Measurement device, measurement method for measurement device, and manufacturing method for measurement device

Abstract

A measurement device (10) comprises: a light projector (22) that emits light; a light receiver (32) that has a light receiving surface (34) for receiving reflected light that is the light emitted from the light projector that has returned after being reflected off an object being measured, the light receiving surface including a plurality of photon-counting light receiving elements arranged in an array; and a controller (42). The controller executes: a histogram generation process for generating a histogram on the basis of output values from the plurality of light receiving elements; a light reception timing estimation process for estimating a light reception timing of the reflected light on the basis of a first reach timing at which the histogram first reaches an intermediate value between the maximum value and the minimum value of the waveform of the histogram, and a second reach timing at which the histogram next reaches the intermediate value; and an output process for outputting information corresponding to the distance to the object being measured on the basis of a light projection timing at which the light projector emits the light and the light reception timing.

Description

Measurement device, measurement method for measurement device, and manufacturing method for measurement device

[0001] The present disclosure relates to a measurement device, a measurement method for the measurement device, and a manufacturing method for the measurement device.

[0002] With the advancement of AD (Autonomous Driving) and ADAS (Advanced Driver-Assistance Systems), research and development of LiDAR (Light Detection and Ranging) is underway as one of the measurement devices used to grasp the surrounding environment and estimate the vehicle's position while the vehicle is traveling. LiDAR includes a projector that projects (irradiates) laser light onto a measurement target, and a photoreceiver that receives the reflected light that is returned from the measurement target after the laser light is reflected. LiDAR outputs information about the measurement target by measuring the distance to the measurement target based on the difference between the projection timing when the projector emits the laser light and the reception timing when the photoreceiver receives the reflected light.

[0003] The photoreceiver has a light receiving surface on which a plurality of photon counting light receiving elements (SPAD: Single Photon Avalanche Diode) are arranged in an array. In the measurement device, the output values ​​from a predetermined number of light receiving elements are integrated over a predetermined time period, and the integrated value is processed as one pixel to generate a histogram. The light receiving timing is determined based on the histogram (see, for example, Patent Document 1).

[0004] Furthermore, conventional measuring devices are configured to write into memory all of the light reception data groups covering an estimated time from when light is emitted from the light projector, reflected by the object to be measured, and returned to the light receiver.

[0005] Japanese Patent Application Publication No. 2018-91760

[0006] However, sometimes the amount of incident reflected light on the light-receiving surface exceeds a predetermined upper limit, resulting in a saturated state. In this saturated state, the histogram waveform reaches the upper limit before the peak timing of the amount of incident reflected light on the light-receiving surface arrives, and the timing at which the histogram waveform first reaches the upper limit is considered the timing at which the reflected light is received. Therefore, in a saturated state, the distance to the measurement target may be measured as shorter than the actual distance. Furthermore, in a saturated state, it is impossible to estimate the amount of incident reflected light at all.

[0007] Furthermore, in the above-described conventional measuring device, the number of light-receiving elements constituting one pixel and the number of times the output signals of the light-receiving elements are integrated are both fixed. Therefore, for example, it is not possible to change the reflected light detection performance or image resolution, and there is room for improvement. Note that this problem is not limited to photon-counting light-receiving elements, but is also a common problem in measuring devices in which multiple analog output light-receiving elements are arranged in an array.

[0008] Furthermore, conventional measuring devices require a large-capacity memory that can store all of the light reception data for an estimated time period. Conversely, reducing the memory capacity shortens the distance that can be measured by the measuring device.

[0009] Furthermore, conventional measurement devices may not be able to accurately measure the distance to the measurement target due to, for example, variations in the timing of light projection from the light projector or variations in the timing of output of the light-receiving signal from the light-receiving element in the light receiver. Factors that cause variations in the timing of light projection include variations in the length of the wiring connected to the multiple light-projecting elements and variations in the timing of output of the trigger signal given to the light projector. Factors that cause variations in the timing of output of the light-receiving signal include variations in the length of the wiring connected to the multiple light-receiving elements and variations in response characteristics.

[0010] The present disclosure aims to solve at least one of the above-mentioned problems.

[0011] A measuring device according to a first aspect of the present disclosure is a measuring device comprising: a light projector that emits light; a light receiver having a light receiving surface on which a plurality of photon counting type light receiving elements are arranged in an array, and that receives reflected light that is returned after the light projected from the light projector is reflected off a measurement object; and a controller, wherein the controller executes: a histogram generation process that generates a histogram based on output values ​​from the plurality of light receiving elements; a light reception timing estimation process that estimates the light reception timing of the reflected light based on a first arrival timing at which the histogram first reaches an intermediate value between the highest and lowest values ​​of the waveform of the histogram, and a second arrival timing at which the histogram next reaches the intermediate value; and an output process that outputs information corresponding to the distance to the measurement object based on the light projecting timing at which the light projector emits light and the light reception timing.

[0012] A measurement device according to a second aspect of the present disclosure is a measurement device comprising: a light projector that emits light; a light receiver having a light receiving surface on which a plurality of photon counting type light receiving elements are arranged in an array, which receives reflected light that is the light emitted from the light projector reflected back onto a measurement object; and a controller, wherein the controller executes: a histogram generation process that generates a histogram based on output values ​​from the plurality of light receiving elements; and a first incident light amount estimation process that estimates the amount of incident light of the reflected light based on the time difference between a first arrival timing at which the histogram first reaches an intermediate value between the highest and lowest values ​​of the histogram waveform, and a second arrival timing at which the histogram next reaches the intermediate value.

[0013] A measurement device according to a third aspect of the present disclosure is a measurement device comprising: a light projector that emits light; a light receiver having a light receiving surface on which a plurality of light receiving elements are arranged in an array to receive reflected light that is returned when the light emitted from the light projector is reflected off a measurement object; and a controller, wherein the controller executes: a first measurement process that acquires output values ​​of the plurality of light receiving elements arranged in at least a part of a target area of ​​the light receiving surface in a first acquisition unit determined by the number of light receiving elements that constitute one pixel and the number of times the output values ​​of the light receiving elements are accumulated, and measures the distance to the measurement object based on the obtained output results; and a second measurement process that acquires output values ​​of the plurality of light receiving elements arranged in the target area in a second acquisition unit in which at least one of the number of light receiving elements that constitute one pixel and the number of times the output values ​​of the light receiving elements are accumulated is different from the first acquisition unit, and measures the distance to the measurement object based on the obtained output results.

[0014] A measuring device according to a fourth aspect of the present disclosure is a measuring device comprising: a light projector that emits light; a light receiver having a light receiving surface on which a plurality of light receiving elements arranged in an array are arranged to receive reflected light that is returned when the light emitted from the light projector is reflected off a measurement object; a controller; and a memory, wherein the controller performs an update process that, after the light projector emits light, writes the light receiving data output from the plurality of light receiving elements into the memory, and when the amount of the light receiving data written reaches a reference value of the memory, repeatedly performs an update operation to overwrite the memory with the light receiving data output from the plurality of light receiving elements thereafter; and a measurement process that, when reflected light receiving data, which is the light receiving data corresponding to the reflected light, is written into the memory, extracts a group of light receiving data including the reflected light receiving data and measures the distance to the measurement object.

[0015] A measurement method for a measurement device according to one aspect of the present disclosure is a measurement method for a measurement device including a light projector that emits light, a light receiver having a light receiving surface on which a plurality of light receiving elements arranged in an array are arranged to receive reflected light that is returned when the light emitted from the light projector is reflected off a measurement object, and a memory, and includes the following steps: an update process in which, after the timing of emitting light from the light projector, light reception data output from the plurality of light receiving elements is written to the memory, and when the amount of light reception data written reaches a reference value of the memory, an update operation is repeatedly performed to overwrite the memory with the light reception data output from the plurality of light receiving elements; and a measurement process in which, when reflected light reception data, which is the light reception data corresponding to the reflected light, is written to the memory, a group of light reception data including the reflected light reception data is extracted and the distance to the measurement object is measured.

[0016] A method for manufacturing a measuring device according to one aspect of the present disclosure includes a light projector having one or more light-emitting elements, a light receiver having a light receiving surface on which a plurality of light-receiving elements are arranged in an array to receive reflected light emitted by the light projector and returned from a measurement object, and a controller for controlling the light projector and the light receiver, and includes: a preparation step of preparing a sample measurement object having a flat surface; an acquisition step of emitting light from the light projector toward the flat surface of the sample measurement object, receiving the reflected light reflected from the flat surface and returned by the light receiving surface of the light receiver, and acquiring measurement results of the distance to the flat surface for each pixel based on a plurality of sample light-receiving signals output from each of the plurality of light-receiving elements; and a correction step of correcting at least one of the light-emitting timing of the light-emitting element and the output timing of the light-receiving signal from the light-receiving element so that the measurement results of the distance for each pixel become closer to uniform.

[0017] According to the measurement device of the first aspect of the present disclosure, even when the amount of incident light of reflected light on the light receiving surface becomes saturated, the light receiving timing can be estimated based on the first arrival timing and the second arrival timing.

[0018] According to the measurement device of the second aspect of the present disclosure, even when the amount of incident reflected light on the light receiving surface is saturated, the amount of incident reflected light can be estimated based on the waveform width of the histogram.

[0019] According to the measurement device of the third aspect of the present disclosure, for example, by changing at least one of the number of light receiving elements constituting one pixel and the number of times the output values ​​of the light receiving elements constituting one pixel are integrated, it is possible to change the reflected light detection performance, resolution, and frame rate, thereby making it possible to measure the distance to the measurement object while changing the reflected light detection performance, etc.

[0020] According to the measuring device of the fourth aspect of the present disclosure, it is possible to measure the distance to the object to be measured while reducing the memory capacity required to store the light reception data, compared to a configuration in which light reception data for the estimated time it takes for light to be emitted from the light projector, reflected by the object to be measured, and returned to the light receiver is written into memory.

[0021] According to a measurement method of a measurement device relating to one aspect of the present disclosure, it is possible to measure the distance to the object to be measured while reducing the memory capacity required to store the received light data, compared to a method in which light is emitted from a light projector, reflected by the object to be measured, and returned to a receiver and the received light data is written into memory for an estimated time.

[0022] According to the manufacturing method of the measuring device relating to the first aspect of the present disclosure, it is possible to suppress a decrease in the measurement accuracy of the distance to the object to be measured due to, for example, variations in the timing of light projection from the light projector or variations in the timing of output of the light receiving signal from the light receiving element in the light receiver.

[0023] FIG. 1 is a block diagram showing the configuration of a measurement device 10 according to a first embodiment; FIG. 2 is an explanatory diagram showing the relationship between the output value of a SPAD constituting one pixel and a histogram; FIG. 3 is a flowchart showing the measurement process executed by the TOF measurement device 40 and the control circuit 42; FIG. 4 is an explanatory diagram showing an example of a histogram in a non-saturated state; FIG. 5 is an explanatory diagram showing an example of a histogram in a saturated state; FIG. 6 is an explanatory diagram showing the relationship between the level of incident light intensity of ambient light L3 and the second arrival timing T2; FIG. 7 is an explanatory diagram showing the relationship between the level of incident light intensity of reflected light L2 and the timing of receiving reflected light L2; a flow chart showing the measurement process executed by the TOF measurement device 1040 and the control circuit 1042; a diagram showing the difference in the number of SPAD integration times U between the reference measurement process and the low frame rate measurement process; a diagram showing the difference in the number of SPAD units M between the reference measurement process and the low resolution measurement process; a block diagram showing the general configuration of the measurement device 2010 in the third embodiment; a diagram showing variations in the light projection timing and the output timing of the light reception signal, and the results of distance measurement; a flow chart showing a part of the memory processing; a diagram illustrating memory processing in an example and a comparative example; a diagram illustrating memory processing of a first pixel and a second pixel; a block diagram showing the general configuration of the measurement device 3010 in the fourth embodiment; a diagram showing variations in the light projection timing and the output timing of the light reception signal, and the results of distance measurement; a flow chart showing a part of the manufacturing process of the measurement device 3010;

[0024] A. First Embodiment: A-1. Configuration of Measuring Device 10: FIG. 1 is a block diagram showing a schematic configuration of a measuring device 10 according to a first embodiment. As shown in FIG. 1, the measuring device 10 includes a projector 20 that irradiates a measurement target W with emitted light L1 (e.g., a light beam (laser light)), and a photoreceiver 30 that receives reflected light L2 (return light) that is generated when the emitted light L1 is reflected back from the measurement target W, and functions as a LiDAR. The measuring device 10 acquires information about the measurement target W by measuring the difference between the timing at which the projector 20 emits the emitted light L1 and the timing at which the photoreceiver 30 receives the reflected light L2 (hereinafter referred to as "TOF (Time of Flight)" of the laser light).

[0025] The measuring device 10 is mounted on, for example, a vehicle (not shown) equipped with AD or ADAS. The measuring device 10 assists in detecting objects such as people and other vehicles while the vehicle is traveling, and provides various information to other devices and users that is useful for ensuring the safety of the driver of the vehicle and those around the vehicle, and for reducing damage to surrounding objects while the vehicle is being driven.

[0026] The light projector 20 includes a light source 22 , a light projecting optical system 24 , a light projecting control device 26 , and a current source 28 .

[0027] The light source 22 includes a light-emitting source having one or more light-emitting elements (not shown), or one or more light-emitting element arrays (e.g., light-emitting elements arranged linearly (one-dimensionally) or planarly (two-dimensionally)). The light-emitting element may be, for example, a laser diode, a surface-emitting laser light-emitting element (e.g., a VCSEL (Vertical Cavity Surface Emitting Laser, hereinafter referred to as "surface-emitting element")), a surface-emitting element array (e.g., a VCSEL array) in which a plurality of surface-emitting elements are arranged one-dimensionally or two-dimensionally on a substrate (e.g., a semiconductor substrate, a ceramic substrate, etc.), or the like.

[0028] The current source 28 supplies a current corresponding to a control signal input from the light projection control device 26 to the light emitting element that constitutes the light source 22. The current source 28 supplies, for example, a periodic square wave current to the light emitting element for turning on and off the current flowing through the light emitting element.

[0029] The light-projection control device 26 generates a control signal for the current source 28 and inputs it to the current source 28, thereby controlling the current (drive current) supplied to the light-emitting element from the current source 28. The light-projection control device 26 inputs a signal indicating the timing at which the light-emitting element emits light (the timing at which the light-emitting element emits light; hereinafter referred to as "light-projection timing") to the TOF measurement device 40. The light-projection control device 26 periodically controls the on / off of a current flowing through the light-emitting element, for example, to cause the light-emitting element to periodically emit light.

[0030] The light projection optical system 24 adjusts the light distribution of the output light L1 by, for example, applying an optical effect (such as refraction, scattering, or diffraction) to the light emitted by the light source 22. The light projection optical system 24 is configured using optical components such as various lenses, such as collimating lenses, and reflecting mirrors (mirrors).

[0031] The light receiver 30 includes a light receiving section 32 and a light receiving optical system 34 .

[0032] The light receiving optical system 34 collects reflected light L2, which is light L1 emitted from the projector 20 and reflected by the measurement target W or the like, onto the light receiving unit 32. The light receiving optical system 34 is configured using optical components such as various lenses, such as a collecting lens, various filters, such as a wavelength filter, and a reflecting mirror (mirror).

[0033] The light receiving unit 32 has a plurality of light receiving elements. The light receiving unit 32 performs photoelectric conversion on the reflected light L2 incident from the light receiving optical system 34 to generate a light receiving signal with a current level or voltage level corresponding to the intensity of the reflected light L2. The light receiving unit 32 inputs to the TOF measurement device 40 a signal indicating the timing at which the light receiving elements constituting the light receiving unit 32 receive the reflected light L2 (hereinafter referred to as "light receiving timing"), and the light receiving signal generated by the light receiving elements.

[0034] The measurement apparatus 10 further includes a TOF measurement device 40, a control circuit 42, and a communication I / F (Inter face) 50. In this embodiment, the TOF measurement device 40 and the control circuit 42 are an example of a controller.

[0035] The TOF measurement device 40 calculates the TOF based on a signal indicating the light projection timing input from the light projection control device 26 and a signal indicating the light reception timing input from the light receiving unit 32. The TOF measurement device 40 has, for example, a time measurement IC (Integrated Circuit) equipped with a TDC (Time to Digital Converter) circuit. The TOF measurement device 40 inputs the calculated TOF and the light reception signal input from the light receiving unit 32 to the control circuit 42.

[0036] The control circuit 42 has a processor (such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor)). Based on the light receiving signal and TOF input from the TOF measurement device 40, the control circuit 42 generates information used for various measurements such as detection of the measurement object W and distance measurement. The information includes, for example, a histogram used in time-correlated single photon counting, distances to each point on the measurement target W, a point cloud (point cloud information), etc. The control circuit 42 also controls the light-projection control device 26 and the light-receiving unit 32. The control circuit 42 controls the light-projection timing and light-receiving timing described above, for example, by controlling the light-projection control device 26 and the light-receiving unit 32, so as to speed up or optimize the processing involved in generating the histogram. The information generated by the control circuit 42 is provided (transmitted) via the communication I / F 50 to devices that use the information (hereinafter referred to as "various use devices 60").

[0037] The various utilization devices 60 perform, for example, the creation of an environmental map using a point cloud, and self-position estimation (SLAM (Simultaneous Localization and Mapping)) using a scan matching algorithm (NDT (Normal Distributions Transform), ICP (Iterative Closest Point), etc.).

[0038] A-2. Signal Processing in the TOF Measurement Device 40: In this embodiment, the photoreceiver 30 (light receiving unit 32) has a light receiving surface 32A on which a plurality of light receiving elements are arranged in a linear (one-dimensional) array or a planar (two-dimensional) array. In this embodiment, the light receiving elements are SPADs, which are an example of photon-counting light receiving elements. The SPAD responds with a predetermined probability (e.g., 20%) when a photon is incident, and outputs one pulse when a photon is detected.

[0039] In the light receiving unit 32 using SPADs, one pixel in an image (hereinafter referred to as a "received light image") acquired based on the light receiving signal from the SPAD is defined by the "SPAD unit number M" and the "SPAD integration number U." The SPAD unit number M is the number of adjacent SPADs on the light receiving surface 32A. For example, in a configuration in which multiple SPADs are arranged two-dimensionally on the light receiving surface 32A, the SPAD unit number M is 9 (= 3 × 3) or 16 (= 4 × 4). The larger the SPAD unit number M, the lower the resolution of the received light image. The SPAD integration number U is the number of times the output value of each SPAD is integrated at a predetermined sampling timing. A larger SPAD integration number U means a longer integration time for the SPAD output values. The larger the SPAD integration number U, the lower the frame rate of the received light image (the number of images (frames) constituting a moving image per second). The TOF measurement device 40 generates a histogram for each pixel based on the output values ​​of the SPADs that make up each pixel.

[0040] FIG. 2 is an explanatory diagram showing the relationship between the output values ​​of the SPADs constituting one pixel and a histogram. The upper part of FIG. 2 shows the temporal change in the output values ​​of each SPAD. The lower part of FIG. 2 shows a graph G showing the temporal change in the amount of incident light (received light intensity, irradiation intensity) per unit area of ​​light (including reflected light L2 and ambient light (background light) L3) irradiated to one pixel of the light-receiving surface 32A, and a histogram H generated by the TOF measurement device 40. In FIG. 2, Q1 represents one pulse output when each SPAD detects a photon when reflected light L2 is incident, and Q2 represents one pulse output when each SPAD detects a photon when only ambient light L3 is incident without reflected light L2.

[0041] In graph G, the waveform showing a relatively high peak intensity is the waveform when reflected light L2 is incident on the light-receiving surface 32A, while the waveform showing a relatively low peak intensity is the waveform when only disturbance light L3 is incident on the light-receiving surface 32A without reflected light L2. The incident light amount of reflected light L2 is higher than the incident light amount of disturbance light L3. Therefore, as shown in histogram H, when reflected light L2 is incident on the light-receiving surface 32A, the number of detected photons per pixel is statistically higher, and when only disturbance light L3 is incident on the light-receiving surface 32A without reflected light L2, the number of detected photons per pixel is statistically lower. The TOF measurement device 40 estimates the light-receiving timing based on the timing corresponding to the waveform showing a relatively high peak in histogram H.

[0042] A-3. Estimation process of the timing of receiving reflected light L2 and the reflectance of the measurement object W: Figure 3 is a flowchart showing the measurement process executed by the TOF measurement device 40 and the control circuit 42. In this measurement process, the timing of receiving reflected light L2 and the reflectance of the measurement object W are estimated.

[0043] (Estimation of Receiving Timing of Reflected Light L2): As shown in FIG. 3 , the TOF measurement device 40 executes a histogram generation process (S110) to generate a histogram based on output values ​​from multiple SPADs. The TOF measurement device 40 sequentially stores the output values ​​from the multiple SPADs in a predetermined memory area. At each predetermined sampling timing, the TOF measurement device 40 reads the output values ​​from the multiple SPADs stored in the memory area and sequentially generates a histogram for each pixel. The TOF measurement device 40 extracts a histogram including the signal waveform of the reflected light L2 based on a comparison between the sequentially generated histograms and a signal detection threshold. In this embodiment, the signal detection threshold is the same value as the determination threshold for determining the presence or absence of reflected light L2, but may be a value lower than the determination threshold.

[0044] Next, the TOF measurement device 40 acquires a first arrival time T1 and a second arrival time T2 in a histogram including the signal waveform of the extracted reflected light L2 (S120). The first arrival time T1 is the time when the histogram waveform first reaches the median value Lh, and the second arrival time T2 is the time when the histogram waveform next reaches the median value Lh. The median value Lh is the value between the highest value (corresponding to the peak of the signal waveform of the reflected light L2) and the average value (corresponding to the signal waveform of the ambient light L3) of the histogram waveform. Furthermore, the median value Lh is preferably a value obtained by adding a height value that is a predetermined percentage of the difference between the highest value and the lowest value to the lowest value. In this embodiment, the predetermined percentage is 1 / 2, but is not limited to this and may be, for example, 3 / 4.

[0045] Fig. 4 is an explanatory diagram illustrating a histogram in a non-saturated state. Fig. 5 is an explanatory diagram illustrating a histogram in a saturated state. The non-saturated state is a state in which the peak intensity Lp of the histogram is less than a predetermined saturation level Lq, as shown in Fig. 4, and the saturated state is a state in which the peak intensity Lp of the histogram has reached the predetermined saturation level Lq, as shown in Fig. 5. In the non-saturated state, the peak timing at which the peak intensity is reached can be identified from the waveform of the histogram and estimated as the light receiving timing.

[0046] On the other hand, in a saturated state, the histogram waveform reaches the saturation level Lq and flattens out before the peak timing of the incident light amount of the reflected light L2 on the light-receiving surface 32A (hereinafter referred to as the "light-receiving timing of the reflected light L2") arrives, and there are multiple times when the histogram waveform reaches a peak, so the peak timing cannot be identified. If the timing when the histogram waveform first reaches the saturation level Lq is taken as the light-receiving timing of the reflected light L2, a problem occurs in that the distance to the measurement target W is measured as shorter than the actual distance.

[0047] In response to this, the inventors have newly discovered that the temporal relationship between the first arrival time T1 and the second arrival time T2 in the histogram correlates with the timing of receiving the reflected light L2. That is, the timing of receiving the reflected light L2 can be estimated based on the first arrival time T1 and the time difference ΔT between the first arrival time T1 and the second arrival time T2. Therefore, in this embodiment, the timing of receiving the reflected light L2 can be estimated not only in a non-saturated state but also in a saturated state.

[0048] Next, the TOF measurement device 40 executes an arrival timing correction process (S130). The arrival timing correction process corrects the second arrival timing T2 depending on the level of the incident light amount of the disturbance light L3 on the light-receiving surface 32A. Specifically, when the light-receiving surface 32A is not irradiated with the reflected light L2 and only with the disturbance light L3, the TOF measurement device 40 appropriately executes a disturbance light histogram generation process to generate a disturbance light histogram based on output values ​​from multiple SPADs. The TOF measurement device 40 corrects the second arrival timing T2 depending on the height of the average value of the disturbance light histogram generated in the disturbance light histogram generation process.

[0049] FIG. 6 is an explanatory diagram showing the relationship between the level of the incident light intensity of disturbance light L3 and the second arrival time T2. Symbols Hg1 to Hg5 in FIG. 6 represent disturbance light histograms with different incident light intensities of disturbance light L3. Symbol R1 represents an approximate straight line of multiple first arrival times T1 under conditions where the incident light intensity of reflected light L2 is the same but the incident light intensity of disturbance light L3 is different. Symbol R2 represents an approximate straight line of multiple second arrival times T2 under conditions where the incident light intensity of reflected light L2 is the same but the incident light intensity of disturbance light L3 is different. As can be seen from FIG. 6, the first arrival time T1 is constant regardless of the level of the incident light intensity of disturbance light L3 as long as the incident light intensity of reflected light L2 is the same. On the other hand, the second arrival time T2 is closer to the first arrival time T1 as the incident light intensity of disturbance light L3 increases, even when the incident light intensity of reflected light L2 is the same. Therefore, based on the linear function of the approximate straight line R2, the TOF measurement device 40 corrects the second arrival time T2 to a later timing (a timing further away from the first arrival time T1) as the amount of incident light in the ambient light histogram increases.

[0050] Next, the TOF measurement device 40 executes a light-reception timing estimation process (S140). The TOF measurement device 40 estimates the reception timing of the reflected light L2 based on the first arrival timing T1 and the second arrival timing T2 (or the corrected second arrival timing T2). For example, the timing obtained by adding a value obtained by multiplying the time difference ΔT between the first arrival timing T1 and the second arrival timing T2 by a predetermined time ratio (e.g., 1 / 2, 3 / 8, etc.) to the first arrival timing T1 is determined to be the reception timing of the reflected light L2.

[0051] Next, the TOF measurement device 40 executes a light-receiving timing correction process (S150). The light-receiving timing correction process corrects the light-receiving timing of the reflected light L2 depending on the level of the incident light amount of the reflected light L2 on the light-receiving surface 32A.

[0052] FIG. 7 is an explanatory diagram showing the relationship between the level of the incident light intensity of the reflected light L2 and the timing of receiving the reflected light L2. Symbols Hh to Hl in FIG. 7 represent histograms with different incident light intensities of the reflected light L2. Symbol Gp represents an approximate line of multiple light receiving timings (peak timings of the incident light intensity of the reflected light L2) under conditions with different incident light intensities of the reflected light L2. As can be seen from FIG. 7, the higher the incident light intensity of the reflected light L2, the earlier the light receiving timing. Therefore, based on the linear function of the approximate line Gp, the TOF measurement device 40 corrects the second arrival timing T2 to a later timing as the peak intensity of the histogram increases.

[0053] The inventors have newly discovered that the time difference ΔT between the first arrival time T1 and the second arrival time T2 correlates with the peak intensity (height) of the incident light amount of the reflected light L2 at the light receiving surface 32A. That is, the higher the peak intensity of the incident light amount of the reflected light L2, the wider the width of the histogram waveform, and accordingly, the larger the time difference ΔT between the first arrival time T1 and the second arrival time T2. Therefore, even in a saturated state, for example, the peak intensity of the incident light amount of the reflected light L2 can be estimated based on the time difference ΔT between the first arrival time T1 and the second arrival time T2. Based on a linear function of the approximate line Gp, the TOF measurement device 40 corrects the second arrival time T2 to a later timing as the time difference ΔT between the first arrival time T1 and the second arrival time T2 becomes longer.

[0054] Thereafter, the TOF measurement device 40 measures the distance to the measurement target W based on the light-receiving timing (or the corrected light-receiving timing), and outputs information corresponding to the distance to the utilization device 60 (S160). The processing of S160 is an example of output processing. As described above, the TOF measurement device 40 can measure the distance to the measurement target W using the following formula A. <Formula A> Distance (m) = [A × "Bin number corresponding to the first arrival timing T1" + B × ("Bin number corresponding to the second arrival timing T2" + C × BG intensity)] + (D × peak intensity) + E] × F A, B: weighting coefficients C: correction coefficient in the arrival timing correction process BG intensity: intensity of disturbance light L3 D: correction coefficient in the light reception timing correction process E: overall offset coefficient (for example, correction coefficient for offsetting a deviation in the light projection timing, etc.) F: coefficient for converting from the bin number of the histogram to the distance (m) to the measurement target W. Note that A to F can be changed as appropriate depending on individual differences in measurement device 10, changes in the surrounding environment, etc.

[0055] (Estimation of reflectance of measurement target W): Next, the TOF measurement device 40 executes a process for estimating the reflectance of the measurement target W. The reflectance ρ of the measurement target W can be estimated using the following equation B (LiDAR equation). <Equation B> P ＲＸ =P ＴＸ ρ (D ＲＸ / 2R) ２ ・γ P ＲＸ : Amount of incident light of reflected light L2 P ＴＸ : Amount of emitted light L1 ρ: Reflectance of measurement object W D ＲＸ : diameter of the light receiving surface 32A R: distance to the measurement target W γ: light transmittance of the optical system (light projecting optical system 24 and light receiving optical system 34) In other words, the incident light amount P of the reflected light L2 ＲＸ If it is possible to estimate the reflectance ρ of the measurement object W, it is possible to estimate the reflectance ρ of the measurement object W from the above formula B.

[0056] 8 is an explanatory diagram showing the relationship between the amount of incident light of reflected light L2 and the peak intensity of the histogram (SPAD output value) when reflected light L2 is incident on the light-receiving surface 32A. As shown in FIG. 8, in a measurement device using a photon-counting light-receiving element (SPAD), unlike a measurement device using an analog-output light-receiving element, the correlation between the amount of incident light of reflected light L2 and the peak intensity of the histogram is nonlinear. Therefore, in this embodiment, the amount of incident light of reflected light L2 is estimated using different estimation methods depending on whether the amount of incident light of reflected light L2 belongs to the "first region E1," "second region E2," or "third region E3." "First region E1": saturated region (peak intensity of histogram = saturation level Lq, incident light amount of reflected light L2 ≧ saturation level Lq) "Second region E2": non-saturated region (peak intensity difference between histogram and ambient light L3 ≧ reference value, incident light amount of reflected light L2 < saturation level Lq) "Third region E3": non-saturated region (peak intensity difference between histogram and ambient light L3 < reference value, incident light amount of reflected light L2 < saturation level Lq) Note that Figure 9 shows the relationship between the incident light amount of reflected light L2 and the peak intensity of the histogram (SPAD output value) for each region.

[0057] The TOF measurement device 40 determines whether the incident light amount of the reflected light L2 is saturated based on the histogram when the reflected light L2 is incident on the light receiving surface 32A (S170). If the TOF measurement device 40 determines that the incident light amount is saturated (S170: YES), the TOF measurement device 40 estimates the peak intensity of the incident light amount of the reflected light L2 based on the time difference ΔT between the first arrival time T1 and the second arrival time T2, as in the light receiving timing correction process (S180). This process of S180 is an example of a first incident light amount estimation process. Specifically, the peak intensity of the incident light amount of the reflected light L2 is estimated using the following formula C. <Formula C> Incident light amount of reflected light L2=10 （ｅ・ΔＴ＋ｆ）9 shows a logarithmic graph illustrating the relationship between the time difference ΔT and the amount of incident reflected light L2 in the first region E1. On the logarithmic graph, the relationship between the time difference ΔT and the amount of incident reflected light L2 is approximately linear, and the smaller the amount of incident disturbance light L3, the larger the time difference ΔT (see graphs G1a (small amount of disturbance light L3) to G1c (large amount of disturbance light L3)).

[0058] When the TOF measurement device 40 determines that the image is in a non-saturated state (S170: NO), it determines whether the peak intensity difference between the histogram and the ambient light L3 is equal to or greater than a reference value (S190). The reference value can be changed as appropriate. When the TOF measurement device 40 determines that the peak intensity difference is equal to or greater than the reference value (S190: YES), it estimates the peak intensity of the incident light amount of the reflected light L2 based on the peak intensity of the histogram (SPAD output value) and the following logarithmic function (Equation D) (S200). The process of S200 is an example of a second incident light amount estimation process. Specifically, the peak intensity of the incident light amount of the reflected light L2 is estimated using the following Equation D: <Equation D> Incident light amount of reflected light = 10 (a × output value from light receiving element + b), where a and b are coefficients. The middle part of FIG. 9 shows a logarithmic graph showing the relationship between the peak intensity of the histogram (SPAD output value) and the incident light amount of the reflected light L2 in the second region E2. On the logarithmic graph, the relationship between the peak intensity of the histogram and the amount of incident light of reflected light L2 is approximately linear, and the greater the amount of incident disturbance light L3, the gentler the slope becomes (see graphs G2a (low disturbance light L3) to G2c (high disturbance light L3)). A first coefficient correction process may be executed to correct the coefficient a of the logarithmic function to a smaller value as the amount of incident disturbance light L3 increases.

[0059] When the TOF measurement device 40 determines that the peak intensity difference is less than the reference value (S190: NO), it estimates the peak intensity of the incident light amount of the reflected light L2 based on the peak intensity of the histogram (SPAD output value) and the following real number function (Equation E) (S210). The process of S210 is an example of a third incident light amount estimation process. Specifically, the peak intensity of the incident light amount of the reflected light L2 is estimated using the following Equation E: <Equation E> Incident light amount of reflected light = c × output value from light receiving element + d, where c and d are coefficients. The lower part of Figure 9 shows a real number graph showing the relationship between the peak intensity of the histogram (SPAD output value) and the incident light amount of the reflected light L2 in the third region E3. On the real number graph, the relationship between the peak intensity of the histogram and the amount of incident light of reflected light L2 is approximately linear, and the greater the amount of incident light of disturbance light L3, the gentler the slope becomes (see graphs G3a (low disturbance light L3) to G3c (high disturbance light L3)). A second coefficient correction process may be executed to correct the coefficient c of the real number function to a smaller value as the amount of incident disturbance light L3 increases.

[0060] Thereafter, the TOF measurement device 40 calculates the peak intensity of the incident light amount of the reflected light L2 (incident light amount P ＲＸ ), the distance (R) measured in S160, and the above equation B (LiDAR equation), the reflectance ρ of the measurement object W is estimated (S220), and the process returns to S110.

[0061] A-4. Process for estimating the reflectance of the measurement object W at pixels where reflected light L2 is not detected: In the measurement process shown in FIG. 3, the reflectance ρ of the measurement object W was estimated using equation B (the LiDAR equation). However, this estimation method presupposes that the amount of incident light of reflected light L2 can be estimated. The light receiving surface 32A may include pixels where reflected light L2 is detected (hereinafter referred to as "detection pixels") and pixels where reflected light L2 is not detected (hereinafter referred to as "non-detection pixels"). For detection pixels, the reflectance ρ of the measurement object W can be estimated by the measurement process of FIG. 3. However, for non-detection pixels, the amount of incident light of reflected light L2 cannot be estimated, and therefore the reflectance ρ of the measurement object W cannot be estimated by the measurement process of FIG. 3.

[0062] Therefore, in this embodiment, the control circuit 42 estimates the reflectance ρ of the measurement target W for the non-detection pixel based on the reflectance ρ of the measurement target W estimated for the detection pixel and the ratio of the amount of disturbance light L3 incident on the non-detection pixel to the detection pixel. It is assumed that the disturbance light L3 is uniformly irradiated over the entire light-receiving surface 32A and that the amount of disturbance light L3 incident on the light-receiving surface 32A is independent of the distance from the measurement target W. For example, if the reflectance ρ at the detection pixel is 80% and the ratio of the amount of disturbance light L3 incident on the non-detection pixel to the amount of disturbance light L3 incident on the detection pixel is ½, the reflectance ρ at the non-detection pixel is estimated to be 40% (= 80% × (½)).

[0063] A-5. Advantages of this embodiment: The inventors have newly discovered that the relative relationship between the first arrival time T1 and the second arrival time T2 in the histogram correlates with the peak timing of the incident light amount of the reflected light L2 on the light-receiving surface 32A. Therefore, according to this embodiment, even when the incident light amount of the reflected light L2 on the light-receiving surface 32A is saturated, the light-receiving timing can be estimated based on the first arrival time T1 and the second arrival time T2 (see S110 to S150 in FIG. 3).

[0064] The inventors have also newly discovered that the time difference ΔT between the first arrival time T1 and the second arrival time T2 correlates with the peak intensity of the incident light amount of the reflected light L2 on the light receiving surface 32 A. Therefore, in this embodiment, even in a saturated state where the histogram peak cannot be detected, the peak intensity of the incident light amount of the reflected light L2 can be estimated based on the time difference ΔT of the histogram (see S180 in FIG. 3).

[0065] B. Modifications of the First Embodiment The technology disclosed in this specification is not limited to the above-described embodiment, and can be modified in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[0066] The configuration of the measuring device 10 in the above embodiment is merely an example, and various modifications are possible.

[0067] In the flowchart of the measurement process in the above embodiment, the correction process of either S130 or S150 may not be executed.

[0068] C. Second Embodiment: C-1. Configuration of Measuring Apparatus 1010: FIG. 10 is a block diagram showing a schematic configuration of a measuring apparatus 1010 according to the second embodiment. As shown in FIG. 10, the measuring apparatus 1010 includes a projector 1020 that irradiates a measurement object W with emitted light L1001 (e.g., a light beam (laser light)), and a photoreceiver 1030 that receives reflected light L1002 (return light) that is generated when the emitted light L1001 is reflected back from the measurement object W, and functions as a LiDAR. The measuring apparatus 1010 acquires information about the measurement object W by measuring the time of flight (TOF) between the time when the projector 1020 emits the emitted light L1001 and the time when the photoreceiver 1030 receives the reflected light L1002.

[0069] The measuring device 1010 is mounted on, for example, a vehicle (not shown) equipped with an AD or ADAS. The measuring device 1010 assists in detecting objects such as people and other vehicles while the vehicle is traveling, and provides various information to other devices and users that is useful for ensuring the safety of the driver of the vehicle and those around the vehicle, and for reducing damage to surrounding objects while the vehicle is being driven.

[0070] The light projector 1020 includes a light source 1022 , a light projecting optical system 1024 , a light projecting control device 1026 , and a current source 1028 .

[0071] The light source 1022 includes a light emitting source having one or more light emitting elements (not shown), or one or more light emitting element arrays (for example, light emitting elements arranged linearly (one-dimensionally) or planarly (two-dimensionally)). The light emitting element may be, for example, a laser diode, a surface-emitting laser light emitting element (for example, a VCSEL), or a surface-emitting element array (for example, a VCSEL array) in which a plurality of surface-emitting elements are arranged one-dimensionally or two-dimensionally on a substrate (such as a semiconductor substrate or a ceramic substrate).

[0072] The current source 1028 supplies a current corresponding to a control signal input from the light projection control device 1026 to the light emitting element that constitutes the light source 1022. The current source 1028 supplies, for example, a periodic square wave current to the light emitting element for turning on and off the current flowing through the light emitting element.

[0073] The light-projection control device 1026 generates a control signal for the current source 1028 and inputs it to the current source 1028, thereby controlling the current (drive current) supplied to the light-emitting element from the current source 1028. The light-projection control device 1026 inputs a signal indicating the timing at which the light-emitting element emits light (the timing at which the light-emitting element emits light; hereinafter referred to as "light-projection timing") to the TOF measurement device 1040. The light-projection control device 1026 periodically controls the on / off of a current flowing through the light-emitting element, for example, to cause the light-emitting element to periodically emit light.

[0074] The light projection optical system 1024 adjusts the light distribution of the output light L1001 by, for example, applying an optical effect (such as refraction, scattering, or diffraction) to the light emitted by the light source 1022. The light projection optical system 1024 is configured using optical components such as various lenses, such as collimating lenses, and reflecting mirrors (mirrors).

[0075] The light receiver 1030 includes a light receiving section 1032 and a light receiving optical system 1034 .

[0076] The light receiving optical system 1034 collects reflected light L1002, which is light L1001 emitted from the projector 1020 and reflected by the measurement target W or the like, onto the light receiving unit 1032. The light receiving optical system 1034 is configured using optical components such as various lenses, such as a condenser lens, various filters, such as a wavelength filter, and a reflecting mirror (mirror).

[0077] The light receiving unit 1032 has a plurality of light receiving elements. The light receiving elements are, for example, photodiodes, SPADs, balanced photodetectors, etc. The light receiving unit 1032 performs photoelectric conversion on the reflected light L1002 incident from the light receiving optical system 1034 to generate a light receiving signal with a current level or voltage level corresponding to the intensity of the reflected light L1002. The light receiving unit 1032 inputs, to the TOF measurement device 1040, a signal indicating the timing at which the light receiving elements constituting the light receiving unit 1032 receive the reflected light L1002 (hereinafter referred to as "light receiving timing"), and the light receiving signal generated by the light receiving elements.

[0078] The measurement device 1010 further includes a TOF measurement device 1040, a control circuit 1042, and a communication I / F (Inter face) 1050. In this embodiment, the TOF measurement device 1040 and the control circuit 1042 are an example of a controller.

[0079] The TOF measurement device 1040 calculates the TOF based on a signal indicating the light projection timing input from the light-projection control device 1026 and a signal indicating the light reception timing input from the light-receiving unit 1032. The TOF measurement device 1040 has, for example, a time measurement IC equipped with a TDC circuit. The TOF measurement device 1040 inputs the calculated TOF and the light-receiving signal input from the light-receiving unit 1032 to the control circuit 1042.

[0080] The control circuit 1042 has a processor (CPU, MPU, ASIC, FPGA, DSP, etc.). Based on the light receiving signal and TOF input from the TOF measurement device 1040, the control circuit 1042 generates information used for various measurements such as detection and distance measurement of the measurement target W. This information includes, for example, a histogram used in time-correlated single photon counting, distances to each point on the measurement target W, and a point cloud (point cloud information). The control circuit 1042 also controls the light projection control device 1026 and the light receiving unit 1032. The control circuit 1042 controls the light projection timing and light receiving timing described above, for example, by controlling the light projection control device 1026 and the light receiving unit 1032, so as to speed up or optimize the processing related to the generation of the histogram. The information generated by the control circuit 1042 is provided (transmitted) via the communication I / F 1050 to a device that uses the information (hereinafter referred to as "various utilization device 1060").

[0081] The various utilization devices 1060 perform, for example, the creation of an environmental map using a point cloud, self-localization (SLAM) using a scan matching algorithm (NDT, ICP, etc.), and the like.

[0082] C-2. Signal Processing in TOF Measurement Device 1040: In this embodiment, the photoreceiver 1030 (light receiving unit 1032) has a light receiving surface 1032A on which a plurality of light receiving elements are arranged in a linear (one-dimensional) array or a planar (two-dimensional) array. The following describes an example in which the light receiving elements are SPADs, an example of photon-counting light receiving elements. The SPAD responds to incident photons with a predetermined probability (e.g., 20%) and outputs one pulse when a photon is detected.

[0083] In the light receiving unit 1032 using SPADs, one pixel in an image (hereinafter referred to as a "received light image") acquired based on a light receiving signal from the SPAD is defined by the "number of SPAD units M" and the "number of SPAD accumulations U." The number of SPAD units M is the number of adjacent SPADs on the light receiving surface 1032A. For example, in a configuration in which multiple SPADs are arranged two-dimensionally on the light receiving surface 1032A, the number of SPAD units M is 9 (= 3 × 3) or 16 (= 4 × 4). The larger the number of SPAD units M, the lower the resolution of the received light image. The number of SPAD accumulations U is the number of times the output value of each SPAD is accumulated at a predetermined sampling timing. A larger number of SPAD accumulations U means a larger number of times the output value of the SPAD is accumulated. The larger the number of SPAD accumulations U, the lower the frame rate of the received light image (the number of images (frames) constituting a moving image per second). The TOF measurement device 1040 generates a histogram for each pixel based on the output values ​​of the SPADs that make up each pixel.

[0084] FIG. 11 is an explanatory diagram showing the relationship between the output values ​​of the SPADs constituting one pixel and a histogram. The upper part of FIG. 11 shows the temporal changes in the output values ​​of each SPAD (SPADs 1001 to 1003). The lower part of FIG. 11 shows a graph G showing the temporal changes in the incident light amount (received light intensity, irradiation intensity) per unit area of ​​light (including reflected light L1002 and ambient light (background light) L1003) irradiated to one pixel of the light-receiving surface 1032A, and a histogram H generated by the TOF measurement device 1040. Q1001 in FIG. 11 represents one pulse output when each SPAD detects a photon when reflected light L1002 is incident, and Q1002 represents one pulse output when each SPAD detects a photon when only ambient light L1003 is incident without reflected light L1002.

[0085] In graph G, the waveform showing a relatively high peak intensity is the waveform when reflected light L1002 is incident on the light-receiving surface 1032A, and the waveform showing a relatively low peak intensity is the waveform when reflected light L1002 is not incident and only disturbance light L1003 is incident on the light-receiving surface 1032A. The incident light amount of reflected light L1002 is higher than the incident light amount of disturbance light L1003. Therefore, as shown in histogram H, when reflected light L1002 is incident on the light-receiving surface 1032A, the number of detected photons per pixel is statistically higher, and when reflected light L1002 is not incident and only disturbance light L1003 is incident on the light-receiving surface 1032A, the number of detected photons per pixel is statistically lower. The TOF measurement device 1040 determines the light-receiving timing based on the timing corresponding to the waveform showing a relatively high peak in histogram H.

[0086] 12 is a flowchart showing the measurement process executed by the TOF measurement device 1040 and the control circuit 1042. In this measurement process, TOF measurement can be performed while changing at least one of the number M of SPAD units constituting one pixel and the number U of SPAD accumulations.

[0087] 12 , the TOF measurement device 1040 executes a reference measurement process at the beginning of measurement (S1110). In the reference measurement process, output values ​​of a plurality of light-receiving elements arranged over the entire light-receiving surface 1032A are acquired in a reference unit in which the number of SPAD units M and the number of SPAD accumulations U are each set to predetermined values ​​(initial values), and the distance to the measurement target W is measured based on the acquired output results. The reference unit is an example of a first acquisition unit, and the reference measurement process is an example of a first measurement process.

[0088] Next, the TOF measurement device 1040 determines whether the detection performance improvement condition is satisfied (S1120). Detection performance refers to the accuracy of detecting the presence or absence of reflected light L1002 in one pixel. The greater the number M of SPAD units constituting one pixel, the greater the area-wise overlap of output values ​​of the SPADs that detect photons when the reflected light L1002 is irradiated onto the light-receiving surface 1032A, resulting in a relatively higher peak intensity in the histogram. As a result, the detection performance for the reflected light L1002 is improved. Furthermore, the greater the number U of SPAD integrations constituting one pixel, the greater the time-wise overlap of output values ​​of the SPADs that detect photons when the reflected light L1002 is irradiated onto the light-receiving surface 1032A, resulting in a relatively higher peak intensity in the histogram. As a result, the detection performance for the reflected light L1002 is improved. The detection performance improvement condition is an example of a switching condition.

[0089] Examples of conditions for improving detection performance include the following: (1) The measurement results of the reference measurement process show a predetermined change pattern (for example, a change pattern indicating the detection of the measurement object W, a change pattern indicating the detection of an abnormal image, etc.). (2) The relative speed between the measurement device 1010 and the measurement object W (for example, the moving speed of the vehicle on which the measurement device 1010 is mounted) has exceeded a predetermined speed or fallen below a predetermined speed. (3) A signal indicating an instruction to improve detection performance has been received from outside (for example, a control unit of the vehicle on which the measurement device 1010 is mounted).

[0090] If the TOF measurement device 1040 determines that the detection performance improvement condition is not satisfied (S1120: NO), it continues the reference measurement process. On the other hand, if the TOF measurement device 1040 determines that the detection performance improvement condition is satisfied (S1120: YES), it determines whether the frame rate restriction condition is satisfied (S1130). The frame rate restriction condition is a condition for restricting a decrease in the frame rate, and examples thereof include the following: (1) The relative speed between the measurement device 1010 and the measurement target W (e.g., the traveling speed of the vehicle on which the measurement device 1010 is mounted) is equal to or greater than a predetermined speed; or (2) The current frame rate has reached a predetermined lower limit.

[0091] (Low Frame Rate Measurement Process): When the TOF measurement device 1040 determines that the frame rate limit condition is not satisfied (S1130: NO), it executes the low frame rate measurement process (S1140). In the low frame rate measurement process, output values ​​of multiple SPADs arranged over the entire light receiving surface 1032A are acquired in low frame rate units, and the distance to the measurement target W is measured based on the acquired output results. The low frame rate unit has the same number of SPAD units M as the reference unit and a larger number of SPAD accumulations U. The low frame rate unit is an example of a second acquisition unit, and the low frame rate measurement process is an example of a second measurement process.

[0092] 13 is an explanatory diagram showing the difference in the SPAD accumulation count U between the reference measurement process and the low frame rate measurement process. Since the reference measurement process and the low frame rate measurement process have the same number of SPAD units M, a histogram (hereinafter referred to as a "unit histogram") is generated in which the output values ​​of the SPADs for the number of SPAD units M are area-superimposed for each sampling timing (sampling period) in both processes.

[0093] Here, the number of SPAD integrations U in the reference measurement process is assumed to be, for example, "1." In this case, as shown in the upper part of FIG. 13 , the TOF measurement device 1040 outputs a unit histogram corresponding to the amount of incident light on the light receiving surface 1032A as data for one pixel at each sampling timing, and the control circuit 1042 determines the presence or absence of reflected light L1002 based on a comparison between the unit histogram and a determination threshold. At each sampling timing, the signal waveform corresponding to the reflected light L1002 is relatively small, resulting in a low S / N ratio. The S / N ratio here refers to the ratio of the peak value S of the signal corresponding to the reflected light L1002 to the peak value N of the signal corresponding to the ambient light L1003. As such, in the reference measurement process, the S / N ratio is low, and therefore the peak intensity of the unit histogram constituting one pixel is relatively low, resulting in relatively low detection performance for the reflected light L1002.

[0094] On the other hand, the number of SPAD integrations U in the low frame rate measurement process is set to, for example, "3." In this case, as shown in the lower part of FIG. 13 , the TOF measurement device 1040 generates a histogram (hereinafter referred to as a "superimposed histogram") by superimposing unit histograms generated at three sampling times, and outputs the superimposed histogram as data for one pixel. The control circuit 1042 determines the presence or absence of reflected light L1002 based on a comparison between the superimposed histogram and a determination threshold. The signal waveform corresponding to the reflected light L1002 superimposed at three sampling times is relatively large, resulting in a high S / N ratio. As such, the low frame rate measurement process has a high S / N ratio, and therefore the peak intensity of the superimposed histogram constituting one pixel is relatively high, resulting in relatively high detection performance for the reflected light L1002. However, the frame rate of the received light image is lower in the low frame rate measurement process than in the reference measurement process.

[0095] For example, when a vehicle equipped with the measuring device 1010 is moving at a low speed (moving speed less than a predetermined speed), the possibility of overlooking an image is relatively low even if the frame rate is lowered, so it is preferable to perform low frame rate measurement processing to improve the detection performance for the reflected light L1002. Note that in the low frame rate measurement processing, the number of SPAD units M may be reduced compared to the reference measurement processing to increase the resolution of the received light image. Furthermore, the number of SPAD accumulations U for each of the reference measurement processing and the low frame rate measurement processing is not limited to the number in the example of FIG. 13 and can be changed as appropriate.

[0096] (Low-resolution measurement process): When the TOF measurement device 1040 determines that the frame rate limit condition is satisfied (S1130: YES), it executes low-resolution measurement process (S1150). In the low-resolution measurement process, output values ​​of multiple light-receiving elements arranged over the entire light-receiving surface 1032A are acquired in low-resolution units, and the distance to the measurement target W is measured based on the acquired output results. The low-resolution unit has a larger number of SPAD units M than the reference unit and the same number of SPAD accumulations U. The low-resolution unit is an example of a second acquisition unit, and the low-resolution measurement process is an example of a second measurement process.

[0097] 14 is an explanatory diagram showing the difference in the number of SPAD units M between the reference measurement process and the low-resolution measurement process. The reference measurement process and the low-resolution measurement process have the same SPAD integration number U, and therefore the same frame rate.

[0098] Here, the number M of SPAD units in the reference measurement process is assumed to be, for example, "2 x 2." In this case, as shown in the upper right of FIG. 14 , the TOF measurement device 1040 outputs a unit histogram corresponding to the amount of incident light in the "2 x 2" SPAD arrangement area as data for one pixel, and the control circuit 1042 determines the presence or absence of reflected light L1002 based on a comparison between the unit histogram and a determination threshold. In the "2 x 2" SPAD arrangement area, the signal waveform corresponding to the reflected light L1002 is relatively small, resulting in a low S / N ratio. As such, in the reference measurement process, the S / N ratio is low, and therefore the peak intensity of the unit histogram constituting one pixel is relatively low, resulting in a relatively low detection performance for the reflected light L1002.

[0099] On the other hand, the number of SPAD units M in the low-resolution measurement process is, for example, "4 x 4." In this case, as shown in the lower right of FIG. 14 , the TOF measurement device 1040 outputs a unit histogram corresponding to the amount of incident light in the "4 x 4" SPAD arrangement area as data for one pixel, and the control circuit 1042 determines the presence or absence of reflected light L1002 based on a comparison between the unit histogram and a determination threshold. In the "4 x 4" SPAD arrangement area, the signal waveform corresponding to the reflected light L1002 is relatively large, resulting in a low S / N ratio. As such, in the low-resolution measurement process, the S / N ratio is high, and therefore the peak intensity of the unit histogram constituting one pixel is relatively high, resulting in relatively high detection performance for the reflected light L1002. However, in the low-resolution measurement process, the resolution of the received light image is lower than in the reference measurement process.

[0100] For example, if the vehicle carrying the measuring device 1010 is moving at high speed (moving at a predetermined speed or higher), it is not preferable to perform low frame rate measurement processing because increasing the frame rate increases the possibility of overlooking an image. In this case, it is preferable to perform low resolution measurement processing to improve detection performance for the reflected light L1002. Note that in the low resolution measurement processing, the SPAD integration count U may be reduced relative to the reference measurement processing to increase the frame rate of the received light image. Furthermore, the number of SPAD units M for each of the reference measurement processing and the low resolution measurement processing is not limited to the example shown in FIG. 14 and can be changed as appropriate. After the TOF measuring device 1040 has performed the low frame rate measurement processing or the low resolution measurement processing for, for example, a predetermined time, it returns to S1120.

[0101] C-4. Generation of Determination Threshold: The determination threshold is preferably appropriately corrected according to the average value of the histogram. Here, when determining the presence or absence of reflected light L1002 based on a comparison between the superimposed histogram and the determination threshold, the following comparative example is an example of a method for generating the determination threshold. This method calculates the average value of the unit histogram at each sampling timing, sums up the average values ​​of the unit histograms for multiple samplings, and corrects the determination threshold according to the sum of the average values. With this method, a rounding error occurs due to rounding each time the average value of the unit histogram is calculated, and the average values ​​of the unit histograms for multiple samplings, including the rounding error, are summed. In other words, the sum of the average values ​​includes the rounding error for multiple samplings. Therefore, there is a risk that the determination threshold cannot be accurately corrected using such a sum of the average values ​​including the rounding error for multiple samplings.

[0102] In contrast, in this embodiment, multiple unit histograms are integrated before calculating the average value, and the average value of the superimposed histogram obtained by this integration is used to correct the determination threshold. In the method of this embodiment, rounding errors occur only once in the calculation of the average value used to correct the determination threshold. Therefore, the method of this embodiment is less affected by rounding errors than the method of the comparative example, and the determination threshold can be corrected with high accuracy.

[0103] In this embodiment, it is preferable to calculate the average value used to correct the determination threshold value based on a histogram generated when the light receiving surface 1032A is not irradiated with the reflected light L1002 and only irradiated with the disturbance light L1003. Fluctuations (fluctuations) in the histogram are mainly due to fluctuations in the disturbance light L1003. Therefore, with the method of this embodiment, it is possible to eliminate the influence of the reflected light L1002 and accurately correct the determination threshold value in accordance with fluctuations in the disturbance light L1003, compared to when the average value is calculated based on a histogram generated when the reflected light L1002 is irradiated on the light receiving surface 1032A.

[0104] C-5. Advantages of this embodiment: As described above, according to the measuring device 1010 of this embodiment, the detection performance, resolution, and frame rate of the reflected light L1002 can be changed by changing at least one of the number M of SPAD units that make up one pixel and the number U of SPAD integrations that make up one pixel. This makes it possible to measure the distance to the measurement target W while changing the detection performance, etc. of the reflected light L1002.

[0105] D. Modifications of the Second Embodiment The technology disclosed in this specification is not limited to the above-described embodiment, and can be modified in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[0106] The configuration of the measuring device 1010 in the above embodiment is merely an example and can be modified in various ways. For example, in the above embodiment, a SPAD is used as the light receiving element, but the present invention is not limited to this and an analog output type light receiving element may also be used.

[0107] In the above embodiment, the detection performance improvement condition is exemplified as the switching condition, but this is not limiting and other conditions may be used, such as receiving a switching instruction signal from an external device. In the flowchart of the measurement process in the above embodiment, the process of S1140 or S1150 may be executed without making the determination in S1130. The second measurement process may have a larger number of light receiving elements and a smaller number of integration times of the output values ​​of the light receiving elements than the first measurement process, or may have a smaller number of light receiving elements and a larger number of integration times of the output values ​​of the light receiving elements than the first measurement process.

[0108] In the above embodiment, the second measurement process (low frame rate measurement process, low resolution measurement process) is performed on the entire light-receiving surface 1032A. However, the second measurement process may be performed on only a partial region of the light-receiving surface 1032A. For example, in the measurement process flowchart of the above embodiment, if the detection performance improvement condition and the region restriction condition are satisfied, the low-resolution measurement process may be performed on a specific region that is part of the light-receiving surface 1032A, and the reference measurement process may be continued on the remaining region of the light-receiving surface 1032A excluding the specific region. The region restriction condition may be, for example, receiving a signal instructing region restriction from an external device or the appearance of a predetermined change pattern (e.g., a change pattern indicating the detection of the measurement object W or a change pattern indicating the detection of an abnormal image) in an image acquired during the reference measurement process (detection of the measurement object W or an abnormal image).

[0109] E. Third Embodiment: E-1. Configuration of Measuring Apparatus 2010: FIG. 15 is a block diagram showing a schematic configuration of a measuring apparatus 2010 according to the third embodiment. As shown in FIG. 15, the measuring apparatus 2010 includes a projector 2020 that irradiates a measurement object W with emitted light L2001 (e.g., a light beam (laser light)), and a photoreceiver 2030 that receives reflected light L2002 (return light) that is generated when the emitted light L2001 is reflected back from the measurement object W, and functions as a LiDAR. The measuring apparatus 2010 acquires information about the measurement object W by measuring the time of flight (TOF) between the time when the projector 2020 emits the emitted light L2001 and the time when the photoreceiver 2030 receives the reflected light L2002.

[0110] The measuring device 2010 is mounted on, for example, a vehicle (not shown) equipped with an AD or ADAS. The measuring device 2010 assists in detecting objects such as people and other vehicles while the vehicle is traveling, and provides various types of information to other devices and users that are useful for ensuring the safety of the driver of the vehicle and those around the vehicle, and for reducing damage to surrounding objects while the vehicle is being driven.

[0111] The light projector 2020 includes a light source 2022 , a light projecting optical system 2024 , a light projecting control device 2026 , and a current source 2028 .

[0112] The light source 2022 includes a light emitting source having one or more light emitting elements (not shown), or one or more light emitting element arrays (for example, light emitting elements arranged linearly (one-dimensionally) or planarly (two-dimensionally)). The light emitting element is, for example, a laser diode, a surface emitting type laser light emitting element (for example, a VCSEL), a surface emitting element array (for example, a VCSEL array) in which a plurality of surface emitting elements are arranged one-dimensionally or two-dimensionally on a substrate (semiconductor substrate, ceramic substrate, etc.), etc.

[0113] The current source 2028 supplies a current corresponding to a control signal input from the light projection control device 2026 to the light emitting element constituting the light source 2022. The current source 2028 supplies, for example, a periodic square wave current to the light emitting element for turning on and off the current flowing through the light emitting element.

[0114] The light-projection control device 2026 generates a control signal for the current source 2028 and inputs it to the current source 2028, thereby controlling the current (drive current) supplied to the light-emitting element from the current source 2028. The light-projection control device 2026 inputs a signal indicating the timing at which the light-emitting element emits light (the timing at which the light-emitting element emits light; hereinafter referred to as "light-projection timing") to the TOF measurement device 2040. The light-projection control device 2026 periodically controls the on / off of a current flowing through the light-emitting element, for example, to cause the light-emitting element to periodically emit light.

[0115] The light projection optical system 2024 adjusts the light distribution of the output light L2001 by, for example, applying an optical effect (such as refraction, scattering, or diffraction) to the light emitted by the light source 2022. The light projection optical system 2024 is configured using optical components such as various lenses, such as collimating lenses, and reflectors (mirrors).

[0116] The light receiver 2030 includes a light receiving section 2032 and a light receiving optical system 2034 .

[0117] The light receiving optical system 2034 collects reflected light L2002, which is light L2001 emitted from the projector 2020 and reflected by the measurement target W or the like, onto the light receiving unit 2032. The light receiving optical system 2034 is configured using optical components such as various lenses such as a condenser lens, various filters such as a wavelength filter, and a reflecting mirror (mirror).

[0118] The light receiving unit 2032 has a plurality of light receiving elements. The light receiving unit 2032 generates a light receiving signal with a current level or voltage level corresponding to the intensity of the reflected light L2002 by photoelectrically converting the reflected light L2002 incident from the light receiving optical system 2034. The light receiving unit 2032 inputs, to the TOF measurement device 2040, a signal indicating the timing at which the light receiving elements constituting the light receiving unit 2032 receive the reflected light L2002 (hereinafter referred to as "light receiving timing") and the light receiving signal generated by the light receiving elements.

[0119] The measuring device 2010 further includes a TOF measuring device 2040, a control circuit 2042, and a communication I / F (Inter face) 2050. In this embodiment, the TOF measuring device 2040 and the control circuit 2042 are an example of a controller.

[0120] The TOF measurement device 2040 calculates the TOF based on a signal indicating the light projection timing input from the light-projection control device 2026 and a signal indicating the light reception timing input from the light-receiving unit 2032. The TOF measurement device 2040 has, for example, a time measurement IC (Integrated Circuit) equipped with a TDC (Time to Digital Converter) circuit. The TOF measurement device 2040 inputs the calculated TOF and the light reception signal input from the light-receiving unit 2032 to the control circuit 2042.

[0121] The control circuit 2042 has a processor (CPU, MPU, ASIC, FPGA, DSP, etc.). The control circuit 2042 generates information used for various measurements such as detection and distance measurement of the measurement target W based on the light reception signal and TOF input from the TOF measurement device 2040. This information includes, for example, a histogram used in time-correlated single photon counting, distances to each point on the measurement target W, and a point cloud (point cloud information). The control circuit 2042 also controls the light-projection control device 2026 and the light-receiving unit 2032. The control circuit 2042 controls the light-projection timing and light-receiving timing described above, for example, by controlling the light-projection control device 2026 and the light-receiving unit 2032, so as to speed up or optimize the processing related to the generation of the histogram. The information generated by the control circuit 2042 is provided (transmitted) via the communication I / F 2050 to a device that uses the information (hereinafter referred to as "various utilization devices 2060").

[0122] The various utilization devices 2060 perform, for example, the creation of an environmental map using a point cloud, self-location estimation (SLAM) using a scan matching algorithm (NDT, ICP, etc.), and the like.

[0123] E-2. Signal Processing in TOF Measurement Device 2040: In this embodiment, the photoreceiver 2030 (light receiving unit 2032) has a light receiving surface 2032A on which a plurality of light receiving elements are arranged in a linear (one-dimensional) or planar (two-dimensional) array. The following description will be given taking as an example a configuration in which the light receiving elements are SPADs, which are an example of photon-counting light receiving elements. The SPAD responds with a predetermined probability (e.g., 20%) when a photon is incident, and outputs one pulse when a photon is detected.

[0124] In the light receiving unit 2032 using SPADs, one pixel in an image (hereinafter referred to as a "received light image") acquired based on a light receiving signal from the SPAD is defined by the "number of SPAD units M" and the "number of SPAD integrations U." The number of SPAD units M is the number of adjacent SPADs on the light receiving surface 2032A. For example, in a configuration in which multiple SPADs are arranged two-dimensionally on the light receiving surface 2032A, the number of SPAD units M is 9 (= 3 × 3) or 16 (= 4 × 4). The larger the number of SPAD units M, the lower the resolution of the received light image. The number of SPAD integrations U is the number of times the output value of each SPAD is integrated at a predetermined sampling timing. A larger number of SPAD integrations U means a longer integration time for the SPAD output values. The larger the number of SPAD integrations U, the lower the frame rate of the received light image (the number of images (frames) constituting a moving image per second). The TOF measurement device 2040 generates a histogram for each pixel based on the output values ​​of the SPADs that make up each pixel.

[0125] FIG. 16 is an explanatory diagram showing the relationship between the output values ​​of the SPADs constituting one pixel and a histogram. The upper part of FIG. 16 shows the temporal changes in the output values ​​of each SPAD (SPADs 2001 to 2003). The lower part of FIG. 16 shows a graph G showing the temporal changes in the incident light amount (received light intensity, irradiation intensity) per unit area of ​​light (including reflected light L2002 and ambient light (background light) L2003) irradiated to one pixel of the light-receiving surface 2032A, and a histogram H generated by the TOF measurement device 2040. Q2001 in FIG. 16 represents one pulse output when each SPAD detects a photon when reflected light L2002 is incident, and Q2002 represents one pulse output when each SPAD detects a photon when only ambient light L2003 is incident without reflected light L2002.

[0126] In graph G, the waveform showing a relatively high peak intensity is the waveform when reflected light L2002 is incident on the light-receiving surface 2032A, and the waveform showing a relatively low peak intensity is the waveform when reflected light L2002 is not incident and only disturbance light L2003 is incident on the light-receiving surface 2032A. The incident light amount of reflected light L2002 is higher than the incident light amount of disturbance light L2003. Therefore, as shown in histogram H, when reflected light L2002 is incident on the light-receiving surface 2032A, the number of detected photons per pixel is statistically higher, and when reflected light L2002 is not incident and only disturbance light L2003 is incident on the light-receiving surface 2032A, the number of detected photons per pixel is statistically lower. The TOF measurement device 2040 determines the light-receiving timing based on the timing corresponding to the waveform showing a relatively high peak in histogram H.

[0127] E-3. Memory Processing: Fig. 17 is a flowchart showing part of the memory processing. This memory processing is a process for measuring the distance to the measurement target W while reducing the memory capacity for storing the light reception data. The TOF measurement device 2040 uses the memory 2044 to store the light reception data (output values) output from the multiple SPADs.

[0128] 17 , the TOF measurement device 2040 determines the memory area to be used (S2110). The memory area to be used is an area of ​​the memory 2044 to be used for storing received light data, and is determined by a reference value. The reference value is the upper limit of the memory area to be used (used memory capacity) of the memory 2044. The memory area to be used can be changed by changing the reference value (an example of a reference value change process). The reference value may be set to a value corresponding to the entire memory area of ​​the memory 2044, or may be set to a value corresponding to a portion of the memory area of ​​the memory 2044.

[0129] Next, the TOF measurement device 2040 starts the update process, triggered by the light projection timing of the projector 2020. In the update process, the TOF measurement device 2040 writes the light reception data output from the plurality of SPADs into the usable memory area of ​​the memory 2044 after the light projection timing of the projector 2020, and when the amount of written light reception data reaches the reference value, repeatedly executes an update operation in which the usable memory area is overwritten with the light reception data output from the plurality of SPADs thereafter.

[0130] Specifically, the TOF measurement device 2040 first starts a process of writing light reception data (histograms) output from multiple SPADs into a used memory area of ​​the memory 2044 (S2120). Next, the TOF measurement device 2040 determines whether or not the reflected light L2002 has been detected based on the light reception data (histograms) written in the used memory area (S2130). Specifically, the TOF measurement device 2040 compares the light reception data (histograms) written in the used memory area with a signal detection threshold to determine whether or not the reflected light reception data, which is light reception data corresponding to the reflected light L2002, is included.

[0131] If the TOF measurement device 2040 determines that the reflected light L2002 has not been detected (S2130: NO), it determines whether the amount of written light reception data has reached the reference value (S2150). If the TOF measurement device 2040 determines that the amount of written light reception data has not reached the reference value (S2150: NO), it returns to S2130 and continues writing the light reception data to the used memory area. On the other hand, if the TOF measurement device 2040 determines that the amount of written light reception data has reached the reference value (S2150: YES), it starts a process of overwriting the used memory area with the light reception data output from the multiple SPADs (S2160). In other words, a light reception data group (histogram) that does not include reflected light reception data is basically not used to measure the distance to the measurement target W, so there is no problem if it is lost by overwriting. After starting the overwriting process, the TOF measurement device 2040 adds 1 to the number of overwrites N (S2170) and returns to S2130. That is, the TOF measurement device 2040 counts the number of times of overwriting.

[0132] On the other hand, if the TOF measuring device 2040 determines that reflected light L2002 has been detected (S2130: YES), it copies and extracts the light reception data group (histogram) contained in the reflected light reception data and outputs it to the control circuit 2042 (S2140), and the control circuit 2042 measures the distance to the measurement object W based on the extracted light reception data group and the number of overwrites N.

[0133] 18 is an explanatory diagram illustrating memory processing in the example and the comparative example. The memory processing in the comparative example is a process in which all light reception data output from multiple SPADs is continuously written to memory without overwriting until reflected light L2002 is detected. For example, to be able to measure the distance to a measurement target W up to 200 m away, a large memory capacity M is required that can store all of the light reception data for the estimated time from when light is emitted from the projector 2020, when it is reflected by the measurement target W up to 200 m away, and when it returns to the light receiver 2030. Therefore, a memory area equivalent to the memory capacity M must be secured in the memory 2044.

[0134] On the other hand, in the memory processing of the embodiment, the light reception data output from the multiple SPADs is written to a usable memory area with a memory capacity m(K) that is smaller than the memory capacity M, and when the amount of written light reception data reaches the upper limit of the memory capacity m(K), the usable memory area is overwritten. In the example of Fig. 18, since reflected light reception data exceeding the signal detection threshold is included during the third and fourth overwriting processes, the third and fourth light reception data groups are output to the control circuit 2042. Therefore, in the memory processing of the embodiment, if a memory area with a memory capacity m(K) is secured in the memory 2044, it is possible to measure the distance to the measurement target W that is up to 200 m away.

[0135] After outputting the light reception data group including the reflected light reception data (S2140), the TOF measurement device 2040 stops the update process, frees up the memory area for memory processing for other pixels (S2180), and terminates this memory processing. FIG. 19 is an explanatory diagram illustrating memory processing for a first pixel and a second pixel. As shown in FIG. 19 , if the reflected light L2002 is detected for a relatively short period ΔTS for the first pixel and the update process is stopped, the memory area for the first pixel is not used for the remaining time Δtf. Therefore, the memory area for the first pixel is freed up for update processing by the second pixel. This expands the memory area for the second pixel, lengthening the time until the reflected light L2002 is detected after the light projection timing, making it possible to measure the distance to a measurement target W that is farther away.

[0136] E-4. Advantages of this embodiment: As described above, according to the measuring device 2010 of this embodiment, it is possible to measure the distance to the measurement object W while reducing the memory capacity required to store the light reception data, compared to a configuration in which light reception data for the estimated time period from when light is emitted from the light projector 2020, reflected by the measurement object W, and returned to the light receiver 2030 is written to memory.

[0137] F. Modifications of the Third Embodiment The technology disclosed in this specification is not limited to the above-described embodiment, and can be modified in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[0138] The configuration of the measuring device 2010 in the above embodiment is merely an example, and various modifications are possible.

[0139] In the flowchart of the measurement process in the above embodiment, either the process of S2110 or the process of S2180 does not have to be executed.

[0140] G. Fourth Embodiment: G-1. Configuration of Measuring Apparatus 3010: FIG. 20 is a block diagram showing a schematic configuration of a measuring apparatus 3010 according to the fourth embodiment. As shown in FIG. 20, the measuring apparatus 3010 includes a projector 3020 that irradiates a measurement object W with emitted light L3001 (e.g., a light beam (laser light)), and a photoreceiver 3030 that receives reflected light L3002 (return light) that is generated when the emitted light L3001 is reflected back from the measurement object W, and functions as a LiDAR. The measuring apparatus 3010 acquires information about the measurement object W by measuring the time of flight (TOF) between the time when the projector 3020 emits the emitted light L3001 and the time when the photoreceiver 3030 receives the reflected light L3002.

[0141] The measuring device 3010 is mounted on, for example, a vehicle (not shown) equipped with an AD or ADAS. The measuring device 3010 assists in detecting objects such as people and other vehicles while the vehicle is traveling, and provides various types of information to other devices and users that are useful for ensuring the safety of the driver of the vehicle and those around the vehicle, and for reducing damage to surrounding objects while the vehicle is being driven.

[0142] The light projector 3020 includes a light source 3022 , a light projecting optical system 3024 , a light projecting control device 3026 , and a current source 3028 .

[0143] The light source 3022 includes a light emitting source having one or more light emitting elements (not shown), or one or more light emitting element arrays (for example, light emitting elements arranged linearly (one-dimensionally) or planarly (two-dimensionally)). The light emitting element may be, for example, a laser diode, a surface emitting laser light emitting element (for example, a VCSEL), or a surface emitting element array (for example, a VCSEL array) in which a plurality of surface emitting elements are arranged one-dimensionally or two-dimensionally on a substrate (such as a semiconductor substrate or a ceramic substrate).

[0144] The current source 3028 supplies a current corresponding to a control signal input from the light projection control device 3026 to the light emitting element constituting the light source 3022. The current source 3028 supplies, for example, a periodic square wave current to the light emitting element for turning on and off the current flowing through the light emitting element.

[0145] The light-projection control device 3026 generates a control signal for the current source 3028 and inputs it to the current source 3028, thereby controlling the current (drive current) supplied to the light-emitting element from the current source 3028. The light-projection control device 3026 inputs a signal indicating the timing at which the light-emitting element emits light (the timing at which the light-emitting element emits light; hereinafter referred to as "light-projection timing") to the TOF measurement device 3040. The light-projection control device 3026 periodically controls the on / off of a current flowing through the light-emitting element, for example, to cause the light-emitting element to periodically emit light.

[0146] The light projection optical system 3024 adjusts the light distribution of the output light L3001 by, for example, applying an optical effect (such as refraction, scattering, or diffraction) to the light emitted by the light source 3022. The light projection optical system 3024 is configured using optical components such as various lenses such as collimating lenses, and reflecting mirrors (mirrors).

[0147] The light receiver 3030 includes a light receiving section 3032 and a light receiving optical system 3034 .

[0148] The light receiving optical system 3034 collects reflected light L3002, which is light L3001 emitted from the projector 3020 and reflected by the measurement target W or the like and returns, onto the light receiving unit 3032. The light receiving optical system 3034 is configured using optical components such as various lenses such as a collecting lens, various filters such as a wavelength filter, and a reflecting mirror (mirror).

[0149] The light receiving unit 3032 has a plurality of light receiving elements. The light receiving unit 3032 generates a light receiving signal with a current level or voltage level corresponding to the intensity of the reflected light L3002 by photoelectrically converting the reflected light L3002 incident from the light receiving optical system 3034. The light receiving unit 3032 inputs to the TOF measurement device 3040 a signal indicating the timing at which the light receiving elements constituting the light receiving unit 3032 receive the reflected light L3002 (hereinafter referred to as "light receiving timing"), and the light receiving signal generated by the light receiving elements.

[0150] The measuring device 3010 further includes a TOF measuring device 3040, a control circuit 3042, and a communication I / F (Inter face) 3050. In this embodiment, the TOF measuring device 3040 and the control circuit 3042 are an example of a controller.

[0151] The TOF measurement device 3040 calculates the TOF based on a signal indicating the light projection timing input from the light-projection control device 3026 and a signal indicating the light reception timing input from the light-receiving unit 3032. The TOF measurement device 3040 has, for example, a time measurement IC equipped with a TDC circuit. The TOF measurement device 3040 inputs the calculated TOF and the light reception signal input from the light-receiving unit 3032 to the control circuit 3042.

[0152] The control circuit 3042 has a processor (CPU, MPU, ASIC, FPGA, DSP, etc.). The control circuit 3042 generates information used for various measurements such as detection and distance measurement of the measurement target W based on the light reception signal and TOF input from the TOF measurement device 3040. This information includes, for example, a histogram used in time-correlated single photon counting, distances to each point on the measurement target W, and a point cloud (point cloud information). The control circuit 3042 also controls the light-projection control device 3026 and the light-receiving unit 3032. The control circuit 3042 controls the light-projection timing and light-receiving timing described above, for example, by controlling the light-projection control device 3026 and the light-receiving unit 3032, so as to speed up or optimize the processing related to the generation of the histogram. The information generated by the control circuit 3042 is provided (transmitted) via the communication I / F 3050 to a device that uses the information (hereinafter referred to as "various utilization devices 3060").

[0153] The various utilization devices 3060 perform, for example, the creation of an environmental map using a point cloud, self-location estimation (SLAM) using a scan matching algorithm (NDT, ICP, etc.), and the like.

[0154] G-2. Variation in light projection timing and light reception signal output timing: Figure 21 is an explanatory diagram showing the variation in light projection timing and light reception signal output timing and the results of distance measurement. In the light projector 3020 of this embodiment, light emitting elements are arranged in a line, and vertical pixel NO is a number assigned to each pixel composed of a predetermined number of light emitting elements. The following description will be given using an example where the number of units is one, but the number of units may be multiple.

[0155] The lower left of Fig. 21 shows the time from input of a trigger signal to light emission for each of the multiple light-emitting elements (VCSELs) included in the projector 3020. As shown in the lower left of Fig. 21, there is variation in the light-projection timing of the multiple light-emitting elements. Causes of the variation in light-projection timing include variation in the length of the wiring connected to the multiple light-projecting elements and variation in the output timing of the trigger signal given to the projector 3020.

[0156] The lower right of Fig. 21 shows the time from input of a trigger signal to output of a light-receiving signal for each of the multiple light-receiving elements (SPAD) included in the light receiver 3030. As shown in the lower right of Fig. 21, there is variation in the output timing of the light-receiving signals from the multiple light-receiving elements. Causes of the variation in the output timing of the light-receiving signals include variations in the length of the wiring connected to the multiple light-receiving elements and variations in response characteristics.

[0157] The upper part of Figure 21 shows the results of measuring the distance to the measurement target W when there is variation in the light projection timing and the output timing of the light reception signal. This measurement target W has, for example, a flat surface. Emitted light L3001 is irradiated onto the flat surface from multiple light-emitting elements of the light projector 3020, and reflected light L3002 reflected back from the flat surface is received by multiple light-receiving elements of the light receiver 3030. Assume that the distance between each of the multiple light-emitting elements and the flat surface is uniform, and that the distance between each of the multiple light-receiving elements and the flat surface is also uniform. In this case, the distance from W should be uniform for all pixels. However, in the upper part of Figure 21, the measurement results of the distance to the measurement target W vary from pixel to pixel. This is due to variations in the light projection timing and the output timing of the light reception signal.

[0158] G-3. Manufacturing method of the measuring device 3010: Figure 22 is a flowchart showing part of the manufacturing process of the measuring device 3010. First, the measuring device 3010 is assembled (S3110). Note that assembly of the measuring device 3010 does not need to be complete at this point; it is sufficient if the acquisition step of acquiring a sample light reception signal, which will be described later, can be carried out.

[0159] Next, a sample measurement object having a flat surface is prepared (S3120). This step S3120 is an example of a preparation step. The sample measurement object is, for example, a plate-shaped object and has a flat surface. Furthermore, the flat surface has a uniform reflectance overall.

[0160] Next, multiple sample light reception signals are acquired (S3130). This step S3130 is an example of an acquisition step. Specifically, multiple light-emitting elements of the light projector 3020 irradiate emitted light L3001 toward the flat surface, and multiple light-receiving elements of the light receiver 3030 receive reflected light L3002 reflected by the flat surface, thereby acquiring multiple sample light reception signals output from each of the multiple light-receiving elements. It is assumed that the distance between each of the multiple light-emitting elements and the flat surface is uniform, and that the distance between each of the multiple light-receiving elements and the flat surface is also uniform. Based on the acquired multiple sample light reception signals, the distance to the flat surface for each pixel is measured, and an image reflecting the measurement results (hereinafter referred to as a "measurement result image") is obtained. Here, if there is no variation in the light projection timing or the output timing of the light-receiving signals, the measurement result image will be a uniform, monochromatic image. On the other hand, if there is variation in the timing of light projection or the timing of output of the light reception signal (see the upper part of FIG. 21), the measurement result image will have variations in density for each pixel, resulting in an uneven mixed-color image.

[0161] Therefore, the timing of projecting light and the timing of outputting the light-receiving signal are corrected (S3140). This step S3140 is an example of a correction step. At least one of the timing of light emission from the light-emitting element and the timing of outputting the light-receiving signal from the light-receiving element is corrected so that the distance measurement results for each pixel become closer to uniform. Specifically, there are correction methods using wiring, correction methods using a light-emission trigger signal, and correction methods using measurement results.

[0162] (1) Wiring-Based Correction Method The wiring-based correction method involves changing at least one of the wiring lengths connected to the light-emitting element and the light-receiving element. FIG. 23 is an explanatory diagram of an example of the wiring-based correction method. The current source 3028 is omitted from FIG. 23 . The upper left side of FIG. 23 shows the wiring state before correction for the light-emitter 3020. Each of the multiple light-emitting elements (V3001, V3002, V3003, etc.) is electrically connected to the light-emitter control device 3026 via wiring (L3001, L3002, L3003, etc.). The lengths of the wiring (L3001, L3002, L3003, etc.) connected to each of the multiple light-emitting elements are the same. The light-emitter control device 3026 sequentially outputs trigger signals to the multiple light-emitting elements at uniform time intervals ΔT. However, as shown in the upper right side of FIG. 23 , the light-emitter timing intervals for the multiple light-emitting elements vary due to variations in the response performance of the multiple light-emitting elements, etc. The light projection timing interval ΔT1 between the light emitting elements V3001 and V3002 is longer than the time interval ΔT, and the light projection timing interval ΔT2 between the light emitting elements V3002 and V3003 is shorter than the time interval ΔT.

[0163] Therefore, the lengths of the wiring are changed so as to suppress the variation in the light-projection timing intervals of the multiple light-emitting elements. For example, as shown in the lower left side of Fig. 23 , the wiring L3002a connected to the light-emitting element V3002 is made shorter than the wiring L3001 connected to the light-emitting element V3001, and the wiring L3003a connected to the light-emitting element V3003 is made longer than the wiring L3001 connected to the light-emitting element V3001. This suppresses the variation in the light-projection timing intervals of the multiple light-emitting elements, as shown in the lower right side of Fig. 23 . According to the wiring-based correction method, by adjusting the hardware configuration, it is possible to suppress a decrease in the measurement accuracy of the distance to the measurement target W.

[0164] (2) Correction Method Using Light Emission Trigger Signal The correction method using a light emission trigger signal changes the output timing of a trigger signal that the light-projection control device 3026 provides to the light-emitting element. In the example in the upper part of Fig. 23 , the output timing of the trigger signal that the light-projection control device 3026 outputs is changed without changing the wiring length so as to suppress variations in the light-projection timing intervals of multiple light-emitting elements. According to the correction method using a light-emission trigger signal, by adjusting the configuration on the light-projector 3020 side, it is possible to suppress a decrease in the measurement accuracy of the distance to the measurement target W.

[0165] (3) Correction Method Based on Measurement Results The correction method based on measurement results of the distance for each pixel generates correction information for correcting the measurement results obtained by the TOF measurement device 3040 and the control circuit 3042 based on the light-receiving signals from the light-receiving elements so that the measurement results approach uniformity. The correction information is, for example, a correction value that offsets the error in the distance measurement between the pixels in the upper row of Fig. 21. The correction method based on measurement results makes it possible to prevent a decrease in the measurement accuracy of the distance to the measurement target W by correcting software processing without requiring adjustment work on the hardware configuration.

[0166] G-4. Advantages of this embodiment: As described above, it is possible to prevent a decrease in the measurement accuracy of the distance to the measurement target W due to, for example, variations in the timing of light projection from the light projector 3020 or variations in the timing of output of the light reception signal from the light receiving element in the light receiver 3030.

[0167] H. Modifications of the Fourth Embodiment The technology disclosed in this specification is not limited to the above-described embodiment, and can be modified in various forms without departing from the spirit of the invention. For example, the following modifications are also possible.

[0168] The configuration of the measuring device 3010 in the above embodiment is merely an example, and various modifications are possible.

[0169] The flowchart of the manufacturing method of the measurement process in the above embodiment is an example and can be modified in various ways.

[0170] This application is based on a Japanese patent application filed on August 10, 2023 (Patent Application No. 2023-131178), a Japanese patent application filed on August 10, 2023 (Patent Application No. 2023-131175), a Japanese patent application filed on August 10, 2023 (Patent Application No. 2023-131177), and a Japanese patent application filed on August 10, 2023 (Patent Application No. 2023-131176), the contents of which are incorporated herein by reference.

[0171] According to the present disclosure, even when the amount of incident light of reflected light on the light receiving surface is saturated, it is possible to estimate the light receiving timing based on the first arrival timing and the second arrival timing.

Claims

1. A measuring device comprising: a light projector that emits light; a light receiver having a light receiving surface in which a plurality of photon counting type light receiving elements are arranged in an array, and that receives reflected light that is returned from the light projector and reflected by an object to be measured; and a controller, wherein the controller executes the following steps: a histogram generation process that generates a histogram based on output values ​​from the plurality of light receiving elements; a light receiving timing estimation process that estimates the light receiving timing of the reflected light based on a first arrival timing at which the histogram first reaches an intermediate value between the maximum and minimum values ​​of the waveform of the histogram, and a second arrival timing at which the histogram next reaches the intermediate value; and an output process that outputs information corresponding to the distance to the object to be measured, based on the light projecting timing at which the light projector emits light and the light receiving timing.

2. A measuring device according to claim 1, wherein the controller further executes a light receiving timing correction process for correcting the light receiving timing according to the height of the peak of the histogram when the histogram is below a saturation level.

3. A measuring device as described in claim 1 or claim 2, wherein the controller further executes a light receiving timing correction process for correcting the light receiving timing in accordance with the length of the time difference between the first arrival time and the second arrival time when the histogram exceeds a saturation level.

4. A measuring device as described in claim 1 or claim 2, wherein the controller further executes a disturbance light histogram generation process that generates a disturbance light histogram based on output values ​​from the multiple light receiving elements when the reflected light is not irradiated onto the light receiving surface, and an arrival timing correction process that corrects the second arrival timing depending on the height of the peak of the disturbance light histogram.

5. A measuring device comprising: a light projector that emits light; a light receiver having a light receiving surface on which a plurality of photon counting type light receiving elements are arranged in an array, and that receives reflected light that is the light emitted from the light projector and reflected back to an object to be measured; and a controller, wherein the controller executes a histogram generation process that generates a histogram based on output values ​​from the plurality of light receiving elements; and a first incident light amount estimation process that estimates the amount of incident light of the reflected light based on the time difference between a first arrival timing at which the histogram first reaches an intermediate value between the maximum and minimum values ​​of the histogram waveform, and a second arrival timing at which the histogram next reaches the intermediate value.

6. A measuring device according to claim 5, wherein the controller further executes a second incident light amount estimation process for estimating the amount of incident reflected light based on output values ​​from the plurality of light receiving elements and a logarithmic function expressed by the following formula when the peak of the histogram is lower than a saturation level and the difference between the peak of the histogram and the amount of incident disturbance light is equal to or greater than a reference value. （ａ×受光素子からの出力値＋ｂ） Here, a and b are coefficients.

7. A measuring device according to claim 5 or 6, wherein the controller further executes a third incident light amount estimation process for estimating the amount of incident light of the reflected light based on output values ​​from the multiple light receiving elements and a real function represented by the following formula when the peak of the histogram is lower than a saturation level and the difference between the peak of the histogram and the amount of incident disturbance light is less than a reference value: Incident light amount of reflected light = c × output value from light receiving element + d, where c and d are coefficients.

8. A measuring device according to claim 6, wherein the controller further executes a first coefficient correction process for correcting the coefficient a of the logarithmic function to a smaller value as the amount of incident ambient light increases.

9. A measuring device according to claim 7, wherein the controller further executes a second coefficient correction process for correcting the coefficient c of the real function to a smaller value as the amount of incident ambient light increases.

10. A measuring device comprising: a light-projector that emits light; a light-receiving device having a light-receiving surface in which a plurality of light-receiving elements are arranged in an array to receive light that is reflected back from the light-projector by a measurement target; and a controller, wherein the controller executes a first measurement process to acquire output values ​​of the plurality of light-receiving elements arranged in at least a part of a target area of ​​the light-receiving surface in a first acquisition unit determined by the number of the light-receiving elements that constitute one pixel and the number of times the output values ​​of the light-receiving elements are accumulated, and to measure the distance to the measurement target based on the obtained output result; and a second measurement process to acquire output values ​​of the plurality of light-receiving elements arranged in the target area in a second acquisition unit in which at least one of the number of the light-receiving elements that constitute one pixel and the number of times the output values ​​of the light-receiving elements are accumulated is different from the first acquisition unit, and to measure the distance to the measurement target based on the obtained output result.

11. A measuring device as described in claim 10, wherein the second acquisition unit has a larger output sum value than the first acquisition unit, the sum value being determined based on the number of light receiving elements constituting one pixel and the number of times the output values ​​of the light receiving elements are integrated, and the controller switches to and executes the second measurement process when a switching condition is satisfied during execution of the first measurement process.

12. A measuring device as described in claim 10 or 11, wherein the second acquisition unit has a larger number of light receiving elements constituting one pixel than the first acquisition unit, and the number of times the output values ​​of the light receiving elements are integrated is equal to or less than that of the first acquisition unit.

13. A measuring device as described in claim 10 or 11, wherein the second acquisition unit has an equal or smaller number of light receiving elements constituting one pixel than the first acquisition unit, and the number of times the output values ​​of the light receiving elements are integrated is greater than that of the first acquisition unit.

14. A measuring device as described in claim 10 or 11, wherein the controller, when a switching condition is satisfied and an area restriction condition is satisfied during execution of the first measurement process, executes the second measurement process on a specific area that is part of the light-receiving surface, and executes the first measurement process on other areas of the light-receiving surface excluding the specific area.

15. A measuring device comprising: a light projector that emits light; a light receiver having a light receiving surface in which a plurality of light receiving elements are arranged in an array for receiving reflected light that is returned when the light emitted from the light projector is reflected off a measurement object; a controller; and a memory, wherein the controller executes an update process that writes light reception data output from the plurality of light receiving elements into the memory after the light projector emits light, and when the amount of light reception data written reaches a reference value of the memory, repeatedly executes an update operation of overwriting the memory with the light reception data output from the plurality of light receiving elements thereafter; and a measurement process that, when reflected light reception data, which is the light reception data corresponding to the reflected light, is written into the memory, extracts a group of light reception data including the reflected light reception data and measures the distance to the measurement object.

16. A measuring device according to claim 15, wherein said controller further executes a reference value changing process for changing said reference value in said memory.

17. A measuring device according to claim 15, wherein said controller stops said updating process when said reflected light reception data is written to said memory.

18. A measuring device as described in claim 17, wherein when the controller stops the update process for a first pixel in the optical receiver, the controller uses the memory area allocated to the first pixel for updating the received light data for a second pixel.

19. A measurement method for a measuring device comprising a light projector that emits light, a light receiver having a light receiving surface in which a plurality of light receiving elements are arranged in an array for receiving reflected light that is emitted from the light projector and reflected back from a measurement object, and a memory, the method comprising: an update step of writing light reception data output from the plurality of light receiving elements to the memory after the timing of light projection by the light projector, and when the amount of light reception data written reaches a reference value of the memory, repeatedly performing an update operation of overwriting the memory with the light reception data output from the plurality of light receiving elements thereafter; and a measurement step of, when reflected light reception data, which is the light reception data corresponding to the reflected light, is written to the memory, extracting a group of light reception data including the reflected light reception data and measuring the distance to the measurement object.

20. A method for manufacturing a measuring device comprising a light-projector having one or more light-emitting elements, a light-receiver having a light-receiving surface in which a plurality of light-receiving elements are arranged in an array to receive light emitted by the light-projector and reflected back from a measurement object, and a controller for controlling the light-projector and the light-receiver, the method comprising: a preparation step of preparing a sample measurement object having a flat surface; an acquisition step of emitting light from the light-projector toward the flat surface of the sample measurement object, receiving the reflected light reflected back from the flat surface at the light-receiving surface of the receiver, and acquiring measurement results of the distance to the flat surface for each pixel based on a plurality of sample light-receiving signals output from each of the plurality of light-receiving elements; and a correction step of correcting at least one of the light-emitting timing of the light-emitting element and the output timing of the light-receiving signal from the light-receiving element so that the measurement results of the distance for each pixel become closer to uniform.

21. A method for manufacturing a measuring device according to claim 20, wherein in the correction step, at least one of the length of the wiring connected to the light-emitting element and the length of the wiring connected to the light-receiving element is changed.

22. A method for manufacturing a measuring device according to claim 20, wherein in the correction step, the output timing of a trigger signal that the controller provides to the light-emitting element is changed.

23. A method for manufacturing a measuring device as described in claim 20, wherein in the correction process, the controller generates correction information for correcting the measurement results determined based on the light receiving signal from the light receiving element so that the measurement results approach uniformity based on the distance measurement results for each pixel.

Images