Distance measuring device, distance measuring method, and program

Abstract

A distance measuring device according to one embodiment of the present technology is provided with a light source, a plurality of light receiving element portions, a histogram generating unit, a feature extracting unit, and an intensity calculating unit. The light source emits light. The plurality of light receiving element portions receive reflected light from an object that has reflected the emitted light, and output electric signals. The histogram generating unit generates, for each of the plurality of light receiving element portions, a histogram relating to the time from when the light source emits the light until the light receiving element portion receives the reflected light, on the basis of the electric signals output from the light receiving element portions. The feature extracting unit extracts from the histogram, which has a saturation region in which a saturation signal level of the light receiving element portions is reached, a feature including information relating to the time width of the saturation region. The intensity calculating unit calculates the intensity of the reflected light on the basis of the extracted feature.

Description

Ranging apparatus, ranging method, and program

[0001] The present technology relates to a ranging apparatus, a ranging method, and a program.

[0002] In recent years, ranging apparatuses that directly measure the distance to an object using ToF (Time of Flight) have been proposed. Patent Document 1 discloses a technology related to a ranging apparatus using ToF, which makes it possible to improve measurement accuracy while reducing power consumption.

[0003] Japanese Unexamined Patent Application Publication No. 2020-106397

[0004] For ranging apparatuses such as those exemplified in Patent Document 1, there is a demand for a technology capable of achieving high measurement accuracy.

[0005] In view of the above circumstances, an object of the present technology is to provide a ranging apparatus, a ranging method, and a program capable of achieving high measurement accuracy.

[0006] To achieve the above object, a ranging apparatus according to one aspect of the present technology includes a light source, a plurality of light receiving element units, a histogram generation unit, a feature amount extraction unit, and an intensity calculation unit. The light source emits light. The plurality of light receiving element units receive reflected light from an object that reflects the emitted light and output electric signals. For each of the plurality of light receiving element units, the histogram generation unit generates a histogram related to a time period from when the light source emits the light to when the light receiving element unit receives the reflected light, based on the electric signal output from the light receiving element unit. The feature amount extraction unit extracts a feature amount including information on a time width of a saturation region from the histogram that has the saturation region where a signal reaches a saturation signal level of the light receiving element unit. The intensity calculation unit calculates an intensity of the reflected light based on the extracted feature amount.

[0007] In this ranging apparatus, a feature amount including information on the time width of the saturation region is extracted from the histogram based on the time from emission of light to reception of the reflected light. The intensity of the reflected light is calculated based on the extracted feature amount. This makes it possible to achieve high measurement accuracy.

[0008] Each of the aforementioned plurality of light-receiving element sections may consist of one or more light-receiving elements.

[0009] The one or more light-receiving elements may be one or more SPADs (Single Photon Avalanche Diodes).

[0010] The saturation signal level of the light-receiving element may be defined in accordance with the number of SPADs (Surface-to-Air Devices) that the light-receiving element has.

[0011] The aforementioned feature quantity may include the saturation signal level of the photodetector. In this case, the intensity calculation unit may calculate the intensity of the reflected light based on the information regarding the time width of the saturation region and the saturation signal level.

[0012] The intensity calculation unit may calculate the intensity of the reflected light by adding the information regarding the time width of the saturation region and the saturation signal level.

[0013] The information relating to the time width of the saturation region may be the time width of the saturation region.

[0014] The distance measuring device may further include a grayscale value calculation unit that calculates a grayscale value corresponding to each of the plurality of light-receiving element units based on the intensity of the reflected light.

[0015] The grayscale value calculation unit may classify the object into a plurality of object groups based on the light reflection characteristics, and calculate the grayscale value based on a plurality of threshold values ​​relating to the intensity of the reflected light for distinguishing each of the classified plurality of object groups.

[0016] The plurality of object groups may include a weakly reflective object group with a relatively low intensity of reflected light, a strongly reflective object group with a relatively higher intensity of reflected light than the weakly reflective object group, and an extremely strongly reflective object group with a relatively higher intensity of reflected light than the strongly reflective object group. In this case, the gradation value calculation unit may calculate the gradation value based on a first threshold value calculated from the weakly reflective object group based on the intensity of reflected light, a second threshold value calculated from the strongly reflective object group based on the intensity of reflected light, and a third threshold value calculated from the extremely strongly reflective object group based on the intensity of reflected light.

[0017] The gradation value calculation unit may linearly associate the intensity of the reflected light included in the range from 0 to the first threshold with a predetermined first gradation value range, linearly associate the intensity of the reflected light included in the range from the first threshold to the second threshold with a second gradation value range which has higher gradation values ​​than the first gradation value range, linearly associate the intensity of the reflected light included in the range from the second threshold to the third threshold with a third gradation value range which has higher gradation values ​​than the second gradation value range, and linearly associate the intensity of the reflected light included in the range from the third threshold to the maximum value of the intensity of the reflected light with a fourth gradation value range which has higher gradation values ​​than the third gradation value range.

[0018] The group of weakly reflective objects may include roads. In this case, the group of strongly reflective objects may include road markings. The group of extremely strongly reflective objects may also include retroreflective members.

[0019] A distance measuring method according to one embodiment of this technology is a distance measuring method performed by a distance measuring device, and includes emitting light from a light source. Each of a plurality of light-receiving elements receives reflected light from an object that has reflected the emitted light, and outputs an electrical signal. For each of the plurality of light-receiving elements, a histogram relating to the time from when the light source emits the light until when the light-receiving element receives the reflected light is generated based on the electrical signal output from the light-receiving element. A feature quantity including information about the time width of the saturation region is extracted from the histogram which has a saturation region where the saturation signal level of the light-receiving element is reached. Based on the extracted feature quantity, the intensity of the reflected light is calculated.

[0020] A program relating to one embodiment of this technology is a program that causes a distance measuring device to execute the distance measuring method.

[0021] This figure shows a basic configuration example of a distance measuring device according to one embodiment of this technology. This is a schematic diagram showing an example configuration of the light emitting unit and the light receiving unit. This is a schematic diagram showing an example of a histogram of arrival time. This is a schematic diagram to explain an example of a feature quantity (histogram when unsaturated). This is a schematic diagram to explain an example of a feature quantity (histogram when saturated). This is a block diagram showing an example configuration of the distance / intensity measuring unit. This is a schematic diagram showing an example of calculation of grayscale values. This figure shows an example of a measurement result image (ambient light). This figure shows an example of a measurement result image (saturated ⇒ saturated signal level). This figure shows an example of a measurement result image (saturated ⇒ full width at half maximum). This figure shows an example of a measurement result image (saturated ⇒ saturated signal level + peak width). This figure shows an example of a measurement result image (saturated ⇒ saturated signal level + peak width, and grayscale value adjustment). This is a flowchart showing an example of processing from the calculation of the intensity of reflected light and the calculation of grayscale values ​​by the distance measuring device. This is a schematic diagram showing an example of operation of grayscale value adjustment. This is a diagram to explain the calculation of multiple thresholds for grayscale value adjustment. This is a schematic diagram showing an example of calculation of grayscale values ​​by grayscale value adjustment. This is a schematic diagram showing an example of calculating gradation values ​​by adjusting gradation values. This is a graph of the gradation values ​​of the measurement result image before gradation value adjustment shown in Figure 11. This is a graph of the gradation values ​​of the measurement result image after gradation value adjustment shown in Figure 12. This is a flowchart showing an example of the distance measurement method (method for calculating the intensity of reflected light L2) using this distance measuring device. This is a schematic diagram showing an example of the use of the distance measuring device and distance measuring system related to this technology. This is a block diagram showing an example of the general configuration of a vehicle control system. This is an explanatory diagram showing an example of the installation position of the external information detection unit and the imaging unit.

[0022] The embodiments of this technology will be described below with reference to the drawings.

[0023] [Basic Configuration Example of Distance Measuring Device] Figure 1 is a block diagram showing a basic configuration example of a distance measuring device according to one embodiment of this technology. The distance measuring device 1 according to this embodiment is configured as a distance measuring sensor using the direct ToF method.

[0024] For example, the distance measuring device 1 can be configured as a direct ToF-based LiDAR (Light Detection and Ranging). Of course, the types of distance measuring sensors that can be configured using this technology are not limited, and other types of distance measuring sensors can also be configured.

[0025] As shown in Figure 1, the distance measuring device 1 includes a light-emitting unit 2, a light-receiving unit 3, a clock signal generation unit 4, a distance measurement processing unit 5, and a control unit 6. These components can be implemented, for example, using an LSI (Large Scale Integration) or a SoC (System on a chip).

[0026] The distance measuring device 1 also has a communication IF unit 7 for outputting distance measurement data, including distance and intensity calculated by the distance measuring processing unit 5, to the outside. Although not shown, for example, the distance measuring device 1 is configured to communicate with an external host IC (integrated circuit) or the like via the communication IF unit 7.

[0027] The control unit 6 is a component that comprehensively controls the operation of the distance measuring device 1. The control unit 6 is composed of, for example, a microprocessor. In this embodiment, the control unit 6 outputs trigger pulses and control signals to the light-emitting unit 2, the light-receiving unit 3, the clock signal generation unit 4, and the distance measuring processing unit 5.

[0028] Figure 2 is a schematic diagram showing an example configuration of the light-emitting unit 2 and the light-receiving unit 3. The light-emitting unit 2 emits light L1 for distance measurement using the direct ToF method. For example, laser light can be used as the light L1.

[0029] As shown in Figure 2, the light-emitting unit 2 includes a light source 9, an emitter lens 10, a light-emitting mirror 11, and a micromirror 12. The light source 9 emits light L1. For example, an end-emitting semiconductor laser or a surface-emitting semiconductor laser can be used as the light source 9. Of course, a light source with a different configuration may be used as the light source 9.

[0030] The light source 9 is driven by a trigger pulse from the control unit 6. The trigger pulse is, for example, a pulsed signal having a predetermined frequency.

[0031] The emitter lens 10, the light projection mirror 11, and the micromirror 12 function as a scanning mechanism for raster scanning of light L1. In this embodiment, these devices realize a mirror-scan type scanning mechanism.

[0032] The micromirror 12 changes the orientation of its reflective surface that reflects light L1 according to a control signal from the control unit 6. Light L1 emitted from the light source 9 is emitted in a direction corresponding to the orientation of the reflective surface of the micromirror 12, via the emitter lens 10, the projection mirror 11, and the micromirror 12. As shown in Figure 2, in this embodiment, the band-shaped irradiation area IA extending in the vertical direction in the figure is scanned in the left-right direction.

[0033] When light L1, reflected and emitted by the micromirror 12, is irradiated onto an object T, the object T reflects the light L1. The reflected light L2 from the object T is then incident on the light receiving unit 3 via the micromirror 12 and the light-emitting mirror 11. Examples of objects T include roads (ground), road markings, traffic signs, and preceding vehicles when the distance measuring device 1 is mounted on a vehicle. Of course, the objects T are not limited to those that can be used in this technology.

[0034] The light-receiving unit 3 has a receiver lens 13 and a plurality of light-receiving element units 14. The receiver lens 13 focuses the reflected light L2 and irradiates the plurality of light-receiving element units 14.

[0035] Multiple light-receiving elements 14 are arranged in a two-dimensional matrix to form a pixel array. Each of the multiple light-receiving elements 14 receives reflected light L2 from an object T that has reflected light L1 and outputs an electrical signal.

[0036] Each of the multiple light-receiving element sections 14 consists of one or more light-receiving elements that convert light into electrical signals through photoelectric conversion. In other words, in this embodiment, one light-receiving element section 14 is composed of one or more light-receiving elements.

[0037] When one light-receiving element section 14 is composed of multiple light-receiving elements, the multiple light-receiving elements are arranged in a matrix. One light-receiving element section 14 can also be called one pixel or one micropixel.

[0038] In this embodiment, one or more SPADs (Single Photon Avalanche Diodes) are used as one or more light-receiving elements. For example, one light-receiving element section 14 is composed of four 2x2 SPADs, nine 3x3 SPADs, or twenty-five 5x5 SPADs.

[0039] The number of SPADs constituting one light-receiving element section 14 is not limited and can be designed arbitrarily. Furthermore, all of the multiple light-receiving element sections 14 may be composed of the same number of SPADs, or multiple light-receiving element sections 14 may be configured such that there are a mix of light-receiving element sections 14 with different numbers of SPADs.

[0040] SPAD has the characteristic of generating a large current in response to the incidence of photons. By utilizing this characteristic, it is possible to detect the incidence of photons contained in reflected light L2 onto the SPAD with high sensitivity.

[0041] Each of the multiple photodetector units 14 outputs an electrical signal in response to the incidence of photons onto one or more SPADs constituting the photodetector unit 14. The electrical signals output from each of the multiple photodetector units 14 are output to the distance measurement processing unit 5.

[0042] The clock signal generation unit 4 generates a clock signal according to the control signal from the control unit 6. The clock signal is output to the distance measurement processing unit 5.

[0043] The distance measurement processing unit 5 is a component for calculating the distance to the target object T and the intensity of the reflected light L2 from the target object T. The distance to the target object T is calculated based on the timing when the light source 9 emits light L1 and the timing when the light receiving element 14 receives the reflected light L2. The intensity of the reflected light L2 is calculated based on the electrical signal output from the light receiving element 14.

[0044] The ranging processing unit 5 is constituted by, for example, a signal processor. As shown in FIG. 1, the ranging processing unit 5 includes a TDC (Time-to-Digital Converter) 16, a histogram generation unit 17, a feature amount extraction unit 18, and a distance / intensity measurement unit 19.

[0045] These components can be implemented using ICs such as FPGAs (Field-Programmable Gate Arrays) and ASICs (Application-Specific Integrated Circuits) configured in a signal processor, for example.

[0046] The present invention is not limited thereto, and these components can also be implemented as software blocks. That is, these components may be implemented by a CPU or the like disposed in the signal processor executing a predetermined program.

[0047] The ranging processing unit 5 operates in accordance with a clock signal input to the TDC 16.

[0048] The TDC 16 converts, into a digital value, the time (hereinafter also referred to as arrival time) from when the light source 9 emits the light L1 to when each of the plurality of light receiving element units 14 receives the reflected light L2, based on a trigger pulse from the control unit 6 (a trigger pulse for driving the light source 9) and an electrical signal from the light receiving unit 3 (each light receiving element unit 14).

[0049] In the present embodiment, the time (arrival time) from the timing at which the light source 9 emits the light L1 to the timing at which a photon is detected by the SPAD constituting each light receiving element unit 14 is converted into a digital value. Note that the timing at which a photon is detected by the SPAD can also be said to be the timing at which the SPAD reacts. The digital value of the arrival time acquired by the TDC 16 is output to the histogram generation unit 17.

[0050] For each of the plurality of light receiving element units 14, the histogram generation unit 17 generates a histogram relating to the time from when the light source 9 emits the light L1 to when the light receiving element unit 14 receives the reflected light L2, based on an electrical signal output from the light receiving element unit 14.

[0051] In this embodiment, the histogram generation unit 17 generates a histogram showing the time distribution of arrival times output from the TDC 16. Specifically, the histogram generation unit 17 classifies the arrival times output from the TDC 16 based on classes (bins) and generates a histogram.

[0052] A histogram is generated for each photodetector unit 14. In other words, a histogram is generated for each photodetector unit 14. The histogram generation unit 17 updates the histogram by incrementing the corresponding bin value each time it receives the arrival time output from the TDC 16.

[0053] Figure 3 is a schematic diagram showing an example of a histogram of arrival times. In Figure 3, the horizontal axis represents the bins (time intervals), and the vertical axis represents the frequency for each bin.

[0054] In this embodiment, a SPAD is used as the photodetector, and the frequency for each bin on the vertical axis corresponds to the count value (number of photons input) of photon detection by the SPAD for each bin.

[0055] In the example shown in Figure 3, the arrival times are classified by n+1 bins, from bin #0 to bin #n. The smaller the time width d of the bins, the higher the resolution of the measurable distance.

[0056] The histogram generation unit 17 counts the number of times the arrival time was obtained for each bin, and generates a histogram.

[0057] The reflected light L2 from the object T is received by the light receiving unit 3 at a timing corresponding to the distance to the object T. Therefore, the frequency of arrival times output by TDC 16 increases for bins corresponding to the light receiving timing corresponding to the distance to the object T. As a result, typically, the bin showing the frequency of peaks corresponds to the bin corresponding to the light receiving timing RT corresponding to the distance to the object T.

[0058] On the other hand, the light received by the light-receiving unit 3 is not limited to reflected light L2; for example, it may also receive ambient light around the distance measuring device 1 (light-receiving unit 3). Such ambient light other than reflected light L2 is incident on the light-receiving unit 3 at random timings and becomes noise for the reflected light L2 that is to be detected.

[0059] Even when noise light such as ambient light is received, it is output as arrival time by TDC 16. Then, the histogram generation unit 17 counts the arrival time in the bin corresponding to the timing of noise light reception. Therefore, as shown in Figure 3, the arrival time output due to the reception of noise light is counted in bins other than the bin corresponding to the reception timing RT which corresponds to the distance to the object T.

[0060] The feature extraction unit 18 extracts features from the histogram generated by the histogram generation unit 17.

[0061] Figures 4 and 5 are schematic diagrams illustrating an example of histogram features. Figures 4 and 5 show an example of a histogram generated based on an electrical signal (echo signal) output from a single photodetector 14 that receives reflected light L2. Note that noise light, such as ambient light, is not shown in Figures 4 and 5.

[0062] Figure 4 shows a histogram generated when the intensity of the reflected light L2 received by the photodetector 14 is not saturated and does not exceed the saturation signal level (Limit) of the photodetector 14.

[0063] Figure 5 shows a histogram generated when the intensity of the reflected light L2 received by the photodetector 14 saturates, exceeding the saturation signal level (Limit) of the photodetector 14.

[0064] As shown in Figure 5, the histogram at saturation has a saturation region SR that reaches the saturation signal level of the photodetector 14.

[0065] In this embodiment, SPADs are used as the light-receiving elements. Therefore, the saturation signal level of the light-receiving element unit 14 is the maximum count value of the SPADs and is defined in accordance with the number of SPADs that the light-receiving element unit 14 has. The more SPADs there are, the higher the saturation signal level becomes.

[0066] For example, the following parameters can be extracted as features of the histogram.

[0067] "Peak Amplitude (PA)"...the peak value of the echo signal. As shown in Figure 4, when unsaturated, the peak amplitude PA is lower than the saturated signal level and changes depending on the reflectivity of the object T. As shown in Figure 5, when saturated, the peak amplitude PA is always limited to the saturated signal level.

[0068] The saturation region (SR) can be described as the period during which the peak amplitude (PA) is always at its maximum value (saturation signal level). The saturation signal level is also included as a feature in the histogram.

[0069] "Echo Start (ES): Echo_start"... The start timing of the echo signal. That is, the start timing of receiving reflected light L2 from the object T (the start timing of photon detection). "Echo End (EE): Echo_end"... The end timing of the echo signal. That is, the end timing of receiving reflected light L2 from the object T (the end timing of photon detection).

[0070] "FWHM_start (FS)": The start of the half-width of the echo signal. That is, the timing when the amplitude of the echo signal reaches half of the peak amplitude PA. "FWHM_end (FE)": The end of the half-width of the echo signal. That is, the timing when the amplitude of the echo signal drops to half of the peak amplitude PA.

[0071] "Peak Start (PS): The start timing of the peak width at saturation. The peak width at saturation corresponds to the time width of the saturation region SR, and the Peak Start (PS) corresponds to the timing of the start of saturation. "Peak End (PE): The end timing of the peak width at saturation. The Peak End (PE) corresponds to the timing of the end of saturation.

[0072] These features can be extracted, for example, based on the frequency of each bin in a histogram and the representative time of each bin (e.g., the time at the midpoint of the bin). For example, the representative time of the bin with the frequency closest to half the peak amplitude PA can be extracted as the half-width start FS and half-width end FE. Alternatively, these features can be extracted by performing calculations that take the frequency of each bin in a histogram and the representative time of each bin as input.

[0073] Furthermore, as shown in Figure 3, even when noise light such as ambient light is received, it is possible to extract the above-mentioned feature quantities by performing, for example, a predetermined filtering process.

[0074] There are no specific algorithms required to extract features from a histogram; any algorithm can be used. For example, features can be extracted using an algorithm that employs a machine learning model constructed using a neural network or the like (a machine learning algorithm).

[0075] For example, any machine learning algorithm using DNN (Deep Neural Network), RNN (Recurrent Neural Network), CNN (Convolutional Neural Network), etc., may be used. For example, by using AI (Artificial Intelligence) that performs deep learning, it becomes possible to extract features with high accuracy.

[0076] Furthermore, the parameters derived from the features exemplified above—peak amplitude PA, echo start ES, echo end EE, half-width start FS, half-width end FE, peak width start PS, and peak width end PE—are also included in the histogram features.

[0077] For example, the time width of the echo signal (Echo_Width) derived from (echo end EE - echo start ES), the full width at half maximum (FWHM) derived from (FWHM end FE - FWHM start FS), and the peak width (Peak_Width) derived from (peak end PE - peak start PS) are also included in the histogram features.

[0078] These features may also be extracted by the feature extraction unit 18. Furthermore, these features may be calculated and used based on the features extracted by the feature extraction unit 18 using other components.

[0079] Furthermore, the frequency and representative time of specific bins, such as bins showing peaks, are also included as features in the histogram.

[0080] Figure 6 is a block diagram showing an example configuration of the distance / intensity measuring unit 19. As shown in Figure 6, the distance / intensity measuring unit 19 includes a distance calculation unit 21, an intensity calculation unit 22, and a grayscale value calculation unit 23.

[0081] These components can be implemented, for example, using integrated circuits (ICs). However, they are not limited to this; they can also be implemented as software blocks.

[0082] The distance calculation unit 21 calculates the distance to the object T based on the features of the histogram. For example, the distance calculation unit 21 calculates the time difference t1 from the emission timing of light L1 to the reception timing of reflected light L2 based on the features of the histogram.

[0083] For example, for the histogram shown in Figure 4 when the signal is unsaturated, it is possible to calculate the representative time of the bin showing the peak as a time difference t1.

[0084] Furthermore, for the histograms shown in Figure 4 (unsaturated) and Figure 5 (saturated), it is possible to calculate the time difference t1 as the time at the midpoint of the full width at half maximum (FWHM), which is derived by (FWHM end FE - FWHM start FS).

[0085] Furthermore, for the histogram at saturation shown in Figure 5, it is possible to calculate the time at the center of the peak width (time width of the saturation region SR), which is derived from (peak width end PE - peak width start PS), as the time difference t1.

[0086] The time difference t1 may be calculated by performing correction processing on the representative time of the bin showing the peak, the midpoint time of the full width at half maximum, and the midpoint time of the peak width (time width of the saturation region SR). Furthermore, there are no limitations on the specific algorithm for calculating the time difference t1 based on the features, and any algorithm may be adopted.

[0087] Since the time difference t1 is the round-trip time of light L1 to the object T, the distance to the object T can be calculated for each photodetector 14 by multiplying the time difference t1 by c / 2 (where c is the speed of light). Then, image information (hereinafter referred to as the measurement result image) including the calculated distance for each of the multiple photodetector 14 can be obtained. In other words, distance information can be stored for each pixel corresponding to the photodetector 14.

[0088] The intensity calculation unit 22 calculates the intensity of reflected light L2 from the object T based on the features of the histogram.

[0089] For example, with respect to the histogram shown in Figure 4 when the light is unsaturated, it is possible to calculate the intensity of reflected light L2 based on the peak amplitude PA.

[0090] In this embodiment, since a SPAD is used as the light-receiving element, the peak amplitude PA, that is, the frequency of bins showing a peak (the count value of photon detection by the SPAD), can be calculated as the intensity of the reflected light L2 received by the light-receiving element 14.

[0091] Furthermore, for the histogram at saturation shown in Figure 5, it is possible to calculate the intensity of reflected light L2 based on the full width at half maximum (FWHM), which is derived, for example, by (FWHM end FE - FWHM start FS).

[0092] For example, based on the assumption that the larger the half-width (FWHM), the higher the intensity of the reflected light L2 received by the photodetector 14, it is possible to use the FWHM as a parameter that defines the intensity of the reflected light L2. That is, it is possible to calculate the intensity of the reflected light L2 for each photodetector 14 using a calculation formula such that the intensity increases as the FWHM increases. It is also possible to use the FWHM value directly as a parameter indicating the intensity of the reflected light L2.

[0093] Furthermore, by calculating the standard deviation using the full width at half maximum and fitting it to a Gaussian distribution model, it is possible to estimate the peak amplitude PA assuming that it is not limited by the saturation signal level. Based on the estimated peak amplitude PA, it is possible to calculate the intensity of the reflected light L2.

[0094] Furthermore, for the histogram at saturation shown in Figure 5, it is possible to calculate the intensity of reflected light L2 based on the peak width (time width of the saturation region SR) derived from (peak width end PE - peak width start PS).

[0095] For example, based on the assumption that a larger peak width corresponds to a higher intensity of reflected light L2 received by the photodetector 14, the peak width can be used as a parameter to define the intensity of the reflected light L2. That is, it is possible to calculate the intensity of the reflected light L2 for each photodetector 14 using a calculation formula such that a larger peak width corresponds to a higher intensity. It is also possible to use the peak width value directly as a parameter indicating the intensity of the reflected light L2.

[0096] Furthermore, it is possible to calculate the intensity of reflected light L2 based on the peak amplitude PA at saturation, i.e., the saturation signal level, and the peak width. Based on the assumption that a larger peak width corresponds to a higher intensity of reflected light L2, it is also possible to calculate the intensity of reflected light L2 by correcting the saturation signal level with the peak width.

[0097] For example, the intensity of reflected light can be calculated using the following formula: Intensity of reflected light = Saturation signal level (Limit) + Peak width (Peak_Width) In this way, the intensity calculation unit 22 can calculate the intensity of reflected light by adding the peak width and the saturation signal level.

[0098] At saturation, the saturation signal level (Limit) equals the peak amplitude (PA). Therefore, the above equation can also be expressed as follows: Reflected light intensity (Intensity) = Peak amplitude (PA) + Peak width (Peak_Width)

[0099] Of course, there are no specific algorithms for calculating the intensity of reflected light L2 using the saturation signal level and peak width; any algorithm that increases as the peak width increases may be used. For example, integration or exponential functions may be used.

[0100] The intensity calculation unit 22 calculates the intensity of the reflected light L2 for each light-receiving element unit 14. This makes it possible to store information about the intensity of the reflected light L2, in addition to distance information, for each pixel of the measurement result image.

[0101] The intensity of the reflected light L2 can be used as information about the reflectivity (reflectance) of the object T. In other words, it is possible to store reflectance information of the object T for each pixel of the measurement result image. For example, it is possible to store information such as the attributes of the object T based on its reflectance for each pixel.

[0102] For example, based on reflectivity, it becomes possible to classify objects T into weakly reflective, strongly reflective, and extremely strongly reflective objects for each pixel.

[0103] For example, in the measurement result image, if the distance measuring device 1 is mounted on a vehicle, it becomes possible to classify and identify, for example, roads such as asphalt with relatively low reflectivity (weakly reflective objects), road markings including white lines and lane markings of pedestrian crossings with higher reflectivity than the road (highly reflective objects), and retroreflective materials such as traffic signs with very high reflectivity (extremely strongly reflective objects).

[0104] In this disclosure, "road" refers to the black area without road markings. Of course, "road" can also be replaced with expressions such as "the black area of ​​the road without road markings."

[0105] The grayscale value calculation unit 23 calculates a grayscale value corresponding to each of the multiple light-receiving element units 14 based on the intensity of the reflected light L2. In other words, the grayscale value calculation unit 23 calculates a grayscale value for each pixel corresponding to the light-receiving element unit 14 when displaying the measurement result image on a display or the like.

[0106] Figure 7 is a schematic diagram showing an example of gradation value calculation. As shown in Figure 7, for example, a grayscale gradation value (8 bits) ranging from 0 (black) to 255 (white) is calculated for each pixel based on the intensity of reflected light L2.

[0107] For example, if the intensity of reflected light L2 is 0, a black gradation value of 0 is set. If the intensity of reflected light L2 is Max, a white gradation value of 255 is set. A linear correspondence is established between the gradation values ​​of 0 (black) and 255 (white) for the intensity of reflected light L2 from 0 to Max. This method of calculating gradation values ​​can be adopted. The Max value of reflected light L2 can be set as appropriate.

[0108] By setting the grayscale value based on the intensity of reflected light L2 in this way, it becomes possible to display, for example, a measurement result image in a way that allows for the identification of weakly reflective objects, strongly reflective objects, and extremely strongly reflective objects, ranging from black to white.

[0109] Of course, the number of bits used to define the gradation value is not limited. Furthermore, the specific algorithm for calculating the gradation value from reflected light L2 is also not limited; any algorithm may be used.

[0110] The distance measurement data (distance measurement image, etc.) including distance and intensity calculated by the distance measurement processing unit 5 is output via the communication IF unit 7 to, for example, a 3D model generation system or a distance measurement image display system.

[0111] [Example of operation of distance measuring device 1] An example of operation of distance measuring device 1 will be explained, mainly focusing on the calculation of the intensity of reflected light L2. The following explanation of operation will be based on the measurement result images 25 generated based on the measurement data of distance measuring device 1 shown in Figures 8 to 12.

[0112] The measurement result images 25 shown in Figures 8 to 12 are generated based on measurement data taken in real-world urban areas. The measurement result images 25 display target objects T, including roads such as sphalt (weakly reflective objects) 26, road markings such as white lines and lane markings (highly reflective objects) 27, and retroreflective materials such as traffic signs (extremely highly reflective objects) 28.

[0113] Furthermore, in the measurement result images 25 shown in Figures 8 to 12, the central left portion and the rightmost portion are masked. This mask is intended to hide some of the content within the image, but it does not hide any content that negates the effectiveness of this technology. If necessary, the inventors can provide the data for the measurement result images 25 without the mask.

[0114] The measurement result image 25a shown in Figure 8 is an image of the measurement result obtained by measuring using only ambient light, without emitting light L1 from the light-emitting unit 2. The measurement result image 25a is mainly intended to clearly show the environment in which the measurement was performed.

[0115] The measurement result images 25b to 25e shown in Figures 9 to 12 are measurement result images obtained by emitting light L1 from the light-emitting unit 2.

[0116] When generating the measurement result image 25b shown in Figure 9, the peak amplitude PA (= saturation signal level) is calculated as the intensity (= Max) of reflected light L2 for the saturation histogram as exemplified in Figure 5. Then, as shown in Figure 7, the grayscale values ​​from 0 (black) to 255 (white) are linearly associated with the intensity values ​​of reflected light L2 from 0 to Max.

[0117] Therefore, for all light-receiving elements 14 (pixels) where the intensity of reflected light L2 is saturated, a grayscale value of 255 (white) is calculated. As a result, overexposure occurs over a large area of ​​the measurement result image 25b. For example, in the central part of the measurement result image 25b shown in Figure 9, there is almost no contrast between the road 26 and the road markings 27, making it difficult to distinguish between the two.

[0118] When generating the measurement result image 25c shown in Figure 10, the value of the full width at half maximum (FWHM) derived from (FWHM end FE - FWHM start FS) for a saturation histogram as illustrated in Figure 5 is directly calculated as the intensity of reflected light L2. Then, as shown in Figure 7, the grayscale values ​​from 0 (black) to 255 (white) are linearly associated with the intensity values ​​of reflected light L2 from 0 to Max.

[0119] As shown in Figure 10, in the measurement result image 25c, where the half-width was calculated as intensity, the overall grayscale value was low, and the contrast of the road 26, road markings 27, and retroreflective material 28 was also low. In other words, the measurement result image was difficult to distinguish between these elements.

[0120] As shown in Figure 3, the histogram is affected by noise light such as ambient light. This noise light often reduces the extraction accuracy of the echo start ES, echo end EE, half-width start FS, and half-width end FE. As a result, the accuracy of the half-width calculation also decreases, leading to low-contrast measurement images like the one shown in Figure 10.

[0121] When generating the measurement result image 25d shown in Figure 11, the intensity of reflected light L2 is calculated based on the peak width (time width of the saturation region SR) derived from the histogram at saturation, as illustrated in Figure 5, by (peak width end PE - peak width start PS). Specifically, the value of (saturation signal level + peak width) is calculated as the intensity of reflected light L2.

[0122] As shown in Figure 7, the gradation values ​​from 0 (black) to 255 (white) are linearly associated with the intensity values ​​of reflected light L2 from 0 to Max.

[0123] The peak width start PS and peak width end PE can be extracted as feature quantities with the influence of noise light sufficiently suppressed. Therefore, by using the peak width (time width of the saturation region SR), it is possible to calculate the intensity of reflected light L2 at saturation with high accuracy.

[0124] Furthermore, the peak width (time duration of the saturation region SR) can be used as the optimal parameter to represent the reaction time of the SPAD in the saturation region SR. The reaction time of the SPAD is a parameter that is approximately proportional to the reaction intensity of the SPAD (count value of photon detection). In other words, by using the optimal peak width (time duration of the saturation region SR) to represent the reaction time of the SPAD, it becomes possible to calculate the intensity of reflected light L2 at saturation with high accuracy.

[0125] As shown in Figure 11, in the measurement result image 25d, where the intensity is calculated based on the peak width, the contrast of the road 26, road markings 27, and retroreflective material 28 is increased. As a result, it becomes possible to distinguish these elements.

[0126] When generating the measurement result image 25e shown in Figure 12, the intensity of reflected light L2 is calculated based on the peak width (time width of the saturation region SR) derived from the histogram at saturation using (peak width end PE - peak width start PS), similar to when generating the measurement result image 25d shown in Figure 11. Specifically, the value of (saturation signal level + peak width) is calculated as the intensity of reflected light L2.

[0127] Then, when the grayscale value is calculated by the grayscale value calculation unit 23 shown in Figure 6, grayscale value adjustment is performed. As shown in Figure 12, the grayscale value adjustment further increases the contrast of the road 26, road markings 27, and retroreflective material 28. As a result, it becomes possible to distinguish them with even higher accuracy.

[0128] The adjustment of grayscale values ​​will be explained in detail later.

[0129] As can be seen by comparing the measurement result images 25a to 25d shown in Figures 8 to 12, the method of calculating the intensity of reflected light L2 based on the peak width of the histogram at saturation is a highly effective technique that is unprecedented (Figures 11 and 12). Furthermore, it can be seen that even more effective measurement result images can be obtained by performing grayscale value adjustment (Figure 12).

[0130] Figure 13 is a flowchart showing an example of the process from calculating the intensity of reflected light L2 and the grayscale value by the distance measuring device 1.

[0131] In the example shown in Figure 13, the light-emitting unit 2 first emits light multiple times (step 101). That is, light L1 is emitted from the light source 9 a predetermined number of times. The predetermined number of times can be set arbitrarily. Of course, this technology is also applicable when light L1 is emitted only once.

[0132] The histogram generation unit 17 accumulates the arrival time (digital value) for each bin for multiple light emission events, and generates a histogram (step 102).

[0133] The feature extraction unit 18 extracts features from the histogram (step 103). In this embodiment, the intensity of reflected light L2 is calculated based on the peak width (time width of the saturation region SR) as explained with reference to Figures 11 and 12. Specifically, the value of (saturation signal level + peak width) is calculated as the intensity of reflected light L2.

[0134] Accordingly, the feature extraction unit 18 extracts features including the peak width start PS and peak width end PE, or features including the peak width (time width of the saturation region SR), from a histogram having a saturation region SR that reaches the saturation signal level of the photodetector unit 14. The feature extraction unit 18 also calculates the peak amplitude PA (which becomes the saturation signal level when saturated) as a feature.

[0135] The peak width start PS, peak width end PE, and peak width (time width of the saturation region SR) are one embodiment of the information regarding the time width of the saturation region related to this technology.

[0136] The intensity calculation unit 22 determines whether the histogram generated for each of the multiple light-receiving element units 14 is saturated or not (step 104). If the histogram is not saturated (NO in step 104), the intensity of the reflected light L2 is calculated based on the peak amplitude PA (step 105).

[0137] If the histogram is saturated (YES in step 104), the intensity of reflected light L2 is calculated based on the peak width (time duration of the saturated region SR). Specifically, the value of (saturation signal level + peak width) is calculated as the intensity of reflected light L2 (step 106).

[0138] Furthermore, as shown in Figure 4, the peak width is set to 0 as a feature of the histogram when it is unsaturated. When it is saturated, the peak amplitude PA is set to the saturation signal level. Under these conditions, by using the formula (reflected light intensity = peak amplitude PA + peak width), it becomes possible to calculate the intensity of each reflected light L2 from the photodetector 14 without having to determine whether or not the histogram is saturated. As a result, the processing time for calculating the intensity of reflected light L2 can be shortened, and processing resources can be saved.

[0139] The grayscale value calculation unit 23 calculates the grayscale value of each pixel in the measurement result image 25 based on the intensity of the reflected light L2 (step 107). For example, as shown in Figure 7, grayscale values ​​from 0 (black) to 255 (white) are linearly associated with the intensity of the reflected light L2 from 0 to Max. This makes it possible to generate the measurement result image 25d shown in Figure 11, which has high contrast for the road 26, road markings 27, and retroreflective material 28.

[0140] [Gradation Value Adjustment] The gradation value adjustment that can be performed by the gradation value calculation unit 23 in step 107 of Figure 13 will be explained below. By adjusting the gradation value, it becomes possible to set gradation values ​​that can achieve higher contrast for multiple objects T with different reflection characteristics.

[0141] For example, a GUI for performing threshold adjustment is provided to the user, and the user classifies the object T into multiple object groups based on its light reflection characteristics. In this disclosure, the term "object group" is not limited to cases where it consists of multiple types of object T, but may also consist of a single type of object T.

[0142] In this embodiment, the following three object groups are classified based on their light reflection characteristics: Object Group A: Weakly reflective object group (including road 26) Object Group B: Highly reflective object group (including road markings 27) Object Group C: Extremely highly reflective object group (including retroreflective material 28)

[0143] The weakly reflective object group consists of objects with relatively low reflected light intensity. The strongly reflective object group consists of objects with relatively higher reflected light intensity than the weakly reflective object group. The extremely strongly reflective object group consists of objects with even higher reflected light intensity than the strongly reflective object group. Tone value adjustment is performed on these object groups.

[0144] Figure 14 is a schematic diagram showing an example of the operation of grayscale value adjustment. First, the intensity of reflected light L2 for each object group is calculated (step 201). In this embodiment, the intensity of reflected light L2 for the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28) is calculated.

[0145] For example, the user is provided with the measurement result image before grayscale value adjustment, as shown in Figure 11, via a GUI for performing threshold adjustment. The user then uses a pointing device such as a mouse to specify the positions (pixels) of the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28).

[0146] In that case, the user may be instructed to specify the brightest position (the position with the highest grayscale value).

[0147] Furthermore, instead of specifying the position (pixel) after the object group has been classified, it is also possible to configure the system so that the user specifies the position (pixel), and the object T containing the specified position (pixel) is classified as a single object group.

[0148] As shown in Figure 15, the grayscale value calculation unit 23 extracts a 3x3 pixel region centered on the pixel specified by the user. Then, it calculates the average value of the reflected light intensity L2 of the nine pixels.

[0149] In the example shown in Figure 15, nine pixels 30, R1 to R9, become pixels displaying the weakly reflective object group (road 26), and the average value of the reflected light intensity L2 is calculated as the reflected light intensity L2 of the weakly reflective object group (road 26).

[0150] Furthermore, nine pixels 31, T1 to T9, become pixels on which the highly reflective object group (road markings 27) is displayed, and the average value of the intensity of the reflected light L2 is calculated as the intensity of the reflected light L2 of the highly reflective object group (road markings 27).

[0151] Furthermore, nine pixels 32, M1 to M9, become pixels on which the highly reflective object group (retroreflective material 28) is displayed, and the average value of the intensity of the reflected light L2 is calculated as the intensity of the reflected light L2 of the highly reflective object group (retroreflective material 28).

[0152] Based on the intensity of reflected light L2 for each object group, multiple thresholds for the intensity of reflected light L2 are calculated to distinguish each of the classified object groups A to C (step 202). These multiple thresholds can also be described as thresholds that define the boundaries between the classified object groups A to C.

[0153] In this embodiment, a first threshold th1 is calculated based on the intensity of reflected light L2 calculated from a weakly reflective object group (road 26). Specifically, the average value of the intensity of reflected light L2 of the nine pixels 30 shown in Figure 15 is used directly as the first threshold th1.

[0154] Furthermore, a second threshold th2 is calculated based on the intensity of reflected light L2 calculated from the highly reflective object group (road markings 27). Specifically, the average value of the intensity of reflected light L2 of the nine pixels 31 shown in Figure 15 is used directly as the second threshold th2.

[0155] Furthermore, a third threshold th3 is calculated based on the intensity of reflected light L2 calculated from the highly reflective object group (retroreflective material 28). Specifically, the average value of the intensity of reflected light L2 from the nine pixels 32 shown in Figure 15 is used directly as the third threshold th3.

[0156] The grayscale value calculation unit 23 calculates grayscale values ​​based on the first threshold th1, the second threshold th2, and the third threshold th3 calculated by the unit. The calculation of grayscale values ​​based on these multiple thresholds constitutes grayscale value adjustment (step 203).

[0157] Figures 16 and 17 are schematic diagrams showing examples of calculating grayscale values ​​by adjusting grayscale values.

[0158] The user specifies the brightest position (position with the highest grayscale value) for a group of weakly reflective objects (road 26), a group of strongly reflective objects (road markings 27), and a group of extremely strongly reflective objects (retroreflective material 28), and a first threshold th1, a second threshold th2, and a third threshold th3 are calculated.

[0159] Therefore, as shown in Figures 16 and 17, the group of weakly reflective objects with low reflectivity (road 26) is approximately within the range of reflected light intensity L2 from 0 to the first threshold th. The group of strongly reflective objects with high reflectivity (road markings 27) is within the range from the first threshold th1 to a value slightly exceeding the second threshold th2. The group of extremely strongly reflective objects with very high reflectivity (retroreflective material 28) is within the range from a value slightly below the third threshold t3 to Max.

[0160] Of course, the distribution of each object group within the intensity range from 0 to Max will vary depending on the environment. However, the weakly reflective object group (road 26) is mainly distributed in the range from 0 to the first threshold th1. The strongly reflective object group (road markings 27) is mainly distributed in the range from the first threshold th1 to the second threshold th2. The extremely strongly reflective object group (retroreflective material 28) is thought to be mainly distributed in the range from the third threshold t3 to Max.

[0161] As shown in Figure 16, for grayscale gradation values ​​from 0 to 255, gradation values ​​PV1, PV2, and PV3 are calculated corresponding to the first threshold th1, the second threshold th2, and the third threshold th3.

[0162] The gradation value PV1 is set to the value to be calculated as a gradation value when the intensity of reflected light L2 reaches the first threshold th1. The gradation value PV2 is set to the value to be calculated as a gradation value when the intensity of reflected light L2 reaches the second threshold th2. The gradation value PV3 is set to the value to be calculated as a gradation value when the intensity of reflected light L2 reaches the third threshold th1. This setting may be performed by the user or may be performed automatically by the gradation value calculation unit 23.

[0163] As shown in Figure 16, if the intensity of the reflected light L2 falls within the range from 0 to the first threshold th1, a grayscale value corresponding to that intensity is calculated that falls within the range from 0 to the grayscale value PV1.

[0164] If the intensity of reflected light L2 falls within the range of the first threshold th1 to the second threshold th2, a corresponding grayscale value is calculated that falls within the range of grayscale values ​​PV1 to PV2.

[0165] If the intensity of reflected light L2 falls within the range of the second threshold th2 to the third threshold th3, a corresponding grayscale value is calculated that falls within the range of grayscale values ​​PV2 to PV3.

[0166] If the intensity of reflected light L2 falls within the range of the third threshold th3 to Max, a corresponding grayscale value is calculated that falls within the range of grayscale values ​​PV3 to 255.

[0167] For example, the following formula can be used to calculate the grayscale value.

[0168]

[0169] In equation (1) above, x represents the intensity of reflected light L2, and y represents the grayscale value. When calculating the grayscale value according to equation (1), as shown in Figure 17, the grayscale values ​​from 0 (black) to grayscale value PV1 are linearly associated with the intensity of reflected light L2 from 0 to the first threshold th. Also, the grayscale values ​​from grayscale value PV1 to grayscale value PV2 are linearly associated with the intensity of reflected light L2 from the first threshold th1 to the second threshold th2.

[0170] Furthermore, the intensity of reflected light L2 is linearly correlated with the grayscale values ​​PV2 to PV3 for values ​​from the second threshold th2 to the third threshold th3. Also, the intensity of reflected light L2 is linearly correlated with the grayscale values ​​PV3 to 255 for values ​​from the third threshold th3 to Max.

[0171] The dotted line shown in Figure 17 is a graph that linearly maps the gradation values ​​from 0 (black) to 255 (white) to the intensity values ​​of reflected light L2 shown in Figure 7, ranging from 0 to Max.

[0172] Thus, in this embodiment, as a grayscale value adjustment, the intensity L2 of reflected light included in the range from 0 to a first threshold th1 is linearly associated with a predetermined first grayscale value range (the range from 0 (black) to grayscale value PV1).

[0173] Furthermore, the intensity of reflected light L2 included in the range from the first threshold th1 to the second threshold th2 is linearly mapped to a second grayscale range (the range from grayscale value PV1 to grayscale value PV2), which is a range with higher grayscale values ​​than the first grayscale range.

[0174] Furthermore, the intensity of reflected light L2 within the range from the second threshold th2 to the third threshold th3 is linearly mapped to a third grayscale range (the range from grayscale value PV2 to grayscale value PV3), which is a range with higher grayscale values ​​than the second grayscale range.

[0175] Furthermore, the intensity of reflected light L2, which falls within the range from the third threshold th3 to the maximum value (Max) of the reflected light intensity, is linearly mapped to a fourth grayscale value range (a range of grayscale values ​​from PV3 to 255), which has higher grayscale values ​​than the third grayscale value range.

[0176] In this way, the desired grayscale values ​​PV1 to PV3 are selected as appropriate, and the grayscale values ​​are mapped so that the first threshold th1, the second threshold th2, and the third threshold th3 become inflection points.

[0177] This makes it possible to generate a measurement result image 25d with very high contrast for the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28), as shown in Figure 12, and to distinguish them with high accuracy.

[0178] Figure 18 is a graph showing the grayscale values ​​of the measurement result image 25d shown in Figure 11 before grayscale value adjustment. Figure 19 is a graph showing the grayscale values ​​of the measurement result image 25e shown in Figure 12 after grayscale value adjustment.

[0179] In the graphs in Figures 18 and 19, the horizontal axis represents the grayscale value, and the vertical axis represents the frequency (number of pixels). The vertical axis is normalized to a range of 0.0 to 1.0. 0.0 corresponds to 0 pixels, and 1.0 corresponds to the maximum number of pixels.

[0180] In Figure 18A, in the measurement result image 25d before grayscale value adjustment, the grayscale values ​​of pixels from the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28) are extracted, and the number of pixels is counted for each grayscale value.

[0181] Figure 18A also shows the results of fitting a Gaussian distribution curve to the histograms of the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28). Figure 18B shows only the fitted Gaussian distribution.

[0182] In Figure 19A, in the measurement result image 25e after grayscale value adjustment, the grayscale values ​​of pixels for the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28) are extracted, and the number of pixels is counted for each grayscale value.

[0183] Figure 19A also shows the results of fitting Gaussian distribution curves to the histograms of the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28). Figure 19B shows the fitted Gaussian distribution.

[0184] Let's compare the Gaussian distributions in Figures 18 and 19. First, let's focus on the overlapping areas between the groups of objects: the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28). In the Gaussian distribution of the measurement result image 25d before grayscale adjustment, there are overlapping areas, but in the Gaussian distribution of the measurement result image 25 after grayscale adjustment, the overlapping areas have disappeared.

[0185] Furthermore, we focus on the variance of each Gaussian distribution. When we do this, compared to the Gaussian distribution of the measurement result image 25d before adjusting the grayscale values, the Gaussian distribution of the measurement result image 25 after adjusting the grayscale values ​​shows that the variance of the Gaussian distribution of the weakly reflective object group (road 26) and the extremely reflective object group (retroreflective material 28) is reduced, resulting in a shape that is concentrated at the predetermined grayscale values.

[0186] Thus, the Gaussian distribution of the measurement result image 25 after adjusting the grayscale values ​​shows a significant reduction in overlapping areas between each object group and in the variance of the Gaussian distribution. As a result, the contrast of the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28) is improved.

[0187] The calculation of the intensity of reflected light L2 for each object group in steps 201 and 202 of Figure 14, and the calculation of multiple thresholds related to the intensity of reflected light L2, may be performed in advance by calibration or the like. The multiple thresholds calculated may then be stored in the memory of the distance measuring device 1, and in the grayscale value calculation step 107 of Figure 13, the stored multiple thresholds may be used to perform grayscale value adjustment.

[0188] The number of object groups to be classified is not limited; two object groups or four or more object groups may be classified. Regardless of the number of classifications, high contrast can be achieved among the multiple classified object groups by adjusting the gradation values. Furthermore, the specific method of adjusting the gradation values ​​is not limited to the method using equation (1) above; any other method may be employed.

[0189] For example, the method for calculating multiple thresholds such as the first threshold th1, the second threshold th2, and the third threshold th3 is not limited, and any other arbitrary method may be used. For example, in the example shown in Figure 15, a 3x3 pixel region was extracted, but this is not limited to this. Pixel regions of any size, such as a 2x2 or 5x5 pixel region, may be extracted. Furthermore, instead of calculating the average intensity of reflected light L2 in the pixel region, multiple thresholds may be calculated based on the median intensity, the mode, or the like. Alternatively, multiple thresholds may be set in advance as fixed values.

[0190] Figure 20 is a flowchart showing an example of a distance measurement method (method for calculating the intensity of reflected light L2) using this distance measuring device 1.

[0191] Light L1 is emitted from the light source 9 (step 301). Each of the multiple light-receiving element units 14 receives the reflected light from the object T that has reflected the emitted light L1 and outputs an electrical signal (step 302). The histogram generation unit 17 generates a histogram for each of the multiple light-receiving element units 14, based on the electrical signal output from the light-receiving element unit 14, relating to the time from when the light source 9 emits light L1 until the light-receiving element unit 14 receives the reflected light L2 (step 303). The feature extraction unit 18 extracts features from the histogram that has a saturation region where the saturation signal level of the light-receiving element unit 14 is reached, including information about the time width of the saturation region (step 304). The intensity calculation unit 22 calculates the intensity of the reflected light L2 based on the extracted features (step 305).

[0192] The distance measuring device 1 is configured to have the necessary computer hardware, such as a processor (CPU, GPU, DSP, etc.), memory (ROM, RAM, etc.), and storage devices (HDD, etc.). The processor loads the program related to this technology, which is stored in the storage unit or memory, into the RAM and executes it, thereby enabling the distance measuring method shown in Figure 20 to be performed.

[0193] Step 101 shown in Figure 13 is included in step 301 shown in Figure 20. Step 102 is a step that combines steps 302 and 303. Step 103 is included in step 304. Steps 104 to 106 are included in step 305.

[0194] Based on the intensity of reflected light L2 calculated in the flowchart of Figure 20, the calculation of the grayscale value (including grayscale value adjustment) described above is performed (steps 106, 201-203). Of course, it is also possible to include the grayscale value calculation step as part of the distance measurement method by this distance measuring device 1.

[0195] In the distance measuring device 1 according to this embodiment, feature quantities including information about the time width of the saturation region SR are extracted from a histogram based on the time from the emission of light L1 to the reception of reflected light L2. Based on the extracted feature quantities, the intensity of the reflected light L2 is calculated. This makes it possible to achieve high measurement accuracy, especially in the calculation of the intensity of reflected light L2.

[0196] In LiDAR technology, photodetectors such as PDs (Photo Diodes), APDs (Avalanche Photo Diodes), and SPADs are used as elements to detect reflected light L2 that returns from the target.

[0197] Of these, SPAD can be an effective light-receiving element for LiDAR due to its high sensitivity, which allows for detection of even a single photon, and the possibility of integration and high pixel count similar to image sensors.

[0198] On the other hand, SPAD has the drawback of being prone to saturation due to its narrow dynamic range. Therefore, when measuring roads, compared to PD and APD, which have lower sensitivity but wider dynamic ranges, both the road and road markings may become saturated, resulting in a loss of contrast and making identification difficult.

[0199] In the distance measuring device 1 according to this embodiment, it is possible to calculate the intensity of reflected light L2 based on the peak width (time width of the saturated region SR) of the histogram at saturation. Therefore, as illustrated in the measurement result images 25d and 25e shown in Figures 11 and 12, it is possible to improve the contrast between the road and the road markings, making it possible to distinguish between the two.

[0200] When the distance measuring device 1 is mounted on a vehicle, the detection of road markings, especially lane marks, becomes an important factor, and this technology, which can achieve high contrast with the road, is extremely effective.

[0201] Furthermore, by performing gradation value correction, it becomes possible to stably enhance the contrast and generate an effective measurement result image 25.

[0202] Furthermore, this technology can be applied even when a photodetector other than SPAD, such as a PD or APD, is used as the photodetector, and the effects described above can be achieved.

[0203] [Reducing Data Volume] When generating histograms as shown in Figure 3, it is possible to improve the resolution of measurable distances by setting a higher sampling frequency and reducing the time width d of the bins. On the other hand, the number of bins required will increase.

[0204] Furthermore, if the distance measurable by the distance measuring device 1 is increased while maintaining the frame rate of distance measurement, for example, a larger number of bins will be required. In addition, increasing the resolution (number of pixels) of the measurement result image 25 will also increase the number of histograms generated.

[0205] Therefore, in order to realize a ranging sensor that can measure over long distances at a high frame rate and generate high-resolution ranging images, it is necessary to generate a very large number of bins and calculate the distance and intensity for each pixel.

[0206] However, the information from a very large number of bins would amount to a massive amount of data, and transmitting all of it would often be difficult from the perspective of interface transfer volume and speed, bandwidth limitations, etc.

[0207] In this distance measuring device 1, the feature extraction unit 18 shown in Figure 1 extracts features from the histogram. Then, the distance / intensity measurement unit 19 can calculate the distance and grayscale value based on the features of the histogram. Therefore, it is not necessary to use the information of all bins, and it is possible to calculate the distance and intensity with high accuracy based on features consisting of a small amount of data, which is very effective.

[0208] For example, by combining the histogram generation unit 17 and the feature extraction unit 18 shown in Figure 1 into a single IC, it becomes unnecessary to transmit information for all bins within the distance measuring device 1, thus avoiding problems related to interface transfer volume and speed. It is also possible to refer to the component combining the histogram generation unit 17 and the feature extraction unit 18 as, for example, a histogram feature extraction unit.

[0209] Thus, this distance measuring device 1 makes it possible to reduce the amount of data, and to avoid problems related to interface transfer volume and speed, bandwidth limitations, etc.

[0210] [Color Mapping] In this distance measuring device 1, grayscale value adjustment is performed so that each of the following object groups can be emphasized, based on a plurality of thresholds that define the boundaries between the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28).

[0211] It is also possible to perform color mapping based on the gradation values ​​calculated by adjusting the gradation values. For example, it is possible to perform color mapping based on gradation values ​​using color maps such as Jet, Hot, Cool, HSV, Turbo, and Plasma.

[0212] By adjusting the gradation values, the contrast of the weakly reflective object group (road 26), the strongly reflective object group (road markings 27), and the extremely strongly reflective object group (retroreflective material 28) is improved. Consequently, these object groups are displayed with very high contrast in the color-mapped color measurement result image 25. As a result, it becomes possible to identify these object groups with very high accuracy.

[0213] Furthermore, the inventors can provide a measurement result image 25 in which, using the Jet color map, weakly reflective object groups (roads 26) are highlighted in blue, strongly reflective object groups (road markings 27) in light blue or green, and extremely strongly reflective object groups (retroreflective materials 28) in red.

[0214] [Regarding distance measurement data] The distance measurement data (distance measurement image, etc.) including distance and intensity calculated by the distance measurement processing unit 5 is output to external components and systems via the communication IF unit 7. For example, it can be output to a 3D model generation system, and a 3D model can be generated based on the distance measurement data related to this technology. Of course, it is also possible to generate 3D data such as point clouds based on the distance measurement data.

[0215] Furthermore, if the 3D model generation system does not require grayscale data, it is possible to output only distance and intensity information to the 3D model generation system.

[0216] Furthermore, it is possible to output distance measurement data related to this technology to components and systems that perform processing including the display of the measurement result image 25. In this case, the distance measurement data is output along with the grayscale value data. This makes it possible to display a measurement result image 25 with high contrast for, for example, a weakly reflective object group (road 26), a strongly reflective object group (road markings 27), and an extremely strongly reflective object group (retroreflective material 28).

[0217] Of course, it is also possible to output distance measurement data related to this technology to components and systems that use grayscale value information for processing different from the display of the measurement result image 25. For example, distance measurement data related to this technology is effective for systems that classify and process weakly reflective object groups (roads 26), strongly reflective object groups (road markings 27), and extremely strongly reflective object groups (retroreflective materials 28).

[0218] This distance measuring device 1, along with external components and systems that output distance measurement data, can be considered as one embodiment of the system related to this technology.

[0219] <Other Embodiments> This technology is not limited to the embodiments described above, and various other embodiments can be realized.

[0220] In the above, a set of peak width start PS and peak width end PE, and the peak width (time width of the saturated region SR) were given as one embodiment of the information regarding the time width of the saturated region related to this technology. While the information regarding the time width of the saturated region is typically the time width of the saturated region, it is not limited to this and may include any information regarding the time width of the saturated region.

[0221] For example, in the histogram at saturation shown in Figure 5, we focus on the bins where the frequency reaches a predetermined percentage (e.g., 90%) of the saturation signal level. In other words, we focus on the bins that have not yet reached the saturation signal level, but can be said to be nearly saturated.

[0222] Furthermore, this technique can be applied by using the time width in which bins whose frequency reaches a predetermined percentage (e.g., 90%) of the saturation signal level are continuously arranged as information about the time width of the saturation region. Note that 90% is just an example, and other values ​​may be used.

[0223] The configurations of the distance measuring device, light emitting unit, light receiving unit, distance measuring processing unit, and distance / intensity calculation unit, as well as the processing flows such as distance / intensity calculation, histogram generation, feature extraction, grayscale value calculation, and grayscale value adjustment, as described with reference to each drawing, are merely one embodiment and can be arbitrarily modified without departing from the spirit of this technology. In other words, other arbitrary configurations and algorithms may be adopted to implement this technology.

[0224] In this disclosure, words such as "approximately," "about," "nearly," and "approximately" may be used as appropriate to facilitate understanding of the explanation. However, there is no clear distinction defined between when these words are used and when they are not. In other words, in this disclosure, concepts that define shape, size, positional relationships, states, etc., such as "center," "central," "uniform," and "equal," are considered to include concepts such as "substantially centered," "substantially central," "substantially uniform," and "substantially equal." For example, states that fall within a predetermined range (e.g., a range of ±10%) based on "perfectly centered," "perfectly central," "perfectly uniform," and "perfectly equal" are also included. Therefore, even when words such as "about," "nearly," and "approximately" are not added, concepts that would otherwise be expressed with these words added may be included. Conversely, a state expressed with "about," "nearly," and "approximately" does not necessarily exclude a perfect state.

[0225] In this disclosure, expressions using "greater than A" such as "greater than A" and "less than A" are expressions that comprehensively include both concepts that include cases where something is equivalent to A and concepts that do not include cases where something is equivalent to A. For example, "greater than A" is not limited to cases where something is not equivalent to A, but also includes "greater than or equal to A". Similarly, "less than A" is not limited to "less than A", but also includes "less than or equal to A". When implementing this technology, you may appropriately adopt specific settings from the concepts included in "greater than A" and "less than A" so that the effects described above are achieved.

[0226] It is also possible to combine at least two of the feature features of the present technology described above. In other words, the various feature features described in each embodiment may be combined arbitrarily, regardless of the specific embodiment. Furthermore, the various effects described above are merely examples and not limiting, and other effects may also be exhibited.

[0227] [Examples of application of this technology] This technology can be applied to any distance measuring device and distance measuring system that employs direct ToF (Time of Flight) distance measurement.

[0228] Figure 21 is a schematic diagram showing an example of the use of the distance measuring device and distance measuring system. The distance measuring device and distance measuring system according to this technology can be used in various cases of sensing light such as visible light, infrared light, ultraviolet light, and X-rays, for example, as shown below.

[0229] Devices that capture images for viewing purposes, such as digital cameras and portable devices with camera functions. Devices used for traffic purposes, such as in-vehicle sensors that capture images of the front, rear, surroundings, and interior of a vehicle for safe driving such as automatic stopping and recognition of the driver's condition, surveillance cameras that monitor moving vehicles and roads, and distance measuring sensors that measure distances between vehicles. Devices used in home appliances such as TVs, refrigerators, and air conditioners that capture user gestures and allow the device to be operated according to those gestures. Devices used for medical and healthcare purposes, such as endoscopes and devices that perform angiography using infrared light reception. Devices used for security purposes, such as surveillance cameras for crime prevention and cameras for person recognition. Devices used for beauty purposes, such as skin measuring devices that capture images of the skin and microscopes that capture images of the scalp. Devices used for sports purposes, such as action cameras and wearable cameras for sports use. Devices used for agriculture, such as cameras for monitoring the condition of fields and crops.

[0230] <Examples of application to mobile devices> The technology disclosed herein (this technology) can be applied to various products. For example, the technology disclosed herein may be implemented as a device mounted on any type of mobile device such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

[0231] Figure 22 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0232] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 22, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0233] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0234] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0235] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0236] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0237] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0238] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0239] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0240] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0241] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 22, the output devices include an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0242] Figure 22 shows an example of the installation position of the imaging unit 12031.

[0243] In Figure 22, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0244] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0245] Figure 22 shows an example of the imaging ranges of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0246] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0247] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0248] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0249] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0250] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to the external information detection unit 12030 in the configuration described above when constructing a distance measuring device or distance measuring system. For example, the imaging unit 12031 can also be used as the light receiving unit 3 shown in Figure 1. By applying this technology, distance measurement with high accuracy can be achieved, and external information can be detected with high accuracy.

[0251] Furthermore, this technology can also be configured as follows: (1) A distance measuring device comprising: a light source that emits light; a plurality of light-receiving element units that receive reflected light from an object that has reflected the emitted light and output an electrical signal; a histogram generation unit that generates a histogram relating to the time from when the light source emits light until when the light-receiving element unit receives the reflected light, based on the electrical signal output from the light-receiving element unit for each of the plurality of light-receiving element units; a feature extraction unit that extracts feature quantities including information relating to the time width of the saturation region from the histogram which has a saturation region that reaches the saturation signal level of the light-receiving element unit; and an intensity calculation unit that calculates the intensity of the reflected light based on the extracted feature quantities. (2) A distance measuring device according to (1), wherein each of the plurality of light-receiving element units consists of one or more light-receiving elements. (3) A distance measuring device according to (2), wherein the one or more light-receiving elements are one or more SPADs (Single Photon Avalanche Diodes). (4) A distance measuring device according to (3), wherein the saturation signal level of the light-receiving element is defined in correspondence with the number of SPADs that the light-receiving element has. (5) A distance measuring device according to any one of (1) to (4), wherein the feature quantity includes the saturation signal level of the light-receiving element, and the intensity calculation unit calculates the intensity of the reflected light based on information regarding the time width of the saturation region and the saturation signal level. (6) A distance measuring device according to (5), wherein the intensity calculation unit calculates the intensity of the reflected light by adding information regarding the time width of the saturation region and the saturation signal level. (7) A distance measuring device according to any one of (1) to (6), wherein the information regarding the time width of the saturation region is the time width of the saturation region. (8) A distance measuring device according to any one of (1) to (7), further comprising a gradation value calculation unit that calculates a gradation value corresponding to each of the plurality of light receiving element units based on the intensity of the reflected light.(9) A distance measuring device according to (8), wherein the gradation value calculation unit classifies the object into a plurality of object groups based on the reflection characteristics of the light, and calculates the gradation value based on a plurality of thresholds relating to the intensity of the reflected light for distinguishing each of the classified plurality of object groups. (10) A distance measuring device according to (9), wherein the plurality of object groups include a weakly reflective object group in which the intensity of the reflected light is relatively low, a strongly reflective object group in which the intensity of the reflected light is relatively higher than that of the weakly reflective object group, and an extremely strongly reflective object group in which the intensity of the reflected light is relatively higher than that of the strongly reflective object group, wherein the gradation value calculation unit calculates the gradation value based on a first threshold based on the intensity of the reflected light calculated from the weakly reflective object group, a second threshold based on the intensity of the reflected light calculated from the strongly reflective object group, and a third threshold based on the intensity of the reflected light calculated from the extremely strongly reflective object group. (11) A distance measuring device according to (10), wherein the gradation value calculation unit linearly associates the intensity of the reflected light included in the range from 0 to the first threshold with a predetermined first gradation value range, linearly associates the intensity of the reflected light included in the range from the first threshold to the second threshold with a second gradation value range which has higher gradation values ​​than the first gradation value range, linearly associates the intensity of the reflected light included in the range from the second threshold to the third threshold with a third gradation value range which has higher gradation values ​​than the second gradation value range, and linearly associates the intensity of the reflected light included in the range from the third threshold to the maximum value of the intensity of the reflected light with a fourth gradation value range which has higher gradation values ​​than the third gradation value range. A distance measuring device according to (12), (10), or (11), wherein the weakly reflective object group includes a road, the strongly reflective object group includes road markings, and the extremely strongly reflective object group includes a retroreflective member.(13) A distance measuring method performed by a distance measuring device, comprising: emitting light from a light source; each of a plurality of light-receiving elements receiving reflected light from an object that has reflected the emitted light and outputting an electrical signal; generating a histogram for each of the plurality of light-receiving elements based on the electrical signal output from the light-receiving element, relating to the time from when the light source emits the light until the light-receiving element receives the reflected light; extracting a feature quantity from the histogram having a saturation region where the saturation signal level of the light-receiving element is reached, including information about the time width of the saturation region; and calculating the intensity of the reflected light based on the extracted feature quantity. (14) A program for causing a distance measuring device to execute a distance measuring method, wherein the distance measuring method includes: emitting light from a light source; receiving reflected light from an object that has reflected the emitted light using each of a plurality of light-receiving elements and outputting an electrical signal; generating a histogram for each of the plurality of light-receiving elements based on the electrical signal output from the light-receiving element, relating to the time from when the light source emits the light until the light-receiving element receives the reflected light; extracting a feature quantity from the histogram having a saturation region that reaches the saturation signal level of the light-receiving element, which includes information about the time width of the saturation region; and calculating the intensity of the reflected light based on the extracted feature quantity.

[0252] EE...Echo end ES...Echo start ES FE...Half-width end FS...Half-width start L1...Light L2...Reflected light PA...Peak amplitude PS...Peak width start PE...Peak width end RT...Light reception timing according to distance to object T...Object 1...Distance measuring device 9...Light source 10...Emitter lens 11...Light projection mirror 12...Micro mirror 14...Light receiving element 25...Measurement result image 26...Road (weakly reflective object group) 27...Road markings (strongly reflective object group) 28...Retroreflective material (extremely strongly reflective object group) 12000...Vehicle control system

Claims

1. A distance measuring device comprising: a light source that emits light; a plurality of light-receiving element units that receive reflected light from an object that has reflected the emitted light and output an electrical signal; a histogram generation unit that generates a histogram relating to the time from when the light source emits light until the light-receiving element unit receives the reflected light, based on the electrical signal output from the light-receiving element unit for each of the plurality of light-receiving element units; a feature quantity extraction unit that extracts feature quantities including information relating to the time width of the saturation region from the histogram having a saturation region that reaches the saturation signal level of the light-receiving element unit; and an intensity calculation unit that calculates the intensity of the reflected light based on the extracted feature quantities.

2. A distance measuring device according to claim 1, wherein each of the plurality of light-receiving element sections comprises one or more light-receiving elements.

3. A distance measuring device according to claim 2, wherein the one or more light-receiving elements are one or more SPADs (Single Photon Avalanche Diodes).

4. A distance measuring device according to claim 3, wherein the saturation signal level of the light-receiving element is defined in correspondence with the number of SPADs that the light-receiving element has.

5. A distance measuring device according to claim 1, wherein the feature quantity includes the saturation signal level of the light receiving element, and the intensity calculation unit calculates the intensity of the reflected light based on information regarding the time width of the saturation region and the saturation signal level.

6. A distance measuring device according to claim 5, wherein the intensity calculation unit calculates the intensity of the reflected light by adding information relating to the time width of the saturation region and the saturation signal level.

7. A distance measuring device according to claim 1, wherein the information relating to the time width of the saturation region is the time width of the saturation region.

8. A distance measuring device according to claim 1, further comprising a gradation value calculation unit that calculates a gradation value corresponding to each of the plurality of light-receiving element units based on the intensity of the reflected light.

9. A distance measuring device according to claim 8, wherein the grayscale value calculation unit classifies the object into a plurality of object groups based on the light reflection characteristics, and calculates the grayscale value based on a plurality of thresholds relating to the intensity of the reflected light for distinguishing each of the classified plurality of object groups.

10. A distance measuring device according to claim 9, wherein the plurality of object groups include a weakly reflective object group having a relatively low intensity of reflected light, a strongly reflective object group having a relatively higher intensity of reflected light than the weakly reflective object group, and an extremely strongly reflective object group having a relatively higher intensity of reflected light than the strongly reflective object group, and the grayscale value calculation unit calculates the grayscale value based on a first threshold value based on the intensity of reflected light calculated from the weakly reflective object group, a second threshold value based on the intensity of reflected light calculated from the strongly reflective object group, and a third threshold value based on the intensity of reflected light calculated from the extremely strongly reflective object group.

11. A distance measuring device according to claim 10, wherein the gradation value calculation unit linearly associates the intensity of the reflected light included in the range from 0 to the first threshold with a predetermined first gradation value range; linearly associates the intensity of the reflected light included in the range from the first threshold to the second threshold with a second gradation value range which has higher gradation values ​​than the first gradation value range; linearly associates the intensity of the reflected light included in the range from the second threshold to the third threshold with a third gradation value range which has higher gradation values ​​than the second gradation value range; and linearly associates the intensity of the reflected light included in the range from the third threshold to the maximum value of the intensity of the reflected light with a fourth gradation value range which has higher gradation values ​​than the third gradation value range.

12. A distance measuring device according to claim 10, wherein the weakly reflective object group includes a road, the strongly reflective object group includes road markings, and the extremely strongly reflective object group includes retroreflective members.

13. A distance measuring method performed by a distance measuring device, comprising: emitting light from a light source; receiving reflected light from an object that has reflected the emitted light using each of a plurality of light-receiving elements and outputting an electrical signal; generating a histogram for each of the plurality of light-receiving elements based on the electrical signal output from the light-receiving element, relating to the time from when the light source emits the light until the light-receiving element receives the reflected light; extracting a feature quantity from the histogram having a saturation region where the saturation signal level of the light-receiving element is reached, including information about the time width of the saturation region; and calculating the intensity of the reflected light based on the extracted feature quantity.

14. A program for causing a distance measuring device to execute a distance measuring method, wherein the distance measuring method involves: emitting light from a light source; each of a plurality of light-receiving elements receiving reflected light from an object that has reflected the emitted light and outputting an electrical signal; generating a histogram for each of the plurality of light-receiving elements based on the electrical signal output from the light-receiving element, relating to the time from when the light source emits the light until the light-receiving element receives the reflected light; extracting feature quantities including information about the time width of the saturation region from the histogram having a saturation region where the saturation signal level of the light-receiving element is reached; and calculating the intensity of the reflected light based on the extracted feature quantities.

Images