Information processing device, distance measuring device, distance measuring system, and information processing method
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- KK TOSHIBA
- Filing Date
- 2023-09-14
- Publication Date
- 2026-08-05
Smart Images
Figure 0007901057000001 
Figure 0007901057000002 
Figure 0007901057000003
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to an information processing apparatus, a distance measuring apparatus, a distance measuring system, and an information processing method.
Background Art
[0002] With the evolution of LiDAR (Light Detection & Ranging) technology, it is expected that the importance of three-dimensional data recognition and classification technology will increase even more in the future, similar to two-dimensional images.
[0003] Fog or rain, etc., is a major factor that degrades the performance of systems such as monitoring using sensors such as cameras or LiDAR, assisting or automating various manual operations. In order to change the control of the system according to the weather, it is essential to easily and instantaneously determine the environment such as fog or rain at as little cost as possible. In addition, fog or rain itself is a factor that significantly degrades the performance of LiDAR. LiDAR often uses infrared light, but infrared light is easily absorbed or scattered by moisture. Therefore, in a situation where fog or rain exists, the measurable distance by LiDAR is limited.
[0004] Many existing LiDAR devices have a multi-echo function that selects the true reflected light peak among multiple reflected light peaks. However, if the light intensity of the light reflected by particles such as fog or rain is large, the multi-echo function does not operate normally, and the distance measurement accuracy decreases.
Prior Art Documents
Patent Documents
[0005]
Patent Document 1
Patent Document 2
Patent Document 3
Non-Patent Documents
[0006] [Non-Patent Document 1] https: / / web.media.mit.edu / ~guysatat / fog / [Overview of the project] [Problems that the invention aims to solve]
[0007] In embodiments of the present invention, an information processing device, a distance measuring device, a distance measuring system, and an information processing method are provided that can determine environmental conditions such as fog or rain simply and instantly without incurring costs. [Means for solving the problem]
[0008] To solve the above problems, according to one embodiment of the present invention, a histogram generation unit generates a histogram showing the relationship between the light reception time and the light reception frequency based on a plurality of light reception signals including reflected light from an object, A first peak detection unit detects the peak of the light reception frequency in the histogram, The system includes a determination unit that determines whether or not there is a light propagation interference factor, including at least one of fog, mist, haze, rain, or snow, based on the histogram and the light reception time of the peaks. The information processing device provided determines whether or not the light propagation interference factor exists based on the light reception time of the peak and the degree of change in the light reception frequency of the histogram before and after the light reception time of the peak. [Brief explanation of the drawing]
[0009] [Figure 1] Block diagram of a distance measuring system equipped with an information processing device according to one embodiment. [Figure 2] A block diagram showing the internal configuration of the information processing unit of an information processing device according to one embodiment. [Figure 3] A schematic diagram illustrating how the optical scanning unit scans an object by projecting light onto it. [Figure 4]Block diagram showing a schematic configuration of a light receiving section for one pixel of a light detection section. [Figure 5] Flowchart showing a processing operation of an information processing apparatus according to an embodiment. [Figure 6] Diagram showing an example of a curve generated based on the light reception frequency of each bin of a histogram. [Figure 7] Diagram showing an example in which a threshold comparison section detects a first time and a second time using a threshold having a signal level smaller than the threshold in FIG. 6. [Figure 8] Diagram for explaining the processing of a fitting section. [Figure 9] Diagram showing the shape of a curve generated based on the light reception frequency of each bin of a histogram when there is a light propagation disturbance factor. [Figure 10A] Diagram showing the experimental results of a first experiment in which ranging is performed by switching on / off fog removal in a fog environment with a visibility of 40 m. [Figure 10B] Diagram showing the experimental results of an experiment in which ranging is performed by switching on / off rain removal in a rain environment with a rainfall of 50 mm / h. [Figure 11] External view of a second experiment. [Figure 12] Diagram showing the results of a second experiment.
Embodiments for Carrying Out the Invention
[0010] Hereinafter, embodiments of an information processing apparatus, a ranging apparatus, a ranging system, and an information processing method will be described with reference to the drawings. Hereinafter, the description will focus on the main components of the information processing apparatus, the ranging apparatus, and the ranging system, but there may be components and functions that are not shown or described in the information processing apparatus, the ranging apparatus, and the ranging system. The following description does not exclude components and functions that are not shown or described.
[0011] FIG. 1 is a block diagram of a ranging system 2 including an information processing apparatus 1 according to an embodiment. The ranging system 2 in FIG. 1 includes an information processing apparatus 1, a light projecting section 3, and a light detection section 4. In this specification, the information processing apparatus 1 and the light detection section 4 are collectively referred to as a ranging apparatus 10.
[0012] The light projection unit 3 includes a light source unit 5 and an optical scanning unit 6. The light source unit 5 intermittently emits an optical pulse signal. The light source unit 5 emits a plurality of optical pulse signals in a linear or planar shape.
[0013] The optical scanning unit 6 scans the propagation direction of the optical pulse signal emitted from the light source unit 5 in a one-dimensional or two-dimensional direction. Thereby, the object 20 existing in the three-dimensional space can be irradiated with the optical pulse signal. The object 20 may be a stationary object or a moving object. When the light source unit 5 is a surface light emitter, the optical scanning unit 6 can be omitted.
[0014] The light detection unit 4 receives the reflected light signal of the optical pulse signal irradiated on the object 20. The light detection unit 4 includes a plurality of light receiving units 7. The light detection unit 4 receives, for example, the reflected light signal incident in a two-dimensional region. In this specification, the minimum unit in the two-dimensional region where the light detection unit 4 receives the reflected light signal is called a pixel. The light detection unit 4 includes a plurality of light receiving units 7 that receive the reflected light signal for each pixel.
[0015] The information processing device 1 measures the distance of the object 20 based on the reflected light signal received by the light detection unit 4 and generates three-dimensional data including the distance information of the object 20. The information processing device 1 may generate a two-dimensional image including the luminance information around the object 20. The information processing device 1 includes an information processing unit 8 that measures the distance of the object 20 and an image processing unit 9 that generates three-dimensional data including the distance information of the object 20.
[0016] FIG. 2 is a block diagram showing the internal configuration of the information processing unit 8 included in the information processing device 1 according to an embodiment. As shown in FIG. 2, the information processing unit 8 according to an embodiment includes a histogram generation unit 11, a first peak detection unit 12, a determination unit 13, a removal unit 14, a second peak detection unit 15, and a distance measurement unit 16.
[0017] The histogram generation unit 11 generates a histogram showing the relationship between the time of reception and the frequency of reception, based on multiple received light signals, including reflected light from the object 20. The histogram is a graph of the frequency of reception for each of the multiple bins, each representing a different time. The histogram generation unit 11 generates a histogram for each pixel that receives light from the light receiving unit 7, for example. Alternatively, the histogram generation unit 11 may generate a histogram for multiple pixels.
[0018] The first peak detection unit 12 detects the peak of the light reception frequency in the histogram generated by the histogram generation unit 11. More specifically, the first peak detection unit 12 detects the light reception frequency in the bin with the highest light reception frequency in the histogram as the peak.
[0019] The determination unit 13 determines whether or not there is a light propagation interference factor, including at least one of fog, mist, haze, rain, or snow, based on the histogram generated by the histogram generation unit 11 and the light reception time of the peak detected by the first peak detection unit 12. Fog is a meteorological term referring to a state where visibility is less than 1 km. Mist is a meteorological term referring to a state where visibility is 1 km or more and less than 10 km. Haze is not a meteorological term, but it refers to a state where visibility is limited by water droplets, dust, dirt, or smoke. The light propagation interference factors determined by the determination unit 13 include not only moisture in the atmosphere but also various particles that make up dust and dirt. Particles such as moisture that can be light propagation interference factors may either absorb the light emitted by the light projection unit 3 or reflect or scatter the light.
[0020] If the determination unit 13 determines that a light propagation interference factor exists, the removal unit 14 removes the light reception frequency information due to the light propagation interference factor included in the histogram generated by the histogram generation unit 11 and extracts the light reception frequency information after removal. The histogram generated by the histogram generation unit 11 includes the light reception frequency information of light reflected by the object 20, the light reception frequency information of light reflected or scattered by particles or the like that are light propagation interference factors, and the light reception frequency information due to ambient light. The removal unit 14 processes the histogram to remove the light reception frequency information of light reflected or scattered by particles or the like that are light propagation interference factors, and the light reception frequency information due to ambient light. The removal unit 14 generates a light reception frequency curve that does not include the light reception frequency information due to the light propagation interference factor.
[0021] The second peak detection unit 15 detects the peak of the light reception frequency curve after removal by the removal unit 14. That is, the second peak detection unit 15 detects the peak of the light reception frequency curve that does not contain light reception frequency information due to light propagation interference factors. If the determination unit 13 determines that there are no light propagation interference factors, the second peak detection unit 15 detects the same peak as the peak detected by the first peak detection unit 12.
[0022] The distance measurement unit 16 measures the distance to the object 20 based on the peak detected by the second peak detection unit 15. This allows for peak detection while eliminating the influence of light propagation interference factors such as fog or rain, and enables accurate measurement of the distance to the object 20 based on the detected peak.
[0023] The image processing unit 9 generates three-dimensional data including the object 20 based on the distance of the object 20 measured by the distance measurement unit 16. The three-dimensional data may be represented as three-dimensional point cloud data that represents the positions of the measured points in three-dimensional space, or as a distance image that represents distance with color. The more times the light projection unit 3 is emitted, the more points in the three-dimensional data will be, and the fewer times it is emitted, the fewer points in the data will be. Also, if the time interval between emission is constant, the horizontal or vertical spacing of the points in the data is determined by the interval obtained by dividing the field of view as seen from the light detection unit 4 into constant angular widths. If the angular width of the field of view is constant, the number of points in the object 20 will be greater the closer the object 20 is, and less the farther the object is, even for the same object 20.
[0024] The determination unit 13 determines whether or not there is a light propagation interference factor based on the peak light reception time and the degree of change in the light reception frequency of the histogram before and after the peak light reception time. As a specific example, the determination unit 13 has a threshold comparison unit 17 and a peak time determination unit 18. The threshold comparison unit 17 detects the period during which the light reception frequency of the histogram is above a predetermined threshold. The peak time determination unit 18 determines whether or not there is a light propagation interference factor based on the period detected by the threshold comparison unit 17 and the peak light reception time detected by the first peak detection unit 12.
[0025] In a more specific example, the threshold comparison unit 17 detects a first time point at which the light reception frequency of the histogram generated by the histogram generation unit 11 matches a predetermined threshold, and a second time point at which the period detected by the threshold comparison unit 17 has elapsed from the first time point. The peak time determination unit 18 compares a first period from the first time point to the peak time with a second period from the peak time to the second time point, and determines that a light propagation interference factor exists if the second period is longer than the first period by a predetermined period or more.
[0026] The threshold value described above is a fixed value set, for example, based on the average value of the sum of the light reception frequencies of all bins in the histogram. Alternatively, the threshold value may be a value that changes according to the light reception frequency of the peak detected by the first peak detection unit 12. The threshold comparison unit 17 may also change the threshold value in multiple ways to change the length of the period from the first time point to the second time point. In this case, the determination unit 13 may determine for each of the multiple threshold values whether the second period is longer than the first period by a predetermined period or more, and finally determine whether or not there are light propagation interference factors based on these determination results.
[0027] In addition, the information processing device 1 according to one embodiment may include a fitting unit 19. The fitting unit 19 generates a function that fits to a curve generated by the light reception frequency of each bin of the histogram generated by the histogram generation unit 11. More specifically, the fitting unit 19 generates a curve of a first function that fits to a first curve generated by the light reception frequency of each bin of the histogram from a first time to a peak time detected by the threshold comparison unit 17, and also generates a curve of a second function that fits to a second curve generated by the light reception frequency of each bin of the histogram from a peak time to a second time.
[0028] The removal unit 14 generates a third curve containing the removed light reception frequency information by taking the difference between the first and second curves generated by the light reception frequency of each bin of the histogram and the first and second function curves generated by the fitting unit 19.
[0029] Figure 3 schematically shows how the optical scanning unit 6 scans the object 20 with light projected onto it. The optical scanning unit 6 scans while simultaneously projecting multiple beams of light from a light source unit 5 having multiple laser light sources arranged in one direction or two dimensions. Each circle in Figure 3 schematically represents the beam spot bs of the laser light. The laser light projected onto the object 20 is reflected by the object 20 and received by the light detection unit 4. The light detection unit 4 receives the reflected light from the object 20 on a pixel-by-pixel basis. The light detection unit 4 has multiple light receiving units 7 for multiple pixels.
[0030] Figure 4 is a block diagram illustrating the schematic configuration of the light-receiving unit 7 for one pixel of the light-detecting unit 4. The light-receiving unit 7 includes a SiPM (Silicon Photo-multiplier) 23 having multiple SPADs (Single Photon Avalanche Diodes) 22, an ADC (Analog Digital Converter) 24, and a storage unit 25.
[0031] SPAD22 can detect light on a single-photon basis, and when a photon is detected, the output voltage decreases. The detection of light by SPAD22 is called firing. SiPM23 has multiple SPAD22s, and the greater the amount of light received by the light-receiving unit 7, the more SPAD22s fire, and the lower the output voltage of SiPM23 becomes. Therefore, ADC24 can output a multi-level digital signal corresponding to the amount of light received by performing AD conversion on the output voltage of SiPM23. In this specification, the digital signal output by ADC24 is sometimes referred to as RAW data.
[0032] Figure 5 is a flowchart showing the processing operation of the information processing device 1 according to one embodiment. The flowchart in Figure 5 starts after the light receiving unit 7 begins detecting light in synchronization with the light emitting unit 3 intermittently emitting light pulse signals.
[0033] First, the histogram generation unit 11 generates a histogram based on the light received signal from the light receiving unit 7 (step S1). Next, the first peak detection unit 12 detects the peak of the light reception frequency in the histogram (step S2). Figure 6 shows an example of a curve w0 generated by the light reception frequency of each bin of the histogram. The curve w0 in Figure 6 shows an example where the light reception frequency is maximum (peak) at the bin of light reception time tp. In the example in Figure 6, the first peak detection unit 12 detects the light reception time tp corresponding to the peak of the curve w0.
[0034] Next, the threshold comparison unit 17 compares the light reception frequency in the histogram with a threshold and detects the first and second time points when the light reception frequency matches the threshold (step S3). In the example in Figure 6, the threshold comparison unit 17 detects the first time point t1 and the second time point t2.
[0035] The first time point t1 and the second time point t2 vary depending on the threshold signal level. Figure 7 shows an example where the threshold comparison unit 17 detects the first time point t1' and the second time point t2' using a threshold with a smaller signal level than the threshold in Figure 6. In Figure 7, the first time point t1' is earlier than the first time point t1, and the second time point t2' is later than the second time point t2. Therefore, the period T between the first time point t1 and the second time point t2 is shorter than the period T' between the first time point t1' and the second time point t2'.
[0036] Next, the peak time determination unit 18 compares the first period LL between the first time t1 and the peak time tp with the second period LR between the peak time tp and the second time t2, and determines whether the second period LR is longer than the first period LL by a predetermined period or more (step S4). In other words, it determines whether or not there are optical propagation interference factors.
[0037] If the second period LR is determined to be longer than the first period LL by a predetermined period or more, it is determined that a light propagation interference factor exists. In the examples of Figures 6 and 7, the peak time determination unit 18 determines that the second period LR is longer than the first period LL by a predetermined period. The specific length of the predetermined period, which is the difference between the second period LR and the first period LL, is arbitrary. For example, the predetermined period may be set to a value of 10% or more of the first period LL or the second period LR.
[0038] According to the inventor's verification, the curve shape of the reception frequency of light reflected or scattered by moisture in the atmosphere, or particles other than moisture constituting dust and dirt, which are factors that interfere with light propagation, is the same as the curve shape of the gamma distribution. If the second period LR is longer than the first period LL by a predetermined period or more, it indicates the presence of factors that interfere with light propagation, and the curve shape after the peak time of the curve generated by the reception frequency of each bin of the histogram is the same as the curve shape of the gamma distribution.
[0039] On the other hand, if there are no light propagation interference factors, the curve shape after the peak time of the curve generated by the light reception frequency of each bin in the histogram will be the same as that of a Gaussian distribution.
[0040] Thus, the peak time determination unit 18 can determine that if the curve in the second period LR is a gamma-distributed curve, there is a light propagation interference factor, and if the curve in the second period LR is a Gaussian-distributed curve, there is no light propagation interference factor.
[0041] If it is determined in step S4 that a light propagation interference factor exists, the fitting unit 19 generates a function curve to fit to the curve generated by the light reception frequency of each bin of the histogram (step S5). Figure 8 is a diagram illustrating the processing of the fitting unit 19. The fitting unit 19 divides the process into a first period LL before the peak light reception time tp and a second period LR after the peak light reception time, and performs fitting processing separately. The reason for performing fitting processing separately in the first period LL and the second period LR is that the curve shapes of the first curve w1 generated by the light reception frequency of each bin of the histogram in the first period LL and the second curve w2 generated by the light reception frequency of each bin of the histogram in the second period LR are different.
[0042] Figure 9 shows the shape of curve w0 generated by the light reception frequency of each bin in the histogram when light propagation interference factors are present. As shown in Figure 9, the largest peak in curve w0 is the peak caused by the light propagation interference factors, and the second largest peak is the peak based on reflected light from object 20.
[0043] As shown in Figure 8, the fitting unit 19 generates a curve w3 of a first function consisting of a linear or quadratic function that passes through two points: the intersection point with the threshold (first time t1) and the peak position (time tp), and matches the shape of the first curve w1 as closely as possible. Similarly, in the second period LR, the fitting unit 19 generates a curve w4 of a second function consisting of a linear or quadratic function that passes through two points: the intersection point with the threshold (second time t2) and the peak position (time tp), and matches the shape of the second curve w2 as closely as possible.
[0044] Thus, the fitting unit 19 generates a curve w3 of the first function in the first period LL through fitting processing, and generates a curve w4 of the second function in the second period LR. As can be seen from Figure 8, there is a large difference between the curve w0 generated by the light reception frequency of each bin of the original histogram and the curves w3 and w4 generated by the fitting unit 19 around the time when reflected light from object 20 is received (around time t3).
[0045] Next, the removal unit 14 detects the difference curve between the curve generated by the light reception frequency of each bin of the histogram and the curves of the first and second functions (step S6). The detected difference curve is, for example, a curve (third curve) w5 as shown in Figure 8, and is a light reception frequency curve that includes reflected light from object 20 and does not include light due to light propagation interference factors.
[0046] Next, the second peak detection unit 15 detects the peak of the third curve w5 based on the difference in step S5 as the second peak (step S7). The second peak is used to measure the distance of object 20.
[0047] On the other hand, if it is determined in step S4 that there are no light propagation interference factors, the second peak detection unit 15 detects the first peak detected by the first peak detection unit 12 as the second peak without performing processing in the fitting unit 19 and the removal unit 14 (step S8).
[0048] Next, the distance measuring unit 16 measures the distance to the object 20 based on the second peak (step S9).
[0049] (Experiment 1) The inventors conducted an experiment (hereinafter referred to as the first experiment) to compare the distance measurement success rate when the determination unit 13 and the removal unit 14 were performed and when they were not. Hereinafter, performing the determination unit 13, fitting unit 19, and removal unit 14 will be referred to as fog (rain) removal on, and not performing the determination unit 13, fitting unit 19, and removal unit 14 will be referred to as fog (rain) removal off.
[0050] Figure 10A shows the experimental results of the first experiment, in which distance measurements were taken by switching fog removal on and off in a foggy environment with visibility of 40m, and Figure 10B shows the experimental results of the same experiment, in which distance measurements were taken by switching rain removal on and off in a rainy environment with rainfall of 50mm / h.
[0051] In Figures 10A and 10B, the horizontal axis represents the distance to object 20 [m], and the vertical axis represents the distance measurement success rate [%]. Figure 10A shows curve w11 when fog removal is on and curve w12 when fog removal is off. Figure 10B shows curve w13 when rain removal is on and curve w14 when rain removal is off.
[0052] As can be seen from Figures 10A and 10B, in both foggy and rainy conditions, performing the processing of the determination unit 13, fitting unit 19, and removal unit 14 significantly improves the success rate of measuring the distance of an object 20 located at a distance of more than 15m. In foggy conditions, even with the processing of the determination unit 13, fitting unit 19, and removal unit 14, the success rate of measuring the distance at distances exceeding 40m decreases. However, in rainy conditions, even at distances exceeding 40m, performing the processing of the determination unit 13, fitting unit 19, and removal unit 14 maintains a high success rate of measuring the distance.
[0053] (Second experiment) The inventors placed multiple objects 20 at equal intervals from the distance measuring device 10 and switched the fog removal on / off to determine which object 20 distance images could be generated (hereinafter referred to as the second experiment).
[0054] Figure 11 is an external view of the second experiment. A rangefinder 10 is positioned in front of Figure 11, and multiple objects 20 are placed at regular intervals from the rangefinder 10 to a distance away. The objects 20 are reflectors that reflect light emitted from the light-emitting unit 3 of the rangefinder 10. The second experiment was conducted in a foggy environment with visibility of 40m.
[0055] Figure 12 shows the results of the second experiment. The left side of Figure 12 shows the three-dimensional data set of the experiment with fog removal turned off, and the right side of Figure 12 shows the three-dimensional data set of the experiment with fog removal turned on. As shown in the figure, with fog removal turned on, light from a more distant object 20 can be received more stably compared to when fog removal is off, and the distance that can be measured can be extended.
[0056] In this embodiment, a histogram is generated based on the light received signal at the light receiving unit 7, and it is determined whether or not there is a light propagation interference factor from the peak position of the histogram and the shape of the curve generated by the light reception frequency of the histogram. If it is determined that there is a light propagation interference factor, the light reception frequency due to the light propagation interference factor is removed from the light reception frequency of each bin of the histogram to generate a curve that includes the light reception frequency due to reflected light from object 20, and the distance to object 20 is measured by detecting the peak of this curve. This makes it possible to perform distance measurement processing with high accuracy even in environments where light propagation interference factors are present. The curve that includes the light reception frequency due to reflected light from object 20 can be easily detected by taking the difference between the curve generated by the light reception frequency of each bin of the histogram and the curve of a linear or quadratic function fitted to this curve.
[0057] At least a portion of the information processing device 1, distance measuring device 10, and distance measuring system 2 described in the above-described embodiment may be configured as hardware or as software. If configured as software, a program that implements at least some of the functions of the information processing device 1, distance measuring device 10, and distance measuring system 2 may be stored on a recording medium such as a flexible disk or CD-ROM, and loaded into a computer for execution. The recording medium is not limited to removable ones such as magnetic disks or optical disks, but may also be a fixed recording medium such as a hard disk drive or memory.
[0058] Furthermore, a program that implements at least some of the functions of the information processing device 1, the distance measuring device 10, and the distance measuring system 2 may be distributed via communication lines such as the Internet (including wireless communication). In addition, the program may be encrypted, modulated, or compressed and distributed via wired or wireless lines such as the Internet, or stored on a recording medium.
[0059] [Note] [Item 1] A histogram generation unit generates a histogram showing the relationship between the time of reception and the frequency of reception based on multiple reception signals, including reflected light from an object. A first peak detection unit detects the peak of the light reception frequency in the histogram, The system includes a determination unit that determines whether or not there is a light propagation interference factor, including at least one of fog, mist, haze, rain, or snow, based on the histogram and the light reception time of the peaks. The determination unit determines whether or not the light propagation interference factor exists based on the time of reception of the peak and the degree of change in the light reception frequency of the histogram before and after the time of reception of the peak. Information processing device. [Item 2] If the determination unit determines that the light propagation interference factor exists, the removal unit removes the light reception frequency information due to the light propagation interference factor included in the histogram and extracts the light reception frequency information after removal. A second peak detection unit detects the peak of the light reception frequency information after the aforementioned removal, The system includes a distance measuring unit that measures the distance to the object based on the peak detected by the second peak detection unit, The information processing apparatus according to claim 1. [Item 3] The system includes an image processing unit that generates three-dimensional data including the object based on the distance of the object measured by the distance measuring unit. The information processing device described in item 2. [Item 4] The determination unit, A threshold comparison unit detects the period during which the light reception frequency of the histogram exceeds a predetermined threshold, The system includes a peak time determination unit that determines whether or not the light propagation interference factor exists based on the aforementioned period and the light reception time of the peak, An information processing device as described in item 2 or 3. [Item 5] The threshold comparison unit detects a first time point in which the light reception frequency of the histogram matches the predetermined threshold, and a second time point at a time period after the first time point. The peak time determination unit compares a first period from the first time to the peak time with a second period from the peak time to the second time, and determines that the optical propagation interference factor exists if the second period is longer than the first period by a predetermined period or more. The information processing device described in item 4. [Item 6] The system includes a fitting unit that generates a curve of a first function that fits to a first curve generated by the light reception frequency of each bin of the histogram from the first time to the peak time, and a curve of a second function that fits to a second curve generated by the light reception frequency of each bin of the histogram from the peak time to the second time, The removal unit generates a third curve containing the removed light reception frequency information based on the difference between the first curve and the second curve and the curve of the first function and the curve of the second function. The information processing device described in item 5. [Item 7] Each of the above first function and the above second function is either a linear function or a quadratic function, The fitting unit generates a first function that passes through the first intersection point and the peak of the first curve and the threshold at the first time, and approximates the shape of the first curve between the first intersection point and the peak, and generates a second function that passes through the second intersection point and the peak of the second curve and the threshold at the second time, and approximates the shape of the second curve between the second intersection point and the peak. The information processing device described in item 6. [Item 8] The second peak detection unit detects the peak of the third curve. The information processing device described in item 7. [Item 9] The predetermined threshold is a fixed value set based on the average value of the sum of the light reception frequencies of all bins in the histogram. An information processing device as described in any one of items 4 through 8. [Item 10] The predetermined threshold changes according to the light reception frequency of the peak detected by the first peak detection unit. An information processing device as described in any one of items 4 through 8. [Item 11] The threshold comparison unit detects multiple periods by changing the threshold in multiple ways. The peak time determination unit determines whether or not the optical propagation interference factor exists based on each of the plurality of periods and the peak time. An information processing device as described in any one of items 4 through 8. [Item 12] If the determination unit determines that there are no light propagation interference factors, the second peak detection unit will detect the same peak as the peak detected by the first peak detection unit. An information processing device as described in any one of items 2 through 11. [Item 13] The determination unit determines that the light propagation interference factor exists if the curve generated by the light reception frequency of each bin of the histogram after the time of the peak becomes a gamma distribution curve. An information processing device as described in any one of items 2 through 12. [Item 14] The determination unit determines that there are no light propagation interference factors if the curve generated by the light reception frequency of each bin of the histogram after the time of the peak is a Gaussian distribution curve. An information processing device as described in any one of items 2 through 12. [Item 15] The removal unit removes the light reception frequency information due to the light propagation interference factors included in the histogram and the light reception frequency information due to ambient light. An information processing device as described in any one of items 2 through 14. [Item 16] If the determination unit determines that the light propagation interference factor is present, the histogram generation unit generates a histogram that includes the frequency of reception due to ambient light, the frequency of reception due to reflected light from the object, and the frequency of reception due to the light propagation interference factor. An information processing device as described in any one of items 2 through 15. [Item 17] A light-receiving unit that repeatedly receives light reflected from an object, A device comprising an information processing device as described in any one of items 2 to 16, Ranging device. [Item 18] A light-emitting unit that repeatedly projects light onto the aforementioned object, The device includes the distance measuring device described in item 17, The distance measuring unit measures the distance to the object based on the time difference between the time the light-emitting unit emitted light and the time of the peak detected by the second peak detection unit. Distancing system. [Item 19] Based on multiple received light signals, including reflected light from an object, a histogram is generated showing the relationship between the time of reception and the frequency of reception. The peak of the light reception frequency in the aforementioned histogram is detected, Based on the histogram and the light reception time of the peak, it is determined whether or not there is a light propagation interference factor, including at least one of fog, mist, or haze. Based on the time of reception of the peak and the degree of change in the reception frequency of the histogram before and after the time of reception of the peak, it is determined whether or not the light propagation interference factor exists. Information processing methods.
[0060] The aspects of this disclosure are not limited to the individual embodiments described above, but include various modifications that a person skilled in the art could conceive, and the effects of this disclosure are not limited to those described above. In other words, various additions, modifications, and partial deletions are possible, as long as they do not depart from the conceptual idea and spirit of this disclosure derived from the claims and their equivalents. [Explanation of symbols]
[0061] 1. Information processing device, 2. Distance measuring system, 3. Light projection unit, 4. Light detection unit, 5. Light source unit, 6. Light scanning unit, 7. Light receiving unit, 8. Information processing unit, 9. Image processing unit, 10. Distance measuring device, 11. Histogram generation unit, 12. First peak detection unit, 13. Judgment unit, 14. Removal unit, 15. Second peak detection unit, 16. Distance measurement unit, 17. Threshold comparison unit, 18. Peak time determination unit, 19. Fitting unit, 20. Object, 25. Storage unit
Claims
1. A histogram generation unit generates a histogram showing the relationship between light reception time and light reception frequency based on multiple light reception signals, A first peak detection unit detects a first peak of the light reception frequency in the histogram, A determination unit detects a period during which the light reception frequency of the histogram exceeds a predetermined threshold, detects a first time point at which the light reception frequency of the histogram matches the predetermined threshold, and a second time point at a period after the first time point, and determines whether or not there is a light propagation interference factor, including at least one of fog, mist, haze, rain, or snow, based on the first time point, the second time point, and the time of the first peak. If the determination unit determines that the light propagation interference factor exists, the fitting unit generates a curve of a first function that fits to a first curve generated by the light reception frequency of each bin of the histogram from the first time to the time of the first peak, and generates a curve of a second function that fits to a second curve generated by the light reception frequency of each bin of the histogram from the time of the first peak to the time of the second time. The system includes a removal unit that generates a third curve by the difference between the first curve and the second curve, and the curve of the first function and the curve of the second function, The determination unit determines whether or not the light propagation interference factor exists based on the light reception time of the first peak and the degree of change in the light reception frequency of the histogram before and after the light reception time of the first peak. Each of the first and second functions is either a linear or quadratic function. The fitting unit generates a first function that passes through the first intersection point and the first peak of the first curve and the threshold at the first time, and approximates the shape of the first curve between the first intersection point and the first peak, and generates a second function that passes through the second intersection point and the first peak of the second curve and the threshold at the second time, and approximates the shape of the second curve between the second intersection point and the first peak. Information processing device.
2. A second peak detection unit for detecting the second peak of the third curve, The system includes a distance measuring unit that measures the distance of an object based on the second peak, The information processing apparatus according to claim 1.
3. The system includes an image processing unit that generates three-dimensional data including the object based on the distance of the object measured by the distance measuring unit. The information processing apparatus according to claim 2.
4. The determination unit compares a first period from the first time to the time of the first peak with a second period from the time of the first peak to the second time, and determines that the optical propagation interference factor exists if the second period is longer than the first period by a predetermined period or more. The information processing apparatus according to claim 1.
5. The predetermined threshold is a fixed value set based on the average value of the sum of the light reception frequencies of all bins in the histogram. The information processing apparatus according to claim 1.
6. The predetermined threshold changes according to the light reception frequency of the first peak detected by the first peak detection unit. The information processing apparatus according to claim 1.
7. The determination unit detects multiple periods by changing the threshold value in multiple ways. Based on each of the aforementioned multiple periods and the time of the first peak, it is determined whether or not the optical propagation interference factor exists. The information processing apparatus according to claim 1.
8. If the determination unit determines that there are no light propagation interference factors, the second peak detection unit detects the first peak detected by the first peak detection unit as the second peak. The information processing apparatus according to claim 2.
9. The determination unit determines that the light propagation interference factor exists when the curve generated by the light reception frequency of each bin of the histogram after the time of the first peak becomes a gamma distribution curve. The information processing apparatus according to claim 1.
10. The determination unit determines that there are no light propagation interference factors if the curve generated by the light reception frequency of each bin of the histogram after the time of the first peak is a Gaussian distribution curve. The information processing apparatus according to claim 1.
11. The removal unit removes the light reception frequency information due to the light propagation interference factors included in the histogram and the light reception frequency information due to ambient light. The information processing apparatus according to claim 1.
12. If the determination unit determines that the light propagation interference factor is present, the histogram generation unit generates a histogram that includes the frequency of reception due to ambient light, the frequency of reception due to reflected light from an object, and the frequency of reception due to the light propagation interference factor. The information processing apparatus according to claim 1.
13. A light receiving unit that generates multiple light receiving signals, The information processing device according to claim 2 comprises, Ranging device.
14. A light-emitting unit that repeatedly projects light, The distance measuring device is as described in claim 13, The distance measuring unit is a distance measuring system that measures the distance to an object based on the time difference between the time the light-emitting unit emits light and the time of the peak detected by the second peak detection unit.
15. Based on multiple received light signals, a histogram is generated showing the relationship between the time of light reception and the frequency of light reception. The first peak of the light reception frequency in the histogram is detected, The system detects a period during which the light reception frequency of the histogram exceeds a predetermined threshold, detects a first time point at which the light reception frequency of the histogram matches the predetermined threshold, and detects a second time point at a period after the first time point. Based on the first time point, the second time point, and the time of the first peak, it determines whether or not there is a light propagation interference factor, including at least one of fog, mist, haze, rain, or snow. If it is determined that the aforementioned light propagation interference factor exists, a curve of a first function is generated that fits to a first curve generated by the light reception frequency of each bin of the histogram from the first time to the time of the first peak, and a curve of a second function is generated that fits to a second curve generated by the light reception frequency of each bin of the histogram from the time of the first peak to the second time. A third curve is generated by the difference between the first curve and the second curve, and the curve of the first function and the curve of the second function. Based on the reception time of the first peak and the degree of change in the reception frequency of the histogram before and after the reception time of the first peak, it is determined whether or not the light propagation interference factor exists. Each of the first and second functions is either a linear or quadratic function. A first function is generated that passes through the first intersection point and the first peak of the first curve and the threshold at the first time, and approximates the shape of the first curve between the first intersection point and the first peak; and a second function is generated that passes through the second intersection point and the first peak of the second curve and the threshold at the second time, and approximates the shape of the second curve between the second intersection point and the first peak. Information processing methods.