Object detection device and vehicle control system

Abstract

To provide an object detection device capable of enhancing the detection accuracy of a high-reflection object.SOLUTION: An object detection device 1 detects an object in a measurement area MA. The object detection device 1 includes: a light-emitting section 10 that has light sources 11a to d for emitting light and projects the light to the measurement area MA; a light-receiving section 30 that has a light-receiving element 32 and detects the light received by the light-receiving element 32; and a data processing section 50 that processes reflected light data obtained by detecting the light reflected from the object with the light-receiving section 30 when the light sources 11a to d emits light and background light data obtained by detecting with the light-receiving section 30 when the light sources 11a to d do not emit light. The data processing section 50 has a flare identification section 52 for identifying a flare in the reflected light data on the basis of a comparison of the reflected light data and the background light data.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The disclosure of this specification relates to a technique for detecting an object and a technique for using the same to control a vehicle. [Background technology]

[0002] There is known an object detection device. Patent Document 1 discloses that the number of light emission pulses and the intensity of light emission pulses are reduced for highly reflective objects. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-169336 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the configuration of Patent Document 1 cannot completely eliminate the occurrence of flare because light is emitted toward a highly reflective object. Therefore, there is a demand for a technology that can accurately detect an object even when flare occurs.

[0005] One of the objectives of the disclosure of this specification is to provide an object detection device that improves the detection accuracy for highly reflective objects. Another objective is to provide a vehicle control system that can appropriately respond to highly reflective objects. [Means for solving the problem]

[0006] One aspect disclosed herein is an object detection device for detecting an object in a measurement area (MA), comprising: a light emitting unit (10, 110) having light sources (11a-d, 111a-d) that emit light and projects the light toward a measurement area; a light receiving section (30) having a light receiving element (32) and detecting light received by the light receiving element; a data processing unit (50) for processing reflected light data obtained by detecting light reflected from an object when the light source emits light and background light data obtained by detecting light reflected from an object when the light source emits light, and The data processing unit includes a flare identifying unit (52) that identifies flare in the reflected light data based on a comparison of the reflected light data and the background light data.

[0007] According to this aspect, by comparing the reflected light data with the background light data, the accuracy of flare identification in the reflected light data is improved. That is, the influence of flare that may be caused by a highly reflective object can be more accurately determined in the object detection results. Therefore, it is possible to provide an object detection device that improves the detection accuracy for highly reflective objects.

[0008] Another disclosed aspect is an object detection device that detects an object in a measurement area (MA), a light emitting unit (10, 110) having light sources (11a-d, 111a-d) that emit light and projects the light toward a measurement area; a light receiving section (30) having a light receiving element (32) and detecting light received by the light receiving element; The light emitting unit emits light from the light source in a direction toward the highly reflective object among the objects with a weaker emission intensity than in other directions.

[0009] According to this aspect, by weakening the light emitted to the highly reflective object, the intensity of the light reflected by the highly reflective object and received by the light receiving unit can be kept relatively low. Therefore, the occurrence of flare caused by the highly reflective object can be suppressed. Therefore, it is possible to provide an object detection device that improves the detection accuracy for highly reflective objects.

[0010] Another disclosed aspect is a vehicle control system for controlling a vehicle, comprising: an object detection device (1) for detecting an object in a measurement area (MA); a driving control device (60) that controls driving of the vehicle based on information output from the object detection device, The object detection device a light emitting unit (10, 110) having light sources (11a-d, 111a-d) that emit light and projects the light toward a measurement area; a light receiving section (30) having a light receiving element (32) and detecting light received by the light receiving element; a data processing unit (50) for processing reflected light data obtained by detecting light reflected from an object when the light source emits light and background light data detected by the light receiving unit when the light source does not emit light; The data processing unit has a flare identifying unit (52) that identifies the occurrence of flare in the reflected light data based on a comparison between the reflected light data and the background light data, and when the flare identifying unit determines that a flare has occurred, outputs a signal notifying the occurrence of a flare to the operation control device; The operation control device includes a collision avoidance control unit (62) that, when receiving a signal notifying the occurrence of a flare, restricts the operation of a collision avoidance action in the direction of the flare occurrence.

[0011] According to this aspect, the object detection device compares the reflected light data with the background light data, thereby improving the accuracy of flare identification in the reflected light data. Information about the highly accurate flare identification is provided to the driving control device, which restricts the execution of collision avoidance actions in the direction of the flare. This makes it possible to provide a vehicle control system that can appropriately respond to highly reflective objects that cause flare while avoiding unnecessary actions in response to the flare.

[0012] The reference numerals in parentheses are intended to exemplify the correspondence with the parts of the embodiments described later, and are not intended to limit the technical scope. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an object detection device. [Figure 2] FIG. 3 is a diagram illustrating scanning of a projected beam. [Figure 3] FIG. 2 is a configuration diagram illustrating functions of a data processing unit. [Figure 4] 10 is a graph illustrating the relationship between reflection intensity and background light intensity. [Figure 5] 10 is a graph illustrating the relationship between reflection intensity and distance. [Figure 6] FIG. 1 is a diagram illustrating the relationship between images. [Figure 7] 10 is a flowchart illustrating an example of a processing method. [Figure 8] 10 is a graph illustrating the relationship between reflection intensity and flare likelihood. [Figure 9] 10 is a graph illustrating the relationship between pulse width and flare likelihood. [Figure 10] 10A to 10C are diagrams illustrating a process of reducing an object size. [Figure 11] FIG. 2 is a diagram illustrating the relationship between a light source and an image area. [Figure 12] FIG. 2 is a configuration diagram illustrating functions of a data processing unit. [Figure 13] FIG. 2 is a configuration diagram illustrating functions of a data processing unit. [Figure 14] 5A and 5B are diagrams illustrating the relationship between light emission intensity control and an image. [Figure 15] FIG. 10 is a diagram illustrating a flare removal model. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, several embodiments will be described with reference to the drawings. Note that corresponding components in each embodiment are given the same reference numerals, and redundant description may be omitted. When only a portion of the configuration is described in each embodiment, the configuration of another embodiment described previously can be applied to the remaining portion of the configuration. Furthermore, in addition to the combinations of configurations explicitly stated in the description of each embodiment, configurations of several embodiments can also be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.

[0015] (First embodiment) As shown in Fig. 1, an object detection device 1 according to a first embodiment of the present disclosure is mounted on a vehicle as a moving body and detects objects around the vehicle. The object detection device 1 includes a LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) 2 and an image processing unit 3. The object detection device 1 may also be referred to as a LiDAR unit. In the following description, the directions indicated by front, rear, up, down, left, and right are defined with respect to the vehicle on a horizontal plane.

[0016] The LIDAR 2 is a light detection device that projects light from light sources 11a to 11d and detects reflected light from an object that is a measurement target. The LIDAR 2 includes a light emitting unit 10, a scanning unit 20, a light receiving unit 30, and a controller 40.

[0017] The light emitting unit 10 emits a projection beam toward the measurement area MA. The light emitting unit 10 includes, for example, a light source array 11 and a light projection optical system 12. As shown in FIG. 2, the light source array 11 is formed by arranging a plurality of light sources 11a-d, for example, in the vertical direction. The number of light sources 11a-d may be any number. Each of the light sources 11a-d may be a laser oscillation element such as a laser diode (LD). Each of the light sources 11a-d may also be an LED. Each of the light sources 11a-d can emit light at a light emission timing according to an electrical signal from the controller 40.

[0018] The light projection optical system 12 collects the light emitted from the light sources 11a to 11d and projects a beam-shaped projection light toward the measurement area MA. The light projection optical system 12 includes one or more lenses.

[0019] As shown in FIG. 2, the scanning unit 20 scans the projected beam from the light emitting unit 10 within the measurement area MA. The scanning unit 20 includes a scanning mirror 21. The scanning mirror 21 includes a drive motor and a reflector. The drive motor is, for example, a voice coil motor, a brushed DC motor, or a stepping motor. The drive motor drives the rotation shaft of the reflector at an amount of rotation and a rotation speed corresponding to an electrical signal from the controller 40. The reflector is a mirror having a reflective surface that reflects the projected beam toward the measurement area MA. The reflective surface is, for example, flat.

[0020] In particular, in this embodiment, the reflector has an elongated shape with a longitudinal dimension along the rotation axis direction. This causes the reflector to face both the light projecting optical system 12 of the light emitter 10 and the light receiving optical system 31 of the light receiving unit 30. Therefore, the reflector can simultaneously reflect not only the projected beam but also the reflected beam that is the projected beam reflected by an object in the measurement area MA, and make it incident on the light receiving unit 30.

[0021] The light receiving unit 30 includes a light receiving optical system 31, a light receiving element 32, and a decoder 33. The light receiving optical system 31 collects the reflected beam reflected by the reflector of the scanning unit 20 and makes it incident on the light receiving element 32. The light receiving optical system 31 includes one or more lenses.

[0022] The light receiving element 32 receives light from the light receiving optical system 31. The light receiving element 32 is, for example, a single photon avalanche diode (SPAD) sensor. The light receiving element 32 is formed by two-dimensionally arranging multiple SPADs in a highly integrated state on a rectangular detection surface.

[0023] When a SPAD receives a single photon, it generates a single electrical pulse through avalanche multiplication (the so-called Geiger mode). In other words, a SPAD can generate an electrical pulse as a digital signal directly, without going through an AD conversion circuit that converts analog signals to digital signals. Therefore, the light reception results can be read out at high speed.

[0024] The decoder 33 is provided to output the electrical pulses generated by the SPADs, and includes a selection circuit and a clock oscillator. The selection circuit sequentially selects the SPADs from which to extract the electrical pulses. The selected SPADs output the electrical pulses to the controller 40. When the selection circuit has selected each SPAD, one sampling cycle is completed. This sampling cycle corresponds to the clock frequency output from the clock circuit.

[0025] When the light emitter 10 is not emitting a light beam, i.e., when no light is being emitted, the light receiver 30 can receive and detect ambient light from a direction corresponding to the angle of the reflector. Furthermore, the ambient light is imaged on the light receiver 32, making it possible to capture an object in the measurement area MA. The ambient light referred to here is also referred to as background light.

[0026] The controller 40 controls the operation of the lidar 2. Specifically, the controller 40 controls the light emission of each light source 11a-d in the light-emitting unit 10 and the orientation of the scanning mirror 21 in the scanning unit 20 so as to be linked. Furthermore, the controller 40 outputs detection data obtained by processing the electrical pulses output from the light-receiving unit 30 to the image processing unit 3. The controller 40 may be configured by a dedicated computer having at least one memory and one processor, or may be realized by an FPGA (Field-Programmable Gate Array), ASIC (Application Specific Integrated Circuit), or the like.

[0027] Here, the controller 40 may have a function of calculating the distance from the lidar 2 to an object in the measurement area MA. The calculated distance may be included in the detection data. Alternatively, the distance may be calculated in the image processing unit 3. The distance may also be referred to as depth. This distance may be measured by a TOF (Time Of Flight) method, which measures the so-called time of flight of light.

[0028] The image processing unit 3 generates and processes images based on the detection data acquired from the lidar 2. The image processing unit 3 may be realized by a dedicated computer. The image processing unit 3 may have at least one memory 3a and one processor 3b. The memory 3a may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores programs and data readable by the processor 3b. Furthermore, the memory 3a may be provided with a rewritable volatile storage medium, such as a random access memory (RAM). The processor 3b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0029] Based on the detection data acquired from the controller 40, the image processing unit 3 generates various images such as a reflection intensity image P1, a distance image P0, a pulse width image P2, and a background light image P3.

[0030] The reflection intensity image P1 is a two-dimensional image that shows the received light intensity of the reflected beam in the detection data corresponding to the light emission timing of the light emitting unit 10, and indicates the reflection intensity from each direction in the measurement area MA.

[0031] Distance image P0 is a two-dimensional image of measurement area MA obtained by visualizing the distance measured by the TOF method from the reflected beam in the detection data corresponding to the light emission timing of light-emitting unit 10.

[0032] The pulse width image P2 is a two-dimensional image of the measurement area MA obtained by imaging the pulse width of the reflected beam in the detection data corresponding to the light emission timing of the light emitting unit 10.

[0033] The background light image P3 is a two-dimensional image of the measurement area MA, which is obtained by visualizing the received light intensity of the background light in the detection data corresponding to the timing when the light emitting unit 10 is not emitting light.

[0034] Here, the reflection intensity image P1, distance image P0, and pulse width image P2 may correspond to reflected light data. The reflected light data is data obtained by detecting light reflected from an object when the light source emits light, using the light receiving unit 30. The reflected light data is not limited to images, but is a concept that includes any form of data, such as simple numerical data.

[0035] The background light image P3 may correspond to background light data. The background light data is data obtained by detecting the light receiving unit when the light source is not emitting light. The background light data is not limited to an image, but is a concept that includes any data format, such as simple numerical data.

[0036] Furthermore, the image processing unit 3 may output a feedback signal to the controller 40 for the controller 40 to control the light emitting unit 10 and the light receiving unit 30 based on the generated images or the results of the image processing.

[0037] 3 is a diagram showing the functional architecture of the object detection device 1. Here, the controller 40 and the image processing unit 3 cooperate with each other to control the light-emitting unit 10, the scanning unit 20, and the light-receiving unit 30, and also function as a data processing unit 50 for providing detection results to the outside. The data processing unit 50 includes an image generating unit 51, a flare identifying unit 52, and a detection result output unit 53 as functional blocks realized by at least one processor that executes a program.

[0038] As described above, the image generation unit 51 generates various images such as the reflection intensity image P1, distance image P0, pulse width image P2, and background light image P3 using the image processing unit 3. The image generation unit 51 provides the various generated images to the flare identification unit 52.

[0039] The flare identifying unit 52 identifies flare from the data provided by the image generating unit 51. The flare identifying unit 52 may simply determine whether or not flare is occurring in the image. The flare identifying unit 52 may also identify a portion of the image or measurement area where flare is occurring. Flare tends to occur when a highly reflective object is present in the measurement area MA. Examples of highly reflective objects include signs, reflective materials attached to vehicles or bicycles, etc.

[0040] The flare identifying unit 52 identifies flare in the reflected light data based on a comparison of the reflected light data and the background light data. For example, by comparing the reflection intensity image P1, the distance image P0, and the background light image P3, it is possible to identify areas in the reflection intensity image P1 or the distance image P0 where flare is occurring.

[0041] Specifically, when no flare is occurring, i.e., when detection is normal, a comparison of the reflection intensity in reflection intensity image P1 with the background light intensity in background light image P3 between corresponding pixels or corresponding pixel regions of the image shows a positive correlation (see FIG. 4). Furthermore, a comparison of the reflection intensity in reflection intensity image P1 with the distance in distance image P0 between corresponding pixels or corresponding pixel regions of the image shows a negative correlation (see FIG. 5). If these correlations are disrupted, it is highly likely that a flare is occurring, i.e., detection is not normal.

[0042] Therefore, the flare identifying unit 52 identifies a flare by determining whether or not these relationships are established. The detailed method is described below. Either the first method or the second method described below may be implemented.

[0043] In the first method, a determination is made for each pixel. Specifically, the flare identifying unit 52 calculates the ratio of the reflection intensity to the background light intensity for corresponding pixels in the reflection intensity image P1 and the background light image P3. For a pixel where a flare occurs, the intensity ratio of the pixel to be determined, which is the reflection intensity divided by the background light intensity, is significantly larger than for a pixel where a flare does not occur.

[0044] That is, the flare identifying unit 52 sets in advance an allowable range for the intensity ratio, which is the reflection intensity divided by the background light intensity, according to the distance, and identifies whether a flare has occurred depending on whether the intensity ratio at the object to be identified exceeds the allowable range.

[0045] When a pixel in which a flare occurs is found, the flare identifying unit 52 performs the same determination on the surrounding pixels, thereby preventing pixel omissions in the flare determination.

[0046] In the second method, a determination is made for each pixel region, which is a collection of multiple adjacent pixels. Specifically, flare identifying unit 52 refers to distance image P0 and divides the image into multiple pixel regions by distance segments with the same or similar distance. Next, flare identifying unit 52 calculates the intensity ratio in each pixel region and identifies whether a flare has occurred based on whether the intensity ratio exceeds an allowable range.

[0047] Here, the flare identifying unit 52 does not have to perform the determination for all divided pixel regions. For example, the flare identifying unit 52 may search for a region where a strong reflected beam is likely to be returned, and perform the determination for this pixel region and surrounding pixel regions. This reduces the load of the flare identifying process.

[0048] As a specific example, consider the example in Figure 6. In the distance image P0 at the top, the object area OA, where an object exists, and the flare occurrence area FA, where flare is occurring, are calculated as a single area in distance calculation. Comparing the background light image P3 at the middle and the reflection intensity image P1 at the bottom, we see that in the object area OA, both the reflection intensity and the background light are strong enough to make the image appear white, and the intensity ratio is close to 1. In contrast, in the flare occurrence area FA, the reflection intensity makes the image appear slightly white, but the background light makes the image appear black, and the intensity ratio is significantly larger. Using these relationships, it is possible to identify the area in the distance image P0 where flare is occurring.

[0049] The detection result output unit 53 outputs, to an external output target seen from the data processing unit 50, data necessary for the output target, including various images generated based on the detection data, flare detection results, and information on detected objects. The output data may include a signal notifying the occurrence of a flare. The signal notifying the occurrence of a flare may include a signal notifying the presence or absence of a flare. The signal notifying the occurrence of a flare may include a signal notifying the area in which a flare has occurred.

[0050] The output target may include a light-emitting unit 10 to realize light-emitting control according to the detection result. The output target may include a light-receiving unit 30 to realize light-receiving control according to the detection result. The output target may include a driving control device 60 to realize vehicle control according to the detection result.

[0051] The driving control device 60 is mounted on a vehicle together with, for example, the object detection device 1. The driving control device 60 controls the driving of the vehicle based on information output from the object detection device 1, etc. Controlling the driving of the vehicle may involve performing automatic driving or assisting the driver in driving. The driving control device 60, together with the object detection device 1, may constitute a vehicle control system VCS that controls the vehicle.

[0052] The operation control device 60 may be realized by a dedicated computer. The operation control device 60 may have at least one memory and one processor. The memory may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores programs and data that can be read by the processor. Further, the memory may be a rewritable volatile storage medium, such as a random access memory (RAM). The processor may include at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0053] As shown in FIG. 3, the driving control device 60 includes a driving mode management unit 61 and a collision avoidance control unit 62 as functional blocks realized by at least one processor that executes a program.

[0054] The driving mode management unit 61 manages the driving mode of the vehicle. For example, the driving mode management unit 61 may switch between automatic driving and manual driving depending on the driver's switch operation, the environment in which the vehicle is placed, etc. The driving mode management unit 61 may switch between an operating mode and an inoperating mode of an app that supports collision avoidance actions such as collision damage mitigation and emergency steering avoidance depending on the driver's switch operation, etc.

[0055] The collision avoidance control unit 62 controls the collision avoidance behavior of the vehicle to avoid a collision between the vehicle and an object detected in the measurement area MA. When the collision avoidance control unit 62 receives a signal notifying the occurrence of a flare from the object detection device 1, it restricts the activation of the collision avoidance behavior in the direction of the flare occurrence. In other words, the vehicle is controlled assuming that no object is present at a position the distance indicated by the distance image in the direction of the flare occurrence from the vehicle.

[0056] Next, examples of an object detection method and a vehicle control method will be described using the flowchart of Fig. 7. The series of processes shown in Fig. 7 is executed at a predetermined execution cycle or based on a predetermined trigger. The series of processes may be executed mainly by at least one processor included in the object detection device 1 and the driving control device 60 so as to realize the functions of the data processing unit 50 and the collision avoidance control unit 62, etc.

[0057] In S1, the measurement area is measured by the lidar 2. After the processing in S1, the process proceeds to S2.

[0058] In S2, the image generating unit 51 generates various images such as a reflection intensity image P1, a distance image P0, a pulse width image P2, and a background light image P3 based on the results of S1. After the processing of S2, the process proceeds to S3.

[0059] In S3, the flare identifying unit 52 identifies flares occurring in the reflection intensity image P1 using the image generated in S2. These detection results are transmitted to the operation control device 60 by the detection result output unit 53. After processing in S3, the process proceeds to S4.

[0060] In S4, the collision avoidance control unit 62 of the operation control device 60 restricts the collision avoidance behavior in the direction of the flare occurrence in response to the detection result transmitted in S3, particularly the signal notifying the occurrence of the flare. The series of processes ends with S4.

[0061] To summarize the first embodiment described above, by comparing reflected light data with background light data, the accuracy of identifying flare in the reflected light data is improved. In other words, the influence of flare that may be caused by a highly reflective object can be determined more accurately in the object detection results. Therefore, the accuracy of detecting highly reflective objects can be improved.

[0062] Furthermore, according to the first embodiment, flare is identified using the correlation between the reflection intensity in the reflected light data and the background light intensity in the background light data, and the correlation between the reflection intensity in the reflected light data and the object distance. Since the characteristics of the reflected light data and background light data that appear due to flare are accurately detected, the accuracy of flare identification can be further improved.

[0063] Furthermore, according to the first embodiment, the object detection device 1 compares the reflected light data with the background light data, thereby improving the accuracy of flare identification in the reflected light data. Information on the highly accurate flare identification is thus provided to the driving control device 60, which restricts the operation of collision avoidance actions in the direction of flare generation. It is possible to provide a vehicle control system VCS that can appropriately respond to highly reflective objects that cause flare generation while avoiding unnecessary actions in response to flare.

[0064] (Second embodiment) 8 to 10, the second embodiment is a modification of the first embodiment. The second embodiment will be described, focusing on the differences from the first embodiment.

[0065] The flare identifying unit 52 of the second embodiment identifies a flare by calculating a probability that the flare is likely to occur, instead of identifying the flare based on a correlation. The probability that the flare is likely to occur may be calculated on a pixel-by-pixel basis, or on a pixel region-by-pixel basis. The pixel region here may be divided by distance segments as in the first embodiment, or may be divided by other methods.

[0066] Specifically, the probability of indicating a flare can be calculated from the intensity ratio between the reflection intensity image P1 in the reflected light data and the background light image P3 in the background light data, the pulse width image P2 in the reflected light data, and the reflection intensity image P1.

[0067] Furthermore, it may be utilized that the probability of indicating a flare increases gradually as the reflection intensity of reflection intensity image P1 increases, for example, as shown in Fig. 8. It may also be utilized that the probability of indicating a flare increases gradually as the pulse width of pulse width image P2 increases, for example, as shown in Fig. 9.

[0068] When the probability of flare occurrence is calculated from two or more types of intensity ratios and images, these are probability integrated using likelihood. For example, the integrated probability is calculated using the following Equation 1.

[0069] X=A B / [A B+(1-A) (1-B)] (Formula 1) Here, A is the first likelihood and is a number ranging from 0 to 1. B is the second likelihood and is a number ranging from 0 to 1. Equation 1 is a formula for integrating two likelihoods (A, B). When integrating three likelihoods, it is possible to handle this by repeating the calculation of Equation 1 as follows: ((A, B), C).

[0070] For example, by substituting a value indicating the likelihood of flare based on the reflection intensity image P1 into A and a value indicating the likelihood of flare based on the pulse width image P2 into B, a value X can be calculated that combines the two values ​​indicating the likelihood of flare.

[0071] The flare identifying unit 52 may determine whether or not a flare has occurred based on the calculated probability indicating the likelihood of a flare. In this case, the flare identifying unit 52 provides the flare detection result including information indicating the presence or absence of a flare to the detection result output unit 53 or the like.

[0072] On the other hand, the flare identification unit 52 may provide a probability indicating the likelihood of a flare as a flare detection result without determining whether a flare has occurred. The image generation unit 51 or the detection result output unit 53 may reduce the size of an object detected based on various images, based on the probability indicating the likelihood of a flare. The object is detected using, for example, a distance image.

[0073] For example, as shown in Fig. 10, suppose a pixel region is detected that has been calculated to have a 60% probability of being a flare. In this case, rather than determining that the entire 60% pixel region is a single object, an area that is reduced by α times the pixel region is determined to be a single object. Here, α is a preset value greater than 0 and less than 1.

[0074] In particular, in this embodiment, the anisotropy in the occurrence of flare is estimated based on the configuration of the lidar 2, and the direction of reduction in the size of the object is set based on the estimated anisotropy. Specifically, in the light-emitting unit 10 of this embodiment, multiple light sources are arranged along the vertical direction of the vehicle, and flare is likely to occur in the vertical direction in the image. Therefore, under the assumption that flare occurs in the vertical direction relative to the area where the object is originally imaged, it is estimated that the area where the object is originally imaged is the area where 60% of the pixel area is reduced in size in the vertical direction.

[0075] Furthermore, the detection result output unit 53 transmits a signal notifying the driving control device 60 of the probability that a flare is likely to occur. In the driving control device 60, the collision avoidance control unit 62 determines at least one of braking force and steering control to be activated in the collision avoidance action, depending on the probability that a flare is likely to occur.

[0076] More specifically, when the probability of a region indicating the possibility of the presence of an object in the measurement region MA indicating a flare exceeds a preset threshold, the collision avoidance control unit 62 weakens the braking force set as a collision avoidance action for the region indicating the possibility of the presence of an object compared to when the probability of a flare is equal to or less than the threshold. In addition, the collision avoidance control unit 62 sets the steering control to careful steering rather than sudden steering.

[0077] For example, if the next time the driving control device 60 receives information about a detected object, it is determined that no object is present in the area indicating the possibility of an object being present, then sudden braking or sudden steering will not be required.

[0078] (Third embodiment) The third embodiment is a modification of the second embodiment, and the third embodiment will be described focusing on the differences from the second embodiment.

[0079] The flare identifying section 52 of the third embodiment determines whether a flare has occurred based on the calculated probability of flare occurrence. Here, if the calculated probability of flare occurrence is too unreliable to be used in determining whether a flare has occurred, the flare identifying section increases the amount of information used for identification. The case where the probability of flare occurrence is too unreliable to be used in determining whether a flare has occurred may be when the probability indicates an intermediate value (for example, 45 to 55%).

[0080] Specifically, the flare identifying unit 52 controls the light emitting unit to change the light emission intensity of the light source when the reliability of identifying the occurrence of a flare is low. The lidar 2 measures the object in the measurement area MA under conditions where the light emission intensity has been changed. This increases the amount of reflected light data (e.g., reflection intensity image P1, pulse width image P2) used for identification. The flare identifying unit 52 integrates the probabilities calculated from data under different conditions, thereby increasing the reliability. In other words, the probability indicating a flare is more likely to fluctuate from an intermediate value to a value close to 0 or 1.

[0081] (Fourth embodiment) As shown in Fig. 11, the fourth embodiment is a modification of the second embodiment. The fourth embodiment will be described, focusing on the differences from the second embodiment.

[0082] In the light-emitting unit 110 of the fourth embodiment, each light source constituting the light source array 111 emits light in a state in which a characteristic is given to it that makes it possible to identify which light source among the plurality of light sources 111a to 111d is detected when detected by the light-receiving element 32.

[0083] The characteristics that enable identification of which of the multiple light sources 111a-d has detected light may include at least one of light emission intensity, light emission time, light phase, and light wavelength. Specifically, the light sources may be identified in the reflected light by making the light emission intensities of the light sources 111a-d different from one another. Alternatively, the light sources may be identified in the reflected light by making the light emission times (e.g., pulse widths) of the light sources 111a-d different from one another. Alternatively, the light sources may be identified in the reflected light by making the phases of the projected beams emitted by the light sources 111a-d different from one another. Alternatively, the light sources may be identified in the reflected light by making the wavelengths of the projected beams emitted by the light sources 111a-d different from one another.

[0084] The flare identifying unit 52 may extract features for identifying the light sources 111a-d from the detection data from the light receiving unit 30 or from each image and use the extracted features to identify a flare. For example, the flare identifying unit 52 may determine that a flare has occurred when it receives light having characteristics different from the characteristics of an expected light source. Alternatively, for example, the flare identifying unit 52 may determine that a flare has occurred when it receives abnormally strong reflected light of a certain wavelength from a direction in which the light source emits weak light of that wavelength.

[0085] 11 shows an example in which the wavelengths emitted by the light sources 111a-d arranged in the vertical direction in the light source array 111 are made different from one another. In this case, the distance image P0, the reflection intensity image P1, and the pulse width image P2 can be analyzed as having image regions PAa-d obtained by dividing the image in the vertical direction by the number of light sources.

[0086] Specifically, when no flare is occurring, reflected light originating from light source 111a is detected in image area PAa. Reflected light originating from light source 111b is detected in image area PAb. Reflected light originating from light source 111c is detected in image area PAc. Reflected light originating from light source 111d is detected in image area PAd. For example, if reflected light originating from light source 11b is detected in image area PAc, the possibility of flare occurring increases.

[0087] Furthermore, even when the flare identifying unit 52 is configured to include a neural network and identifies flares using a machine-learned identification model (see also the seventh embodiment), the above-described features increase the number of feature amounts that can be used for identification, thereby improving the accuracy of identification.

[0088] (Fifth embodiment) As shown in Fig. 12, the fifth embodiment is a modification of the first embodiment. The fifth embodiment will be described, focusing on the differences from the first embodiment.

[0089] The data processing unit 50 of the fifth embodiment further includes, as functional blocks, a direction-specific light emission intensity changing unit 54 and a light receiving sensitivity changing unit 55. The direction-specific light emission intensity changing unit 54 controls the light emitting unit 10 so as to change the light emission intensity of light emitted from the light sources 11a to 11d according to the light emission direction.

[0090] When flare identification unit 52 of the fifth embodiment determines that a flare has occurred, it identifies the flare source that caused the flare, based on the flare recognition result, various images such as distance image P0, and object information detected from distance image P0, etc. The flare source is selected from among objects present in measurement area MA.

[0091] The direction-specific light emission intensity changing unit 54 sets the light emission intensity for each light emission direction according to the flare identification result obtained from the flare identifying unit 52. The direction-specific light emission intensity changing unit 54 reduces the light emission intensity in the direction where the flare source exists compared to the light emission intensity in other directions. Hereinafter, the light emission intensity in other directions will be referred to as the reference intensity.

[0092] The direction-specific light emission intensity changing unit 54 may set the amount of reduction in light emission intensity in the direction where the flare source exists, depending on the distance from the lidar 2 to the flare source (hereinafter referred to as the object distance). For example, when the object distance to the flare source is greater than a preset threshold, the direction-specific light emission intensity changing unit 54 may reduce the light emission intensity in the direction where the flare source exists by a first reduction amount with respect to the reference intensity. On the other hand, when the object distance to the flare source is equal to or less than this threshold, the direction-specific light emission intensity changing unit 54 may reduce the light emission intensity in the direction where the flare source exists by a second reduction amount with respect to the reference intensity.

[0093] Here, the first reduction amount is larger than the second reduction amount. That is, as the object distance of the flare source becomes closer, the reflection intensity received by the light receiving unit 30 increases, making it easier for flare to occur, and therefore the emission intensity is weakened.

[0094] Here, the direction-specific light emission intensity changing unit 54 may further turn off light emission in the direction in which the flare source exists when the object distance to the flare source is equal to or less than the above-mentioned threshold value or equal to or less than a second threshold value that is set to a value smaller than the above-mentioned threshold value.

[0095] The light-receiving sensitivity change unit 55 changes the light-receiving sensitivity of the light-receiving element 32 in accordance with the emission intensity set by the direction-specific emission intensity change unit 54. For example, by setting high sensitivity to a reflected beam incident from a direction in which the emission intensity is reduced and a flare source is present, it becomes easier to detect an object even if the emission intensity is reduced. Here, increasing the sensitivity may mean changing the threshold value used by the light-receiving element 32 to distinguish between a signal and noise, making it easier to pick up a weak signal as a signal.

[0096] According to the fifth embodiment described above, by weakening the light emitted to a highly reflective object, the intensity of the light reflected by the highly reflective object and received by the light receiving unit 30 can be kept relatively low. Therefore, the occurrence of flare caused by a highly reflective object can be suppressed. Therefore, the detection accuracy for highly reflective objects can be improved.

[0097] (Sixth embodiment) 13 and 14, the sixth embodiment is a modification of the fifth embodiment. The sixth embodiment will be described, focusing on the differences from the fifth embodiment.

[0098] The data processing unit 50 of the sixth embodiment acquires map information from a map database (hereinafter referred to as map DB) 91 mounted on the vehicle. The map DB 91 includes at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium. The map DB 91 can acquire and store the latest map information from an external map server 92, for example, by V2X communication via a communication device mounted on the vehicle.

[0099] The map DB 91 stores two-dimensional or three-dimensional high-precision map data as map information. The high-precision map data may include object position information. Furthermore, the high-precision map data may be supplemented with the reflection characteristics of static objects or reflection estimation information that allows the reflection characteristics to be estimated. The reflection characteristics may simply be the reflectance of the static object. The reflection estimation information may be the type of static object (signboard, tree, etc.), or may be the color, surface shape, etc.

[0100] In addition to the light emission control of the fifth embodiment, the direction-specific light emission intensity changing unit 54 of the sixth embodiment may adjust the light emission intensity using map data from the map DB 91. When a stationary object is recognized as a highly reflective object based on the reflection characteristics or reflection estimation possible information in the map data, the direction-specific light emission intensity changing unit 54 may reduce the light emission intensity in the direction in which the stationary object exists.

[0101] By using map data to reduce the emission intensity toward highly reflective objects in advance, it is possible to prevent flare from occurring in the first place.

[0102] As a specific example, the case of Fig. 14 will be described. When the direction-specific emission intensity changing unit 54 acquires information about a signboard SB as a highly reflective object from map data, it reduces the emission intensity in the direction toward the signboard SB. Specifically, of the multiple light sources 11a-d, the emission intensity of light sources 11a and 11b that can project a projection beam in the direction toward the signboard SB is weakened only during the period when the angle of the scanning mirror 21 is aligned with the signboard SB. In the reflection intensity image P1 obtained by this control, the area WA around the signboard SB becomes a reflection intensity image obtained with a weak emission intensity, and the other area SA becomes a reflection intensity image obtained with the standard intensity.

[0103] Furthermore, the driving control device 60 may use the map data in the map DB 91. For example, the collision avoidance control unit 62 of the driving control device 60 determines whether to perform a collision avoidance action based on the map data and a signal notifying the occurrence of a flare.

[0104] Specifically, when collision avoidance control unit 62 receives a signal from object detection device 1 notifying the occurrence of a flare, it refers to map data and determines whether or not a highly reflective object is present in the direction of flare occurrence. If a highly reflective object is present in the direction of flare occurrence, collision avoidance control unit 62 estimates that the highly reflective object is the source of the flare, and determines to execute collision avoidance action based on the position information of a static object that is stored in map DB 91 and recognized as a highly reflective object. This can increase the accuracy of collision avoidance.

[0105] Seventh embodiment As shown in Fig. 15, the seventh embodiment is a modification of the first embodiment. The seventh embodiment will be described, focusing on the differences from the first embodiment.

[0106] The data processing unit 50 of the seventh embodiment has a flare removal model 52a that functions as a flare identification unit 52. The flare removal model 52a is a trained model mainly configured using a neural network. The flare removal model 52a is capable of outputting reflected light data from which flare has been removed by inputting reflected light data (e.g., a distance image P0, a reflection intensity image P1, and a pulse width image P2) and background light data (e.g., a background light image P3).

[0107] The flare removal model 52a can be trained by providing, for example, reflected light data, background light data, and corresponding correct data. The correct data may be manually input at the discretion of an operator who creates the flare removal model 52a. The correct data may also be provided by another sensor that measures the same measurement area as the reflected light data and background light data.

[0108] The data processing unit 50 can obtain object information in a state in which flare has been removed using the flare removal model 52a. That is, based on the distance image P0 in a state in which flare has been removed and output from the flare removal model, the data processing unit 50 can detect an object and provide the detected object to an external operation control device 60 or the like.

[0109] (Other embodiments) Although multiple embodiments have been described above, the present disclosure should not be construed as being limited to those embodiments, and can be applied to various embodiments and combinations within the scope that does not deviate from the gist of the present disclosure.

[0110] Specifically, the object detection device 1 does not have to be mounted on a vehicle. The object detection device 1 may be provided in a roadside unit that constitutes infrastructure and detect objects on the road in a fixed state.

[0111] Furthermore, the object information detected by the object detection device 1 does not have to be used for vehicle control. For example, the object information detected by the object detection device 1 may be used for collecting big data for developing a road network.

[0112] In addition, a configuration that adds a feature that makes it possible to identify which of the multiple light sources 111a to 111d the light source is detected from in detection by the light receiving element 32 in the fourth embodiment may be applied to the fifth to seventh embodiments to improve the accuracy of flare identification.

[0113] The controller and methods described herein may be implemented by a special-purpose computer comprising a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and methods described herein may be implemented by special-purpose hardware logic circuitry. Alternatively, the apparatus and methods described herein may be implemented by one or more special-purpose computers comprising a processor executing a computer program in combination with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium. [Explanation of symbols]

[0114] 1: object detection device, 10, 110: light emitting unit, 11a-d, 111a-d: light source, 30: light receiving unit, 32: light receiving element, 50: data processing unit, 52: flare identification unit, 60: driving control device, 62: collision avoidance control unit, VCS: vehicle control system, MA: measurement area

Claims

1. An object detection device for detecting an object in a measurement area (MA), a light emitting unit (10, 110) having a light source (11a-d, 111a-d) that emits light and projects the light toward the measurement area; a light receiving section (30) having a light receiving element (32) and detecting the light received by the light receiving element; a data processing unit (50) that processes reflected light data obtained by detecting light reflected from the object when the light source emits light and background light data obtained by detecting light reflected from the object when the light source emits light, and background light data obtained by detecting light reflected from the object when the light source does not emit light, The object detection device, wherein the data processing unit has a flare identifying unit (52) that identifies flare in the reflected light data based on a comparison between the reflected light data and the background light data.

2. 2. The object detection device according to claim 1, wherein the flare identifying unit identifies the flare using a correlation between a reflection intensity in the reflected light data and a background light intensity in the background light data, and a correlation between a reflection intensity in the reflected light data and an object distance.

3. the light source emits pulsed light, 2. The object detection device according to claim 1, wherein the flare identifying unit calculates a probability of a flare being present based on at least one of an intensity ratio between a reflection intensity image (P1) in the reflected light data and a background light image (P3) in the background light data, a pulse width image (P2) in the reflected light data, and the reflection intensity image.

4. The object detection device according to claim 3 , wherein the probability of indicating the flare likelihood is calculated by probability integrating the flare likelihood based on the pulse width image and the flare likelihood based on the reflection intensity image using likelihood.

5. The object detection device according to claim 3 , wherein the data processing unit reduces a size of the detected object based on the probability of the object being a flare.

6. 2. The object detection device according to claim 1, wherein when reliability in identifying the occurrence of a flare is low, the flare identifying unit changes the emission intensity of the light source and increases the reflected light data used for identification to increase the reliability.

7. 2. The object detection device according to claim 1, wherein the light sources are provided in a plurality of positions offset from each other, and emit light in a state in which a characteristic is imparted that makes it possible to identify which of the plurality of light sources has detected light in detection by the light receiving element.

8. The object detection device according to claim 7 , wherein the characteristics include at least one of light emission intensity, light emission time, light phase, and light wavelength.

9. The data processing unit a flare removal model (52a) that functions as the flare identification unit and is a trained model including a neural network, and that receives the reflected light data and the background light data as input and outputs the reflected light data in a state in which flare has been removed; The object detection device according to claim 1 , wherein the object information is obtained in a state where flare has been removed using the flare removal model.

10. a direction-specific light emission intensity change unit (54) that can change the light emission intensity of the light emitted from the light source according to the light emission direction, The object detection device according to claim 1 , wherein the direction-specific light emission intensity change unit sets the light emission intensity for each light emission direction in accordance with a flare identification result obtained by the flare identification unit.

11. the flare identifying unit identifies a flare source that causes a flare; The direction-specific light emission intensity changing unit When an object distance of the flare source is greater than a preset threshold, a light emission intensity in a direction in which the flare source exists is reduced by a first reduction amount; 11. The object detection device according to claim 10, wherein, when the object distance to the flare source is equal to or less than the threshold, the emission intensity in the direction in which the flare source exists is reduced by a second reduction amount greater than the first reduction amount.

12. 12. The object detection device according to claim 11, wherein the direction-specific light emission intensity changing unit turns off light emission in a direction in which the flare source is present when the object distance to the flare source is equal to or less than the threshold value or equal to or less than a second threshold value that is set to a value smaller than the threshold value.

13. The direction-specific light emission intensity changing unit acquiring map data including position information of a static object, the map data having added thereto the reflection characteristics of the static object or reflection estimation information that allows the reflection characteristics to be estimated; The object detection device according to claim 10 , wherein when the stationary object is recognized as a highly reflective object based on the reflection characteristics or the reflection estimation information, the light emission intensity in the direction in which the stationary object exists is reduced.

14. 14. The object detection device according to claim 10, further comprising a light receiving sensitivity change unit (55) that changes the light receiving sensitivity of the light receiving element in accordance with the light emission intensity set by the direction-specific light emission intensity change unit.

15. The object detection device according to claim 1 , wherein the flare identifying unit identifies a flare source that causes the flare.

16. An object detection device for detecting an object in a measurement area (MA), a light emitting unit (110) having light sources (111a-d) that emit light and that projects the light toward the measurement area; a light receiving section (30) having a light receiving element (32) and detecting the light received by the light receiving element; The light emitting unit emits light from the light source in a direction toward a highly reflective object among the objects with a weaker emission intensity than in other directions.

17. 17. The object detection device according to claim 16, wherein the light sources are provided in a plurality of positions offset from each other, and emit light in a state in which a characteristic is imparted that makes it possible to identify which of the plurality of light sources has detected light in detection by the light receiving element.

18. A vehicle control system for controlling a vehicle, An object detection device (1) for detecting an object in a measurement area (MA); a driving control device (60) that controls driving of the vehicle based on information output from the object detection device, The object detection device a light emitting unit (10, 110) having a light source (11a-d, 111a-d) that emits light and projects the light toward the measurement area; a light receiving section (30) having a light receiving element (32) and detecting the light received by the light receiving element; a data processing unit (50) that processes reflected light data obtained by detecting light reflected from the object when the light source emits light and background light data detected by the light receiving unit when the light source does not emit light, the data processing unit has a flare identifying unit (52) that identifies the occurrence of flare in the reflected light data based on a comparison between the reflected light data and the background light data, and when the flare identifying unit determines that a flare has occurred, outputs a signal notifying the occurrence of a flare to the operation control device; The vehicle control system includes a collision avoidance control unit (62) that, when receiving a signal notifying the occurrence of a flare, restricts the operation of collision avoidance actions in the direction of the flare.

19. the light source emits pulsed light, the data processing unit calculates a probability of indicating flare likelihood based on at least one of an intensity ratio between a reflection intensity image (P1) in the reflected light data and a background light image (P3) in the background light data, a pulse width image (P2) in the reflected light data, and the reflection intensity image, and further outputs a signal notifying the probability of indicating flare likelihood to the operation control device; The vehicle control system according to claim 18 , wherein the collision avoidance control unit determines at least one of braking force and steering control to be performed in the collision avoidance behavior, depending on the probability of indicating flare likelihood.

20. The system further includes a map database (91) storing map data including position information of a static object, the map data being added with the reflection characteristics of the static object or reflection estimation information that can estimate the reflection characteristics, 19. The vehicle control system according to claim 18, wherein the collision avoidance control unit determines whether to take a collision avoidance action in response to the map data and the signal notifying the occurrence of a flare.

Images