Object detection device

Abstract

To provide an object determination device which can accurately determine whether an object detected ahead of the vehicle influences travel of the vehicle or not.SOLUTION: An object determination device comprises: a first determination part which determines, when an object with a speed that is a reference speed or less is detected ahead of an own vehicle, whether the other vehicle that is predicted to pass over the object exists or not based on environmental information ahead of the own vehicle before the time when the object is detected; a second determination part which determines, when it is determined that the other vehicle exists, whether the other vehicle is a first vehicle whose distance between the vehicle body and a road surface is narrow, or the other vehicle is a second vehicle whose distance between the vehicle body and the road surface is wider than that of the first vehicle based on the environmental information during a prescribed period of time before the time when the object is detected; and a third determination part which determines, when it is determined that the other vehicle is the first vehicle, that the object does not influence travel of the own vehicle, and determines, when the speed of the object is faster than the reference speed; the other vehicle does not exit; or the other vehicle is the second vehicle, that the object influences the travel of the own vehicle.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] This disclosure relates to an object determination device.

Background Art

[0002] An automatic control system mounted on a vehicle generates a driving route of the vehicle based on the current position of the vehicle, the destination position of the vehicle, and map information, and controls the vehicle to travel along this driving route.

[0003] The automatic control system of the vehicle uses a sensor such as a camera to detect an object in front of the vehicle during driving. When an object stationary in front of the vehicle is detected, if this object is a manhole, a puddle or a plastic bag, it is not an obstacle that affects the driving of the vehicle. On the other hand, if this object is a tire or a log, it is an obstacle.

[0004] When it is determined that it is an obstacle, the automatic control system of the vehicle drives the vehicle to avoid the obstacle (for example, see Patent Document 1). On the other hand, when it is not possible to avoid the obstacle, the automatic control system stops the vehicle to prevent the vehicle from coming into contact with the obstacle.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] When the object in front of the vehicle is a manhole, a puddle or a plastic bag, it may be difficult for the automatic control system to accurately determine that the object is a manhole, a puddle or a plastic bag based on the output information of a sensor such as a camera. Therefore, the automatic control system may erroneously determine an object that is not an obstacle as an obstacle.

[0007] If the automatic control system mistakenly identifies manholes, puddles, or plastic bags, which do not affect vehicle movement, as obstacles, it may obstruct vehicle movement by either avoiding the obstacle or stopping the vehicle to prevent contact with the obstacle.

[0008] Therefore, the object detection device can accurately determine whether or not an object detected in front of a vehicle affects the vehicle's movement. [Means for solving the problem]

[0009] According to one embodiment, an object detection device is provided. This object detection device is characterized by comprising: a first determination unit that, when an object having a speed of less than or equal to a predetermined reference speed is detected in front of the vehicle, determines whether there is another vehicle that is presumed to have passed over the object based on environmental information representing the environment in front of the vehicle prior to the time the object was detected; a second determination unit that, when the first determination unit determines that there is another vehicle, determines, based on environmental information for a predetermined period prior to the time the object was detected, whether the other vehicle is a first vehicle with a narrow distance between its body and the road, or a second vehicle with a wider distance between its body and the road than the first vehicle; and a third determination unit that, when the second determination unit determines that the other vehicle is a first vehicle, determines that the object does not affect the driving of the vehicle, and determines that the object affects the driving of the vehicle if the object's speed is faster than the reference speed, or there is no other vehicle, or the other vehicle is a second vehicle. [Effects of the Invention]

[0010] The object detection device described herein can accurately determine whether or not an object detected in front of a vehicle will affect the vehicle's movement. [Brief explanation of the drawing]

[0011] [Figure 1] This is a diagram (part 1) illustrating the general operation of the object detection device of this embodiment. [Figure 2]This is a diagram (part 2) illustrating the general operation of the object detection device of this embodiment. [Figure 3] This is a schematic diagram of a vehicle on which the object detection device of this embodiment is implemented. [Figure 4] This is an example of an operation flowchart relating to the object determination process of the object detection device of this embodiment. [Modes for carrying out the invention]

[0012] Figures 1 and 2 illustrate an overview of the operation of the object detection device 11 of this embodiment. Hereinafter, an overview of the operation of the object determination process of the object detection device 11 disclosed herein will be described with reference to Figures 1 and 2. The object detection device 11 is an example of an object determination device.

[0013] Vehicle 10 is traveling on road 50. Vehicle 10 has a camera 2, an object detection device 11, and an automatic control device 12. The object detection device 11 uses the camera 2 to detect objects in front of vehicle 10. The automatic control device 12 drives vehicle 10 based on object detection information regarding the objects detected by the object detection device 11. Vehicle 10 is, for example, an autonomous vehicle.

[0014] In the example shown in Figure 1, the object detection device 11 detects an object 40 in front of the vehicle 10 based on environmental information such as a camera image representing the environment in front of the vehicle 10. The object detection device 11 also detects the speed of the object 40 based on the environmental information.

[0015] Since the speed of object 40 is below a predetermined reference speed (for example, 5 km / h), the object detection device 11 determines, based on environmental information prior to the time of detection of object 40, whether or not there are other vehicles that are presumed to have passed over object 40. When the speed of object 40 is below the reference speed, object 40 is presumed to be a stationary object such as a manhole, puddle, or plastic bag.

[0016] In the example shown in FIG. 1, when the object detection device 11 detects an object 40, it determines based on environmental information such as a camera image 70 before the time of detecting the object 40 that there is another vehicle 60 estimated to have passed over the object 40.

[0017] Then, the object detection device 11 determines that the vehicle 60 estimated to have passed over the object 40 is a first vehicle with a narrow distance between the vehicle body and the road. The first vehicle indicates a vehicle with a low ground clearance. Since a vehicle 60 with a low ground clearance is passing over the object 40, it is estimated that the object 40 is an object that does not affect the running of the vehicle 10 such as a manhole, a puddle, or a plastic bag.

[0018] Therefore, the object detection device 11 determines that the object 40 does not affect the running of the vehicle 10. The object detection device 11 does not notify the automatic control device 12 of the object detection information regarding the object 40. The automatic control device 12 drives the vehicle 10 to go straight on the road 50 without considering the object 40.

[0019] Also in the example shown in FIG. 2, the object detection device 11 detects an object 41 in front of the vehicle 10 based on environmental information such as a camera image representing the environment in front of the vehicle 10. The object detection device 11 detects the speed of the object 41 based on the environmental information.

[0020] Since the speed of the object 41 is below the reference speed, the object detection device 11 determines the presence or absence of other vehicles estimated to have passed over the object 41 based on the environmental information before the time of detecting the object 41.

[0021] In the example shown in FIG. 2, the object detection device 11 determines based on environmental information such as a camera image 70 before the time of detecting the object 41 that there is another vehicle 61 estimated to have passed over the object 41.

[0022] Then, the object detection device 11 determines that the vehicle 61 estimated to have passed over the object 41 is a second vehicle with a wider distance between the vehicle body and the road than the first vehicle. The second vehicle indicates a vehicle with a higher ground clearance than the first vehicle. Since the vehicle 61 with a higher ground clearance is passing over the object 41, the object 41 may be an obstacle affecting the travel of the vehicle 10.

[0023] Therefore, the object detection device 11 determines that the object 41 affects the travel of the vehicle 10. The object detection device 11 notifies the automatic control device 12 of the object detection information regarding the object 41. The automatic control device 12 drives the vehicle 10 considering the object 41.

[0024] Also, even when the speed of the object 41 is faster than the reference speed or there are no other vehicles, the object detection device 11 determines that the object 41 affects the travel of the vehicle 10. When the speed of the object 41 is faster than the reference speed, the object 41 may be a moving object such as a bicycle (the moving object is an example of an obstacle). Also, when there are no other vehicles, since there is no information on whether the object 41 affects the travel of other vehicles, the object 41 may be an obstacle affecting the travel of the vehicle 10.

[0025] As described in detail above, based on the other vehicles 60, 61 estimated to have passed over the objects 40, 41, the object detection device 11 can accurately determine whether the objects 40, 41 detected in front of the vehicle 10 affect the travel of the vehicle 10.

[0026] FIG. 3 is a schematic configuration diagram of the vehicle 10 in which the object detection device 11 is installed. The vehicle 10 includes a camera 2, a LiDAR sensor 3, an object detection device 11, an automatic control device 12, etc. Further, the vehicle 10 may have other distance measuring sensors (not shown) for measuring the distance to objects around the vehicle 10, such as a radar sensor.

[0027] Camera 2, LiDAR sensor 3, object detection device 11, and automatic control device 12 are connected to each other via an in-vehicle network 13 that conforms to standards such as a controller area network.

[0028] Camera 2 is mounted on vehicle 10 so as to face forward. Camera 2 captures camera images representing the environment of a predetermined area in front of vehicle 10, for example, at predetermined intervals. The camera images may show roads included within the predetermined area in front of vehicle 10, and road features such as lane markings on the road surface. The camera images captured by camera 2 may show other vehicles located in front of vehicle 10. Camera 2 has a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system that forms an image of the area to be captured on the two-dimensional detector. The camera image is an example of environmental information representing the environment in front of vehicle 10.

[0029] Each time camera 2 captures a camera image, it outputs the camera image and the time the image was captured to the object detection device 11, etc., via the in-vehicle network 13. The camera image is used by the object detection device 11 to detect objects around the vehicle 10.

[0030] The LiDAR sensor 3 is mounted, for example, on the outer surface of the vehicle 10 so as to face forward of the vehicle 10. At a reflected wave information acquisition time set at a predetermined period, the LiDAR sensor 3 emits a pulsed laser in a scanning manner toward the front of the vehicle 10 and receives reflected waves reflected by reflective objects. The time required for the reflected wave to return contains distance information between the vehicle 10 and an object located in the direction from which the laser was emitted. The LiDAR sensor 3 outputs reflected wave information, including the direction of laser irradiation and the time required for the reflected wave to return, along with the reflected wave information acquisition time when the laser was emitted, to the object detection device 11 via the in-vehicle network 13. The reflected wave information is used by the object detection device 11 to detect objects around the vehicle 10. The reflected wave information is an example of environmental information representing the environment in front of the vehicle 10.

[0031] The object detection device 11 performs detection processing and determination processing. To this end, the object detection device 11 has a communication interface (IF) 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 are connected via a signal line 24. The communication interface 21 has an interface circuit for connecting the object detection device 11 to the in-vehicle network 13.

[0032] Memory 22 is an example of a storage unit and includes, for example, volatile semiconductor memory and non-volatile semiconductor memory. Memory 22 stores computer programs and various data of applications used in information processing performed by the processor 23.

[0033] All or part of the functions of the object detection device 11 are functional modules implemented by, for example, a computer program running on the processor 23. The processor 23 has a detection unit 231 and a determination unit 232. The detection unit 231 detects an object in front of the vehicle 10 based on a camera image and / or reflected wave information, and outputs object detection information, including information about this object, to the automatic control device 12 via the in-vehicle network 13. Alternatively, the functional modules of the processor 23 may be dedicated arithmetic circuits provided on the processor 23. The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may further have other arithmetic circuits such as a logic unit, a numerical unit, or a graphics processing unit. The object detection device 11 is, for example, an Electronic Control Unit (ECU). Details of the operation of the object detection device 11 will be described later.

[0034] The automatic control device 12 controls the operation of the vehicle 10, including its movement. Based on object detection information output from the object detection device 11, the automatic control device 12 generates a driving plan that controls operations such as steering, driving, and braking. The automatic control device 12 outputs automatic control signals based on this driving plan to actuators (not shown), drive units (not shown), or brakes (not shown) that control the steering wheels via the in-vehicle network 13.

[0035] In Figure 3, the object detection device 11 and the automatic control device 12 are described as separate devices, but all or part of these devices may be configured as a single device.

[0036] Figure 4 is an example of an operation flowchart relating to the object determination process of the object detection device 11 in this embodiment. The object determination process of the object detection device 11 will be described below with reference to Figure 4. The object detection device 11 executes the object determination process according to the operation flowchart shown in Figure 4 at an object determination time having a predetermined period.

[0037] The detection unit 231 determines whether or not an object has been detected in front of the vehicle 10 based on the camera image and reflected wave information (step S101).

[0038] The detection unit 231 detects objects in front of the vehicle 10 that are located within a predetermined range from the vehicle 10, and their types, based on the camera image captured by the camera 2. These objects include other vehicles traveling around the vehicle 10. The detection unit 231 has a classifier that, for example, detects objects represented in an image by inputting a camera image. As the classifier, for example, a deep neural network (DNN) pre-trained to detect objects represented in an input image can be used. Preferably, this classifier is pre-trained to identify vehicles with a cargo bed. It is also preferable that this classifier is pre-trained to identify trucks. The detection unit 231 may use a classifier other than a DNN. For example, the detection unit 231 may use a support vector machine (SVM) as the classifier, which is pre-trained to take feature quantities (e.g., Histograms of Oriented Gradients, HOG) calculated from a window set on the camera image as input and output a confidence level that the object to be detected is represented in that window. Alternatively, the detection unit 231 may detect an object region by performing template matching between a template representing the object to be detected and an image.

[0039] Furthermore, the detection unit 231 detects an object in front of the vehicle 10 based on the reflected wave information output by the LiDAR sensor 3. Based on the position of the object in the camera image, the detection unit 231 determines the orientation of the object relative to the vehicle 10, and based on this orientation and the reflected wave information output by the LiDAR sensor 3, it determines the distance between the object and the vehicle 10. Based on the current position of the vehicle 10 and the distance and orientation to the other object relative to the vehicle 10, the detection unit 231 estimates the position of the other object, for example, expressed in a world coordinate system. If no object is detected in front of the vehicle 10 (step S101-No), the series of processes ends.

[0040] On the other hand, if an object is detected in front of the vehicle 10 (step S101-Yes), the detection unit 231 detects the speed of the object (step S102). Here, the detection unit 231 detects the speed of the object regardless of the type of object.

[0041] The detection unit 231 tracks the object detected in the latest camera image by associating it with objects detected in past images, following a tracking process based on optical flow. The detection unit 231 then determines the trajectory of other objects being tracked based on the position of the object in the latest image in the world coordinate system, based on the position of the object in the latest image, based on the position of the object in past images. The detection unit 231 estimates the speed of the object relative to the vehicle 10 based on the change in the position of the other objects over time. The detection unit 231 determines the speed of the object relative to the ground based on the estimated speed of the object relative to the vehicle 10 and the speed of the vehicle 10. Furthermore, the detection unit 231 identifies the lane in which the object is traveling based on the lane markings represented in the map information and the position of the object. For example, the detection unit 231 determines that the object is traveling in a lane identified by two adjacent lane markings located on either side of the horizontal center position of the object. The detection unit 231 stores object detection information in the memory 22, which includes information indicating the type of detected object, information indicating its position, speed, acceleration, and lane of travel.

[0042] Next, the determination unit 232 determines whether the object's speed is below a predetermined reference speed (step S103). The determination unit 232 determines whether the object's speed is below a reference speed in order to determine whether the object is stationary. The reference speed can be, for example, 5 km / hour. If the object is light, such as a plastic bag, it may move due to wind even if it is stationary.

[0043] If the object's speed is below the reference speed (step S103-Yes), the determination unit 232 determines, based on camera images taken before the object was detected, whether or not there are other vehicles that are presumed to have passed over the object (step S104).

[0044] The determination unit 232 determines whether or not another vehicle has been detected based on the time when the object was detected and the object detection information detected based on the camera image taken a predetermined time earlier (for example, 1 to 5 minutes) from that time.

[0045] If another vehicle is detected, the determination unit 232 determines whether the travel path of this other vehicle overlaps with the position of the object. If the travel path of the other vehicle overlaps with the position of the object, the determination unit 232 determines that there is another vehicle that is presumed to have passed over the object. On the other hand, if the travel path of the other vehicle does not overlap with the position of the object, or if no other vehicle is detected, the determination unit 232 determines that there is no other vehicle that is presumed to have passed over the object.

[0046] If there is another vehicle that is presumed to have passed over the object (step S104-Yes), the determination unit 232 determines, based on camera images from a predetermined period prior to the time the object was detected, whether the other vehicle is the first vehicle with a narrow distance between its body and the road (step S105).

[0047] Based on the information indicating the type of other vehicle included in the object detection information, the determination unit 232 determines that the other vehicle is the second vehicle if it is identified as a vehicle with a cargo bed or as a truck, and determines that the other vehicle is the first vehicle if it is not the second vehicle.

[0048] The first vehicle includes passenger cars and buses and other vehicles with low ground clearance. When the distance between another vehicle's body and the road is narrow, passing over an obstacle will affect the vehicle's movement. Therefore, if an object is an obstacle, other vehicles will not drive over it. On the other hand, if the object is a manhole, puddle, or plastic bag, passing over these will not affect the vehicle's movement. Therefore, other vehicles will pass over manholes, puddles, or plastic bags.

[0049] The second category of vehicles includes trucks with high ground clearance. Because trucks have a large distance between their body and the road, passing over obstacles may not affect the truck's movement. Therefore, trucks may pass over obstacles even if they are present. In addition, since trucks carry cargo in their cargo beds, the cargo may fall onto the road and be detected as a stationary object in front of vehicle 10.

[0050] Trucks have a greater height than passenger cars, but buses also have a greater height because the distance between the vehicle and the road is narrower. Therefore, it is not appropriate to use only vehicle height for determination. For this reason, the determination unit 232 may add the height of other vehicles as a determination criterion. For example, if another vehicle has a cargo bed and its height is above a predetermined threshold, or if it is identified as a truck and its height is above a predetermined threshold, the other vehicle may be determined to be the second vehicle. The height of the other vehicle can be estimated, for example, based on the number of pixels in the height direction of the other vehicle in the camera image and the distance from vehicle 10 to the other vehicle.

[0051] If the other vehicle is the first vehicle (step S105-Yes), the determination unit 232 determines that the object does not affect the movement of vehicle 10 (step S106) and terminates the series of processes. Since the other vehicle, which has a low ground clearance, is passing over the object, it is presumed that the object is an object that does not affect the movement of vehicle 10, such as a manhole, puddle, or plastic bag. The determination unit 232 does not notify the automatic control device 12 of the object detection information.

[0052] On the other hand, if the other vehicle is not the first vehicle (step S105-No), the determination unit 232 determines that the other vehicle is the second vehicle, whose distance from the vehicle body to the road is greater than that of the first vehicle (step S107). The determination unit 232 then determines that the object will affect the movement of vehicle 10 (step S108). In other words, since the object may be an obstacle that will affect the movement of vehicle 10, vehicle 10 must take the object into consideration when driving.

[0053] Next, the determination unit 232 notifies the automatic control device 12 of object detection information, which includes information indicating the type of detected object, its position, speed, acceleration, and lane (step S109), and ends the series of processes.

[0054] The automatic control unit 12 drives the vehicle 10 based on object detection information. For example, the automatic control unit 12 drives the vehicle 10 to avoid contact with an object. If the automatic control unit 12 determines that contact with an object cannot be avoided, it stops the vehicle 10. Alternatively, if the automatic control unit 12 determines that contact with an object cannot be avoided, it may transfer control of the vehicle 10 from the automatic control unit 12 to the driver.

[0055] Furthermore, if the object's speed is not below a predetermined reference speed (step S103-No), and if there are no other vehicles that are presumed to have passed over the object (step S104-No), the process proceeds to step S108. If the object's speed is faster than the reference speed, the object may be a moving object such as a bicycle (a moving object is an example of an obstacle). Also, if there are no other vehicles, there is no information as to whether the object affects the movement of other vehicles, so the object may be an obstacle that affects the movement of vehicle 10.

[0056] As described in detail above, the object detection device of this embodiment, the object detection device 11, can accurately determine whether an object detected in front of a vehicle will affect the vehicle's movement, based on other vehicles that are presumed to have passed over the object.

[0057] In this disclosure, the object detection devices of the embodiments described above may be modified as appropriate without departing from the spirit of this disclosure. Furthermore, the technical scope of this disclosure is not limited to those embodiments, but extends to the inventions described in the claims and their equivalents. [Explanation of Symbols]

[0058] 2 cameras 3 LiDAR sensors 10 vehicles 11. Object detection device 21 Communication Interface 22 memory 23 processors 231 Detection unit 232 Judgment section 12 Automatic control system 13. In-vehicle network

Claims

[Claim 1] When an object traveling at a speed below a predetermined standard speed is detected in front of the vehicle, a first determination unit determines whether or not there is another vehicle that is presumed to have passed over the object, based on environmental information representing the environment in front of the vehicle prior to the time the object was detected. If the first determination unit determines that another vehicle is present, the second determination unit determines, based on environmental information for a predetermined period prior to the time the object was detected, whether the other vehicle is a first vehicle with a narrow distance between its body and the road, or a second vehicle with a wider distance between its body and the road than the first vehicle. If the second determination unit determines that the other vehicle is the first vehicle, the third determination unit determines that the object does not affect the driving of the own vehicle, and if the object's speed is faster than the standard speed, or there is no other vehicle, or the other vehicle is the second vehicle, the third determination unit determines that the object affects the driving of the own vehicle. An object determination device characterized by having the following features.

Images