Mobile object detection device and autonomous driving device

Abstract

To obtain a mobile object detection device capable of detecting both stationary and moving objects. [Solution] The system includes a first detection unit 14 that calculates the position and feature quantity of stationary feature points, which are feature points indicating a stationary object 40 among the first feature points indicating an object in the surrounding environment from observation data of the surrounding environment; a difference extraction unit 17 that extracts the difference between the observation data and the stationary feature points; a second feature point extraction unit 18 that extracts second feature points indicating an object in the surrounding environment from the observation data after difference extraction; a second feature point tracking unit 19 that tracks the second feature points based on multiple pieces of information about the second feature points input from the second feature point extraction unit 18 at different timings; and a second detection unit 20 that calculates the position and feature quantity of moving feature points, which are feature points indicating a moving object 50 among the second feature points, using its own position and orientation and the tracking result of the second feature points.

Description

Technical Field

[0007] ,

[0001] The present disclosure relates to a moving object detection device and an autonomous driving device for detecting a moving object.

Background Art

[0002] An autonomous driving device such as a robot needs to travel while avoiding surrounding objects, and thus includes an object detection device for detecting surrounding objects.

[0003] As a method for an object detection device to detect a target object, there are a stereo vision method and a SLAM (Simultaneous Localization and Mapping) method.

[0004] The stereo vision method is a technique that simultaneously observes a target using a plurality of sensors from different locations and calculates the position of the target from the observation results and the sensor position information. In the stereo vision method, the target position is calculated using the observation results of the target at a certain time.

[0005] On the other hand, as disclosed in Patent Document 1, in the SLAM method, self-position estimation and environmental map creation are simultaneously performed based on self-position information and information on surrounding stationary objects. In the SLAM method, since self-position estimation and environmental map construction are performed, it is not necessary to grasp the situation of the target area and acquire position information in advance, and, unlike the stereo vision method, it is not necessary to arrange sensors in advance. ​​​​​​​​​​​​​​​​​​​​​In an environment where other moving objects are present, an autonomous vehicle must navigate autonomously, avoiding not only stationary objects but also moving objects. To navigate autonomously while avoiding moving objects, the speed of those objects must be known.

[0008] In stereoscopic imaging, while it is theoretically possible to calculate the target's velocity if the distance between the sensor and the target is above a certain level and the sensor can observe the target's attitude with high accuracy, this is practically difficult. Therefore, stereoscopic object detection devices detect stationary and moving objects as objects without distinguishing between them. As a result, if an autonomous vehicle using a stereoscopic object detection device is operated autonomously in an environment where other moving objects are present, the autonomous vehicle may not be able to avoid the moving objects.

[0009] In SLAM methods such as the one disclosed in Patent Document 1, the positional relationship between the self-position and stationary objects is used for self-position estimation and environmental map creation, so information on moving objects is treated as noise and discarded. For this reason, SLAM methods such as the one disclosed in Patent Document 1 can detect stationary objects but cannot detect moving objects. Consequently, if an autonomous vehicle using a SLAM object detection device is operated autonomously in an environment where other moving objects are present, the autonomous vehicle may not be able to avoid the moving objects.

[0010] Thus, in order to enable autonomous driving devices to operate autonomously in environments where other moving objects are present, a moving object detection device is needed that can detect moving objects in addition to stationary objects.

[0011] This disclosure has been made in view of the above, and aims to provide a mobile object detection device capable of detecting stationary and moving objects. [Means for solving the problem]

[0012] To solve the above-mentioned problems and achieve the objective, the mobile object detection device according to this disclosure comprises: a sensor that observes the surrounding environment and measures the distance of self-movement; a first feature point extraction unit that extracts first feature points indicating objects in the surrounding environment from observation data of the surrounding environment; a first feature point tracking unit that tracks the first feature points based on a plurality of pieces of information about the first feature points input from the first feature point extraction unit at different timings; a self-position and posture estimation unit that estimates the self-position and self-posture using the distance of self-movement and the tracking result of the first feature points by the first feature point tracking unit; and a first detection unit that calculates the position and feature quantity of stationary feature points, which are feature points indicating stationary objects among the first feature points, based on the estimation results of the self-position and self-posture by the self-position and posture estimation unit and the first feature points extracted by the first feature point extraction unit. The moving object detection device includes a difference extraction unit that extracts the difference between observation result data and stationary object feature points; a second feature point extraction unit that extracts second feature points indicating objects in the surrounding environment from the observation result data after difference extraction; a second feature point tracking unit that tracks the second feature points based on multiple pieces of information about the second feature points input from the second feature point extraction unit at different timings; and a second detection unit that calculates the position and feature quantity of the moving object feature point, which is a feature point indicating a moving object among the second feature points, using the self-position and self-position calculated by the self-position and self-position estimation unit and the tracking result of the second feature point. [Effects of the Invention]

[0013] This disclosure provides the advantage of obtaining a mobile object detection device capable of detecting both stationary and moving objects. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows the configuration of an autonomous driving system using a mobile object detection device according to Embodiment 1. [Figure 2] This figure shows an example of a travel path for an autonomous driving device using a mobile object detection device according to Embodiment 1. [Figure 3] Diagram showing the configuration of the moving object detection device according to Embodiment 1. [Figure 4]This figure shows an example of the hardware configuration of the moving object detection device according to Embodiment 1. [Modes for carrying out the invention]

[0015] The mobile object detection device and autonomous driving device according to the embodiment will be described in detail below with reference to the drawings.

[0016] Embodiment 1. Figure 1 shows the configuration of an autonomous driving device using a moving object detection device according to Embodiment 1. The autonomous driving device 70 comprises a driving device 4 which is a device for performing operations such as movement, stopping, acceleration / deceleration and changing direction; a moving object detection device 1 which detects stationary objects 40 and moving objects 50; a driving path setting unit 2 which sets a driving path that avoids the stationary objects 40 and moving objects 50 detected by the moving object detection device 1; and a driving control unit 3 which controls the driving device 4 to drive along the driving path set by the driving path setting unit 2. The driving device 4 includes mechanisms for movement, stopping and changing direction, such as a prime mover, a transmission, a steering device and a braking device.

[0017] Figure 2 shows an example of a driving path for an autonomous driving device using a moving object detection device according to Embodiment 1. As shown in Figure 2, the autonomous driving device 70 equipped with the moving object detection device 1 detects stationary objects 40 and moving objects 50, and autonomously drives while avoiding the detected stationary objects 40 and moving objects 50. In detecting stationary objects 40 and moving objects 50, the moving object detection device 1 constructs a map based on the position information of the stationary objects 40 and the self-position estimation result, while also detecting the moving objects 50.

[0018] FIG. 3 is a diagram showing the configuration of the moving body detection device according to Embodiment 1. The moving body detection device 1 includes a noise removal unit 10, a first feature point extraction unit 11, a first feature point tracking unit 12, a self-position and orientation estimation unit 13, a first detection unit 14, a first result storage unit 15, a map construction unit 16, a difference extraction unit 17, a second feature point extraction unit 18, a second feature point tracking unit 19, a second detection unit 20, and a second result storage unit 21. In the following description, the difference extraction unit 17, the second feature point extraction unit 18, the second feature point tracking unit 19, the second detection unit 20, and the second result storage unit 21, which perform processes related to the detection of the moving body 50, are collectively referred to as the moving body detection unit 30.

[0019] The sensor 105 is a sensor that observes the surrounding environment of the moving body detection device 1, and is, for example, a visible sensor or an ultrasonic sensor. The sensor 105 outputs the observation result data of the surrounding environment to the noise removal unit 10. Further, the sensor 105 outputs the self-movement distance, which is the distance that the autonomous driving device 70 has moved by the traveling device 4, to the self-position and orientation estimation unit 13.

[0020] The noise removal unit 10 receives the observation result data of the surrounding environment from the sensor 105. In Embodiment 1, it is assumed that the observation result data is a frame image obtained by imaging the surrounding environment using a visible sensor, an ultrasonic sensor, or the like. Note that the observation result data may be the observation result by a distance measurement sensor instead of image information. The noise removal unit 10 performs threshold determination and correlation processing on the frame image to remove the noise included in the frame image.

[0021] The first feature point extraction unit 11 receives a denoised frame image as input. The first feature point extraction unit 11 calculates first feature points representing objects in the surrounding environment from the denoised frame image and outputs information about the first feature points to the first feature point tracking unit 12. The first feature points are, for example, a point cloud that forms the contour of an object in the surrounding environment, and include a point cloud that forms the contour of a stationary object 40 and a point cloud that forms the contour of a moving object 50. The first feature point tracking unit 12 tracks the first feature points by performing correlation processing or the like to compare the first feature points input from the first feature point extraction unit 11 with past first feature points, and outputs the tracking result to the self-position and attitude estimation unit 13. That is, the first feature point tracking unit 12 calculates the displacement of the first feature points along the time series based on multiple pieces of information about the first feature points input from the first feature point extraction unit 11 at different timings, and outputs the calculation result to the self-position and attitude estimation unit 13. The self-position and attitude estimation unit 13 uses the self-movement distance measured by the sensor 105 and the tracking results of the first feature point tracking unit 12 to estimate the self-position, which is the current position of the mobile object detection device 1, and the self-attitude, which is the attitude of the mobile object detection device 1, and outputs the estimation results of the self-position and self-attitude to the first detection unit 14. Based on the estimation results of the self-position and self-attitude and the extraction results of the first feature points, the first detection unit 14 calculates the position and feature quantity of the stationary feature points, which are the feature points that form the contour of the stationary object 40, from the first feature points, taking into account the self-position and self-attitude, and outputs the calculation results to the first result storage unit 15. The map construction unit 16 obtains information on the positions of the stationary feature points, which are the point cloud that forms the contour of the stationary object 40, from the first result storage unit 15 and creates a map.

[0022] The differential extraction unit 17 receives the frame image after noise removal, similar to the first feature point extraction unit 11. Also, the differential extraction unit 17 acquires information on the positions of the static object feature points at the same time as the frame image from the first result storage unit 15. The differential extraction unit 17 extracts the difference between the frame image after noise removal and the information on the positions of the static object feature points acquired from the first result storage unit 15, and outputs the frame image after differential extraction to the second feature point extraction unit 18. That is, the differential extraction unit 17 performs a process of removing the static object 40 from the frame image after noise removal. The second feature point extraction unit 18 performs feature point extraction again using the frame image after differential extraction, and outputs information on the second feature points to the second feature point tracking unit 19. The second feature points extracted by the second feature point extraction unit 18 are a point group forming the contour of the moving object 50. The second feature point tracking unit 19 collates the information on the second feature points input from the second feature point extraction unit 18 with the past second feature points by correlation processing or the like, performs tracking of the second feature points, and outputs the calculation result to the second detection unit 20. That is, the second feature point tracking unit 19 calculates the displacement along the time series of the second feature points based on a plurality of information on the second feature points input from the second feature point extraction unit 18 at different timings, and outputs the calculation result to the second detection unit 20. The feature points tracked by the second feature point tracking unit 19 are a point group forming the contour of the moving object 50. The second detection unit 20 calculates the positions and feature amounts of the moving object feature points forming the contour of the moving object 50 from among the second feature points based on the estimation results of the self-position and self-orientation and the extraction results of the second feature points, and outputs the calculation result to the second result storage unit 21. That is, the second detection unit 20 detects the moving object 50 by excluding the change in the second feature points due to self-movement, and outputs the detection result to the second result storage unit 21. The second result storage unit 21 stores the detection result of the moving object 50.

[0023] In this way, the moving object detection device 1 detects the moving object 50 by removing the information of the static object 40 from the objects included in the frame image, which is the observation result data of the surrounding environment, by a method equivalent to the conventional SLAM method.

[0024] Figure 4 shows an example of the hardware configuration of a mobile object detection device according to Embodiment 1. The mobile object detection device 1 comprises a computer 110, an input / output interface (IF) 103, a communication device 104, and a sensor 105. The computer 110 may be one unit or multiple units. The computer 110 comprises a processor 100, a memory 101, and an auxiliary storage device 102.

[0025] The computer 110, the input / output IF 103, the communication device 104, and the sensor 105 are appropriately connected via the signal line 106.

[0026] The processor 100 is an integrated circuit (IC) that performs arithmetic operations and controls the hardware of the computer 110. Specific examples of the processor 100 include a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit).

[0027] The processor 100 may be a single arithmetic processing unit, or it may be a group of arithmetic processing units that share the processing.

[0028] Memory 101 is typically a volatile storage device, specifically RAM (Random Access Memory). Memory 101 is also called main memory. Data stored in memory 101 is saved to auxiliary storage device 102 as needed.

[0029] The auxiliary storage device 102 is typically a non-volatile storage device, specifically a ROM (Read Only Memory), an HDD (Hard Disk Drive), or flash memory. Data stored in the auxiliary storage device 102 is loaded into memory 101 as needed.

[0030] The memory 101 and the auxiliary storage device 102 may be configured as a single unit.

[0031] Input / Output IF103 is a port to which input and output devices are connected. A specific example of an input / output IF103 is a USB (Universal Serial Bus) terminal. Specific examples of input devices connected to input / output IF103 include a keyboard and a mouse. A specific example of an output device connected to input / output IF103 is a display. Note that the mobile object detection device 1 may be configured without an input / output IF103.

[0032] The communication device 104 consists of a receiver and a transmitter. Specific examples of the communication device 104 include a communication chip or a NIC (Network Interface Card).

[0033] Each part of the mobile object detection device 1 may use the input / output IF 103 and the communication device 104 as appropriate when communicating with other devices.

[0034] The auxiliary storage device 102 stores the mobile object detection program. The mobile object detection program is a program that enables the computer 110 to implement the functions of each part of the mobile object detection device 1. The mobile object detection program is loaded into memory 101 and executed by the processor 100. The functions of each part of the mobile object detection device 1 are realized through software processing by the processor 100 executing the mobile object detection program.

[0035] When the processor 100 executes the motion detection program, observation data acquired by the sensor 105 and data calculated by the processor 100 executing the motion detection program are appropriately stored in the memory. Each part of the motion detection device 1 makes appropriate use of the memory. Specific examples of the memory include memory 101, auxiliary memory 102, registers within the processor 100, and at least one of the cache memory within the processor 100. Note that data and information can sometimes have the same meaning. The memory may also be an external memory independent of the computer 110.

[0036] The functions of memory 101 and auxiliary storage device 102 may be implemented by an external storage device independent of the computer 110.

[0037] The motion detection program may be recorded on a non-volatile recording medium readable by the computer 110. Specific examples of non-volatile recording media include optical discs and flash memory. The motion detection program may also be provided as a program product.

[0038] The hardware configuration of other devices that communicate with the mobile object detection device 1 according to Embodiment 1 may be the same as the hardware configuration of the mobile object detection device 1.

[0039] The moving object detection device 1 according to Embodiment 1 can detect stationary objects 40 and moving objects 50 by using information on moving objects 50 that would have been discarded in a typical SLAM method in the moving object detection unit 30, thus eliminating the need for special equipment to detect moving objects. Therefore, the autonomous driving device 70 equipped with the moving object detection device 1 according to Embodiment 1 can autonomously drive in an unknown environment, avoiding stationary objects 40 and moving objects 50 without using special equipment to detect moving objects 50.

[0040] The configurations shown in the above embodiments are merely examples of the content, and can be combined with other known technologies. It is also possible to omit or modify parts of the configuration without departing from the gist of the invention.

[0041] The various aspects of this disclosure are summarized below as an appendix.

[0042] (Note 1) A sensor that observes the surrounding environment and measures the distance traveled by itself, A first feature point extraction unit extracts first feature points indicating objects in the surrounding environment from the observation data of the surrounding environment, A first feature point tracking unit tracks the first feature point based on a plurality of pieces of information about the first feature point input from the first feature point extraction unit at different timings, A self-position and self-pose estimation unit that estimates the self-position and self-pose using the self-movement distance and the tracking result of the first feature point by the first feature point tracking unit, A first detection unit calculates the position and feature quantity of stationary feature points, which are feature points that indicate a stationary object, based on the self-position and self-position estimation results by the self-position and self-position estimation unit and the first feature points extracted by the first feature point extraction unit. A difference extraction unit that extracts the difference between the observation result data and the stationary feature points, A second feature point extraction unit extracts second feature points indicating objects in the surrounding environment from the observation result data after difference extraction, A second feature point tracking unit tracks the second feature point based on multiple pieces of information about the second feature point input from the second feature point extraction unit at different timings, A moving object detection device comprising a second detection unit that calculates the position and feature quantity of a moving object feature point, which is a feature point indicating a moving object among the second feature points, using the self-position and self-position calculated by the self-position and self-position estimation unit and the tracking result of the second feature point. (Note 2) The moving object detection device according to Appendix 1, characterized in that the observation result data is a frame image of the surrounding environment. (Note 3) A moving object detection device according to Appendix 1 or Appendix 2, characterized by comprising a map construction unit that creates a map of the surrounding environment based on the positions of the stationary object feature points. (Note 4) Traveling device and A moving object detection device described in any one of Appendix 1 to Appendix 3, A travel path setting unit sets a travel path that avoids the stationary body and the moving body, An autonomous driving device comprising: a driving control unit that controls the driving device to travel along a driving route set by the driving route setting unit. [Explanation of symbols]

[0043] 1 Mobile object detection device, 2 Driving path setting unit, 3 Driving control unit, 4 Driving device, 10 Noise reduction unit, 11 First feature point extraction unit, 12 First feature point tracking unit, 13 Self-position and attitude estimation unit, 14 First detection unit, 15 First result storage unit, 16 Map construction unit, 17 Difference extraction unit, 18 Second feature point extraction unit, 19 Second feature point tracking unit, 20 Second detection unit, 21 Second result storage unit, 30 Mobile object detection unit, 40 Stationary object, 50 Moving object, 70 Autonomous driving device, 100 Processor, 101 Memory, 102 Auxiliary storage device, 103 Input / Output IF, 104 Communication device, 105 Sensor, 106 Signal line, 110 Computer.

Claims

1. A sensor that observes the surrounding environment and measures the distance traveled by itself, A first feature point extraction unit extracts first feature points indicating objects in the surrounding environment from the observation data of the surrounding environment, A first feature point tracking unit tracks the first feature point based on a plurality of pieces of information about the first feature point input from the first feature point extraction unit at different timings, A self-position and self-pose estimation unit that estimates the self-position and self-pose using the self-movement distance and the tracking result of the first feature point by the first feature point tracking unit, A first detection unit calculates the position and feature quantity of a stationary feature point, which is a feature point representing a stationary object among the first feature points, based on the self-position and self-position estimation results by the self-position and self-position estimation unit and the first feature point extracted by the first feature point extraction unit. A difference extraction unit that extracts the difference between the aforementioned observation result data and the aforementioned stationary object feature points, A second feature point extraction unit extracts second feature points indicating objects in the surrounding environment from the observation result data after difference extraction, A second feature point tracking unit tracks the second feature point based on a plurality of pieces of information about the second feature point input from the second feature point extraction unit at different timings, A moving object detection device comprising a second detection unit that calculates the position and feature quantity of a moving object feature point, which is a feature point indicating a moving object among the second feature points, using the self-position and self-position calculated by the self-position and self-position estimation unit and the tracking result of the second feature point.

2. The moving object detection device according to claim 1, characterized in that the observation result data is a frame image obtained by imaging the surrounding environment.

3. The moving object detection device according to claim 1, further comprising a map construction unit that creates a map of the surrounding environment based on the positions of the stationary object feature points.

4. Traveling device and A moving object detection device according to any one of claims 1 to 3, A travel path setting unit sets a travel path that avoids the stationary body and the moving body, An autonomous driving device comprising: a driving control unit that controls the driving device to travel along a driving route set by the driving route setting unit.

Images