Obstacle detection system, moving object, program, and obstacle detection method

Abstract

To perform obstacle detection, when a moving object is stopped, in blind spots such as areas outside the observation range of the sensor or occlusions caused by the housing of the moving object.SOLUTION: An obstacle detection system for detecting obstacles, comprises: a three-dimensional distance measuring sensor capable of measuring distances in three dimensions to detect the distance and height direction to a road surface on which a moving object is traveling; and an obstacle detection unit that detects obstacles on the road surface when the moving object is stopped. The obstacle detection unit includes height history data generation means for generating height history data supplemented with height information from the road surface that is held until the moving object stops, the height information being based on the amount of movement of the moving object during a predetermined period until the moving object stops.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] The present invention relates to an obstacle detection system, a moving object, a program, and an obstacle detection method. [Background technology]

[0002] Traditionally, safety technology has been essential for traveling devices (mobile bodies) operating outdoors to detect obstacles and stop or avoid them in order to enable autonomous traveling. This safety technology uses a variety of sensors, including RGBD cameras, stereo cameras, radar, and ultrasonic sensors. In recent years, sensors used in safety technology have often been LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), which emits laser light and measures the distance to an object and the shape of the object based on information from the reflected light, and in particular 2D-LiDAR, which can detect obstacles on a two-dimensional plane.

[0003] Patent Document 1 discloses a technology in which one-dimensional laser scanning is performed K times by a distance measuring device installed on a moving body, and all laser scannings from the a-th scan (Ka scans) to the K-th scan are consecutively detected as concave obstacles. Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the conventional technology, when the moving body is stopped, there is a problem in that it is not possible to detect obstacles outside the observation range of the sensor or in blind spots such as occlusion by the housing of the moving body.

[0005] The present invention has been made in consideration of the above, and aims to enable obstacle detection in blind spots such as outside the observation range of the sensor or occlusion by the casing of the moving body when the moving body is stopped. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the present invention provides an obstacle detection system for detecting obstacles, comprising: a three-dimensional ranging sensor capable of measuring distances in three dimensions to detect the distance and height direction to a road surface on which a moving body is traveling; and an obstacle detection unit that detects obstacles on the road surface when the moving body is stopped, wherein the obstacle detection unit comprises a height history data generation means that generates height history data supplemented with height information from the road surface that is held until the moving body stops, the height information being based on the amount of movement of the moving body during a predetermined period until the moving body stops. [Effects of the Invention]

[0007] According to the present invention, when a moving body is stopped, it is possible to detect an obstacle outside the observation range of the sensor or in a blind spot such as occlusion by the housing of the moving body. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an external perspective view showing an example of the configuration of a traveling device as an obstacle detection system according to a first embodiment. [Figure 2] FIG. 2 is a side view showing an example of the configuration of the traveling device. [Figure 3] FIG. 3 is a plan view showing the traveling device as viewed from above. [Figure 4] FIG. 4 is a block diagram illustrating an example of a hardware configuration of the traveling device. [Figure 5] FIG. 5 is a block diagram showing a functional configuration related to autonomous traveling of the traveling device. [Figure 6] FIG. 6 is a block diagram showing a functional configuration related to speed control of the steering control unit. [Figure 7] FIG. 7 is a diagram for explaining an outline of the curbstone detection process. [Figure 8] FIG. 8 is a diagram for explaining an outline of the groove detection process. [Figure 9] FIG. 9 is a diagram showing an example of a result of the detection process of a road surface obstacle. [Figure 10] FIG. 10 is a flowchart showing an example of a procedure for detecting a road surface obstacle in the traveling device. [Figure 11] FIG. 11 is a flowchart illustrating an example of a procedure for the point cloud data saving process. [Figure 12] FIG. 12 is a flowchart illustrating an example of a procedure for height map generation processing. [Figure 13] FIG. 13 is a diagram showing an outline of the noise removal process. [Figure 14] FIG. 14 is a diagram illustrating an example of a result of the noise removal process. [Figure 15] FIG. 15 is a diagram illustrating an example of a height map. [Figure 16] FIG. 16 is a diagram for explaining an outline of the blind spot handling process for obstacle detection. [Figure 17] FIG. 17 is a flowchart showing an example of a procedure for processing that is always executed as an algorithm. [Figure 18] FIG. 18 is a flowchart illustrating an example of a procedure for generating a height history map. [Figure 19] FIG. 19 is a flowchart illustrating an example of the procedure for the moving object region removal process. [Figure 20] FIG. 20 is a flowchart illustrating an example of a procedure for height history map clear service processing in client-server processing. [Figure 21] FIG. 21 is a sequence diagram showing an example of calling the height history map clear service process. [Figure 22] FIG. 22 is a sequence diagram showing another example of calling the height history map clear service process. [Figure 23] FIG. 23 is a flowchart illustrating an example of a procedure for generating a height obstacle map. [Figure 24] FIG. 24 is a flowchart illustrating an example of a procedure for a differential obstacle map generation process. [Figure 25]FIG. 25 is a flowchart illustrating an example of a procedure for generating a groove obstacle labeling map. [Figure 26] FIG. 26 is a flowchart illustrating an example of a procedure for obstacle map generation processing. [Figure 27] FIG. 27 is a diagram for explaining an outline of the obstacle map addition process. [Figure 28] FIG. 28 is a diagram illustrating an example of a result of applying the obstacle map addition process. [Figure 29] FIG. 29 is a diagram illustrating an automobile as an obstacle detection system according to the second embodiment. [Figure 30] FIG. 30 is a diagram illustrating a configuration of an obstacle detection system according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] (First embodiment) DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of an obstacle detection system, a moving object, a program, and an obstacle detection method will be described in detail with reference to the accompanying drawings.

[0010] (Example of running device configuration) 1 is an external perspective view showing an example of the configuration of a traveling device 1 as an obstacle detection system according to a first embodiment. The traveling device 1 is a mobile body that travels on uneven ground and is intended for use in facility inspections in factories and the like.

[0011] In this specification, the Y direction refers to the width direction of the traveling device 1. The X direction refers to the direction of travel of the traveling device 1, and the Z direction refers to the height direction of the traveling device 1. The coordinates represented by these X, Y, and Z coordinates are sometimes called robot coordinates. In other words, the robot coordinates are coordinates centered on the traveling device 1.

[0012] The traveling device 1 serving as an obstacle detection system has track-type traveling bodies 11a and 11b and a main body 10.

[0013] The track-type traveling bodies 11a and 11b are units that serve as driving members of the traveling device 1. The track-type traveling bodies 11a and 11b are crawler-type traveling bodies that use metal or rubber belts.

[0014] A tracked vehicle has a larger contact area than a vehicle that runs on tires, such as an automobile, and can run stably even in environments with poor footing. Also, while a vehicle that runs on tires needs space to turn when turning, a traveling device equipped with a tracked vehicle can perform so-called pivot turns, allowing it to turn smoothly even in a limited space.

[0015] Here, "spin turn" refers to turning on the spot around the center of the vehicle body by rotating the left and right tracks in opposite directions at a constant speed. This type of turning method is also called a spin turn.

[0016] The two track-type running bodies 11a, 11b are installed with the main body 10 sandwiched between them, in a state in which the running device 1 can run. The number of track-type running bodies is not limited to two, and may be three or more. For example, the running device 1 may be installed in a state in which the running device 1 can run, such as by arranging three track-type running bodies in parallel in three rows. Also, for example, the running device 1 may be installed with four track-type running bodies arranged front to back, left to right like automobile tires.

[0017] The tracked vehicle 11 (11a, 11b) has a triangular shape. With a triangular tracked vehicle 11, for example, when there are restrictions on the front and rear size, it is possible to increase the ground contact area within the limited front and rear size. This improves stability during running, as described above. On the other hand, so-called tank-type tracks, in which the upper side (drive wheel side) is longer than the lower side (rolling wheel side), have a smaller overall ground contact area and become unstable when there are restrictions on the front and rear size. In this way, the tracked vehicle 11 is effective when improving the running performance of a relatively small running device 1.

[0018] The main body 10 is a support that supports the track-type traveling bodies 11a and 11b in a state in which they can travel, and also a control device that controls the driving of the traveling device 1. The main body 10 is also equipped with a battery (not shown) that supplies power to drive the track-type traveling bodies 11a and 11b.

[0019] The main body 10 of the traveling device 1 is equipped with a power button 12, a start button 13, an emergency stop button 14, a status indicator lamp 15, and bumpers 16 and 17. The main body 10 of the traveling device 1 also has, at its upper part, a sensor installation stand 18, via poles 19, on which a distance measuring sensor 112 for detecting horizontal directions and a distance measuring sensor 113 for detecting diagonal directions are installed. The sensor installation stand 18 is supported on the main body 10 by four poles 19.

[0020] The power button 12 is an operating means that a person near the traveling device 1 presses to turn the power on or off for the traveling device 1. The start button 13 is an operating means that a person near the traveling device 1 presses to start the two track-type traveling bodies 11a, 11b. The emergency stop button 14 is an operating means that a person near the traveling device 1 presses to stop the traveling device 1 while it is running.

[0021] The status indicator lamp 15 is a notification means for notifying the status of the traveling device 1. For example, when the status of the traveling device 1 changes, such as when the remaining battery power is low, the status indicator lamp 15 lights up to notify people around the traveling device 1 of the change in status. The status indicator lamp 15 also lights up when there is a risk of an abnormality occurring, such as when the presence of an obstacle that hinders the traveling device 1 from traveling is detected.

[0022] 1 shows an example in which the traveling device 1 is provided with one status indicator lamp 15, but the number of status indicator lamps 15 may be two or more. Also, the notification means may be configured to notify the status of the traveling device 1 not only by the status indicator lamp 15 but also by a warning sound emitted from a speaker or the like.

[0023] Bumpers 16 and 17 are provided on the front and rear surfaces of main body 10, respectively, to protect main body 10 of traveling device 1 and crawler-type traveling bodies 11a and 11b.

[0024] The traveling device 1 is provided with a horizontal direction detection distance measuring sensor 112 on a sensor installation stand 18 at a height of 0.5 m, with the direction of travel of the main body 10 facing forward. The horizontal direction detection distance measuring sensor 112 is a 3D-LiDAR (Light Detection And Ranging) three-dimensional distance measuring sensor capable of measuring distances in three dimensions.

[0025] The 3D-LiDAR used is a non-repetitive scanning type with a 360-degree field of view that can measure a cone range of 70.4 degrees from the center of the sensor detection surface and can measure distances up to 90 m. The non-repetitive scanning method scans like drawing a flower while slightly shifting the phase in the horizontal and vertical directions, and is characterized by the fact that the point cloud coverage rate within the measurement range increases over time.

[0026] The distance measurement sensor 112 shines laser light onto an object such as a road obstacle, and measures the distance to the object and the direction in which the object is located based on the measurement result of the time it takes for the light to hit the object and bounce back, and obtains the measurement result as data.

[0027] Additionally, the traveling device 1 is provided with a distance measuring sensor 113 for detecting oblique directions on the sensor installation stand 18, with the traveling direction of the main body 10 being the front. This installation position is considered to enable detection of the traveling road surface up to just before the bumper 16 on the front of the traveling device 1. The distance measuring sensor 113 for detecting oblique directions is a three-dimensional distance measuring sensor using a 3D-LiDAR method, similar to the distance measuring sensor 112, that is capable of measuring distances in three dimensions. The traveling road surface is the road surface around the traveling device. Furthermore, the traveling road surface is not limited to outdoors, but also includes indoors. Furthermore, the traveling road surface is not limited to flat road surfaces, and includes, for example, road surfaces with grooves or steps, and uneven road surfaces.

[0028] Here, FIG. 2 is a side view showing an example of the configuration of the traveling device 1, and FIG. 3 is a plan view showing the traveling device 1 as viewed from above.

[0029] The distance measuring sensor 113 for detecting diagonal directions is attached so that the angle of irradiation of the laser light is a predetermined depression angle with respect to the horizontal road surface. The traveling device 1 is inverted and equipped with the distance measuring sensor 113 for detecting diagonal directions at a height of 0.8 m with the traveling direction of the main body 10 facing forward, at a depression angle of 52 degrees and an elevation angle of 7 degrees. This installation position is designed to enable detection of the road surface and height direction up to just before the bumper 16 on the front of the traveling device 1.

[0030] Note that at the installation position of the ranging sensor 113 for diagonal direction detection, there are blind spots at the feet and in the vertical direction due to the measurement range of the 3D-LiDAR. Furthermore, at the installation position of the ranging sensor 113 for diagonal direction detection, there are blind spots due to the fact that the ranging sensor 113 is installed on the traveling device 1, and part of the field of view is blocked by the four poles 19 supporting the sensor installation stand 18 on which the ranging sensor 113 is installed. The range of the blind spots is as shown in Figure 3, and the blind spot from the front bumper 16 is 0.325 m.

[0031] The distance measurement sensor 113 shines laser light onto an object such as a road obstacle, and measures the distance to the object and the direction in which the object is located based on the measurement result of the time it takes for the light to hit the object and bounce back, and obtains the measurement result as data.

[0032] The vertical measurement range has a depression angle of 52 degrees and an elevation angle of 7 degrees, as shown in Figure 2. On the other hand, the horizontal measurement range is 360 degrees, as mentioned above.

[0033] The installation position of the distance measuring sensor 113 has an optimum value depending on the width and length of the tracks of the tracked vehicles 11a and 11b, and also on the size, width, depth, and height of the object to be detected.

[0034] (Example of hardware configuration for running device) Next, an example of the hardware configuration of the traveling device 1 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the hardware configuration of the traveling device 1.

[0035] As shown in FIG. 4, the traveling device 1 has a CPU (Central Processing Unit) 101, memory 102, auxiliary storage device 103, camera 111, ranging sensors 112, 113, satellite positioning system 114, IMU 115, battery 121, motor drivers 122a, 122b, traveling motors 132a, 132b, brake drivers 123a, 123b, braking motors 133a, 133b, power SW 141, start SW 142, and emergency stop SW 143.

[0036] The CPU 101 controls the entire traveling device 1. The CPU 101 functions as an obstacle detection unit 155 (see FIG. 5) by operating in accordance with a traveling program stored in the memory 102. The CPU 101 travels using an obstacle map generated by a map generation unit by operating in accordance with the traveling program stored in the memory 102. The memory 102 is a temporary storage area for the CPU 101 to execute the traveling program, etc. The auxiliary storage device 103 stores the traveling program, etc. executed by the CPU 101.

[0037] The running program executed by the running device 1 is provided as a file in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disc).

[0038] The traveling program executed by the traveling device 1 may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. The traveling program executed by the traveling device 1 may be provided or distributed via a network such as the Internet. The traveling program executed by the traveling device 1 may be provided by being pre-installed in a ROM or the like.

[0039] The motor drivers 122 (122a, 122b) are drivers of the travel motors 132 (132a, 132b) provided on the two track-type traveling bodies 11a, 11b, respectively. The brake drivers 123 (123a, 123b) are drivers of the brake motors 133 (133a, 133b) provided on the two track-type traveling bodies 11a, 11b, respectively. The motor driver 122 and the brake driver 123 receive commands from the CPU 101 and control the travel motor 132 and the brake motor 133, respectively.

[0040] The power SW 141 is a switch that turns on or off the power supply to the traveling device 1. The power SW 141 operates in conjunction with pressing the above-mentioned power button 12 (see FIG. 1). The start SW 142 is a switch that starts the two track-type traveling bodies 11a, 11b. The start SW 142 operates in conjunction with pressing the above-mentioned start button 13 (see FIG. 1). The emergency stop SW 143 is a switch that brings the two track-type traveling bodies 10a, 10b to an emergency stop. The emergency stop SW 143 operates in conjunction with pressing the above-mentioned emergency stop button 14 (see FIG. 1).

[0041] The camera 111 includes a spherical camera, a stereo camera, an infrared camera, etc. The distance measurement sensor 112 is a 3D-LiDAR sensor for horizontal direction detection as described above. The distance measurement sensor 113 is a 3D-LiDAR sensor for diagonal direction detection as described above.

[0042] The satellite positioning system 114 receives radio waves from satellites and measures the position of the traveling device 1 on the Earth based on the reception results. The satellite positioning system 114 uses an RTK (Real Time Kinematic)-GNSS (Global Navigation Satellite System). In position estimation using RTK-GNSS, when a high-precision positioning solution (fix solution) is obtained in RTK-GNSS positioning, an accuracy of several centimeters can be obtained.

[0043] The satellite positioning system 114 is equipped with two antennas for GNSS reception. When there is a discrepancy of a certain distance or more between the position information obtained by the two antennas, the satellite positioning system 114 determines that the reliability (accuracy) of the position information has decreased.

[0044] The IMU 115 includes a three-axis acceleration sensor, a rotational angular velocity sensor, etc. The traveling device 1 detects the amount of tilt of the main body 10 of the traveling device 1 using the measurement data of the IMU 115, and corrects the difference in elevation with respect to the traveling road surface measured by the distance measurement sensor 113 based on the amount of tilt of the traveling device 1.

[0045] It is preferable that the IMU 115 is provided near the distance measurement sensor 113. When the distance measurement sensor 113 is provided on the main body 10 via an arm, it is preferable that the IMU 115 is provided on the arm. In this embodiment, the IMU 115 also serves as an IMU for self-position estimation that estimates the attitude of the traveling device 1.

[0046] This solves the problem that the height position detected by the distance measuring sensor 113 changes due to the main body 10 swaying back and forth when traveling on rough ground.

[0047] The CPU 101, which functions as the obstacle detection unit 155, etc., has judgment parameters that match the track-type traveling bodies 11a, 11b of the traveling device 1. This has the effect of enabling settings to be made so that the width of a groove that the track-type traveling bodies 11a, 11b of the traveling device 1 can traverse or the height of an obstacle that can be overcome are not detected as obstacles.

[0048] Next, the operation of the traveling device 1 for autonomous traveling will be described.

[0049] Fig. 5 is a block diagram showing a functional configuration related to the autonomous traveling of the traveling device 1. As shown in Fig. 5, the autonomous traveling of the traveling device 1 includes a function related to self-position estimation of the traveling device 1 using the satellite positioning system 114 and a function related to traveling control of the traveling device 1.

[0050] First, the self-position estimation of the traveling device 1 will be described.

[0051] As shown in FIG. 5, the traveling apparatus 1 includes an odometry calculation unit 151 and a self-position estimation unit 152 as functions related to the self-position estimation of the traveling apparatus 1.

[0052] As described above, high-precision position information obtained using the satellite positioning system 114 is used to estimate the self-position of the traveling device 1, but the position update period varies widely, ranging from about 200 ms to 1 s. Therefore, the odometry calculation unit 151 performs odometry calculations to estimate the self-position of the traveling device 1 from the rotation angle ω of the traveling motors 132a and 132b that drive the two tracked traveling bodies 11a and 11b, respectively. The rotation angle ω of the traveling motors 132a and 132b can be obtained from motor drive hall sensor pulses of the traveling motors 132a and 132b, or from external sensors such as an encoder or tachometer.

[0053] The self-position estimation unit 152 outputs the self-position estimation result (X, Y, θ). More specifically, the self-position estimation unit 152 corrects the odometry information calculated by the odometry calculation unit 151 using the position information obtained by the satellite positioning system 114 including RTK-GNSS and the attitude change measured by the IMU 115, thereby performing stable self-position estimation.

[0054] Incidentally, position information obtained by RTK-GNSS can suddenly deviate by several tens of centimeters to several meters due to the influence of reflecting objects, etc. Therefore, to prevent unstable operation due to such a sudden deviation in position estimation, the self-position estimation unit 152 corrects the odometry information calculated by the odometry calculation unit 151 as described above when the deviation in the position information acquired by the two antennas described above is not greater than a certain level. On the other hand, when the deviation in the position information acquired by the two antennas described above is greater than a certain level or when a high-precision positioning solution (fix solution) has not been obtained in the RTK-GNSS positioning, the self-position estimation unit 152 estimates the self-position using only the odometry calculation result by the odometry calculation unit 151.

[0055] Next, the speed control of the traveling device 1 will be described.

[0056] As shown in FIG. 5, the traveling device 1 includes a global planner unit 153, a local planner unit 154, an obstacle detection unit 155, and a steering control unit 156 as functions related to speed control of the traveling device 1.

[0057] The global planner unit 153 performs global planning (WP list) from the start point to the goal and outputs a sequence of reachable WAYPOINTs (WP(n)). A WAYPOINT is a set of points on the travel route of the traveling device 1. A WAYPOINT has information on direction and speed (Vmax) in addition to position (x, y, z).

[0058] The local planner unit 154 generates a route that actually connects the waypoints based on the information of the waypoints (WP(n)) obtained from the global planner unit 153, and calculates the target direction (WP vector).

[0059] The obstacle detection unit 155 detects obstacles based on the detection results of the distance measurement sensors 112 and 113. The obstacle detection unit 155 also includes a movement distance calculation unit 155a that calculates a predetermined movement distance of the traveling device 1. The obstacle detection unit 155 includes a height map generation unit 155b that generates a height map that holds height information from the traveling road surface based on the distance to the traveling road surface and the height direction measurement results obtained by the distance measurement sensor 113. The obstacle detection unit 155 also includes a height history map generation unit 155c that generates a height history map (height history data) that holds height information for blind spots that the distance measurement sensor 113 could not measure, taking into account the height information held in the height map for the movement distance of the traveling device 1. In other words, the height history map generation unit 155c functions as a height history data generation unit that generates height history data that is height information from the traveling road surface that was held until the traveling device 1 stopped, supplemented with height information based on the movement distance of the traveling device 1 during the predetermined period until the traveling device 1 stopped. Depending on the detected obstacle, the obstacle detection unit 155 outputs an avoidance direction (θ), a speed command (V), and a stop command (V) to the steering control unit 156. The obstacle detection unit 155 also generates an obstacle map of the road surface on which the traveling device 1 is traveling, using the results of the obstacle detection.

[0060] Steering control unit 156 calculates the speed and angular velocity of traveling device 1, and also calculates the velocities (VL, VR) of left and right crawler-type traveling bodies 11a, 11b for steering control. Steering control unit 156 passes the velocities (VL, VR) of left and right crawler-type traveling bodies 11a, 11b to serial IF (Interface) 157 and motor drivers 122a, 122b, and controls traveling motors 132a, 132b.

[0061] Here, a description will be given of the functional configuration related to the speed control of the steering control unit 156. Fig. 6 is a block diagram showing the functional configuration related to the speed control of the steering control unit 156.

[0062] The traveling device 1 is a crawler-type robot that independently drives left and right tracked traveling bodies 11a and 11b, and turns depending on the speed difference between the left and right tracked traveling bodies 11a and 11b. In order to travel stably on various road surfaces with different traveling loads, it is necessary to rotate the left and right tracked traveling bodies 11a and 11b at a specified speed regardless of the load.

[0063] For this reason, as shown in FIG. 6, the steering control unit 156 performs travel control of the traveling device 1 by dividing the loop into a major loop that controls the steering and a minor loop that controls the speed of each of the left and right track-type traveling bodies 11a, 11b.

[0064] As shown in FIG. 6, the steering control unit 156 includes a position PID (Proportional-Integral-Differential) 161, speed PIDs (Proportional-Integral-Differential) 162a and 162b, and speed calculation units 163a and 163b.

[0065] The position PID 161 calculates the operation amount of the left and right crawler-type traveling bodies 11a, 11b for steering control from the difference (position deviation) between the set value and the measured value.

[0066] The speed PIDs 162a and 162b calculate the change in the manipulated variable from the speed deviation.

[0067] The speed calculation units 163a and 163b estimate the motor speeds based on signals (motor pulses) from hall sensors attached to the travel motors 132a and 132b.

[0068] In the major loop, the steering control unit 156 calculates the difference (position deviation) between the target direction (θset) to the target point and the target position (Pset), using the odometry calculation result in the odometry calculation unit 151, which uses the motor pulses of the driving motors 132a and 132b, the self-position estimation result corrected in the self-position estimation unit 152, which uses the position information obtained from the satellite positioning system 114 including RTK-GNSS and the attitude change (θgps) and velocity change (Vgps) measured by the IMU 115.

[0069] More specifically, the steering control unit 156 outputs the calculated difference (position deviation) to the position PID 161.

[0070] The position PID 161 calculates the operation amounts VLset, VRset (command speed V and rotation angle ω of the travel motors 132a, 132b) of the left and right crawler-type traveling bodies 11a, 11b for steering control from the difference (position deviation) between the set value and the measured value.

[0071] The traveling device 1 can achieve stable traveling by controlling the left and right crawler-type traveling bodies 11a, 11b in accordance with the operation amounts VLset, VRset from the major loop.

[0072] Additionally, in the minor loop, steering control unit 156 estimates motor speeds using speed calculation units 163a and 163b based on signals (motor pulses) from Hall sensors attached to travel motors 132a and 132b. Steering control unit 156 then controls the motor applied voltage to travel motors 132a and 132b using speed PIDs 162a and 162b based on the difference (speed deviation) between the speed estimation results from speed calculation units 163a and 163b and the target speed, thereby performing speed feedback control. The reason for feedback controlling speed in this manner using the minor loop is to suppress speed fluctuations due to disturbance torque on travel motors 132a and 132b.

[0073] (Example of obstacle detection process for running equipment) Next, the detection of road surface obstacles on the road surface on which the traveling device 1 is traveling will be described, which is part of the control process for the autonomous traveling of the traveling device 1.

[0074] First, an overview of how the traveling device 1 detects road surface obstacles on the traveling road surface will be described. The traveling device 1 acquires detailed road surface data by performing vibration correction using plane detection on data obtained from the ranging sensor 113, which is a 3D-LiDAR installed facing diagonally downward, and then superimposing the data over time. The obtained vibration-corrected detailed road surface data is converted into a height map in which maximum heights as height information are stored in cells of a two-dimensional map so that the relationship between front, back, left, right, and diagonal heights can be grasped. An obstacle is then determined from the height difference in this height map. The height map is a two-dimensional map that records height information for identifying the height of the traveling road surface on which the traveling device 1, which is a traveling body, is traveling.

[0075] FIG. 7 is a diagram illustrating an overview of the curbstone detection process. The traveling device 1 accumulates raw 3D-LiDAR data obtained from the distance measurement sensor 113 shown in FIG. 7(a) for two seconds, and then generates a height map shown in FIG. 7(b). The traveling device 1 uses the height map data to determine whether an object is a curbstone based on the difference in height and gentleness of the slope. Specifically, the obstacle detection unit 155 of the traveling device 1 detects an object as a curbstone if the difference in height from the ground in the height map of FIG. 7(b) is equal to or greater than a threshold (height obstacle map) and if the differential value with respect to a nearby cell in the two-dimensional map is equal to or greater than a certain value (differential obstacle map).

[0076] FIG. 8 is a diagram illustrating an overview of the groove detection process. The traveling device 1 accumulates 3DLiDAR raw data obtained from the distance measurement sensor 113 shown in FIG. 8(a) for two seconds, and then generates a height map shown in FIG. 8(b). The obstacle detection unit 155 uses the height map data to cluster cells in the two-dimensional map whose height difference from the ground is less than a threshold and determine whether they are grooves. That is, the obstacle detection unit 155 of the traveling device 1 clusters cells in the two-dimensional map whose height difference from the ground is less than a threshold, and if any of the cells in the two-dimensional map of the cluster has a differential value with respect to neighboring cells that is greater than a certain value, the cluster is detected as a groove. If there is even one high edge information in a groove candidate cluster, the entire groove candidate cluster is determined to be a groove.

[0077] This makes it possible to detect airborne objects as obstacles in addition to road surface obstacles such as curbs and grooves, and also enables detection from the front or at steep approach angles. Basically, whether an object is an obstacle is determined from the created height map, but there are separate dedicated processes for obstacles with convex features such as curbs (hereafter referred to as curb obstacles) and obstacles with concave features such as grooves (hereafter referred to as groove obstacles).

[0078] Figure 9 shows an example of the results of the road obstacle detection process. Figure 9(a) shows an example of the curb detection process result, and Figure 9(b) shows an example of the groove detection process result. Figure 9(a) shows the curb detection result at an approach angle of 45 degrees, and Figure 9(b) shows the groove detection result from the front. The black cells in the two-dimensional map indicate road obstacles.

[0079] By performing processing for each and integrating the processing results, it is possible to detect both bumpy and uneven road obstacles. Below, we will explain the detailed flow of the obstacle detection process using a flowchart.

[0080] FIG. 10 is a flowchart showing an example of a procedure for detecting a road obstacle in the traveling device 1.

[0081] 10, the obstacle detection unit 155 performs obstacle detection by sequentially performing a point cloud data storage process (step S1), a height map generation process (step S2), a blind spot response process (step S3), a height obstacle map generation process (step S4), a differential obstacle map generation process (step S5), a groove obstacle labeling map generation process (step S6), and an obstacle map generation process (step S7). Each process will be described in detail below.

[0082] First, the point cloud data saving process (step S1) will be described. Fig. 11 is a flowchart showing an example of the procedure of the point cloud data saving process.

[0083] As shown in FIG. 11, the obstacle detection unit 155 first starts scanning by the distance measurement sensor 113 for diagonal detection (step S11).

[0084] Next, the obstacle detection unit 155 converts the acquired point cloud data of the distance measurement sensor 113 into robot coordinates of the traveling device 1, and generates road surface point cloud data, which is traveling road surface data (step S12). Based on the data thus obtained from the three-dimensional distance measurement sensor, the obstacle detection unit 155 generates traveling road surface data, which is three-dimensional data based on the traveling device 1 on the traveling road surface.

[0085] Next, the obstacle detection unit 155 uses the odometry information to move the road surface point cloud data that has already been saved (step S13). Note that, if the traveling device 1 is not moving, there may be no odometry information.

[0086] Next, the obstacle detection unit 155 performs plane detection using the acquired road surface point cloud data, and calculates the inclination of the ground surface from the plane detection result (step S14). Specifically, the obstacle detection unit 155 extracts a certain rectangular range (vertical width x: 1.0 m, horizontal width y: 1.4 m) from the acquired road surface point cloud data, and performs plane detection using that data. The obstacle detection unit 155 calculates the pitch angle from the detected plane, and outputs it as the inclination of the ground surface.

[0087] Next, the obstacle detection unit 155 corrects the inclination of the acquired road surface point cloud data using the calculated inclination of the ground (step S15).

[0088] Next, the obstacle detection unit 155 removes the road surface point cloud data that is within the rectangle of the traveling device 1 from the acquired road surface point cloud data that has been tilt-corrected, in order to exclude sensor data that has hit the traveling device 1 itself (step S16).

[0089] Next, the obstacle detection unit 155 saves the acquired road surface point cloud data after the tilt correction (step S17). The obstacle detection unit 155 stores the road surface point cloud data in the saved road surface point cloud data, but only saves the most recent 2 seconds' worth of data, and deletes any data older than that.

[0090] By carrying out the above procedure, it is possible to correct the forward and backward shaking of the traveling device 1 caused by traveling on rough terrain, and obtain the corrected traveling road surface data, which is detailed road surface point cloud data.

[0091] Next, the height map generation process (step S2) will be described. Fig. 12 is a flowchart showing an example of the procedure for the height map generation process. The obstacle detection unit 155 includes a height map generation means 155b that generates a height map that holds height information from the road surface, based on the distance to the road surface and the height direction measurement results obtained by the distance measurement sensor 113. As shown in Fig. 12, the height map generation process performs processing in the following order using the stored road surface point cloud data generated as described above.

[0092] 12, the obstacle detection unit 155 (height map generation means 155b) first extracts road surface point cloud data of the detection range (Pass Through Filter) (step S21). The detection range is a rectangle with a vertical width x of 2.0 m when viewed from the front of the bumper and a horizontal width y of 2.8 m from the center of the robot.

[0093] Next, the obstacle detection unit 155 generates voxel grid data using the road surface point cloud data within the detection range (step S22). The resolution is set to 2.5 cm.

[0094] Next, the obstacle detection unit 155 performs noise removal processing (Radius Outlier Filter) (step S23). Here, Fig. 13 is a diagram showing an overview of the noise removal processing. Specifically, as shown in Fig. 13, if there are less than n other points within a radius dm from the target point, for example, the obstacle detection unit 155 removes those points (indicated by a cross in Fig. 13).

[0095] Next, the obstacle detection unit 155 generates a height map using the voxel grid data from which noise has been removed (step S24). The resolution is set to 5.0 cm for the plane (xy) resolution and 2.5 cm for the height (z) resolution. As an example, the z value of the point cloud data with the largest z value among the point clouds included in each voxel is stored as the height value of that voxel. Then, a map is generated in which the z value of the voxel with the largest z value among multiple voxels with a certain xy value is stored as the height value of the position indicated by that xy value, and the maximum height from the road surface is stored on a two-dimensional map to form a height map.

[0096] By performing the above processing, a noise-removed height map can be obtained. The effect of this height map is that it reduces the processing load (speeds up processing) by reducing the number of points, and it makes it possible to traverse street gutters (gratings) by creating a larger height map.

[0097] When traveling outdoors, gutters must be detected as road surface obstacles because there is a risk of the traveling device 1 falling through them. However, in some locations, mesh-like gutters (gratings) are installed to allow the traveling device 1 to traverse these areas. Since areas with gratings are traversable by the traveling device 1, they should be detected as drivable areas rather than as road surface obstacles. Because the gratings are mesh-like, if the point cloud is examined closely, both flat areas and areas detected as grooves will be present, raising the risk of incorrectly detecting the grooves and preventing the vehicle from traversing these areas. Therefore, by converting the 2D map into a height map with a somewhat coarser cell resolution, localized detection is avoided, preventing the gratings from being incorrectly detected as grooves and allowing them to be detected as part of the traveling road surface. In this embodiment, the resolution of the height map is set to 5.0 cm, taking into account the size of the tracks of the traveling device 1.

[0098] Here, the details of the noise removal process in step S23 will be described.

[0099] Fig. 14 is a diagram illustrating an example of the result of the noise removal process. The 3D-LiDAR, which is the distance measurement sensor 113, may erroneously detect raindrops in rainy weather. When the amount of rainfall increases, scattered point cloud data is erroneously detected in the air, even though there are no obstacles in the air, as shown in Fig. 14(a). Therefore, if the height map conversion process is performed as is, there is a risk that the raindrops will be detected as obstacles, and the vehicle will stop or avoid obstacles even though there are no obstacles in the area.

[0100] Therefore, in this embodiment, by performing a noise removal process as shown in FIG. 13, raindrops are removed as noise, and it is expected that the obstacle detection function will be able to be performed even in rainy weather. The noise removal process as shown in FIG. 13 is a process that removes points that are less than n points within a radius dm of the target point. The data shown in FIG. 14(b) is the result of performing the noise removal process of FIG. 13 with d = 0.06 m and n = 2, and compared to the data shown in FIG. 14(a), raindrops in the air have been removed. This has the effect of removing false detection data due to raindrops in rainy weather using this noise removal, making it possible to operate even in rainy weather (allowing the obstacle detection function to be performed).

[0101] Here, Fig. 15 is a diagram showing an example of a height map. Fig. 15 is an example of a height map obtained by the traveling device 1 configured as shown in Fig. 1 through the above process. Fig. 15(a) shows an example of a height map obtained by the traveling device 1 when stopped, and Fig. 15(b) shows an example of a height map obtained by the traveling device 1 when traveling.

[0102] As shown in FIG. 15, the distance measurement sensor 113 of this embodiment can acquire point cloud data covering the entire 360-degree periphery, thereby expanding the applicable range for obstacle detection (see FIG. 3). Because the distance measurement sensor 113 can acquire point cloud data covering the entire periphery of the traveling apparatus 1, it can detect obstacles on the sides or behind. Therefore, the traveling apparatus 1 of this embodiment can safely avoid or stop obstacles on the sides or behind by taking into account their trajectories. In other words, because the traveling apparatus 1 of this embodiment can detect obstacles on the sides or behind, it can perform avoidance operations taking into account obstacles on the sides or behind, thereby increasing safety.

[0103] As described above, the height map is a two-dimensional map that represents the relationship between front, back, left, right, and diagonal heights, and stores the maximum height of data obtained from the ranging sensor 113, which is a 3D-LiDAR, in cells of a certain resolution. In the example shown in Fig. 15, data obtained from the ranging sensor 113 that has been accumulated for two seconds is used to generate the height map.

[0104] The area in Figure 15(a) where no colored cells exist is the blind spot area also shown in Figure 3, and this blind spot exists when the traveling device 1 is stopped. On the other hand, while the traveling device 1 is traveling, it accumulates data from the distance measurement sensor 113 for two seconds while traveling, so this blind spot area almost disappears, as shown in Figure 15(b).

[0105] As shown in Figure 15(b), the traveling device 1 can detect obstacles in blind spots while traveling. However, when the traveling device 1 stops after traveling, it returns to the state shown in Figure 15(a), causing blind spots to appear again and making it impossible to detect obstacles in the blind spots. If an obstacle exists in the blind spots in this state, it will not be able to detect the obstacle in the blind spots, and the traveling device 1 will mistakenly believe that there is no obstacle and continue traveling, creating the risk of a collision.

[0106] To prevent such a situation, the traveling device 1 of this embodiment stores data spatially rather than temporally, thereby dealing with blind spots in obstacle detection. Below, we will explain how the traveling device 1 of this embodiment deals with blind spots in obstacle detection.

[0107] FIG. 16 is a diagram illustrating an overview of the blind spot handling process for obstacle detection. As shown in FIG. 16, the traveling device 1 of this embodiment basically shifts previously observed sensor data by the amount of movement of the traveling device 1, and stores the observation history of sensor data only within blind spots. The traveling device 1 of this embodiment then performs obstacle detection processing using a height history map, which will be described later, including the observation history of the stored sensor data. This allows the obstacle detection processing to be performed in blind spot areas while taking into account the surrounding conditions, thereby improving detection accuracy.

[0108] If only stationary objects are to be detected, blind spot processing for obstacle detection is not necessary. However, in outdoor environments, moving objects such as people and cars also exist. If this moving object moves back and forth within a blind spot, an afterimage of the moving object may remain in the blind spot area, and performing detection processing in this state may result in erroneous detection. To prevent this, the traveling device 1 of this embodiment recognizes and tracks the moving object, erases historical data only in the area of ​​the moving object, and uses real-time tracking data to deal with the afterimage of the moving object.

[0109] In addition, in this embodiment, in case an afterimage does remain, a client-server system is provided that can call a process to clear the height data history itself from an external device. By calling the process to clear the height data history itself, the client-server system can take measures against afterimages in blind spots from an external device.

[0110] As described above, blind spot handling for obstacle detection can be performed while taking into consideration erroneous detection in the blind spot area. The traveling device 1 of this embodiment performs blind spot handling processing for obstacle detection (step S3 shown in FIG. 10) by providing the following two processing flows. 1. Processing that is always executed as an algorithm 2. Processing of a client-server system that can be called from external devices

[0111] The specific flows of the above two processes will be explained below using flowcharts.

[0112] FIG. 17 is a flowchart showing an example of a procedure for processing that is always executed as an algorithm.

[0113] As shown in Figure 17, the obstacle detection unit 155 sequentially performs a height history map generation process (step S101) and a moving object area removal process (step S102) using three items: a height map generated from observation data from the ranging sensor 113, which is a 3D-LiDAR; a height history map that stores height data; and moving object recognition data.

[0114] First, the height history map generation process (step S101) will be described. The obstacle detection unit 155 includes a height history map generation means 155c that generates a height history map that holds height information even for blind spots where the distance measurement sensor 113 was unable to measure, taking into account the height information held in the height map in relation to the amount of movement of the traveling device 1.

[0115] 18 is a flowchart showing an example of the procedure for the height history map generation process. As shown in Fig. 18, the obstacle detection unit 155 (height history map generation means 155c) performs the height history map generation process using the height map and height history map data in the following order.

[0116] As shown in Fig. 18, the obstacle detection unit 155 uses the odometry information (the amount of movement of the traveling device 1) calculated by the movement amount calculation unit 155a to move the height history map data by the amount of odometry (step S1011). Note that the obstacle detection unit 155 excludes data that has gone outside the range of the area. However, since there is no data in the height history map initially, no movement of the data occurs. Movement occurs from the second time onwards when there is data.

[0117] Next, the obstacle detection unit 155 updates the height history map by storing the current height map in the height history map (step S1012). More specifically, when height data exists in each cell of the height map, the obstacle detection unit 155 performs an update process by storing the current height data in the same cell number of the height history map. By performing such an update process, the obstacle detection unit 155 can obtain a height history map that holds height data in blind spot areas.

[0118] Next, the moving object region removal process (step S102) will be described.

[0119] Fig. 19 is a flowchart showing an example of the procedure for the moving object region removal process. As shown in Fig. 19, the obstacle detection unit 155 performs the moving object region removal process on the height history map in the following order to remove height data of the moving object region based on the recognition and tracking results of the moving object, thereby removing the influence of the blind spot region due to the moving object.

[0120] 19, the obstacle detection unit 155 performs a moving object region removal process (step S1021) using the height history map generated in the height history map generation process (step S101) and the moving object recognition data. More specifically, the obstacle detection unit 155 compares the rectangular region of the recognized moving object with the height history map, and removes height information from the rectangular region of the moving object.

[0121] That is, by performing the moving object region removal process, the obstacle detection unit 155 can remove residual image obstacles in blind spot regions caused by moving objects from the height history map, thereby suppressing the influence of moving objects on the height history map.

[0122] Next, processing of a client-server system that can be called from an external device will be described. In this embodiment, a client-server system is assumed in which an information processing device (external device) functions as a client for a traveling device 1 having a remote control system.

[0123] FIG. 20 is a flowchart illustrating an example of a procedure for height history map clear service processing in client-server processing.

[0124] As shown in FIG. 20, the obstacle detection unit 155 performs the height history map processing in the client-server processing using the height history map data in the following order.

[0125] As shown in FIG. 20, the obstacle detection unit 155 clears (deletes) all data of the height history map in accordance with an instruction from a client (information processing device) (step S201).

[0126] Next, the obstacle detection unit 155 stores the processing completion status as a response result to the client (information processing device) (step S202).

[0127] As mentioned above, in case an afterimage does remain, by calling a process to clear the height data history itself from an external device, it is possible to execute the process to clear the height history map data from the external device, rather than within the algorithm. This provides an operational measure to prevent obstacles in blind spots from remaining as afterimages. An example of this service will be explained below.

[0128] Fig. 21 is a sequence diagram showing an example of calling the height history map clear service process. The example shown in Fig. 21 shows an example in which the user calls the height history map clear service process at any timing.

[0129] In the example shown in Fig. 21, operation is considered using a traveling device 1 equipped with a remote control system. The traveling device 1 can be sent various commands from the user via the remote control system app (hereinafter referred to as the remote control app). In this case, as shown in Fig. 21, the user can also call the height history map clear service via the remote control app.

[0130] As shown in FIG. 21, when the user presses the height history map clear button displayed on the remote control application (step S301), the remote control application sends a height history map clear command to the traveling device 1 (step S302).

[0131] When the height history map clear service is called from the outside (remote control application), the traveling device 1 executes height history map clear service processing (step S303). After completing the height history map clear service processing, the traveling device 1 sends a completion notification to the remote control application (step S304). This allows the user to know the completion notification of the height history map clear service processing.

[0132] With this configuration, if an afterimage does remain, the user can execute a process to clear the height history map data from an external device (remote control application) at any time of their choosing.

[0133] Consider, for example, a scenario in which a user is manually driving the traveling device 1 by remote control. During manual driving, assume that the blind spot obstacle detection of this embodiment prevents the traveling device 1 from colliding with an obstacle and safely stops the traveling device 1. After the moving object is removed and safety is ensured, if an afterimage of the moving object remains in the blind spot, the traveling device may stop due to the afterimage of the object even though there is no actual obstacle. In this case, once the user confirms that there are no obstacles in the blind spot, the user can call this service via the remote control app to clear the height history map and resume manual driving with the blind spot area safely secured.

[0134] Next, consider operation in a system where the traveling device 1 also has a remote control app and the remote control app can issue instructions for autonomous traveling of the traveling device 1. When an instruction related to autonomous traveling is issued by the user, it is also possible to automatically send a command for the height history map clear service before sending the command to the traveling device 1.

[0135] Fig. 22 is a sequence diagram showing another example of calling the height history map clear service process. The example shown in Fig. 22 shows an example in which the height history map clear service process is called automatically by the system. In the example shown in Fig. 22, consider a series of flows from the start of autonomous driving to its suspension, resumption, and end.

[0136] 22, first, the user presses the autonomous driving start button on the remote control app (Web app) (step S401), and the remote control app sends an autonomous driving start command to the traveling device 1 (step S402). As a result, the traveling device 1 starts autonomous driving and sends an autonomous driving start notification to the remote control app (step S403).

[0137] Next, the traveling device 1 performs blind spot obstacle detection of this embodiment (step S405) while moving to the destination (step S404).

[0138] When the traveling device 1 stops for a certain period of time due to a road obstacle such as a curb or a gutter while traveling, the traveling device 1 itself suspends its autonomous traveling (step S406) and sends a suspension notice to the remote control application (step S407). This allows the user to know that the autonomous traveling of the traveling device 1 has been suspended.

[0139] Thereafter, when the user presses the manual driving command button of the remote control application (step S408), the remote control application sends a manual driving command to the traveling device 1 (step S409). As a result, the user manually drives the traveling device 1 to a safe location where there are no obstacles in the blind spot of the traveling device 1.

[0140] When the traveling device 1 moves to a safe location where there are no obstacles in the blind spot, the user presses the autonomous traveling resume button on the remote control app (step S410), and the remote control app sends a height history map clear command to the traveling device 1 before sending an autonomous traveling resume command (step S411).

[0141] When the traveling device 1 receives the height history map clear command, it executes height history map clear service processing (step S412). After completing the height history map clear service processing, the traveling device 1 sends a completion notification to the remote control application (step S413).

[0142] When the remote control application receives the completion notification, it sends an autonomous driving resume command to the traveling device 1, causing the traveling device 1 to resume autonomous driving.

[0143] Next, the traveling device 1 performs the blind spot obstacle detection of this embodiment (step S416) while moving to the destination (step S415).

[0144] When the traveling device 1 reaches the destination (step S417), it sends an autonomous traveling end notification to the remote control application (step S418). This allows the user to know that the traveling device 1 has reached the destination and that the autonomous traveling has ended.

[0145] As in the above scene, by sending a height history map clear command before resuming autonomous driving after the user has moved the traveling device 1 to a location where there are no obstacles in the blind spot, even if an obstacle in the form of an afterimage of the moving object remains in the blind spot area, the height history map can be cleared, and autonomous driving can be resumed with the blind spot area secured. In this case, the system automatically calls the service, which is effective in that it can be operated without the user being aware of it.

[0146] The above is the blind spot handling process for obstacle detection in the traveling apparatus 1 of this embodiment (step S3 shown in FIG. 10).

[0147] Next, the height obstacle map generation process (step S4) will be described. Fig. 23 is a flowchart showing an example of the procedure of the height obstacle map generation process. As shown in Fig. 23, the height obstacle map generation process uses the height history map to perform the following steps:

[0148] As shown in Fig. 23, the obstacle detection unit 155 sets the threshold to ±6 cm and generates a height obstacle map (step S41). Specifically, the obstacle detection unit 155 sets a value of 100 to cells in the two-dimensional map that are higher than the threshold of +6 cm, as normal obstacles, sets a value of 99 to cells in the two-dimensional map that are lower than -6 cm, as groove obstacles, sets -1 to cells for which no point cloud data exists, so-called unknown obstacle cells, and sets the rest to 0 as drivable cells. By performing this process, a height obstacle map can be obtained that is divided into four types: normal obstacles, groove obstacles, unknown obstacles, and drivable area cells, and conditional branching of processing can be performed depending on the value of the cells in the two-dimensional map.

[0149] Next, the differential obstacle map generation process (step S5) will be described. Fig. 24 is a flowchart showing an example of the procedure of the differential obstacle map generation process. As shown in Fig. 24, the differential obstacle map generation process uses the height history map to perform the following steps:

[0150] As shown in FIG. 24, the obstacle detection unit 155 first corrects unknown obstacles in the height history map (step S51). The height history map contains so-called unknown obstacles for which no point cloud data exists due to factors such as obstacle occlusion. For such unknown obstacles, the obstacle detection unit 155 stores the average height of the four neighboring two-dimensional map cells in the unknown obstacle cell to perform correction. If all four neighboring cells are unknown obstacles, the obstacle detection unit 155 stores a height value of 0.

[0151] Next, the obstacle detection unit 155 performs median filtering on the corrected height history map (step S52).

[0152] Next, the obstacle detection unit 155 performs Laplacian filtering on the corrected height history map that has been subjected to the median filtering, to generate a differential map (step S53).

[0153] Next, the obstacle detection unit 155 sets a threshold value of ±0.20 using the differential map generated by the Laplacian filter and generates a differential obstacle map (step S54). Cells greater than the threshold value of +0.20 or less than the threshold value of −0.20 are set to 100 as differential obstacles in the two-dimensional map, and other cells are set to 0 as passable cells.

[0154] By performing the above processing, a differential obstacle map can be obtained that is divided into two types: differential obstacles and drivable area cells. A differential map is a map that numerically represents the difference in height between neighboring cells in the height history map, and the larger the difference in height between neighboring cells, the larger the value of the cell. For example, the value increases in places where the difference in height between neighboring cells suddenly increases, such as a curb, in other words, where there is a sharp edge. This allows you to know the gradualness of the height history map.

[0155] The correction process in step S51 is for implementing a median filter, and the median filter process in step S52 is for smoothing out variations in height in the height history map due to noise from the distance measurement sensor 113.

[0156] A differential map like this is classified by setting a threshold and is called a differential obstacle map. Cells above the threshold indicate areas with sharp edges, such as curbs, and are potential obstacle locations.

[0157] Next, the groove obstacle labeling map generation process (step S6) will be described. Fig. 25 is a flowchart showing an example of the procedure of the groove obstacle labeling map generation process. As shown in Fig. 25, the groove obstacle labeling map generation process uses height obstacle map data to perform the following steps.

[0158] 25, the obstacle detection unit 155 first executes the expansion process of the height obstacle map (step S61). More specifically, if there is a groove obstacle in the vicinity of eight surroundings of an unknown obstacle cell in the height obstacle map, the obstacle detection unit 155 converts the target unknown obstacle into a groove obstacle.

[0159] Next, the obstacle detection unit 155 generates a groove obstacle labeling map using the expanded height obstacle map (step S62). More specifically, the obstacle detection unit 155 performs labeling in the eight neighborhoods of the groove obstacle.

[0160] By performing the above processing, it is possible to label (cluster) groove obstacles in the height obstacle map.

[0161] The expansion process in step S61 is performed to expand unknown obstacles to groove obstacles, under the assumption that unknown obstacles existing near a groove obstacle cell are the same groove obstacle, since unknown obstacles often occur due to occlusion when viewing grooves using 3D-LiDAR.

[0162] Next, the obstacle map generation process (step S7) will be described. Fig. 26 is a flowchart showing an example of the procedure for the obstacle map generation process. As shown in Fig. 26, the obstacle map generation process uses the three pieces of data obtained in the previous processes: height obstacle map, differential obstacle map, and groove obstacle labeling map data, in the following order:

[0163] 26, the obstacle detection unit 155 integrates the height obstacle map data, differential obstacle map data, and groove obstacle labeling map data to generate an obstacle map (step S71). More specifically, the obstacle detection unit 155 looks at the same cell in both the differential obstacle map and the height obstacle map, and if the same cell in both maps contains a value of 99 (groove obstacle) or 100 (curb obstacle), it performs an AND operation and sets the cell as an obstacle cell. At this time, in the case of a groove obstacle, all label data that became an obstacle cell is converted to an obstacle cell.

[0164] Next, the obstacle detection unit 155 executes an obstacle map addition process to detect groove obstacles from differences in height near target points on the height history map, in order to make it easier to find the groove obstacles (step S72).

[0165] Here, Fig. 27 is a diagram for explaining an overview of the obstacle map addition process, and Fig. 28 is a diagram for explaining an example of the result of applying the obstacle map addition process. The obstacle detection unit 155 calculates the average height of three cells on the left and right of the target cell in the lateral direction (Y direction) of the traveling device 1. Using the calculated average height, the obstacle detection unit 155 compares the average height of three cells in front and behind in the traveling direction (X direction) of the traveling device 1. For example, as shown in Fig. 27, the obstacle detection unit 155 determines that a cell is an obstacle cell if (maximum - minimum) of the three cells is greater than 6 cm.

[0166] In the case of a groove obstacle in front, the detection sensitivity is low with only the above-mentioned processing, as shown in Figure 28(a), so to enable detection with higher sensitivity, the obstacle map addition processing shown in step S72 is also executed. By also executing the obstacle map addition processing, it becomes possible to detect groove obstacles in front with high sensitivity, as shown in Figure 28(b).

[0167] By carrying out the above processing, an obstacle map can be obtained that includes both groove obstacles and curb obstacles.

[0168] The process in step S71 determines obstacles based on height difference and gentleness, and integrates both the height obstacle map, which corresponds to height difference, and the differential obstacle map, which corresponds to gentleness. Also, since groove obstacles cannot necessarily be identified by ANDing height difference and gentleness, labeling (cluster) map data for groove obstacles is used, and if there is even one edge cell in the label map that can be ANDed with the edge cell of the differential obstacle map, all cells with that label are determined to be groove obstacles.

[0169] As described above, according to this embodiment, when a moving body is stopped, the height information is supplemented with height information based on the amount of movement of the moving body during a predetermined period before the moving body comes to a stop, so that height information can be retained even for blind spots that the three-dimensional ranging sensor was unable to measure, and therefore obstacles can be avoided by detecting obstacles outside the observation range of the sensor or in blind spots occluded by the housing of the moving body. It goes without saying that "when a moving body is stopped" includes not only a state in which the moving body is stopped, but also a state in which the moving body starts moving again from a stopped state.

[0170] (Second embodiment) Next, a second embodiment will be described.

[0171] In the first embodiment, an example was described in which a traveling device 1, which is a mobile body having a small crawler-type traveling body that moves on rough ground and is intended for use in facility inspections in factories and the like, is applied as an obstacle detection system. In the second embodiment, an autonomously driven automobile is applied as an obstacle detection system. In the following description of the second embodiment, descriptions of the same parts as in the first embodiment will be omitted, and only differences from the first embodiment will be described.

[0172] Conventional autonomous driving vehicles often use on-board stereo cameras to capture images of the front and rear of the vehicle and detect obstacles, etc. However, conventional autonomous driving vehicles do not support the detection of obstacles, etc. on the sides of the vehicle. While it is possible to install multiple sensors on the sides of the vehicle to eliminate blind spots, this approach comes with the problem of increased costs.

[0173] Fig. 29 is a diagram showing an automobile 200 as an obstacle detection system according to the second embodiment. As shown in Fig. 29, automobile 200, which is a moving body, is provided with sensor installation stand 18 on roof 201 via pole 19, on which distance measuring sensor 112 for horizontal detection and distance measuring sensor 113 for diagonal detection are installed.

[0174] The automobile 200 as an obstacle detection system of this embodiment has the effect of being able to detect obstacles in blind spots such as outside the observation range of the sensor or occlusion by the housing of the moving body when the automobile 200 is stopped during automatic driving of the automobile 200.

[0175] It goes without saying that the obstacle detection system is not limited to application to autonomously driven automobiles, but can also be applied to autonomously driven mobile bodies such as various vehicles, aircraft, ships, robots, etc.

[0176] (Third embodiment) Next, a third embodiment will be described.

[0177] In the first embodiment, measures against blind spots are taken using only one traveling device 1, which is a mobile body having a crawler-type traveling body. In the third embodiment, accumulated data such as a height map and a height history map is stored in a cloud server and shared with other mobile bodies (such as crawler robots and automobiles). In the following description of the third embodiment, descriptions of the same parts as in the first and second embodiments will be omitted, and differences from the first and second embodiments will be described.

[0178] 30 is a diagram showing the configuration of an obstacle detection system 300 according to the third embodiment. As shown in Fig. 30, the obstacle detection system 300 according to this embodiment includes a plurality of moving bodies (such as the traveling device 1 and the automobile 200) 301, and a cloud server 303 connected to the plurality of moving bodies 301 via a network (such as the Internet) 302. The moving bodies 301 are, for example, the traveling device 1 according to the first embodiment or the automobile 200 according to the second embodiment.

[0179] The cloud server 303 stores accumulated data such as a height map based on road surface data obtained from the ranging sensor 113 of the mobile body 301, and a height history map that holds height information even for blind spots where the ranging sensor 113 was unable to measure distances.

[0180] For example, the obstacle detection unit 155 of one mobile object 301 successively stores accumulated data, such as a height map based on road surface data obtained from the distance measurement sensor 113 and a height history map that holds height information even for blind spots that the distance measurement sensor 113 was unable to measure, in the cloud server 303 in a shareable manner via the network 302, thereby constantly updating the accumulated data in the cloud server 303 to the latest state. The obstacle detection units 155 of other mobile objects 301 share the accumulated data stored in the cloud server 303 via the network 302.

[0181] Also, for example, the obstacle detection unit 155 of one mobile body 301 stores data of a blind spot area that could not be measured by the distance measurement sensor 113 in a shareable manner in the cloud server 303 via the network 302. The obstacle detection units 155 of other mobile bodies 301 share the data of the blind spot area stored in the cloud server 303 via the network 302, thereby complementing the blind spot area.

[0182] That is, the obstacle detection unit 155 of one mobile body 301 stores at least one of the height map, the height history map, and the data on the blind spot area in a shareable manner in the cloud server 303. Note that, for example, when the obstacle detection unit 155 of one mobile body 301 stores at least one of the height map and the height history map in addition to the data on the blind spot area in the cloud server 303 as accumulated data, the obstacle detection unit 155 of another mobile body 301 may acquire only the data on the blind spot area in order to reduce the network load on the network 302.

[0183] Each function of each embodiment described above can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the functions described above.

[0184] For example, aspects of the present invention are as follows. <1> An obstacle detection system for detecting an obstacle, comprising: a three-dimensional distance measuring sensor capable of measuring distances in three dimensions to detect the distance and height direction to a road surface on which a moving object is traveling; an obstacle detection unit that detects an obstacle on the road surface when the moving object is stopped; Equipped with The obstacle detection unit a height history data generating means for generating height history data supplemented with height information from the road surface that is held until the moving body stops, the height information being based on the amount of movement of the moving body during a predetermined period until the moving body stops; An obstacle detection system comprising: <2> the height history data generating means compares the rectangular area of ​​the recognized moving object with the height history data and removes height information from the rectangular area of ​​the moving object; Characterized by <1> 2. An obstacle detection system according to claim 1 . <3> the height history data generating means erases the height information stored in the height history data in response to an instruction from an external device; Characterized by <1> 2. An obstacle detection system according to claim 1 . <4> the measurement results by the three-dimensional distance measuring sensor are three-dimensional road surface data based on the moving object; the obstacle detection unit generates height map data that holds a maximum height from the traveling road surface based on the traveling road surface data as the height information, and provides the height map data to the height history data generation means; Characterized by <1> Or <3> 10. An obstacle detection system according to claim 9, wherein: <5> the obstacle detection unit generates the height map data based on the corrected road surface data obtained by performing vibration correction on the road surface data by plane detection and then superimposing the data over time; Characterized by <4> 2. An obstacle detection system according to claim 1 . <6> the three-dimensional distance measuring sensor is installed at a height and at an inclination angle that allows it to detect the road surface up to just before the leading end of the moving object in the traveling direction; Characterized by <4> 2. An obstacle detection system according to claim 1 . <7> The obstacle detection unit determining whether there is a height obstacle on the travel road surface based on the characteristics of the height difference and gentleness; determining whether a groove obstacle is present on the road surface based on a difference in height of the vicinity; Characterized by <4> 2. An obstacle detection system according to claim 1 . <8> the obstacle detection unit generates a voxel grid based on the result of the vibration correction to obtain the height map data generation result. Characterized by <5> 2. An obstacle detection system according to claim 1 . <9> the obstacle detection unit performs noise removal based on the point cloud distribution resulting from the vibration correction. Characterized by <5> 2. An obstacle detection system according to claim 1 . <10> the drive member of the moving body is a crawler-type traveling body having a predetermined contact length, The obstacle detection unit has a determination parameter that is tailored to the driving member. Characterized by <4> 2. An obstacle detection system according to claim 1 . <11> The three-dimensional distance measuring sensor can acquire the traveling road surface data over a 360-degree entire circumference. Characterized by <4> Or <10> 10. An obstacle detection system according to claim 9, wherein: <12> Equipped with a network-connected cloud server, the obstacle detection unit stores at least one of the height map data, the height history data, and data on blind spots that could not be measured by the three-dimensional ranging sensor in the cloud server in a shareable manner; Characterized by <4> 2. An obstacle detection system according to claim 1 . <13> the obstacle detection unit acquires only the data of the blind spot area when at least one of the height map data and the height history data is stored in the cloud server in addition to the data of the blind spot area; Characterized by <12> 2. An obstacle detection system according to claim 1 . <14> a driving member that travels on a travel surface; a three-dimensional distance measuring sensor capable of measuring distances in three dimensions to detect the distance and height direction to the road surface; an obstacle detection unit that detects an obstacle on the travel road surface when the drive member is stopped; Equipped with The obstacle detection unit a height history data generating means for generating height history data supplemented with height information from the road surface that is held until the drive member stops, the height information being based on the amount of movement of the drive member during a predetermined period until the drive member stops; A moving object characterized by: <15> Computer, When the moving object is stopped, the obstacle detection unit detects obstacles on the road surface, The obstacle detection unit a height history data generating means for generating height history data supplemented with height information from the road surface that is held until the moving body stops, the height information being based on the amount of movement of the moving body during a predetermined period until the moving body stops; program. <16> an obstacle detection step of detecting an obstacle on a road surface when the moving body is stopped; The obstacle detection step includes: a height history data generation step of generating height history data supplemented with height information from the traveling road surface that is held until the moving body stops, the height information being based on a movement amount of the moving body that has moved during a predetermined period until the moving body stops; Obstacle detection methods. [Explanation of symbols]

[0185] 1. Moving object and obstacle detection system 11a, 11b driving member 113 Three-dimensional distance sensor 155 Obstacle detection unit 155c Height history data generation means 200 Mobile object, obstacle detection system 300 Obstacle Detection System 301 Mobile [Prior art documents] [Patent documents]

[0186] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-075032

Claims

1. An obstacle detection system for detecting an obstacle, comprising: a three-dimensional distance measuring sensor capable of measuring distances in three dimensions to detect the distance and height direction to a road surface on which a moving object is traveling; an obstacle detection unit that detects an obstacle on the road surface when the moving object is stopped; Equipped with The obstacle detection unit a height history data generating means for generating height history data supplemented with height information from the road surface that is held until the moving body stops, the height information being based on the amount of movement of the moving body during a predetermined period until the moving body stops; An obstacle detection system comprising:

2. the height history data generating means compares the rectangular area of ​​the recognized moving object with the height history data and removes height information from the rectangular area of ​​the moving object; 2. The obstacle detection system of claim 1.

3. the height history data generating means erases the height information stored in the height history data in response to an instruction from an external device; 2. The obstacle detection system of claim 1.

4. the measurement results by the three-dimensional distance measuring sensor are three-dimensional road surface data based on the moving object; the obstacle detection unit generates height map data that holds a maximum height from the traveling road surface based on the traveling road surface data as the height information, and provides the height map data to the height history data generation means; 4. An obstacle detection system according to claim 1, wherein the obstacle detection system is a vehicle.

5. the obstacle detection unit generates the height map data based on the corrected road surface data obtained by performing vibration correction on the road surface data by plane detection and then superimposing the data over time; 5. The obstacle detection system according to claim 4.

6. the three-dimensional distance measuring sensor is installed at a height and at an inclination angle that allows it to detect the road surface up to just before the leading end of the moving object in the traveling direction; 5. The obstacle detection system according to claim 4.

7. The obstacle detection unit determining whether there is a height obstacle on the travel road surface based on the characteristics of the height difference and gentleness; determining whether a groove obstacle is present on the road surface based on a difference in height of the vicinity; 5. The obstacle detection system according to claim 4.

8. the obstacle detection unit generates a voxel grid based on the result of the vibration correction to obtain the height map data generation result.

6. The obstacle detection system according to claim 5.

9. the obstacle detection unit performs noise removal based on the point cloud distribution resulting from the vibration correction.

6. The obstacle detection system according to claim 5.

10. the drive member of the moving body is a crawler-type traveling body having a predetermined contact length, The obstacle detection unit has a determination parameter that is tailored to the driving member.

5. The obstacle detection system according to claim 4.

11. The three-dimensional distance measuring sensor can acquire the traveling road surface data over a 360-degree entire periphery.

5. The obstacle detection system according to claim 4.

12. Equipped with a network-connected cloud server, the obstacle detection unit stores at least one of the height map data, the height history data, and data on blind spots that could not be measured by the three-dimensional ranging sensor in the cloud server in a shareable manner; 5. The obstacle detection system according to claim 4.

13. the obstacle detection unit acquires only the data of the blind spot area when at least one of the height map data and the height history data is stored in the cloud server in addition to the data of the blind spot area; 13. The obstacle detection system of claim 12.

14. a driving member that travels on a travel surface; a three-dimensional distance measuring sensor capable of measuring distances in three dimensions to detect the distance and height direction to the road surface; an obstacle detection unit that detects an obstacle on the travel road surface when the drive member is stopped; Equipped with The obstacle detection unit a height history data generating means for generating height history data supplemented with height information from the road surface that is held until the drive member stops, the height information being based on the amount of movement of the drive member during a predetermined period until the drive member stops; A moving object characterized by:

15. Computer, When the moving object is stopped, the obstacle detection unit detects obstacles on the road surface, The obstacle detection unit a height history data generating means for generating height history data supplemented with height information from the road surface that is held until the moving body stops, the height information being based on the amount of movement of the moving body during a predetermined period until the moving body stops; program.

16. an obstacle detection step of detecting an obstacle on a road surface when the moving body is stopped; The obstacle detection step includes: a height history data generation step of generating height history data supplemented with height information from the traveling road surface that is held until the moving body stops, the height information being based on a movement amount of the moving body that has moved during a predetermined period until the moving body stops; Obstacle detection methods.

Images