Self-discipline driving system of mine vehicle

Abstract

The purpose of the present invention is to provide a mine vehicle autonomous travel system that, when a non-detection region is generated within a region to be detected by an external detection sensor, can appropriately handle the cause of the non-detection region, and can suppress a decrease in the working efficiency of a mine vehicle. Therefore, the controller establishes a corresponding relation between the three-dimensional point cloud acquired by the external detection sensor and the point cloud picture, calculates the position change of the external detection sensor based on the measurement result of the vehicle position measurement sensor, and determines the position change of the external detection sensor under the condition that the external detection sensor fails to acquire the three-dimensional point cloud to be acquired. And a control unit that extracts a region of the point cloud as a non-detection region, determines the cause of the non-detection region on the basis of whether or not the position of the non-detection region in the point cloud changes in accordance with a change in the position of the vehicle, and determines the operation of the vehicle in accordance with the cause of the non-detection region.

Description

Technical Field

[0001] The invention relates to an autonomous driving system for a mining vehicle. Background Art

[0002] As a document that discloses the background technology in the technical field, there is, for example, Patent Document 1. Patent Document 1 discloses "an external recognition system, which has a camera mounted on a vehicle, analyzes the image of the camera, and recognizes the state of the periphery of the vehicle based on the analysis result of the image of the camera, and the external recognition system is characterized in that it has a dirt detection unit that detects whether dirt is attached to the camera, the dirt detection unit sets a line of at least a part of an object that does not change in the image of the camera even if the vehicle moves as at least one baseline, calculates an observation line corresponding to the baseline from the image of the camera, calculates the displacement of the observation line from the baseline, detects whether dirt is attached to the camera based on the displacement, and calculates the displacement of the observation line from the baseline by calculating the area of ​​the region between the observation line and the baseline".

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 6246014 Summary of the invention

[0006] In mining sites, for example, autonomous dump trucks and other work machines are used to improve productivity. For autonomous dump trucks, in addition to operating management using control information, each autonomous dump truck is also controlled as follows to ensure driving safety: the surrounding conditions during driving are always understood, obstacles and other vehicles are detected, and autonomous driving is controlled. In addition, in non-road environments such as mining sites, obstacles may not be detected due to mud splashing, road surface liquid, etc.

[0007] On the other hand, if LiDAR is used for obstacle detection, if there is liquid on the road, it will be affected by the light reflected by the liquid on the road. If there is dirt on the irradiation surface of the LiDAR, it will be affected by the dirt, and there may be a situation where a non-detection area is generated in the area that should be detected. If there is liquid on the road, it needs to be avoided, and if the irradiation surface is dirty, it needs to be cleaned. However, when a non-detection area is generated without distinguishing the cause for a certain non-detection, it is difficult to automatically determine the cause. In this case, it is necessary to perform on-site confirmation work based on the operator after stopping the dump truck every time, which reduces the work efficiency.

[0008] Patent document 1 discloses the following technology, which calculates the displacement of the brightness value on the observation line based on an object (such as a bumper) that does not change in the camera image even if the vehicle moves, and judges whether there is dirt attached to the camera based on the displacement. The premise for the establishment of this disclosed technology is that a part of the vehicle is always projected into the same position of the image and always drives on a paved road. On the other hand, in a mining environment, the surrounding environment of the vehicle changes all the time, so it is not necessarily appropriate to use parameters that change regularly, such as the displacement of the brightness value.

[0009] The present invention is made in view of the above-mentioned problems, and its purpose is to provide an autonomous driving system for a mining vehicle, which, when a non-detection area is generated in an area that should be detected by an external detection sensor, can appropriately deal with the cause of the non-detection area, thereby suppressing the reduction in the operating efficiency of the mining vehicle.

[0010] In order to achieve the above-mentioned purpose, the autonomous driving system of a mining vehicle of the present invention comprises: an external detection sensor for acquiring a three-dimensional point cloud of the surrounding environment of the vehicle; a position measurement sensor installed on the vehicle; and a controller to which the signals of the external detection sensor and the position measurement sensor are input, the controller comprising: a point cloud map generating unit for establishing a correspondence between the three-dimensional point cloud acquired by the external detection sensor and the point cloud map; a sensor position change calculating unit for calculating the position change of the external detection sensor based on the measurement result of the position measurement sensor; and a non-detection area detecting unit for extracting an area of ​​the point cloud map as a non-detection area when the external detection sensor fails to acquire the three-dimensional point cloud that should be acquired, the autonomous driving system of the mining vehicle is characterized in that the controller comprises: a non-detection area cause judging unit for judging the cause of the non-detection area based on whether the position of the non-detection area in the point cloud map changes according to the position change of the vehicle; and an action determining unit for determining the action of the vehicle according to the judgment result of the non-detection area cause judging unit.

[0011] According to the present invention thus constituted, when a non-detection area occurs in an area to be detected by the outside detection sensor, the mining vehicle can be caused to take appropriate actions according to the cause of the non-detection area, thereby suppressing a decrease in the working efficiency of the mining vehicle.

[0012] Effects of the Invention

[0013] According to the present invention, when a non-detection area occurs in an area to be detected by an outside detection sensor, appropriate measures can be taken against the cause of the non-detection area, thereby suppressing a decrease in the operating efficiency of a mining vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1This is a schematic diagram of a mining vehicle.

[0015] Figure 2 This is a diagram of the configuration of a driving control system installed in a mining vehicle.

[0016] Figure 3 It is the functional block diagram of the controller.

[0017] Figure 4 This is a diagram showing an example of a coordinate system.

[0018] Figure 5 This is a flowchart showing the processing of the mining vehicle autonomous driving system.

[0019] Figure 6 Detailed description of the process of detecting the non-detection area.

[0020] Figure 7 This is a diagram showing the detectable range of LiDAR.

[0021] Figure 8 Detailed description of the process for determining the cause of the non-detection area.

[0022] Fig. 9 Detailed description of the process of tracking the non-detectable area.

[0023] Fig.10 Detailed description of the process for determining the cause of the non-detectable area.

[0024] Fig.11 This is a diagram showing the concept of a process for determining the cause of a non-detectable area.

[0025] Fig.12 This is a diagram showing a specific example of a process of classifying non-detection cells and setting them as non-detection areas.

[0026] Fig.13 This is a diagram showing the data structure of the registration buffer.

[0027] Fig.14 This is a flowchart showing the details of a process for determining the behavior of a vehicle according to the cause of a non-detection area.

[0028] Fig.15 This is a flowchart showing details of a process for determining the behavior of the vehicle when the cause of the non-detection area is dirt on the irradiation surface of the LiDAR.

[0029] Fig.16 This is a diagram showing a method of changing a route.

[0030] Fig.17 This is a flowchart showing a process for determining the behavior of the vehicle when the cause of the non-detection area is water accumulation on the road surface.

[0031] Fig.18 This is a diagram showing a scene in which there is water accumulation on the road within the route range.

[0032] Fig.19 This figure shows an example of a point cloud map created using the vertical illumination angle and the horizontal illumination angle of LiDAR. DETAILED DESCRIPTION

[0033] Hereinafter, the embodiments of the present invention will be described using the drawings. In each of the drawings, the same reference numerals are given to the same elements, and duplicate descriptions are omitted as appropriate.

[0034] Figure 1 This is a schematic diagram of a mining vehicle 101 in this embodiment. Figure 1 In the figure, the mining vehicle 101 has a vehicle body 102; a cargo box 103 mounted on the upper rear side of the vehicle body 102 and capable of rotating in the up-down direction; a driver's seat 104 provided on the upper front side of the vehicle body 102; a pair of left and right driven wheels 105 mounted on the lower front side of the vehicle body 102; and a pair of left and right driving wheels 106 mounted on the lower rear side of the vehicle body 102. A GNSS (Global Navigation Satellite System) 107 is mounted on the upper front side of the vehicle body 102 as a vehicle position measurement sensor. The GNSS 107 uses satellite radio waves such as GPS (Global Positioning System) to measure the position of the vehicle body 102. In front of the vehicle body 102, a LiDAR 108 is mounted as an external detection sensor.

[0035] LiDAR 108 has a structure that irradiates laser light from vehicle 101 and uses reflected waves to obtain data. LiDAR 108 has a field of view that is unique to the model in the horizontal and vertical directions. It irradiates multiple lasers within the field of view and uses the reflected waves to obtain data such as the three-dimensional coordinates and reflection intensity of the position where the laser light is reflected. This data is generally called point cloud data. Unlike a camera, LiDAR 108 can obtain point cloud data without relying on the brightness of the surrounding area, and can detect objects regardless of day or night and weather.

[0036] Figure 2 1 is a block diagram of a driving control system mounted on a mining vehicle 101. The driving control system 201 includes a LiDAR 108, a vehicle position measuring sensor 202, a controller 203, and a vehicle control unit 204. The controller 203 is connected to the vehicle control unit 204 via a CAN interface 205 and a CAN bus 206, is connected to the LiDAR 108 via an Ethernet line 207, and is connected to the vehicle position measuring sensor 202 via a serial line 208.

[0037] The controller 203 includes a CPU 209, a memory 210, and a storage device 211. The storage device 211 stores map data, data necessary for autonomous driving, values ​​of calibration parameters of the LiDAR 108, and the like.

[0038] The vehicle position measurement sensor 202 is a sensor for measuring the position of the vehicle 101, and is composed of a wheel speed sensor 209, GNSS 107, and IMU (Inertial Measurement Unit) 210. The wheel speed sensor 209 and IMU 210 are devices for self-position estimation based on dead reckoning, and can function even when the GNSS 107 cannot receive satellite radio waves. By using these devices in combination, the vehicle position can be measured in various situations.

[0039] The CPU 209 processes the point cloud data acquired by the LiDAR 108 based on the vehicle position information from the vehicle position measurement sensor 202, and in the process, stores the necessary data and processing results in the memory 210. In addition, the information of the processing results is sent to the vehicle control unit 204 via the CAN interface 205 and the CAN bus 206. The vehicle control unit 204 controls the direction and speed of the vehicle 101 based on the processing results of the CPU 209. In addition, the vehicle control unit 204 creates and stores the dead reckoning information of the direction and speed of the vehicle 101. The controller 203 can acquire this information via the CAN bus 206 and the CAN interface 205.

[0040] Figure 2 2 is a functional block diagram of the controller 203. The controller 203 includes a point cloud map generation unit 301, a non-detection area detection unit 302, a sensor position change calculation unit 303, a non-detection area cause determination unit 304, and an action determination unit 305 as functions realized by software.

[0041] The point cloud map generation unit 301 expands the point cloud data acquired from the LiDAR 108 in a two-dimensional plane and creates a map. The map preferably uses an occupancy grid map (OGM). The point cloud map generation unit 301 converts each point cloud coordinate into a cell position of the grid map and places a mark in the cell indicating the existence of the point cloud.

[0042] The non-detection area detection unit 302 uses the above point cloud map to determine the cell area where the LiDAR 108 cannot obtain the point cloud data in the cell area corresponding to the point cloud data that the LiDAR 108 should obtain (that the LiDAR 108 can obtain). As the cause of the non-detection area, dirt on the irradiation surface of the LiDAR 108, liquid accumulation on the road surface, occlusion due to obstacles, sand and dust, etc. can be cited. According to these causes, cells without point clouds are generated in the point cloud map. The non-detection area is detected by classifying them.

[0043] In addition, the point cloud of LiDAR108 is discrete, and the farther away, the larger the interval between the points, and non-detection areas will also be generated. However, this problem can be avoided by accumulating point cloud data, for example, in the latest one second. At this time, it is only necessary to convert the acquired point cloud data into a global coordinate system, convert all the accumulated point cloud data into relative positions from LiDAR108, and then convert them into cell coordinates. Alternatively, for the purpose of reducing data capacity and speeding up processing, it is also possible to accumulate only the information of the cells where the point clouds in the point cloud map exist, and convert the cell coordinates into relative positions from LiDAR108 based on changes in the vehicle position. In addition, the acquired information of the non-detection area is accumulated in the storage device 211 in a manner that grasps the position changes of the non-detection area.

[0044] The sensor position change calculation unit 303 calculates the position change of the LiDAR 108 based on the result of the vehicle position measurement sensor 202. This is a relative position change from the position of the LiDAR 108 at the time of the previous measurement, and the moving direction and moving speed are calculated.

[0045] The non-detection area cause determination unit 304 determines the cause of the non-detection area detected by the non-detection area detection unit 302. Specifically, any one of dirt, road surface liquid, occlusion, and dust is determined. In addition, the center of gravity coordinates of the non-detection area are also calculated. The non-detection area cause determination unit 304 determines the cause of the non-detection area based on the history of non-detection area detection accumulated in the storage device 211 and the position change obtained by the sensor position change calculation unit 303, using the position relationship of the non-detection area at the previous moment and the current moment.

[0046] The control center 306 instructs the traffic control, destination, and driving route in the mine site. In order to instruct the driving route, the control center 306 obtains information on obstacles and bad roads detected by each vehicle 101 in the mine site, generates a route for avoiding them, and sends it to the vehicle 101. In addition, when dirt is detected on the irradiation surface of the LiDAR 108 in each vehicle 101, an instruction to stop or wash is issued according to the situation.

[0047] The action decision unit 305 determines the action of the vehicle 101 according to the cause of the non-detection area obtained by the non-detection area cause determination unit 304. Specifically, if the road is wet, it is determined to be any one of crossing, avoiding, and parking. If it is dirt, it is determined whether to stop or use the area without dirt to drive according to the location of the dirt.

[0048] Figure 4 1 is a diagram showing an example of a coordinate system used in the present embodiment. In the present embodiment, a coordinate system with the position of the vehicle 101 as the origin is used, the traveling direction of the vehicle 101 is set as the x-axis, the direction perpendicular to the x-axis and along the vehicle width direction is set as the y-axis, and the direction perpendicular to the road surface is set as the z-axis. In addition, in the y-axis, the direction from right to left in the traveling direction of the vehicle 101 is positive.

[0049] Figure 5 : This is a flowchart showing the processing of the autonomous driving system of a mining vehicle in the present embodiment. First, a non-detection area is detected in step 501. The processing of step 501 is performed by the point cloud map generation unit 301 and the non-detection area detection unit 302. After step 501, it is determined in step 502 whether there is a non-detection area. If the judgment result of step 502 is no, the process ends. If the judgment result of step 502 is yes, the cause of the non-detection area is determined in step 503. The processing of steps 502 and 503 is performed by the non-detection area cause determination unit 304. After step 503, the action of the vehicle 101 corresponding to the cause of the non-detection area is determined in step 504, and the process ends. The processing of step 504 is performed by the action determination unit 305.

[0050] Figure 6 is a detailed representation of the process of detecting non-detection areas ( Figure 5 First, in step 601, point cloud data within a certain period of time is obtained from LiDAR108. After step 601, the front slope of vehicle 101 is calculated in step 602. When calculating the front slope, for example, RANSAC (RANdom Sampling Consensus) is preferably used. After step 602, the range of the valid point cloud obtained from LiDAR108 is calculated in step 603.

[0051] Here, the necessity of the processing of step 602 and step 603 is explained. LiDAR108 is preferably set so that no non-detection area is generated even if there is a high obstacle or an uphill road in front. At this time, if the road surface is flat and there is no obstacle in front, the laser irradiated from the upper part of the vertical field of view of LiDAR108 will not irradiate any object, and there is a possibility that no point cloud will be obtained. Therefore, this non-detection area is not taken as the object of processing, and the point cloud obtained in the other area is taken as the object of processing.

[0052] After step 603, a loop of the number of valid point clouds is entered, and step 604 is executed for each valid point cloud. In step 604, the valid point cloud is projected onto the point cloud map. The processing of steps 602 to 604 is performed by the point cloud map generation unit 301. After jumping out of the loop of the number of valid point clouds, in step 605, the non-detection cells are classified and set as non-detection areas. After step 605, a loop of the number of non-detection areas is entered, and steps 606 and 607 are executed for each non-detection area. In step 606, the centroid coordinates of the non-detection area are calculated and converted into the global coordinate system. In the subsequent step 607, the information of the non-detection area is registered to the new non-detection buffer. After jumping out of the loop of the number of non-detection areas, the process ends. The processing of steps 605 to 607 is performed by the non-detection area detection unit 302.

[0053] Here, in association with the processing of steps 602 and 603, use Figure 7 Indicates the area where the point cloud could not be obtained. Figure 7 (a) indicates a situation where there is an uphill road in front of the vehicle 101. The installation pitch angle of LiDAR 108 is set to θp, the field of view angle on the upper side from a line perpendicular to the irradiation surface of LiDAR 108 is set to φu, the field of view angle on the lower side is set to φl, the road surface gradient of the uphill road is set to θg, and the installation height of LiDAR 108 is set to h.

[0054] The longest detection distance lv obtained based on LiDAR108 can be calculated using the slope length lg and by lv=lg×cos(θg). The slope length lg can be calculated using the sine theorem. In the case where the point reached by the laser at the upper limit of the field angle of LiDAR108 exists on the slope, the angle between the laser and the slope is θg+θp-φu. When it takes a positive value, point clouds are obtained from all lasers even if there is an uphill road ahead. That is, when the upper limit irradiation angle (φu-θp) of LiDAR108 observed from the horizontal plane is less than the road surface slope (θg), point clouds are obtained from all lasers.

[0055] on the other hand, Figure 7(b) is a case where a point cloud cannot be obtained from the laser above the irradiated surface, that is, when φu-θp ≥ θg. At this time, if there is no obstacle in front, at least a point cloud cannot be obtained from the laser whose field of view angle of LiDAR108 is greater than φu-θp-θg. However, for practical application, it is preferable to determine the longest detection distance lmax and treat the point cloud with a distance shorter than it as the processing object.

[0056] Figure 8 is a process for determining the cause of the non-detection area ( Figure 5 First, in step 801, the non-detection area is tracked. Specifically, the position of the non-detection area of ​​each time series is tracked using Kalman filtering or the like. After step 801, in step 802, the cause of the non-detection area is determined based on the tracking result of the non-detection area.

[0057] Fig. 9 is a detailed representation of the process of tracking non-detectable areas ( Figure 8 First, in step 901, the centroid position of the non-detection area detected at this moment and the registration buffer 1300 ( Fig.13 The Mahalanobis' distance is calculated by using all combinations of the previously estimated values ​​of the centroid positions of the non-detection areas detected in the past and stored in the image processing unit 101 (as shown in the figure) (the centroid coordinates 1314 at the subsequent moment). After step 901, a correspondence is established between these in step 902. In step 902, a cost matrix is ​​prepared for each combination, and the Hungarian algorithm is preferably used to obtain the best combination. In addition, in steps 901 and 902, the Euclidean distance can also be used to establish a correspondence, but if it is considered that the vehicle 101 will not move in the positive lateral direction, the Mahalanobis distance that can pay attention to the distance in the direction of travel is preferably used.

[0058] After step 902, a loop of the non-detection areas registered in the registration buffer 1300 is entered, and the processing after step 903 is performed for each registered non-detection area. In step 903, it is determined whether there is a registered non-detection area with a corresponding relationship established in step 902. If the result of the determination in step 903 is yes, it is determined in step 904 whether the moving distance is less than a threshold. The moving distance can be set as the difference between the position of the registered non-detection area at the previous moment and the position of the detected non-detection area with a corresponding relationship established. If the result of the determination in step 904 is no, the stationary flag 1305 of the registration buffer 1300 is set to 1 in step 905. If the result of the determination in step 904 is yes, the stationary flag 1305 is set to 0 in step 906. If the result of the determination in step 903 is no, it is determined that the registered non-detection area has moved out of the field of view of the LiDAR 108 or disappeared, and thus, the registered non-detection area is deleted from the registration buffer 1300 in step 907.

[0059] After steps 905 to 907, a loop for detecting the number of non-detection areas is entered, and the processing after step 908 is performed for each detected non-detection area. In step 908, it is determined whether a corresponding relationship between the detected non-detection area and the registered non-detection area is established. If the judgment result of step 908 is yes, the detected non-detection area has been registered in the registration buffer 1300, and nothing is done. If the judgment result of step 908 is no, it is regarded as a non-detection area that appears for the first time, and the detected non-detection area is registered in the registration buffer 1300 in step 909.

[0060] After jumping out of the loop of detecting the number of non-detection areas, the loop of registering the number of non-detection areas is entered again. In the loop of registering the number of non-detection areas, steps 910 to 912 are executed for each registered non-detection area. In this loop, the non-detection area is tracked using a Kalman filter. In step 910, the Kalman filter is used to perform post-prediction and pre-prediction of the center of gravity position of the non-detection area. In the subsequent step 911, the post-prediction result (current center of gravity coordinate 1307) and the pre-prediction result (center of gravity coordinate 1314 at the subsequent moment) of the registration buffer 1300 are updated. In the subsequent step 912, the number of detections 1303 of the registration buffer 1300 is increased by one. After jumping out of the loop of registering the number of non-detection areas, the process ends.

[0061] Fig.10 The process of determining the cause of the non-detection area 1306 is shown in detail ( Figure 8 First, a loop of registering the number of non-detection areas in the buffer 1300 (registering the number of non-detection areas) is entered, and the processing after step 1001 is executed for each non-detection area.

[0062] In step 1001, it is compared whether the number of detections 1303 of the non-detection area is above the threshold value. If the judgment result of step 1001 is no, the cause 1306 of the non-detection area is set to dust in step 1004. If the judgment result of step 1001 is yes, it is determined in step 1002 whether the stationary flag 1305 is 1. If the judgment result of step 1002 is yes, the cause 1306 of the non-detection area is set to dirt in step 1007. If the judgment result of step 1002 is no, it is determined in step 1003 whether there is a tall object near the non-detection area. If the judgment result of step 1002 is yes, the cause 1306 of the non-detection area is set to occlusion in step 1005. If the judgment result of step 1002 is no, the cause 1306 of the non-detection area is set to road surface water in step 1006.

[0063] After exiting the loop of registering the number of non-detectable areas, the content of the registration buffer 1300 is sent to the action determination unit 305 in step 1008. After step 1008, the content of the registration buffer 1300 is sent to the control center 306 in step 1009, and the process ends.

[0064] Regarding the processing of the non-detection area cause determination unit 304 described here, the Fig.11 Explain its concept. Fig.11 (a) is a case where the non-detection area 1101 is a road surface with water accumulated, and is a scene where the vehicle 101 is traveling in the direction of the arrow. In the case of water accumulated on the road surface, the position of the non-detection area 1101 on the road surface does not change, but the relative position relationship with the LiDAR 108 changes. In other words, the relative position relationship between the vehicle and the non-detection area changes according to the position change of the vehicle 101.

[0065] Fig.11 (b) is a case where the non-detection area 1102 is caused by dirt on the irradiated surface, and is a scene where the vehicle 101 is traveling in the direction of the arrow. In the case of dirt, even if the vehicle 101 moves, the relative positional relationship between the vehicle 101 and the non-detection area 1102 does not change.

[0066] Fig.11 (c) is a case where the cause of the non-detection area 1104 is the occlusion of the obstacle 1103, and is a scene where the vehicle 101 is traveling in the direction of the arrow. In the case of occlusion, the position change of the non-detection area 1104 on the road surface is small. In this case, it is difficult to distinguish it from the road surface water in (a), but in the case of occlusion, it can be distinguished by whether the obstacle 1103 exists in front of the non-detection area 1104. The non-detection area cause judgment unit 304 judges the cause of the non-detection area by using the above properties.

[0067] Fig.12 Indicates the process of classifying non-detection cells and setting them as non-detection areas ( Figure 6 State (a) represents the initial state of the point cloud, and the cells marked with the number "0" are non-detected cells. First, in state (a), the point cloud is scanned, and in state (b), cell 1201 is found as a non-detected cell. This becomes the starting point for classification, and the ID 1302 of the non-detected area to which cell 1201 belongs is set to 1.

[0068] Then, eight cells nearby are searched with cell 1201 as the center, and when a non-detected cell is found, it is included in the same non-detected area. In state (c), cell 1202 is found as a non-detected cell and added as a constituent cell with ID 1302 = 1.

[0069] Next, the eight nearby cells are recursively explored. In state (d), the eight nearby cells are explored with cell 1202 as the center. Then, in state (e), cell 1203 is found as a non-detected cell and added as a constituent cell of ID1302=1. The eight nearby cells are further explored with cell 1203 as the center. In state (f), cell 1204 is found as a non-detected cell and added as a constituent cell of ID1302=1. The eight nearby cells are further explored with cell 1204 as the center. No non-detected cells are found here except for the cells that have been classified, so the eight nearby cells are explored again with cell 1203 as the center. No non-detected cells are found here except for the cells that have been classified, so the eight nearby cells are explored again with cell 1202 as the center. Then, in state (g), cell 1205 is found as a non-detected cell and added as an element of ID1302=1. The eight nearby cells are further explored with cell 1201 as the center. Here, no non-detection cells are found except the cells that have been classified, so eight nearby cells are explored with cell 1201 as the center. Here, no non-detection cells are found except the cells that have been classified, so eight nearby cells are explored with cell 1205 as the center. Here, no non-detection cells are found except the cells that have been classified. As a result, the classification of all non-detection cells found by the exploration based on cell 1201 is completed, and the non-detection area with ID1302=1 is determined.

[0070] Then, the classification number is incremented by 1 and the search for non-detection cells is continued. Then, in state (h), cell 1206 is found as a non-detection cell, which becomes an element of classification ID 1302 = 2. The processing described so far is repeated to classify non-detection cells.

[0071] Fig.13 2 is a diagram showing the contents of the registration buffer 1300 stored in the storage device 211. The registration buffer 1300 is composed of a non-detection area number 1301 and non-detection area information 1326 whose number is indicated by the non-detection area number 1301.

[0072] The non-detection area information 1326 is composed of ID 1302 , detection times 1303 , detection time 1304 , stationary flag 1305 , cause 1306 , current centroid coordinates 1307 , subsequent time centroid coordinates 1314 , and constituent cell information 1321 .

[0073] ID1302 is the ID of the non-detection area. Fig.12 The detection count 1303 indicates the number of times the non-detection area is detected. The detection time 1304 is the time when the non-detection area is first detected. The stationary flag 1305 is a flag indicating that the non-detection area is stationary within the field of view of the LiDAR 108, and is assigned by the non-detection area cause determination unit 304. Fig. 9 Set 0 or 1 in step 905 or step 906.

[0074] Now the centroid coordinates 1307 are the coordinates of the non-detection area at the current point, which are composed of cell coordinates 1308 and global coordinates 1311. Cell coordinates 1308 are cell coordinates in the point cloud, and global coordinates 1311 are coordinates in the global coordinate system. In addition, they are composed of x coordinates 1309 and y coordinates 1310, and x coordinates 1312 and y coordinates 1313, respectively.

[0075] The subsequent moment centroid coordinates 1314 are the coordinates of the position where the non-detection area is predicted to exist at the subsequent moment, and are composed of cell coordinates 1315 and global coordinates 1318. The cell coordinates 1315 are the cell coordinates in the point cloud map, and the global coordinates 1318 are the coordinates in the global coordinate system. In addition, they are composed of x coordinates 1316 and y coordinates 1317, and x coordinates 1319 and y coordinates 1320, respectively.

[0076] The constituent cell information 1321 is information about cells constituting the non-detection area, and is composed of the constituent cell number 1322 and the cell coordinates 1323. The cell coordinates 1323 store the number represented by the constituent cell number 1322. The cell coordinates 1323 are composed of an x ​​coordinate 1324 and a y coordinate 1325.

[0077] Fig.14 The process of determining the action of the vehicle 101 according to the cause of the non-detection area is shown in detail. Figure 5 First, a loop of registering the number of non-detection areas 1301 stored in the buffer is entered, and the processing after step 1401 is executed for each non-detection area.

[0078] In step 1401, non-detection area information 1326 is obtained from the registration buffer 1300. In the subsequent step 1402, the cause 1306 is determined. In the case where the cause 1306 is dirt, the action of the vehicle 101 is determined in step 1403. In the case where the cause 1306 is road surface water, the action of the vehicle 101 is determined in step 1404. In the case where the cause 1306 is dust, the action of the vehicle 101 is determined in step 1405. After jumping out of the loop of the number of non-detection areas, the process ends.

[0079] Fig.15 The process of determining the behavior of the vehicle 101 when the non-detection area is caused by dirt on the irradiation surface of the LiDAR 108 is shown in detail. Fig.14 First, in step 1401, it is determined whether the dirt position overlaps with the path range. This is equivalent to determining whether the non-detection area 1102 exists within the path range. If the determination result of step 1501 is no, the vehicle continues to drive as it is in step 1502, and the process ends.

[0080] If the result of the determination in step 1501 is yes, in step 1503, it is determined whether the dirty position can be made outside the path range by changing the route of the vehicle 101. If the result of the determination in step 1503 is no, in step 1504, a warning is issued to the control center 306, and in the subsequent step 1505, the vehicle 101 is stopped, and the process ends. If the result of the determination in step 1503 is no, the route is changed in step 1506, and the vehicle continues to travel in step 1502, and the process ends.

[0081] use Fig.16 The method of changing the route will be further described. Fig.16(a) represents a scenario where the non-detection area 1102 exists within the path range. Here, the width of the path range is set to w, the width of the non-detection area 1102 is set to wd, the width of the vehicle 101 is set to wv, the distance between the right end of the non-detection area 1102 and the right end of the path range is set to wr, and the distance between the left end of the non-detection area 1102 and the left end of the path range is set to wl. In addition, the distance between the right end of the vehicle 101 and the left end of the non-detection area 1102 is set to wdv.

[0082] At this time, if wl and wr are compared, wr is smaller, so it is considered to set the non-detection area 1102 to the right of the path range. The condition that can make the non-detection area 1102 outside the path range is that wdv is greater than 0. As long as the vehicle 101 moves to the right with wd+wr, the non-detection area 1102 becomes outside the path range. Fig.16 (b) represents this situation.

[0083] Fig.17 The process of determining the behavior of the vehicle 101 when the cause of the non-detection area is water accumulation on the road is shown in detail. Fig.14 1404), it is envisaged that Fig.18 The scene shown. Fig.18 In the process, the distance between the left end of the non-detection area 1102 and the left end of the path range is set to wl, the distance between the right end of the non-detection area 1102 and the right end of the path range is set to wr, and the width of the vehicle 101 is set to wv. First, in step 1701, it is determined whether the vehicle width wv is greater than the road surface fluid width wd. If the determination result of step 1701 is yes, the vehicle 101 can cross the road surface fluid, and the action of crossing the non-detection area 1102 is determined in step 1702, and the process ends.

[0084] If the result of the judgment in step 1701 is negative, it is judged in step 1703 whether the distance wl is greater than the width wv of the vehicle, or whether the distance wr is greater than the width wv of the vehicle. This judgment is to judge whether there is space within the path range to bypass the road surface liquid. If the distance wl is greater than the width wv of the vehicle, or the distance wr is greater than the width wv of the vehicle, the road surface liquid can be bypassed within the path range.

[0085] If the result of the determination in step 1703 is yes, the action of bypassing the non-detection area 1102 is determined in step 1704, and the process ends. If the result of the determination in step 1703 is no, the action of stopping the vehicle 101 is determined in step 1705, and the process ends.

[0086] (Summarize)

[0087] In this embodiment, the autonomous driving system of a mining vehicle comprises: an external detection sensor 108 for acquiring a three-dimensional point cloud of the surrounding environment of the vehicle 101; a position measurement sensor 202 installed on the vehicle 101; and a controller 203 to which signals of the external detection sensor 108 and the position measurement sensor 202 are input, and the controller 203 comprises: a point cloud map generating unit 301 for establishing a correspondence between the three-dimensional point cloud acquired by the external detection sensor 108 and the point cloud map; a sensor for calculating the position change of the external detection sensor 108 based on the measurement result of the vehicle position measurement sensor 202; The controller 203 includes a device position change calculation unit 303; and a non-detection area detection unit 302 for extracting an area of ​​the point cloud map as a non-detection area when the external detection sensor 108 fails to obtain the three-dimensional point cloud that should be obtained. In the mining vehicle autonomous driving system, the controller 203 has: a non-detection area cause judgment unit 304 for judging the cause of the non-detection area based on whether the position of the non-detection area in the point cloud map changes according to the position change of the vehicle 101; and an action determination unit 305 for determining the action of the vehicle 101 according to the cause of the non-detection area.

[0088] According to the present embodiment configured as described above, when a non-detection area occurs in the area to be detected by the outside detection sensor 108, the mining vehicle 101 takes appropriate action according to the cause of the non-detection area, thereby suppressing a decrease in the working efficiency of the mining vehicle 101.

[0089] In addition, in the present embodiment, when the non-detection area cause determination unit 304 determines that the cause of the non-detection area 1102 is road accumulation, the action determination unit 305 determines whether to change the route to avoid the road accumulation, drive across the road accumulation, or stop the vehicle based on the size relationship between the width wv of the vehicle 101 and the width wd of the road accumulation determined as the cause of the non-detection area 1102. Thus, when a non-detection area of ​​the external detection sensor 108 is generated due to road accumulation, the reduction in the operating efficiency of the mining vehicle 101 can be suppressed.

[0090] In addition, the external detection sensor 108 in this embodiment is composed of LiDAR 108 (laser ranging sensor), and the action determination unit 305 determines whether to send an alarm to the control center 306 that manages the vehicle 101, change the forward route in a manner that the non-detection area 1102 does not overlap with the path range in the detection range of LiDAR 108, or stop the vehicle when the non-detection area cause determination unit 304 determines that the cause of the non-detection area 1102 is dirt on the irradiation surface of LiDAR 108. Thus, when a non-detection area is generated due to dirt on the irradiation surface of LiDAR 108, the reduction in the operating efficiency of the mining vehicle 101 can be suppressed.

[0091] In addition, the external detection sensor 108 in this embodiment is composed of LiDAR 108 (laser ranging sensor), and the point cloud map generation unit 301 calculates the longest detection distance lmax according to the road slope in front of the vehicle 101, and only processes the point cloud data of the three-dimensional point cloud measured within the object range below the longest detection distance lmax. Therefore, when a non-detection area is generated in the area that should be detected by LiDAR 108, the mining vehicle 101 takes appropriate actions corresponding to the cause of the non-detection area, thereby suppressing the reduction of the operating efficiency of the mining vehicle 101.

[0092] The embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments and includes various modifications. For example, the above embodiments are described in detail to explain the present invention in an easy-to-understand manner, but the present invention is not limited to having all the described configurations.

[0093] In this embodiment, an example of creating a point cloud map using the three-dimensional coordinates of the point cloud acquired by LiDAR 108 is described, but the point cloud map may not use the three-dimensional coordinates. The situation where a non-detection area is generated means that there is an irradiation direction from which point cloud data cannot be acquired. Therefore, as long as only the irradiation surface is judged whether it is dirty, the point cloud map can be created using the irradiation angle of LiDAR 108. Fig.19 This is an example of a point cloud image 1901 created using the vertical illumination angle (VFoV) and the horizontal illumination angle (HFoV) of the LiDAR 108. If the illumination surface is dirty, a non-detectable area 1902 will appear in the point cloud image 1901. Fig.12 The classification method shown detects the non-detection area 1902 and registers it in the registration buffer 1300 .

[0094] In addition, the above-mentioned various structures, functions, processing units, processing mechanisms, etc. can be partially or completely implemented by hardware, for example, through integrated circuit design. In addition, the above-mentioned various structures, functions, etc. can also be implemented by software by interpreting the programs that implement each function through the processor and executing them. The information such as programs, tables, files, etc. that implement each function can be placed in a memory and a recording device such as a hard disk, SSD (Solid State Drive), or a recording medium such as an IC card, SD card, DVD, etc. In addition, the control lines and information lines represent the parts that are considered necessary in the description, and are not limited to representing all the control lines and information lines required for the product. In fact, it can be considered that all structures are interconnected.

[0095] Description of Reference Numerals

[0096] 101… mining vehicle, 102… vehicle body, 103… cargo box, 104… driver's seat, 105… driven wheel, 106… driving wheel, 108… LiDAR (external detection sensor), 201… driving control system, 202… vehicle position measurement sensor, 203… controller, 204… vehicle control unit, 205… CAN interface, 206… CAN bus, 207… Ethernet line, 208… serial line, 209… wheel speed sensor, 210… memory, 211… storage device, 301… point cloud map generation unit, 302… non-detection area detection unit, 303… sensor position change calculation unit, 304… non-detection area cause judgment unit, 305… action determination unit, 306… control center, 1101, 1102… non-detection area, 1103… obstacle, 1104… non-detection area, 1201~1202 06…cell, 1300…registration buffer, 1301…number of non-detection areas, 1303…number of detections, 1304…detection time, 1305…stationary flag, 1306…cause, 1307…current center of gravity coordinates, 1308…cell coordinates, 1309…x coordinates, 1310…y coordinates, 1311…global coordinates, 1312…x coordinates, 1313…y coordinates, 1314…center of gravity coordinates at subsequent moments, 1315…cell coordinates, 1316…x coordinates, 1317…y coordinates, 1318…global coordinates, 1319…x coordinates, 1320…y coordinates, 1321…composition cell information, 1322…number of composition cells, 1323…cell coordinates, 1324…x coordinates, 1325…y coordinates, 1326…non-detection area information, 1901…point cloud map, 1902…non-detection area.

Claims

1. A mining vehicle autonomous driving system, comprising: An external detection sensor that obtains a three-dimensional point cloud of the vehicle's surroundings; A position measuring sensor mounted on the vehicle; and a controller to which the signals of the external environment detection sensor and the position measurement sensor are input, The controller has: A point cloud map generating unit that establishes a corresponding relationship between the three-dimensional point cloud acquired by the external detection sensor and the point cloud map; a sensor position change calculation unit for calculating a position change of the outside detection sensor based on a measurement result of the position measurement sensor; and When the external detection sensor fails to acquire the three-dimensional point cloud that should be acquired, a non-detection area detection unit extracts an area of ​​the point cloud image as a non-detection area, The mine vehicle autonomous driving system is characterized in that: The controller has: a non-detection area cause determination unit for determining a cause of the non-detection area based on whether a position of the non-detection area in the point cloud image changes according to a change in a position of the vehicle; and An action determination unit determines an action of the vehicle according to a determination result of the non-detection area cause determination unit.

2. The autonomous driving system for mining vehicles according to claim 1, characterized in that: When the non-detection area cause judgment unit determines that the cause of the non-detection area is road accumulation, the action decision unit determines whether to change the forward route to avoid the road accumulation, drive across the road accumulation, or stop the vehicle based on the relationship between the width of the vehicle and the width of the road accumulation determined to be the cause of the non-detection area.

3. The autonomous driving system for mining vehicles according to claim 1, characterized in that: The external detection sensor is composed of a laser distance measuring sensor. When the non-detection area cause judgment unit determines that the cause of the non-detection area is dirt on the irradiation surface of the laser ranging sensor, the action decision unit decides whether to send an alarm to the control center that manages the vehicle, change the forward route in a way that the non-detection area and the path range do not overlap in the detection range of the external detection sensor, or stop the vehicle.

4. The autonomous driving system for mining vehicles according to claim 1, characterized in that: The external detection sensor is composed of a laser distance measuring sensor. The point cloud map generating unit calculates a maximum detection distance according to a road surface gradient in front of the vehicle, and processes only point cloud data of a three-dimensional point cloud measured within a target range below the maximum detection distance.

Images