Obstacle avoidance trajectory for mining vehicles

Abstract

According to the invention, there is provided a method comprising: detecting (200), by an obstacle detection function of an underground mining vehicle (10) performing an autonomous driving task along a first route comprising a first set of route points, an obstacle on the basis of a scan by a scanner (40) of the mining vehicle; generating (240), by a route planning function, a second route comprising a second set of route points for the mining vehicle to bypass the obstacle on the basis of the determined expected position in which the vehicle is approaching, the determined position of the obstacle, and a worksite model indicative of limitations of route generation; providing (250) the second route to an autonomous driving controller of the mining vehicle; and controlling (260), by the autonomous driving controller, the mining vehicle to autonomously drive along the second route.

Description

Technical Field

[0001] The present invention relates to route adaptation of a mining vehicle for use at an underground worksite. Background Art

[0002] A mining or construction excavation site, such as an underground hard rock or soft rock mine, may include an area for automated operation of mining vehicles, such as loaders and / or haulers and drills. Such mining vehicles may include, for example, unmanned vehicles that are remotely controlled from a control room, or manned vehicles, such as vehicles operated by an operator in the cab of the vehicle.

[0003] The mining vehicle may be configured to perform at least some tasks autonomously.An automated mining vehicle operating in an autonomous mode may be configured to operate independently, but may be taken over under external control in certain operating areas or conditions, such as during a state of emergency.

[0004] The route from the starting point to the end point may be pre-planned for autonomous driving. In some systems, an experienced operator may teach the optimal path by driving the vehicle, or the route may be designed at least partially automatically based on input parameters and a model of the site. Conditions in an underground site may change during mining operations and affect driving performance along the route. For example, loose rock may cause the autonomous mining vehicle to stop and require manual control by an operator. This production interruption negatively affects the efficiency of the mining operation at the site. Summary of the invention

[0005] The invention is defined by the features of the independent claims. Some particular embodiments are defined in the dependent claims.

[0006] According to a first aspect, a device is provided, the device being configured to perform at least the following operations, or the device comprising means configured to perform at least the following operations: detecting an obstacle by an obstacle detection function of an underground mining vehicle performing an autonomous driving task along a first route comprising a first set of route points, based on scanning by a scanner of the mining vehicle; determining the position of the obstacle; in response to detecting the obstacle, determining an expected position that the vehicle is approaching before the obstacle so as to start a new route of the vehicle to an end point of the first route; providing the expected position that the vehicle is approaching and the position of the obstacle to a route planning function; generating a second route by the route planning function based on the vehicle approaching the expected position, the position of the obstacle and a site model indicating constraints for route generation, the second route comprising a second set of route points for the mining vehicle to bypass the obstacle; providing the second route to an autonomous driving controller of the mining vehicle; and controlling the mining vehicle to autonomously drive along the second route by the autonomous driving controller.

[0007] According to a second aspect, a method for mining vehicle route planning is provided, the method comprising: detecting an obstacle by an obstacle detection function of an underground mining vehicle performing an autonomous driving task along a first route including a first set of route points, based on scanning by a scanner of the mining vehicle; determining the position of the obstacle; in response to detecting the obstacle, determining an expected position that the vehicle is approaching before the obstacle so as to start a new route of the vehicle to an end point of the first route; providing the expected position that the vehicle is approaching and the position of the obstacle to a route planning function; generating a second route by the route planning function based on the vehicle approaching the expected position, the position of the obstacle and a site model indicating constraints for route generation, the second route including a second set of route points for the mining vehicle to bypass the obstacle; providing the second route to an autonomous driving controller of the mining vehicle; and controlling the mining vehicle to autonomously drive along the second route by the autonomous driving controller.

[0008] According to a third aspect, a device is provided, comprising at least one processor and at least one memory, wherein the at least one memory comprises computer program code, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, enable the device to at least perform the method or an embodiment of the method.

[0009] According to a fourth aspect, a computer program, a computer program product or a (non-tangible) computer readable medium comprising computer program code which, when executed in a data processing apparatus, causes the apparatus to perform the method or embodiments of the method is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 An example of a mining vehicle is shown;

[0011] Figure 2 A method according to some embodiments is shown;

[0012] Figure 3 illustrates the functionality of a control unit according to some embodiments;

[0013] Figure 4a and Figure 4b An example of a top view of a mining vehicle performing a driving task is shown;

[0014] Figure 5 A pit bottom model generation procedure is shown; and

[0015] Figure 6 An example device is shown that can support at least some embodiments. DETAILED DESCRIPTION

[0016] Figure 1 A simplified example of a portion of a mining site 1 (comprising an underground tunnel portion or tunnel 2 in this example) is shown. A mining site may comprise a mine or a construction site, such as a railway or road tunnel site. However, it will be appreciated that a site may also comprise an area on the surface of the ground. The term underground mining vehicle herein generally refers to a mobile working machine suitable for use in at least the operation of different kinds of underground mining or construction excavation sites, such as a truck, a bulldozer, a dumper, a van, a mobile rock drilling or milling drill, a mobile reinforcement machine and a bucket loader or other kinds of mobile working machines that may be used in different kinds of excavation sites.

[0017] In this example, the mining vehicle 10 is a loader or load and haul (LHD) vehicle including a bucket 11 connected to a boom 12. The vehicle 10 may be, for example, an articulated vehicle including two sections connected by a joint 13. However, it should be understood that the application of the features disclosed herein is not limited to any particular type of vehicle that may be used at an excavation site. Some other examples of such vehicles include inspection vehicles, autonomous inspection vehicles (AIVs), trucks, dump trucks, vans, mobile rock drilling or milling rigs, or mobile reinforcement machines.

[0018] exist Figure 1In the example of , the vehicle 10 includes a system 14 consisting of a pump for generating hydraulic pressure to operate various parts of the machine, such as lifting the boom 12, rotating the bucket 11, etc. The vehicle 10 may include one or more other energy sources, such as accumulators, hydrogen containers, fuel tanks, etc. The vehicle 10 may include a motor 15, such as a combustion engine or an electric motor. The power from the motor 15 can be provided to the front and / or rear wheels directly by the crankshaft or via a gearbox. The drive train of the vehicle 10 may include, for example, a motor 15, a crankshaft and / or a transmission.

[0019] The vehicle 10 includes at least one control unit 20, which is configured to control at least some functions and / or actuators of the vehicle. The control unit 20 may include one or more computing units / processors that execute computer program codes stored in a memory. The control unit 20 may be connected to one or more other control units of the control system of the vehicle, for example, via a controller area network (CAN) bus. The control unit 20 may include or be connected to a user interface (UI) having a display device and an operator input interface. Such an interface may be used to receive operator commands and information to the control unit. It should be understood that the control unit 20 may be configured to perform at least some of the features shown below, or multiple control units or controllers may be applied to perform these features. It should be understood that at least some of these control functions may even be implemented outside the vehicle (e.g., at a worksite management system).

[0020] The vehicle 10 may comprise a wireless communication device 30 by means of which the control unit 20 of the vehicle 10 and / or another unit of the control system may be configured to establish a data connection. The data connection comprises at least one of data reception and / or data transmission. The data connection may be formed between the control system of the vehicle and at least one control device of a (mining operation) control system 6 external to the vehicle. The data connection may make use of a wireless connection provided by a base station or access node 4. Thus, the communication device 30 may be operably connected to a communication system of the worksite, such as a wireless access system comprising a wireless local area network (WLAN) and / or a cellular communication network (such as a 4G, 5G or some other generation of cellular network).

[0021] The control system 6 may include or be connected to additional networks and / or data processing systems, such as a site management system, a cloud service, a data analysis system, an intermediate communication network, such as the Internet, etc. The control system 6 may include or be connected to additional devices or control units, such as a handheld user unit, a vehicle unit, a site management device / system, a remote control and / or monitoring device, a data analysis device, a sensor system / device, etc.

[0022] The vehicle 10 may be unmanned. An unmanned vehicle may include a user interface to allow a user to control any function of the vehicle, such as steering. In addition to or as an alternative to the user interface, the vehicle may include an operator input interface. The interface of the vehicle may be located in a position away from the vehicle, and the vehicle may be remotely controlled. Examples of remote control include: controlled by an operator in a tunnel, or controlled by a device in a control room at a construction site, or even controlled by a device from a long distance away from the construction site via a communication network. The vehicle 10 may be an automated vehicle that can be independently operated / driven in an autonomous operation or driving mode without continuous user control. Such an automated vehicle may be taken over under external control, for example, during a state of emergency. When the vehicle is in manual driving mode, the operator manually drives the vehicle by remote control, or drives the vehicle locally by operator control. The operator may set the vehicle to a (default) automatic driving mode, in which the vehicle is configured to automatically drive a designated route (e.g., between a loading point and a dumping lane).

[0023] Equipment at the worksite, such as the control system 6 and the control device of the mining vehicle 10, may be configured to store and use at least one worksite model representing the current state and / or target state of the worksite environment. The worksite model may be an environment model or a map. For example, the worksite model may be a two-dimensional (2D) model or a three-dimensional (3D) model indicating surface contours and obstacles at the worksite at the area operated by the mining vehicle. In the case of an underground worksite, the worksite model may be a tunnel model indicating the contours of the tunnel.

[0024] In some embodiments, the control system 6 and / or the mining vehicle 10 is configured to store a 3D (tunnel) model of the underground site, which shows the pit floor, pit walls and pit roof of the tunnel network. The 3D model may include point cloud data generated on the basis of the scan or be formed based on these point cloud data. In some other embodiments, the 3D model may be stored in some other format (such as a mesh model including vertices, edges and faces). In some embodiments, the 3D model may be a design model, or may be generated based on a design model (such as a CAD model) created by mining design software or a 3D model created based on tunnel lines and contours designed in drilling and blasting design software.

[0025] Vehicle 10 may include one or more scanning units or scanners 40 configured to perform a scan of the environment surrounding the vehicle. In some embodiments, the scan results are applied to detect the position and orientation of the vehicle and one or more other elements thereof, such as the position and orientation of scanner 40 or bucket 11.

[0026] The control unit 20 may be configured to compare the scanned tunnel profile data with reference profile data stored in the site model. The control unit 20 may be configured to locate the mining vehicle based on finding a match between the surface profile indicated by the scan data and the surface profile indicated by the site model. The control unit 20 may be configured to correct the positioning by dead reckoning positioning based on the scanning-based positioning. In some embodiments, the scanning results are applied to detect the position and orientation of the mining vehicle and one or more additional elements thereof, such as the position of the scanner 40 or the bucket 11.

[0027] In an embodiment, for example, the scanner 40 is a 2D scanner configured to monitor a tunnel wall at a desired height. The mining vehicle 10 may be configured to apply a 2D site model for monitoring the position of the mining vehicle based on an environmental scan performed by a 2D scanner included in the mining vehicle. In another embodiment, the scanner 40 is a 3D scanner configured to generate 3D scan data or point cloud data, such as a light detection and ranging (lidar) device.

[0028] A driving plan or route plan may define a route to be driven by the vehicle 10 and may be used as an input to the autonomous driving control of the vehicle. In addition, such a plan may be used as part of a mission or a planned mission. When a mission includes a route plan, the plan may form a route to be driven during the mission. The plan may be generated offline and off-site (e.g., in an office). The plan may be generated on the vehicle, for example, by teach driving. The plan may define a starting point, an end point, and a set of route points for autonomous driving. If new obstacles appear at the route after the route plan is defined, the autonomously driven mining vehicle may need to stop. After stopping, the route may need to be redefined, for example, based on input from an operator of a fleet of vehicles remotely monitoring the mining vehicles through the control system 6.

[0029] Improvements are now provided to facilitate dynamic mining vehicle route adaptation during autonomous driving missions.

[0030] Figure 2 A method for controlling a mining vehicle and autonomous driving of a mining vehicle according to some embodiments is shown. The method may be performed by a device, which in some embodiments is a control device configured to control a mining vehicle and autonomous driving related operations thereof. The method may be performed by a mining vehicle and its control device (such as a mining vehicle 10) and its control unit 20, respectively. The method may be a computer-implemented method.

[0031] Block 200 includes detecting obstacles based on scanning by a scanner of the mining vehicle by an obstacle detection function (ODF) of the underground mining vehicle performing an autonomous driving mission along a first route including a first set of route points.

[0032] Block 210 includes determining the location of the obstacle.

[0033] Block 220 includes, in response to detecting an obstacle, determining an expected position that the vehicle is approaching before the obstacle so as to start a new route for the vehicle to the end of the first route. Thus, the expected position can be used as a starting point for generating a new route to avoid the obstacle.

[0034] Block 230 includes providing the expected position of the vehicle approaching and the position of the obstacle to a route planning function (RPF). For example, the control unit or a function thereof may be configured to send the position information of the obstacle received from the ODF and the determined expected position to the RPF in block 230.

[0035] Block 240 includes generating, by the RPF, a second route including a second set of route points. The second set of route points for the mining vehicle to circumvent the obstacles is generated based on the expected location that the vehicle is approaching, the location of the obstacles, and the site model indicating limitations for route generation. Thus, the information provided in block 230 may be used as input parameters for generating the second route. Thus, the expected location may be used as a starting point for the second route requested from the RPF in block 230 to avoid the obstacles.

[0036] Block 250 includes providing the second route to an autonomous driving controller (ADC) of the mining vehicle. For example, the RPF may be configured to send information of at least the locations of route points of the second route to the ADC.

[0037] Block 260 includes the ADC controlling the mining vehicle to autonomously drive along the second route.

[0038] A route point may generally refer to information including at least position information to control autonomous driving of a vehicle. For example, a route point may define the x and y coordinates of a midpoint between tunnel pit walls (or between an obstacle and a tunnel pit wall) at which the ADC controls the midpoint of the vehicle. An autonomous driving mission may refer to route control information provided to a mining vehicle, such as the start and end points of a route. For example, the driving mission information may be based on input received at a user interface of the vehicle 10 or the control system 6. The RPF of the vehicle may be configured to define route points for the vehicle to follow in order to perform the driving mission and navigate around obstacles.

[0039] Features of the present invention and the collaboration of RPF, ODF and ADC facilitate obstacle avoidance through dynamic rerouting performed on-board the vehicle immediately upon detection of an obstacle. Thus, it may be feasible to avoid stopping the vehicle, and instead the vehicle may continue to drive (possibly at a reduced speed) while a new route is calculated and commissioned. For example, delays that affect production efficiency may be reduced because the need for manual intervention and control by an operator may be avoided. Features of the present invention enable major improvements to the on-board adaptive capabilities of autonomous mining vehicles.

[0040] refer to Figure 3 The mining vehicle 10 may include an ODF, an RPF, and an ADC, which may also be referred to as an autonomous driving control function. These functions may correspond to Figure 2 The control unit 20 may be configured to perform at least some of these functions. Alternatively, multiple control units may be applied, for example, a separate control unit for each function. It should be understood that there may also be additional functions related to autonomous driving, such as positioning functions.

[0041] The ODF may be configured to receive scan data from the scanner 40 or from a scanner application or function that provides scan data based on a scan of the environment performed by the scanner. Figure 4a The scanning data may be based on a scanning signal set 400 of a scanner. The scanning data may indicate that an obstacle 410 exists in a driving direction D on a first route including a first set of route points 420 to a destination 430.

[0042] The ODF may be part of (or operatively connected to) a collision avoidance system (CAS). The ODF may be configured to perform a collision check based on scan data received from at least the scanner 40. In another embodiment, the CAS is configured to: perform a collision check based on an indication of a detected obstacle from the ODF, and control further actions based on the collision check. The scan data may come from multiple scanners. For example, one scanner may cover the rear of the vehicle, and another scanner may cover the front of the vehicle with a directional beam. The ODF or CAS may be configured to control or cause a stop of the mining vehicle in response to detecting an object as an obstacle in the area.

[0043] The ODF may be configured to process the scan data, which may indicate the distance to the tunnel portion and further obstacles or their locations, to detect obstacles in a detection area ahead in the driving direction D of the vehicle. To detect obstacles, the scan data may be processed in or already before block 200. The processing may be based on obstacle detection control parameters, such as control parameters that control a detection area and / or a safety area defining a surrounding of the vehicle.

[0044] The device (such as ODF or CAS) can be configured to apply one or more obstacle detection or safety zones around the vehicle. For example, at least one safety zone or envelope 440 can be defined on at least one side of the vehicle 10.

[0045] ODF / CAS can be configured to determine whether the obstacle will fall into the safe area when the vehicle follows the route based on the location of at least some of the route points 420, the obstacle detection parameters and the determined location of the obstacle 50. If yes, then box 220 can be entered. If not, ODF can continue to process new scan data. Alternatively, processing can be before box 200, and box 200 can involve detecting obstacles to be avoided. In the latter embodiment, the location of the obstacle can already be determined before box 200 (rather than box 210).

[0046] In an example embodiment, the device is configured to apply a warning zone and a collision zone or envelope. For example, if an obstacle is detected within the warning zone, a warning may be issued and the speed of the vehicle may be reduced. If an obstacle is detected within the collision zone, the vehicle will hit the obstacle or is too close to the obstacle and further safety action is required. Instead of the mining vehicle simply stopping, block 220 may now be entered to attempt to go around the obstacle.

[0047] ODF / CAS can be configured to analyze the severity or category of the obstacle to determine whether it needs to be avoided or bypassed. This can be performed when an obstacle is detected that will fall into the safety zone of the vehicle. The analysis can be another box between box 210 and box 220. For example, ODF / CAS can be configured to define the vertical z-direction size or height of the obstacle and compare it with the relevant threshold parameters of the vehicle. If the threshold is exceeded, the obstacle is classified as an obstacle to be avoided (for the vehicle in question). If the obstacle is classified as an obstacle to be avoided, box 220 can be entered. For example, ODF can be configured to classify the set of lidar measurements so that: if there is a measurement value above a threshold (e.g., 30 cm) from the bottom of the pit defined relative to the mining vehicle, it indicates an obstacle to be avoided.

[0048] Devices such as the control unit 20 and further ADCs may be configured to control a reduction in the speed of the mining vehicle in response to the detection 200 of an obstacle. For example, the ODF may be configured to transmit an indication of an obstacle or a request to decelerate to the ADC in response to the block 200 (and the detection of an obstacle to be avoided). In response to such an indication or request, the ADC may be configured to reduce the speed to a predefined target speed. This makes it possible to further reduce the risk of the mining vehicle hitting an obstacle before the second route has been used. The reduced speed provides time for the device to prepare and put the second route into operation, thereby avoiding a stop of the vehicle. This is a significant advantage for heavy mining vehicles, especially at uphill places, such as ramps. The device may be configured to dynamically limit the speed to avoid: stopping the mining vehicle before the obstacle (in some embodiments before the expected position) and starting to drive along the second route. For example, the ODF may be configured to limit the speed (in order to avoid stopping) based on a determined distance of the vehicle to the obstacle (or the expected position that may have been defined) and an expected time to define and put the second route into operation.

[0049] The apparatus may be configured to increase the speed when controlling the mining vehicle to be driven along the second route by the ADC after block 250. For example, the control unit 20 may be configured to control the mining vehicle to a target speed defined for the portion of the route currently being driven by the vehicle.

[0050] The position of the obstacle determined in block 210 may include a position in a worksite reference system (such as a global coordinate system). The ODF may be configured to determine 3D position information of the obstacle. The ODF may be configured to determine the position of the obstacle relative to the scanner 40. As described above, the position of the scanner 40 may be determined based on an environmental scan. Then, the position of the obstacle in the global coordinate system may be determined based on the position of the scanner. This may be determined based on the measured distance and direction of the obstacle from the scanner.

[0051] In some example embodiments, the ADC is configured to control navigation of the vehicle along the route based on the route points provided by the RPF. The ADC may be configured to define steering control signals to other units or actuators of the mining vehicle based on position information of the route points, such as x and y coordinates in a global coordinate system. Thus, the ADC may define and provide control signals after block 260 such that the mining vehicle autonomously drives along the route points of the second route in order to avoid obstacles during autonomous driving toward an end point that is a destination of the vehicle. The second route may include or otherwise be associated with additional parameters, such as target speed information, on the basis of which the ADC may control the autonomous driving.

[0052] Figure 4b The vehicle is shown after having successfully generated a second route with a second set of route points 450 at block 240. At least some of the route points in set 450 are positioned so that the vehicle will pass obstacle 410 without the obstacle falling into safety zone 440.

[0053] It should be understood that there are several implementation options for generating a route and defining route points in block 240, some example embodiments are shown below. The second route can be a new route defined in block 240 further based on the initial route endpoint location information. Therefore, the RPF can be configured to generate a second route in block 240 to completely replace the first route. The RPF can be configured to calculate a second set of route points from the expected location of the vehicle to the endpoint of the first route.

[0054] In another embodiment, a partial route regeneration is performed in block 240. The second set of route points may include new route points 460 that replace route points in the first set of route points that are close to the obstacle. For example, close to the obstacle may mean that the new route point (or the replaced route point) is within a predetermined distance from the obstacle. The device may be configured to generate 240 route points in an area within a predetermined distance from the location of the obstacle. In another example, a predetermined number of route points may be generated in block 240. Thus, a device such as an RPF may be configured to: - based on the position of the obstacle and the vehicle size information, defining waypoints in the first set of waypoints which are unavailable to the mining vehicle and which are affected by the detected obstacle; and - determining the second set of waypoints to replace waypoints defined as unusable for the mining vehicle.

[0055] The device may be configured to define unusable route points based on the locations of the route points in the first set and their distances to the locations of obstacles 410. Route points in the first route that are within a predetermined distance from obstacles may be omitted or removed as unusable route points. The RPF may be configured to generate 240 route points further based on vehicle attribute parameter data. For example, the vehicle attribute parameters may include vehicle dimension parameter data indicating at least the dimensions of the vehicle and / or safety parameters (such as the size of the safety or collision zone required at the side of the vehicle). The RPF may be configured to generate 240 route points at predetermined intervals (e.g., 0.5 m).

[0056] Based on the vehicle attribute parameter data, the RPF can be configured to determine the location of the route points so that they allow sufficient safety margins between one side of the vehicle and the obstacle and between the other side of the vehicle and the tunnel pit wall (or obstacle) on the other side. If no such route point is found, the RPF can be configured to indicate this to the ADC and / or on the user interface to notify the operator. The ADC can be configured to control the stopping of the vehicle in response to such an indication. The ADC or RPF can be configured to indicate the interruption of the driving task and the stopping of the vehicle to the operator via a UI (e.g., a UI of a control system). Further information of the detected obstacles can be provided to the UI, such as a 2D or 3D UI view indicating the location and size of the obstacles in the site model.

[0057] Thus, the RPF may be configured to determine a second route from the desired location as a starting point to the initial destination (in some embodiments, the second route may be referred to as a (second) path). For a given route, the RPF obtains the required inputs in block 240 and may be configured to determine an available route or path to reach the destination location from the starting location. The RPF may be configured to generate one or more available reference lines to represent available routes to reach the destination location from the starting location (i.e., by bypassing the obstacles detected in block 200). A reference line may refer to an ideal route or path without interference from other objects (such as other vehicles or other obstacles).

[0058] If there are multiple available route options, the RPF or another module (such as a decision module) selects one of the route options as the second route based on route selection parameter information (such as driving or traffic state information, driving performance optimization parameters, etc.). The RPF can be configured to determine or adapt a second set of route points based on the selected route option or reference line. The ADC can be configured to steer the mining vehicle to accurately or closely follow the (selected) reference line of the second route (as defined by the second set of route points), or to steer to the route point without defining a reference line. The ADC can be configured to determine driving parameters such as speed and / or turning angle for one or more second route points. Each route point or a section of route points can be associated with a target speed, based on which the ADC controls the traction of the vehicle 10. For example, since the safety zone 440 is closer to the pit wall, a lower speed can be assigned to the route point 460 close to the obstacle 410 than other route points of the route.

[0059] The expected position that the mining vehicle is approaching may include an expected future position of the mining vehicle. The RPF may be configured to select the expected position as a starting point for the second route calculation in block 240. The device (in some embodiments, the ADC) may be configured to estimate 220 the expected position for the second route generation in response to block 200. For example, the ODF may be configured to transmit an indication of an obstacle to be avoided or a request to stop or slow down to the ADC, thereby triggering the ADC to estimate the expected position.

[0060] In some embodiments, the expected position is an expected stopping position of the mining vehicle. The device (in some embodiments, the ADC) may be configured to: in response to detecting an obstacle, estimate the expected stopping position of the mining vehicle due to the object based on the current position of the vehicle, the current speed of the mining vehicle and the position of the obstacle. The estimation may further apply (stop distance impact) information as input to define the distance required for stopping and / or the stopping position relative to the site model. For example, such further information may include vehicle deceleration parameters and / or route attributes based on the site model. In block 240, the ADC may be configured to transmit the expected stopping position to the RPF. Therefore, the second route may start from the stopping position, and the ADC may change to use the (first) route point in the second set when approaching the stopping position. By utilizing the expected stopping position and generating the second route before the position, the vehicle may be avoided from stopping, thereby making it possible to save energy consumption and improve hauling efficiency. When the expected stopping position is estimated, the device may control the vehicle to reduce speed, generate the second route while the vehicle is driving (rather than only after stopping), and change to dynamically use the second route when approaching the expected stopping position without stopping the vehicle.

[0061] The expected position may be another position, other than the expected stopping position, selected at a sufficient distance in front of the obstacle so that the mining vehicle avoids stopping (or risk of hitting the obstacle). For example, the expected position may be defined as the position where the vehicle would have to initiate emergency braking to avoid hitting the obstacle, if possible.

[0062] In some embodiments, the site model in box 240 includes point cloud data, which can be generated based on scanning the environment by a 3D scanner. In another embodiment, the site model is a 2D model, which can be generated by a 2D scanner. Depending on whether the ADC is configured to control autonomous driving based on 2D or 3D route point input, at least 2D positions of the route points are defined in box 240. It should be noted that, for example, if 2D (horizontal plane or x and y directions) coordinates are defined for the route points, vertical direction (z) position information can be associated with the route points at the tunnel level.

[0063] A device such as the control unit 20 may be configured to update the site model based on the scan data from the scanner 40. The device may be configured to update the site model to include information indicating the location of obstacles. This may be another box after box 210 (in some embodiments, before box 240). The device may include another site model processing function (MPF) configured to update the model based on the obstacle information from the ODF. Providing the location to the RPF may include, for example, providing the updated site model to the RPF by the MPF. Box 240 may include generating a second route based on the updated site model.

[0064] The vehicle 10 may include a simultaneous localization and mapping (SLAM) unit. Such a unit is configured to localize the vehicle and (enhance) map the environment, such as geographical features of the environment, based on (2D or 3D) scanning information while the vehicle is driving, for example in an autonomous mode. The SLAM unit may be configured to update the site model based on detected obstacles.

[0065] The device may be configured to apply a driving performance map (which may also be referred to as a drivable map) as a site model. The driving performance map includes driving performance data indicating the driving performance of the vehicle at a set of locations in the tunnel. The driving performance map is based on a subset of points of the 3D site model. The driving performance map (which may also be referred to as a driving performance model) includes a plurality of points enhanced with driving performance data. The RPF may be configured to generate 240 a second route for the vehicle based on the driving performance map. The RPF may be configured to update the driving performance map to include information about detected obstacles.

[0066] In some embodiments, the site model includes or is based on a pit floor model generated on the basis of the 3D site model. The term pit floor model generally refers to a model that includes a set of points indicating the tunnel pit floor at least in a horizontal plane (e.g., 2D or x, y coordinates). Such points may also be referred to as pit floor points. A more efficient route or path calculation or position tracking of a vehicle may be achieved by including only a map of the pit floor of the mine, wherein the driving performance attributes are associated with some or all of the pit floor points.

[0067] Also refer to the simplified Figure 5, the device can be configured to apply the 3D site model as input to a model processing program / algorithm, such as MPF. The device can be configured to execute a program for generating a pit bottom model, the program comprising: identifying pit bottom points among the points of the 3D model. This may refer to detecting points at the pit bottom level of the bottom of the tunnel on the basis of performing one or more evaluation or filtering programs. Extracting the identified pit bottom points. This may refer to forming a further data set based on the identified pit bottom points. At least one set of extracted pit bottom points is applied as a pit bottom model of the tunnel for positioning and / or route planning of mobile objects in the tunnel. For example, the pit bottom model can therefore be stored in a database accessible by a positioning unit or application or a route planning unit or a site server or an application of a mine vehicle. The pit bottom model can be applied as a map of the tunnel and used for route planning, and the pit bottom points can therefore be considered as map points.

[0068] It should be understood that the pit floor model does not necessarily define the absolute lowest bottom level of the underground tunnel, but it may be more practical to extract the pit floor model and the pit floor points that define the accessible area at a certain height from the tunnel bottom. Figure 5 As further shown in FIG. 1 , in some embodiments, the 3D site model includes 3D point cloud data. The point cloud data may be generated based on scanning the tunnel. Thus, the pit bottom model may be a point cloud model of the pit bottom that includes a subset of points extracted from the 3D site point cloud model data representing the pit bottom of the tunnel.

[0069] The drivability map may include pit bottom point records for pit bottom points of the pit bottom model, wherein at least some of the pit bottom point records include enhanced drivability data for the vehicle.

[0070] The drivability map, which may be based on the pit floor model, may be applied by a route planning application for route calculation, in some embodiments, by the RPF in block 240. The application may be configured to calculate a route / path along the pit floor defined by the pit floor points based on the pit floor model. A simplified pit floor model and drivability map may be beneficial to the route planning algorithm instead of a full 3D site model, since, for example, the cave roof may be unnecessary information for route planning. In some embodiments, the drivability map is applied by a navigation application (such as an ADC) controlling the autonomously operated mining vehicle 10. The navigation application may be configured to apply the drivability map to vehicle path definition and / or steering control along the pit floor defined by the pit floor points.

[0071] In some embodiments, the driving performance data includes vehicle-specific driving performance data that is specific to an automated or semi-automated vehicle or vehicle type. The driving performance data may be generated based on vehicle attribute data that may indicate the size of the vehicle or the type of vehicle. Appropriate vehicle attribute data may be retrieved from a memory or another device based on a received vehicle or vehicle type identifier for driving performance map generation and / or route planning. For example, a given family or product family of loading & hauling equipment may have specific common dimensions that are indicated in the associated vehicle attribute data and used to generate driving performance data and / or route information for these vehicles.

[0072] The drivability map may indicate points in a subset of points of the worksite model that are drivable by a vehicle or vehicle type. For example, the drivability map may include a subset of points that are drivable by a vehicle or vehicle type (such as a pit bottom point), or an indication of the drivability (drivable or non-drivable) of a vehicle or vehicle type may be included in at least some points of the drivability map.

[0073] In some embodiments, the height of the detected obstacle is included in the drivability data. In response to the height of the obstacle exceeding an obstacle height threshold (which may be vehicle or vehicle type specific), the device may be configured to include an indication of the height of the local obstacle and / or an indication of undrivability in the drivability map and / or the pit floor model. The device may be configured to apply the indication in route generation 240, for example, the corresponding point may be omitted from the available route options, or the availability of the point may be determined based on the height indication.

[0074] As an example, consider a vehicle having a width W of, for example, 2 meters and having a ground clearance value G (e.g., 20 centimeters). The device may be configured to define a subset of a pit bottom model for route planning, in which the vehicle can be driven without the risk of hitting an obstacle. The subset can be found from a pit bottom model enhanced with a local obstacle height h. The device may be configured to: for each point p in the pit bottom model, define a point at a radius of W / 2 based on the width W of the vehicle in question. The device may be configured to assign such a point zero to the driving performance of the vehicle if any point in the radius has an obstacle height h greater than the ground clearance G. If no point has an obstacle height higher than G, the device may be configured to assign true or one to the driving performance of the point. The result may be a binary map indicating the driving performance of the vehicle.

[0075] The information generated based on the height of obstacles can be used to extract a subset of the pit floor model, such as a pit floor grid, for route generation. When the route is calculated along the graph formed by the edges of this smaller pit floor grid, the route will automatically be at a sufficient distance from the pit walls and objects and free of obstacles. Using such a driving-specific map, different routes can be given to narrow-veined vehicles that can drive through narrow passages, for example. On the other hand, large vehicles require a certain tunnel width.

[0076] In some embodiments, an update of the site model triggers an automatic update of the pit floor model and possibly further associated models (such as a surface mesh model) to match the updated model. The device may be configured to update a subset of points of the pit floor model and a driving performance map (if separate from the pit floor model) in response to detecting an update of the site model. In some embodiments, the device is configured to include information of detected obstacles into the driving performance map. In addition, some driving performance data may be dynamic and repeatedly updated. For example, an obstacle may be mobile and obstacle-related driving performance data may be repeatedly updated.

[0077] An electronic device including an electronic circuit may be used to implement at least some of the embodiments described above (such as in combination with Figure 2 The apparatus may be comprised in at least one computing device connected to or integrated into a control system of a mining vehicle. Such a control system may be an intelligent onboard control system that controls the operation of various subsystems of the mining vehicle, such as hydraulic systems, motors, rock drilling tools, etc. Such control systems are typically distributed and include many independent modules connected by a bus system of, for example, Controller Area Network (CAN) nodes.

[0078] Figure 6 A simplified example device capable of supporting at least some embodiments of the present invention is shown. A device 60 is shown that can be configured to perform at least some embodiments related to the above-described obstacle avoidance, mining vehicle route planning and autonomous driving control related operations. In some embodiments, the device 60 includes or implements the control unit 20.

[0079] The device 60 includes a processor 61, which may include, for example, a single-core or multi-core processor. The processor 61 may include more than one processor. The processor may include at least one application-specific integrated circuit ASIC. The processor may include at least one field-programmable gate array FPGA. The processor may be configured at least in part by computer instructions to perform actions.

[0080] The device 60 may include a memory 62. The memory may include random access memory and / or permanent memory. The memory may be at least partially accessible by the processor 61. The memory may be at least partially included in the processor 61. The memory may be at least partially external to the device 60, but accessible by the device. The memory 62 may be a means for storing information affecting the operation of the device, such as parameters 64. In particular, the parameter information may include parameter information affecting, for example, route adaptation and related features, such as thresholds.

[0081] The memory 62 may be a non-transitory computer-readable medium that includes a computer program code 63, which includes computer instructions that the processor 61 is configured to execute. When computer instructions configured to cause the processor to perform certain actions are stored in the memory and the device as a whole is configured to run under the direction of the processor using the computer instructions from the memory, the processor and / or at least one of its processing cores may be considered to be configured to perform the certain actions. The processor may form, together with the memory and the computer program code, means for performing at least some of the above-described method steps in the device.

[0082] The device 60 may include a communication unit 65 including a transmitter and / or a receiver. The transmitter and the receiver may be configured to transmit and receive data and control commands, respectively, within or outside the mining vehicle. The transmitter and / or the receiver may be configured to operate according to, for example, a Global System for Mobile Communications GSM, a Wideband Code Division Multiple Access WCDMA, a Long Term Evolution LTE, a 3GPP New Radio Access Technology (N-RAT), a Wireless Local Area Network WLAN and / or an Ethernet standard. The device 60 may include a Near Field Communication NFC transceiver. The NFC transceiver may support at least one NFC technology, such as NFC, Bluetooth or similar technology.

[0083] The device 60 may include or be connected to a UI, such as a UI of the control unit 20 of the vehicle 10. The UI may include at least one of a display 66, a speaker, an input device 67 (such as a keyboard, a joystick, a touch screen and / or a microphone). The UI may be configured to display a view based on the embodiments described above. The user can operate the device and control at least some of the features described above. In some embodiments, in response to user authentication and appropriate permissions associated with the user, etc., the user can control the device and the vehicle 10 via the UI, such as manually driving the vehicle, changing the operating mode to autonomous driving and changing the operating mode from autonomous driving, changing the display view, modifying the parameters 64.

[0084] The device 60 may further include and / or be connected to additional units, devices and systems, such as one or more sensor devices that sense the environment of the device 60 and / or sensor devices that detect the position of the joint.

[0085] The processor 61, the memory 62, the communication unit 65 and the UI can be interconnected in a variety of different ways via electrical leads inside the device 60. For example, each of the aforementioned devices can be individually connected to a main bus inside the device to allow the devices to exchange information. However, as will be appreciated by those skilled in the art, this is only an example, and depending on the embodiment, various ways of interconnecting at least two of the aforementioned devices may be selected without departing from the scope of the present invention.

[0086] The device (such as the device 60, the mining vehicle 10 or the control unit 20) can be configured at least to: detect an obstacle based on a scan by a scanner of the mining vehicle through an obstacle detection function of the underground mining vehicle performing an autonomous driving task along a first route including a first set of route points; determine the position of the obstacle; in response to detecting the obstacle, determine an expected position that the vehicle is approaching before the obstacle so as to start a new route of the vehicle to the end of the first route; provide the expected position that the vehicle is approaching and the position of the obstacle to a route planning function, so that based on the expected position that the vehicle is approaching, the position of the obstacle and a site model indicating constraints for route generation, the route planning function generates a second route, the second route including a second set of route points for the mining vehicle to bypass the obstacle; provide the second route to an autonomous driving controller of the mining vehicle; and control the mining vehicle to autonomously drive along the second route by the autonomous driving controller.

[0087] The apparatus may be or be comprised in a mining vehicle.

[0088] The apparatus may be configured to: estimate an expected stopping position of the mining vehicle as the expected position based on the speed of the mining vehicle in response to detecting the obstacle; and transmit the expected stopping position as the expected position from the autonomous driving controller to the route planning function.

[0089] The device may be configured to determine three-dimensional position information of an obstacle in a worksite reference system based on a position of a scanner determined based on a three-dimensional environment scan.

[0090] The apparatus may be configured to update the worksite model to include information indicative of the location of the obstacle, and generate a second route based on the updated worksite model.

[0091] The apparatus may be configured to generate a second route based on a traversable or drivable map, wherein the traversable or drivable map includes a subset of points of the three-dimensional worksite model and indicates obstacles.

[0092] The apparatus may be configured to: check whether a detected obstacle will be in a safety area for the mining vehicle when the mining vehicle drives to a further route point in the first set of route points; and in response to detecting that the detected obstacle will be in the safety area, provide the position of the obstacle to the route planning function to generate a second route.

[0093] The apparatus may be configured to control a reduction in the speed of the mining vehicle in response to detecting an obstacle to avoid stopping the mining vehicle before the desired location and to start driving along the second route.

[0094] The apparatus may be configured to control an increase in speed when controlling the mining vehicle to drive along the second route.

[0095] Although the foregoing examples illustrate the principles of the present invention in one or more specific applications, it will be apparent to those skilled in the art that many modifications can be made in the form, use, and details of the implementation without exercising creative ability and without departing from the principles and concepts of the present invention as defined in the claims.

[0096] The verbs "comprise" and "include" are used in this document as open limitations that neither exclude nor require the presence of features not already recited. The features recited in the dependent claims are mutually freely combinable unless expressly stated otherwise. Furthermore, it should be understood that the use of "a" or "an" (i.e. the singular) throughout this document does not exclude a plurality.

Claims

1. An apparatus for controlling a mining vehicle, the apparatus comprising at least one processor and at least one memory, the at least one memory comprising computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the apparatus to at least perform: - detecting (200) obstacles by an obstacle detection function of an underground mining vehicle (10) performing an autonomous driving task along a first route comprising a first set of route points, based on scanning by a scanner (40) of the mining vehicle; - determining (210) the position of the obstacle; - in response to detecting the obstacle, determining (220) an expected stopping position that the vehicle is approaching before the obstacle in order to start a new route of the vehicle to an end point of the first route; - providing (230) the expected stopping position and the position of the obstacle that the vehicle is approaching to a route planning function; - generating (240) by the route planning function a second route comprising a second set of route points for the mining vehicle to circumvent the obstacle based on the vehicle approaching the expected stopping position, the location of the obstacle and a model of the site indicating limitations for route generation; - providing (250) the second route to an autonomous driving controller of the mining vehicle; and - controlling (260) the mining vehicle to drive autonomously along the second route by the autonomous driving controller.

2. The apparatus of claim 1 , wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the apparatus to: - in response to detecting the obstacle, estimating the expected stopping position of the mining vehicle on the basis of the speed of the mining vehicle; and - communicating the expected stopping position from the autonomous driving controller to the route planning function.

3. An apparatus according to any preceding claim, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, enable the apparatus to determine three-dimensional position information of the obstacle in a construction site reference system based on the position of the scanner determined based on a three-dimensional environment scan.

4. The apparatus of any preceding claim, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the apparatus to: - updating the site model to include information indicating the location of the obstacle; and - Generating said second route based on said updated site model.

5. An apparatus according to any preceding claim, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the apparatus to generate the second route based on a traversable or drivable map, wherein the traversable or drivable map includes a subset of points of a three-dimensional site model and indicates obstacles.

6. The apparatus of any preceding claim, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the apparatus to: - when the mining vehicle drives to a further waypoint in the first set of waypoints, checking whether the detected obstacle will be in a safe area for the mining vehicle; and - In response to detecting that the detected obstacle is to be within the safety area, providing the location of the obstacle to the route planning function to generate the second route.

7. Apparatus according to any preceding claim, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to control a reduction in the speed of the mining vehicle in response to detecting the obstacle to avoid stopping the mining vehicle before the expected stopping position and to start driving along the second route.

8. Apparatus according to any preceding claim, wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to control the increase in speed when controlling the mining vehicle to drive along the second route.

9. A mining vehicle (10) comprising an apparatus according to any preceding claim.

10. A method for controlling autonomous driving of a mining vehicle, the method comprising: - detecting (200) obstacles by an obstacle detection function of an underground mining vehicle (10) performing an autonomous driving task along a first route comprising a first set of route points, based on scanning by a scanner (40) of the mining vehicle; - determining (210) the position of the obstacle; - in response to detecting the obstacle, determining (220) an expected stopping position that the vehicle is approaching before the obstacle in order to start a new route of the vehicle to an end point of the first route; - providing (230) the expected stopping position and the position of the obstacle that the vehicle is approaching to a route planning function; - generating (240) by the route planning function a second route comprising a second set of route points for the mining vehicle to circumvent the obstacle based on the vehicle approaching the expected stopping position, the location of the obstacle and a model of the site indicating limitations for route generation; - providing (250) the second route to an autonomous driving controller of the mining vehicle; and - controlling (260) the mining vehicle to drive autonomously along the second route by the autonomous driving controller.

11. The method according to claim 10, comprising: - in response to detecting the obstacle, estimating the expected stopping position of the mining vehicle on the basis of the speed of the mining vehicle; as well as - communicating the expected stopping position from the autonomous driving controller to the route planning function.

12. The method according to claim 10 or 11, further comprising: - determining the three-dimensional position information of the obstacle in the construction site reference system based on the position of the scanner determined based on the three-dimensional environment scanning; - Updating the site model to include information indicating the location of the obstacle; as well as - Generating said second route based on said updated site model.

13. The method according to any one of claims 10 to 12, wherein: The second route is generated (240) based on a traversable or drivable map, wherein the traversable or drivable map includes a subset of points of the three-dimensional site model and indicates obstacles.

14. The method according to any one of claims 10 to 13, further comprising: In response to detecting the obstacle, a reduction in the speed of the mining vehicle is controlled to avoid stopping the mining vehicle before the expected stopping position and to start driving along the second route.

15. A computer program comprising code for causing the method according to any one of claims 10 to 14 to be performed when the code is executed in a data processing device.

Images