Obstacle detection for mining vehicles

Abstract

According to the invention, there is provided a method comprising: assigning (200) a set of obstacle analysis parameters for a mining vehicle (10), where the set of obstacle analysis parameters is specific to a portion of the mining vehicle; detecting (210) an obstacle candidate by an obstacle detection function of the mining vehicle on the basis of processing scan data from a scanner of the mining vehicle; performing (220) an obstacle analysis on the basis of the determined size of the obstacle candidate and the set of obstacle analysis parameters assigned for the mining vehicle to classify the obstacle candidate and to determine whether the obstacle candidate is an obstacle to avoid; and in response to the obstacle analysis indicating that the obstacle candidate is an obstacle to evade, providing (230) an alert or control signal to avoid the mining vehicle from impacting the obstacle.

Description

Technical Field

[0001] The present invention relates to obstacle detection, object analysis and related control by mining vehicles. Background Art

[0002] A mining or construction excavation site, such as an underground hard rock or soft rock mine, may include an area for automated operation of mining vehicles, such as loaders and / or haulers and drills. Such mining vehicles may include, for example, unmanned vehicles that are remotely controlled from a control room, or manned vehicles, such as vehicles operated by an operator in the cab of the vehicle.

[0003] The mining vehicle may be configured to perform at least some tasks autonomously.An automated mining vehicle operating in an autonomous mode may be configured to operate independently, but may be taken over under external control in certain operating areas or conditions, such as during a state of emergency.

[0004] The route from the starting point to the end point can be pre-planned for autonomous driving. In some systems, an experienced operator can teach the optimal path by driving the vehicle, or the route can be designed at least partially automatically based on input parameters and a site model. Conditions in an underground site may change during mining operations and affect driving performance along the route. For example, loose rock may cause the stop of an autonomous mining vehicle and manual control by an operator. This production interruption has a negative impact on the efficiency of mining operations at the site. The safety system of the mining vehicle should be configured to prevent the mining vehicle from hitting obstacles that cause damage or stop the mining vehicle, but to avoid unnecessary stops of the mining vehicle due to small new obstacles. Summary of the invention

[0005] The invention is defined by the features of the independent claims. Some particular embodiments are defined in the dependent claims.

[0006] According to a first aspect, a device is provided, the device being configured to perform at least the following operations, or the device comprising means configured to perform at least the following operations: assigning an obstacle analysis parameter set to a mining vehicle, wherein the obstacle analysis parameter set is specific to a part of the mining vehicle; detecting obstacle candidates by an obstacle detection function of the mining vehicle based on processing scan data from a scanner of the mining vehicle; performing obstacle analysis based on a determined size of the obstacle candidate and the obstacle analysis parameter set assigned to the mining vehicle to classify the obstacle candidate and determine whether the obstacle candidate is an obstacle to be avoided; and providing an alarm or control signal in response to the obstacle analysis indicating that the obstacle candidate is an obstacle to be avoided to avoid the mining vehicle from colliding with the obstacle.

[0007] According to a second aspect, a method for mining vehicle route planning is provided, the method comprising: assigning an obstacle analysis parameter set to a mining vehicle, wherein the obstacle analysis parameter set is specific to a portion of the mining vehicle; detecting obstacle candidates by an obstacle detection function of the mining vehicle based on processing scan data from a scanner of the mining vehicle; performing obstacle analysis based on a determined size of the obstacle candidate and the obstacle analysis parameter set assigned to the mining vehicle to classify the obstacle candidate and determine whether the obstacle candidate is an obstacle to be avoided; and providing an alarm or control signal in response to the obstacle analysis indicating that the obstacle candidate is an obstacle to be avoided to avoid the mining vehicle from hitting the obstacle.

[0008] According to a third aspect, a device is provided, comprising at least one processor and at least one memory, wherein the at least one memory comprises computer program code, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, enable the device to at least perform the method or an embodiment of the method.

[0009] According to a fourth aspect, a computer program, a computer program product or a (non-tangible) computer readable medium comprising computer program code which, when executed in a data processing apparatus, causes the apparatus to perform the method or embodiments of the method is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 An example of a mining vehicle is shown;

[0011] Figure 2 A method according to some embodiments is shown;

[0012] Figure 3 illustrates the functionality of a control unit according to some embodiments;

[0013] Figure 4 An example of a top view of a mining vehicle performing a driving task is shown;

[0014] Figure 5 illustrates the functionality of a control unit according to some embodiments;

[0015] Figure 6 shows an example of a top view of a mining vehicle performing a driving task; and

[0016] Figure 7 An example device is shown that can support at least some embodiments. DETAILED DESCRIPTION

[0017] Figure 1 A simplified example of a portion of a mining site 1 (comprising an underground tunnel portion or tunnel 2 in this example) is shown. A mining site may comprise a mine or a construction site, such as a railway or road tunnel site. However, it will be appreciated that a site may also comprise an area on the surface of the ground. The term underground mining vehicle herein generally refers to a mobile working machine suitable for use in at least the operation of different kinds of underground mining or construction excavation sites, such as a truck, a bulldozer, a dumper, a van, a mobile rock drilling or milling drill, a mobile reinforcement machine and a bucket loader or other kinds of mobile working machines that may be used in different kinds of excavation sites.

[0018] In this example, the mining vehicle 10 is a loader or load and haul (LHD) vehicle including a bucket 11 connected to a boom 12. The vehicle 10 may be, for example, an articulated vehicle including two body parts connected by a joint 13, which may be referred to as a front body part (including the bucket 11 and the boom 12) and a rear body part. However, it should be understood that the application of the features disclosed herein is not limited to any particular type of vehicle that may be used at an excavation site. Some other examples of such vehicles include inspection vehicles, autonomous inspection vehicles (AIVs), trucks, dump trucks, vans, mobile rock drilling or milling rigs, or mobile reinforcement machines.

[0019] exist Figure 1 In the example of , the vehicle 10 includes a system 14 consisting of a pump for generating hydraulic pressure to operate various parts of the machine, such as lifting the boom 12, rotating the bucket 11, etc. The vehicle 10 may include one or more other energy sources, such as accumulators, hydrogen containers, fuel tanks, etc. The vehicle 10 may include a motor 15, such as a combustion engine or an electric motor. The power from the motor 15 can be provided to the front and / or rear wheels directly by the crankshaft or via a gearbox. The drive train of the vehicle 10 may include, for example, a motor 15, a crankshaft and / or a transmission.

[0020] The vehicle 10 includes at least one control unit 20, which is configured to control at least some functions and / or actuators of the vehicle. The control unit 20 may include one or more computing units / processors that execute computer program codes stored in a memory. The control unit 20 may be connected to one or more other control units of the control system of the vehicle, for example, via a controller area network (CAN) bus. The control unit 20 may include or be connected to a user interface (UI) having a display device and an operator input interface. Such an interface may be used to receive operator commands and information to the control unit. It should be understood that the control unit 20 may be configured to perform at least some of the control features shown below related to obstacles, or multiple control units or controllers may be applied to perform these features. It should be understood that at least some of these control functions may even be implemented outside the vehicle (e.g., at a site management system).

[0021] The vehicle 10 may comprise a wireless communication device 30 by means of which the control unit 20 of the vehicle 10 and / or another unit of the control system may be configured to establish a data connection. The data connection comprises at least one of data reception and / or data transmission. The data connection may be formed between the control system of the vehicle and at least one control device of a (mining operation) control system 6 external to the vehicle. The data connection may make use of a wireless connection provided by a base station or access node 4. Thus, the communication device 30 may be operably connected to a communication system of the worksite, such as a wireless access system comprising a wireless local area network (WLAN) and / or a cellular communication network (such as a 4G, 5G or some other generation of cellular network).

[0022] The control system 6 may include or be connected to additional networks and / or data processing systems, such as a site management system, a cloud service, a data analysis system, an intermediate communication network, such as the Internet, etc. The control system 6 may include or be connected to additional devices or control units, such as a handheld user unit, a vehicle unit, a site management device / system, a remote control and / or monitoring device, a data analysis device, a sensor system / device, etc.

[0023] The vehicle 10 may be unmanned. An unmanned vehicle may include a user interface to allow a user to control any function of the vehicle, such as steering. In addition to or as an alternative to the user interface, the vehicle may include an operator input interface. The interface of the vehicle may be located in a position away from the vehicle, and the vehicle may be remotely controlled. Examples of remote control include: controlled by an operator in a tunnel, or controlled by a device in a control room at a construction site, or even controlled by a device from a long distance away from the construction site via a communication network. The vehicle 10 may be an automated vehicle that can be independently operated / driven in an autonomous operation or driving mode without continuous user control. Such an automated vehicle may be taken over under external control, for example, during a state of emergency. When the vehicle is in manual driving mode, the operator manually drives the vehicle by remote control, or drives the vehicle locally by operator control. The operator may set the vehicle to a (default) automatic driving mode, in which the vehicle is configured to automatically drive a designated route (e.g., between a loading point and a dumping lane).

[0024] Equipment at the worksite, such as the control system 6 and the control means of the mining vehicle 10, may be configured to store and use at least one worksite model representing a current state and / or a target state of the worksite environment. The worksite model may be an environment model or a map. For example, the worksite model may be a 2D model or a 3D model indicating surface contours and obstacles at the worksite at the area operated by the mining vehicle. In the case of an underground worksite, the worksite model may be a tunnel model indicating the contours of the tunnel.

[0025] In some embodiments, the control system 6 and / or the mining vehicle 10 is configured to store a 3D (tunnel) model of the underground site, which shows the pit floor, pit walls and pit roof of the tunnel network. The 3D model may include point cloud data generated on the basis of the scan or be formed based on these point cloud data. In some other embodiments, the 3D model may be stored in some other format (such as a mesh model including vertices, edges and faces). In some embodiments, the 3D model may be a design model, or may be generated based on a design model (such as a CAD model) created by mining design software or a 3D model created based on tunnel lines and contours designed in drilling and blasting design software.

[0026] Vehicle 10 may include one or more scanning units or scanners 40 configured to perform a scan of the environment surrounding the vehicle. In some embodiments, the scan results are applied to detect the position and orientation of the vehicle and one or more other elements thereof, such as the position and orientation of scanner 40 or bucket 11.

[0027] The control unit 20 may be configured to compare the scanned tunnel profile data with reference profile data stored in the site model. The control unit 20 may be configured to locate the mining vehicle on the basis of finding a match between the surface profile indicated by the scan data and the surface profile indicated by the site model. The control unit 20 may be configured to correct the positioning by dead reckoning positioning based on the scanning-based positioning. In some embodiments, the scanning results are applied to detect the position and orientation of the mining vehicle and one or more further elements thereof, such as the position of the scanner 40 or the bucket 11.

[0028] The scanner 40 may be a 3D scanner configured to generate 3D scan data or point cloud data. For example, the 3D scanner may include a light detection and ranging (lidar) detector device, a radar device, or a camera-based detector device.

[0029] A driving plan or route plan may define a route to be driven by the vehicle 10 and may be used as an input to the autonomous driving control of the vehicle. In addition, such a plan may be used as part of a mission or a planned mission. When a mission includes a route plan, the plan may form a route to be driven during the mission. The plan may be generated offline and off-site (e.g., in an office). The plan may be generated on the vehicle, for example, by teaching driving. The plan may define a starting point, an end point, and a set of route points for autonomous driving.

[0030] The mining vehicle 10 may include an obstacle detection / avoidance system or functionality. The obstacle detection system is configured to monitor the driving path of the mining vehicle to detect obstacles 50. The mining vehicle needs to be configured to avoid collisions with obstacles that may cause danger and / or damage. Small objects may be ignored, but dangerous objects require a warning or alarm signal that causes the autonomous vehicle to stop.

[0031] Improvements are now provided to facilitate improved object handling of a mining vehicle through object analysis configured to classify objects based on obstacle analysis parameters that are specific to the mining vehicle portion.

[0032] Figure 2 A method according to some embodiments is shown. The method may be performed by a device, which in some embodiments is a control device configured to control autonomous driving related operations of a mining vehicle. The method may be performed by a mining vehicle and its control device (such as a mining vehicle 10) and its control unit 20, respectively. The method may be a computer-implemented method.

[0033] Box 200 includes assigning an obstacle analysis parameter set to a mining vehicle. The obstacle analysis parameter set is specific to a portion of the mining vehicle. The parameters may be dynamically defined during driving of the mining vehicle. When entering box 200, the mining vehicle may perform an autonomous driving task.

[0034] Block 210 includes detecting obstacle candidates by an obstacle detection function of the mining vehicle based on processing scan data from a scanner of the mining vehicle.

[0035] Block 220 includes performing an obstacle analysis based on the determined size of the obstacle candidate and the set of obstacle analysis parameters assigned to the mining vehicle to classify the obstacle candidate and determine whether the obstacle candidate is an obstacle to be avoided.

[0036] Block 230 includes providing an alert or control signal to prevent the mining vehicle from striking the obstacle in response to the obstacle analysis indicating that the obstacle candidate is an obstacle to be avoided. For example, the control signal may be an alert or stop signal to an autopilot controller configured to control or cause a stop or deceleration of the mining vehicle in response to the control signal.

[0037] The obstacle candidate may be an object detected in the scan data or a set of measurements selected from the measurements of the scan data by an obstacle candidate detection function. Determining the size may involve determining at least some size or reach of the obstacle in a given direction that may affect or prevent the drivability of the mining vehicle. The device performing the method may be configured to determine the size in box 220 or in another box between box 210 and box 220. This may include determining the vertical size or height of the obstacle. Determining the size may also include determining the horizontal size or width of the object. In a simple example, the obstacle analysis may include comparing the at least one determined size or dimension with at least one threshold parameter value assigned to the mining vehicle portion. If the threshold parameter value is exceeded, the device may be configured to assign an associated classification to the obstacle, and if box 230 is configured for the assigned classification, box 230 is performed according to the input. The autonomous driving task may involve route control information provided to the mining vehicle, such as the start and end points of the route.

[0038] The obstacle analysis parameter set may be vehicle type specific. The parameters may be assigned or adapted based on user input via a user interface of the mining vehicle 10 or a user interface of the control system 6. Additionally or alternatively, the parameters may be assigned or adapted dynamically and automatically during the driving task based on input data and parameter assignment or adaptation criteria, some examples of which are further illustrated below.

[0039] The obstacle analysis parameter set may include an adaptable trigger condition set defining two or more obstacle severity categories (or classifications) to control the triggering of associated actions due to obstacles. In a simple example, obstacles may be classified in block 220 as: to be avoided; to be warned; or to be ignored. Each category may be associated with at least one trigger parameter value (such as a threshold or range for the size of the obstacle).

[0040] The severity categories may be defined by obstacle analysis parameters specific to the mining vehicle part. Alternatively, the obstacle analysis parameters may be applied without being specific to the mining vehicle part. Figure 2 In such an embodiment, the obstacle analysis parameters may be specific to a mining vehicle and define two or more obstacle severity categories.

[0041] Adaptable trigger conditions and obstacle severity categories may be associated with different output actions. In a simple example, a first criterion may be defined by a first threshold, exceeding which triggers a stop or emergency signal in box 230. A second criterion may be defined by a second threshold, exceeding which triggers a warning or deceleration control signal in box 230. These thresholds may be associated with obstacle size. Thus, box 220 may include a set of comparisons of obstacle size with thresholds. For example, if the obstacle size does not exceed the first threshold, the program compares the size with the second threshold. If none of the thresholds determined by the obstacle analysis parameter set (triggering box 230) are exceeded, the mining vehicle may continue driving without interruption. The obstacle analysis program may return to process new scan data and additional obstacle candidates and return to box 210.

[0042] The device may be configured to perform 220 an obstacle analysis based on at least one of the following parameters: a height of the obstacle candidate; an area of ​​a cross section of the obstacle candidate; or another size of the obstacle candidate. Thus, the determination of the size may include, for example, calculating the cross-sectional area in a direction from the scanner based on point cloud data indicating measurements of the impact obstacle.

[0043] In addition to the obstacle size, other inputs and obstacle analysis parameters may also be applied. For example, the slope of the surface ahead in the driving direction of the mining vehicle (e.g., ahead of the obstacle candidate). For example, the slope of the surface may be defined based on a plane set along the pit bottom points detected in the scan data based on analyzing the 3D point cloud data.

[0044] In some embodiments, the scan data includes point cloud data, which can be generated based on a scan of an environment by a 3D scanner. The device can be configured to select a set of measurement value points for obstacle candidate detection and / or obstacle analysis based on the measurement values ​​in the scan data, which in some embodiments are points in the point cloud. The selected measurement value set can be further processed during obstacle analysis. In an example embodiment, adjacent measurement value points of the point being evaluated or analyzed are selected and analyzed. For example, adjacent points within a radius (from the analyzed point) selected from 10-100 cm (20-50 cm in another embodiment) are selected from the measurement value point set for analysis.

[0045] The apparatus may be configured to apply a vehicle portion specific obstacle height threshold.Block 230 may be entered in response to detecting that the height of the detected obstacle exceeds the obstacle height threshold.

[0046] In a simple obstacle candidate analysis example, the device can be configured to compare the defined vertical positions of the measurement points in the selected measurement set. The lowest measurement point can be selected to represent the pit bottom or ground level. The vertical positions of the other points in the group are analyzed and compared with the vertical position of the lowest measurement point / level. If the difference between one or more points and the lowest point / level exceeds the obstacle avoidance height threshold (which is defined in the obstacle analysis parameter set), the obstacle is determined to be an obstacle to be avoided and enter box 230. It should be noted that there can be multiple such obstacle height thresholds to classify obstacles into different categories or severity categories.

[0047] The device may be configured to adapt obstacle analysis parameters within a preconfigured range and apply minimum and / or maximum values ​​that cannot be exceeded to avoid setting obviously dangerous or erroneous conditions. For example, for the type of mining vehicle in question, a non-adaptable maximum value may be preconfigured for obstacle height. In this example, the system may be configured to prevent assignment of an obstacle height parameter in block 200 to trigger an alarm or stop signal above a preconfigured maximum obstacle height.

[0048] Features of the present invention facilitate adapting, for example, trigger points based on the current worksite environment and / or vehicle state to determine the obstacle category if the obstacle candidate is a dangerous obstacle. By adapting the trigger points to tolerate larger obstacles that still do not damage the vehicle, unnecessary alarms and production interruptions due to small obstacles can be reduced. Features of the present invention facilitate adapting obstacle analysis and its results for different parts of a mining vehicle. For example, the obstacle analysis can be adapted to a work tool used in the vehicle, or to different articulated body parts of the vehicle. A detected object can be classified based on its relevance to a specific part of the mining vehicle. Object classification can be performed separately for multiple mining vehicle parts. If the result of at least one of the object analysis procedures is that an obstacle needs to be avoided, then enter block 230. The obstacle detection system can also adapt to changes in worksite environmental conditions (such as road conditions) based on adaptable alarm trigger levels or conditions.

[0049] The part of the mining vehicle may be a work tool of the mining vehicle. Examples of such work tools include a bucket 11 and / or boom 12 of the vehicle 10, a drilling boom or drilling unit of a drilling rig, or a platform of a dumper.

[0050] An apparatus performing the method, such as the control unit 20, may be configured to assign the set of obstacle analysis parameters to the mining vehicle on the basis of the determined position of the work tool relative to the body of the mining vehicle.

[0051] The device may be configured to determine the position of the work tool relative to the body of the vehicle based on information from one or more sensors sensing the orientation of the work tool, or based on work tool status data of a vehicle control system. In another example, the device is configured to define the position of the work tool relative to the body based on analysis of scan data (such as image data or point cloud data) indicating reference points of the work tool and reference points of the body.

[0052] The device may be configured to adapt vehicle-part-specific obstacle size parameters based on the determined vertical position of the work tool relative to the body of the mining vehicle. Thus, one or more thresholds for obstacle classification, such as obstacle height parameters, may depend on the distance between a reference point of the work tool and a reference point of the body and be adapted based on the distance between the reference point of the work tool and the reference point of the body. The distance between the reference point of the work tool and the reference point of the body may comprise a vertical distance. For example, the thresholds are adapted in response to moving the bucket 11 to or from an extreme position of the bucket.

[0053] The apparatus may be configured to determine driving state data indicative of current driving parameters of the mining vehicle. In block 200, a set of obstacle analysis parameters for the mining vehicle may be dynamically assigned or adapted based on the determined driving state data of the mining vehicle. The driving state data may include at least one of a speed of the mining vehicle and a driving direction of the mining vehicle. The driving state data may be received from a control system of the mining vehicle, such as from an autonomous driving controller via a bus system of the vehicle.

[0054] In some embodiments, when performing an autonomous driving task, the device may be configured to detect path conditions during driving. In block 200, a set of obstacle analysis parameters may be dynamically assigned or adapted based on the detected path conditions. Thus, the device may be configured to enter or return to block 200 in response to detecting a triggering path condition. The path conditions may include one or more roadway or tunnel pit surface conditions, such as a soft or wet path, or a rutted path. The path conditions may be detected based on an environmental scan performed by a scanner and / or other sensor of the mining vehicle.

[0055] refer to Figure 3 The mining vehicle 10 may include an obstacle detection function (ODF), an obstacle control function (OCF) and an autonomous driving controller (ADC), which may also be referred to as an autonomous driving control function. The control unit 20 may be configured to perform at least some of these functions. Alternatively, multiple control units may be applied, such as a separate control unit for each function. It should be understood that there may also be additional functions related to autonomous driving, such as a positioning function.

[0056] The ODF may be configured to receive scan data from the scanner 40 or from a scanner application or function that provides scan data based on a scan of the environment performed by the scanner. Figure 4 The scanning data may be based on a scanning signal set 400 of a scanner. The scanning data may indicate that an obstacle 50 exists in a driving direction D on a route of an autonomous driving task, the route including a set of route points 420 to an end point 430.

[0057] The ODF and OCF may be part of (or operably connected to) a collision avoidance system (CAS). The ODF may be configured to perform a collision check based on scan data received from at least the scanner 40. In another embodiment, the OCF is configured to: perform a collision check based on an indication of a detected obstacle from the ODF, and control further actions based on the collision check. The scan data may come from multiple scanners. For example, one scanner may cover the rear of the vehicle, and another scanner may cover the front of the vehicle with a directional beam.

[0058] The ODF may be configured to process the scan data, which may indicate the distance to or location of the tunnel portion and further obstacles, to detect obstacles in a detection area ahead in the driving direction D of the vehicle. To detect obstacles, the scan data may be processed in or before block 210. The processing may be based on obstacle detection control parameters, such as control parameters that control a detection area and / or a safety area defining a vehicle surrounding.

[0059] The device (such as ODF or OCF) may be configured to apply one or more obstacle detection or safety zones around the vehicle.At least one safety zone or envelope 440 may be defined on at least one side of the vehicle 10 . Figure 4 A first safety envelope 440 is shown assigned to a first / front body portion of the vehicle, and a second safety envelope 442 is assigned to a second / rear body portion.

[0060] The ODF or OCF may be configured to determine or predict whether an obstacle candidate will fall into a safe area when the vehicle follows the route based on the locations of at least some of the route points 420, the obstacle detection parameters, and the determined location of the obstacle 50. If so, block 210 may be entered. If not, the ODF may continue to process new scan data. However, it should be noted that obstacle candidate detection need not involve checking based on future route points, but rather objects or measurement point sets may be detected as obstacle candidates for further analysis based on processing scan data indicating the object / measurement point set without using information about the future path of the vehicle.

[0061] In an example embodiment, the device is configured to apply a warning zone and a collision zone or envelope. For example, if an obstacle to be avoided is detected in the warning zone, a warning may be issued and the speed of the vehicle may be reduced. If an obstacle to be avoided is detected in the collision zone, the vehicle is about to hit the obstacle or is too close to the obstacle and further safety action is required.

[0062] The OCF may be configured to determine whether an obstacle candidate needs to be avoided or bypassed based on the information of the obstacle received from the ODF. The OCF may be configured to control or cause the stopping of the mining vehicle in response to detecting an object classified as an obstacle to be avoided. The OCF may be configured to execute box 220 and classify the obstacle candidate based on the obstacle analysis parameter set. This may be performed when an obstacle candidate that will fall into the safety zone of the vehicle (or a part of the vehicle) is detected. In a simple example, the OCF may be configured to compare the vertical z-direction size of the obstacle candidate detected by the ODF with the threshold parameter associated with the vehicle part. If the threshold is exceeded, the obstacle candidate is classified as an obstacle to be avoided (for the vehicle part in question). If the obstacle candidate is classified as an obstacle to be avoided, box 230 may be entered. In another embodiment, the OCF defines the size of the obstacle candidate based on the obstacle information from the ODF. In yet another embodiment, the ODF decides whether to enter box 230 based on the obstacle analysis information from the ODF. The ODF may further determine whether or which alarm or control signal is caused in response to the obstacle classification.

[0063] The apparatus may be configured to adapt or assign the vertical (z) reach of the safety zone / envelope 440, 442 based on a determined vertical position of the work tool relative to the body of the mining vehicle 10. In one embodiment, pit wall monitoring is turned on or off depending on the vertical position so determined. Thus, the safety zone height (and therefore the monitoring area height) may depend on the distance between the reference point of the work tool and the reference point of the body, and be adapted based on the distance between the reference point of the work tool and the reference point of the body. The distance between the reference point of the work tool and the reference point of the body may include a vertical distance. For example, the safety zone height (and therefore the monitoring area height) is increased in response to raising the bucket 11. Thus, the bucket may define the uppermost portion of the vehicle and define a vertical upper limit for monitoring obstacles and their analysis.

[0064] The apparatus may be configured to adapt or assign the horizontal reach of the safety zone / envelope 440, 442 based on the determined driving state of the mining vehicle. For example, the width of the safety zone may be increased in response to increasing the speed of the vehicle.

[0065] In or before block 210, the position of the obstacle candidate may be determined in a worksite reference system (in some embodiments, a global coordinate system). The ODF may be configured to determine 3D position information of the obstacle. The ODF may be configured to determine the position of the obstacle relative to the scanner 40. As described above, the scanner 40 may be positioned based on a scan of the environment. Then, the position of the obstacle in the global coordinate system may be determined based on the determined position relative to the scanner.

[0066] The apparatus may be configured to generate a new or updated route for the mining vehicle to circumvent the obstacle in response to the obstacle analysis indicating that the obstacle candidate is an obstacle to be avoided or in response to the control signal. The new or updated route may be generated based on the vehicle's position information, the location of the obstacle, and a site model indicating constraints to route generation.

[0067] Figure 5 It is shown that the OCF may be connected to a route planning function (RPF). The OCF may be configured to connect to the RPF in or after block 230 in response to detecting an obstacle to be avoided. The OCF may transmit a control signal that triggers the RPF to determine a new route around the obstacle, if possible. Thus, a route recalculation may be attempted to avoid the obstacle rather than causing the mining vehicle to stop immediately.

[0068] The vehicle's RPF may be configured to define waypoints for the vehicle to follow in order to perform driving tasks and navigate around detected obstacles.

[0069] In some example embodiments, the ADC is configured to control navigation of the vehicle along the route based on the route points provided by the RPF. The ADC may be configured to define steering control signals to other units or actuators of the mining vehicle based on position information of the route points, such as x and y coordinates in a global coordinate system. Thus, the ADC may define and provide control signals after block 230 such that the mining vehicle autonomously drives along the route points in order to avoid obstacles during autonomous driving toward the destination point. Route planning may also include other parameters that the ADC may be configured to control.

[0070] Figure 6 The vehicle is shown after having successfully generated a new / second route having a new / second set of route points 600 after block 230 for the vehicle initially driving the first route indicated by route points 420. At least some of the route points in set 600 are positioned so that the vehicle will pass obstacle 50 without the obstacle falling into safety zones 440, 442.

[0071] This makes it possible to avoid obstacles by dynamic rerouting performed on the vehicle immediately when the obstacle is detected. Therefore, it may be possible to avoid stopping the vehicle, and instead the vehicle can continue to drive (possibly at a reduced speed) until a new route is calculated and put into operation. For example, delays that affect production efficiency can be reduced because the need for manual intervention and control by an operator can be avoided.

[0072] It should be understood that there are several implementation options for generating a new route and new route points after block 230. The second route may be an entirely new route defined further based on the route endpoint location information after block 230. The RPF may be configured to calculate a second set of route points from the vehicle's location to the endpoint of the first route.

[0073] In another embodiment, only a partial route regeneration is performed after block 230. The second set of route points may include new route points that replace route points in the first set of route points that are proximate to obstacles.

[0074] The RPF may be configured to generate route points further based on vehicle attribute parameter data. For example, the vehicle attribute parameters may include vehicle dimension parameter data indicating at least the dimensions of the vehicle and / or safety parameters such as the size of a required safety or collision zone at the side of the vehicle. The RPF may be configured to generate route points at predetermined intervals (e.g., 0.5 m).

[0075] Based on the vehicle attribute parameter data, the RPF can be configured to determine the positions of the route points so that they allow sufficient safety margins between one side of the vehicle and the obstacle and between the other side of the vehicle and the tunnel pit wall (or obstacle) on the other side. If no such route point is found, the RPF can be configured to indicate this to the ADC and / or to the user interface to notify the operator. The ADC can be configured to control the stopping of the vehicle in response to such an indication. The ADC or the RPF can be configured to indicate the interruption of the driving task and the stopping of the vehicle to the operator via a UI (e.g., a UI of a control system). Further information of the detected obstacles can be provided to the UI, such as a 2D or 3D UI view indicating the position and size of the obstacles in the site model.

[0076] The position information of the mining vehicle used for the new route calculation may be the current position of the mining vehicle or the future expected position of the mining vehicle. The RPF may be configured to select the current position or the expected position as the starting point for the new route calculation.

[0077] The determined indication of the obstacle height may be applied in the new route generation, for example, the corresponding point may be omitted from the available path options, or the availability of the point may be determined based on the height indication.

[0078] As an example, consider a vehicle with a width W (e.g., 2 meters) and with a ground clearance value G (e.g., 20 centimeters). The device may be configured to define a subset of the pit bottom model for path planning, in which the vehicle can be driven without the risk of hitting an obstacle. The subset can be found from the pit bottom model enhanced with a local obstacle height h. The device may be configured to: for each point p in the pit bottom model, define a point at a radius of W / 2 based on the width W of the vehicle in question. The device may be configured to assign such a point zero to the driving performance of the vehicle if any point in the radius has an obstacle height h greater than the ground clearance G. If no point has an obstacle height higher than G, the device may be configured to assign true or one to the driving performance of the point. The result may be a binary map indicating the driving performance of the vehicle.

[0079] Devices such as the control unit 20 and further ADCs may be configured to respond to detecting 210 an obstacle and / or control a reduction in the speed of the mining vehicle in block 230. For example, the ODF or OCF may be configured to transmit an indication of an obstacle or a request to slow down to the ADC in response to blocks 210 or 230. In response to such an indication or request, the ADC may be configured to reduce the speed to a predefined target speed.

[0080] A device such as the control unit 20 may be configured to update a site model based on the scan data from the scanner 40. The device may be configured to: in response to the obstacle analysis indicating that the obstacle candidate is an obstacle to be avoided, update the site model for autonomous driving of the mining vehicle to include information indicating the location of the obstacle. A second route may be generated based on the updated site model.

[0081] The vehicle 10 may include a simultaneous localization and mapping (SLAM) unit. Such a unit is configured to localize the vehicle and (enhance) map the environment, such as geographical features of the environment, based on (2D or 3D) scanning information while the vehicle is driving, for example in an autonomous mode. The SLAM unit may be configured to update the site model based on detected obstacles.

[0082] The device can be configured to apply a driving performance map (which may also be referred to as a traversable map) as a site model (for generating a new route and including information about obstacles). The driving performance map includes driving performance data indicating the driving performance of the vehicle at a set of locations in the tunnel. The driving performance map is based on a subset of points of the 3D site model. The driving performance map (which may also be referred to as a driving performance model) includes a plurality of points enhanced with driving performance data. The RPF can be configured to generate a new route for the vehicle based on the driving performance map. The RPF can be configured to: in response to an obstacle analysis indicating that an obstacle candidate is an obstacle to be avoided, update the driving performance map to include information about the obstacle.

[0083] In some embodiments, the site model includes or is based on a pit bottom model generated on the basis of the 3D site model. The term pit bottom model generally refers to a model that includes a set of points indicating the tunnel pit bottom at least in a horizontal plane (i.e., 2D or x, y coordinates). Such points may also be referred to as pit bottom points. A more efficient path calculation or position tracking of a vehicle may be achieved by including only a map of the pit bottom of a mine, wherein driving performance attributes are associated with some or all of the pit bottom points. The driving performance map may include pit bottom point records of the pit bottom points of the pit bottom model, wherein at least some of the pit bottom point records include enhanced driving performance data for the vehicle. The device may be configured to apply the driving performance map to vehicle path definition and / or maneuvering control along the pit bottom defined by the pit bottom points.

[0084] The driving performance data may include vehicle-specific driving performance data specific to an automated or semi-automated vehicle or vehicle type. The route and / or driving performance data may be generated based on vehicle attribute data indicating at least the dimensions of a vehicle of that type. Appropriate vehicle attribute data may be retrieved from a memory or another device based on a received vehicle or vehicle type identifier for driving performance map generation and / or route planning. For example, a given family or product family of loading & hauling equipment may have certain common dimensions that are indicated in the associated vehicle attribute data and used to generate driving performance data and / or route information for those vehicles.

[0085] The drivability map may indicate points in a subset of points of the worksite model that are drivable by the vehicle or vehicle type. For example, the drivability map may include a subset of points that are drivable by the vehicle or vehicle type (such as a pit bottom point), or an indication of the drivability (drivable or non-drivable) of the vehicle or vehicle type is included in at least some points of the drivability map.

[0086] In some embodiments, the height of an obstacle (defined as an obstacle to be avoided) is included in the driving performance data or used to generate the driving performance data. The height of the obstacle may be defined in or before block 220 .

[0087] An electronic device including an electronic circuit may be used to implement at least some of the embodiments described above (such as in combination with Figure 2 The apparatus may be comprised in at least one computing device connected to or integrated into a control system of a mining vehicle. Such a control system may be an intelligent onboard control system that controls the operation of various subsystems of the mining vehicle, such as hydraulic systems, motors, rock drilling tools, etc. Such control systems are typically distributed and include many independent modules connected by a bus system of, for example, Controller Area Network (CAN) nodes.

[0088] Figure 7 A simplified example device capable of supporting at least some embodiments of the present invention is shown. A device 70 is shown that can be configured to perform at least some embodiments related to the obstacle handling related operations described above. In some embodiments, the device 70 includes or implements the control unit 20.

[0089] A processor 71 is included in the device 70, which may include, for example, a single-core or multi-core processor. The processor 71 may include more than one processor. The processor may include at least one application-specific integrated circuit ASIC. The processor may include at least one field programmable gate array FPGA. The processor may be configured at least in part by computer instructions to perform actions.

[0090] The device 70 may include a memory 72. The memory may include random access memory and / or permanent memory. The memory may be at least partially accessible by the processor 71. The memory may be at least partially included in the processor 71. The memory may be at least partially external to the device 70, but accessible by the device. The memory 72 may be a means for storing information affecting the operation of the device, such as parameters 74. In particular, the parameter information may include obstacle analysis parameters and further parameter information affecting, for example, obstacle detection and autonomous mining vehicle control related features, such as thresholds.

[0091] The memory 72 may be a non-transitory computer-readable medium including a computer program code 73, which includes computer instructions that the processor 71 is configured to execute. When computer instructions configured to cause the processor to perform certain actions are stored in the memory and the device as a whole is configured to run under the direction of the processor using the computer instructions from the memory, the processor and / or at least one of its processing cores may be considered to be configured to perform the certain actions. The processor may form, together with the memory and the computer program code, means for performing at least some of the above-described method steps in the device.

[0092] The device 70 may include a communication unit 75 including a transmitter and / or a receiver. The transmitter and the receiver may be configured to transmit and receive data and control commands, respectively, within or outside the mining vehicle. The transmitter and / or the receiver may be configured to operate according to, for example, a Global System for Mobile Communications GSM, a Wideband Code Division Multiple Access WCDMA, a Long Term Evolution LTE, a 3GPP New Radio Access Technology (N-RAT), a Wireless Local Area Network WLAN and / or an Ethernet standard. The device 70 may include a Near Field Communication NFC transceiver. The NFC transceiver may support at least one NFC technology, such as NFC, Bluetooth or similar technology.

[0093] The device 70 may include or be connected to a UI, such as a UI of the control unit 20 of the vehicle 10. The UI may include at least one of a display 76, a speaker, an input device 77 (such as a keyboard, a joystick, a touch screen and / or a microphone). The UI may be configured to display views based on the embodiments described above. The user can operate the device and control at least some of the features described above. In some embodiments, in response to user authentication and appropriate permissions associated with the user, etc., the user can control the device 70 or the associated vehicle 10 via the UI, for example, to manually operate the boom, change the operating mode to autonomous driving and change the operating mode from autonomous driving, change the display view, modify the parameters 74.

[0094] The device 70 may further include and / or be connected to additional units, devices and systems, such as one or more sensor devices that sense the environment of the device 70 and / or sensor devices that detect the position of the joint.

[0095] The processor 71, the memory 72, the communication unit 75 and the UI can be interconnected in a variety of different ways via electrical leads inside the device 70. For example, each of the aforementioned devices can be individually connected to a main bus inside the device to allow the devices to exchange information. However, as will be appreciated by those skilled in the art, this is only an example, and depending on the embodiment, various ways of interconnecting at least two of the aforementioned devices may be selected without departing from the scope of the present invention.

[0096] The apparatus (such as device 70, mining vehicle 10 or control unit 20) may be configured at least to: assign an obstacle analysis parameter set to the mining vehicle, wherein the obstacle analysis parameter set is specific to a portion of the mining vehicle; detect obstacle candidates through an obstacle detection function of the mining vehicle based on processing scan data from a scanner of the mining vehicle; perform obstacle analysis based on a determined size of the obstacle candidate and the obstacle analysis parameter set assigned to the mining vehicle to classify the obstacle candidate and determine whether the obstacle candidate is an obstacle to be avoided; and provide an alarm or control signal in response to the obstacle analysis indicating that the obstacle candidate is an obstacle to be avoided to avoid the mining vehicle from colliding with the obstacle.

[0097] The apparatus may be or be comprised in a mining vehicle. The part of the mining vehicle may be a work tool of the mining vehicle. The obstacle analysis parameter set may comprise an adaptable set of trigger conditions defining two or more obstacle severity categories.

[0098] The apparatus may be configured to assign a set of obstacle analysis parameters to the mining vehicle based on the determined position of the work tool relative to the body of the mining vehicle.

[0099] The device may be configured to perform obstacle analysis based on at least one of: a height of the obstacle candidate; an area of ​​a cross-section of the obstacle candidate; and a slope of a surface in front of the obstacle candidate.

[0100] The apparatus may be configured to determine driving state data indicative of current driving parameters of the mining vehicle and dynamically assign the obstacle analysis parameter set based on the determined driving state data of the mining vehicle.

[0101] The apparatus may be configured to distribute the obstacle analysis parameter set while the mining vehicle is performing an autonomous driving mission.

[0102] The device may be configured to: detect path conditions when performing an autonomous driving task, and assign or adapt an obstacle analysis parameter set based on the detected path conditions.

[0103] Although the foregoing examples illustrate the principles of the present invention in one or more specific applications, it will be apparent to those skilled in the art that many modifications can be made in the form, use, and details of the implementation without exercising creative ability and without departing from the principles and concepts of the present invention as defined in the claims.

[0104] The verbs "comprise" and "include" are used in this document as open limitations that neither exclude nor require the presence of features not already recited. Unless expressly stated otherwise, the features recited in the dependent claims are mutually freely combinable. Furthermore, it should be understood that the use of "a" or "an" (i.e. the singular) throughout this document does not exclude a plurality.

Claims

1. A device comprising at least one processor and at least one memory, the at least one memory comprising computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the device to at least perform: - allocating (200) a set of obstacle analysis parameters specific to a portion of a mining vehicle (10), wherein The obstacle analysis parameter set includes an adaptable set of trigger conditions defining two or more obstacle severity categories, each of the severity categories being associated with at least one trigger parameter value for a size of an obstacle; - detecting (210) obstacle candidates by an obstacle detection function of the mining vehicle based on processing scan data from a scanner (40) of the mining vehicle; - based on the determined size of the obstacle candidate and the set of obstacle analysis parameters assigned to the mining vehicle, performing (220) an obstacle analysis to classify the obstacle candidate and determine whether the obstacle candidate is an obstacle to be avoided; as well as - In response to the obstacle analysis indicating that the obstacle candidate is an obstacle to be avoided, providing (230) an alarm or control signal to prevent the mining vehicle from impacting the obstacle.

2. The device according to claim 1, wherein: The part of the mining vehicle is a work tool (11) of the mining vehicle.

3. The apparatus of claim 2, wherein the at least one memory and the computer program code are configured to: with the at least one processor, cause the apparatus to assign (200) the set of obstacle analysis parameters to the mining vehicle (10) based on the determined position of the work tool relative to a body of the mining vehicle.

4. Apparatus according to any preceding claim, wherein: The at least one trigger parameter value includes a threshold or range for the size of the obstacle.

5. Apparatus according to any preceding claim, wherein: The at least one memory and the computer program code are configured to, with the at least one processor, cause the device to perform (220) the obstacle analysis based on at least one of: a height of the obstacle candidate; an area of ​​a cross section of the obstacle candidate; and a slope of a surface in front of the obstacle candidate.

6. The apparatus according to any preceding claim, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the apparatus to: determine driving state data indicative of current driving parameters of the mining vehicle, and dynamically allocate (200) the obstacle analysis parameter set based on the determined driving state data of the mining vehicle.

7. The apparatus of any preceding claim, wherein the mining vehicle is configured to perform an autonomous driving task, and the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus to assign (200) the set of obstacle analysis parameters while the mining vehicle is performing the autonomous driving task.

8. The apparatus of claim 7, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the apparatus to: - detecting path conditions while performing the autonomous driving task; and - allocating or adapting the obstacle analysis parameter set based on the detected path conditions.

9. A mining vehicle (10) comprising an apparatus according to any preceding claim.

10. A method for controlling a mining vehicle, the method comprising: - allocating (200) an obstacle analysis parameter set specific to a portion of a mining vehicle (10), wherein the obstacle analysis parameter set comprises an adaptable set of trigger conditions defining two or more obstacle severity categories, each of the severity categories being associated with at least one trigger parameter value for a size of an obstacle; - detecting (210) obstacle candidates by an obstacle detection function of the mining vehicle based on processing scan data from a scanner (40) of the mining vehicle; - based on the determined size of the obstacle candidate and the set of obstacle analysis parameters assigned to the mining vehicle, performing (220) an obstacle analysis to classify the obstacle candidate and determine whether the obstacle candidate is an obstacle to be avoided; and - In response to the obstacle analysis indicating that the obstacle candidate is an obstacle to be avoided, providing (230) an alarm or control signal to prevent the mining vehicle from striking the obstacle.

11. The method according to claim 10, wherein: The set of obstacle analysis parameters of the mining vehicle (10) is assigned based on the determined position of a work tool of the mining vehicle relative to a body of the mining vehicle.

12. The method according to claim 10 or 11, wherein: The obstacle analysis is based on at least one of: a height of the obstacle candidate; an area of ​​a cross section of the obstacle candidate; and a slope of a surface in front of the obstacle candidate.

13. The method according to any one of claims 10 to 12, comprising: Driving state data indicative of current driving parameters of the mining vehicle is determined, and the obstacle analysis parameter set is dynamically assigned (200) based on the determined driving state data of the mining vehicle.

14. The method according to any one of claims 10 to 13, comprising: When the mining vehicle is performing an autonomous driving task, detecting a path condition; as well as When the mining vehicle is performing an autonomous driving mission, the obstacle analysis parameter set is assigned or adapted based on detected path conditions.

15. A computer program comprising code for causing the method according to claims 10 to 14 to be performed when the code is executed in a data processing device.

Images