Method and system for updating a representation of a mining environment

Abstract

The present disclosure relates to a computer-implemented method for updating a distance-based representation of a mine using sensor data received from a mining and / or construction machine. The mining and / or construction machine comprises at least one sensor for sensing distances to obstacles in a plurality of directions in the surrounding of the machine, the method comprising: obtaining the distance-based representation of the mine; obtaining a set of sensor signals representing distances in a plurality of directions to obstacles in the surroundings of the machine from at least one position of the at least one sensor; positioning the obstacles represented by the set of sensor signals in the distance-based representation of the mine; comparing distances to obstacles in the surroundings sensed by the at least one sensor with corresponding distances determined from the corresponding sensor position in the distance-based representation of the mine; and updating the distance-based representation of the mine based on differences in the compared distances.

Description

[0001] METHOD AND SYSTEM FOR UPDATING A REPRESENTATION OF A MINING ENVIRONMENT

[0002] Technical Field

[0003] The present disclosure relates to representations of a surroundings of a mining and / or construction machine, in particular a mining and / or tunnelling environment, and more specifically to a method and system for updating a distance-based representation of a mine. The disclosure also relates to a corresponding system, as well as a mining and / or construction machine comprising such a system. The disclosure also relates to a computer program and a computer-readable medium comprising instructions that causes a computer to carry out the method according to the disclosure.

[0004] Background

[0005] With regard to mining and tunnelling, for example, there is a constant ongoing process of improving, e.g., efficiency, productivity and safety. Examples of changes / improvements that are carried out to an increasing extent is the automation of, fully or partly, and / or remote control of, various processes occurring in mining.

[0006] It is, for example, often desirable that at least part of the machines that are used in mining / tunnelling can be operated in an autonomous mode, i.e. , without an operator being required to influence the manoeuvring of the machine.

[0007] Autonomous operation may be utilised, e.g., to autonomously manoeuvre a machine from one location to another. This may be achieved based on a principle where the machine is first driven along a route in a path recording step, i.e., along the tunnels and / or drifts, along which the machine will later be driven autonomously, while at the same time signals from various sensors arranged on the machine are recorded. The recorded path may then be played back and autonomously followed by the machine by simultaneously determining its position in a map that is also generated while recording the path. This provides for an automation of a task that is repeated in precisely the same manner over and over again. There also exist work tasks that would be desirable to automate, but where the conditions change over time as task are carried out. Such work tasks may include autonomous operation in an environment where, e.g., excavations change the general appearance of the environment, so that a pre-recorded route no longer is valid, and thereby no longer may be used.

[0008] Summary

[0009] It is an object of the disclosure to provide a method for allowing autonomous operation in environments in which obstacles such as rock walls change over time, e.g., due to excavation. It is a further object of the disclosure to update an existing representation of a mine to improve the accuracy of the representation of the mine.

[0010] According to an aspect of the disclosure, it is provided a computer-implemented method for updating a distance-based representation of a mine using sensor data received from a mining and / or construction machine, the mining and / or construction machine comprising at least one sensor for sensing distances to obstacles in a plurality of directions in the surrounding of the machine. The method comprises: obtaining the distance-based representation of the mine; obtaining a set of sensor signals representing distances in a plurality of directions to obstacles in the surroundings of the machine from at least one position of the at least one sensor; positioning the obstacles represented by the set of sensor signals in the distance-based representation of the mine; comparing distances to obstacles in the surroundings sensed by the at least one sensor with corresponding distances determined from the corresponding sensor position in the distance-based representation of the mine; and updating the distance-based representation of the mine based on differences in the compared distances.

[0011] As was mentioned above, there exists various situations in which it may be desirable to obtain an autonomous operation of a mining and / or construction machine. The present disclosure relates, in principle, to any situation where a machine is used to autonomously navigate in a mine, e.g., to perform tasks, which may, e.g., include, but not be limited to, load and transport away material that, e.g., result from blasting. Such navigation, as was mentioned, may be carried out in a manner where a path is first recorded to then be played back and autonomously followed by the machine by simultaneously determining its position in a map that was generated while recording the path.

[0012] However, it may not always be possible to operate the machine in this manner, due to the requirement that the machine must have been driven beforehand along the route along which the machine is intended to subsequently operate in an autonomous manner. For example, the geographical extension of a mine may be regularly changing, e.g., due to ongoing excavation and / or addition or removal of various types of obstacles to the mining environment. Therefore, there may oftentimes exist parts of the mine which has not been properly set up for autonomous operation of the machine, or where conditions may have changed so that a prerecorded path is no longer usable or not feasible to set up.

[0013] An alternative method for effectuating, e.g., autonomous navigation in parts of a mine where no pre-recorded routes exist also utilizes a map, i.e. representation of the environment, but where the machine may determine distances to surrounding obstacles and compare the determined distances with corresponding distances in a representation of the environment in order to thereby determine the position of the machine in the mine, and navigate the mine.

[0014] Such navigation heavily rely on the accuracy of the representation of the environment, and if this representation of the mine is not sufficiently accurate the autonomous navigation may stop due to non-ability to determine the position of the machine. According to the disclosure, it is provided a method that allows navigation such as autonomous navigation, e.g., in situations where no pre-recorded routes exist, where a representation of the mine is updated when the environment changes to allow for continued autonomous operation.

[0015] In particular it is provided a computer-implemented method for updating a distancebased representation of a mine using sensor data received from a mining and / or construction machine comprising at least one sensor for sensing distances to obstacles in a plurality of directions in the surrounding of the machine. The updating may hence be carried out using signals recorded by the machine as it travels in the mine. The representation of the mine, i.e. map of the mine, is a distance-based representation, such as a spatial or cartesian representation, and hence a representation where obstacles in the mine, such as rock walls are represented in a way that allow distances to objects in the representation to be determined, e.g., from any other position in the representation. The method comprises to obtain the distance-based representation of the mine. This representation of the mine may be generated, e.g., by mine surveyors that generates the representation using suitable measurement equipment. The representation of the mine may be in any suitable data format, and such data formats are known in the art. The representation may further be generated in a global coordinate system of the mine, where global in this regard is a coordinate system valid throughout the mine but not necessarily being used outside the mine.

[0016] The method further comprises to obtain a set of sensor signals representing distances in a plurality of directions to obstacles in the surroundings of the machine from at least one position of the at least one sensor. For example, the sensor signals may be recorded as the machine travels to subsequently be utilized according to the disclosure. The obstacles represented by the sensor signals are then positioned in the distance-based representation of the mine, where this may be carried out, e.g. by determining the position of the sensor in the distance-based representation of the mine, and then position the obstacles as detected by the sensor signals, i.e. by distances and directions from the sensor to the obstacle in the representation of the mine. The sensor may be a rotating sensor, and in case the speed of rotation of the sensor is low, a single sweep of the sensor, such as a single rotation, and thereby the performing of the measurements in the plurality of directions in which the sensor is configured to measure distances, may not be determined at the same machine position, but the machine may have moved during the sweep. Different sensor positions may then be determined for different measurements of a single set of sensor signals. The distances to obstacles in the surroundings sensed by the at least one sensor are then compared with corresponding distances determined from the corresponding one or more sensor positions in the distance-based representation of the mine, and when there are discrepancies the distance-based representation of the mine can be updated based on differences in the compared distances.

[0017] In this way, sensor signals of a sensor being positioned on the machine and recording distances to obstacles in a plurality of different directions can be used to determine discrepancies between the actual extension of the mine and the extension of the mine according to the representation of the mine. That is, any changes to the mine that has occurred after the generation of the distance-based representation of the mine may be detected using the sensor signals and the sensor signals may be used to update the representation of the mine to facilitate subsequent navigation using machines where, e.g., an updated representation of the mine can be downloaded to other machines navigating in the mine.

[0018] According to aspects of the disclosure, the sensor signals may be provided, e.g., through a wireless interface, to a remote system such as a remote control centre, which may comprise a computer configured to perform the calculations required to update the representation of the mine. The updated representation may also be transmitted to machines, e.g., over any suitable wireless interface. The distancebased representation may also be updated using signals received from a plurality of machines. The calculations may also be performed, e.g., by a control unit or other suitable means in a machine, and the updated representation of the mine may then be uploaded to, e.g., a remote control centre, and / or other machines.

[0019] According to aspects of the disclosure, the distance-based representation of the mine is updated through addition of obstacles to, and / or and remove and obstacles from, the distance-based representation of the mine based on the differences in compared distances. Consequently, in case the sensor signals indicate that an obstacle that is present in the representation of the mine no longer is present in reality the obstacle may be removed from the representation of the mine. Conversely, in case the sensor signals indicate presence of an obstacle in reality but which is not present in the representation of the mine, such an obstacle may instead be added to the representation of the mine so that the updated representation of the mine comprising the detected obstacle may be used in subsequent navigation.

[0020] According to aspects of the disclosure, the comparing of distances to obstacles in the surroundings sensed by the at least one sensor with corresponding distances determined from the corresponding sensor position in the distance-based representation of the mine comprises to perform ray tracing in the distance-based representation of the mine from the position of the sensor in the plurality of directions of the set of sensor signals. The ray tracing in turn comprises to determine any presence of obstacles in the distance-based representation of the mine being closer to the sensor than the sensor detected distance to an obstacle in the plurality of directions of the set of sensor signals. The distance-based representation of the mine may then be updated by removing obstacles in the distance-based representation of the mine being detected by the ray tracing prior to reaching the distances indicated by the set of sensor signals. The ray tracing may comprise to follow the path of a simulated laser beam from the sensor position in the direction of a sensor signal until an obstacle hinders its propagation in the representation of the surroundings, or until the distance indicated by the sensor signal is reached. In this way a particular sensor signal, which hence has a direction and a length, may be followed in the representation of the mine to determine whether an obstacle is encountered prior to reaching the distance indicated as free from obstacles according to the sensor signal. In case an obstacle is encountered along the ray that the sensor signal constitutes the encountered obstacle may be removed from the representation of the mine.

[0021] According to aspects of the disclosure, in addition to removing obstacles detected by the ray tracing prior to reaching the distances indicated by the set of sensor signals, the new obstacles being located at a distance further away from the sensor may be added to thereby obtain an updated representation of the mine that more correctly reflects the actual propagation of the mine.

[0022] According to aspects of the disclosure, the distance-based representation of the mine is updated by adding obstacles being indicated by the set of sensor signals as being closer to the sensor than the ray tracing reaching an obstacle in the distance-based representation of the mine. That is, it may also be detected whether obstacles are encountered along the rays that the sensor signals represent to determine whether obstacles are present at a closer distance to the sensor than indicated by the representation of the mine. When this is the case such new obstacles may be added to the representation of the mine.

[0023] According to aspects of the disclosure, a plurality of sets of sensor signals are obtained, where each set of sensor signals represents distances to obstacles in different directions from the sensor position, wherein the plurality of sets of sensor signals represent sets of sensor signals from a plurality of sensor positions in the distance-based representation of the mine. Hence, a set of sensor signals may be obtained for each such sensor position. The ray tracing may then be performed for each of the plurality of sets of sensor signals, and the distance-based representation of the mine may be updated based on the differences detected by the ray tracing where consequently sensor signals from a plurality of sensor positions may be used in the updating. In this way a large number of sensor signals may be used where also presence of obstacles, or non-presence thereof, may be detected from a plurality of different sensor locations.

[0024] According to aspects of the disclosure, the method further comprises, when updating the distance-based representation of the mine based on the compared distances, to assign weighs to the sensor signals, wherein a distance to an obstacle detected from a sensor position comparatively further away from the obstacle may be given a lower weight in comparison to a distance to the obstacle measured from a sensor position being comparatively closer to the obstacle. The distance-based representation of the mine may then be updated based on differences detected by the ray tracing and the weights of the sensor signals.

[0025] According to aspects of the disclosure, each set of sensor signals represents sensor signals measured by the sensor for a position and rotation of the sensor in relation to the distance-based representation of the mine, where the method further comprises to obtain an estimated orientation of the sensor in the distance-based representation of the mine for the set of sensor signals and align the set of sensor signals in the distance-based representation of the mine. The aligned sensor signals may then be used when performing the ray tracing.

[0026] The surface upon which the machine travels in a mine may oftentimes not be level, with the consequence that the sensor being located on the machine and sensing distances to obstacles may also not be level. This may have the result that sensor signals may not be fully representative, e.g. in case the machine is inclined. According to aspects of the disclosure, therefore, the sensor signals may be aligned with the representation of the mine. This may be accomplished, e.g., by utilizing the position of the machine and the inclination of the machine which in general is known by the control system of the machine. The position of the sensor (or the position of the machine from which the position of the sensor may be calculated, e.g., through knowledge of the position of the sensor in relation to a reference position of the machine in the local coordinate system of the machine) and inclination may then be used to align the sensor signals so that an inclination of the sensor signals may be accounted for when used in the updating of the representation of the mine. The aligning may hence comprise to align the sensor orientation in one or more of x, y, z directions and also with regard to one or more of roll, pitch, yaw of the machine (and hence sensor).

[0027] According to aspects of the disclosure, the method further comprises to generate a grid representation of the distance-based representation of the mine, the grid forming a plurality of grid cells, each grid cell representing a region, and each grid cell being represented by a point of data. In this way the amount of data necessary to generate the representation of the mine may be substantially reduced. For example, each grid cell may represent a region such as an area of the mine, e.g. part of a rock wall etc., where each such region may be represented by a single point of data. It is to be understood that the single point of data may comprise a plurality of values describing the region. According to aspects of the disclosure, the region is represented by an average value of points of the representation of the mine that otherwise belong to the specific region. The method may then further comprise to convert the one or plurality of sets of sensor data to a corresponding grid representation of the region covered by the sensor data, such that each cell of the grid representation of the sensor data also comprises a point of data. Any sensor data such as a plurality of sensor measurements that belong to particular grid cell may then be converted to single point of data, e.g. an average value or any other suitable kind of value.

[0028] The grid representation of the distance-based representation of the mine may then be updated utilizing the grid representation of the set of sensor data. This may substantially facilitate the updating since fewer values need to be compared to each other.

[0029] According to aspects of the disclosure, the point of data is generated as a normal distributions representation of sensor data points within the particular cell. In this way e.g. a plurality of sensor signals belonging to particular grid cell may be represented in a manner that while being efficient from a data amount point of view may still provide a description of the sensor data that reflects the actual distribution of the sensor data within a grid cell. For example, in case sensor data are distributed over the full region of a grid cell this may be reflected by the normal distributions representation exhibiting a high standard deviation for the grid cell. Correspondingly, in case the sensor data are more concentrated this may be reflected by a lower standard deviation, where the normal distributions representation may further comprise an average value of the various sensor signals.

[0030] According to aspects of the disclosure, probabilities may be designated to each grid cell, where each probability may represent a probability that the obstacle exists. For example, a plurality of sensor signals belonging to a particular grid cell may indicate a high probability that the obstacle actually exists, while a single sensor signal, or, in comparison, only few in comparison sensor signals may indicate a low probability of the obstacle actually existing, in particular in a case where the sensor signals are from a sensor position being located comparatively further away than more closely generated sensor signals. According to aspects of the disclosure, a removal of an obstacle may require a plurality of sensor signals indicating the non-presence of the obstacle, and / or the addition of an obstacle may require a plurality of sensor signals indicating presence of the obstacle to be added to the distance-based representation of the mine. In this way it can be assured that an obstacle is not removed from the representation only because of a single sensor value, which, e.g., may origin from a distant sensor position, in a situation when it is in fact the sensor signal that is not accurate.

[0031] According to the disclosure, the method further comprises to obtain a plurality of sets of sensor signals representing distances in a plurality of directions to obstacles in the surroundings of the machine, wherein the sets of sensor signals are obtained at sensor positions being located at least at a predetermined distance from each other. Consequently, sensor signals not only from a single sensor position but from a plurality of a sensor positions may be used when updating the representation of the mine. A predetermined minimum distance between sensor positions may be utilized to thereby reduce the overall amount of sensor signals being used when updating the representation of the mine.

[0032] According to aspects of the disclosure, sensor signal data is recorded for a distance of travel of the machine, and then used to update the representation of the surroundings of the machine. In this way the machine may record sensor signals when navigating through the mine, and the sensor signals consequently recorded from various positions in the mine may then be used according to the above for updating the representation of the mine where such data may be transmitted to remote location in case the updating is performed at the remote location or be used in the machine in case the updating is performed by the machine. Again, a minimum spacing between sensor positions from which sensors signals are collected may be utilized.

[0033] According to aspects of the disclosure, sensor means are configured to scan a region in front of the machine in a plurality of angles in relation to a vertical plane of the machine intersecting a longitudinal axis of the of the machine. Such sensors are commonly used in autonomous navigation, and may also be used to obtain sensor signals for updating the representation of the mine.

[0034] The sensor means may also be configured to scan a region in front of the machine in a plurality of angles in relation to a horizontal plane as well as in relation to a vertical plane. Hence not only two-dimensional scanning may be used, but also three- dimensional scanning of distances to obstacles.

[0035] According to aspects of the disclosure, the mining and / or construction machine is an articulated machine having a front portion and a rear portion, wherein each of the front portion and rear portion comprises a sensor measuring distances and directions to obstacles in the surrounding of the machine, the method comprising use of sets of sensor signals from each of the sensors.

[0036] The disclosure also relates to a system for updating a distance-based representation of a mine. It will be appreciated that all features described for the method aspects of the disclosure are applicable also to the system aspects of the disclosure. Also, the system and its aspects have advantages corresponding to the advantages discussed above with regard to the method. This also applies to a mining and / or construction machine comprising such a system.

[0037] Further characteristics of aspects of the disclosure and advantages thereof are indicated in the detailed description of exemplary embodiments set out below and the attached drawings.

[0038] Brief description of the drawings

[0039] Figs. 1A-B illustrates an exemplary machine which may be configured to operate according to embodiments of the disclosure;

[0040] Fig. 2 illustrates an exemplary representation of a portion of an environment in which the machine of Figs. 1A-B may be autonomously operating;

[0041] Fig. 3 illustrates an exemplary method according to the disclosure; Fig. 4A illustrates an exemplary portion of a representation of a surrounding of a machine, where the representation is in the form of points of data at a distance from each other;

[0042] Fig. 4B illustrates the exemplary portion of a representation of a surrounding of a machine according to Fig. 4A, and exemplary sensor signals of a sensor on a machine;

[0043] Fig. 4C illustrates the exemplary portion of the representation of the surrounding where obstacles have been removed based on sensor signals;

[0044] Fig. 4D illustrates the exemplary portion of the representation of the surrounding according to Fig. 4A where data is added based on the sensor signals;

[0045] Fig. 4E illustrates the exemplary portion of the representation of the surrounding according to Fig. 4A following the addition of data based on the sensor signals.

[0046] Fig. 5 illustrates an exemplary computer which may be utilized to implement aspects of the disclosure.

[0047] Detailed description of aspects of the disclosure

[0048] Aspects of the present disclosure will be exemplified in the following in view of a particular kind of mining and / or construction machine. The disclosure is, however, applicable for all kinds of mining and / or construction machines that may be used for mining operations, such as, but not limited to, autonomous navigation, in a mine. In particular, in addition to collecting sensor signals using autonomously navigating machines, sensor signals being collected for use in the updating of the representation of the mine may be collected by sensors being located on manually operated machines, or on remote-controlled machines, where such machines as they are driven around the mine may collect sensor signals to be used in the updating of the representation of the mine.

[0049] Figs. 1 A and 1 B illustrates a side view and elevated view, respectively, of an exemplary machine 100, which may be utilized to collect sensor signals for use to update a representation of a mine according to the disclosure. According to the present example, the machine 100 is a load-haul-dump (LHD) machine, and is used to load and transport away materials such as excavated rock through the use of a bucket 101. The machine 100 comprises, apart from the bucket 101 , wheels 102 - 105 for allowing the machine to be set in motion and a control system comprising at least one control unit 106. The control unit 106 is configured to control various of the functions of the machine 100. Machines of the disclosed kind may comprise more than one control unit, e.g., a plurality of control units, where each control unit, respectively, may be arranged to be responsible for monitoring and carrying out different functions of the machine 100. For reasons of simplicity, however, it will be assumed in the following that the various functions according to the disclosure that are carried out using the machine are controlled by the control unit 106. The control unit 106 may also, for example, be configured to control manoeuvring of the machine, e.g., in terms of setting the machine in motion, stopping the machine, and controlling and manoeuvring equipment forming part of or being attached to the machine, such as the bucket 101 e.g. when loading or unloading rock. Requests for setting the machine in motion may be initiated by a part of the control system controlling autonomous driving of the machine.

[0050] According to the disclosure, the machine collects sensor signals, and although the method according to the disclosure may be arranged to be carried out by the machine, the updating of the representation of the mine may preferably be carried out at a remote location, such as in a remote control centre, in which case the machine may transmit obtained sensor signals to the remote location for further processing, e.g., through a wireless interface. Once the representation of the mine has been updated, the updated representation may, e.g., be downloaded to the machine for use in continued autonomous navigation. It should be appreciated that the machine may collect the sensor signals concurrently with performing other operations, such as during transport of excavated rock through the mine.

[0051] The machine 100 further constitutes an articulated machine, where a front portion 100a is connected to a rear portion 100b by means of a hinge 107, and the machine is steered by means of articulated steering to facilitate manoeuvring of the machine. Machines of the disclosed kind are often driven in surroundings where the distance to surrounding rock walls may be small, and articulated machines may provide manoeuvrability advantages over non-articulated machines in such environments. As is appreciated by a person skilled in the art, the illustrated machine merely forms an example of usability of the disclosure, and, in principle, the disclosure is applicable for essentially any kind of movable machine being utilized in mining and / or construction.

[0052] Motion of the machine 100 and / or equipment thereof, may be generated by setting one or more actuators in motion. Such actuators may comprise cylinders / motors / pumps etc. For example, the machine 100 may comprise actuators in the form of, e.g., hydraulic motors for propelling the machine 100. According to the disclosed example there may also be an actuator in the form of one or more hydraulic cylinders for controlling articulation of the joint 107.

[0053] The machine 100 may also comprise actuators, e.g., in the form of hydraulic cylinders for controlling raising / lowering the bucket 101 , where further actuators may be present in this regard, e.g., to control tipping of the bucket 101.

[0054] The machine also comprises range detectors such as laser / LIDAR scanners 111 , 112 to determine distances and directions e.g., to surrounding rock and / or obstacles in the travel path of the machine 100. Also other types of range detectors, such as radars or stereo cameras, may be used instead of or in addition to laser / LIDAR scanners. The range detectors 111 , 112 may scan distances in various different directions, e.g., in relation to a longitudinal axis of the machine, and, e.g., in a horizontal plane of the machine.

[0055] As was mentioned above, machines of the kind disclosed in figs. 1 A-B may be of very large dimensions and exhibit substantial mass, and even more so when carrying load, where machines of the disclosed kind may be designed to carry e.g., 10-30 tonnes of broken rock. It is highly desirable that machines of this kind can be driven autonomously as much as possible. According to the present disclosure, it is provided means for increasing the number of situations in which autonomous navigation can be utilised. Fig. 2 illustrates a part of an exemplary portion 200 of an underground mine in which the machine according to Figs. 1A-B may be configured to operate, and where the present disclosure may be utilised. According to the illustrated example, the machine 100 uses the bucket 101 (shown in figs. 1A-B) to load material such as broken rock and / or ore from one location (not shown), to then, e.g., haul the load for dumping at some other location (also not shown). The machine 100 is configured to autonomously navigate the environment based on a representation of the mine.

[0056] According to aspects of the disclosure, it is provided a method for updating the representation of the mine that allows continued autonomous navigation when the environment changes, e.g., due to excavation, or for other reasons, such as new obstacles arising. An exemplary method for updating a representation of the surroundings is schematically illustrated in Fig. 3. The method is computer- implemented, and may, as was mentioned, be carried out in the machine 100 by the control system of the machine, such as by the control unit 106, but perhaps preferably, as was also mentioned, at least partly at a location remote from the machine, in which case sensor data may be transmitted from the machine 100 to the remote location.

[0057] The method 300 according to Fig. 3 starts in step 301 , where the distance-based representation of the mine is obtained. As was mentioned above, the representation, i.e., map, of the mine, may be generated, e.g., by mine surveyors that generate the map of the mine. The representation of the mine may be generated in different data formats, having varying degrees of accuracy. In general, the more accurate a map representation is, the more data is in general required to represent the map.

[0058] According to the disclosure, the representation of the mine may be received in any suitable data format, but according to aspects of the disclosure, the representation of the mine is converted to a format that is less data consuming, but that still provide sufficient accuracy in terms of, e.g., being used for navigation by the machine. According to aspects of the disclosure, a normal distributions representation is used, where, when obtaining the representation of the mine, this representation may be converted to such a normal distributions representation. This will be explained further below, and is also schematically illustrated by Fig. 4A.

[0059] Fig. 4A illustrates only a small portion 401 of a representation of mine, such as a fraction of a rock wall. Instead of using, e.g., continuous line segments or densely packed points of data to represent obstacles such as rock walls etc., the representation of the mine illustrated in Fig. 4A as a discontinuous relatively sparse row of points of data 402. As is realized, a mine may be large comprising a plurality of drifts and tunnels and Fig. 4A only illustrates a very small portion in the immediate vicinity of the machine 100. It is to be understood that the complete representation of the mine to be converted to a format of the kind illustrated in Fig. 4A.

[0060] It is further to be noted that the mine may be represented by points of data as disclosed in a single plane, and hence form a 2D representation of the actual 3D object that the mine in reality constitutes. The points of data may also represent, e.g. a region, or area, having a width and a height. This may substantially reduce the amount of data required to represent the mine as was mentioned. The points of data 402 will be described further below. The points of data 402 may be located to represent any suitable distance from each other, such as, e.g., any distance in the interval 0.1 to 5 m apart, but may also be denser or more sparsely located.

[0061] In step 302, a set of sensor signals from the one or more sensors 111 , 112 being located on the machine 100 is obtained. The set of sensor signals may comprise a single sweep, i.e. a single sweep of one of the sensors 111 , 112 where, for example, sensor may be arranged to measure the distance to surrounding obstacles, e.g., every degree of rotation from a direction along the longitudinal axis of the machine to a direction being perpendicular to the longitudinal axis of the machine, where measurements may be carried out in both directions in relation to the longitudinal axis of the machine and, e.g., in a horizontal plane of the sensor. In principle, the sensor may be a rotating sensor performing measurements during the full, or only a part of, revolution, such as, e.g. through the 180 degrees exemplified. As is realized, the sensor may also be configured to measure distances to obstacles at closer or more sparse intervals than every degree of rotation. Furthermore, the sensor signals being used in the updating of the representation of the mine may comprise sensor signals not only from a single location of the sensor and hence the machine, but the sensor signals may comprise sets of sensor signals from a plurality of sensor positions being recorded, e.g., as the machine is traveling in the mine.

[0062] In step 303 the obtained sensor signals are positioned in the distance-based representation of the mine. In order to do this, the set of sensor signals may be transmitted together with a position of the sensor in the mine, where, e.g., the machine may comprise means for determining the position of the machine in the mine using the representation of the mine, and through the use of which the position of the sensor in the mine may also be determined in a manner known per se, e.g. using the position of the machine, which in general is determined by a specific reference point of the machine, and the local coordinate system of the machine in which the position of the sensor in general is known so that the position of the sensor in the mine thereby can be calculated. It can be noted in this regard that means for determining the position of the machine in the mine may actually use sensor signals of the kind being used for the updating of the representation of the mine, e.g., by performing scans of distances and directions to obstacles in the surroundings of the machine, and then translating this to a position in the representation of the mine through the use of comparison of distances measured by the sensor and corresponding distances measured in the representation of the mine. It should also be appreciated that it is possible to position the set of sensor signals in the distancebased representation of the mine by performing a scan registration algorithm between the representation of the mine and the set of sensor signals themselves.

[0063] The sensor may furthermore have an inclination in relation to, e.g., horizontal plane of the mine for example because the machine currently being inclined. The position of the sensor may therefore be a position, e.g., expressed in xyz coordinates as well as an inclination, which may be expressed in one or more from roll (rotation around the longitudinal axis of the machine), pitch (rotation around a transversal axis of the machine) and yaw (rotation around a vertical axis of the machine). The consequently obtained position and orientation of the sensor may then be used to align the sensor signals to the representation of the mine so that thereby the points represented by distances and directions determined by the sensor may be accurately determined in the representation of the mine. These points may hence be added, in step 303, to the representation of the mine.

[0064] This is schematically illustrated in figure 4B, which illustrates the machine 100 in the same position as in Fig. 4A, where the measurements of the sensor 111 are illustrated as dots. As was mentioned above, the sensor 111 may scan in various different directions, of which only some of the measurements are illustrated in figure 4B. In reality, a sensor scan may comprise a plurality of more measurements as was mentioned above, but for reasons of illustration the number of measurements has been reduced. As can also be seen in the figure, there are measurements 410 indicating obstacles that coincide with the obstacles of the representation of the mine. However, there are also measurements, some of which being explicitly exemplified by being denoted 411 , going beyond the obstacles indicated by the original representation of the mine, and hence indicating that an obstacle such as a portion of, e.g., a rock wall, or any other obstacle, no longer is present in the environment.

[0065] In step 304, distances to obstacles in the surroundings of the machine sensed by the sensor 111 from the position of the sensor in the representation of the mine are compared with corresponding distances determined from the corresponding sensor position in the distance-based representation of the mine, and when there are discrepancies, as shown in Fig. 4B, the representation of the mine is updated in step 305.

[0066] The updating of the distance-based representation of the mine is carried out by adding obstacles to, and / or and removing obstacles from, the distance-based representation of the mine based on the differences in compared distances. This is illustrated in Figs. 4C to 4E. In Fig. 4C the points of data representing obstacles in the original representation of the mine have been removed for positions where the distance measurements of the sensor indicate non-presence of such obstacles. These points of data are indicated by being enclosed by dashed box 412 in Fig. 4B. This may be determined, e.g., using ray tracing in the distance-based representation of the mine from the position of the sensor in the plurality of directions of the set of sensor signals. That is, the rays represented by the directions and distances measured by the sensor 111 are followed from the position of the sensor in the representation of the mine while determining whether an obstacle is countered. For example, with reference to Fig. 4B, the following of array 413 from the position of the sensor 111 will encounter an obstacle denoted by 414. consequently, the ray tracing is used to determine presence of obstacles in the distance-based representation of the mine being closer to the sensor 111 than the sensor detected distance to an obstacle in the particular direction, where this may be carried out for each sensor signal, and hence in each direction in which the sensor has measured the distance. The distance-based representation of the mine is then updated by removing obstacles in the distance-based representation of the mine being detected by the ray tracing prior to reaching the distances indicated by the set of sensor signals. This removal is, as was mentioned, illustrated in Fig. 4C.

[0067] Furthermore, the updating may, in addition to removing obstacles detected by the ray tracing, also include to add the obstacles that has been detected by being indicated by the set of sensor signals. That is, the obstacles being represented by the rays 411 in fig. 4B. These obstacles are consequently added to the representation of the mine and this is illustrated in Fig. 4D where new points of data, exemplified by points of data 420, have been added to the representation of the mine. The result of the updating according to the disclosure is illustrated in Fig. 4E. This updated representation of the mine may then be utilized for subsequent navigation, both by the machine from which sensor signals have been recorded and also, e.g., from other machines operating in the mine.

[0068] Furthermore, as can be seen in Fig. 4E, there is a gap 430 in the representation of the mine. This is because this particular area cannot be reached from the current sensor position. However, the updating according to the disclosure has been illustrated for sensor signals captured from a single sensor position. As was mentioned above, sensor signals from a plurality of sensor positions may be utilized in the updating of the representation of the mine. This means that when the machine has moved a bit further to the left in the figure, a subsequent sets of sensor signals will reach the area of 450 so that this area as well can be updated with possible presence of obstacles.

[0069] According to aspects of the disclosure, a machine may travel any suitable distance while recording sensor signals, e.g., continuously and / or regularly such as for every predetermined distance of travel of the machine, such as 0.1 m or 1 m or any other suitable distance as was mentioned. Thereby, sensor signals may be recorded for a plurality of sensor positions, where all of these sensor signals may then be used in the updating of the representation of the mine. Furthermore, it is to be noted that a set of sensor signals, such as a single sweep of a sensor in the plurality of directions in which the sensor is configured to measure distances, need not be determined for precisely the same machine position, since the machine may move during the sweep. In such situations different positions may be determined for different measurements of a single sweep. Also, Figs. 4A to 4E only illustrates sensor signals captured by front sensor 111 but as is realized, sensor signals captured by a rear sensor 112 may simultaneously be utilized in the updating of the representation of the mine, where such sensor signals in particular may reach the area 430 in Fig. 4E.

[0070] The use of sensor signals captured from a plurality of different sensor locations means that, e.g., a same obstacle may be detected by sensor signals from a plurality of different distances. However, an obstacle detected from a closer distance may be considered to be more likely to be present than an obstacle being detected from a position further away. In order to account for possible accuracies in measurements from different positions, weights may be assigned to the sensor signals, where a distance to an obstacle detected from a sensor position comparatively further away from the obstacle may be given a lower weight in comparison to a distance to the obstacle being measured from a sensor position being comparatively closer to the obstacle. The updating of the distance-based representation of the mine may then take into account the weights when removing and adding obstacles to the representation. For example, if an obstacle indicated by a sensor signal having lower weight contradicts the detection of a sensor signal having a higher weight the sensor signal having the lower weight may, according to aspects of the disclosure, be discarded.

[0071] As was mentioned above, and as has been described with reference to Figs. 4A to 4E, the representation of the mine may be generated using discrete points of data to thereby reduce the amount of data being required to generate the representation of the mine. Each point of data may represent, e.g. a region, such as a rock surface segment, of the mine. According to aspects of the disclosure, this may be accomplished by generating a grid representation of the distance-based representation of the mine. The grid may form a plurality of grid cells where each grid cell represents a region of the mine, such as, for example, a region having a width and a height. Each such region may then be presented by the point of data, e.g. in a matter illustrated in figs. 4A-E. The sensor data, however, may be considerably denser than a single measurement hitting a particular grid cell. In fact, a plurality of sensor measurements may hit an area that is represented by a particular grid cell and / or a new area having the size of a grid cell. The set of sensor data, or the plurality of sets of sensor data may therefore be converted to a corresponding grid representation of the region covered by the sensor data, such that each cell of the grid representation of the sensor data comprises a point of data, where this grid representation of the sensor data may then be used to update the distance-based representation of the mine.

[0072] In this regard, all sensor measurements striking region representing a grid cell may be identified and used to calculate a single point of data for the grid cell. For example, mean values of the xyz coordinates, and distances of the sensor signals striking in particular grid cell may be calculated to obtain a single point of data representing the grid cell. Thereby, the oftentimes large number of sensor measurements may be converted to a representation comprising considerable fewer points of data, while still the sensor measurements are used to generate the points of data and where also weights according to the above may be utilized in the generation of the points of data. Furthermore, as was briefly mentioned above, according to aspects of the disclosure, a normal distributions representation may be utilized to reduce the amount of data required to properly generate the representation of the mine. This may be carried out using the plurality of sensor signals striking a particular grid cell where the spread in the cell may be used to calculate a normal distributions representation indicating, e.g., whether the sensor signals are concentrated towards a particular portion of the grid cell or more evenly distributed. In case the sensor signals are more densely located, this may indicate, for example, presence of an obstacle taking up only part of a grid cell, such as a light post or other narrow object, whereas a more evenly distributed presence of sensor signals may indicate, e.g. a rock wall. The normal distributions representation may hence indicate such differences where this may also be used in the updating of the representation of the mine.

[0073] In addition to, or as an alternative to the normal distributions representation probabilities may be assigned to each grid cell. That is, probabilities stating the probability of an obstacle actually existing in the grid cell. For example, a large number of sensor signals being present for a particular grid cell may indicate a large probability of the presence of an obstacle, whereas a single sensor signal indicating an obstacle may give rise to a low probability, since actual presence of an obstacle likely would generate more sensor signal indications.

[0074] The presence of a number of sensor signals in a grid cell, or otherwise in a region of the mine, may also be used when determining whether an obstacle actually is to be removed or included in the representation of the mine. According to aspects of the disclosure, a plurality of sensor signals indicating presence or non-presence of an obstacle may be required prior to actually adding or removing the obstacle in question.

[0075] Furthermore, so far the updating the distance-based representation of the mine has been illustrated for a situation where obstacles in the representation being closer to the sensor than the distance measured by the sensor signals have been removed and additional obstacles have been added at distances been further away from the sensor. However, according to aspects of the disclosure, the opposite situation may also arise. That is, the sensor of the machine may indicate presence of an obstacle being closer to the sensor than presence being indicated by the representation of the mine. Such new obstacles may be added to the representation of the mine in a manner similar to what has been described above.

[0076] As was mentioned above, the updating of the representation of the mine may be carried out in a computer in a remote control centre, which may comprise a computer configured to perform the calculations required to update the representation of the mine, and / or in a machine which may then also comprise such a computer, which may form, or form part of a control unit, such as control unit 106 above. Fig . 5 illustrates an exemplary computer 500 that may be utilized to carry out aspects of the disclosure.

[0077] The computer 500, which hence may form, or form part of, control unit 106, or part of a system of the remote location, comprises a computing unit 501 , which can be constituted by essentially any suitable type of processor or microcomputer, e.g., a microprocessor, or a circuit for digital signal processing (Digital Signal Processor, DSP). The computing unit 501 is connected to a memory storage 502 which provides the computing unit 501 with, e.g., the stored program code and / or the stored data 503 that is required in order for the computing unit 501 to be able to perform computations. The computing unit 501 is also arranged to store partial or final results of computations, such as the updated representation of the mine, in the memory unit 502.

[0078] In addition, the computer is provided with devices 511 , 512 for receiving and transmitting input and output signals. These input and output signals can contain the representation of the mine and sensor signals, and the updated representation of the mine, where the signals may follow any suitable communication protocol, such as a wired or wireless protocol. Received signals can be converted into signals which can be processed by the computing unit 501 , and the result following processing in the computing unit can correspondingly be converted to a suitable format for transmission, e.g., to one or more machines. The connections to the devices for receiving and transmitting input and output signals can be constituted by one or more of a cable; a data bus, such as a Controller Area Network CAN bus, a Media Orientated Systems Transport MOST bus, or some other bus configuration; or by a wireless connection. The aspects of the disclosure may also be implemented wholly or partially in one or more computers of the illustrated kind.

[0079] Finally, aspects of the disclosure have been described largely with reference to an LHD machine. The disclosure may however be utilized in any kind of movable mining and / or construction machine being adapted to record sensor signals according to the above, where such machines may or may not be autonomously operating. The disclosure is also applicable for underground machines as well machines operating above ground for as long as operation is carried out using sensors measuring distances to objects in the surroundings of the machine for comparison with a representation of a mine.

Claims

Claims1 . A computer-implemented method for updating a distance-based representation of a mine using sensor data received from a mining and / or construction machine, the mining and / or construction machine comprising at least one sensor for sensing distances to obstacles in a plurality of directions in the surrounding of the machine, the method comprising: obtaining the distance-based representation of the mine; obtaining a set of sensor signals representing distances in a plurality of directions to obstacles in the surroundings of the machine from at least one position of the at least one sensor; positioning the obstacles represented by the set of sensor signals in the distance-based representation of the mine; comparing distances to obstacles in the surroundings sensed by the at least one sensor with corresponding distances determined from the corresponding sensor position in the distance-based representation of the mine; and updating the distance-based representation of the mine based on differences in the compared distances.

2. A method according to claim 1 , further comprising: updating the distance-based representation of the mine by adding obstacles to, and / or and removing obstacles from, the distance-based representation of the mine based on the differences in compared distances.

3. A method according to claim 1 or 2, the comparing of distances to obstacles in the surroundings sensed by the at least one sensor with corresponding distances determined from the corresponding sensor position in the distancebased representation of the mine further comprising: performing ray tracing in the distance-based representation of the mine from the at least one position of the sensor in the plurality of directions of the set of sensor signals; the ray tracing comprising to determine presence of obstacles in thedistance-based representation of the mine being closer to the sensor than the sensor detected distance to an obstacle in the plurality of directions of the set of sensor signals; and updating the distance-based representation of the mine by removing obstacles in the distance-based representation of the mine being detected by the ray tracing prior to reaching the distances indicated by the set of sensor signals.

4. A method according to claim 3, further comprising: in addition to removing obstacles detected by the ray tracing prior to reaching the distances indicated by the set of sensor signals, adding obstacles detected indicated by the set of sensor signals.

5. A method according to claim 3 or 4, further comprising: updating the distance-based representation of the mine by adding obstacles being indicated by the set of sensor signals as being closer to the sensor than the ray tracing in the direction reaching an obstacle in the distance-based representation of the mine.

6. A method according to any one of the claims 3-5, further comprising: obtaining a plurality of sets of sensor signals, each set of sensor signals representing distances to obstacles in different directions from the sensor position, wherein the plurality of sets of sensor signals represent sets of sensor signals from a plurality of sensor positions in the distance-based representation of the mine; performing the ray tracing for each of the plurality of sets of sensor signals; and updating the distance-based representation of the mine based on differences detected by the ray tracing.

7. A method according to claim 6, further comprising, when updating the distance-based representation of the mine based on the compared distances: assigning weighs to the sensor signals, wherein a distance to an obstacle detected from a sensor position comparatively further away from theobstacle is given a lower weight in comparison to a distance to the obstacle measured from a sensor position being comparatively closer to the obstacle, and updating the distance-based representation of the mine based on differences detected by the ray tracing and the weights of the sensor signals.

8. A method according to any one of the claims 6-7, wherein: each set of sensor signals represent sensor signals measured by the sensor for a position and rotation of the sensor in relation to the distancebased representation of the mine, the method further comprising: obtaining an estimated orientation of the sensor in the distance-based representation of the mine for the set of sensor signals; aligning the set of sensor signals in the distance-based representation of the mine; and utilizing the aligned sensor signals when performing the ray tracing.

9. A method according to any one of the claims 1 -8, further comprising: generating a grid representation of the distance-based representation of the mine, the grid forming a plurality of grid cells, each grid cell representing a region, and each grid cell being represented by a point of data, the method comprising: converting the one or plurality of sets of sensor data to a corresponding grid representation of the region covered by the sensor data, such that each cell of the grid representation of the sensor data comprises a point of data, and updating the grid representation of the distance-based representation of the mine utilizing the grid representation of the set of sensor data.

10. A method according to claim 9, further comprising: generating the point of data from an average of any plurality of points of the set of sensor data being located within the particular cell of the grid representation.11 . A method according to claim 10, further comprising: generating the point of data as a normal distribution representation of sensor data points within the particular cell.

12. A method according to any one of the claims 9-11 , further comprising: assigning probabilities to each cell, each probability representing a probability that the obstacle exists.

13. A method according to any one of the claims 1-12, wherein the removal of an obstacle requires a plurality of sensor signals indicating the non-presence of the obstacle, and / or wherein the addition of an obstacle requires a plurality of sensor signals indicating presence of the obstacle to be added to the distance-based representation of the mine.

14. A method according to any one the claims 1-13, further comprising: obtaining a plurality of sets of sensor signals representing distances in a plurality of directions to obstacles in the surroundings of the machine, wherein the sets of sensor signals are obtained at sensor positions being located at least at a predetermined distance from each other.

15. A method according to any one of the claims 1 -14, further comprising: recording sensor signal data of the at least one sensor for a distance of travel of the machine, and utilizing the recorded sensor signal data to update the representation of the surroundings of the machine.

16. Computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method according to any one of the claims 1 -15.

17. Computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of the claims 1-15.

18. A system for updating a distance-based representation of a mine using sensor data received from a mining and / or construction machine, the mining and / or construction machine comprising at least one sensor for sensing distances to obstacles in a plurality of directions in the surrounding of the machine, the system comprising: means for obtaining the distance-based representation of the mine; means for obtaining a set of sensor signals representing distances in a plurality of directions to obstacles in the surroundings of the machine from at least one position of the at least one sensor; means for positioning the obstacles represented by the set of sensor signals in the distance-based representation of the mine; means for comparing distances to obstacles in the surroundings sensed by the at least one sensor with corresponding distances determined from the corresponding sensor position in the distance-based representation of the mine; and means for updating the distance-based representation of the mine based on differences in the compared distances.

19. A mining and / or construction machine comprising a system according to claim18.

Images