A method for localization of a mining or construction vehicle in an underground mine, a control system for localization of a mining or construction vehicle, and a mining or construction vehicle comprising the control system

Abstract

The disclosure relates to a method for localization of a mining or construction machine in an underground mine, the method comprising: - generating a virtual image comprising: -- intensity information associated with portions of reflected light from bolt ends comprised in a tunnel wall exposed to a light emission from a sensor system, respectively, and from an area surrounding each bolt end; and -- depth information associated with distances between the sensor system and the bolt ends, respectively; - generating a local map based on the virtual image, the local map comprising: -- nodes associated with positions of the bolt ends, respectively, and with a signature of the area surrounding each bolt end; and -- edges linking two nodes together; and - localizing the mining or construction machine based on the local map and the global mine map.

Description

[0001] A METHOD FOR LOCALIZATION OF A MINING OR CONSTRUCTION VEHICLE IN AN UNDERGROUND MINE, A CONTROL SYSTEM FOR LOCALIZATION OF A MINING OR CONSTRUCTION VEHICLE, AND A MINING OR CONSTRUCTION VEHICLE COMPRISING THE CONTROL SYSTEM

[0002] Technical Field

[0003] The disclosure relates to a method and a control system for localization of a mining or construction machine in an underground mine. More specifically, the disclosure relates to bolt pattern based underground localization of a mining or construction machine.

[0004] Furthermore, the disclosure also relates to a corresponding computer program and a computer-readable medium causing a computer to carry out the method.

[0005] Background

[0006] Underground localization of mining or construction vehicles, such as determination of a vehicle position in an underground / subterranean mine tunnel, is a generally a difficult task. Positioning systems commonly used for localization and navigation on the surface of the earth, such as for example the global positioning system (GPS), do net work in a mine, because signalling between the underground vehicle and satellites or terrestrial nodes is generally impossible.

[0007] Some conventional methods for underground localization of a mining or construction vehicle involve manual operation of a total station, i.e. of a theodolite apparatus utilising electronic distance measuring to measure vertical and horizontal angles. Typically, each time the mining or construction vehicle is moved within the underground mine, a total station and its operator has to move along with the mining or construction vehicle. At each new position of the mining or construction vehicle, the operator then has to perform a manual localization of the mining or construction vehicle by utilization of the total station. Thus, this conventional method for underground localization requires purchasing of total stations and extensive manual operation of these total stations. The method is thus expensive and time demanding. Alternatively, some conventional methods for underground localization of a mining or construction vehicle involve initial distribution of a large number of position markers at predetermined and well-defined positions in the mine. These markers may then be identified and utilized for localization of the mining or construction vehicle when it moves around in the mine. However, the distribution of the numerous markers at the predefined positions in the mine requires a lot of manual work and is time consuming. Also, the process needs to be repeated each time the topology of the mine changes when new tunnels and / or drifts are created.

[0008] Summary

[0009] Conventional solutions for underground localization of mining or construction vehicles involve extensive manual operation of total stations and / or extensive manual work related to distribution of numerous position markers in the mine. The conventional solutions are thus time consuming and costly.

[0010] Also, since the conventional solutions involve manual operation, there is a considerable risk for manually caused errors, which may considerably degrade the accuracy of the localization.

[0011] An objective of the embodiments of the disclosure is to provide a solution which mitigates or solves the drawbacks of conventional solutions.

[0012] Another objective of the embodiments of the disclosure is to provide a solution which provides an accurate and efficient localization of a mining or construction vehicle in an underground mine.

[0013] Yet another object of the disclosure is to provide a robust and reliable solution for localization of a mining or construction vehicle in an underground mine.

[0014] Yet another object of the disclosure is to provide an alternative solution for localization of a mining or construction vehicle in an underground mine.

[0015] Yet another object of the disclosure is to improve automation of the process of localization of a mining or construction vehicle in an underground mine. The above and further objectives are solved by the subject matter of the appended independent claims.

[0016] According to a first aspect of the disclosure, the above mentioned and other objectives are achieved with a method performed by a control system for localization of a mining or construction machine in an underground mine, the method comprising:

[0017] - providing and / or receiving a global mine map;

[0018] - receiving one or more sensor measurements from a sensor system capable of emitting and detecting light;

[0019] - generating a virtual image based on the one or more sensor measurements, the virtual image comprising:

[0020] -- intensity information associated with portions of reflected light from two or more bolt ends comprised in a tunnel wall exposed to a light emission from the sensor system, respectively, and from an area surrounding each bolt end; and

[0021] -- depth information associated with distances between the sensor system and the two or more bolt ends, respectively;

[0022] - generating a local map based on the virtual image, the local map comprising:

[0023] -- two or more nodes associated with positions of the two or more bolt ends on the tunnel wall, respectively, and with a signature of the area surrounding each bolt end; and

[0024] -- one or more edges, each edge linking two nodes together; and

[0025] - localizing the mining or construction machine based on the local map and the global mine map.

[0026] Hereby, an accurate and reliable localization of a mining or construction machine in an underground mine is provided. The bolts comprised in the tunnel walls, i.e. bolts being fastened in the rock, and also the specific pattern in which the bolts are positioned in the rock, are utilized for the localization of the mining or construction machine. The local map, which comprises a detailed pattern of nodes and edges for a small portion of the mine, is here matched with, i.e. mapped onto, a global mine map, which comprises a corresponding detailed pattern of nodes and edges for a more complete environment of the mine, e.g. for most of the complete mine. Generally, when the pattern of nodes and edges in the local map is found somewhere in the global mine map, the position of the sensor system used for creating the local map, and thus also the location of the mining or construction machine, is found.

[0027] The two-channel / two dimensional virtual image comprises both intensity information and depth information, which makes the following generation of the local map more reliable. By including both the intensity information and the depth information in the virtual image, also bolt ends being covered, e.g. being sprayed with concrete, may be accurately positioned and included in the local map. If only intensity information had been included in the virtual image, such a covered bolt end would easily be missed in the local map, since the covered bolt end does not reflect more light than the rest of the rock of the tunnel wall, and may be essentially invisible in the intensity information.

[0028] To associate each node with a position of a bolt end on the tunnel wall and with also a signature of the area surrounding each bolt end provides additional information useful for localizing the mining or construction machine. The signature, comprising information related to geometry and / or an appearance of the area surrounding the bolt, is utilized in combination with the bolt end position when matching / mapping the local map onto the global mine map. Since both the position and the signature of each bolt are associated with the nodes, both in the local map and in the global mine map, all of this information may be used as a basis for determining a relative transformation between the local map and the global mine map. Hereby, the reliability and the accuracy of the localization of the mining or construction machine is considerably improved.

[0029] A further advantage of the presented method for localization of a mining or construction machine in an underground mine is that an automated localization process is provided. The manual work that was necessary in conventional solutions, such as handling of total stations and / or positioning of markers in the mine, may be omitted with the presented method. Hereby, a lot of time is saved and manual operations are made unnecessary. Therefore, error sources due to the human factor are minimized and costs are reduced, since the human interference is avoided. Thus, a more efficient, accurate and reliable localization of a mining or construction vehicle in an underground mine is provided by the herein presented disclosure.

[0030] In an embodiment of a method according to the first aspect, the generation of the virtual image is further based on at least one projection parameter and / or at least one distortion parameter for a sensor model utilized by the sensor system when performing the one or more sensor measurements.

[0031] Generally, the virtual image information is much faster to process than the raw sensor information. Thus, it is easier and more efficient for the subsequent steps of localizing and describing the bolts based on the virtual image than based on the raw sensor information. Furthermore, to generate the virtual image based on the projection model like this, makes it possible to adjust the field of view of the sensor, thereby removing irrelevant parts from the virtual image. This makes the subsequent processing even faster, and also reduces the risk of false positives.

[0032] In an embodiment of a method according to the first aspect, the distance between the sensor system and each of the two or more bolt ends is determined at a point on each bolt end being closest to the sensor system within a field of view cone associated with the sensor system.

[0033] Hereby, an accurate determination of the distance between the sensor system and the bolt end is provided, which results in an accurate depth information being included in the virtual image.

[0034] In an embodiment of a method according to the first aspect, the method further comprises:

[0035] - discarding the one or more sensor measurements after the virtual image has been generated.

[0036] An advantage with this embodiment is that memory space is saved, and the computational complexity is reduced. After the virtual image has been generated, the information associated with the one or more sensor measurements is not used anymore anyway. In an embodiment of a method according to the first aspect, the generation of the local map comprises:

[0037] - estimating a three-dimensional position for each of the two or more bolt ends in a coordinate system used by the sensor system;

[0038] - determining the area surrounding each of the two or more bolt ends;

[0039] - extracting features associated with a geometry and / or an appearance of the area surrounding each of the two or more bolt ends as the signature of the area;

[0040] - associating the three dimensional positions of the two or more bolt ends and the signatures of the areas surrounding the two or more bolt ends with two or more nodes, respectively; and

[0041] - linking nodes by edges based on at least one linkage condition.

[0042] An advantage with this embodiment is that each node in the local map is associated with both a position of a bolt end on the tunnel wall and with a signature of the area surrounding the bolt end, which provides a lot of information useful as a basis for localizing the mining or construction machine. The signature comprises information related to geometry and / or an appearance of the area surrounding the bolt, such as e.g. shape, form, general and / or specific geometry features, material, color and / or structure of the tunnel wall surrounding the bolt ends. This information, in combination with the bolt end position, is utilized when matching / mapping the local map onto the global mine map. Since both the position and the signature of each bolt are associated with the nodes, both in the local map and in the global mine map, all of this information may be used as a basis for determining a relative transformation between the local map and the global mine map, which improves the reliability and the accuracy of the localization of the mining or construction machine.

[0043] In an embodiment of a method according to the first aspect, the at least one linkage condition comprises one or more in the group of:

[0044] - two nodes are linked together if the two associated bolt ends are located at a node distance ND from each other, the node distance ND being smaller than a node distance threshold NDth; ND<NDth;

[0045] - two nodes are linked together if both the two associated bolt ends are determined to be located on a common tunnel wall surface;

[0046] - two nodes are linked together if a line between the two associated bolt ends is separated from a tunnel wall surface by a wall distance WD being smaller than a wall distance threshold WDth; WD<WDth; for a portion P of the line exceeding a portion threshold Pth; P>Pth.

[0047] Thus, various embodiments are provided for reliably and accurately determining if two nodes should be linked by an edge in the local map. Basically, it is here determined if the two nodes are close to each other and are located on a common tunnel wall surface, and if this is the case, the two nodes should be linked by an edge. Hereby, a well-defined pattern of nodes and edges is reliably and accurately created in the local map, and is eventually also included in the global mine map by the incremental updates of the global mine map based on the local map.

[0048] In an embodiment of a method according to the first aspect, the method further comprises:

[0049] - discarding the virtual image after the local map has been generated.

[0050] An advantage with this embodiment is that memory space is saved, and the computational complexity is reduced. After the local map has been generated, the virtual image is not used anymore anyway.

[0051] In an embodiment of a method according to the first aspect, the localization of the mining or construction machine comprises:

[0052] - determining a relative transformation between the local map and the global mine map;

[0053] - determining, based on the relative transformation, a three dimensional position and an orientation of the sensor system relative to a coordinate frame of the global mine map.

[0054] Since both the local map and the global mine map have been generated with high accuracy, the pose of the sensor system, i.e. the three dimensional position and orientation of the sensor system relative to a coordinate frame of the global mine map, is efficiently and accurately determined based on the relative transformation between the local map and the global mine map. The relative transformation between the local map and the global mine map is generally a matching / mapping of the local map onto the global mine map, based on the detailed patterns of nodes and edges comprised both in the local map and in the global mine map. In an embodiment of a method according to the first aspect, the determination of the relative transformation is performed by utilization of at least one selected from the group consisting of:

[0055] - an iterative randomized fitting algorithm;

[0056] - a random sample consensus algorithm;

[0057] - an algorithm based on soft data association; and

[0058] - an algorithm based on machine learning.

[0059] Hereby, an algorithm suitable for the determination of the relative transformation may be chosen based on the current application and / or situation, such that a high accuracy relative transformation is reliably determined.

[0060] In an embodiment of a method according to the first aspect, the method further comprises:

[0061] - providing as an output the determined three-dimensional position and orientation of the sensor system as an indication of the localization of the mining or construction machine.

[0062] Hereby, an automated localization of a mining or construction machine in an underground mine is provided. The manual work that was necessary in conventional solutions, such as handling of total stations and / or positioning of markers in the mine, may be omitted with the presented localization method. Hereby, a lot of time is saved and manual operations may be avoided. Costs are hereby reduced, and error sources due to the human factor are minimized.

[0063] In an embodiment of a method according to the first aspect, the method further comprises:

[0064] - updating the global mine map based on the generated local map and the determined three-dimensional position and orientation of the sensor system.

[0065] Hereby, the global mine map may be continuously updated as the drilling in the mine develops / proceeds. The information comprised in the high quality global map, such as the nodes, the edges, and the signatures, is utilized as a basis for updating the global mine map. Since the local maps are generated over time as the mining proceeds, the global mine map is hereby kept up to date, without a need for manually updating the global mine map. In an embodiment of a method according to the first aspect, the update of the global mine map is performed by utilization of at least one selected from the group consisting of:

[0066] - at least one probabilistic rule;

[0067] - information accumulation; and

[0068] - at least one pooling operation.

[0069] Hereby, an algorithm suitable for updating the global mine map may be chosen based on the current application and / or situation, such that a reliable and high accuracy global mine map is generated.

[0070] In an embodiment of a method according to the first aspect, the one or more sensor measurements comprise at least one selected from the group consisting of:

[0071] - point cloud information;

[0072] - raw range information;

[0073] - bearing and intensity information; and

[0074] - stereo image information.

[0075] The measurements may thus comprise a high accuracy set of data points in space, such as a three dimensional digital representation of the tunnel wall, having been detected by the one or more sensor. Based on this digital representation, a high- quality generation of the virtual image may be performed. Hereby, also the local map, and finally the global mine map, may be generated with high accuracy.

[0076] According to a second aspect of the disclosure, the above mentioned and other objectives are achieved with a control system for localization of a mining or construction machine in an underground mine, wherein the control system is configured to perform the herein described method.

[0077] The control system according to the second aspect can be extended into embodiments corresponding to the embodiments of the method according to the first aspect. Hence, an embodiment of the control system comprises the feature(s) of the corresponding embodiment of the method. The advantages of the control system according to the second aspect and its embodiments are the same as those for the corresponding aspect and embodiments of the method according to the first aspect mentioned above.

[0078] According to a third aspect of the disclosure, the above mentioned and other objectives are achieved with a mining or construction machine comprising the herein described control system and a herein described sensor system.

[0079] The advantages of the mining or construction machine according to the third aspect and its embodiments are the same as those for the corresponding aspect and embodiments of the method according to the first aspect mentioned above.

[0080] According to an embodiment of the third aspect of the disclosure, the mining or construction machine is a drilling rig.

[0081] The advantages of the drilling rig are the same as those for the corresponding aspect and embodiments of the method according to the first aspect mentioned above.

[0082] Embodiments of the disclosure also relate to a computer program, characterized in program code, which when run by at least one processor causes the at least one processor to execute any method according to embodiments of the disclosure. Further, embodiments of the disclosure also relate to a computer program product comprising a computer readable medium and the mentioned computer program, wherein the computer program is included in the computer readable medium, and may comprises one or more from the group of: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), flash memory, electrically erasable PROM (EEPROM), hard disk drive, etc.

[0083] Further applications and advantages of embodiments of the disclosure will be apparent from the following detailed description.

[0084] Brief Description of the Drawings

[0085] The appended drawings are intended to clarify and explain different embodiments of the disclosure, in which: - Fig. 1 shows a flow chart diagram for an exemplary method according to embodiments of the disclosure;

[0086] - Fig. 2 schematically illustrates a block diagram according to embodiments of the disclosure;

[0087] - Figs. 3a-b schematically illustrate some details associated with bolts of a tunnel wall in an underground mine;

[0088] - Fig. 4 shows an exemplary local map according to embodiments of the disclosure;

[0089] - Fig. 5 shows an exemplary mining or construction machine, in which embodiments of the disclosure may be utilized; and

[0090] - Fig. 6 schematically illustrates a control unit according to some embodiments of the disclosure.

[0091] Detailed Description

[0092] Accurate, efficient and reliable underground localization of mining or construction vehicles, for example in an underground / subterranean mine tunnel, is generally complicated and difficult to provide. Positioning systems, such as e.g. the global positioning system, that work well on the surface of the earth do generally not work in a mine, since reliable communication between the underground mining or construction vehicle and satellites or terrestrial nodes is not possible.

[0093] Figure 1 illustrates a flow chart for a method 200 performed by a control system 400 for localization of a mining or construction machine 100 in an underground mine 500. Figure 2 schematically shows a block diagram for some entities of the control system 400. Figure 3a shows an example tunnel wall 510 comprising some bolts ends 512a, 512b, 512c of bolts. Figure 4 shows a non-limiting example of a local map 431 .

[0094] Figure 5 schematically shows mining or construction machine 100 in an underground mine 500. Figures 2, 3a, 4 and 5 are explained more in detail below, and are mentioned here in order to help understanding the method of the flow chart in figure 1.

[0095] In a first step 210 of the method 200, the control system 400 provides and / or receives a global mine map 461 . Thus, the control system 400 may itself generate and provide the global min map 461 , as described more in detail below, and / or may receive the global mine map 461 from an external entity, such as for example a more or less remotely located control center. As explained below, the global mine map 461 may comprise a bolt feature graph covering one or more parts of the mine 500, and may be generated and / or updated incrementally / iteratively over time. In some cases the global mine map 461 may cover large parts of the mine, for example the complete mine 500.

[0096] In a second step 220, the control system 400 receives one or more sensor measurements 411 from a sensor system 410 capable of emitting and detecting light. The sensor system 410 is described more in detail below.

[0097] In a third step 230, the control system 400 generates a virtual image 421 based on the one or more sensor measurements 411 provided by the sensor system 410. The virtual image 421 may for example be generated by a virtual view rendering entity 420. The virtual image 411 may be a two-channel / two-dimensional image / representation comprising intensity information 422 and depth information 423.

[0098] The intensity information 422 is associated with portions of reflected light from two or more bolt ends 512a, 512b, 512c comprised in a tunnel wall 510 exposed to a light emission from the sensor system 410, respectively. The intensity information 422 is also associated with portions of reflected light from an area 513a, 513b surrounding each bolt end 512a, 512b, 512c.

[0099] The depth information 423 is associated with distances D between the sensor system 410 and the two or more bolt ends 512a, 512b, 512c, respectively.

[0100] In this document, each of the two or more bolt ends 512a, 512b, 512c may comprise an end part of the actual bolt and / or a surface of a grouted hole with the bolt installed within the hole using cement- or resin-based grout material.

[0101] Generally, metal bolt ends are better reflectors than the surrounding rock, wherefore visible metal bolt ends may be easily identified based only on the intensity information 422. However, if the bolt ends are covered somehow, e.g. by concrete or dirt, the identification of the bolt ends is more complicated / difficult. According to the herein presented disclosure, the virtual image 421 comprises both the intensity information 422 and the depth information 423, that may be utilized in combination to identify also covered bolts. Thus, if the intensity information 422 is combined with the depth information 423, whereby also covered bolt ends may be detected / identified based on the virtual image 421 .

[0102] In a fourth step 240, the control system 400 generates a local map 431 based on the virtual image 421 . The local map 431 may for example be generated by a bolt feature extractor entity 430.

[0103] The local map 431 comprises two or more nodes 432a, 432b associated with positions of the two or more bolt ends 512a, 512b, 512c on the tunnel wall 510, respectively. The two or more nodes 432a, 432b are also associated with a signature of the area 513a, 513b surrounding each bolt end 512a, 512b, 512c. The local map 431 further comprises one or more edges 433, where each edge 433 links two nodes 432a, 432b together.

[0104] In figure 4, the two or more nodes and the one or more edges of the loval map 431 are exemplified / illustrated as two nodes 432a, 432b linked / connected together by an edge 433 for readability reasons. As realized by a skilled person, however, such edges 433 may link / connect also other nodes shown in figure 4.

[0105] In a fifth step 250, the control system 400 localizes the mining or construction machine 100 based on the local map 431 and the global mine map 461. The localization 250 of the mining or construction machine 100 may be performed by a localizer entity 440. The global mine map 461 may comprise a bolt feature graph / pattern covering one or more parts of the mine 500, as mentioned above. The local map 431 is a locally generated map generated for a more restricted part of the mine at the current position of the mining or construction machine 100. Therefore, the two or more nodes 432a, 432b and the one or more edges 433 of the local map 431 may be compared to the corresponding information of the global mine map 461. Based on this comparison, which is described more in detail below, the position of the generated local map 431 in in relation to the global mine map 461 , and therefore also the position of the sensor system 410 and thus the mining or construction machine 100, may be determined.

[0106] According to an embodiment, the generation 230 of the virtual image 421 is further based on at least one projection parameter and / or at least one distortion parameter 412 for a sensor model utilized by the sensor system 410 when performing the one or more sensor measurements 411 . Various sensor models may be utilized by the sensor system 410 when performing the one or more sensor measurements 411 , and parameters of such sensor models may then be taken into account by the virtual view rendering entity 420 when generating the virtual image 421 . As a non-limiting example, a fisheye projection model may be utilized as a sensor model to cover a wide tunnel wall surface with the measurements 411.

[0107] The at least one projection parameter and / or at least one distortion parameter 412 utilized by the sensor model, and here taken into account in the generation 230 of the virtual image 421 , may be determined in any suitable way, for example at least partially by manual determination, by classic calculations, by computer calculations, by utilization of machine learning algorithms, and / or by utilization of artificial intelligence algorithms.

[0108] The one or more sensor measurements 411 may comprise point cloud information, raw range information, bearing and intensity information and / or stereo image information. This information may be provided by any therefore suitable optical measuring device / instrument, such as for example a light detection and ranging (LIDAR) device, one or more radar arrangements, a photogrammetry sensor arrangement, and / or a fixed offset camera arrangement.

[0109] According to an embodiment schematically illustrated in figure 3a, the distance D between the sensor system 410 and each of the two or more bolt ends 512a, 512b, 512c is determined at a point 514b on each bolt end 512a, 512b, 512c being closest to the sensor system 410 within a field of view cone 413b associated with the sensor system 410.

[0110] For example, according to one sensor model, the virtual image 421 may be generated by controlling a virtual ray to pass through a virtual camera and a virtual lens of the sensor system 410, and to reach the tunnel wall 510, such that a field of view cone 413b is created. Then, each pixel of the virtual image 421 is generated to correspond to a certain point of the tunnel wall 510. The intensity information 422 and the depth information 423 is then generated for the point on each bolt end 512a, 512b, 512c being closest to the sensor system 410 within the field of view cone 413b.

[0111] According to an embodiment, the one or more sensor measurements 411 received from the sensor system 410 are discarded 231 after the virtual image 421 has been generated 230 e.g. by the virtual view rendering entity 420. After the virtual image 421 has been generated 230, the information associated with the one or more sensor measurements 411 are not used anymore. Therefore, to save memory, this information may be deleted.

[0112] According to an embodiment, the generation 240 of the local map 431 shown schematically in figure 4 comprises a number of steps.

[0113] A three-dimensional position 515a, 515b, 515c for each of the two or more bolt ends 512a, 512b, 512c is estimated 241 in a coordinate system used by the sensor system 410. Three such three-dimensional positions 515a, 515b, 515c are schematically illustrated in figure 3a.

[0114] Further, the area 513a, 513b surrounding each of the two or more bolt ends 512a, 512b, 512c is determined 242, illustrated for two bolt ends 512a, 512b in figure 3a. Thus, around and adjacent to each of the two or more bolt ends 512a, 512b, 512c an area 513a, 513b is defined for which intensity information 422 should be generated 230.

[0115] Further, features associated with a geometry and / or an appearance of the area 513a, 513b surrounding each of the two or more bolt ends 512a, 512b, 512c are extracted 243 as the signature of the area 513a, 513b. Thus, in addition the detection / identification of the bolt ends themselves, also a shape, form, geometry, material, color and / or structure of the tunnel wall 510 surrounding the bolt ends are extracted 243. These one or more features of the tunnel wall area 513a, 513b surrounding the bolt end are extracted 243 as a signature of that area 513a, 513b.

[0116] Further, the three dimensional positions 515a, 515b, 515c of the two or more bolt ends 512a, 512b, 512c and the signatures of the areas 513a, 513b surrounding the two or more bolt ends 512a, 512b, 512c are associated 244 with two or more nodes 432a, 432b, respectively. Thus, the three dimensional position and the surrounding area signature for each bolt end associated 244 with a node of the local map 431.

[0117] For example, the first three dimensional position 515a and the first signature of the surrounding area 513a for a first bolt end 512a illustrated in figure 3a are associated with a first node 432a of the local map, shown in figure 4. Correspondingly, the second three dimensional position 515b and the second signature of the surrounding area 513b for a second bolt end 512b illustrated in figure 3a are associated with a second node 432b of the local map, shown in figure 4.

[0118] Further, nodes 432a, 432b are then linked 245 by edges 433 based on at least one linkage condition. For example, as shown in figure 4, the first node 432a of the local map is linked 245 to a second node 432b of the local map by an edge 433, if at least one linkage condition is fulfilled.

[0119] The herein described generation 240 of the local map 431 may be determined in any suitable way, for example at least partially by manual determination, by classic calculations, by computer calculations, by utilization of machine learning algorithms, and / or by utilization of artificial intelligence algorithms.

[0120] According to an embodiment, the at least one linkage condition comprises a node distance ND condition, i.e. a proximity measure, as schematically illustrated in figures 3b and 4. Here, two nodes 432a, 432b are linked together by an edge 433 if the two associated bolt ends 512a, 512b are located close to each other. Thus, two nodes 432a, 432b are linked together if the two associated bolt ends 512a, 512b are at a node distance ND from each other, where the node distance ND is smaller than a node distance threshold NDth; ND<NDth.

[0121] According to an embodiment, the at least one linkage condition comprises a common surface condition, such that two nodes 432a, 432b are linked together by an edge 433 if both of the two associated bolt ends 512a, 512b are determined to be located on a common tunnel wall surface 511 , as schematically illustrated in figure 3b.

[0122] According to an embodiment schematically illustrated in figure 3b, two nodes 432a, 432b are linked together if a line 516 between the two associated bolt ends 512a, 512b is separated from a tunnel wall surface 511 by a wall distance WD being smaller than a wall distance threshold WDth; WD<WDth; for a portion P of the line 516 exceeding a portion threshold Pth; P>Pth. In other words, if most of the line 516 between the two associated bolt ends 512a, 512b is close to the wall surface 511 , it is determined that the bolt ends 512a, 512b are located on a common tunnel wall surface 511 , and the two nodes 432a, 432b are linked together by an edge 433.

[0123] According to an embodiment, the virtual image 421 is discarded 246 after the local map 431 has been generated 240. After the local map 431 has been generated 240, the virtual image 421 is not used anymore. Therefore, the virtual image 421 may be deleted to save memory.

[0124] According to an embodiment, the localization 250 of the mining or construction machine 100 comprises a determination 251 of a relative transformation between the local map 431 and the global mine map 461 . This may also be described as the goal is to determ ine / find 251 a relative transformation of the local map 431 with respect to the global mine map 461 , such that the local map 431 is fitted / matched with or mapped to the global mine map 461 . The relative transformation may here be associated with an offset between a currently performed sensor measurement 411 a previously performed measurement.

[0125] It should here be noted that the global mine map 461 is built on the same principles as the local map 431 . Thus, the global map 461 also comprises nodes associated with bolt ends and corresponding signatures, where the nodes are linked by edges based on at least one linkage condition. Thus, the determination of the relative transformation generally comprises finding a portion of the global mine map 461 which matches the currently generated local map 431 .

[0126] When the relative transformation has been determined 251 , a three dimensional position 414 and an orientation 415 of the sensor system 410 relative to a coordinate frame of the global mine map 461 are determined 252 based on the relative transformation. Thus, when there is a match / mapping found between the currently generated local map 431 and a portion of the global mine map 461 , then the three dimensional position 414 and the orientation 415 of the sensor system 410 may be pinpointed based on the global mine map 461 .

[0127] The determination 251 of the relative transformation may be performed based on the local map 431 and the global mine map 461 in essentially any suitable way, for example by at least partially utilizing an iterative randomized fitting algorithm, a random sample consensus algorithm, an algorithm based on soft data association and / or an algorithm based on machine learning.

[0128] According to an embodiment schematically illustrated in figure 2, the control system 400 provides 260 the determined three-dimensional position 414 and orientation 415 of the sensor system 410 as an output 441 .

[0129] This output 441 , provided by the localizer entity 440, may be seen as an indication of the localization of the mining or construction machine 100. Thus, since the three- dimensional position 414 and orientation 415 of the sensor system 410 are comprised in the output 441 , and since the position of the sensor system 410 on the mining or construction machine 100 is well known, a localization of the mining or construction machine 100 is easily determined based on the output 441 .

[0130] Further, the generated local map 431 and the determined three-dimensional position 414 and orientation 415 of the sensor system 410 may be output 442 from the localizer entity 440 to a global map update entity 450. The three-dimensional position 414 and orientation 415 of the sensor system 410 may then be utilized as a basis for updating 270 the global mine map 461 stored in a global map entity 460. Nodes and edges of the global mine map 461 are here updated based on corresponding determined nodes and edges of the local map 431 .

[0131] According to an embodiment, the update 270 of the global mine map 461 is performed by merging the information of the global mine map 461 with the information of the determined local map 431 by utilization of essentially any suitable algorithm, comprising at least partly at least one probabilistic rule, information accumulation and / or at least one optional pooling operation. The hereby updated global mine map 461 is thereafter based on, and comprises information of, the currently determined local map 431 .

[0132] The very first time a local map 431 is generated 423 for the mine 500, this local map 431 may be stored in the global map entity 460 as the global mine map 461 . Also, the determined three-dimensional position 414 and orientation 415 of the sensor system 410 may then be utilized as a global reference frame of the global mine map 461. Alternatively, a manually operated total station may be utilized provide the three- dimensional position 414 and orientation 415 of the sensor system 410 for the first measurements. Such a first initial update 270 of the global mine map 461 is thus actually an update of a previously blank global mine map 461 . Thereafter, the global map update entity 450 may incrementally construct the global mine map 461 representing a bolt feature graph for one or more parts of the mine 500 based on further generated local maps. After a number of, possibly many, iterations / updates 270 of the global mine map 461 based on generated local maps, the global mine map 461 may cover large parts of, possibly the whole environment of, the mine 500.

[0133] According to an aspect of the disclosure schematically illustrated in figures 2 and 5, a control system 400 for localization of a mining or construction machine 100 in an underground mine 500 is presented. The control system 400 is configured to perform the aspects and embodiments of the method described in this document.

[0134] According to an aspect, a mining or construction machine 100 comprising a herein described control system 400 and a herein described sensor system 410 is presented. According to an embodiment, the mining or construction machine 100 is a drilling rig.

[0135] Such a drilling rig 100 is schematicaly illustrated in figure 5. The drilling rig 100 may be utilized for drilling holes in a tunnel wall 510. The illustrated drilling rig 100 is only exemplary mining or construction machine, and the herein presented disclosures may be implemented using various kinds of mining or construction machines and / or drill rigs of various designs.

[0136] More in detail, figure 5 schematically illustrates an example drill rig 100 configured to perform a drilling process, such as drilling of holes, e.g. during tunnelling or mining, in which the aspects and / or embodiments herein described may be implemented. The drill rig 100 includes a boom 101 , one end 101a of which being attached, according to the present example, in such a way that it can pivot in relation to a carrier 102, such as a vehicle, via one or more articulated connections (not shown). A feeder 103 that carries a drilling machine 104 is attached to the other end 101b of the boom 101 via one or more articulated connections, such as one or more rotators (not shown). The drilling machine 104 may be movable along the feeder 103 such that the drill string, and thus also the drill bit at the end of it, continues creating a hole in the rock wall 510. The carrier 102 further comprises crawlers and / or wheels 105 facilitating tramming of the drill rig 100, i.e. facilitating the drill rig 100 to move from one position to another, for example between holes to be drilled, driven by a tramming system.

[0137] The drill rig 100 further comprises a herein described control system 400, possibly implemented in a control unit 130, and a herein described sensor system 410 connected to the control system 400.

[0138] Figure 6 schematically illustrates a control unit 130. The mining or construction machine 100 shown in figure 5 comprises a control system 400 comprising or being implemented in at least one control unit 130, which controls various functions of the mining or construction machine 100, e.g., by suitable control of various actuators / motors / pumps etc. Mining or construction machines 100 of the disclosed kind may comprise more than one control unit, where each control unit, respectively, may be arranged to be responsible for different functions of the mining or construction machine 100. According to examples of the disclosure, the herein described method steps 210, 220, 230, 231 , 240, 241 , 242, 243, 244, 245, 246, 250, 251 , 252, 260, 270, may be controlled by any suitable control unit of the mining or construction machine 100, such as the control unit 130. Correspondingly, the herein disclosed virtual view rendering entity 420, bolt feature extractor entity 430, localizer entity 440, global map update entity 450, and global map entity 460 may be implemented in any suitable control unit of the mining or construction machine 100, such as the control unit 130, possibly as one or more sections of programming code. The functionality of the disclosure may also be divided among more than one control units. According to examples of the disclosure, one control unit may comprise at least part of the functionality of one or more of the disclosed virtual view rendering entity 420, bolt feature extractor entity 430, localizer entity 440, global map update entity 450, and global map entity 460, whereas another control unit may comprise at least part of the functionality of one or more of the disclosed virtual view rendering entity 420, bolt feature extractor entity 430, localizer entity 440, global map update entity 450, and global map entity 460, and so on.

[0139] The control unit 130 comprises a data processing unit 141 which, based on received signals, and by means of suitable calculations, performs the steps according to the examples of the disclosure described herein. The processing unit 141 can, for example, be constituted by a processor, such as a digital signal processor. The control unit 140 may be controlled by means of a computer program 142 that is, e.g., built into the processor or being connected thereto. The computer program may be generated by means of an appropriate programming language and be stored in a non-transitory computer memory 143 that is integrated in the processor or form a separate part of the control unit 130. The control unit 130 may further comprises a transceiver module 144 for receiving / transmitting signals. The transceiver module 144 may, e.g., also constitute an interface for other signals being received and / or transmitted by the control unit 130.

[0140] The processing unit 141 may be referred to and / or may comprise one or more general-purpose central processing units (CPUs), one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more programmable logic devices, or any other one or more discrete or logic devices / components / circuits / chipsets. The computer memory 143 may be a readonly memory (ROM), a random access memory (RAM), or a non-volatile RAM (NVRAM). The transceiver module 144 may be a transceiver circuit, a power controller, or an interface providing capability to communicate with other communication modules or communication devices. The transceiver module 144, computer memory 143 and / or processing unit 141 may be implemented in separate components or may be implemented in a common component.

[0141] Finally, it should be understood that the disclosure is not limited to the embodiments described above, but also relates to and incorporates all embodiments within the scope of the appended independent claims.

Claims

CLAIMS1 . A method (200) performed by a control system (400) for localization of a mining or construction machine (100) in an underground mine (500), the method (200) comprising:- providing and / or receiving (210) a global mine map (461 );- receiving (220) one or more sensor measurements (411) from a sensor system (410) capable of emitting and detecting light;- generating (230) a virtual image (421 ) based on the one or more sensor measurements (411 ), the virtual image (421 ) comprising:-- intensity information (422) associated with portions of reflected light from two or more bolt ends (512a, 512b, 512c) comprised in a tunnel wall (510) exposed to a light emission from the sensor system (410), respectively, and from an area (513a, 513b) surrounding each bolt end (512a, 512b, 512c); and-- depth information (423) associated with distances (D) between the sensor system (410) and the two or more bolt ends (512a, 512b, 512c), respectively;- generating (240) a local map (431 ) based on the virtual image (421 ), the local map (431 ) comprising:-- two or more nodes (432a, 432b) associated with positions of the two or more bolt ends (512a, 512b, 512c) on the tunnel wall (510), respectively, and with a signature of the area (513a, 513b) surrounding each bolt end (512a, 512b, 512c); and -- one or more edges (433), each edge (433) linking two nodes (432a, 432b) together; and- localizing (250) the mining or construction machine (100) based on the local map (431) and the global mine map (461 ).

2. The method (200) according to claim 1 , wherein the generation (230) of the virtual image (421) is further based on at least one projection parameter and / or at least one distortion parameter (412) for a sensor model utilized by the sensor system (410) when performing the one or more sensor measurements (411 ).

3. The method (200) according to any one of claims 1-2, wherein the distance (D) between the sensor system (410) and each of the two or more bolt ends (512a, 512b, 512c) is determined at a point (514b) on each bolt end (512a, 512b, 512c) beingclosest to the sensor system (410) within a field of view cone (413b) associated with the sensor system (410).

4. The method (200) according to any one of claims 1-3, further comprising:- discarding (231 ) the one or more sensor measurements (411 ) after the virtual image (421 ) has been generated (230).

5. The method (200) according to any one of claims 1-4, wherein the generation (240) of the local map (431 ) comprises:- estimating (241 ) a three-dimensional position (515a, 515b, 515c) for each of the two or more bolt ends (512a, 512b, 512c) in a coordinate system used by the sensor system (410);- determining (242) the area (513a, 513b) surrounding each of the two or more bolt ends (512a, 512b, 512c);- extracting (243) features associated with a geometry and / or an appearance of the area (513a, 513b) surrounding each of the two or more bolt ends (512a, 512b, 512c) as the signature of the area (513a, 513b);- associating (244) the three dimensional positions (515a, 515b, 515c) of the two or more bolt ends (512a, 512b, 512c) and the signatures of the areas (513a, 513b) surrounding the two or more bolt ends (512a, 512b, 512c) with two or more nodes (432a, 432b), respectively; and- linking (245) nodes (432a, 432b) by edges (433) based on at least one linkage condition.

6. The method (200) according to claim 5, wherein the at least one linkage condition comprises one or more in the group of:- two nodes (432a, 432b) are linked together if the two associated bolt ends (512a, 512b) are located at a node distance ND from each other, the node distance ND being smaller than a node distance threshold NDth; ND<NDth;- two nodes (432a, 432b) are linked together if both the two associated bolt ends (512a, 512b) are determined to be located on a common tunnel wall surface (511 );- two nodes (432a, 432b) are linked together if a line (516) between the two associated bolt ends (512a, 512b) is separated from a tunnel wall surface (511) by a wall distance WD being smaller than a wall distance threshold WDth; WD<WDth; for a portion P of the line (516) exceeding a portion threshold Pth; P>Pth.

7. The method (200) according to any one of claims 1-6, further comprising:- discarding (246) the virtual image (421 ) after the local map (431 ) has been generated (240).

8. The method (200) according to any one of claims 1-7, wherein the localization (250) of the mining or construction machine (100) comprises:- determining (251 ) a relative transformation between the local map (431 ) and the global mine map (461 );- determining (252), based on the relative transformation, a three dimensional position (414) and an orientation (415) of the sensor system (410) relative to a coordinate frame of the global mine map (461 ).

9. The method (200) according to claim 8, wherein the determination (251 ) of the relative transformation is performed by utilization of at least one selected from the group consisting of:- an iterative randomized fitting algorithm;- a random sample consensus algorithm;- an algorithm based on soft data association; and- an algorithm based on machine learning.

10. The method (200) according to any one of claims 8-9, further comprising:- providing (260) as an output (441 ) the determined three-dimensional position (414) and orientation (415) of the sensor system (410) as an indication of the localization of the mining or construction machine (100).11 . The method (200) according to any one of claims 8-10, further comprising:- updating (270) the global mine map (461) based on the generated local map (431 ) and the determined three-dimensional position (414) and orientation (415) of the sensor system (410).

12. The method (200) according to claim 11 , wherein the update (270) of the global mine map (461) is performed by utilization of at least one selected from the group consisting of:- at least one probabilistic rule;- information accumulation; and- at least one pooling operation.

13. The method (200) according to any one of claims 1-12, wherein the one or more sensor measurements (411 ) comprise at least one selected from the group consisting of:- point cloud information;- raw range information;- bearing and intensity information; and- stereo image information.

14. Computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method according to any one of the claims 1 -13.

15. Computer-readable medium (142) comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of the claims 1 -13.

16. A control system (400) for localization of a mining or construction machine (100) in an underground mine (500), the control system (400) being configured to perform the method according to any one of claims 1-13.

17. A mining or construction machine (100) comprising- a control system (400) according to claim 16; and- a sensor system (410).

18. A mining or construction machine (100) according to claim 17, wherein the mining or construction machine (100) is a drilling rig.

Images