Method and system for enhancing a mining machine's perception of its surrounding environment

Abstract

A method for enhancing a mining machine's perception of its surrounding environment is provided, whereby the mining machine operates in an underground environment : - Remote sensor data associated with the environment surrounding the mining machine is obtained (302) from at least one remote sensor, - The mining machine's perception of its surrounding environment is enhanced by generating (303) perception data associated with the environment surrounding the mining machine based on the remote sensor data obtained from the at least one remote sensor.

Description

[0001] METHOD AND SYSTEM FOR ENHANCING A MINING MACHINE'S PERCEPTION OF ITS SURROUNDING ENVIRONMENT

[0002] TECHNICAL FIELD

[0003] Embodiments herein relate to a system, control unit, mining machine and method for enhancing the mining machine’s perception of it surrounding environment.

[0004] BACKGROUND

[0005] In mining and tunneling, there is a constant ongoing process of improving efficiency, productivity and safety. Examples of changes and / or improvements that are carried out with an increasing extent, especially in mining, is the automation, fully or partly, of various processes occurring in mining. Methods for localization, mapping, control and motion planning have enabled development and deployment of fully or partly autonomous vehicles and / or mobile machines.

[0006] When working in underground environments, it is often important to gather knowledge about where various objects are located. The underground environments may in many cases have a significant extent and be difficult to overview. Collecting information about positions may therefore be partly problematic. Direct visual observation is made difficult by intervening rock, but also by the fact that it is dark. Other methods are therefore used.

[0007] When operating heavy machinery, manually or automatically, it is essential to know the surroundings. This is typically achieved by several sensors and cameras on the machine. In addition, central systems can be used to provide additional information regarding the surroundings and where machines are located. The use of central intelligence is however not always suitable. Due to the harsh conditions, it can also be difficult to achieve optimal viewpoints from local sensors. For example a protruding sensor may be knocked off when passing too closely to the rock wall In addition, machines such as loaders have big moving parts that also obstruct the view.

[0008] SUMMARY As part of developing embodiments herein a problem has been identified and will first be discussed.

[0009] A problem in underground environments relates to the perception of the environment surrounding a mining machine. As mentioned above, sensors and / or cameras on the machine may be used to gain information of the surrounding environment. However, sensors located on a mining machine may suffer from a limited field of view. This may e.g., be caused by large parts of mining machine blocking the field of view of the sensors. One way to solve this problem would be to move the sensors to other locations on the mining machine. This may however not be possible. E.g., the tunnels in the underground environment may be narrow and the sensors may be knocked off, resulting in an even more limited perception of the surrounding environment. Further, moving parts of the mining machine may cause the field of view to be blocked even if the sensor were to the relocated. The mining machine’s limited perception of its surrounding environment may increase the risk of collisions and / or accidents with e.g., other machines, vehicles, operators walls and / or fixed infrastructure in the underground environment. Further, this may decrease the operating efficiency in the underground environment.

[0010] An object of embodiments herein is to provide a mechanism that increases operational safety of mining machines and improves the operational efficiency and productivity in a delimited environment, such as an underground environment. The object is achieved by the independent claims.

[0011] According to a first aspect, a method for enhancing a mining machine’s perception of its surrounding environment is provided. The mining machine operates in an underground environment.

[0012] Remote sensor data associated with the environment surrounding the mining machine is obtained from at least one remote sensor.

[0013] The mining machine’s perception of its surrounding environment is enhanced by generating perception data associated with the space surrounding the mining machine based on the remote sensor data obtained from the at least one remote sensor.

[0014] By generating perception data, the operation efficiency, safety and productivity may be improved. This is enabled by enhancing the mining machine’s perception of it surrounding environment, which may enable the mining machine to operate more efficiently and avoiding collisions and / or accidents. According to an exemplary embodiment, generating the perception data may further comprise combing the remote sensor data with local sensor data obtained from at least one local sensor comprised in the mining machine. The perception data may be generated based on the combined sensor data. By combing the remote sensor data with the local sensor data, the mining machine’s perception of its surrounding environment may be enhanced. This may be enabled by a comprehensive sensor data being used when generating the perception data.

[0015] According to an exemplary embodiment, the remote sensor data may comprise sensor data not comprised in the local sensor data collected by the at least one local sensor. This may enable the perception of the mining machine’s 110 surrounding environment to be enhanced, since more sensor data is available.

[0016] According to an exemplary embodiment, a part of the mining machine’s surrounding environment may be blocked from a field of view of the at least one local sensor, prohibiting the at least one local sensor from collecting local sensor data associated with the blocked part of the surrounding environment.

[0017] According to an exemplary embodiment, the remote sensor data may at least partly overlap the blocked area. By obtaining remote sensor data that at least partly overlaps the blocked area, the mining machine’s perception of its surrounding environment may be enhanced. This is enabled by the remote sensor data complementing the local sensor data such that the combined sensor data may cover more of the mining machine’s surrounding environment.

[0018] According to an exemplary embodiment, generating the perception data may comprise generating a model of the mining machine’s surrounding environment based on the perception data. The model may enable an improved control of operations of the mining machine. This is enabled by that the model may be used when controlling the operation, and since the model is based on the generated perception data, the mining machine to operate more efficiently and avoiding collisions and / or accidents.

[0019] According to an exemplary embodiment, the local sensor data is obtained from the at least one local sensor. The local sensor data is associated with a limited part of the mining machine’s surrounding environment.

[0020] According to an exemplary embodiment, the at least one local sensor and the at least one remote sensor may comprise any one or more out of: A camera, a radar, a lidar, and an ultrasound sensor. By providing the possibility to use types of sensors, perception data may be more flexibly generated, where the type of sensor is chosen based on the type of sensor data most suitable. According to an exemplary embodiment, an operation of the mining machine is controlled taking the generated perception data into account.

[0021] By controlling the operation of the mining machine 110 utilizing the perception data, the operation efficiency, safety and productivity may be improved. This is enabled by that the operation may more efficiently and securely performed since the mining machine’s perception of its surrounding environment is enhanced.

[0022] According to a second aspect, a control unit configured to enhance a mining machine’s perception of its surrounding environment is provided.

[0023] The control unit is configured to obtain, from at least one remote sensor, remote sensor data adapted to be associated with the environment surrounding the mining machine.

[0024] The control unit is configured to enhance the mining machine’s perception of its surrounding environment by generating perception data adapted to be associated with the space surrounding the mining machine based on the remote sensor data obtained from the at least one remote sensor.

[0025] By generating perception data, the operation efficiency, safety and productivity may be improved. This is enabled by enhancing the mining machine’s perception of it surrounding environment, which may enable the mining machine to operate more efficiently and avoiding collisions and / or accidents.

[0026] According to a third aspect, a mining machine operating in an underground and / or delimited environment using methods and systems described herein for an underground environment is provided. The mining machine comprises a control unit according to embodiments herein. These described methods may be used for machines operating in a delimited environment as well.

[0027] According to a fourth aspect, a system configured to enhance a mining machine’s perception of its surrounding environment is provided.

[0028] The system comprises a mining machine operating in an underground and / or delimited environment, and at least one remote sensor configured to provide remote sensor data associated with the environment surrounding the mining machine.

[0029] The mining machine and / or a control unit is configured to obtain remote sensor data associated with the environment surrounding the mining machine from the at least one remote sensor. The mining machine and / or control unit is configured enhance the mining machine’s perception of its surrounding environment by generating perception data associated with the space surrounding the mining machine based on the remote sensor data obtained from the at least one remote sensor. According to a fifth aspect, a computer program comprising instructions, which when executed by a processing circuitry, causes the processing circuitry to perform actions according to any of the methods herein.

[0030] BRIEF DESCRIPTIONS OF DRAWINGS

[0031] Examples of embodiments herein are described in more detail with reference to attached drawings in which:

[0032] Fig. 1 shows an example of a system according to embodiments herein.

[0033] Fig. 2a shows an example of an underground environment according to embodiments herein.

[0034] Fig. 2b shows an example according to embodiments herein.

[0035] Fig. 3 shows a flowchart depicting embodiments of a method.

[0036] Figs. 4a-b shows examples according to embodiments herein.

[0037] Fig. 5 shows an example of a mining machine according to embodiments herein.

[0038] Figs. 6a-b shows schematic block diagrams illustrating embodiments of a control unit.

[0039] DETAILED DESCRIPTION

[0040] An object of embodiments herein is to provide mechanisms that increase the operational efficiency and productivity in a delimited environment, such as an underground environment, of a mining machine operating in a delimited, such as an underground environment.

[0041] Embodiments herein may bring the advantage of an increased operational safety of a mining machine in an underground environment. This may be achieved by, as mentioned above, making it possible to enhance the mining machine’s perception of its surrounding environment based on sensor data obtained from a remote sensor. Further, embodiments herein may bring the advantage of an improved operational efficiency in the underground environment. This by controlling an operation of the mining machine operating in the underground environment taken perception data generated based on the remote sensor data into account. Fig. 1 shows a schematic illustration of a system 105 comprising at least one remote sensor 122 and a mining machine 110. The mining machine 110 may comprise a control unit 111 and at least one local sensor 121. The machine 110 may e.g., be a mine truck, a drill rigg, a load haul dump (LHD) machine and / or a utility machine.

[0042] A sensor, such as the at least one local sensor 121 and the at least one remote sensor 122, may e.g., comprise any one or more out of: A camera, a radar, a lidar, and an ultrasound sensor. A local sensor may comprise a sensor comprised on the mining machine 110. A remote sensor may e.g., comprise a fixed remote sensor 122 and / or a mobile remote sensor. A fixed remote sensor 122 may e.g., comprise a remote sensor 122 that is stationary in the underground environment 100, such as mounted on a wall, roof and / or another stationary object. A mobile remote sensor 122 may e.g., comprise a remote sensor 122 that is non-stationary in the underground environment 100. E.g., the mobile remote sensor may be comprised on another machine or vehicle operating in the underground environment 100, a handheld device and / or a wearable device worn by an operator in the underground environment 100. A sensor according to examples of embodiments herein, may be connected to e.g., electronic circuits, processors, control units, such as the control unit 111, communication units and / or structural elements on the mining machine 110, any one or more of which be configured to control operations, interpret sensor data, process sensor data and or communications. The sensor may be used to control and / or support the control of different functions and / or operations in the mining machine 110. A sensor may e.g., provide input for performing operations, collect sensor data, and / or communicate with other sensors, remote and local, in order to obtain input from said other sensors and / or provide instructions or request to said other sensors.

[0043] Methods according to embodiments herein may be performed by the system 105, the control unit 111 and / or the mining machine 110. Alternatively, or additionally, methods according to embodiments herein may be performed, or partly performed, by Distributed Nodes, e.g., comprised in a cloud.

[0044] Fig. 2a shows a schematic illustration of an underground environment 100 according to embodiments herein. The mining machine 110 may operate in the underground environment 100. Figure 2a further shows a remote sensor 122 comprised in the underground environment 100. In this example only one remote sensor 122 is present, though any number of remote sensors 122 may be present. The remote sensor 122 may be arranged for collecting sensor data associated with the underground environment 100, e.g., the mining machine’s 110 surrounding environment.

[0045] In order to enhance the perception of the mining machine’s 110 surrounding environment, the mining machine 110, or the control unit 111 , may obtain remote sensor data from the remote sensor. The remote sensor data may be used, e.g., by the mining machine 110 or the control unit 111 , to enhance the perception of the mining machine’s 110 surrounding environment. This may be achieved by generating perception data based on the remote sensor data.

[0046] Fig 2b shows a schematic illustration of an example according to embodiments herein. In this example, the mining machine 110 comprises a local sensor 121. In other examples more than one local sensor 121 may be comprised on the mining machine 110. The underground environment comprises a remote sensor 122. As shown in Fig 2b, the field of view 121a of the local sensor 121 is blocked, e.g., by parts of the mining machine 110. Thus, there is a blocked part 121b of the mining machine’s 110 surrounding environment that is not covered by the local sensor data. This results in that the local sensor data obtained from the local sensor 121 does not completely covers the mining machine’s 110 surrounding environment. Further, as shown in Fig. 2b, the field of view 122a of the remote sensor 122 overlaps the part of the mining machine’s 110 surrounding environment that is blocked from the field of view of the local sensor 121 , such as the blocked part 121b. According to this example, the mining machine 110, or the control unit 110 may obtain both the remote sensor data from the remote sensor 122 and the local sensor data from the local sensor 121. The local sensor data and the remote sensor data may be combined, and the enhanced perception data may be generated based on the combined sensor data. This way, the perception of the mining machine’s 110 surrounding environment may be enhanced, since the remote sensor data and the local sensor data, when combined, will cover all parts of the underground environment 100 surrounding the mining machine 110.

[0047] Fig. 3 shows an example embodiment of a method for enhancing the mining machine’s 110 perception of its surrounding environment. The mining machine 110 operates in the underground environment 100. The method may be performed by the system 105, the mining machine 110 and / or the control unit 111 comprised in the mining machine 110. The method comprises the following actions, which may be taken in any suitable order. Optional actions are referred to as dashed boxes in Fig. 3. Action 301

[0048] In some embodiments, the local sensor data is obtained, or received, from the at least one local sensor 121. The local sensor data is associated with a limited part of the mining machine’s 110 surrounding environment. A local sensor as used herein may comprise a sensor comprised on the mining machine 110. The local sensor data may, e.g., comprise data representing the environment surrounding the mining machine 110 obtain, measured or collected by the at least one remote sensor. Thus, the local sensor data may enable the mining machine 110, or control unit 111, to enhance its perception of its surrounding environment, e.g., by detecting obstacles, providing input for navigation, analyze its surrounding environment and / or plan or control operations.

[0049] In some embodiments, the at least one local sensor 121 may comprise any one or more out of: A camera, a radar, a lidar, and an ultrasound sensor. These are some examples of sensors, though any other type of suitable may also be used.

[0050] In some embodiments, a part of the mining machine’s 110 surrounding environment may be blocked from a field of view of the at least one local sensor 121 , prohibiting the at least one local sensor 121 from collecting local sensor data associated with the blocked part of the mining machine’s 110 surrounding environment. The blocked part may e.g., be blocked a part of the mining machine 110. Alternatively, or additionally the blocked part may e.g., be blocked be another object or obstacle in the underground environment 100, such as a wall and / or roof of the underground environment 100, and / or another machine, vehicle or operator currently located in the mining machine’s 110 surrounding environment. When a part of the field of the at least one local sensor 121 is blocked, the sensor data collected, or measured, by the at least one local sensor 121 information of the mining machine’s 110 surrounding environment. This may reduce the perception of the mining machine’s surrounding environment.

[0051] Action 302

[0052] Remote sensor data associated with the environment surrounding the mining machine 110 is obtained, or received, from the at least one remote sensor 122. The remote sensor data may, e.g., comprise data representing the environment surrounding the mining machine 110 obtain, measured or collected by the at least one remote sensor. Thus, the remote sensor data may enable the mining machine 110, or control unit 111 , to enhance its perception of its surrounding environment, e.g., by detecting obstacles, providing input for navigation, analyze its surrounding environment and / or plan or control operations. A remote sensor as used herein, may e.g., comprise a fixed remote sensor 122 and / or a mobile remote sensor. A fixed remote sensor 122 may e.g., comprise a remote sensor 122 that is stationary in the underground environment 100, such as mounted on a wall, roof and / or another stationary object. A mobile remote sensor 122 may e.g., comprise a remote sensor 122 that is non-stationary in the underground environment 100. E.g., the mobile remote sensor may be comprised on another machine or vehicle operating in the underground environment 100, a handheld device and / or a wearable device worn by an operator in the underground environment 100.

[0053] The remote sensor data may e.g., be obtained by wirelessly receiving the remote sensor data from the at least on remote sensor 122. The remote sensor data may be wired and / or wirelessly received using any suitable wired and / or wireless communication technology. Some examples may comprise e.g., WiFi, Bluetooth, ZigBee and / or mobile communications such as 2G, 3G, 4G, 5G, or even 6G, mobile communications networks.

[0054] The remote sensor data may be obtained upon a request for obtaining the remote sensor data. E.g., the mining machine 110, or a control unit 115 comprised in the mining machine 110, may detect the presence of the at least one remote sensor 122 and send a request, to the at least one remote sensor 122, to receive the remote sensor data.

[0055] In some embodiments, the remote sensor data may comprise sensor data not comprised in the local sensor data collected by the at least one local sensor 121. This may e.g., be the case when the field of view of the at least one remote sensor 122 comprises parts of underground environment 100 the at least one local sensor 121 is unable to collect, or measure, sensor data from. This may enable the perception of the mining machine’s 110 surrounding environment to be enhanced, since more sensor data is available.

[0056] In some embodiments, the remote sensor data may at least partly overlap the blocked part. As mentioned above, part of the mining machine’s 110 surrounding environment may be blocked from the field of view of the at least one local sensor 121. By obtaining remote sensor data that at least partly overlaps the blocked area, the mining machine’s 110 perception of its surrounding environment may be enhanced.

[0057] In some embodiments, the at least one remote sensor 122 may comprise any one or more out of: A camera, a radar, a lidar, and an ultrasound sensor. These are some examples of sensors, though any other type of suitable may also be used.

[0058] Action 303

[0059] The mining machine’s 110 perception of its surrounding environment is enhanced by generating perception data associated with the environment surrounding the mining machine 110 based on the remote sensor data obtained from the at least one remote sensor. The perception data may comprise distances between the mining machine 110 and obstacles, such as walls, roof, other machines, vehicles, operators and / or objects in the underground environment. To ensure safe operations of the mining machine 110, distances that is below a threshold may trigger a warning, enabling controlling the mining machine 110 to avoid collisions.

[0060] In some embodiments, generating the perception data may further comprise combing the remote sensor data with local sensor data obtained from the at least one local sensor 121 comprised in the mining machine 110. The perception data may be generated based on the combined sensor data. By combining the local sensor data and the remote sensor data the perception data may be generated based on more information. This may enable an even more enhanced perception of the mining machine’s 110 surrounding environment, than if only the local sensor data or remote sensor data is used. This since the local sensor data and the remote sensor data may complement each other. By complementing each other, is meant that the remote sensor data may comprise sensor data not comprised in the local sensor data, and vice versa. In other words, the remote sensor data may comprise sensor data related to the mining machine’s 110 that has not been collected, or measured, by the local sensor 121. Correspondingly, the local sensor data may comprise sensor data related to the mining machine’s 110 that has not been collected, or measured, by the remote sensor 122. Thus, a combination of the local sensor data and the remote sensor data may comprise more sensor data associated with the mining machine’s 110 surrounding environment than the respective local sensor data and remote sensor data comprises separately, which may result in improved perception data enhancing the perception of the mining machine’s 110 surrounding environment.

[0061] In some embodiments, generating the perception data may comprise generating a model of the mining machine’s 110 surrounding environment based on the perception data. The model may comprise distances between the mining machine 110 and obstacles, such as walls, roof, other machines, vehicles, operators and / or objects in the underground environment. The distances may be explicitly comprised in the model, or at least derivable from the model. The model may e.g., be a two dimensional model or a three dimensional model. The model may also be referred to as a spatial representation of the mining machine’s surrounding environment.

[0062] Action 304

[0063] In some embodiments, an operation of the mining machine 110 is controlled taking the generated perception data into account. In some embodiments, when the model of the mining machine’s 110 surrounding environment has been generated, the operation of the mining machine 110 may be controlled taking the model into account.

[0064] Controlling the operation of the mining machine 110 may e.g., utilizing the perception data when performing operations. E.g., movement of the mining machine 110 may controlled based on the generated perception data. This may comprise utilizing the perception data to avoid collisions with other objects or obstacles present in the mining machine’s 110 surrounding environment. Alternatively, or additionally, controlling the operations of the mining machine 110 may comprise controlling the movement of one or more parts of the mining machine 110, such as e.g., a boom, a feed, a service platform, a bucket and / or a box. Another example of the controlling the operation of the mining machine 110 may comprise a combination of movement of the mining machine 110 and movement one or more parts of the mining machine 110. An example of this may comprise controlling operations of mining machine 110 loading rocks onto another machine, where the operations e.g., comprises filling a bucket with rock, moving the mining machine 110 into a position where the rock may be loaded to the other machine and dumping the rock at the correct position in the other machine. This while avoiding collisions and / or accidents that may incur injuries and / or damage to machines and operators.

[0065] Figs. 4a-b depict examples of signaling diagrams according to embodiments herein. The signaling diagrams also reflect actions corresponding to those disclosed in the detailed description of Fig. 3. Optional actions are referred to as dashed boxes and / or lines in Fig. 4a-b. As mentioned above the method described in relation to Fig. 3 may be performed by the system 105, the mining machine 110 and / or the control unit 111 , and the actions may be performed by the control unit 111 and / or the mining machine 110 comprised in the system 105 in any suitable combination.

[0066] Fig. 4a exemplifies embodiments enhancing the mining machine’s 110 perception of its surrounding environment. In this example, the system 105 comprises the remote sensor 122 and the mining machine 110, which may comprise a control unit 111. The example comprises the following actions, which may be taken in any suitable order.

[0067] S41a. The remote sensor 122 may collect sensor data associated with the environment surrounding the mining machine 110. S42a. The mining machine 110 and / or the control unit 111 may obtain remote sensor data, e.g., in accordance with Action 302 above. The remote sensor data may be obtained, or received, from the remote sensor 122.

[0068] S43a. The mining machine 110 and / or the control unit 111 may generate perception data associated with the environment surrounding the mining machine 110., e.g., in accordance with Action 303 above. The generated perception data may be based on the obtained remote sensor data.

[0069] S44a. The mining machine 110 and / or the control unit 111 may control an operation of the mining machine 110, e.g., in accordance with Action 304 above. The operation may be controlled taking the generated perception data into account.

[0070] Fig. 4b exemplifies embodiments enhancing the mining machine’s 110 perception of its surrounding environment. In this example, the system 105 comprises the remote sensor 122, the local sensor 121 and the mining machine 110, which may comprise a control unit 111. The example comprises the following actions, which may be taken in any suitable order.

[0071] S41b. The remote sensor 122 may collect sensor data associated with the environment surrounding the mining machine 110.

[0072] S42b. The local sensor 121 may collect sensor data associated with the environment surrounding the mining machine 110.

[0073] S43b. The mining machine 110 and / or the control unit 111 may obtain remote sensor data, e.g., in accordance with Action 302 above. The remote sensor data may be obtained, or received, from the remote sensor 122.

[0074] S44b. The mining machine 110 and / or the control unit 111 may obtain local sensor data, e.g., in accordance with Action 301 above. The local sensor data may be obtained, or received, from the local sensor 121.

[0075] S45b. The mining machine 110 and / or the control unit 111 may generate perception data associated with the environment surrounding the mining machine 110., e.g., in accordance with Action 303 above. The generated perception data may be based on the obtained remote sensor data and the obtained local sensor data.

[0076] S46b. The mining machine 110 and / or the control unit 111 may control an operation of the mining machine 110, e.g., in accordance with Action 304 above. The operation may be controlled taking the generated perception data into account.

[0077] Fig. 5 depicts a schematic view of the mining machine 110 according to example of embodiments of the present invention. In Fig. 5, the mining and / or construction machine 20 is depicted as a LHD machine. This should not be seen as limiting to the scope, but is merely shown as an example of a mining machine 110. The mining and / or construction machine 110 may be any other type of mining machine 110. The mining and / or construction machine 110 may be configured to enhance the perception of its surrounding environment according to embodiments herein. Fig. 5 further shows the optional at least one local sensor 121 comprised on the mining machine 110. The at least one local sensor may be configured to provide local sensor data associated to the surrounding environment of the mining machine 110. Fig. 5 further shows the optional control unit 111 comprised in the mining machine 110. The control unit 111 may be configured to enhance the mining machine’s 110 perception of it surrounding environment according to embodiments herein.

[0078] Fig. 6a disclose an example configuration of the control unit 111 configured to enhance a mining machine’s 110 perception of its surrounding environment. The control unit 111 comprises a memory 660 operable to store instructions and processing circuitry 650 operable to execute the instructions. The control unit 111 may be located in the mining machine 110.

[0079] The control unit 111 may comprise an input and output interface 600 configured to communicate with, e.g., the at least one local sensor 121, the at least one remote sensor 122, another control unit and / or another mining machine or vehicle.

[0080] Fig. 6b also discloses an example configuration of processing circuitry for a control unit, e.g., the processing circuitry 650 disclosed in Fig. 6a. The processing circuitry may comprise an obtaining unit 610, a generating unit 620 and a controlling unit 630 configured to perform the methods above.

[0081] The embodiments herein may be implemented through the processing circuitry 650 in the control unit 111 depicted in Figure 6a, together with respective computer program code for performing the functions and actions of the embodiments herein. The processing circuitry 650 may comprise one or more processors and one or more memory units. The memory units may be the memory 660. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code for performing the embodiments herein when being loaded into the control unit 111. One such carrier may be in the form of a CD ROM disc, a USB flash drive, and / or an Over-the-Air (OTA) carrier. It is however feasible with other data carriers such as a memory stick. The computer program code may furthermore be provided as pure program code on a server and downloaded to the control unit 111. The memory 660 of the control unit 111 may further comprise one or more memory units. The memory 660 is configured to store instructions executable by the processing circuitry 650. The memory 660 is arranged to be used to store e.g. information, messages, indications, configurations, operating modes, measurements, sensor data, perception data, locations, positions, operational rules and applications to perform the methods herein when being executed in executed in the control unit 111.

[0082] The control unit 111 and / or the processing circuitry 650 is configured to enhance a mining machine’s (110) perception of its surrounding environment.

[0083] The control unit 111 and / or the processing circuitry 650 is configured to obtain, from at least one remote sensor 122, remote sensor data adapted to be associated with the environment surrounding the mining machine 110.

[0084] The control unit 111 and / or the processing circuitry 650 is configured to enhance the mining machine’s 110 perception of its surrounding environment by generating perception data adapted to be associated with the space surrounding the mining machine 110 based on the remote sensor data obtained from the at least one remote sensor 122.

[0085] In some embodiments, a computer program 670 comprises instructions, which when executed by the processing circuitry 650, e.g., of the respective at least one processor of the processing circuitry 650, cause the processing circuitry 650 of the control unit 111 to perform the actions above.

[0086] In some embodiments, a respective carrier 680 comprises the respective computer program 670, wherein the carrier 680 is one of an electronic signal, an optical signal, an electromagnetic signal, a magnetic signal, an electric signal, a radio signal, a microwave signal, or a computer-readable storage medium.

[0087] Those skilled in the art will appreciate that the units in the control unit 111 described above may refer to a combination of analogue and digital circuits, and / or one or more processors configured with software and / or firmware, e.g., stored in the control unit 111 , that when executed by the respective one or more processors such as the processors described above. One or more of these processors, as well as the other digital hardware, may be included in a single Application-Specific Integrated Circuitry (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a- chip (SoC). It will be appreciated that the foregoing description and the accompanying drawings represent non-limiting examples of the methods and apparatus taught herein. As such, the apparatus and techniques taught herein are not limited by the foregoing description and accompanying drawings. Instead, the embodiments herein are limited only by the following claims and their legal equivalents.

Claims

CLAIMS1. A method for enhancing a mining machine’s (110) perception of its surrounding environment, the mining machine (110) operating in an underground environment (100), the method comprising: obtaining (302), from at least one remote sensor (122), remote sensor data associated with the environment surrounding the mining machine (110), and enhancing the mining machine’s (110) perception of its surrounding environment by generating (303) perception data associated with the environment surrounding the mining machine (110) based on the remote sensor data obtained from the at least one remote sensor (122).

2. The method according to claim 1, wherein generating (303) the perception data further comprises combing the remote sensor data with local sensor data obtained from at least one local sensor (112) comprised in the mining machine (110), wherein the perception data is generated based on the combined sensor data.

3. The method according to any of claims 1-2, wherein the remote sensor data comprises sensor data not comprised in the local sensor data collected by the at least one local sensor (112).

4. The method according to any of claims 1-3, wherein a part of the mining machine’s (110) surrounding environment is blocked from a field of view of the at least one local sensor (112), prohibiting the at least one local sensor (112) from collecting local sensor data associated with the blocked part of the surrounding environment.

5. The method according to claim 4, wherein the remote sensor data at least partly overlaps the blocked area.

6. The method according to any of claims 1-5, wherein generating (303) the perception data comprises generating a model of the mining machine’s (110) surrounding environment based on the perception data.

7. The method according to any of claims 1-6, wherein the method further comprises:obtaining (301) the local sensor data from the at least one local sensor (112), wherein the local sensor data is associated with a limited part of the mining machine’s(110) surrounding environment.

8. The method according to any of claims 1-7, wherein the at least one local sensor(111) and the at least one remote sensor (112) comprises any one or more out of:- a camera,- a radar,- a lidar, and- an ultrasound sensor.

9. The method according to any of claims 1-8, wherein the method further comprises: controlling (304) an operation of the mining machine (110) taking the generated perception data into account.

10. A computer program 670 comprising instructions, which when executed by a processing circuitry 650, causes the processing circuitry 650 to perform actions according to any of the methods 1-9.

11. A control unit (111) configured to enhance a mining machine’s (110) perception of its surrounding environment, the mining machine (110) further being configured to: obtain, from at least one remote sensor (122), remote sensor data adapted to be associated with the environment surrounding the mining machine (110), and enhance the mining machine’s (110) perception of its surrounding environment by generating perception data adapted to be associated with the space surrounding the mining machine (110) based on the remote sensor data obtained from the at least one remote sensor (122).

12. The control unit (115) according to claim 11, wherein the control unit (110) is further configured to perform the method according to any of claims 2-9.

13. A mining machine (110) operating in an underground environment (100), wherein the mining machine comprises the control unit (111) according to claim 11.

14. A system (105) configured to enhance a mining machine’s (110) perception of its surrounding environment, the system comprising: a mining machine (110) according to claim 12 operating in an underground environment (100), at least one remote sensor (122) configured to provide remote sensor data associated with the environment surrounding the mining machine (110), wherein the mining machine (110) and / or a control unit (111) is configured to obtain remote sensor data associated with the environment surrounding the mining machine (110) from the at least one remote sensor (122), and enhance the mining machine’s (110) perception of its surrounding environment by generating perception data associated with the space surrounding the mining machine (110) based on the remote sensor data obtained from the at least one remote sensor (122).

Images