Environment-related data management for mobile mining vehicles

Abstract

Apparatuses, methods, computer programs and systems for environment-related data management of mobile mining vehicles are disclosed. An environment-related data management device for a mobile mining vehicle obtains real-time position information of a structural body member of the mobile mining vehicle. The device obtains spatial information of structural body components of the mobile mining vehicle. Based on the obtained spatial information, the device associates at least one bounding box with at least one structural body member. The apparatus receives point cloud data related to an environment of a mobile mining vehicle from a sensor configured to scan the environment of the mobile mining vehicle. In response to determining that data points in the received point cloud data overlap with the associated at least one bounding box, the apparatus generates an overlap indication indicating that at least the overlapping data points should be excluded.

Description

Technical Field

[0001] The present disclosure relates to the field of industrial systems, and more particularly, the present disclosure relates to environment-related data management for mobile mining vehicles. Background Art

[0002] When installing sensors such as light detection and ranging (lidar) sensors on large mobile mining vehicles, the sensors inevitably see the vehicle itself in addition to the environment. At least in some cases, this can cause problems for, for example, mapping algorithms and / or obstacle detection algorithms (which are designed to see and respond only to the environment).

[0003] If, for example, the boom, bucket and / or body of a mobile mining vehicle cannot be masked out (and removed from the input data to the mapping algorithms, obstacle detection algorithms, etc.), they will be added to the map and displayed as obstacles to the mobile mining vehicle. Summary of the invention

[0004] The scope of protection sought by various exemplary embodiments of the present disclosure is defined by the independent claims. Exemplary embodiments and features described in this specification that do not fall within the scope of the independent claims (if any) should be interpreted as examples that help understand the various exemplary embodiments of the present disclosure.

[0005] An exemplary embodiment of an environment-related data management device for a mobile mining vehicle includes at least one processor and at least one memory, the at least one memory including computer program code. The mobile mining vehicle includes a plurality of structural body components, at least one of which is movable relative to at least another structural body component. The at least one memory and the computer program code are configured to cause the device to at least obtain real-time position information of the structural body component of the mobile mining vehicle through the at least one processor, the real-time position information indicating the real-time position of the structural body component. The at least one memory and the computer program code are also configured to cause the device to at least obtain spatial information of the structural body component of the mobile mining vehicle through the at least one processor, the spatial information indicating the space required by the structural body component. The at least one memory and the computer program code are also configured to cause the device to at least associate at least one bounding box with at least one structural body component of the plurality of structural body components based on at least the obtained spatial information of the structural body component through the at least one processor. The at least one memory and the computer program code are also configured to cause the device to at least receive point cloud data related to the environment of the mobile mining vehicle from at least one sensor configured to scan the environment of the mobile mining vehicle through the at least one processor. In response to determining that one or more data points included in the received point cloud data overlap with the associated at least one bounding box, the at least one memory and the computer program code are also configured to cause the apparatus, via the at least one processor, to at least generate an overlap indication indicating that at least the one or more overlapping data points in the received point cloud data should be excluded.

[0006] An exemplary embodiment of an environment-related data management device for a mobile mining vehicle includes a device for performing the following operations: obtaining real-time position information of a structural body component of the mobile mining vehicle, the mobile mining vehicle including a plurality of structural body components, at least one of which is movable relative to at least another structural body component, the real-time position information indicating the real-time position of the structural body component. The device is also configured to perform: obtaining spatial information of the structural body component of the mobile mining vehicle, the spatial information indicating a space required by the structural body component. The device is also configured to perform: associating at least one bounding box with at least one structural body component of the plurality of structural body components based on the obtained spatial information of the structural body component. The device is also configured to perform: causing the environment-related data management device to receive point cloud data related to the environment of the mobile mining vehicle from at least one sensor configured to scan the environment of the mobile mining vehicle. In response to determining that one or more data points included in the received point cloud data overlap with the associated at least one bounding box, the device is also configured to perform: generating an overlap indication, the overlap indication indicating that at least the one or more overlapping data points in the received point cloud data should be excluded.

[0007] An exemplary embodiment of a method includes: obtaining, by an environment-related data management device for a mobile mining vehicle, real-time position information of a structural body member of the mobile mining vehicle, the real-time position information indicating a real-time position of the structural body member, the mobile mining vehicle including a plurality of structural body members, at least one of which is movable relative to at least another structural body member; obtaining, by the device, spatial information of the structural body member of the mobile mining vehicle, the spatial information indicating a space required by the structural body member; associating, by the device, at least one bounding box with at least one structural body member of the plurality of structural body members based on the obtained spatial information of the structural body member; receiving, by the device, point cloud data related to an environment of the mobile mining vehicle from at least one sensor configured to scan an environment of the mobile mining vehicle; and generating, by the device, an overlap indication in response to the device determining that one or more data points included in the received point cloud data overlap with the associated at least one bounding box, the overlap indication indicating that at least the one or more data points in the received point cloud data that overlap should be excluded.

[0008] An exemplary embodiment of a computer program includes instructions for causing an environment-related data management device for a mobile mining vehicle to perform at least the following operations, the mobile mining vehicle including a plurality of structural body components, at least one of which is movable relative to at least another structural body component: obtaining real-time position information of a structural body component of the mobile mining vehicle, the real-time position information indicating a real-time position of the structural body component; obtaining spatial information of the structural body component of the mobile mining vehicle, the spatial information indicating a space required by the structural body component; associating at least one bounding box with at least one of the plurality of structural body components based on the obtained spatial information of the structural body component; receiving point cloud data related to the environment of the mobile mining vehicle from at least one sensor configured to scan the environment of the mobile mining vehicle; and generating an overlap indication in response to determining that one or more data points included in the received point cloud data overlap with the associated at least one bounding box, the overlap indication indicating that at least the one or more data points in the received point cloud data that overlap should be excluded.

[0009] In an exemplary embodiment, alternatively or in addition to the above exemplary embodiments, the at least one memory and the computer program code are further configured to cause the apparatus, through the at least one processor, to exclude at least the overlapping one or more data points from the received point cloud data.

[0010] In an exemplary embodiment, alternatively or in addition to the above-mentioned exemplary embodiments, the at least one memory and the computer program code are also configured to cause the device to provide the received point cloud data and the generated overlap indication through the at least one processor, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for obstacle detection, so that at least the overlapping one or more data points should be excluded from the obstacle detection.

[0011] In an exemplary embodiment, alternatively or in addition to the above-mentioned exemplary embodiments, the at least one memory and the computer program code are also configured to cause the device to provide the received point cloud data and the generated overlap indication through the at least one processor, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for collision detection, so that at least the overlapping one or more data points should be excluded from the collision detection.

[0012] In an exemplary embodiment, alternatively or in addition to the exemplary embodiments described above, the at least one memory and the computer program code are further configured to cause the apparatus, via the at least one processor, to provide the received point cloud data and the generated indication of overlap, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for use in generating a three-dimensional (3D) model of the environment of the mobile mining vehicle, such that at least the overlapping one or more data points should be excluded from the generation of the 3D model of the environment.

[0013] In an exemplary embodiment, alternatively or in addition to the exemplary embodiments described above, the at least one memory and the computer program code are further configured to cause the apparatus, via the at least one processor, to provide the received point cloud data and the generated overlap indication, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for use in modifying a 3D model of a mine associated with the mobile mining vehicle, such that at least the overlapping one or more data points should be excluded from the modification of the 3D model of the mine.

[0014] In an exemplary embodiment, alternatively or in addition to the above-mentioned exemplary embodiments, the at least one memory and the computer program code are also configured to cause the device to provide, through the at least one processor, the received point cloud data and the generated overlap indication, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for generating an alert about a detected obstacle in driver assistance, so that at least the overlapping one or more data points should be excluded from the generation of the alert.

[0015] In an exemplary embodiment, alternatively or in addition to the above-mentioned exemplary embodiments, the at least one memory and the computer program code are also configured to cause the device to provide the received point cloud data and the generated overlap indication through the at least one processor, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for real-time 3D positioning, so that at least the overlapping one or more data points should be excluded from the real-time 3D positioning.

[0016] In an exemplary embodiment, alternatively or in addition to the exemplary embodiments described above, the real-time position information, the spatial information and the point cloud data are information and data according to a coordinate system associated with the mobile mining vehicle.

[0017] In an exemplary embodiment, alternatively or in addition to the above-described exemplary embodiments, the at least one memory and the computer program code are further configured to cause the apparatus, via the at least one processor, to generate an overlapping area based on the overlapping one or more data points, wherein the overlap indication indicates that the overlapping area should be excluded.

[0018] In an exemplary embodiment, the overlapping region comprises a bounding box type region.

[0019] An exemplary embodiment of a mobile mining vehicle, the mobile mining vehicle comprising a plurality of structural body components, at least one of which is movable relative to at least another structural body component. The mobile mining vehicle further comprises at least one sensor configured to scan an environment of the mobile mining vehicle. The mobile mining vehicle further comprises a control system configured to provide: real-time position information of the structural body components, the real-time position information indicating the real-time position of the structural body components; and spatial information of the structural body components, the spatial information indicating the space required by the structural body components. The mobile mining vehicle further comprises an environment-related data management device according to any of the above exemplary embodiments.

[0020] In an exemplary embodiment, alternatively or in addition to the exemplary embodiments described above, the mobile mining vehicle comprises an autonomous mobile mining vehicle.

[0021] Many of these features will be more readily apparent as they become better understood by reference to the following detailed description considered in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the embodiments and constitute a part of this specification. The accompanying drawings illustrate the embodiments and together with the description help explain the principles of the embodiments. In the drawings:

[0023] Figure 1 An example embodiment of the subject matter described herein is shown, illustrating an example mobile mining vehicle in which various embodiments of the present disclosure may be implemented;

[0024] Figure 2 An example embodiment of the subject matter described herein is shown, showing an example environment-related data management apparatus for a mobile mining vehicle in which various embodiments of the present disclosure may be implemented; and

[0025] Figure 3 An example embodiment of the subject matter described herein is shown, illustrating a method.

[0026] In the drawings, like reference numerals refer to like parts. DETAILED DESCRIPTION

[0027] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. The detailed description provided below in conjunction with the accompanying drawings is intended as a description of this example and is not intended to represent the only form in which this example may be constructed or used. This description sets forth the functions of the example and the sequence of steps for constructing and operating the example. However, the same or equivalent functions and sequences may be implemented by different examples.

[0028] Figure 1 An example mobile mining vehicle 100 is shown in which various embodiments of the present disclosure may be implemented. For example, the mobile mining vehicle 100 may include an autonomous mobile mining vehicle. In addition, the mobile mining vehicle 100 may include, for example, a rock drill, a development drill, a tunnel boring machine, a surface boring machine, an anchoring or reinforcement vehicle, a rock removal machine, a long hole drill, an explosive charging machine, a loader, a transport vehicle, a loader or tractor, a tunnel arch or net setting vehicle, a concrete sprayer, a crusher, a surveying vehicle, or a passenger transport vehicle.

[0029] The mobile mining vehicle 100 includes a plurality of structural body members 111-114 such that at least one structural body member is movable (e.g., at least, occasionally) relative to at least another structural body member. For example, the structural body members 111-114 may include at least one boom. The boom may be provided with a mining work tool. The mining work tool may include one or more devices for underground or open-pit mining. The mining work tool may include, for example, a rock drilling unit, an anchor head, or a bucket.

[0030] The boom may be configured to move between different working positions. The working positions may include, for example, a transport position to which the boom is moved to move a mining vehicle from a first location to a second location. As another example, the working positions may include an operating position for operation of a rock drilling unit disposed at a distal end of the boom. As yet another example, the working positions may include an operating position for bucket filling.

[0031] In an example, the mobile mining vehicle 100 may include articulated steering such that a first carrier and a second carrier of the mobile mining vehicle 100 are connected by an articulated joint. Thus, when the mobile mining vehicle 100 turns, the first carrier and the second carrier may move relative to each other.

[0032] Turning the mobile mining vehicle 100 may include, for example, changing the position of the mobile mining vehicle 100 from a first position to a second position. The change in position may include, for example, movement of the mobile mining vehicle 100 in a particular direction and / or a change in the orientation of the mobile mining vehicle 100.

[0033] The first carrier may include a front carrier, and the second carrier may include a rear carrier. The front carrier may include the first carrier in the forward travel direction. The rear carrier may include a carrier following the front carrier in the forward travel direction.

[0034] In this example, the structural body members 111-114 may include these first and second carriers.At least some of the structural body members 111-114 may be protruding and / or overhanging, such as the at least one boom.

[0035] The mobile mining vehicle 100 also includes at least one sensor 120 configured to scan the environment of the mobile mining vehicle 100. The at least one sensor 120 may include, for example, a light detection and ranging (lidar) sensor. A lidar sensor is configured to determine the distance by aiming a laser at a target and measuring the time for the reflected light to return to a receiver. As another example, the at least one sensor 120 may include a radio detection and ranging (radar) sensor. A radar sensor is configured to transmit electromagnetic energy to an object and observe the echoes returning from them. As yet another example, the at least one sensor 120 may include a time-of-flight (ToF) camera configured to determine the distance between the camera and the subject by measuring the round-trip time of an artificial light signal provided by a laser or a light emitting diode (LED). At least in some embodiments, the at least one sensor 120 may be attached to the body of the mobile mining vehicle 100. For example, the at least one sensor 120 may be attached to the left front corner and / or the right front corner of the mobile mining vehicle 100 relative to the direction of travel of the mobile mining vehicle 100.

[0036] The scan results may be applied to detect the position and / or orientation of the mobile mining vehicle 100 and one or more other elements thereof, such as the position and / or orientation of the at least one sensor 120 or the plurality of structural body members 111 - 114 .

[0037] The control system 130 may be configured to compare the scanned tunnel contour data with reference contour data stored in a site model (e.g., a 3D model of a mine). The control system 130 may be configured to position the mobile mining vehicle 100 based on finding a match between the surface contour indicated by the scanned data and the surface contour indicated by the site model. The mobile mining vehicle 100 also includes the control system 130 configured to provide real-time position information of the structural body components 111-114, the real-time position information indicating the real-time positions of the structural body components 111-114.

[0038] For example, the position information may include the position and / or orientation of the structural body components 111-114 relative to a reference coordinate system. The reference coordinate system may include, for example, a global coordinate system, a coordinate system of a worksite in which the mobile mining vehicle operates, or other coordinate systems associated with predetermined reference points. Alternatively, the reference coordinate system may include, for example, a coordinate system of the main body / carrier of the mobile mining vehicle 100, with other components referenced to this coordinate system.

[0039] The control system 130 is also configured to provide spatial information of the structural body members 111-114, the spatial information indicating the space required by the structural body members 111-114. For example, the control system 130 may include a boom control system configured to calculate forward kinematics of the boom in real time and track the poses (e.g., six degrees of freedom (6DOF) poses) of the various boom sections. Forward kinematics may enable the control system 130 to determine the position and orientation of the structural body members 111-114. For example, the control system 130 may determine the position and orientation of the at least one boom based on joint values ​​associated with the at least one boom. Based on the forward kinematics, a 6DOF transformation of each moving boom section may be calculated. Additionally or alternatively, the control system 130 may include a positioning system configured to track the 6DOF poses of the first and second carriages of the mobile mining vehicle 100 in real time.

[0040] The at least one sensor 120 may be used, for example, to monitor the environment of the mobile mining vehicle 100 in real time, so that the control system 130 is aware of obstacles detected in the environment of the mobile mining vehicle 100. However, since the at least one sensor 120 may be attached to the mobile mining vehicle 100 itself, it is possible that, for example, sometimes some of the structural body components 111-114 may enter the field of view of the at least one sensor 120, causing these structural body components 111-114 to be incorrectly interpreted as obstacles, which may cause a false alarm to the control system 130. On the other hand, it is also possible that the mobile mining vehicle cannot see an obstacle, such as a pedestrian, if the obstacle is located between or below the boom or is obscured by the boom (or removed by a coarse filtering method when trying to eliminate the influence of the mobile mining vehicle components). As a result, the obstacle may not be detected at all, and the control system 130 may not receive an alarm, which may lead to an accident.

[0041] As will be discussed in detail below, the above-mentioned problems are solved by the disclosed environment-related data management device 200 and related methods and computer programs, thereby avoiding computationally complex and burdensome post-processing of the sensor data from the at least one sensor 120.

[0042] The mobile mining vehicle 100 also includes an environment-related data management device 200. The environment-related data management device 200 will be described below in conjunction with Figure 2 Detailed description.

[0043] In the following, various exemplary embodiments will be discussed. At least some of these exemplary embodiments may allow for environment-related data management for a mobile mining vehicle 100. Some of the described features are optional features that may provide further advantages.

[0044] Figure 2 is a block diagram of an environment-related data management apparatus 200 for a mobile mining vehicle 100 according to an exemplary embodiment.

[0045] The environment-related data management device 200 for the mobile mining vehicle 100 includes at least one processor 202 and at least one memory 204, the at least one memory 204 including computer program code. The device 200 may also include Figure 2 In addition, at least in some embodiments, the device 200 can be included or integrated in the control system 130.

[0046] Although the device 200 is depicted as including only one processor 202, the device 200 may include more processors. In one embodiment, the memory 204 is capable of storing instructions, such as an operating system and / or various applications. In addition, the memory 204 may include a storage device for storing information and data used, for example, in the disclosed embodiments.

[0047] In addition, the processor 202 is capable of executing the stored instructions. In one embodiment, the processor 202 may be embodied as a multi-core processor, a single-core processor, or a combination of one or more multi-core processors and one or more single-core processors. For example, the processor 202 may be embodied as one or more of various processing devices, such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), a processing circuit with or without an accompanying DSP, or various other processing devices including integrated circuits, such as application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), microcontroller units (MCUs), hardware accelerators, special purpose computer chips, neural network chips, artificial intelligence (AI) accelerators, etc. In one embodiment, the processor 202 may be configured to perform hard-coded functions. In one embodiment, the processor 202 is embodied as an executor of software instructions, wherein the instructions may specifically configure the processor 202 to perform the algorithms and / or operations described herein when the instructions are executed.

[0048] The memory 204 may be embodied as one or more volatile memory devices, one or more non-volatile memory devices, and / or a combination of one or more volatile memory devices and non-volatile memory devices. For example, the memory 204 may be embodied as a semiconductor memory (such as a mask ROM, a PROM (programmable ROM), an EPROM (erasable PROM), a flash ROM, a RAM (random access memory), etc.).

[0049] The at least one memory 204 and the computer program code are configured to cause the apparatus 200 to at least obtain real-time position information of the structural body components 111-114 of the mobile mining vehicle 100 through the at least one processor 202. As described above, the real-time position information indicates the real-time position of the structural body components 111-114, and the mobile mining vehicle 100 has a plurality of structural body components 111-114, so that at least one structural body component can move relative to at least another structural body component. For example, the real-time position information can be obtained by receiving the real-time position information from the control system 130 or accessing the real-time position information in other ways. In other words, obtaining herein may include accessing or receiving. In addition, obtaining the real-time position information may include calculating the forward kinematic chain of the structural body components 111-114 based on the joint position measurements.

[0050] The at least one memory 204 and the computer program code are further configured to cause the apparatus 200, via the at least one processor 202, to at least obtain spatial information of the structural body components 111-114 of the mobile mining vehicle 100. As described above, the spatial information indicates the space required by the structural body components 111-114. For example, the spatial information may include size information, such as bounding box type information. For example, the spatial information may be obtained by receiving the spatial information from the control system 130 or accessing the spatial information in other ways.

[0051] The at least one memory 204 and the computer program code are further configured to cause the apparatus 200 , through the at least one processor 202 , to associate at least one bounding box with at least one structural body component of the plurality of structural body components 111 - 114 based at least on the obtained spatial information of the structural body components 111 - 114 .

[0052] For example, a bounding box can define an area or space by coordinates. At least in some embodiments, when the positions (e.g., coordinates) of the structural body components 111-114 and the spaces required by them are known, the at least one memory 204 and the computer program code can also be configured to cause the apparatus 200 to generate a bounding box with known coordinates through the at least one processor 202.

[0053] The at least one memory 204 and the computer program code are further configured to cause the apparatus 200, through the at least one processor 202, to at least receive point cloud data related to the environment of the mobile mining vehicle 100 from at least one sensor 120 configured to scan the environment of the mobile mining vehicle 100. For example, the point cloud data may be real-time data. Here, the term "point cloud data" includes mesh type data.

[0054] The point cloud data may include a plurality of data points representing, for example, the distance between the mobile mining vehicle 100 and objects in the environment of the mobile mining vehicle 100 at a particular moment in time. Each data point included in the point cloud may be represented, for example, by X and Y coordinates, or X, Y, and Z coordinates relative to a particular coordinate system.

[0055] The at least one memory 204 and the computer program code may also be configured to cause the apparatus 200, via the at least one processor 202, to analyze the received point cloud data. The analyzing may include determining coordinate information associated with at least one data point.

[0056] In response to determining that one or more data points included in the received point cloud data overlap with the associated at least one bounding box, the at least one memory 204 and the computer program code are further configured to cause the apparatus 200 to at least generate an overlap indication through the at least one processor 202, the overlap indication indicating that at least the one or more data points in the received point cloud data that overlap should be excluded. Excluding overlapping data points may include, for example, ignoring or removing overlapping data points, such as by masking.

[0057] In other words, overlapping data points are data points whose coordinates in the received point cloud fall within the associated at least one bounding box.

[0058] For example, the real-time position information, spatial information, point cloud data, and the position of the at least one sensor 120 (also known) may be information and data according to a coordinate system (e.g., the same coordinate system) associated with the mobile mining vehicle 100. In at least some embodiments, the coordinate system may include a Cartesian coordinate system, where coordinates are expressed in, for example, distance units, such as meters. In other embodiments, the coordinate system may include a non-Cartesian coordinate system (e.g., a polar coordinate system). In at least some embodiments, the origin of the coordinate system may be defined relative to the mobile mining vehicle 100. The origin of the coordinate system may include a predetermined point of the mobile mining vehicle 100. For example, the origin of the coordinate system may be located at a center point of a front axle of the mobile mining vehicle 100. Alternatively, the origin of the coordinate system may be located at, for example, the location of one of the at least one sensor 120.

[0059] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the apparatus 200 to exclude at least the one or more overlapping data points from the received point cloud data via the at least one processor 202 .

[0060] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the apparatus 200 to generate an overlap region based on the one or more data points that overlap, via the at least one processor 202. In these embodiments, the overlap indication may indicate that the overlap region should be excluded. For example, the overlap region may include a bounding box type region. In some embodiments, the overlap region may also include a tolerance distance.

[0061] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the apparatus 200, via the at least one processor 202, to control at least one function of the mobile mining vehicle 100 based on the point cloud data from which the one or more data points from which at least overlap has been excluded.

[0062] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the device 200 to provide, through the at least one processor 202, the received point cloud data and the generated overlap indication, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for use in obstacle detection, so that at least the overlapping one or more data points should be excluded from the obstacle detection.

[0063] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the device 200 to provide, through the at least one processor 202, the received point cloud data and the generated overlap indication, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for collision detection, so that at least the overlapping one or more data points should be excluded from the collision detection.

[0064] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the apparatus 200, via the at least one processor 202, to provide the received point cloud data and the generated indication of overlap, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for use in generating a three-dimensional (3D) model of the environment of the mobile mining vehicle 100, such that at least the overlapping one or more data points should be excluded from the generation of the 3D model of the environment.

[0065] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the apparatus 200, via the at least one processor 202, to provide the received point cloud data and the generated indication of overlap, or to provide the received point cloud data from which at least the one or more data points that overlap have been excluded, for use in modifying a 3D model of a mine associated with the mobile mining vehicle 100, such that at least the one or more data points that overlap should be excluded from the modification of the 3D model of the mine.

[0066] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the device 200 to provide, through the at least one processor 202, the received point cloud data and the generated overlap indication, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for use in generating an alert about a detected obstacle in driver assistance, such that at least the overlapping one or more data points should be excluded from the generation of the alert.

[0067] In at least some embodiments, the at least one memory 204 and the computer program code may also be configured to cause the device 200 to provide, through the at least one processor 202, the received point cloud data and the generated overlap indication, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for real-time 3D positioning, so that at least the overlapping one or more data points should be excluded from the real-time 3D positioning.

[0068] When the mobile mining vehicle 100 includes more than one sensor 120, from which point cloud data related to the environment of the mobile mining vehicle 100 is received, the above-mentioned masking process can be performed on the point cloud data from one sensor at a time. Alternatively, the point cloud data from multiple sensors can be combined first, and then the masking process can be performed on the combined data simultaneously.

[0069] Other features of the apparatus 200 are derived directly from the functions and parameters of the mobile mining vehicle 100 and are therefore not repeated here.

[0070] Figure 3 An exemplary flow chart of a method 300 according to an exemplary embodiment is shown. Figure 3 In the example of , method 300 comprises a computer-implemented method.

[0071] At operation 301, the environment-related data management device 200 for the mobile mining vehicle 100 obtains real-time position information of the structural body components 111-114 of the mobile mining vehicle 100. As described above, the real-time position information indicates the real-time positions of the structural body components 111-114, wherein the mobile mining vehicle 100 includes a plurality of structural body components 111-114 such that at least one structural body component can move relative to at least another structural body component.

[0072] At operation 302, the environment-related data management device 200 obtains spatial information of the structural body components 111-114 of the mobile mining vehicle 100. As described above, the spatial information indicates the space required by the structural body components 111-114.

[0073] At operation 303 , the environment-related data management device 200 associates at least one bounding box with at least one structural vehicle body component among the plurality of structural vehicle body components 111 - 114 based on the obtained spatial information of the structural vehicle body components 111 - 114 .

[0074] At operation 304 , the environment-related data management device 200 receives point cloud data related to the environment of the mobile mining vehicle 100 from at least one sensor 120 configured to scan the environment of the mobile mining vehicle 100 .

[0075] In response to determining at operation 305 that one or more data points included in the received point cloud data overlap with the associated at least one bounding box, at operation 307, the environment-related data management device 200 generates an overlap indication indicating that at least the one or more overlapping data points in the received point cloud data should be excluded. Otherwise, the method 300 exits at operation 306.

[0076] At optional operation 308 , the environment-related data management device 200 may exclude at least the one or more overlapping data points from the received point cloud data.

[0077] Method 300 may be performed by Figure 2 The method 300 is performed by the apparatus 200 in the embodiment of the present invention. Operations 301-308 may be performed, for example, by the at least one processor 202 and the at least one memory 204. Other features of the method 300 are directly derived from the functions and parameters of the apparatus 200 and are therefore not repeated here. The method 300 may be performed by a computer program.

[0078] The apparatus 200 may include means for performing at least one method described herein. In one example, the apparatus may include at least one processor 202 and at least one memory 204, the at least one memory including program code, the program code causing the apparatus 200 to perform the method when executed by the at least one processor 202.

[0079] At least some of the disclosed exemplary embodiments may allow for environment-related data management for a mobile mining vehicle 100 .

[0080] At least some of the disclosed exemplary embodiments may allow for masking to adapt to the real-time location of the mobile mining vehicle 100 .

[0081] At least some of the disclosed exemplary embodiments may allow for ensuring that shielding does not remove objects other than components of the mobile mining vehicle 100 .

[0082] At least some of the disclosed exemplary embodiments may allow for visibility of obstacles between and below the booms.

[0083] At least some of the disclosed example embodiments may allow for avoiding contamination of components of the mobile mining vehicle 100 to a map generated based on point cloud data from the at least one sensor 120 .

[0084] At least some of the disclosed example embodiments may allow for reduced post-processing required for maps generated based on point cloud data from the at least one sensor 120 .

[0085] At least some of the disclosed example embodiments may allow for real-time calculation of which measurements in the raw data (in the point cloud data) overlap with the physical mobile mining vehicle 100 and what is the actual environment of the mobile mining vehicle 100 .

[0086] At least some of the disclosed exemplary embodiments may allow for the definition of bounding boxes that should be filtered directly from three-dimensional computer-aided design (CAD) files of the structural vehicle body members 111 - 114 .

[0087] At least some of the disclosed exemplary embodiments may allow for the separation of sensor 120 hits caused by the mobile mining vehicle 100 from hits caused by the environment of the mobile mining vehicle 100 to automatically clean up the raw data of the point cloud data.

[0088] At least some of the disclosed exemplary embodiments may allow for the proper detection of obstacles (eg, pipes) protruding from the roof of a mine.

[0089] At least some of the disclosed exemplary embodiments may allow a first carrier and a second carrier of a mobile mining vehicle 100 with articulated steering to be correctly ignored when the mobile mining vehicle 100 is turning on a curve. In conventional systems, the first carrier and / or the second carrier may be mistaken as obstacles when turning because the first carrier and / or the second carrier may suddenly enter the field of view of the sensor, depending on where the sensor is installed.

[0090] At least some of the disclosed exemplary embodiments may allow for reduction of delays and heavy and complex calculations caused by cleaning up maps in post-processing.

[0091] At least some of the disclosed exemplary embodiments may, for example, allow a mobile mining vehicle to see obstacles, such as pedestrians, even if the obstacles are located between or under the booms or are otherwise obscured by the booms.

[0092] The functions described herein may be performed at least in part by one or more computer program product components (e.g., software components). According to one embodiment, the environment-related data management device 200 may include a processor or processor circuit, such as, for example, a microcontroller, which is configured by program code when executed to perform the described operations and embodiments of the functions. Alternatively or in addition, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), program application specific integrated circuits (ASICs), program specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and graphics processing units (GPUs).

[0093] Any range or device value given herein may be expanded or altered without losing the effect sought. In addition, any embodiment may be combined with another embodiment unless expressly prohibited.

[0094] Although the subject matter has been described in language specific to structural features and / or actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Instead, the specific features and actions described above are disclosed as examples of implementing the claims, and other equivalent features and actions are intended to fall within the scope of the claims.

[0095] It should be understood that the benefits and advantages described above may relate to one embodiment, or may relate to several embodiments. These embodiments are not limited to those embodiments that solve any or all of the problems described, or embodiments that have any or all of the benefits and advantages described. It should also be understood that reference to an item "an" may refer to one or more of these items.

[0096] The steps of the methods described herein may be performed in any suitable order, or may be performed simultaneously where appropriate. In addition, individual step blocks may be deleted from any method without departing from the spirit and scope of the subject matter described herein. Aspects of any of the above embodiments may be combined with aspects of any of the other embodiments described to form further embodiments without losing the effects sought.

[0097] The term "comprising" is used herein to mean including the identified methods, step blocks or elements, but these step blocks or elements do not include an exclusive list and the method or apparatus may contain additional step blocks or elements.

[0098] It should be understood that the above description is given as an example only, and various modifications may be made by those skilled in the art. The above description, examples and data provide a complete description of the structure and use of the example embodiments. Although various embodiments have been described above with a certain degree of particularity or with reference to one or more individual embodiments, those skilled in the art may make many modifications to the disclosed embodiments without departing from the scope of this specification.

Claims

1. An environment-related data management device (200) for a mobile mining vehicle (100), the mobile mining vehicle (100) comprising a plurality of structural body components (111-114), at least one of the structural body components being movable relative to at least another structural body component, the device (200) comprising: at least one processor (202); and at least one memory (204), the at least one memory comprising computer program code; The at least one memory (204) and the computer program code are configured to cause the apparatus (200) to perform at least the following operations through the at least one processor (202): Obtaining real-time position information of the structural body components (111-114) of the mobile mining vehicle (100), the real-time position information indicating the real-time positions of the structural body components (111-114); Obtaining spatial information of the structural vehicle body components (111-114), the spatial information indicating a space required by the structural vehicle body components (111-114); Associating at least one bounding box with at least one of the plurality of structural body components (111-114) based on the obtained spatial information of the structural body component (111-114), the at least one bounding box defining an area via a real-time position of the structural body component (111-114) and a space required by the structural body component; receiving real-time point cloud data related to the environment of the mobile mining vehicle (100) from at least one sensor (120) configured to scan the environment of the mobile mining vehicle (100); and In response to determining that one or more data points included in the received real-time point cloud data overlap with the associated at least one bounding box: An overlap indication is generated, the overlap indication indicating that at least the one or more data points that overlap in the received point cloud data should be excluded, thereby separating sensor (120) hits caused by the mobile mining vehicle (100) from hits caused by the environment of the mobile mining vehicle (100) to automatically clean up raw data of the point cloud data.

2. The environment-related data management device (200) according to claim 1, wherein the at least one memory (204) and the computer program code are also configured to enable the device (200) to exclude at least one or more overlapping data points from the received point cloud data through the at least one processor (202).

3. The environment-related data management device (200) according to claim 2, wherein the at least one memory (204) and the computer program code are also configured to enable the device (200) to provide the received point cloud data and the generated overlap indication through the at least one processor (202), or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for obstacle detection, so that at least the overlapping one or more data points should be excluded from the obstacle detection.

4. An environment-related data management device (200) according to claim 2 or 3, wherein the at least one memory (204) and the computer program code are also configured to enable the device (200) to provide the received point cloud data and the generated overlap indication through the at least one processor (202), or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for collision detection, so that at least the overlapping one or more data points should be excluded from the collision detection.

5. The environment-related data management device (200) according to any one of claims 2 to 4, wherein the at least one memory (204) and the computer program code are further configured to cause the device (200) to provide, via the at least one processor (202), the received point cloud data and the generated overlap indication, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for use in generating a three-dimensional (3D) model of the environment of the mobile mining vehicle (100), such that at least the overlapping one or more data points should be excluded from the generation of the 3D model of the environment.

6. The environment-related data management device (200) according to any one of claims 2 to 5, wherein the at least one memory (204) and the computer program code are further configured to cause the device (200) to provide, through the at least one processor (202), the received point cloud data and the generated overlap indication, or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for use in modifying a 3D model of a mine associated with the mobile mining vehicle (100), so that at least the overlapping one or more data points should be excluded from the modification of the 3D model of the mine.

7. An environment-related data management device (200) according to any one of claims 2 to 6, wherein the at least one memory (204) and the computer program code are also configured to enable the device (200) to provide the received point cloud data and the generated overlap indication through the at least one processor (202), or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for generating an alert about the detected obstacle in driver assistance, so that at least the overlapping one or more data points should be excluded from the generation of the alert.

8. An environment-related data management device (200) according to any one of claims 2 to 7, wherein the at least one memory (204) and the computer program code are also configured to enable the device (200) to provide the received point cloud data and the generated overlap indication through the at least one processor (202), or to provide the received point cloud data from which at least the overlapping one or more data points have been excluded, for real-time 3D positioning, so that at least the overlapping one or more data points should be excluded from the real-time 3D positioning.

9. The environment-related data management device (200) according to any one of claims 1 to 8, wherein the real-time position information, the spatial information and the point cloud data are information and data according to a coordinate system associated with the mobile mining vehicle (100).

10. An environment-related data management device (200) according to any one of claims 1 to 9, wherein the at least one memory (204) and the computer program code are also configured to enable the device (200) to generate an overlapping area based on the overlapping one or more data points through the at least one processor (202), wherein the overlap indication indicates that the overlapping area should be excluded.

11. The environment-related data management device (200) according to claim 10, wherein the overlapping area comprises an area of ​​a bounding box type.

12. A method (300) comprising: Acquiring (301) real-time position information of the structural body components (111-114) of the mobile mining vehicle (100) by an environment-related data management device (200) for a mobile mining vehicle (100) comprising a plurality of structural body components (111-114), at least one of the structural body components being movable relative to at least another structural body component, the real-time position information indicating the real-time positions of the structural body components (111-114); The device (200) obtains (302) spatial information of the structural vehicle body components (111-114), wherein the spatial information indicates a space required by the structural vehicle body components (111-114); The device (200) associates (303) at least one bounding box with at least one structural body component among the plurality of structural body components (111-114) based on the obtained spatial information of the structural body component (111-114), wherein the at least one bounding box defines an area by the real-time position of the structural body component (111-114) and the space required by the structural body component; receiving (304), by the device (200), point cloud data relating to the environment of the mobile mining vehicle (100) from at least one sensor (120) configured to scan the environment of the mobile mining vehicle (100); and In response to determining (305) by the apparatus (200) that one or more data points included in the received point cloud data overlap with the associated at least one bounding box: An overlap indication is generated (307) by the apparatus (200), the overlap indication indicating that at least the one or more data points that overlap in the received point cloud data should be excluded, thereby separating sensor (120) hits caused by the mobile mining vehicle (100) from hits caused by the environment of the mobile mining vehicle (100) to automatically clean up raw data of the point cloud data.

13. A computer program comprising instructions for causing an environment-related data management device for a mobile mining vehicle, the mobile mining vehicle comprising a plurality of structural body components, at least one of which is movable relative to at least another structural body component: obtaining real-time position information of the structural body component of the mobile mining vehicle, the real-time position information indicating a real-time position of the structural body component; obtaining spatial information of the structural body member of the mobile mining vehicle, the spatial information indicating a space required by the structural body member; Based on the obtained spatial information of the structural body component, at least one bounding box is associated with at least one structural body component of the plurality of structural body components, the at least one bounding box defining an area by a real-time position of the structural body component (111-114) and a space required by the structural body component; receiving point cloud data related to an environment of the mobile mining vehicle from at least one sensor configured to scan an environment of the mobile mining vehicle; as well as In response to determining that one or more data points included in the received point cloud data overlap with the associated at least one bounding box: An overlap indication is generated, the overlap indication indicating that at least the one or more data points that overlap in the received point cloud data should be excluded, thereby separating sensor (120) hits caused by the mobile mining vehicle (100) from hits caused by the environment of the mobile mining vehicle (100) to automatically clean up raw data of the point cloud data.

14. A mobile mining vehicle (100), comprising: a plurality of structural body members (111-114), at least one of which is movable relative to at least another structural body member; at least one sensor (120) configured to scan an environment of the mobile mining vehicle (100); A control system (130), the control system (130) being configured to provide: real-time position information of the structural body components (111-114), the real-time position information indicating the real-time position of the structural body components (111-114); and spatial information of the structural body components (111-114), the spatial information indicating the space required by the structural body components (111-114); and An environment-related data management device (200) according to any one of claims 1 to 11.

15. The mobile mining vehicle (100) of claim 14, wherein the mobile mining vehicle (100) comprises an autonomous mobile mining vehicle.

Images