Providing control information

Abstract

An apparatus, method and computer program product for receiving information of a first position of an element positioned by a boom (450) based on a pre-programmed automation sequence; receiving image information comprising a target location (470) of the element and a representation of at least a portion of the element; determining a second position of the element and a target position of the element based on the image information; determining a position error based on the second position of the component and the target position of the component; and providing control information for adjusting the first position of the element on the basis of the position error until at least one criterion for continuing the pre-programmed automation sequence is met.

Description

Technical Field

[0001] The present application generally relates to providing control information. More specifically, the present application relates to providing control information for adjusting the position of a component. Background Art

[0002] Different types of mobile mining machines are configured to perform different types of operations. Accurate control of the operation of mobile mining machines is required to improve safety and achieve efficient production in challenging environments.

[0003] For example, a rock drilling rig comprises a carrier and at least one boom for positioning the drilling unit. Accurate positioning of the drilling unit plays an important role in rock drilling, as accurate positioning enables accurate drilling. Summary of the invention

[0004] Various aspects of examples of the present invention are set out in the claims. The scope of protection sought by various embodiments of the present invention is set out by the independent claims. Examples and features described in this specification that do not belong to the scope of the independent claims (if any) will be interpreted as examples that help understand various embodiments of the present invention.

[0005] According to a first aspect of the present invention, a device is provided, comprising components for: receiving information of a first position of an element positioned by a boom based on a preprogrammed automation sequence; receiving image information, the image information comprising a target position of the element and a representation of at least a portion of the element; determining a second position of the element and the target position of the element based on the image information; determining a position error based on the second position of the element and the target position of the element; and providing control information for adjusting the first position of the element based on the position error until at least one criterion for continuing the preprogrammed automation sequence is met.

[0006] According to a second aspect of the present invention, a method is provided, comprising: receiving information of a first position of an element positioned by a boom based on a preprogrammed automation sequence; receiving image information, the image information comprising a target position of the element and a representation of at least a portion of the element; determining a second position of the element and the target position of the element based on the image information; determining a position error based on the second position of the element and the target position of the element; and providing control information for adjusting the first position of the element based on the position error until at least one criterion for continuing the preprogrammed automation sequence is met.

[0007] According to a third aspect of the invention, a computer program is provided, comprising instructions for causing a device to perform at least the following operations: receiving information of a first position of an element positioned by a boom based on a preprogrammed automated sequence; receiving image information comprising a target position of the element and a representation of at least a portion of the element; determining a second position of the element and the target position of the element based on the image information; determining a position error based on the second position of the element and the target position of the element; and providing control information for adjusting the first position of the element based on the position error until at least one criterion for continuing the preprogrammed automated sequence is met.

[0008] According to a fourth aspect of the present invention, a device is provided, which includes at least one processor and at least one memory including computer program code, and the at least one memory and the computer program code are configured to, together with the at least one processor, enable the device to perform at least the following operations: receive information about a first position of an element positioned by a boom based on a preprogrammed automation sequence; receive image information, the image information including a target position of the element and a representation of at least a portion of the element; based on the image information, determine a second position of the element and the target position of the element; based on the second position of the element and the target position of the element, determine a position error; and based on the position error, provide control information for adjusting the first position of the element until at least one criterion for continuing the preprogrammed automation sequence is met.

[0009] According to a fifth aspect of the present invention, a non-transitory computer-readable medium is provided, comprising program instructions for causing an apparatus to perform at least the following operations: receiving information of a first position of an element positioned by a boom based on a preprogrammed automated sequence; receiving image information comprising a target position of the element and a representation of at least a portion of the element; determining a second position of the element and the target position of the element based on the image information; determining a position error based on the second position of the element and the target position of the element; and providing control information for adjusting the first position of the element based on the position error until at least one criterion for continuing the preprogrammed automated sequence is met.

[0010] According to a sixth aspect of the present invention, a computer-readable medium is provided, comprising program instructions for causing an apparatus to perform at least the following operations: receiving information of a first position of an element positioned by a boom based on a preprogrammed automated sequence; receiving image information comprising a target position of the element and a representation of at least a portion of the element; determining a second position of the element and the target position of the element based on the image information; determining a position error based on the second position of the element and the target position of the element; and providing control information for adjusting the first position of the element based on the position error until at least one criterion for continuing the preprogrammed automated sequence is met. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Some example embodiments will now be described with reference to the accompanying drawings:

[0012] Figure 1 A block diagram illustrating an example apparatus in which examples of the disclosed embodiments may be applied;

[0013] Figure 2 A block diagram illustrating another example apparatus in which examples of the disclosed embodiments may be applied;

[0014] Figure 3 An example system incorporating aspects of the example embodiments is shown;

[0015] Figure 4 An example of position error is shown; and

[0016] Figure 5 An example method incorporating aspects of examples of the present invention is shown. DETAILED DESCRIPTION

[0017] The following embodiments are exemplary. Although the specification may refer to "one", "an" or "some" embodiments in multiple places in the text, this does not necessarily mean that each reference refers to the same embodiment, or that a particular feature applies only to a single embodiment. Individual features of different embodiments may also be combined to provide other embodiments.

[0018] Example embodiments relate to a mobile mining machine configured to execute at least one preprogrammed automated sequence. A mobile mining machine comprising a carrier and at least one boom may be configured to position at least one element by the at least one boom based on a preprogrammed automated sequence. The preprogrammed automated sequence may include, for example, a sequence for changing a drill bit, a bolting sequence, or a sequence for installing a riser. Each of these automated sequences requires accurate positioning of an element associated with the sequence, such as a drill bit, a bolting sequence, or a riser.

[0019] Currently, the positioning of the element is based on a kinematic model of the mobile mining machine using measurement information from at least one sensor associated with the mobile mining machine. Based on the kinematic model and the measurement information, the position of the element relative to the mobile machine is calculated. However, the accuracy of such a system depends on the accuracy of the sensors, sensor calibration, accuracy of the boom sensors, joint play and wear. Furthermore, there is no feedback between the actual position of the element and the position determined based on the kinematic model. For example, inaccurate positioning can also affect drilling accuracy and production.

[0020] One example embodiment relates to an apparatus for controlling a mobile mining machine, the mobile mining machine comprising a carrier and at least one boom, the at least one boom being configured for positioning at least one element based on a preprogrammed automated sequence, the apparatus being configured to: receive information of a first position of an element positioned by the boom based on the preprogrammed automated sequence; receive image information comprising a target position of the element and a representation of at least a portion of the element; determine a second position of the element and the target position of the element based on the image information; determine a position error based on the second position of the element and the target position of the element; and provide control information for adjusting the first position of the element based on the position error until at least one criterion for continuing the preprogrammed automated sequence is met.

[0021] Figure 1 1 is a block diagram illustrating an apparatus 100 operating in accordance with an example embodiment of the present invention. The apparatus 100 may be, for example, an electronic device, such as a module consisting of an automation or control system, a chip or a chipset. The apparatus 100 includes one or more control circuits (such as at least one processor 110) and at least one memory 160 (including one or more algorithms, such as computer program instructions 120), wherein the at least one memory 160 and the computer program instructions are configured to, together with the at least one processor 110, cause the apparatus to perform any of the example functions described below.

[0022] exist Figure 1 In the example of the embodiment, the processor 110 is a control unit that is operatively connected to read from and write to the memory 160. The processor 110 may also be configured to receive control signals received via the input interface, and / or the processor 110 may be configured to output control signals via the output interface. In an example embodiment, the processor 110 may be configured to convert the received control signals into appropriate commands for controlling the functions of the device.

[0023] The at least one memory 160 stores computer program instructions 120 which, when loaded into the processor 110, control the operation of the apparatus 100, as described below. In other examples, the apparatus 100 may include more than one memory 160 or different types of storage devices.

[0024] The computer program instructions 120 for implementing the exemplary embodiments of the present invention, or a portion of such computer program instructions, may be loaded onto the apparatus 100 by the manufacturer of the apparatus 100, the user of the apparatus 100, or the apparatus 100 itself based on a download procedure, or the instructions may be pushed to the apparatus 100 by an external device. The computer program instructions may reach the apparatus 100 via an electromagnetic carrier signal, or may be copied from a physical entity such as a computer program product, a storage device or a recording medium, such as a USB stick, a compact disc (CD), a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), or a Blu-ray disc.

[0025] 2 is a block diagram illustrating an apparatus 200 according to an exemplary embodiment of the present invention. The apparatus 200 may be an electronic device, such as a module composed of an automation system or a control system, a personal computer (PC), a laptop, a desktop, a wireless terminal, a communication terminal, a computing device, etc. In the following example, it is assumed that the apparatus 200 is a computing device.

[0026] exist Figure 2 In an example embodiment of the present invention, the apparatus 200 is shown to include the apparatus 100, a display 210, a user interface 220 for interacting with the computing device 200, and a communication module 230. The display 210 can also be configured to be used as a user interface. For example, the display can be a touch screen display. In an example embodiment, the display 210 and / or the user interface 220 can be external to the apparatus 200, but in communication with the apparatus 200.

[0027] Additionally or alternatively, the user interface may also include manually operable controls such as buttons, keys, touchpad, joystick, stylus, pen, scroll wheel, rocker, keypad, keyboard, or any suitable input mechanism for entering and / or accessing information.

[0028] The communication module 230 may be configured to communicate with one or more other devices, such as one or more sensors, a control unit, a work machine (such as a mobile mining vehicle), etc. Communicating with one or more devices may include receiving information from the one or more devices and / or sending information to the one or more devices via a wired or wireless connection.

[0029] Figure 2The apparatus 200 of the example may also be configured to establish radio communication with another device using, for example, a cellular network, Bluetooth or WiFi connection, etc. Alternatively or additionally, the apparatus 200 may be configured to communicate with another device using a wired connection.

[0030] According to an example embodiment, the apparatus 200 is configured to communicate with at least one camera. The apparatus 200 may be configured to communicate with the at least one camera via, for example, the communication module 230. The at least one camera may include, for example, a video camera, a stereo camera, or a monocular camera.

[0031] Communicating with the at least one camera may include, for example, receiving image information captured or streamed by the at least one camera. As another example, communicating with the at least one camera may include, for example, instructing a camera to capture or stream image information. The image information may include image data, such as one or more frames of data, including, for example, a visual representation of one or more objects.

[0032] According to an example embodiment, the apparatus 200 comprises a module operatively connected to a mobile mining machine. For example, the apparatus 200 may be implemented in a control system of a mobile mining machine, or the apparatus 200 may be configured to communicate with a control system of a mobile mining machine.

[0033] According to an example embodiment, the apparatus 200 is configured for controlling one or more functions and / or actuators of a mobile mining machine. The apparatus 200 may be configured for controlling the one or more functions and / or actuators directly or via one or more control units of a control system of the mobile mining machine. Controlling the one or more functions and / or actuators via the one or more control units may include, for example, communicating with the one or more control units via, for example, a controller area network (CAN) bus or an Ethernet network.

[0034] According to an example embodiment, the mobile mining machine comprises a drilling rig. The drilling rig may comprise a drilling rig for underground drilling or a drilling rig for surface drilling. The drilling rig may comprise, for example, a rock drilling rig.

[0035] According to an example embodiment, a mobile mining machine comprises a movable carrier and at least one boom. The at least one boom may be equipped with different types of operating units at the distal end of the at least one boom. For example, the at least one boom may be equipped with a rock drilling unit, a bolting unit, a riser installer, a casing installer, a rock support installer, etc.

[0036] The rock drilling unit is configured for drilling in rock. The rock drilling unit comprises a rock drilling machine and a feed beam configured for providing a linear movement along the feed beam to the rock drilling machine when drilling a hole.

[0037] The bolting unit is configured for reinforcing rock by means of bolts or elongated reinforcing material, resin, cement etc. The bolting unit comprises a rock bolting machine and a feed beam for feeding the rock bolt or the corresponding elongated reinforcing material into the drilled hole.

[0038] According to an example embodiment, a mobile mining machine is configured to execute at least one preprogrammed automation sequence. The mobile mining machine may be configured to initiate the at least one preprogrammed automation sequence based on input received, for example, from an operator of the mobile mining machine, or automatically as part of the operation of the mobile mining machine.

[0039] The preprogrammed automation sequence may include a preprogrammed sequence of operations for performing at least one task. The preprogrammed sequence of operations may include, for example, a plurality of consecutive operations performed by the mobile mining machine to perform the task. The plurality of consecutive operations performed by the mobile mining machine may include, for example, controlling one or more functions and / or actuators of the mobile mining machine.

[0040] Preprogrammed automation sequences may include, for example, sensor-based and / or time-based automation sequences.

[0041] The sensor-based automation sequence may be configured to initiate, execute, and / or terminate the automation sequence using data received from at least one sensor. For example, a mobile mining machine may be configured to execute a sensor-based automation sequence for controlling a boom of the mobile mining machine in response to determining that the boom of the mobile mining machine is in a particular position.

[0042] The time-based automation sequence may be configured to determine a specific time instance or duration associated with the operation of the mobile mining machine to initiate, execute and / or end the automation sequence. For example, the mobile mining machine may be configured to execute a time-based automation sequence for controlling the boom in response to determining that a specific time period has elapsed since the boom was moved to a specific position, and so on.

[0043] The preprogrammed automation sequence may also include a combination of a sensor-based automation sequence and a time-based automation sequence. For example, one or more preprogrammed operations of the automation sequence may be sensor-based, and one or more preprogrammed operations may be time-based.

[0044] The pre-programmed automated sequences executed by the mobile mining machine may include, for example, a sequence for changing a drill bit, a bolting sequence, a sequence for installing a riser, a sequence for changing drill rods, or any other applicable sequence.

[0045] As described above, the preprogrammed sequence includes preprogrammed operations for performing a task. As an example, a sequence for changing a drill bit may include multiple operations such as loosening the drill bit, retracting the drill tool, closing the drill rod holder, tightening the drill shank thread, opening the drill rod holder, controlling the position of the drill bit to the drill bit changer position, activating the position adjustment of the drill bit, storing or retrieving the drill bit, retracting the drill tool, and verifying whether the operation was successful.

[0046] According to an example embodiment, the position adjustment includes adjusting the position of the element based on the image information.For example, the position adjustment of an element such as a drill bit, a bolt, a riser, or a drill rod is discussed in more detail in the following examples.

[0047] A preprogrammed automation sequence may include one or more subsequences that are executed as part of the automation sequence. For example, an activation position adjustment included in a sequence for changing a drill bit may include a subsequence consisting of multiple operations for adjusting the position of the drill bit / boom.

[0048] It should be noted that the sequence for installing the riser or the bolting sequence may include operations different from the sequence for changing the drill bit. Alternatively, the sequence for installing the riser or the bolting sequence may have operations that are partially the same as the sequence for changing the drill bit, such as, for example, verifying whether the operation was successful.

[0049] As mentioned above, the apparatus 200 is configured to communicate with different devices, such as different sensors and / or control units.

[0050] According to an example embodiment, the apparatus 200 is configured for receiving information of a first position of an element positioned by a boom based on a preprogrammed automated sequence.

[0051] The apparatus 200 may be configured to receive information of the first position from a separate device, such as a sensor, or the apparatus 200 may be configured to determine the first position based on, for example, measurement data from one or more sensors.

[0052] The information about the first position of the element may include information about the position of the element relative to the mobile mining machine. The position of the element relative to the mobile mining machine may include, for example, the position of the element relative to a predetermined reference point associated with the mobile mining machine. As another example, the information about the first position of the element may include the position of the element in coordinates of a coordinate system, such as the position of the element in mine coordinates.

[0053] The information about the first position may include measurement data indicative of the first position, a simulated position, or a position determined based on a model of the mobile mining machine and measurement data from one or more sensors associated with the mobile mining machine.

[0054] The model of the mobile mining machine may include, for example, a kinematic model of the mobile mining machine.

[0055] A kinematic model of a mobile mining machine comprises a mathematical description of at least a portion of the mobile mining machine. The kinematic model describes the motion of the machine without taking into account the forces causing the motion. The kinematic model may be used to estimate the position of the machine based on measurement data from one or more sensors associated with the machine or the motion of the machine caused by a given control input. The kinematic model of the mobile mining machine comprises at least the size of the mobile mining machine and / or the reach of the mobile mining machine, such as the range of movement of at least one boom of the mobile mining machine.

[0056] According to an example embodiment, the first position comprises a position determined using a kinematic model of the mobile mining machine and measurement data from at least one sensor associated with the mobile mining machine.

[0057] The elements may include elements associated with a pre-programmed automated sequence. The elements may include replaceable elements or fixed elements. Replaceable elements refer to elements that can be removed or replaced due to wear or different stages of operation, while fixed elements refer to elements that are intended to last for a longer period of time but can be replaced, for example, when damaged. Replaceable elements may include, for example, drill bits, risers, anchors, drill rods, etc. Fixed elements may include, for example, booms, feed beams, etc.

[0058] According to an example embodiment, the at least one element comprises a drill bit, a riser or a bolt.

[0059] Thus, the first position of the element may include, for example, the position of a drill head relative to the mobile mining machine, the position of a riser relative to the mobile mining machine, or the position of a bolt relative to the mobile mining machine.

[0060] As described above, the apparatus 200 is configured to communicate with at least one camera. The at least one camera may be associated with the mobile mining machine, or the at least one camera may be located in the environment of the mobile mining machine.

[0061] The apparatus 200 may be configured to receive image information from a single camera or multiple cameras. The image information may include information about the environment of the mobile mining machine and / or information related to the mobile mining machine. The information about the environment of the mobile machine may include a representation of the environment, and the information related to the mobile mining machine may include a representation of at least a portion of the mobile mining machine, an element related to the mobile mining machine, etc. According to an example embodiment, the apparatus 200 is configured to receive image information including a target position of an element and a representation of at least a portion of an element.

[0062] The target position includes the position of the target of the element. For example, the target may include a cavity configured to receive the element.

[0063] According to one example embodiment, the target location for the component includes an existing hole, and the component is to be placed in the hole.

[0064] According to an example embodiment, the existing hole comprises a drilled hole or a man-made hole. The drilled hole may comprise, for example, a hole drilled in a rock, while the man-made hole may comprise, for example, a hole in a drill bit holder.

[0065] The target position of the element may include a target position of the element associated with a preprogrammed automation sequence. For example, if the element is a drill bit, the target position may include a position that allows the drill bit to be replaced. As another example, if the element is a riser, the target position may include a position that allows the riser to be installed. As a further example, if the element is an anchor bolt, the target position may include a position that allows the anchor bolt to be installed.

[0066] The image information may include, for example, a video feed from a camera. A video feed includes a series of images electronically processed into a predefined format. As another example, the image information may include one or more images captured by a camera.

[0067] According to an example embodiment, the image information includes image data, such as one or more frames of data.

[0068] According to an example embodiment, the apparatus 200 is configured to determine, based on the image information, the presence of one or more elements and / or the status of one or more elements in the image information.

[0069] Determining the presence of an element may include determining whether a representation of at least a portion of the element is contained in the image information. Determining the presence of the element may include determining that a representation of the element is contained in the image information, or that a representation of the element is not contained in the image information.

[0070] Determining the state of the element may include determining the state of the element based on the representation of the element in the image information. Determining the state of the element may include comparing the representation of the element with the reference information. Determining the state of the element may include, for example, determining whether the element is of the correct type, determining whether the element is not damaged, and / or determining whether the element is correctly positioned.

[0071] Determining the presence of one or more elements and / or the state of one or more elements may include, for example, processing the image information. Processing the image information may include, for example, filtering the image information, parsing the image information, object detection, or other suitable processing methods.

[0072] The apparatus 200 may be configured to process the image information using, for example, artificial intelligence (AI). Artificial intelligence may include, for example, different types of mechanisms for analyzing image information. AI may include, for example, one or more machine vision algorithms, such as one or more algorithms for object detection and / or object tracking.

[0073] Machine vision algorithms are configured to extract information from image information to obtain the data required to perform an action.

[0074] According to an example embodiment, the apparatus 200 is configured to analyze image information using a machine vision algorithm. Figure 2 In the example of , the machine vision algorithm includes an algorithm for automatically acquiring and analyzing images to determine the target position of the component and at least a portion of the component. The machine vision algorithm can be stored in the memory 160, for example.

[0075] Machine vision algorithms may include, for example, neural networks. A neural network includes multiple algorithms that are configured to identify underlying relationships and / or patterns in a set of data.

[0076] A neural network includes multiple layers of nodes, such as an input layer, one or more hidden layers, and an output layer. Each node has an associated weight and threshold, and each node is connected to the nodes in the next layer. If the output of a single node is above the threshold, the node is activated and it sends data to the next layer of the network.

[0077] Neural networks are configured to learn based on training data, and these neural networks improve their accuracy over time. The learning algorithms for neural networks can include supervised learning or unsupervised learning. Supervised learning uses labeled input data and output data, which means that the expected output is known, while unsupervised learning algorithms do not use labeled input data and output data.

[0078] According to an example embodiment, the apparatus 200 is configured to determine the second position of the component and the target position of the component based on the image information.

[0079] Determining the second position of the element and the target position of the element based on the image information may include using a machine vision algorithm to identify a representation of the element and the target of the element in the image information. The machine vision algorithm may know, for example, the shape of the element and the shape of the target, and identify the element and the target based on the shapes of the corresponding elements. For example, the machine vision algorithm may know the shape of a drill bit, an anchor or a riser, and the shape of a hole.

[0080] Determining the second position of the component and the target position of the component based on the image information may further include determining the position of the component and the position of the target relative to a reference point.

[0081] Reference points may include static reference points or dynamic reference points. Static reference points may include, for example, markings in a tunnel pit wall, etc. Dynamic reference points may include, for example, reference points associated with a mobile mining machine, such as points in a movable boom, etc.

[0082] According to an example embodiment, the apparatus 200 is configured to perform at least one operation affecting at least one of: a second position of an element or a target position of an element while receiving the image information.

[0083] According to an example embodiment, the at least one operation includes an operation that causes a change in the position of the element and / or the target position. The at least one operation may include, for example, causing a controlled collision between the element and the target position. For example, the device 200 may be configured to cause a controlled collision between a drill bit or a drill rod and a drill bit holder. The controlled collision includes an intentionally caused collision.

[0084] The controlled collision may include a collision with a defined collision position. The defined collision position may include, for example, a selected position that causes the element and the target position to collide while allowing an unobstructed view of the representation of the selected object contained in the image information. For example, assuming that the apparatus 200 is configured to cause a controlled collision between a drill bit and a drill bit holder on the right hand side of a mobile mining machine, the defined position may include the lower left corner of the drill bit holder so that the hole in the drill bit holder is not obstructed by the drill bit.

[0085] According to an example embodiment, the apparatus 200 is configured to detect a change in the position of at least one representation of an object contained in the image information. The object may include, for example, at least one element (such as a drill bit, an anchor, a drill rod, or a boom), a target position of an element (e.g., a drill bit holder), or some other object.

[0086] According to an example embodiment, the apparatus 200 is configured for detecting changes in positions of a plurality of representations of an object contained in image information.

[0087] Without limiting the scope of the claims, an advantage of detecting changes in the positions of multiple objects is that a detected change in the position of an object may be confirmed by a detected change in one or more other objects.

[0088] According to an example embodiment, the apparatus 200 is configured to store information related to operations affecting the second position of the element and / or the target position of the element.

[0089] The information about the operation influencing the second position of the element and / or the target position of the element may comprise, for example, information about the position of the unit causing the change in position. For example, in case the operation influencing the second position and / or the target position of the element comprises a controlled collision between the drill bit / drill rod and the drill bit holder, the position of the rock drilling unit may be stored when the rock drilling unit is controlled forward and when the rock drilling unit is retracted.

[0090] According to an example embodiment, the apparatus 200 is configured to determine the second position of the component based on the image information and based on information of at least one operation affecting the second position of the component and / or the target position of the component.

[0091] Without limiting the scope of the claims, an advantage of determining the second position of the element based on the image information and based on information of at least one operation affecting the second position of the element and / or the target position of the element is that more accurate information about the second position of the element can be received. For example, in the case where the operation resulting in a change in the position and / or the target position of the element includes an intentional collision between the drill bit / drill rod and the drill bit holder, the device 200 can be configured to determine three-dimensional (3D) position information about the second position of the element.

[0092] The apparatus 200 may be configured to determine the second position of the component and the target position of the component based on predefined conditions. For example, the apparatus 200 may be configured to determine the second position of the component and the target position in response to detecting that a preprogrammed automation sequence has reached a specific state.

[0093] According to an example embodiment, the apparatus 200 is configured to determine the second position in response to determining that the first position is within a threshold distance from the target position. The threshold distance may include, for example, 1 to 30 centimeters, such as 5, 10, 15, 20 or 25 centimeters, or even higher.

[0094] Without limiting the scope of the claims, an advantage of determining the second position of the element based on the image information is that the position of the element can be determined independently of sensors associated with the mobile mining machine, such as boom sensors. This enables accurate determination of the element despite varying sensor accuracy, sensor calibration, boom structure accuracy and / or wear.

[0095] According to an example embodiment, the apparatus 200 is configured to determine a position error based on the second position of the component and the target position of the component.

[0096] According to an example embodiment, the position error comprises a parameter value indicative of a misalignment of the component relative to a target position of the component.

[0097] The misalignment of the element may include, for example, distance information and / or orientation information. The distance information may include, for example, the distance between the second position and the target position. The orientation information may include, for example, the angle between the second position and the target position.

[0098] According to an example embodiment, the apparatus 200 is configured to determine control information for adjusting a first position of an element. The control information may include one or more instructions to control the position of the element so as to reach a target position.

[0099] According to an example embodiment, the apparatus 200 is configured to determine control information based on image information.

[0100] The control information may include, for example, control information for moving the element from a first position over a specific distance and / or in a specific direction. Alternatively, the control information may include coordinates of a target position or control information corresponding to image information, such as the number of pixels by which the element is to be moved.

[0101] The control information may include an absolute value or a relative value. For example, the control information may include control information for moving the element 5 centimeters to the left or 10 centimeters downward from the current position.

[0102] According to an example embodiment, the apparatus 200 is configured for providing control information for adjusting a first position of a component based on the position error until at least one criterion for continuing a preprogrammed automation sequence is met.

[0103] According to an example embodiment, the apparatus 200 is configured to determine, based on the image information, that the at least one criterion is met.

[0104] The at least one criterion may be related to the position of the element or to a target of a preprogrammed automation sequence.

[0105] According to an example embodiment, the at least one criterion comprises: the position of the element corresponds to a target position, or the operation is completed.

[0106] Completed operations may include, for example, a successful drill bit change, a successful riser installation, or a successful rock bolt drive.

[0107] The apparatus 200 may be configured for providing control information to a control unit configured for controlling at least one actuator of the mobile mining machine, or the apparatus 200 may be configured for directly controlling the at least one actuator of the mobile mining machine.

[0108] According to an example embodiment, the apparatus 200 is configured for controlling at least one actuator using the control information.

[0109] It should be noted that the apparatus 200 may be configured to determine multiple position errors, for example for a mobile mining machine including multiple booms. Assuming that the element controlled by the boom is the boom itself, the apparatus 200 may be configured to determine a first position error of a first boom and a second position error of a second boom based on the image information. The apparatus 200 may be further configured to provide first control information for adjusting the position of the first boom based on the first position error, and to provide second control information for adjusting the position of the second boom based on the second position error. The apparatus 200 may be further configured to use the position of the boom to compensate for errors in the kinematic model.

[0110] According to an example embodiment, the device 200 includes components for performing characteristic functions of the device 200, wherein the components for performing include at least one processor 110, at least one memory 160, the at least one memory including computer code 120, and the computer code 120 is configured to cause the execution of the device 200 together with the at least one processor 110.

[0111] According to an example embodiment, the device 200 includes: a component for receiving information of a first position of an element positioned by a boom based on a preprogrammed automation sequence, a component for receiving image information including a target position of the element and a representation of at least a portion of the element, a component for determining a second position of the element and the target position of the element based on the image information, a component for determining a position error based on the second position of the element and the target position of the element, and a component for providing control information for adjusting the first position of the element based on the position error until at least one criterion for continuing the preprogrammed automation sequence is met.

[0112] According to an example embodiment, the apparatus 200 may further include a component for determining a second position in response to determining that the first position is within a threshold distance from the target position. According to an example embodiment, the apparatus 200 may further include a component for analyzing the image information using a machine vision algorithm and / or a component for determining that the at least one criterion is satisfied based on the image information. According to an example embodiment, the apparatus 200 may further include a component for determining control information based on the image information and / or a component for controlling at least one actuator using the control information, the control information including one or more parameters for controlling the direction of the boom or coordinate information for positioning the boom.

[0113] Figure 3 An example system 300 incorporating aspects of the example embodiments is shown. Figure 3 In the example of the present invention, the system 300 comprises a mobile mining machine and a Figure 2 The device 200 of the example corresponds to the device 200 composition.

[0114] exist Figure 3 In an example of the invention, the mobile mining machine comprises a rock drill comprising at least one boom with a rock drilling unit disposed at a distal end of the at least one boom. The rock drilling unit comprises a feed system configured to keep a drill bit in contact with the rock during drilling.

[0115] The system 300 includes the device 200, an image processing unit 310 operatively connected to a camera 320, a position control unit 330 operatively connected to a boom control actuator 340, and a boom kinematics model 350 configured to receive information from a boom sensor 360. It should be noted that the image processing unit 310 may be included in the device 200, as shown by the dashed line, or the image processing unit 310 may be operatively connected to the device 200 but separate from the device 200.

[0116] The apparatus 200 is configured to receive information from the kinematic model 350 and the image processing unit 310. Figure 3 In the example of , the information from the kinematic model 350 includes information about a first position of an element to be positioned by the boom based on a preprogrammed automation sequence, and the information from the image processing unit 310 includes image information including a target position of the element and a representation of at least a portion of the element.

[0117] The pre-programmed automated sequences may include, for example, an automated sequence for changing a drill bit, a bolting sequence, or a sequence for installing a riser, performed by a mobile mining machine.

[0118] The apparatus 200 is further configured to determine a second position of the component and a target position of the component based on information from the image processing unit 310 , and determine a position error based on the second position of the component and the target position of the component.

[0119] The apparatus 200 is further configured to provide control information to the position control unit 330 based on the position error to adjust the first position of the component until at least one criterion for continuing the preprogrammed automation sequence is met. Figure 3 In the example, the control information is used to control at least one boom control actuator 340, and the at least one boom control actuator 340 is configured to control the position of the boom.

[0120] Figure 4 An example arrangement 400 is shown for determining a position error of a mobile mining machine 410 comprising the apparatus 200. It is assumed that an element 440 is positioned by a boom 450 based on a pre-programmed automated sequence.

[0121] The pre-programmed automated sequences executed by the mobile mining machine may include, for example, a sequence for changing a drill bit, a bolting sequence, a sequence for installing a riser, or any other applicable sequence.

[0122] The apparatus 200 is configured for receiving information of a first position 430 of an element positioned by a boom 450. The first information may comprise a position of the element determined using a kinematic model of the mobile mining machine.

[0123] exist Figure 4 In the example of , the mobile mining machine 410 further includes a camera 320 , and the apparatus 200 is configured to receive image information including a target position 470 of the element 440 and a representation of at least a portion of the element 440 .

[0124] The apparatus 200 is further configured to determine a second position 420 of the component 440 and a target position 470 of the component based on the image information, and determine a position error 460 based on the second position 420 and the target position 470 of the component. The position error 460 includes a parameter value indicating a misalignment of the component relative to the target position of the component.

[0125] although Figure 4 The example shows that the mobile mining machine includes one boom, but the position error can also be determined in a multi-boom system. Assuming that the element controlled by the boom is the boom itself, the device 200 can be configured to determine a first position error of the first boom and a second position error of the second boom based on the image information. The device 200 can be further configured to provide first control information for adjusting the position of the first boom based on the first position error, and to provide second control information for adjusting the position of the second boom based on the second position error. The device 200 can be further configured to use the position of the boom to compensate for errors in the kinematic model.

[0126] Figure 5 An example method 500 is shown that incorporates aspects of previously disclosed embodiments. More specifically, the example method 500 shows a method for controlling a mobile mining machine that includes a carrier and at least one boom configured to position at least one element based on a preprogrammed automated sequence. The method 500 may include a computer-implemented method performed by the apparatus 200.

[0127] The method begins by receiving 505 information of a first position of a component positioned by a boom based on a preprogrammed automated sequence.

[0128] exist Figure 5In an example of the invention, the first position comprises a position determined using a kinematic model of the mobile mining machine and measurement data from at least one sensor associated with the mobile mining machine.

[0129] The elements may include elements such as a drill bit, a riser, or a bolt that are associated with a pre-programmed automated sequence.

[0130] The method continues with receiving 510 image information including a target position of the component and a representation of at least a portion of the component. The image information may include, for example, a video feed from a camera or one or more images captured by a camera.

[0131] Receiving image information may also include processing the image information. Figure 5 In an example of, receiving the image information includes analyzing the image information using a machine vision algorithm. The machine vision algorithm may include, for example, a neural network.

[0132] exist Figure 5 In the example of , the target location of the element includes an existing hole, and the element is to be placed in the hole. The existing hole may include a hole that has been drilled or an artificial hole. The existing hole may include, for example, a hole drilled into rock, and the artificial hole may include, for example, a hole in a drill bit holder.

[0133] The method continues by determining 515 a second position of the component and a target position of the component based on the image information.Apparatus 200 may be configured to determine the second position in response to determining that the first position is within a threshold distance from the target position.

[0134] The method continues by determining 520 a position error based on the second position of the component and the target position of the component.

[0135] exist Figure 5 In an example, the position error includes a parameter value indicating a misalignment of the component relative to a target position of the component.

[0136] The method further continues by providing 525 control information for adjusting the first position of the component based on the position error until at least one criterion for continuing the preprogrammed automation sequence is met.

[0137] The at least one criterion may relate to the position of the element or to a target of a preprogrammed automation sequence.

[0138] exist Figure 5 In an example of , the at least one criterion includes: the position of the element corresponds to the target position, or the operation is completed. The completed operation may include, for example, successful replacement of the drill bit, successful installation of the riser, or successful driving of the anchor.

[0139] Without limiting the scope of the claims, an advantage of determining a position error based on the second position of the element and the target position of the element and providing control information for adjusting the first position of the element based on the position error is that feedback between the calculated position and the actual position can be provided.

[0140] Without limiting the scope, interpretation or application of the appended claims in any way, the technical effect of one or more of the example embodiments disclosed herein is that the accuracy of face drilling, crosscut drilling and riser installation is improved. Another technical effect is that position errors can be used to compensate for errors in the kinematic model.

[0141] In this application, the term "circuitry" may refer to one or more or all of the following: (a) a pure hardware circuit implementation (such as an implementation using only analog and / or digital circuits) and (b) a combination of hardware circuitry and software, such as (as applicable): (i) a combination of analog and / or digital hardware circuitry and software / firmware, and (ii) any portion of a hardware processor with software (including a digital signal processor), software and memory, which work together to enable a device (such as a mobile phone or server) to perform various functions, and (c) a hardware circuit and / or processor, such as a microprocessor or a portion of a microprocessor, which requires software (e.g., firmware) to operate, but the software may not be present when the software is not required for operation.

[0142] This definition of circuitry applies to all uses of this term in this application, including in any claims. By way of further example, as used in this application, the term "circuitry" also covers an implementation of merely a hardware circuit or processor (or multiple processors), or a portion of a hardware circuit or processor and its accompanying software and / or firmware. The term "circuitry" also covers (for example, if applicable to a particular claim element) a baseband integrated circuit or processor integrated circuit of a mobile device, or a similar integrated circuit in a server, cellular network device, or other computing or network device.

[0143] Embodiments of the present invention may be implemented in software, hardware, application logic, or a combination of software, hardware, and application logic. The software, application logic, and / or hardware may reside on an apparatus, a separate device, or multiple devices. If desired, a portion of the software, application logic, and / or hardware may reside on an apparatus, a portion of the software, application logic, and / or hardware may reside on a separate device, and a portion of the software, application logic, and / or hardware may reside on multiple devices. In an example embodiment, the application logic, software, or instruction set is maintained on any of a variety of conventional computer-readable media. In the context of this document, a "computer-readable medium" may be any medium or component that can contain, store, communicate, propagate, or transport instructions for use by or in connection with an instruction execution system, apparatus, or device (such as a computer), and Figure 2 An example of a computer is described and illustrated in . Computer-readable media may include computer-readable storage media, which may be any media or component that can contain or store instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer.

[0144] If desired, the different functions discussed herein may be performed in different orders and / or simultaneously with each other. In addition, if desired, one or more of the above-mentioned functions may be optional or may be combined.

[0145] Although various aspects of the invention are set out in the independent claims, further aspects of the invention comprise other combinations of features from the described embodiments and / or dependent claims with features of the independent claims, not just the combinations explicitly set out in the claims.

[0146] It will be obvious to a person skilled in the art that, as technology advances, the inventive concept can be implemented in various ways.The invention and its embodiments are not limited to the examples described above but may vary within the scope of the claims.

Claims

1. An apparatus for controlling a mobile mining machine, the mobile mining machine comprising a carrier and at least one boom, the at least one boom being configured to position at least one element based on a preprogrammed automated sequence, the apparatus comprising at least one processor and at least one memory comprising computer program code, the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to at least: receiving information of a first position of a component positioned by the boom based on a preprogrammed automated sequence; receiving image information including a target location of the component and a representation of at least a portion of the component; determining a second position of the component and a target position of the component based on the image information; determining a position error based on the second position of the component and the target position of the component, the position error comprising a parameter value indicative of a misalignment of the component relative to the target position of the component; as well as Based on the position error, control information is provided for adjusting the first position of the component until at least one criterion is met for continuing the preprogrammed automation sequence.

2. The device according to claim 1, wherein: The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to determine the second location in response to determining that the first location is within a threshold distance from the target location.

3. The device according to claim 1 or 2, wherein: The target location of the component includes an existing hole, and the component is to be placed in the hole.

4. The device according to claim 3, wherein: The existing holes include drilled holes or artificial holes.

5. An apparatus according to any preceding claim, wherein: The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to analyze image information using a machine vision algorithm.

6. An apparatus according to any preceding claim, wherein: The first position comprises a position determined using a kinematic model of the mobile mining machine and measurement data from at least one sensor associated with the mobile mining machine.

7. An apparatus according to any preceding claim, wherein: The at least one element comprises a drill bit, a riser or a bolt.

8. An apparatus according to any preceding claim, wherein: The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to determine the control information based on the image information.

9. An apparatus according to any preceding claim, wherein: The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to control at least one actuator using the control information.

10. An apparatus as claimed in any preceding claim, the at least one memory and the computer program code being further configured to, with the at least one processor, cause the apparatus to determine that the at least one criterion is met based on the image information.

11. An apparatus according to any preceding claim, wherein: The at least one criterion includes: the position of the element corresponds to the target position, or the operation is completed.

12. An apparatus according to any preceding claim, wherein: The at least one memory and the computer program code are further configured to, together with the at least one processor, cause the apparatus to perform at least one operation affecting at least one of: the second position of the element, or the target position of the element while receiving the image information.

13. A drilling rig comprising an apparatus according to any preceding claim.

14. A method of controlling a mobile mining machine, the mobile mining machine comprising a vehicle and at least one boom, the at least one boom being configured to position at least one element based on a preprogrammed automated sequence, the method comprising: receiving information of a first position of a component positioned by the boom based on a preprogrammed automated sequence; receiving image information including a target location of the component and a representation of at least a portion of the component; determining a second position of the component and a target position of the component based on the image information; determining a position error based on the second position of the component and the target position of the component, the position error comprising a parameter value indicative of a misalignment of the component relative to the target position of the component; as well as Based on the position error, control information is provided for adjusting the first position of the component until at least one criterion is met for continuing the preprogrammed automation sequence.

15. A computer program comprising instructions for causing an apparatus to perform at least the following operations: receiving information of a first position of a component positioned by the boom based on a preprogrammed automated sequence; receiving image information including a target location of the component and a representation of at least a portion of the component; determining a second position of the component and a target position of the component based on the image information; determining a position error based on the second position of the component and the target position of the component, the position error comprising a parameter value indicative of a misalignment of the component relative to the target position of the component; as well as Based on the position error, control information is provided for adjusting the first position of the component until at least one criterion is met for continuing a preprogrammed automation sequence.

Images