Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environment

Abstract

A mining machine, a field computer device, and a kit for the machine are configured to perform a computer- implemented method for monitoring and controlling operation of the machine in a mining site. The machine comprises a drilling implement configured to have one or more rods coupled thereto. The method comprises, as the machine is operating in the mining site, receiving inertial sensor measurements acquired by at least one inertial measurement unit (IMU) sensor over a first period of time; applying a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the machine, for the first period of time; determining at least one operation indicator from the at least one operating state, the operation indicator comprising a duration of the operating state; and initiating an action in dependence on the determining of the operation indicator.

Description

TITLEFIELD COMPUTER DEVICE, MINING MACHINE, KIT, AND METHODS FOR MONITORING AND CONTROLLING MINING MACHINES IN A MINING ENVIRONMENTTECHNICAL FIELD

[0001] The disclosure relates to systems and methods for monitoring and controlling operation of a mining machine in a mining site, in particular to a drilling mining machine. The disclosure further relates to a field computer device, a machine controller, a mining machine, a kit, a computer program product, and a computer- readable storage medium for monitoring and controlling operation of the mining machine.BACKGROUND

[0002] In mining, construction, tunnelling and other areas employing drilling techniques, developments are constantly underway to improve efficiency, productivity, and safety. Mining machines, e.g., drilling and various other machines, for underground and surface mining and tunneling can perform various tasks. Sometimes the machines operate in harsh subsurface environments that are dark and often inaccessible by foot and may generally be not comfortable for human drivers. Other types of challenging environments may be encountered in exploration drilling where the areas and environment may be tough and demanding. Furthermore, drilling is associated with safety concerns.

[0003] Thus, one of the leading areas in which changes / improvements are increasingly taking place is automation, full or partial, of various processes occurring in mining / tunneling. The automation requires accurate assessment of machines' condition and operation.

[0004] Various drilling machines, e.g., surface and underground drilling rigs, employ drilling tools or implements to drill or break rock and other materials. A drilling implement typically includes one or more drill rods having a drill bit at their distal end. A rod positioned at a distal end of the drilling implement is used to perform the drilling operation. Once that rod is used up i.e. becomes worn and / or broken, it may be replaced with another rod, which is typically performed manually though automatic approaches to changing drill rods also exist. Drill rods may also be changed when a rod of another diameter or having another different property needs to be used, depending e.g. on a desired properties of a hole being drilled, a type of the material being drilled, and other factors.

[0005] Monitoring, assessing, and controlling a mining operation are challenging tasks, given the properties of a mining worksite, variety of the machines used, and a multitude of situations which may occur. Accordingly, there remains a need in accurate assessment of operation and conditions of mining machines such as e.g. drilling machines.SUMMARY

[0006] In an aspect, a computer-implemented method for monitoring and controlling operation of a mining machine in a mining site is provided The mining machine comprises a drilling implement configured to have one or more rods coupled thereto. The method comprises, as the mining machine is operating in the mining site to drill at least one borehole, receiving inertial sensor measurements acquired by at least one inertial measurement unit (IMU) sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; applying a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine for the first period of time; determining at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the identified at least one operating state; and initiating an action in dependence on the determining of the at least one operation indicator. The at least one operating state may comprise a plurality of operating states.

[0007] The operation indicators may include a time per operating state, which may be used to obtain a breakdown by shift, month, year, or any other period of time. Machine utilization times may be useful for establishing proactive maintenance plans. The operation indicators may also include, for a certain machine, a total operating time representing a total time during which the machine is operational and productive e.g. performs drilling; a total non-productive time representing a time during which the machine is stopped and / or is not contributing to production; operational efficiency; unplanned or unscheduled downtime; one or more production delays, etc. The operation indicators may be determined, e.g. repeatedly, per a shift, a month, a quarter, or any other period of time, thereby a fleet performance over that period of time may be determined. The operation indicators may be represented on a user interface of a display, e.g., of a fleet control system, in the manner that allows assessing the fleet performance. The operation indicators may also include an overall equipment effectiveness (OEE) indicators determined for the machine. Real-time monitoring of the machines in the fleet may be performed, which allows making decisions during a shift, to ensure that the performance of the fleet during the shift conforms to target performance. The target performance may be defined as, e.g., one or more of a number of drilled boreholes, a combined depth of the boreholes, an amount of time that it took to drill the boreholes, etc. The target performance may be set for a shift, a day, a month, a quarter, a year, and / or any other period of time.

[0008] The technical benefits and advantages comprise automatic prediction or detection of operating states of a drilling machine, which allows improved control over use of individual mining machines and of the entire mine. In this way, overall efficiency, safety and productivity of mining operations in the mine may be increased. Furthermore, the determining of the drilling machine operating states and the operation indicators may be performed in real time, which allows implementation of timely control and intervention in fleet management systems. For a fleet of mining machines, decisions may be made on dynamic operational assignments. As a further advantage, the provided approach is equipment agnostic, such that it may be integrated with various mining machinery and equipment without requiring extensive modifications or specialized hardware. This flexibility not only simplifies implementation, but also allows for wider adoption in different mining configurations.

[0009] In some examples, the method may comprise, for a subsequent operating state from the at least one operating state, verifying a correctness of identification of the subsequent state by confirming a veracity of transition from a preceding operating state to the subsequent operating state. The preceding operating state may be an operating state that immediately precedes the subsequent operating state, and the correctness of identification of the preceding operating state may have been verified before the subsequent operating state is identified. The subsequent operating state may be a currently identified operating states which is subsequent to one or more preceding operating states including the immediately preceding operating state.

[0010] The technical benefits include identifying operating states of the mining machine with improved accuracy. The identification of the operating states corresponds to actual operations of the mining machine.

[0011] In some examples, the method may comprise, if the correctness of the identification of the subsequent operating state is not confirmed, removing the subsequent operating state from the identified at least one operating state prior to determining the at least one operation indicator.

[0012] In some examples, the set of operating states may comprise one or more of a drill motor on state, a drill motor off state, a drilling state, a standby state, a state of adding at least one rod, a state of removing at least one rod, and an inactive drilling implement state.

[0013] In some examples, the at least one identified operating state comprises a plurality of operating states. The method may comprise identifying, using the identified operating states, a drilling operation by at least determining a start time and an end time of the drilling operation performed during the first period of time. The start time of the drilling operation may be determined by determining a start time of an identified operating state that is indicative of a start of the drilling operation, and the end time of the drilling operation may be determined by determining an end time of an identified operating state that is indicative of an end or completion of the drilling operation.

[0014] In some examples, determining the at least one operation indicator comprises determining the at least one operation indicator using the identified drilling operation.

[0015] The technical benefits include determining and assessing performance metrics for the mining machine with improved precision, which ultimately allows controlling operation of the mining machine with improved accuracy

[0016] The drilling operation may also be referred to as a complete drilling operation during which a borehole is drilled to completion by the machine The start of the drilling operation may be defined by a first detected drilling state and the end of drilling operation may be defined by a detection of the state of removing at least one rod such as any of the rods coupled to the machine. The complete drilling operation may comprise one or more drilling states, one or more states of adding a rod, one or more states of removing a rod, and other states. Certain states may demarcate a start and an end of the drilling operation, e.g., the first detected drilling state may demarcate the start of the drilling operation and the state of removing at least one rod may demarcate the end or completion of the drilling operation. In some examples, the detection of drilling operation may include detecting the operating states performed after the start state and before the end state, and confirming that thesequence of the operating states between the start and end states corresponds to a valid sequence of operating states indicative of a drilling operation.

[0017] In some examples, the method may further comprise determining a number of rods used during the drilling operation In some examples, within a time period between the start and end of the drilling operation, one or more operating states may be identified that indicate that the machine has drilled one or more times, such that drilling states may be interspersed with rod addition states. These detected states may be counted, and the number of rods detected to have been added may be counted, whereby a drilling state may be associated with one or more rods. Certain metrics may be calculated for the data included in each time window of the drilling state, and the metrics may be used to improve the precision of the estimation of the drilling state. Non-limiting examples of the metrics include a state duration, magnitude of movement, drilling vibrations, and others.

[0018] In some examples, the method may comprise determining or estimating a depth of the at least one borehole drilled by the mining machine during the drilling operation. The depth of the at least one borehole may be estimated using a known length of each rod and a number of rods that have been added during the drilling operation. The number of rods is determined using the techniques in accordance with the present disclosure.

[0019] The technical benefits include estimating the depth of the borehole indirectly and with improved precision, using the inertial sensor measurements. In some cases, there may advantageously be no need for additional tools to measure the depth of the drilled borehole.

[0020] In some examples, initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of prompting a display of a representation of the identified at least one operating state and / or a representation of the at least one operation indicator; receiving an instruction to control the machine in dependence on the at least one operation indicator; providing the instruction e.g. to a mining machine controller; and adjusting a target requirement for performance of the machine during a second period of time. In some examples, the initiating the action in dependence on the determining of the at least one operation indicator may comprise controlling the machine in dependence on the at least one operation indicator.

[0021] In some examples, the action may be initiated in dependence on identifying the at least one operating state and in dependence on the determining of the at least one operation indicator.

[0022] In some examples, the action may be initiated in dependence on identifying one or more drilling operations

[0023] In an aspect, a field computer device is provided that is configured to perform the method in accordance with examples of the present disclosure. The field computer device may be communicatively coupled to a mining machine controller and a central control system configured to control mining machines in the mining site.

[0024] Advantages and effects of the field computer device are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the field computer device are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa.

[0025] In an aspect, a computer program product is provided that comprises computer-executable instructions, which, when executed by at least one processor, cause the at least one processor to perform the method in accordance with examples of the present disclosure.

[0026] Advantages and effects of the computer program product are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the computer program product are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa.

[0027] In an aspect, a tangible computer-readable storage medium is provided that has stored thereon a computer program product comprising computer-executable instructions which, when executed by at least one processor, cause the at least one processor to perform the method in accordance with examples of the present disclosure.

[0028] Advantages and effects of the computer-readable storage medium are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the computer-readable storage medium are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa.

[0029] In an aspect, a mining machine for operation in a mining site is provided, the mining machine comprising a drilling implement configured to have one or more rods coupled thereto; at least one inertial measurement unit (IMU) sensor configured to be associated with the drilling implement of the mining machine; and at least one processor and memory storing computer-executable instructions which, when executed by the at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site to drill at least one borehole, receive inertial sensor measurements acquired by the at least one IMU sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; apply a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine, for the first period of time; determine at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the identified at least one operating state; and initiate an action in dependence on the determining of the at least one operation indicator.

[0030] Advantages and effects of the mining machine are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the mining machine are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa.

[0031] In some examples, the processor and the memory may be included in a field computer device located in the mining machine or otherwise associated with the mining machine. In some examples, the processor and the memory may be included in a mining machine controller. In some examples, the processor may be distributed among the field computer device and the mining machine controller. In some examples, the memory may be distributed among the field computer device and the mining machine controller. In someexamples, the processor and the memory may be distributed among the field computer device and the mining machine controller.

[0032] In some examples, the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to, for a subsequent state from the identified at least one operating state, verify a correctness of identification of the subsequent state by confirming a veracity of transition from a preceding state to the subsequent state; and if the correctness of the identification of the subsequent state is not confirmed, remove the subsequent state from the identified at least one operating state prior to determining the at least one operation indicator.

[0033] In some examples, the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to identify, using the identified operating states, a drilling operation by determining a start time and an end time of the drilling operation performed during the first period of time. The start time of the drilling operation may be determined by determining a start time of an identified operating state that is indicative of a start of the drilling operation, and the end time of the drilling operation may be determined by determining an end time of an identified operating state that is indicative of an end or completion of the drilling operation.

[0034] The determining of the at least one operation indicator may comprise determining the at least one operation indicator using the identified drilling operation.

[0035] In some examples of the mining machine, the set of operating states may comprise one or more of a drill motor on state, a drill motor off state, a drilling state, a state of adding at least one rod, a state of removing at least one rod, and an inactive drilling implement state.

[0036] In some examples of the mining machine, initiating the action in dependence on the determining of the at least one operation indicator, and in some examples in dependence on the identifying of the at least one operating state, may comprise prompting a display of a representation of the identified at least one operating state and / or a representation of the at least one operation indicator. The representation of the identified at least one operating state and / or a representation of the at least one operation indicator may be displayed on a graphical user interface associated with the mining machine, e g., a dashboard of the machine. In some examples, the representation of the identified at least one operating state and / or a representation of the at least one operation indicator may be displayed on a graphical user interface associated with a central control system or device, or on any other graphical user interface.

[0037] In some examples of the mining machine, the initiating the action in dependence on the determining of the at least one operation indicator may comprise one or more of receiving an instruction to control the machine in dependence on the at least one operation indicator and adjusting a target requirement for performance of the machine during a second period of time. In some examples, the initiating the action in dependence on the determining of the at least one operation indicator may comprise controlling the machine in dependence on the at least one operation indicator.

[0038] In some examples of the mining machine, the IMU sensor or sensor unit comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. In such examples, the inertial sensor measurements may comprise nine-axis measurements

[0039] In some examples, the inertial sensor measurements may comprise six-axis measurements, e.g., the measurements acquired by a three-axis accelerometer and a three-axis gyroscope

[0040] In an aspect, a kit for a mining machine is provided, the mining machine being configured to operate in a mining site and comprising a drilling implement configured to have one or more rods coupled thereto. The kit comprises at least one IMU sensor configured to be associated with the drilling implement of the machine; and a computer-readable storage medium, having stored thereon computer-executable instructions which, when executed by at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site to drill at least one borehole, receive inertial sensor measurements acquired by the at least one IMU sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; apply a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine, for the first period of time; determine at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the at least one operating state; and initiate an action in dependence on the determining of the at least one operation indicator.

[0041] Advantages and effects of the kit are largely analogous to the advantages and effects of the method according to the examples herein. Further, all embodiments of the kit are applicable to and combinable with all embodiments of the method according to the examples herein, and vice versa.

[0042] In some examples of implementation of the kit, the set of operating states comprises one or more of a drill motor on state, a drill motor off state, a standby state, a drilling state, a state of adding at least one rod, a state of removing at least one rod, and an inactive drilling implement state.

[0043] In some examples of the kit, the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to, for a subsequent state from the identified at least one operating state, verify a correctness of identification of the subsequent state by confirming a veracity of transition from a preceding state to the subsequent state; and if the correctness of the identification of the subsequent state is not confirmed, remove the subsequent state from the identified at least one operating state prior to determining the at least one operation indicator.

[0044] In some examples, the at least one identified operating state comprises a plurality of operating states, and the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to identify, using the identified plurality of operating states, a drilling operation by determining a start time and an end time of the drilling operation performed during the first period of time. The start and the end of the drilling operation may be determined based on identifying respective operating states of the mining machine which are indicative of the start and end of the drilling operation. The drilling operation may involve drilling a borehole to completion.

[0045] In some examples, determining the at least one operation indicator may comprise determining the at least one operation indicator using the identified drilling operation.

[0046] In some examples, in the kit, the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to determine, using the identified operating states, a depth of the at least one borehole drilled by the mining machine during the drilling operation performed during the first period of time.

[0047] In some examples of the kit, initiating the action in dependence on the determining of the at least one operation indicator may comprise generating, on a display of a computing device, a representation of at least one of the identified plurality of operating states, the drilling operation, and the at least one operation indicator.

[0048] In some examples of the kit, initiating the action in dependence on the determining of the at least one operation indicator may comprise one or more of prompting a display of a representation of the identified at least one operating state and / or a representation of the at least one operation indicator; receiving an instruction to control the machine in dependence on the at least one operation indicator; providing the instruction e.g. to the mining machine controller; and adjusting a target requirement for performance of the machine during a second period of time.

[0049] In some examples of the kit, the initiating the action in dependence on the determining of the at least one operation indicator may comprise one or more of controlling the machine in dependence on the at least one operation indicator, and adjusting a target requirement for performance of the machine during a second period of time. The initiating of the action may comprise controlling the machine in dependence on the identified complete drilling operation.

[0050] In some examples, the action may be initiated in dependence on the identifying of the at least one operating state and in dependence on the determining of the at least one operation indicator.

[0051] In some examples of the kit, the IM U sensor comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. In some examples, the inertial sensor measurements may comprise nine-axis measurements.

[0052] In an aspect, a mining machine comprising the kit in accordance with examples of the present disclosure is provided. The kit may be operatively installed on the mining machine.

[0053] The computer device, the mining machine, the kit, and the methods that may be performed are developed and configured to operate with low computational resource requirements. This allows the provided techniques to be implemented in onboard computer devices which typically have limited processing capabilities. By using resources efficiently, the provided techniques may ensure optimal performance even in challenging computing environments. Furthermore, various events or actions may be triggered e.g. initiated in response to determining or predicting drilling machine operating states and determining operation indicators such as e.g., visualizing the operating states, drilling operation(s), and / or operation indicators, determining maintenance needs for the machine, adjusting target performance for the machine, determining a location for the machine, optimizing routes traveled by machines, defining and adjusting assignments for the machine and other machines in the mine, and others.

[0054] Additional features and advantages are disclosed in the following description, claims, and drawings. Furthermore, additional advantages will be readily apparent from the present disclosure to those skilled in the art or recognized by practicing the disclosure as described herein. There are also disclosed herein control units, computer program products, and computer-readable media associated with the above discussed technical effects and corresponding advantages.BRIEF DESCRIPTION OF THE DRAWINGS

[0055] With reference to the appended drawings, below follows a more detailed description of aspects of the disclosure cited as examples.

[0056] FIG. 1 A illustrates an example of a mining environment in which a method in accordance with examples of the present disclosure may be implemented.

[0057] FIG. 1 B illustrates an example of a portion of a drilling implement of a machine, comprising a plurality of rods.

[0058] FIG. 2 is a block diagram illustrating an example of a configuration of a system in which a method in accordance with examples of the present disclosure may be implemented.

[0059] FIG. 3 is a flowchart illustrating a method for monitoring and controlling operation of a mining machine in a mining site, in accordance with examples of the present disclosure.

[0060] FIG. 4 is a flowchart illustrating an example of a method for training a machine-learning model, in accordance with examples of the present disclosure.

[0061] FIG. 5 is a flowchart illustrating an example of a method for controlling operations of one or more mining machines in a mining site, in accordance with examples of the present disclosure.

[0062] FIG. 6A is a block diagram illustrating an example of operating states identified for a process of drilling a borehole by the mining machine.

[0063] FIG. 6B is another block diagram illustrating an example of operating states identified for a process of drilling a borehole by the mining machine.DETAILED DESCRIPTION

[0064] A system, a field computer device, a mining machine, a kit and a method are provided that implement monitoring a status of the mining machine, such as a drilling machine, and controlling operation of the drilling machine. The drilling machine comprises a moving or movable implement such as e g. a drilling tool or implement. The approach in accordance with examples of the present disclosure is agnostic to a model and brand of the mining machine comprising the drilling implement, consolidates operational information and integrates this information for use by a control system that may not need to rely on presence of an operator in the machine.

[0065] The techniques herein employ a machine-learning model for identifying one or more operating states out of a set of operating states in which the mining machine may operate, using data acquired from one or more inertial measurement unit (I MU) sensors or sensor units. The machine-learning model enables real-timedetection of operating states of one or more mining machines, and allows determining operation indicators for the machines using the operating states. The operating states determined or identified for the drilling machine may relate to operation of the drilling implement of the machine and may comprise one or more of a drill motor on state, a drill motor off state, a drilling state indicating that the drilling implement is operative or reserved, a standby state, a state of adding at least one rod, a state of removing at least one rod, and an inactive drilling implement state. The operation indicators comprise a duration of the identified at least one operating state. The operation indicators may also comprise a total operating time indicating a total time during which the machine is operational i.e. active and performs drilling. The total operating time may be calculated by summing all periods of time during which the machine is active and performs drilling. Another example of an operation indicator is a nonproductive time such as a time during which the machine is stopped or not directly contributing to production.This may include planned downtime, preventive maintenance, configuration adjustments, and any other period in which the machine is not producing. Other examples of operation indicators comprise operational efficiency of the machine, unplanned downtime of the machine, delays, and others. The operation indicators may be calculated on a shift-by-shift, month-by-month, quarter-by-quarter, or another basis and they can be used to assess performance of the machine as well as performance of other machines in the mining site. A fleet's performance over time may be assessed.

[0066] In an aspect, a computer-implemented method for monitoring and controlling operation of a mining machine in a mining site is provided. The mining machine comprises a drilling implement configured to have one or more rods coupled thereto. The one or more rods may be interconnected rods. The method may comprise, as the mining machine is operating in the mining site to drill at least one borehole and perform other operations, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; applying a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine, for the first period of time; determining at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the identified at least one operating state; and initiating an action in dependence on the determining of the at least one operation indicator.

[0067] The drill rods may be interconnected such that, once a rod used for drilling needs to be replaced, another rod may be automatically advanced or added to replace the previously used. The drilling implement may comprise a plurality of rods configured such that no intervention from an operator of the mining machine or another person may be required to change rods. In other words, an addition of a rod may be performed automatically, which may be detected using the described techniques.

[0068] The IMU sensor or sensor may be coupled to or installed on the drilling implement. In some examples, the IMU sensor may be positioned on the drill motor. In some examples, the IMU sensor may be positioned in proximity to the drill motor of the drilling implement but not on the motor. The IMU sensor may be positioned at any point on the drilling implement, at a location that allows acquiring reliable sensormeasurements that reflect a status of the drilling machine, such that an operating state of the machine may be identified with sufficient accuracy.

[0069] In some examples, the method in accordance with examples of the present disclosure may be implemented by a computing device such as e.g. the field computer device. The field computer device may be positioned in the mining machine e.g. inside an operation compartment of the mining machine, or otherwise associated with the mining machine. The field computer device may in some implementations be part of a mining machine controller such as a main controller of the mining machine.

[0070] In some aspects, the field computer device for the mining machine is provided that is configured to implement the method in accordance with examples of the present disclosure.

[0071] In some aspects, a mining machine, also referred to herein as a drilling machine, is provided in which a method in accordance with examples of the present disclosure may be implemented. The mining machine is configured and adapted to operate in a mining site or in another worksite. The mining machine may comprise a drilling implement configured to have one or more rods coupled thereto, at least one I MU sensor configured to be associated with the drilling implement of the machine; and a field computer device. The field computer device comprises at least one processor and memory storing computer-executable instructions which, when executed by the at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site to drill at least one borehole, receive inertial sensor measurements acquired by the at least one sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; apply a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine, for the first period of time; determine at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the identified at least one operating state; and initiate an action in dependence on the determining of the at least one operation indicator.

[0072] In some aspects, a kit may be provided in accordance with examples of the present disclosure. In some examples, the kit may be or may comprise a standalone sensing and processing unit or system, and it may be configured to be deployed on new or existing mining machines. Thus, a mining machine with a drilling implement can be configured to perform the method in accordance with examples of the present disclosure by installing the kit on that machine Any suitable drilling machine may be configured or reconfigured in this manner.

[0073] The kit may include one or more sensor units e.g. I MU sensor unit and a computer program product e.g. tangible computer-readable medium comprising computer-executable instructions, which, when executed by at least one processor, cause the at least one processor to perform the method in accordance with examples of the present disclosure. In some examples, the computer program product may be in the form of computerexecutable instructions stored in memory of a field computer device which may be associated with the mining machine. The IMU sensor is configured to be coupled to or installed on a mining machine, e.g., on a drilling implement of the mining machine. The computer program product may be configured to be installed on the mining machine, e.g., on at least one processor of the field computer, or a main controller of the mining machine,or on another computer device. In some cases, the computer program product may be in part or completely stored on a remote computer device.

[0074] FIG. 1 A depicts an example of a mining environment or site 10 comprising multiple, three in this example, mining machines 12a, 12b, and 12c. The mining machines 12a-12c may be of the same or different types and they each may comprise a drilling tool or implement, such that the mining machines 12a-12c may be referred to as drilling machines. Non-limiting examples of a machine out of the mining machines 12a, 12b, and 12c comprise an advance drill, a radial drill, a bolt drill, or another type of a drilling machine. In some examples, one or more of the mining machines 12a, 12b, and 12c may be exploration drilling rigs, face drill rigs, production drill rigs, rotary blasthole drill rigs, surface drill rigs, well drilling rigs, etc. The drilling equipment is used to drill holes or boreholes at drilling sites.

[0075] The mining machines 12a- 12c may be underground mining machines or surface mining machines, or they may be suitable for both underground and surface drilling. In some examples, the mining machines 12a- 12c may be autonomous machines, such as e.g., fully or partially autonomous machines. In some examples, the mining machines 12a-12c may be part of one or more mining machines in a fleet that may comprise various mining machines in addition to drilling mining machines. The mining site 10 may comprise various mining machines in addition to the mining machines 12a- 12c.

[0076] The mining site 10 may be a site of surface mining or a site of mining in an underground environment. In some examples, the mining site 10 may be any type of a mine-like underground environment. In some examples, the mining site 10 may be a subway mine, a quarry site, an open pit mine, a construction site, or another type of a mine. Although not shown in FIG. 1A, the mining site 10 may have a specific configuration. For example, if the mining site 10 is an underground mining site, it may include tunnels.

[0077] As shown FIG. 1A, each of the machines 12a- 12c may have a respective inertial sensor 14a, 14b, and 14c, such as an IMU sensor, that is coupled to a corresponding movable implement 15a, 15b, and 15c such as a drilling implement of the machine. An IMU sensor or sensor unit is a sensor that provides motion data in a time-series format. The IMU sensor comprises an accelerometer configured to acquire acceleration measurements, a gyroscope configured to acquire angular velocity measurements, and a magnetometer configured to measure a magnitude and direction of the magnetic field at a location of the magnetometer Thus, combined measurements acquired by the IMU sensor can be used to determine a position, velocity, acceleration, and orientation of an object in a three-dimensional space to which object the IMU sensor is attached. The measurements acquired by each of the accelerometer, gyroscope, and magnetometer are represented along a three-axis coordinate system. Thus, the IMU sensor is configured to provide a nine-dimensional time series data.

[0078] In some examples, more than one IMU sensor may be coupled to one or more of the mining machines 12a-12c.

[0079] In some example, one or more of the IMU sensors 14a, 14b, and 14c may be a single i.e. one IMU sensor or device. In some example, each of the IMU sensors 14a, 14b, and 14c may be one IMU sensor unit. The one or single IMU sensor is an IMU sensor that comprises respective one accelerometer, one gyroscope, and one magnetometer. The present approach allows identifying drilling machine operating states in inertialsensor measurements with precision, and inertial sensor measurements acquired by a single IM U sensor may be sufficient to identify operating states of the drilling machine and in some cases control the machine based on the identified operating states.

[0080] FIG. 1 B shows very schematically an example of a drilling implement 15 which represents any of the drilling implements 15a-15c, or any other drilling implement of a drilling machine. In some implementations, the drilling implement 15 may be coupled to a movable element 13 of the mining machine such as e.g. an arm and it may be electrically coupled to a drill motor 18. The movable element 13 is configured to allow adjustment of the position and inclination of the drilling. The drill motor 18 is configured to provide power to the drilling implement 15 so that at least a part of the drilling implement, such as a rod comprising a drill bit, is moved i.e. rotated to bore or drill a borehole in rock or another material.

[0081] As shown in FIG. 1 B, the drilling implement 15 may comprise a plurality of drill rods 17 comprising, in this example, first, second, and third drill rods 17i, 17ii, and 17iii . As shown for the first, distal rod 17i, the rods comprise a drill bit 19 that is used to drill the borehole. The rods 17 may be interconnected. In some examples, the rods 17 may be arranged such that a distal rod, such as the first rod 17i in the example of FIG. 1 B, is automatically or semi-automatically replaced by a next rod to be used for drilling such as the second rod 17ii. In this way, the second rod 17ii may be considered to be added and this rod may be counted as an added rod in the method herein. Any suitable type of a rod arrangement may be used, which may comprise any suitable number of rods. It should be appreciated that examples of the present disclosure may be agnostic to a type of the drilling implement employed by a mining machine. Also, the drilling implement may include various other components not shown herein.

[0082] As shown in FIG. 1 B, a sensor such as an IMU sensor 14, which represents any of the IMU sensors 14a-14c, may be associated with a suitable location on the drilling implement 15 to sense vibrations and other movements, or lack thereof, of the drill implement 15 comprising the drill rods 17. In some examples, the IMU sensor 14 comprises a gyroscope, an accelerometer, and a magnetometer, and the IMU sensor 14 may acquire 9-axes IMU sensor data or signals. In some examples, the IMU sensor 14 may be positioned on the drill motor 18 or in proximity to the drill motor 18.

[0083] In any case, the IMU sensor 14 is positioned close enough to the drill rods to collect IMU sensor signals or measurements which reflect status of movements of the drill rods 17 and other parts of the drilling implement 15. In this way, the IMU sensor measurements may be informative of an operating or operational state of the drilling machine. The specific position of the IMU sensor on the drilling implement may depend on a configuration of the drilling implement, type and configuration of the mining machine, work environment conditions, a type of the material to be drilled, feasibility of installation of the IMU sensor, and / or on other factors. The IMU sensor is installed on the drilling machine at a position and in the manner such that acquisition of IMU sensor measurements by the IMU sensor is not affected and the acquired IMU sensor measurements allow detection of operating states with desired speed and accuracy.

[0084] It should be noted that the drilling implement 15 is shown by way of example only. Examples of the present disclosure may be agnostic to a type of the drilling implement employed by a mining machine. Also, the drilling implement may include various other components not shown herein.

[0085] As further shown in FIG. 1A, each of the machines 12a-12c may comprise or may be associated with a respective control unit or a field computer device 16a, 16b, and 16c that may be configured to perform the method in accordance with examples of the present disclosure. Each of the field computer devices 16a- 16c includes various components not shown in this example, such as a hardware memory device configured to store computer-executable instructions and processing circuitry e.g. one or more hardware processors. The processors are configured to execute the computer-executable instructions which causes the processing circuitry to perform the method in accordance with examples of the present disclosure, as discussed in more detail below. Each of the field computer devices 16a-16c also has an input and output interface that is configured to communicate with other components, e.g., with the respective IMU sensor, a mining machine controller, as well as with a fleet controller or central control system 22 and with other external systems.

[0086] One or more of the mining machines 12a-12c may be operated in the mining site 10 to drill boreholes in rock or other substrate. The mining machine may be in one or more of a set of operating states which comprises one or more of a drill motor on state, a drill motor off state, a drilling state, a state of replacing at least one rod, a state of adding at least one rod, a state of removing at least one rod, an inactive drilling implement state, and a standby state such as e.g. one or more of the drill motor off state and the inactive drilling implement state. In the drill motor on state, the drill motor is turned on and operating but the drilling implement may not necessarily be moving. In the drill motor off state, the drill motor is not operating i.e. it is turned off. In the drilling state, the drill motor is turned on and the motor is driving the drill bit to drill a borehole. In the state of replacing at least one rod, the drilling implement may be controlled, e.g. automatically, to replace a rod which involves removing a rod and adding another rod. Adding the rod involves coupling the rod being added with another rod. Replacing the rod involves uncoupling of the rod from the machine and coupling of a new rod instead of the uncoupled rod The state of removing at least one rod may involve removing the at least one rod In some examples, the state of replacing at least one rod may be detected as a state of removing a rod and a state of adding a rod. The one or more rods may be manipulated automatically and a state of the rod may be detected using the IMU sensor.

[0087] In the inactive drilling implement state, the drilling implement may be inactive i.e. not used for drilling.

[0088] The drilling machine may be in one or more of additional or alternative operating states. For example, the machine may be moving from one location to another. As the drilling machine is moving to another location, e.g. to drill holes at that location, the machine may be considered to be in the inactive drilling implement state. Furthermore, different operating states may be considered depending on a configuration of the mining machine and / or its drilling implement, e.g., the manner in which rods are changed for proper drilling operation.

[0089] In some examples, a complete drilling operation performed by a drilling machine may be identified based on identifying operating states of the drilling machine. The complete drilling operation involves drilling a borehole to completion e.g. to a desired depth.

[0090] In some examples, a kit 20 may be installed and deployed on a mining machine out of the plurality of mining machines 12a- 12c in the mining site 10 The kit may comprise one or more I MU sensors and a computer program product e.g. tangible computer-readable medium comprising computer-executable instructions, which, when executed by at least one processor, cause the at least one processor to perform the method in accordance with examples of the present disclosure. In some examples, the computer program product may be in the form of computer-executable instructions stored in memory of a field computer device. Thus, in some examples, the kit may comprise the one or more I MU sensors and the field computer device. In some examples, the kit comprises one IMU sensor and a field computer device. The kit may be installed on the mining machine or otherwise associated with the mining machine to be used in conjunction with the machine that may be operating in the mining site.

[0091] FIG. 1A illustrates schematically, for the mining machines 12b and 12c, that the mining machines may comprise respective kits 20b and 20c, shown by a dot-dashed line. The kit may be operatively installed on the mining machine, to be used in conjunction with the machine. The kit 20b may comprise the IMU sensor 14b and a computer program product (not shown separately) that may be installed on the field computer device 16b. Similarly, the kit 20c may comprise the IMU sensor 14c and a computer program product (not shown separately) that may be installed on the field computer device 16c.

[0092] As shown in FIG. 1A, operation of mining machines in the mining site 10, shown by way of example as the mining machines 12a-12c, may be controlled by the central control system 22 such as e.g. a fleet controller or fleet control system. The central control system 22, which is communicatively coupled with one or more of the mining machines 12a-12c, may track locations or positions of each of the mining machines 12a-12c, may receive data from the mining machines, and may send commands or instructions to the mining machines. The fleet control system may also be referred to as a traffic control system, and it may be any type of a control system that monitors multiple machines e.g. receives information from the mining machines, and sends commands or instructions to the mining machines. The central control system 22 comprises processing circuitry 24 that is configured to execute computer-executable instructions which thereby cause the processing circuitry 24 to perform a method for controlling operation of one or more mining machines, in accordance with examples of the present disclosure. The central control system 22 may have various other components not shown herein for simplicity, e.g., memory, user interface, communication interface, etc.

[0093] FIG. 2 illustrates an example of a system 100 in which example embodiments of the present disclosure may be implemented. The system 100 may be employed, at least in part, in the mining environment or site such as e.g. mining site 10 shown schematically in FIG. 1A. The system 100 may comprise one or more mining machines, and a mining machine 12 is shown in FIG. 2 as a representative mining machine. The system 100 may also comprise a central controller or control system 22 such as, e.g. a traffic control system which may be a fleet controller configured to perform coordinated control of machines in the mining site. It should be notedhowever that the central control system 22 may be remote and it may be positioned outside of the mining site 10. The mining machines controlled by the control system 22 may be drilling machines as well as various other mining machines, e.g , load-haul-dump (LHD) trucks, crushers, draglines, shovels, etc.

[0094] As shown in FIG. 2, the mining vehicle 12 may comprise a main control system or mining machine controller 28 and a field computer device 16 which may be configured to perform the method in accordance with examples of the present disclosure. The field computer device 16 may be in operable communication with the mining machine controller 28. In some examples, the field computer device 16 may be part of the mining machine controller 28.

[0095] The mining machine controller 28 may be located onboard the machine 12. The machine controller 28 may be a main controller of the mining machine 12 which is configured to control operations of the mining machine 12. The machine controller 28 may comprise processing circuitry 32 such as one or more processors and memory 34 which may comprise one or more memory units. The memory 34 comprises computerexecutable instructions executable by the processing circuitry 32 of the machine controller 28. The memory 34 may be configured to store information, data, etc., and the computer-executable instructions to perform, when executed by the processing circuitry 32, various processes related to monitoring and controlling operation of the mining machine 12. In some examples, the mining machine controller 28 may receive data and / or control signals from the field computer device 16, and the data and / or control signals may comprise an instruction related to operation of the mining machine 12. The machine controller 28 may comprise or may be associated with various other components not shown herein.

[0096] The field computer device 16 may be adapted to execute instructions from a computer-readable medium to perform the functions or processes described herein. The field computer device 16 may be connected (e.g., networked) to other machines in a local area network (LAN), an intranet, an extranet, or the Internet. The field computer device 16 may comprise processing circuitry 210 and a hardware memory device or memory 220 which may comprise one or more memory units. The memory 220 comprises computer-executable instructions which may be executed by the processing circuitry 210 to cause the processing circuitry 210 to perform the method in accordance with examples of the present disclosure.

[0097] The memory 220 may store various data related to the method in accordance with examples of the present disclosure. As shown in FIG. 2, the memory 220 may store inertial sensor measurements 221 which may be acquired by the IMU sensor unit 14 also shown in FIG. 2. Also, in some cases, the inertial sensor measurements may be sent to the machine controller 28 and / or to an external system, e.g , to control system 22 and / or to another system.

[0098] The memory 220 may comprise an operating states registry 222 storing one or more operating states predicted or identified for the mining machine 12. The operating states registry 222 may store one or more variables for each operating state, which variable may be associated with one or more parameters 224 stored in association with that operating state. The one or more parameters related to the operating state may be one or more features used to predict the operating state, a duration of time of the operating state, a start time and an end time of the operating state, and other parameters.

[0099] The memory 220 may also comprise a machine-learning (ML) model unit 228 that stores at least one trained machine-learning model as well as various information associated with the trained ML model e.g. training data used to train the ML model, extracted / constructed and / or selected features, and other information The ML model unit 228 may comprise one or more subunits or modules, e.g., exploratory data analysis (EDA) unit, a feature construction and selection unit, a model building unit, a model evaluation unit, and other units or subunits.

[0100] The memory 220 may comprise one or more operation indicators unit or registry 232 that may store the one or more operation indicators 234 that may be determined from operating states predicted or identified for the mining machine. Non-limiting examples of the operation indicators 234 comprise a duration of a certain operating state, a total operating time, a total non-productive time representing a time during which the machine is stopped and / or is not contributing to production, operational efficiency, unplanned downtime, one or more production delays, and various other productivity indicators etc. The operation indicators may also comprise a state time of drilling i.e. a first drilling state, an end time of drilling i.e. a last detection of a rod removal, a number of rods used during a certain period of time, estimated drilling depth, etc. The operation indicators 234 may be stored in association with other information such as e.g. an operator identifier identifying an operator of the mining machine 12, a machine identifier identifying the mining machine 12, a type of equipment e.g. drilling equipment used in the mining machine, and others.

[0101] As shown in FIG. 2, the processing circuitry 210 may comprise a ML model execution unit 240 that is configured to execute the trained ML model stored in the ML model unit 228. The trained ML model may be executed by the processing circuitry 210 to perform the method in accordance with examples of the present disclosure. The processing circuitry 210 may also comprise model retraining unit 230 that allows the processing circuitry 210 to automatically train and retrain the ML model for identifying operating states of the mining machine with the drilling implement. In some examples, the same module or unit may perform ML model processing and retraining, e.g. the ML model execution unit 240. The processing circuitry 210 may include various other modules and / units configured to perform actions and methods in accordance with examples herein, as well as other actions and / or methods.

[0102] The processing circuitry 210 may include any number of hardware components for conducting data or signal processing or for executing computer code stored in the memory 220. The processing circuitry 210 may include a general-purpose processor, an application specific processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a circuit containing processing components, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein.

[0103] The memory 220 may be one or more devices for storing data and / or computer code for completing or facilitating methods described herein. The memory 220 may comprise random access memory (RAM), readonly memory (ROM), erasable programmable read-only memory (EPROM), hard drive storage, temporary storage, non-volatile memory, flash memory, optical memory, or any other suitable memory for storing software objects and / or computer instructions. The memory 220 may includedatabase components, object code components, script components, and / or any other type of information structures for supporting the various processes and information structures described in the present disclosure. The memory 220 may be communicably connected to the processing circuitry 210, e.g., via a circuit or any other wired or wireless connection.

[0104] The field computer device 16 may also include a communications interface 242 that may include wired and / or wireless communications interfaces, e.g., jacks, antennas, transmitters, receivers, transceivers, wire terminals, etc., for conducting data communications external systems or devices. In various examples, the communications may be direct, e.g., local wired or wireless communications, or via a communications network, e.g., a WAN. the internet, a cellular network, etc. The field computer device 16 may include various other components not shown herein.

[0105] It should be appreciated that any of the data and computer-executable instructions stored in the memory 220 may be loaded onto the processing circuity 210 or used by the processing circuitry 210, to perform the method in accordance with examples of the present disclosure.

[0106] Those skilled in the art will appreciate that the units of the field computer device 16 described herein may refer to a combination of analogue and digital circuits, and / or one or more processors configured with software and / or firmware, e.g., stored in the field computer device 16, that, when executed by respective one or more processors, may perform the methods in accordance with examples of the present disclosure. One or more of these processors, as well as the other digital hardware, may be included in a single Application-Specific Integrated Circuitry (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a system-on-a-chip.

[0107] It should be appreciated that the field computer device 16, the machine controller 28, and the mining machine 12 may comprise various other components not shown in FIG. 2 for the sake of simplicity. For example, the mining machine 12 may comprise an onboard display which, in some examples, may be configured to display a representation of the identified one or more operating states, one or more operation indicators, as well as other information. The field computer device 16 may comprise a communication interface for communication with the I MU sensor unit 14 to receive one or more inertial sensor measurements, and for communication with remote systems such as e.g. the control system 22.

[0108] FIG. 2 illustrates that the central control system 22 comprises processing circuitry 24 and memory 260. The central control system 22 may be a central controller or fleet controller that may be positioned remotely from a plurality of mining machines operating in the mining site. The mining machines include one or more drill mining machines. In some examples, the central control system 22 may be positioned in the mining environment or site, e.g., it may be part of or associated with one of the mining machines in the mining site. Regardless of its specific implementation and location, the central control system 22 is configured to receive information from and to send information and control commands to one or more mining machines out of the plurality of mining machines in the mining site. The central control system 22 may be configured to control, in a coordinated manner, movements and / or other functions of the mining machines. The mining machines may be fullyautonomous, semi-autonomous, or manually controlled machines, and various type of signals and instructions may be received by the machines from the control system 22.

[0109] The memory 260 may store computer-executable instructions that can be executed by the processing circuitry 24 to cause the processing circuitry 24 to perform monitoring and controlling of operations of the mining machine 12. The memory 260 of the central control system 22 may receive various information regarding the mining machine 12 e.g. from the field computer device 16 and / or the machine controller 28, such as one or more operating states, one or more operator indicators, and various other information. Based on the received information, the control system 22 may generate and send commands to the mining machine 12, as well as to one or more of other mining machines in the mining site.

[0110] As also shown in FIG. 2, the central control system 22 may comprise and / or may be communicatively coupled to a display (not shown) that is configured to render a graphical user interface 280. The user interface 280 may display various information related to mining machines controlled via the control system 22. The user interface 280 may also be configured to receive user input e.g. with respect to the displayed information. The control system 22 may include an input and output device interface (not shown) such as e.g. a circuit for controlling input and output from and to peripheral devices including devices such as a mouse, a keyboard, joystick, touch-sensitive surface or pad, touch-sensitive screen, etc.

[0111] The user interface 280 of the control system 22 may be configured to present various information based on the identified operating states and the operation indicators. The information allows visualizing and assessing a status of mining machines in the fleet including one or more drill machines, as well as a status of the entire fleet. For example, at any point in time, operating states of each of the mining machines currently located in the mining site may be visualized. A representation of the machines in the mining site by operating state may also be visualized such that it is possible to assess a number of machines currently determined to be in a certain operating state. It may be, for example, visualized how many mining machines are currently operative, delayed, reserved, or out of service. The one or more representations of the status of the individual or groups of the machines, and of the entire fleet, may be generated and displayed in real time, such that a real time monitoring, assessment, and control of the fleet of mining machines may be performed. In this way, a user such as e.g. a fleet operator or mine coordinator may be provided with information that can be used to assess the status of the mining machines and to make decisions regarding operation of the mining machines. The information may in some cases be analyzed automatically. Maintenance decisions, machine repositioning, task or job assignments, and other actions may be performed using the operating states and the operation indicators identified or predicted in accordance with examples of the present disclosure.

[0112] Instructions or commands may be generated by the control system 22 and sent to the mining machine 12 and / or other machines, instructing the machines to initiate actions related to maintenance, repositioning, task or job assignments, and other types of actions. The instructions or commands may be sent to the field computer device 16 and / or the mining machine controller 28. In some examples, the field computer device 16 may receive an instruction to control the machine in dependence on at least one operation indicator identified for the machine and / or provide the instruction to the machine e.g. to the mining machine controller 28.

[0113] The approach described herein, in accordance with examples of the present disclosure, may comprise two or more workflows also referred to as pipelines, which in turn may comprise multiple stages. One of the pipelines is a training pipeline, which is developed using historical or batch data, and includes pre-analysis stages such as e.g sensor description and position, exploratory data analysis (EDA), and modeling stages such as data capture and processing, feature extraction, modeling, and validation. Another pipeline is a production pipeline, which is developed with data generated in real time, as the mining machine is operating in a mining environment or site, and includes such stages as e.g, queuing data, preprocessing, feature extraction, generating predictions, performing validation logic which includes confirming veracity of a state transition, determining operation indicators, and generating results representation. Other pipelines may be implemented as well.

[0114] FIG. 3 shows an example of a computer-implemented method or process 300 for monitoring and controlling operation of a mining machine in a mining site, the mining machine comprising a drilling implement configured to have one or more rods coupled thereto. The method or process 300 may be performed by a computer device, e.g. the field computer device 16 or another suitable computer device which may be positioned in the mining machine or may be otherwise associated with the mining machine including remotely. In some examples, the method or process 300 may be performed by one or both the field computer device 16 and / or the mining machine controller 28. The actions at blocks of FIG. 3 do not have to be taken in the order stated below, but may be taken in any suitable order. Dashed boxes indicate optional features.

[0115] At block 302, the method 300 comprises, as the mining machine is operating in the mining site to drill at least one borehole and / or to perform other operations, receiving inertial sensor measurements acquired by at least one IMU sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time. In some examples, the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, and the received sensor measurements may comprise sensor data in 9 axes. Thus, patterns indicative of drilling machine operating states of the mining machine may be detected in the 9-axis IMU sensor data In some examples, during the process for training and optimizing the machine-learning model, an importance of each of the axes and their features in the model may be analyzed, along with determining how data from these axes contributes to the inference. Thus, the ML model can be trained with all the features extracted from the 9 axes of the IMU sensor data, or with some of the features.

[0116] The IMU sensor or sensor unit may be coupled to the drilling implement For example, the IMU sensor may be coupled in proximity to a drill motor of the mining machine The IMU sensor may be attached to the drilling implement at any suitable location, so that sensor measurements acquired by the IMU sensor reliably reflect a status or state of the drilling implement and / or the drilling machine, which may be defined as an operating state in accordance with examples herein.

[0117] The first period of time may be an hour, a shift comprising several hours, more than one shifts, a day i.e. 24 hours, a month, a quarter, or any other period of time.

[0118] At block 304, the method 300 comprises applying a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of the set of operating states of the miningmachine, for the first period of time. In examples herein, the operating states in the set of operating states relate to drilling performed by the machine, though the mining machine may be inactive at certain periods, which periods are also automatically detected using the described approach

[0119] The machine-learning model is a model that has been trained and optimized to recognize in the inertial sensor measurements patterns that correspond to operating states from the set of operating states.

[0120] At a given time, the mining machine may operate in an operating state out of the set of the possible operating states, and that operating state may be identified by the present method by applying the machinelearning model that has been trained to identify one or more patterns in the inertial sensor measurements that are indicative of the certain state. More than one operating state, such as a plurality of operating states may be identified for or during the first period of time. During the first period of time, the mining machine may operate in various operating states. One or more drilling operations, also referred to as complete drilling operations, may be performed by the mining machine during the first period of time.

[0121] In some examples, the operating states in the set may comprise one or more of a drill motor on state, a drill motor off state, a drilling state, a state of adding at least one rod, a state of removing at least one rod, an inactive drilling implement state, and a standby state. The set of the operating states that the machinelearning is trained to recognize may comprise other types of operating states related to drilling.

[0122] One or both the drill motor off state and the inactive drilling implement state may be referred to as a standby state in which the mining machine is not performing drilling. The standby state may occur between complete drilling operations. Furthermore, in some examples, the standby state may be detected as the machine stops drilling during a drilling operation and before the drilling operation is completed.

[0123] In some examples, the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope and a three-axis magnetometer. The inertial sensor measurements may thus comprise data acquired in the nine corresponding axes, three per each of the accelerometer, gyroscope, and magnetometer. Data or signal from each axis may undergo computer-implemented processing to generate new variables, e.g., one or more of signal enveloping, low-pass or high-pass filtering to eliminate noise, and moving average filtering to smooth the signal may be applied to the data. Feature extraction may be performed using these new variables.

[0124] In some examples, the trained machine-learning model may be agnostic to one or more out of a model of the mining machine, a brand of the mining machine, and a manufacturer of the mining machine. The machine-learning model may be trained using training data that is obtained by one or more IMU sensor units coupled externally to drilling implements of one or more mining machines The training data may include simulated data. The machine-learning model may be trained without use of data which may be obtained from measuring values of parameters related to operation of internal components of the mining machine. Furthermore, the machine-learning model may be trained without use of imaging data and without use of positioning data. Although a position of each vehicle in a mining site is tracked, this information, e.g., acquired by a position sensor such as a Global Positioning System (GPS) tracker and / or a radio frequency identification (RFID) device, may not need to be used for the purposes of the present disclosure. Thus, in use, when the trained machinelearning model is applied to inertial sensor measurements acquired by the IMU sensor from the mining machineduring its actual operation in the mine, it may be sufficient to use the inertial sensor measurements acquired only by an IMU sensor that is coupled to the drilling implement and / or the drilling motor of the mining machine, to predict or identify or determine, in real-time, an operating state of the mining machine based on detected movements of the drilling implement.

[0125] In some examples, the trained machine-learning model may be trained in dependency on at least one property of the worksite e.g. mining environment. For example, in some cases, the machine-learning model may be retrained and / or adjusted based on properties or characteristics of the mining environment or site which may include e.g. a layout of the mining environment, distances to be traveled by the mining machine, properties of rock or other material that may be drilled, and other properties that may affect detection of operating states for mining machines in the worksite. It should be noted that the worksite may comprise a mining environment or any other site where drilling operations may be performed by one or more mining machines. The trained machinelearning model may be trained to recognize patterns in the IMU sensor data that correspond to operating states. The trained machine-learning model may be trained to apply a certain time window to the IMU sensor data to identify a pattern within that time window that is indicative of the corresponding operating state.

[0126] In some examples, the machine-learning model may be trained as described in connection with FIG. 4. In some examples, a training pipeline, described in more detail in connection with FIG. 4, may be executed automatically, thereby a trained machine-learning model may be generated that is suitable for properties of the mining environment and takes into consideration other factors, conditions, and properties. Thus, in some examples, the machine-learning model may be generated and trained as part of the processing at block 304.

[0127] Non-limiting examples of features that may be extracted from the inertial sensor measurements comprise, for a certain period of time, a mean, a standard deviation, a slope, a median, a maximum, a minimum, polynomial coefficients, kurtosis, skewness, and frequency response.

[0128] In some examples, the trained machine-learning model is trained in dependency on one or more features selected based on at least one property of the mining site and the position of the IMU sensor on the machine. The machine-learning model may be trained in a training pipeline which comprises a first stage of processing the data to eliminate noise and improve the quality of signals. In this stage, for example, low or high frequency filtering processing and envelope calculations may be applied. Then this data may be labeled according to operating states that are to be detected, and the model is trained and optimized One or more ML models may be trained using supervised learning, e.g., artificial neural networks, decision trees, or a mixture of neural networks and decision trees. An artificial neural network can be implemented in a variety of ways, such as e.g. a simple neural network, a recurrent neural network, a bidirectional recurrent neural network, a convolutional neural network, a deep convolutional neural network, or one or more of other types of neural networks. An artificial neural network may also be implemented using other machine-learning algorithms, such as e.g. a support vector machine, a random forest of decision trees, linear regression, logistic regression, Naive Bayes, k- nearest neighbors (kNN) algorithm, or K-Means.

[0129] At block 306, the method 300 may optionally comprise, for an identified operating state which may be referred to as a subsequent state which follows a preceding operating state, determining whether the state has been identified correctly. In some examples, the method 300 may comprise, for a subsequent state from the identified operating states, verifying a correctness of identification of the subsequent state by confirming a veracity of transition from a preceding operating state to the subsequent operating state The veracity of the transition may be confirmed based on timing and / or based on a type of the operating state such that the machine may transition from certain states to others.

[0130] It should be appreciated that, as used herein, a currently identified operating state may interchangeably be referred to as a subsequent operating state. Thus, each operating state may first be referred to as a subsequent operating state until it is confirmed that this state is correctly identified based on previously identified states e.g. based on transition from the preceding state to the subsequent, currently identified operating state. Once the correctness of the identification of the state is confirmed, it may be referred to as the current operating state. Information on the confirmed current operating state may be recorded e.g. in the operating state registry 222 and / or in another suitable storage.

[0131] In some examples, to confirm a veracity of transition from the preceding operating state to the subsequent operating state, the field computer device may acquire a threshold transition time which may be stored e.g. in the form of a transition matrix. The transition matrix, an example of which is shown in Table 1 , may store threshold transition times for transitions from one operating state to another. In the example of Table 1 , the transitions are shown from the preceding operating state to a predicted subsequent operating state. The preceding operating state may be a verified operating state for the mining machine, and the predicted subsequent operating state may be a newly identified or predicted or determined operating state, e.g., based on real-time data acquired using the I MU sensor unit coupled to the machine. For example, as shown in Table 1, times f, with I ranging from 1 to 6 in this example (with three "zero” entries), are stored for transition between three preceding operating states to three subsequent operating states. A transition matrix may include times for transition to and from any suitable types of operating states in which the mining machine with a drilling implement may operate.

[0132] Table 1.

[0133] The transition matrix shown in Table 1 specifies a threshold transition time that allows confirming veracity of the identified transition from one operating state to another. For example, for State 1, a threshold transition time within which the machine may transition from State 1 to State 2 is Time 1 , and a threshold transition time within which the machine may transition from State 2 to State 1 is Time 3. The threshold transition time may be expressed e.g. in seconds, minutes, or other time units. The States 1, 2, and 3. shown by way of example in Table 1, may be any of the operating states that may be identified for the drilling mining machine. Asused herein, the threshold transition time may be defined as a time that the field computer device waits to confirm and generate a prediction of a new detected operating state. As an example, the drilling machine may be in standby, then it begins to drill a well and the “drilling” state is detected; however, to reduce false positives, the field computer device waits a transition time t (e.g. 10 seconds, only as an example) If no different state is detected, such that a status of the drilling machine is not detected to have chanted during this waiting time, the detected “drilling” operating state is confirmed and recorded or stored. Thus, in some examples, this threshold may represent a minimum duration of the detected state; however, the duration of the state may be greater than the threshold.

[0134] The correctness of the identification of the subsequent operating state may be not confirmed if, for example, a time that it took the mining machine to transition from State 1 to State 2 deviates from Time 1. The correctness of the identification of the subsequent operating state may be confirmed if, for example, the time that it took the mining machine to transition from State 1 to State 2 is Time 1 or does not deviate from Time 1 to a degree that is larger than a certain allowed deviation. In some examples, a margin may be used for the threshold transition time indicating an expected time for transition from the preceding operating state to the predicted subsequent operating state. The margin for the threshold transition time may indicate an allowed deviation from the threshold transition time for the transition from the preceding operating state to the preceding operating state. The margin may define a level of certainty required to confirm a change for the mining machine from one operating state to another.

[0135] Furthermore, for some states, a veracity of transition from the preceding operating state may not be determined or confirmed. For example, a veracity of transition from the preceding operating state may not be determined or confirmed for a state that is identified for the mining machine, e.g., a first operating state in a shift or another type of state after the machine is in standby or in another condition during which operating state detection may not be performed. The correctness of identification of such operating state may be confirmed in other ways. The veracity of transition from the preceding operating state may not be determined for a first drilling event such as a start of a drilling operation, where the machine may have not been drilling before that e g. it could have been in standby. For consistency, such state may still be referred to as a subsequent operating state, even though there may be no preceding state detected before this state, and / or any preceding operating state(s) may be detected for the same mining machine a certain time ago such that a transition matrix would not be applied.

[0136] At decision block 308, it may be determined whether the correctness of identification of the subsequent operating state has been verified. Responsive to determining that the correctness of identification of the subsequent operating state has not been verified, e.g. that the veracity of the transition from the preceding operating state to the subsequent operating state is not confirmed, the method 300 may in some cases return to block 302 where further inertial sensor measurements may be acquired. In some cases, however, the processing may continue to block 309.

[0137] As shown at block 309, in some examples, the method 300 may comprise, if the correctness of the identification of the subsequent state is not confirmed, removing the subsequent state from the identified at leastone operating state prior to determining the at least one operation indicator. Thus, in some examples, a detected or identified operating state may be excluded from further analysis if it is identified incorrectly.

[0138] At block 310, the method 300 comprises identifying, using the identified operating states, a drilling operation by determining a start time and an end time of the drilling operation performed within the first period of time. The at least one operating state, identified in accordance with the described techniques, may comprise a plurality of operating states such as a certain number of operating states.

[0139] The identifying of the drilling operation may include identifying one or more drilling states, as well as other states, between a start operating state and an end operating state. The start and end states are operating states that are indicative of respective start and end of the drilling operation. The drilling operation may encompass one or more drilling states, as well as other states, between the start and the end states. The detection of the drilling operation and related parameters of operating states that form the drilling operation allows determining and assessing performance metrics for the mining machine with improved precision, and to ultimately control operation of the mining machine with improved accuracy.

[0140] The determining of the drilling operation comprises identifying an operating state that is indicative of the start of the drilling operation and identifying another operating state that is indicative of the end of the drilling operation. For example, a first drilling state identified e.g. after a standby state of the mining machine, may be indicative of the start of the drilling operation. A last state of removing of at least one rod used for drilling may be indicative of the end of the drilling operation.

[0141] The start time of the drilling operation may be determined by determining a start time of the identified operating state that is indicative of the start of the drilling operation, and the end time of the drilling operation may be determined by determining an end time of the identified operating state that is indicative of the end or completion of the drilling operation.

[0142] FIG. 6A illustrates an example of a plurality of operating states in which have been identified for a mining machine e.g. drilling machine 12 for the first period of time, represented by an arrow line 601. As shown in FIG. 6A, the a plurality of operating states comprise a standby, a state of drilling using a first rod, a state of adding a second rod, a state of drilling using the second rod, a state of adding a third rod, a state of drilling using the third rod, a state of removing all three rods, and a standby state. As also shown in FIG 6A, a complete drilling operation may be identified to be performed by the mining machine during the first period of time, the drilling operating comprising in this example the state of drilling using the first rod, the state of adding the second rod, the state of drilling using the second rod, the state of adding the third rod, a state of drilling using the third rod, and the state of removing all three rods. Thus, the detection of the state of drilling using the first rod is used to identify the start of the drilling operation, and the detection of the state of removing all three rods is used to identify the end of the drilling operation. In some examples, there may be a state of removing a rod during the drilling operation. The drilling machine may be in a standby state before and after the drilling operation.

[0143] In some examples, the first drilling state may be identified when the performance of the method 300 begins, e.g., once the field computer device 16 is activated to begin acquiring inertial sensor measurements from the IMU sensor coupled to the drilling implement or the drill motor and identifies, using the acquired sensormeasurements, that the mining machine has begun drilling. As another option, the first drilling state may be identified after one or more operating states are identified for the mining machine, which state(s) indicate a period of inactivity of the mining machine, e.g., between shifts, after maintenance, or in other circumstances.

[0144] The end time of the drilling operation may be determined by detecting or identifying a last detection of rod removal, i.e. the detection of the state of removing a rod which is followed by one or more of the state in which the drilling implement is inactive, the drill motor off state, or a period of inactivity of the mining machine. In some examples, the operating state of removing at least one rod is used to determine the end of the drilling operation, since it is a process or state that generally occurs when a drilling is completed. A state of removing of at least one rod may be identified as being indicative of the end of the drilling operation when the removal of the at least rod is followed by a standby operating state or another operating state during which the drilling implement is not active. More than one drilling state may be identified to be performed during the first period of time.

[0145] At block 312, the method 300 optionally comprises determining or estimating, using the identified at least one operating state such as a plurality of operating states, a depth of the at least one borehole drilled by the mining machine during the drilling operation. More than one borehole may be drilled during the first period of time, and a depth of each of the boreholes may be automatically estimated. This may comprise determining or estimating a number of rods used during the drilling operation. The depth of the borehole may be estimated based on a known length of each rod and the number of rods that have been used during the drilling operation performed during the first period of time. In some examples, by determining a duration of time during which the mining machine has been in the operating state of adding at least one rod, while the machine’s location does not change, it may be determined that the same borehole was being drilled.

[0146] In some examples, the drilling machine may not be turned off while it is used to drill the same hole. In some examples, the drilling operation may include a standby operating state. FIG. 6B illustrates an example of operating states identified for a process of drilling a borehole by a mining machine. A sequence of the identified operating states is illustrated in the order which corresponds to the respective operations performed by the mining machine. In FIG. 6B, each block corresponds to an operating state detected in accordance with the method described herein, the duration of which can vary

[0147] As shown in FIG. 6B, a complete drilling operation may include the following states: drilling with a first rod, adding a second rod, drilling with the second rod, adding a third rod, drilling with the third rod, a standby, another drilling with the third rod, adding a fourth rod, drilling with the fourth rod, and removing all rods. A start of the drilling is defined as a first drilling event detected in a time window, and an end of the drilling is defined by detecting the rod removal maneuver. In this non-limiting example, the drilling uses four rods, where there are three operating states of adding rods and five operating states of drilling. As the third rod is being used by the machine, the drilling is interrupted by a period of inactivity, which causes an additional drilling state to be detected; however, this period does not include adding a rod.

[0148] There may be a relationship between the addition of rods, the drilling states and the total number of rods. This relationship may be used to validate, filter, and improve the precision of the estimation of the operatingstates and the drilling operation encompassing certain operating states. In a drilling operation, uninterrupted states may encompass N + 1 drilling states, where N corresponds to the rod aggregate states. This is because drilling begins with a rod, already installed in the drilling implement e.g. an arm of the machine, therefore the first event or state of rod addition may not exist.

[0149] At block 314, the method 300 comprises determining at least one operation indicator from the identified at least one operating state. The at least one operation indicator comprises a duration of the identified at least one operating state. The operation indicator may be or may indicate a duration of each of the identified operating states by shift, month, year, or any other time period. The operation indicators may comprise any suitable one or more performance metrics related to the one or more operating states, the metrics indicating a status, a need for service, and other characteristics of the drilling implement and / or the mining machine.

[0150] In mining, machine utilization times may be determined to automatically generate indicators indicating a need and timing for maintenance and / or servicing of the machine. In some examples, the operation indicators may comprise a total operating time representing a total time during which the mining machine is operational and productive e.g. performs drilling. The total operating time may be determined by summing all periods of time in which the mining machine is active and has been identified as generating value.

[0151] In some examples, the operation indicators may comprise a use of the mining machine per a certain duration of time. For example, a use U of the mining machine per shift may be calculated by dividing the total operating time by a duration of the shift:

[0153] A use of the mining machine during any other periods of time may also be determined.

[0154] In some examples, the operation indicators may comprise an overall operational efficiency E of the mining machine, which may be calculated by dividing the total operating time by a total available time indicating a total duration of time during which the mining machine is or was available. It shows what percentage of the available time is used for production such as drilling:„ Total Operating Time „

[0155] E = - - - - - x 100%Total time available (2)

[0156] In some examples, the operation indicators may comprise an unplanned or unscheduled downtime indicating a time during which the mining machine is down due to unplanned failures. This indicator can allow identifying sources of faults that may occur in the mining machine. Thus, it may be automatically determined, based on identifying one or more operating states indicative of inactivity of the mining machine, that the mining machine was not performing drilling in accordance with target requirements set for the machine.

[0157] In some examples, the operation indicators may comprise unexpected delays during production, such as one or more of a lack of a work resource, quality problems, or unforeseen interruptions. Identifying and reducing such delays can improve efficiency. The work resource may be water and electricity which the drilling machine requires for operation. These resources may in some cases not be immediately available, which may lead to delays in the mining machine operation. Another type of the work resource may be rods and / or drilling bits which may not be always available.

[0158] The operation indicators may be determined per shift, month, quarter, or per any other period of time, thereby performance of the mining machine, as well as performance of other machines in the mine and / or an overall fleet performance over that period of time may be determined

[0159] The operation indicators may be represented on a user interface of a display, e.g., of the mining machine and / or a fleet control system, in the manner that allows assessing the mining machine performance. The operation indicators may include an overall equipment effectiveness (OEE) indicators determined for the machine. Real-time monitoring of the drilling machines in the fleet may be performed, which allows e.g. making decisions during a shift, to ensure that the performance of the fleet during the shift conforms to target performance. The target performance may be defined as, e.g., one or more of a number of drilled holes, a combined depth of the holes, an amount of time that it took to drill the holes, etc. The target performance may be set for a shift, a day, a month, a quarter, a year, and / or any other period of time. The target performance may be set for the specific mining machine, and the actual machine performance, determined using the operation indicators, may be compared to the target drilling performance, to determine whether the drilling machine performs drilling in accordance with the target drilling performance and whether certain control parameters for the drilling machine need to be adjusted.

[0160] At block 316, the method 300 comprises initiating an action in dependence on the identifying of the at least one operating state and / or in dependence on the determining of the at least one operation indicator. The action may additionally be initiated in dependence on the identifying of one or more drilling operations performed by the machine in the first period of time. In some examples, the action may also be initiated in dependence on the estimated depth of the borehole.

[0161] In some examples, the action may be initiated in dependence on the identified drilling operation and the at least one operation indicator. In some examples, initiating the action may comprise sending data on one or more of the identified operating stated, the identified drilling operation, and the determined operation indicators. The data may be sent by the field computer device 16 to the mining machine controller 28. In some examples, additionally or alternatively, the data may be sent by the field computer device 16 to the central control system 22. In examples in which the field computer device 16 is part of the mining machine controller 28, the data may be sent by the mining machine controller 28 to the central control system 22.

[0162] In some examples, initiating the action may comprise one or more of prompting a display of one or more of a representation of the identified operating states, a representation of the identified drilling operation, and a representation of the determined operation indicators; receiving an instruction to control the machine in dependence on the at least one operation indicator; sending the instruction to control the machine in dependence on the at least one operation indicator; and adjusting a target requirement for performance of the machine during a second period of time. In some examples, initiating the action comprises controlling at least one operating parameter of the mining machine in dependence on the at least one operation indicator determined for the machine. In some examples, initiating the action comprises adjusting a target requirement for performance of the machine during a second period of time.

[0163] In some examples, the field computer device 16 may receive the instruction to control the machine in dependence on the at least one operating state and / or the at least one operation indicator from the control system 22. In some examples, the field computer device 16 may provide the instruction to the mining machine, e.g., to the mining machine controller 28 which then controls operation of the mining machine based on the instruction. In some examples, e.g. in which the field computer device 16 is implemented as part of the mining machine controller 28, the field computer device 16 and / or the mining machine controller 28 may be configured to control operation of the mining machine based on the instruction. Furthermore, in some examples, the field computer device 16 may be configured to generate the instruction and to provide the instruction to the mining machine.

[0164] In some examples, regardless of the specific way in which the field computer device 16 and / or the mining machine controller 28 generate or receive or obtain the instruction, such instructions may be a control instruction that is used to adjust one or more operating characteristics of the mining machine 12. Non-limiting examples of the operating characteristics comprise one or more of a speed of drilling, a speed of changing rods, a position of the mining machine, timing and duration of standby periods of the mining machine, and other suitable operating characteristics. For example, the mining machine 12 may be instructed to stop operation if it is determined that it is not currently capable of performing drilling e.g. due to a malfunction, lack of resources such as one or more of electricity, water, and drilling rod(s), or other reasons. As another example, the mining machine may be instructed to move to another location in the mine.

[0165] In some examples, the status of the machine and its operating state as related to the drilling, such as the representation of the at least one operating state and / or the representation of the at least one operation indicator may be displayed on the user interface onboard of the machine, to be viewed by the operator of the machine. The representation of at least one operating state and / or the representation of the at least one operation indicator may be displayed on any one or more user interfaces, including at the same time. For example, the representation of at least one operating state and / or of the at least one operation indicator may be displayed on the user interface onboard of the machine and on the user interface associated with the central control system 22

[0166] The representation of the identified operating states, the identified drilling operation, and / or the operation indicators may be presented on a display of a computer device, e.g., the field computer device 16, the mining machine controller 28, and / or the central control system 22, or any other computer device or system. For example, the representation may be displayed on user interface 280 on rendered on the display communicatively coupled to the central control system 22. The representation may be displayed in real time, as the mining machine is operating, such that a progress of drilling may be visualized in real time. For example, as the operating states are identified, they may be displayed on a user interface accessible to an operator of the mining machine. Once a plurality of operating states are identified and a complete drilling operation is identified to be performed, a representation of the drilling operation may be displayed on the user interface accessible to the operator of the mining machine. Similar information may be presented on the user interface associated with the central control system 22.

[0167] The second period of time may be any period of time after the first period of time. Adjusting the target requirement for performance of the machine may comprise adjusting target parameter values related to operation of the machine This may involve setting different operating parameters for the machine so that the machine's operation during further time periods e.g. further shifts, days, months, etc. is adjusted to meet different target requirements. The machine may be controlled to move to a different location As another example, one or more drilling parameters related to operation of the drilling implement may be adjusted.

[0168] In some examples, the adjustment of the target requirement for performance of the mining machine during the second period of time may involve adjusting operational parameters for the mining machine such that the machine is controlled to operate i.e. drill boreholes in dependence on its previously determined performance. For example, if the machine is determined to be underperforming, its target parameter values, e.g. a number of boreholes bored during a shift, a frequency of use, a length of continuous periods of use, and / or other target parameter values, may be increased or otherwise adjusted. It should be noted that the target performance may be increased if it is additionally determined that the machine does not experience excessive downtime caused by unplanned failures or by other factors that affect machine performance.

[0169] If the machine is determined to be overused, its target parameter values, e.g. a number of boreholes bored during a shift, a frequency of use, a length of continuous periods of use, and / or other target performance parameters, may be decreased or otherwise adjusted.

[0170] In some examples, a reliability ranking or another similar comparison measure may be used to rank the mining machines in the worksite based on the respective operation indicator such as performance metrics. For example, a mining machine may be considered to have a higher reliability when it has a higher utilization efficiency and a lower downtime than one or more of other mining machines. In some example, more reliable drilling machines may be assigned to more critical production points in the worksite. Less reliable drilling machines can be assigned to less critical points to ensure planned production.

[0171] Thus, in some examples, a type of adjustment for the mining machine, based on the identified operating states and operation indicators, may include changing a location of the mining machine in the worksite such as a mine. For example, an instruction may be generated and sent to the mining machine with higher performance metrics, instructing the mining machine to move to a more critical location in the mine The performance metrics, which may be estimated using the method described herein, may be, e.g., a number of boreholes bored by the machine during a shift or another period of time such as a number of drilling operations, a number of rods used to drill a borehole, a length of continuous periods of use of the machine, a frequency of use of the machine, a utilization efficiency, and a downtime. A mining machine with higher performance metrics may be instructed to move to another location in the mine where use of the machine may be increased. A mining machine with lower performance metrics may be instructed to move to a location in the mine where use of that machine may be decreased.

[0172] In some examples, one or more reasons of underperformance of the mining machine may be identified when the machine is determined to be underperforming e.g. due to component failures. For example, itmay be determined that the machine is underperforming due to excessive delays caused by mechanical failures, in which case a measure related to machine maintenance may be taken

[0173] In some examples, if the mining machine is determined to be underperforming due to excessive delays caused by environmental conditions, a plan can be implemented to improve conditions in the work area. Some examples include power supply or water supply failures, which can extend machine downtime.

[0174] FIG. 4 illustrates an example of a training pipeline or process 400 for training a machine-learning model, in accordance with examples of the present disclosure. The training process 400 is performed to obtain a machine-learning model that can be used to detect or predict operational or operating states of the drilling machine. The machine-learning model may be selected from one or more candidate machine-learning models. The operating states may be identified and defined from patterns detected in signals acquired by one or more IMU sensor units. The training process 400 may be performed in advance, and a resulting trained machinelearning model may be provided, for example, as part of a kit provided in accordance with examples of the present disclosure. In some examples, the training may be performed by a central control system e.g. central control system 22, and / or by another external control system.

[0175] In some examples, the machine-learning model may be trained by the field computer device 16, such that the execution of the training pipeline may be part of the method in accordance with examples of the present disclosure as performed by the field computer device 16. The training pipeline may be executed automatically.

[0176] At block 402, the process 400 comprises obtaining inertial sensor measurements acquired by one or more IMU sensor or sensor units. The IMU sensor may be coupled to a drilling implement of a mining machine operating in a worksite such as a mining environment or mining site. The inertial sensor measurements may be acquired, directly or indirectly, from IMU sensors coupled to respective drilling implements of multiple mining machines in the mining environment. The inertial sensor measurements, which may be referred to as training data, are acquired as the one or more mining machines are operating in the mining environment. The one or more mining machines may be any suitable types of mining machines configured to perform drilling. In some examples, each of the mining machines used to acquire training data comprises an IMU sensor coupled thereto, e.g., on the movable implement of the mining machine. In some examples, the IMU sensor may be coupled to a drill motor configured to drive the drilling implement. In some examples, the mining machine comprises one i.e. single IMU sensor. The single IMU sensor comprises an accelerometer, a gyroscope, and a magnetometer The single IMU sensor may be sufficient to perform the methods in accordance with the present disclosure.

[0177] In some examples, the training data may be updated as more inertial sensor measurements are acquired from mining machines in the mining environment. In some examples, the one or more inertial sensor measurements used for the training stage may be simulated data, or a combination of simulated data and actual sensor measurements.

[0178] At block 405, exploratory data analysis (EDA) may be performed on the acquired inertial sensor measurements. The EDA may be performed to ensure that the resulting model is agnostic to a drilling machine and a mining environment. The EDA stage may include pattern analysis, statistical analysis, frequency analysis,and other types of processing. The objective of the exploratory analysis of the inertial sensor measurements is to find patterns that allow identifying a time window in which the mining machines perform one or more operating states. The operating states may comprise one or more of a drill motor on state, a drill motor off state, a drilling state, a state of adding at least one rod, a state of removing at least one rod, and an inactive drilling implement state. In some examples, the time window may be configured to train the model.

[0179] At block 407, the process 400 may comprise data preprocessing, which may involve various techniques to prepare the inertial sensor measurements for training. During this phase or stage, different data cleaning methods may be applied, such as e.g. outlier filtering, elimination of null or corrupted values, and signal smoothing using different techniques such as rolling, low-pass filtering or Fourier transform. In some examples, calibration of the IMU sensor may be performed. Also, secondary signals and / or additional transforms may be obtained that provide relevant information for a feature extraction stage, such as, e.g., envelope calculation, Fourier transform, logarithmic transform, wavelets, and empirical mode decomposition, among others. Preprocessed data may be generated as a result of preprocessing of the one or more inertial sensor measurements.

[0180] At block 409, the process 400 may comprise performing feature construction and / or extraction to identity one or more features to be used in the machine-learning model. For example, the preprocessed data may be labeled according to observed and identified patterns, distinguishing between different operating states or events. The most relevant signals may be selected to form variables, and sliding time windows may be designed or generated with overlapping, adjusted to a duration of the operating states. In some examples, for each of the axes e.g. 9 axes, statistical indicators may be calculated using the time windows as a basis, and a class is assigned with a name of the operating state based on the previously labeled pattern. These calculated characteristics and the class constitute the variables to be used for modeling. Non-limiting examples of the statistical indicators may comprise one or more of a mean, a standard deviation, a slope, a median, a maximum, a minimum, polynomial coefficients, kurtosis, skewness, and frequency response.

[0181] At block 411, at a training and model selection stage, the process 400 may comprise training one or more candidate machine-learning models and selecting a model from the candidate models, which may be performed using any one or more of various approaches. A machine-learning model may be selected from a set of the candidate machine-learning models. For example, training data may be fitted to different supervised classification models, performance of the models may be evaluated, and the process 400 may select a classification model that is able to predict or identify operating states in the inertial sensor data with the greatest accuracy. In some examples, the machine-learning models comprise decision tree models, which use a class variable as a target. One or more machine-learning models may be trained and assessed, and a model that gives a most accurate performance and is less computationally expensive, as compared to the other models, may be selected. For the evaluation, evaluation metrics such as e.g., one or more out of f1-score, recall, precision, accuracy, area under the curve (AUC), and others, may be used. Also, techniques such as e.g. confusion matrices, classification reports, and others related to an importance of the variables may be used. In some examples, a machine-learning model may be selected which may deliver results above 90% for allevaluation metrics used. In some examples, a trained machine-learning model may be selected which may deliver results above 95% for all evaluation metrics used. The trained machine-learning model may be selected based on other criteria

[0182] At block 413, the process 400 may comprise validating the trained machine-learning model. The validation may be performed using one or more of various validation techniques. For example, to ensure appropriate performance of the selected machine-learning model, predictions may be made with unlabeled data, and metrics may be generated to evaluate and validate the machine-learning model. The validation state is used to access whether the model, selected at the training and model selection stage at block 411 , is able to perform, i.e. recognize operating states of a drilling mining machine, with performance similar to that exhibited by that model at the training and model selection stage.

[0183] It should be noted that a choice of a final model is an iterative process, subject to changes in the training data, so that monitoring, readjustment, and / or retraining may be considered in order to ensure that the model reflects an updated, most recent representation of reality. Accordingly, as shown schematically in FIG. 4, by arrow a1 , the process 400 may return, from block 413, to block 411, to continue training the model, which may be performed iteratively.

[0184] Further, as shown in FIG. 4, at decision block 415, it may be determined whether the validation is complete. If this is the case, the trained model may be output at block 417. Otherwise, as shown in FIG. 4, the process 400 may return to block 413 to continue the validation process of the model.

[0185] To perform the method steps of the method for monitoring and controlling operation of a mining machine out of a plurality of mining machines in a mining site, the field computer device and / or the mining machine controller may be configured to perform the processing described in connection with FIG. 3. In some examples, the field computer device and / or the mining machine controller may be configured to perform, at least in part, the processing described in connection with FIG. 4 which involves training a machine-learning model that is configured to be applied to inertial sensor measurements to predict or identify mining machine operating states that are related to drilling operations performed by the mining machine.

[0186] FIG. 5 illustrates an example of a process or method 500 that may be performed by a control system such as e.g. central control system 22 shown in FIGs. 1A and 2. The control system 22, which may be e.g. a fleet control system, comprises processing circuitry 24 e g. at least one processor that is configured to coordinate movements of one or more mining machines out of a plurality of mining machines in a mining site. Each mining machine of the one or more mining machines may comprise a field computer device and a mining machine controller that are communicatively coupled to the central control system 22. The control system 22 may be configured to send commands to the mining machine.

[0187] At block 502, the control system may receive, from the mining machine of the one or more mining machines, information on at least one operating state identified or determined or predicted for the mining machine. The control system may also receive information on at least one operation indicator determined from the identified at least one operating state. The at least one operating state and the at least one operation indicator may be determined, e.g, as described in connection with FIG. 3.

[0188] At block 504, the control system may generate at least one command to the mining machine in dependence on the received information.

[0189] At block 506, the control system may send the at least one command to the mining machine. The generating and sending may be performed in the same processing step or block. The command may be received by the mining machine and an action may be initiated on the mining machine in dependence on the at least one command. In some examples, the at least one command may comprise an instruction to control the machine in dependence on the at least one operation indicator. In some examples, the at least one command may comprise an instruction to control the machine in dependence on the at least one operating state and / or the at least one operation indicator. In some examples, the at least one command may comprise an instruction to control the machine in dependence on the identified drilling operation and / or the at least one operation indicator. The control system 22 may send the instruction to the field computer device 16 that receives the instruction. In some examples, the field computer device 16 may provide the instruction to the mining machine, e.g., to the mining machine controller 28 which then controls operation of the mining machine based on the instruction. In some examples, the field computer device 16 may be configured to control operation of the mining machine based on the instruction received from the control system. Furthermore, in some examples, the field computer device 16 may be configured to generate the instruction and to provide the instruction to the mining machine.

[0190] In an aspect, a kit for a mining machine is provided, the mining machine being configured to operate in a mining site and comprising a drilling implement configured to have one or more rods coupled thereto. The kit comprises at least one I MU sensor configured to be associated with the drilling implement of the machine; and a computer-readable storage medium, having stored thereon computer-executable instructions which, when executed by at least one processor, cause the at least one processor to, as the mining machine is operating in the mining site to drill at least one borehole, receive inertial sensor measurements acquired by the at least one I MU sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; apply a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine for the first period of time; determine at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the at least one operating state; and initiate an action in dependence on the determining of the at least one operation indicator.

[0191] The kit may be configured to be deployed on the mining vehicle. In some examples, the computer program product of the kit may be installed on a field computer device and / or a mining machine controller. In some examples, the computer program product of the kit may be implemented as the field computer device such that the field computer device may be installed and deployed on the mining machine in addition to existing one or more controllers of the mining machine. The at least one IMU sensor from the kit may be coupled or attached to suitable location on the mining machine, such as a drilling implement or a drill motor. In some examples, the at least one IMU sensor comprises one IMU sensor comprising an accelerometer, a gyroscope, and a magnetometer.

[0192] In some examples of implementation of the kit, the plurality of operating states comprise one or more of a drill motor on state, a drill motor off state, a drilling state, a state of adding at least one rod, a state of removing at least one rod, and an inactive drilling implement state.

[0193] In some examples of the kit, the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to, for a subsequent state from the identified at least one operating state, verify a correctness of identification of the subsequent state by confirming a veracity of transition from a preceding state to the subsequent state; and if the correctness of the identification of the subsequent state is not confirmed, remove the subsequent state from the identified at least one operating state prior to determining the at least one operation indicator.

[0194] In some examples, in the kit, the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to identify, using the identified operating states, a drilling operation by determining a start time and an end time of the drilling operation performed during the first period of time. The start time of the drilling operation may be determined by determining a start time of an identified operating state that is indicative of a start of the drilling operation, and the end time of the drilling operation may be determined by determining an end time of an identified operating state that is indicative of an end or completion of the drilling operation.

[0195] In some examples, a number of rods used during the drilling operation may be estimated or determined or detected.

[0196] In some examples of the kit, initiating the action in dependence on the determining of the at least one operation indicator may comprise generating, on a display of a computing device, a representation of the identified operating states, the drilling operation, and the at least one operation indicator.

[0197] In some examples of the kit, initiating the action in dependence on the determining of the at least one operation indicator may comprise one or more of prompting a display of a representation of the identified at least one operating state and / or a representation of the at least one operation indicator; receiving and / or generating an instruction to control the machine in dependence on the at least one operation indicator; providing the instruction e.g. by the field computer device to the mining machine controller, and adjusting a target requirement for performance of the machine during a second period of time.

[0198] In some examples of the kit, the initiating the action in dependence on the determining of the at least one operation indicator may comprise one or more of controlling the machine in dependence on the at least one operation indicator, and adjusting a target requirement for performance of the machine during a second period of time.

[0199] In some examples of the kit, the initiating the action in dependence on the determining of the at least one operation indicator may comprise initiating the action in dependence on identifying of the complete drilling operation.

[0200] In some examples of the kit, the IM U sensor comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. In such examples, the inertial sensor measurements may comprise nine-axis measurements.

[0201] In an aspect, a mining machine comprising the kit in accordance with examples of the present disclosure is provided.

[0202] Operational steps described in any of the exemplary aspects herein are described to provide examples and discussion. The steps may be performed by hardware components, may be embodied in machineexecutable instructions to cause a processor to perform the steps, or may be performed by a combination of hardware and software. Although a specific order of method steps may be shown or described, the order of the steps may differ. In addition, two or more steps may be performed concurrently or with partial concurrence.

[0203] The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a,” "an,” and "the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term "and / or” includes any and all combinations of one or more of the associated listed items. It will be further understood that the terms "comprises,” "comprising,” “includes," and / or “including” when used herein specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0204] It will be understood that, although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element without departing from the scope of the present disclosure.

[0205] Relative terms such as “below” or “above” or “upper” or “lower" may be used herein to describe a relationship of one element to another element as illustrated in the Figures. It will be understood that these terms and those discussed above are intended to encompass different orientations of the device in addition to the orientation depicted in the Figures. It will be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or intervening elements may be present. In contrast, when an element is referred to as being “directly connected” or “directly coupled" to another element, there are no intervening elements present.

[0206] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms used herein should be interpreted as having a meaning consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein

[0207] It is to be understood that the present disclosure is not limited to the aspects described above and illustrated in the drawings; rather, the skilled person will recognize that many changes and modifications may be made within the scope of the present disclosure and appended claims. In the drawings and specification, there have been disclosed aspects for purposes of illustration only and not for purposes of limitation, the scope of the inventive concepts being set forth in the following claims.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method (300) for monitoring and controlling operation of a mining machine in a mining site, the mining machine comprising a drilling implement configured to have one or more rods coupled thereto, the method comprising: as the mining machine is operating in the mining site to drill at least one borehole, receiving (302) inertial sensor measurements acquired by at least one inertial measurement unit (I MU) sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; applying (304) a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine, for the first period of time; determining (314) at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the identified at least one operating state; and initiating (316) an action in dependence on the determining of the at least one operation indicator.

2. The method of claim 1, comprising, for a subsequent state from the at least one operating state, verifying (306) a correctness of identification of the subsequent state by confirming a veracity of transition from a preceding operating state to the subsequent operating state.

3. The method of claim 2, comprising, if the correctness of the identification of the subsequent operating state is not confirmed, removing (309) the subsequent operating state from the identified at least one operating state prior to determining the at least one operation indicator.

4. The method of any one of claims 1 to 3, wherein the set of operating states comprises one or more of a drill motor on state, a drill motor off state, a drilling state, a state of adding at least one rod, a state of removing at least one rod, and an inactive drilling implement state.

5. The method of any one of claims 1 to 4, wherein the at least one identified operating state comprises a plurality of operating states, and wherein the method further comprises identifying (310), using the identified plurality of operating states, a drilling operation by determining a start time and an end time of the drilling operation performed during the first period of time.

6. The method of claim 5, wherein determining (314) the at least one operation indicator comprises determining the at least one operation indicator using the identified drilling operation.

7. The method of any one of claims 5 to 6, further comprising estimating (312) a depth of the at least one borehole drilled by the mining machine during the drilling operation.

8. The method of any one of claims 1 to 7, wherein initiating (316) the action in dependence on the determining of the at least one operation indicator comprises one or more of: prompting a display of a representation of the identified at least one operating state and / or a representation of the at least one operation indicator; receiving an instruction to control the machine in dependence on the at least one operation indicator; providing the instruction to the machine; and adjusting a target requirement for performance of the machine during a second period of time.

9. A field computer device (16) comprising processing circuitry that is configured to perform the method of any one of claims 1 to 8.

10. A computer program product comprising computer-executable instructions, which, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 8.

11. A tangible computer-readable storage medium, having stored thereon a computer program product comprising computer-executable instructions which, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 8.

12. A mining machine (12) for operation in a mining site, comprising: a drilling implement (15) configured to have one or more rods (17) coupled thereto; at least one inertial measurement unit, IMU, sensor (14) configured to be associated with the drilling implement of the mining machine; and at least one processor and memory storing computer-executable instructions which, when executed by the at least one processor, cause the at least one processor to: as the mining machine is operating in the mining site to drill at least one borehole, receive inertial sensor measurements acquired by the at least one IMU sensor associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; apply a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine, for the first period of time;determine at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the identified at least one operating state; and initiate an action in dependence on the determining of the at least one operation indicator.

13. The mining machine of claim 12, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to, for a subsequent state from the identified at least one operating state: verify a correctness of identification of the subsequent state by confirming a veracity of transition from a preceding state to the subsequent state; and if the correctness of the identification of the subsequent state is not confirmed, remove the subsequent state from the identified at least one operating state prior to determining the at least one operation indicator.

14. The mining machine of claim 12 or claim 13, wherein the set of operating states comprises one or more of a drill motor on state, a drill motor off state, a drilling state, a state of adding at least one rod, a state of removing at least one rod, and an inactive drilling implement state.

15. The mining machine of any one of claims 12 to 14, wherein the at least one identified operating state comprises a plurality of operating states, and wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to identify, using the identified plurality of operating states, a drilling operation by determining a start time and an end time of the complete drilling operation performed during the first period of time, and wherein determining the at least one operation indicator further comprises determining the at least one operation indicator using the identified drilling operation16. The mining machine of claim 15, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to estimate a depth of the at least one borehole drilled by the mining machine during the drilling operation.

17. The mining machine of any one of claims 12 to 16, wherein initiating the action in dependence on the determining of the at least one operation indicator comprises one or more of: prompting a display of a representation of the identified at least one operating state and / or a representation of the at least one operation indicator. receiving an instruction to control the machine in dependence on the at least one operation indicator; providing the instruction to the machine; and adjusting a target requirement for performance of the machine during a second period of time.

18. The mining machine of any one of claims 12 to 17, wherein the IMU sensor (14) comprises a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer.

19. A kit (20) for a mining machine (12) configured to operate in a mining site and comprising a drilling implement (15) configured to have one or more rods (17) coupled thereto, the kit (20) comprising: at least one inertial measurement unit, IMU, sensor (14) configured to be associated with the drilling implement of the machine; and a computer-readable storage medium, having stored thereon computer-executable instructions which, when executed by at least one processor (210, 32), cause the at least one processor to: as the mining machine is operating in the mining site to drill at least one borehole, receive inertial sensor measurements acquired by the at least one IMU sensor (14) associated with the drilling implement, wherein the inertial sensor measurements are acquired over a first period of time; apply a trained machine-learning model to the inertial sensor measurements to identify at least one operating state out of a set of operating states of the mining machine, for the first period of time; determine at least one operation indicator from the identified at least one operating state, wherein the at least one operation indicator comprises a duration of the at least one operating state; and initiate an action in dependence on the determining of the at least one operation indicator.

20. The kit of claim 19, wherein the at least one identified operating state comprises a certain number of operating states, and wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to identify, using the identified operating states, a drilling operation by determining a start time and an end time of the drilling operation performed during the first period of time, wherein determining the at least one operation indicator comprises determining the at least one operation indicator using the identified drilling operation.

Images