Field computer device, mining machine, kit, and methods for monitoring and controlling mining machines in a mining environment

Abstract

A method for monitoring and controlling operation of a mining machine (12) in a mining site, the mining machine, a field computer device (16), and a kit for installation on the mining machine are provided. As the mining machine is operating in the mining site, inertial sensor measurements are received over a first period of time by at least one inertial measurement unit (IMU) sensor associated with the mining machine. A first trained machine-learning model is applied to the inertial sensor measurements to identify at least one material moving cycle performed by the machine within the first period of time. At least one operation indicator is determined from the identified material moving cycle, and an action may be initiated in dependence on the operation indicator. Position sensor measurements may be acquired from position tags and used to verify a loading and / or dumping point for the material moving cycle.

Description

Docket No.: PS56142PC00 / P23087WO01TITLE FIELD COMPUTER DEVICE, MINING MACHINE, KIT, AND METHODS FOR MONITORING ANDCONTROLLING MINING MACHINES IN A MINING ENVIRONMENTTECHNICALFIELD

[0001] The disclosure relates to systems and methods for monitoring and controlling operation ofa mining machine from a plurality of mining machine in a mining site. The disclosure further relates to afield computer device configured to be installed on the mining machine, a mining machine, a kit for themining machine, a computer program product, and a computer-readable storage medium for monitoringand controlling operation of a mining machine.BACKGROUND

[0002] In mining and tunnelling, developments are constantly underway to improve efficiency,productivity, and safety. One of the leading areas in which changes / improvements are increasinglytaking place is automation, full or partial, of various processes occurring in mining / tunneling.

[0003] Mining machines, e.g., trucks, for underground mining and tunneling can perform varioustasks in environments that are dark and often inaccessible by foot and may generally be notcomfortable for human drivers. Thus, it is often desirable that mining machines that operate in anunderground environment can be driven in a fully autonomous mode, i.e., without an onboard operatorbeing required to control the machines during machine operation.

[0004] An example of mining machine where automated operation is typically considered to bebeneficial are so-called load-haul-dump (LHD) machines. The LDH machines may be used to removeand transport broken rock / ore from a certain location, e.g., a position where blasting has beenperformed, to a particular place where the broken rock is dumped. After dumping their load, at the placethat may be referred to as a dump point or location, the LHD machines typically return to an initial (start)location to pick up a new load. Thus, these machines often travel the same route over and over again,which makes the travel between load and dump locations well suited for automation. There are alsovarious other situations where automation may prove beneficial.

[0005] The mining machine may be operating in one out of different possible operating states, anda current state is typically recorded by a machine operator. This may however be a tiring andcumbersome process for a human. Also, such manual recording of the process may be error-prone,whereas accurate and timely recording of operating states of the mining machine may affect theproductivity of the entire mining operation.Docket No.: PS56142PC00 / P23087WO01SUMMARY

[0006] In an aspect, a method for monitoring and controlling operation of a mining machine in amining site is provided. The method comprises, with at least one processor, as the mining machine isoperating in the mining site, receiving inertial sensor measurements acquired by at least one IMUsensor associated with the mining machine, wherein the inertial sensor measurements are acquiredover a first period of time; applying a first trained machine-learning model to the inertial sensormeasurements to identify at least one material moving cycle performed by the mining machine withinthe first period of time; determining at least one operation indicator from the identified at least onematerial moving cycle; and initiating an action in dependence on determining of the at least oneoperation indicator.

[0007] The technical benefits and advantages comprise automatic prediction or detection ofmaterial moving cycles, which allows improved control over productivity of individual mining machinesand of the entire mine. In this way, overall efficiency, safety and productivity of mining operations maybe increased. Furthermore, the determining of the material moving cycles and operation indicators maybe performed in real time, which allows implementation of timely control and intervention in fleetmanagement systems. For a fleet of mining machines, decisions may be made on dynamic operationalassignments. As a further advantage, the provided approach is equipment agnostic, such that it may beintegrated with various mining machinery and equipment without requiring extensive modifications orspecialized hardware. This flexibility not only simplifies implementation, but also allows for wideradoption in different mining configurations.

[0008] Additionally, the provided system, mining machine, kit, and methods therein are developedand configured to operate with low computational resource requirements. This feature allows for theprovided technique to be implemented in onboard controllers, e.g. in a field controller or computerdevice, which typically have limited processing capabilities. By using resources efficiently, the providedtechnique may ensure optimal performance even in challenging computing environments. Furthermore,various events or actions may be triggered e.g. initiated in response to predicting the ore moving cyclesand determining productivity indicators such as e.g., predictive maintenance of mining machines,optimizing routes traveled by the machines, defining and adjusting assignments for the miningmachines, and others.

[0009] In some examples, the material moving cycle may comprise a sequence of operatingstates that the mining machine performs during the material moving cycle, the sequence of operatingstates comprising loading the mining machine with material at a loading point, moving the material byDocket No.: PS56142PC00 / P23087WO01the mining machine from the loading point to a dumping point, unloading the material at the dumpingpoint, and moving from the dumping point to the loading point or to another loading point.

[0010] In some examples, the method may further comprise determining a correctness of theidentified at least one material moving cycle. The at least one material moving cycle may be determinedto be correctly identified when the at least one material moving cycle has a duration that is within athreshold range and / or when the at least one material moving cycle is an operational cycle.

[0011] In some examples, the method may further comprise receiving, over the first period of time,position sensor measurements acquired from at least one position beacon positioned in the mining site.

[0012] In some examples, the method may further comprise applying a second trained machine-learning model to the position sensor measurements to verify the loading point and the dumping pointfor the identified at least one material moving cycle.

[0013] In some examples, the second trained machine-learning model is trained in dependency onone or more features selected from time series data previously acquired by measuring signals from oneor more position beacons. The one or more position beacons may comprise the at least one positionbeacon. In some examples, the signals from one or more position beacons may comprise actual dataand / or simulated data.

[0014] In some examples, the first trained machine-learning model is trained using inertial sensormeasurements previously acquired by one or more IMU sensors. The one or more IMU sensors mayinclude the at least one IMU sensor and / or by one or more other sensors. In some examples, traininginertial sensor measurements may be simulated data or a combination of actual data acquired in a real-world environment and the simulated data.

[0015] In some examples, initiating the action in dependence on determining of the at least oneoperation indicator comprises prompting a display of a representation of the identified at least onematerial moving cycle and / or a representation of the at least one operation indicator.

[0016] In some examples, initiating the action in dependence on the determining of the at leastone operation indicator comprises one or more of obtaining an instruction to control the mining machinein dependence on the at least one operation indicator; and adjusting a target requirement forperformance of the mining machine.

[0017] In some aspects, a computer device is provided that comprises at least one processorconfigured to perform the method in accordance with any embodiments of the present disclosure.Docket No.: PS56142PC00 / P23087WO01

[0018] Advantages and effects of the computer device are largely analogous to the advantagesand effects of the method according to the examples herein. Further, all embodiments of the computerdevice are applicable to and combinable with all embodiments of the method according to the examplesherein, and vice versa.

[0019] In some aspects, a computer program product is provided that comprises computer-executable instructions, which, when executed by at least one processor, cause the at least oneprocessor to perform the method in accordance with any embodiments of the present disclosure. Insome examples, the computer program product may be provided as part of a kit.

[0020] Advantages and effects of the computer program product are largely analogous to theadvantages and effects of the method according to the examples herein. Further, all embodiments ofthe computer program are applicable to and combinable with all embodiments of the method accordingto the examples herein, and vice versa.

[0021] In some aspects, a tangible computer-readable storage medium is provided that has storedthereon a computer program product comprising computer-executable instructions. The computer-executable instructions, when executed by at least one processor, cause the at least one processor toperform the method in accordance with any embodiments of the present disclosure. In some examples,the tangible computer-readable storage medium may be provided as part of a kit.

[0022] Advantages and effects of the tangible computer-readable storage medium are largelyanalogous to the advantages and effects of the method according to the examples herein. Further, allembodiments of the tangible computer-readable storage medium are applicable to and combinable withall embodiments of the method according to the examples herein, and vice versa.

[0023] In some aspects, a mining machine for operation in a mining site is provided. The miningmachine comprises a movable implement comprising a tool configured to load and unload material; atleast one IMU sensor associated with mining machine and configured to acquire inertial sensormeasurements as the mining machine is operating in the mining site; and at least one processor and amemory comprising computer-executable instructions that, when executed by the at least oneprocessor, cause the at least one processor to, as the mining machine is operating in the mining site,receive the inertial sensor measurements acquired by the at least one IMU sensor, wherein the inertialsensor measurements are acquired over a first period of time; apply a first trained machine-learningmodel to the inertial sensor measurements to identify at least one material moving cycle performed bythe mining machine within the first period of time; determine at least one operation indicator from theDocket No.: PS56142PC00 / P23087WO01identified at least one material moving cycle; and initiate an action in dependence on determining of theat least one operation indicator.

[0024] Advantages and effects of the mining machine are largely analogous to the advantagesand effects of the method according to the examples herein. Further, all embodiments of the miningmachine are applicable to and combinable with all embodiments of the method according to theexamples herein, and vice versa.

[0025] In some examples, the material moving cycle may comprise a sequence of operatingstates that the mining machine performs during the material moving cycle, the sequence of operatingstates comprising loading the mining machine with material at a loading point, moving the material bythe mining machine from the loading point to a dumping point, unloading the material at the dumpingpoint, and moving from the dumping point to the loading point or to another loading point.

[0026] In some examples, the computer-executable instructions, when executed by the at leastone processor, may further cause the at least one processor to determine a correctness of the identifiedat least one material moving cycle, wherein the at least one material moving cycle is determined to becorrectly identified when the at least one material moving cycle has a duration that is within a thresholdrange and / or when the at least one material moving cycle is an operational cycle.

[0027] In some examples, the computer-executable instructions, when executed by the at leastone processor, further cause the at least one processor to receive, over the first period of time, positionsensor measurements acquired from at least one position beacon positioned in the mining site, andapply a second trained machine-learning model to the position sensor measurements to verify theloading point and the dumping point for the identified at least one material moving cycle.

[0028] In some examples, initiating the action in dependence on the determining of the at leastone operation indicator comprises prompting, by the at least one processor, a display of arepresentation of the identified at least one material moving cycle and / or a representation of the at leastone operation indicator.

[0029] In some examples, initiating the action in dependence on the determining of the at leastone operation indicator comprises one or more of obtaining an instruction to control the mining machinein dependence on the at least one operation indicator, providing the instruction to the mining machine,and adjusting a target requirement for performance of the mining machine.

[0030] In some aspects, a kit for installation on a mining machine is provided. The mining machineis configured to operate in a mining site. The kit comprises at least one IMU sensor configured to beassociated with mining machine and computer program product comprising computer-executableDocket No.: PS56142PC00 / P23087WO01instructions configured to be installed on the mining machine. The computer-executable instructions,when executed by at least one processor, cause the at least one processor to, as the mining machine isoperating in the mining site, receive inertial sensor measurements acquired by the at least one IMUsensor associated with the mining machine, wherein the inertial sensor measurements are acquiredover a first period of time; apply a first trained machine-learning model to the inertial sensormeasurements to identify at least one material moving cycle performed by the mining machine withinthe first period of time; determine at least one operation indicator from the identified at least onematerial moving cycle; and initiate an action in dependence on determining of the at least one operationindicator.

[0031] In some examples, the at least one IMU sensor included in the kit comprises one e.g.single IMU sensor.

[0032] In some examples of the kit, the IMU sensor comprises a three-axis accelerometer, athree-axis gyroscope, and a three-axis magnetometer. In such examples, the inertial sensormeasurements may comprise nine-axis measurements.

[0033] Advantages and effects of the kit are largely analogous to the advantages and effects ofthe method according to the examples herein. Further, all embodiments of the kit are applicable to andcombinable with all embodiments of the method according to the examples herein, and vice versa.

[0034] In some examples, wherein the material moving cycle may comprise a sequence ofoperating states that the mining machine performs during the material moving cycle, the sequence ofoperating states comprising loading the mining machine with material at a loading point, moving thematerial by the mining machine from the loading point to a dumping point, unloading the material at thedumping point, and moving from the dumping point to the loading point or to another loading point.

[0035] In some examples, the computer-executable instructions, when executed by the at leastone processor, further cause the at least one processor to receive, over the first period of time, positionsensor measurements acquired from at least one position beacon positioned in the mining site; andapply a second trained machine-learning model to the position sensor measurements to verify theloading point and the dumping point for the identified at least one material moving cycle.

[0036] Additional features and advantages are disclosed in the following description, claims, anddrawings. Furthermore, additional advantages will be readily apparent from the present disclosure tothose skilled in the art or recognized by practicing the disclosure as described herein. There are alsodisclosed herein control units, computer program products, and computer-readable media associatedwith the above discussed technical effects and corresponding advantages.Docket No.: PS56142PC00 / P23087WO01BRIEF DESCRIPTION OF THE DRAWINGS

[0037] With reference to the appended drawings, below follows a more detailed description ofaspects of the disclosure cited as examples.

[0038] FIG.1A illustrates an example of a mining environment in which a method in accordancewith examples of the present disclosure may be implemented.

[0039] FIG.1B illustrates an example of a mining machine.

[0040] FIG.2 is a block diagram illustrating an example of a system comprising a mining machineand a central control system, in which a method in accordance with examples of the present disclosuremay be implemented.

[0041] FIG.3 is a flowchart illustrating a method for monitoring and controlling operation of amining machine in a mining site, in accordance with examples of the present disclosure.

[0042] FIG.4 is a flowchart illustrating a method for generating and training a machine-learningmodel, in accordance with examples of the present disclosure.

[0043] FIG.5 is another flowchart illustrating a method for monitoring and controlling operation ofa mining machine in a mining site, in accordance with examples of the present disclosure.

[0044] FIG.6 is another flowchart illustrating a method for monitoring and controlling operation ofa mining machine in a mining site, in accordance with examples of the present disclosure.

[0045] FIG.7 is a flowchart illustrating a method performed by a central control system, inaccordance with examples of the present disclosure.

[0046] FIGs.8, 9, 10, and 11 illustrate examples of information that can be presented on a userinterface of a computer device, in accordance with examples of the present disclosure.DETAILED DESCRIPTION

[0047] Aspects of the present disclosure relate to a method performed by a field computer devicefor monitoring and controlling operation of a mining machine in a mining site, to the field computerdevice, a kit comprising at least one inertial measurement unit (IMU) sensor, and the mining machinecomprising the field computer device.

[0048] The techniques described herein allow using inertial sensor measurements acquired by atleast one inertial measurement unit (IMU) to identify one or more operating cycles for a mining machinethat is operating in the mining site. A trained machine-learning model may be used to identify the one ormore operating cycles. The cycle may comprise a sequence of operating states comprising loading themining machine at a loading point, moving of the loaded mining machine from the loading point to adumping point, unloading the mining machine at the dumping point, and moving the unloaded miningDocket No.: PS56142PC00 / P23087WO01machine from the dumping point to the loading point. However, in examples in accordance with thepresent disclosure, an ore hauling cycle is detected as a whole, without separately identifying asequence of operating states forming the cycle. The machine-learning model may be trained to detectthe entire cycle rather than to detect separate operating states within an ore hauling cycle.

[0049] Furthermore, in examples herein, in addition to the inertial sensor measurements, the fieldcomputer device is configured to also acquire position sensor measurements also referred to herein astag sensor measurements, e.g., from at least one position beacon or tag. The position beacon or tagmay be positioned at a known location in the mine. The field computer device may acquire the positionsensor measurements from at least one sensor that is configured to communicate with the at least one,typically multiple position tags. The at least one sensor may be installed in or otherwise associated withthe field computer device, or it may be positioned in another location on the mining machine. The fieldcomputer device installed on the mining machine may be configured to acquire the position sensormeasurements as the mining machine is moved through the mine, and as the inertial sensormeasurements are acquired by the field computer device.

[0050] In some examples, the position or tag sensor measurements may be pre-processed andstored e.g. in a memory of the field computer device until an ore hauling cycle is detected. Once thecycle is identified and validated using the inertial sensor measurements, the position sensormeasurements acquired during a time period when the inertial sensor measurements are acquired, areprocessed. The processing may be performed using a trained machine-learning model, referred toherein as a second machine-learning model. The second machine-learning model may be applied tothe pre-processed position sensor measurements to identify and / or verify an origin or loading point fromwhich the material is taken and a destination or dumping point to which the material is taken during theidentification of the material hauling cycle.

[0051] The second machine-learning model may provide an output such as a value indicating aprobability that a given signal comes from a certain loading point. More than one loading point may beidentified, each assigned a corresponding probability. In some examples, a loading point associatedwith a highest probability may be selected as an actual loading point from which the ore was takenduring the identified ore hauling cycle. In some examples, the dumping point may be identified and / orverified, for the material hauling cycle, in a similar manner. In some examples, the dumping point maybe identified and / or verified based on a portion of the position sensor measurements that were acquiredduring a dumping time window.

[0052] In aspects, the computer-implemented method for monitoring and controlling operation of amining machine in a mining site, in accordance with examples of the present disclosure comprises, by aprocessor, as the mining machine is operating in the mining site, receiving inertial sensorDocket No.: PS56142PC00 / P23087WO01measurements acquired by at least one IMU sensor associated with the mining machine, wherein theinertial sensor measurements are acquired over a first period of time; applying a first trained machine-learning model to the inertial sensor measurements to identify at least one material moving cycleperformed by the mining machine within the first period of time; determining at least one operationindicator from the identified at least one material moving cycle; and initiating an action in dependenceon determining of the at least one operation indicator.

[0053] FIG.1A depicts an example of a mining environment or site 10 comprising multiple, four inthis example, mining vehicles or machines 12a, 12b, 12c, and 12d. The mining machines may be of thesame or different types. The mining machines may be underground mining machines or surface miningmachines. One or more of the mining vehicles 12a-12d may be load-haul-dump (LHD) machines orother types of machines configured to carry material from one location to another. For example, themining machine may be an excavator, a backhoe, or another type of a mining machine with a movableimplement. In some examples, the mining machines 12a-12d may be autonomous machines, such ase.g., fully or partially autonomous machines. In some examples, the mining machines 12a-12d may beoperating in a fleet of mining machines. The mining machines in the fleet may be assigned workassignments or tasks.

[0054] The mining environment or site 10 may be a surface mining site or an underground miningsite. The mining site 10 may be any type of a mine-like environment. In some examples, the mining site10 may be a subway mine. Although not shown in FIG.1A, The mining site 10 may have a specificconfiguration. For example, if the mining site 10 is an underground mining site, it may include tunnels.

[0055] A mining machine in the mining site 10 may have a moving or movable implement, suchas, e.g., an arm having a bucket attached thereto. As shown FIG.1A, each of the mining machines12a-12d may have a respective IMU sensor 14, also referred to herein as an inertial sensor associatedwith mining machine and configured to acquire inertial sensor measurements as the mining machine isoperating in the mining site. An IMU sensor or sensor unit is a sensor that provides motion data in atime-series format. The IMU sensor comprises an accelerometer configured to acquire accelerationmeasurements, a gyroscope configured to acquire angular velocity measurements, and amagnetometer configured to measure a magnitude and direction of the magnetic field at a location ofthe magnetometer. Thus, combined measurements acquired by the IMU sensor can be used todetermine a position, velocity, acceleration, and orientation of an object in a three-dimensional space towhich object the IMU sensor is attached. The measurements acquired by each of the accelerometer,gyroscope, and magnetometer are represented along a three-axis coordinate system. Thus, the IMUsensor may be configured to provide a nine-dimensional time series data.Docket No.: PS56142PC00 / P23087WO01

[0056] The mining machines 12a-12d comprise respective IMU sensors 14a, 14b, 14c, and 14d,coupled to a corresponding movable implement 15, such as movable implements 15a, 15b, 15c, and15d of the mining machine. In some examples, each of the inertial sensors 14a, 14b, 14c, and 14d maybe a single i.e. one IMU sensor. Thus, in some examples, inertial sensor measurements acquired by asingle IMU sensor or sensor unit may be sufficient to perform the method in accordance with examplesof the present disclosure, which makes the provided approach less costly and requiring lessmaintenance. The single IMU sensor unit comprises an accelerometer, a gyroscope, and amagnetometer.

[0057] The movable implement may comprise a tool configured to load and unload material, whichmay be any suitable load picking and carrying implement or tool, e.g., a bucket or another work tool fordigging and / or load picking and carrying. The movable implement may comprise an arm such as e.g. ahydraulic arm which may comprise the tool configured to load and unload material e.g. a bucket as inan LHD machine. In some examples, the IMU sensor unit may be coupled to the arm, e.g, to a joint ofthe arm. In some examples, the inertial sensor may be positioned in proximity to the joint of the arm andin proximity to a tool for digging and / or load picking and carrying, e.g., a bucket or another work tool ofthe movable implement. In some examples, the IMU sensor unit may be positioned on the bucket, butsuch that the use of the bucket does not interfere with the accuracy of measurements acquired by theIMU sensor. In any case, the IMU sensor unit may be positioned to detect and report acceleration,orientation, angular rates, and other gravitational forces as the mining machine and / or its parts moveand vibrate.

[0058] At least one IMU sensor is positioned on the mining machine where the acquired sensormeasurements are informative of a current state of the movable implement and of the entire machine.Also, an ease of installation of the inertial sensor is taken into consideration when selecting a specificlocation at which to associate the inertial sensor with the mining machine. The specific position of theinertial sensor on the mining machine may depend on a configuration of the mining machine and themachine’s movable implement, type of work performed by the mining machine, work environmentconditions to which the mining machine is subjected, the feasibility of installation of the inertial sensorand / or on other factors. In any case, in examples herein, the inertial sensor is positioned outside of anoperator compartment or cabin of the mining machine. The IMU sensor cannot be positioned in theoperator compartment or cabin because information about the movement of the arm and / or bucketwould be lost in such case.

[0059] It should be appreciated that the mining machine may comprise various other sensors.

[0060] As further shown in FIG.1A, each of the machines 12a-12d may comprise or may beassociated with a respective control unit or a field computer device 16a, 16b, 16c, 16d that may beDocket No.: PS56142PC00 / P23087WO01configured to perform the method in accordance with examples of the present disclosure. Each of thefield computer devices 16a-16d includes various components not shown in this example, such as amemory device configured to store computer-executable instructions and processing circuitry e.g. oneor more processors. The processors are configured to execute the computer-executable instructions tocause the processing circuitry to perform the method in accordance with examples of the presentdisclosure, as discussed in more detail below. Each of the field computer devices 16a-16d also has aninput and output interface that is configured to communicate with other components, e.g., with therespective IMU sensor, as well as with a fleet controller or central control system 22 and with otherexternal systems.

[0061] Furthermore, in some examples, one or more of the field computer devices 16a-16d maycomprise or may otherwise be associated with one or more position sensors configured tocommunicate with at least one position beacon positioned in the mining site. For example, as shown inFIG.1A in connection with the first mining vehicle 12a, the mining site 10 may comprise positionbeacons 11a, 11b, and 11c. Although the three position beacons are shown, multiple position beaconsmay be present in the mining site. Position beacons may be located in proximity to a material origin orloading point and in proximity to a material destination or dumping point, to assist in determining withmore precision the origin of the material and a point of dumping the material. Position beacons may beuseful in determining a location of the mining machine in the mine, as the machine is moving e.g. in atunnel. The mining environment may in some cases be deep underground and tracking locations ofmining machines operating in such environments may be a challenge, let alone automaticallydetermining production of the machine.

[0062] The position beacon may be an active position tag or a passive position tag. In someexamples, the position beacon be configured as a transmitter, in some cases as a transceiver, and theposition of the position beacon may be known. The position beacons may be installed in any suitablelocations in the mining site. In some examples, as the mining machine is moving and / or handlesmaterial in the mining site, position sensor measurements may be acquired based on detection of theposition beacons. For example, the one or more position sensors associated with the field computerdevice of the mining machine may detect signals, e.g. radio frequency (RF) or other types of signals,emitted by the position beacons. In some implementations, the position beacon may emit signals inresponse to corresponding signals from the position sensors associated with the mining machine.Furthermore, in some cases, the position sensors may be passive tag with a known position, and theone or more position sensors associated with the mining machine may read the position informationcarried by the passive position tag. Regardless of a specific implementation of position beacons in themining site, the mining machine, e.g. a field computer device associated with the mining machine, isDocket No.: PS56142PC00 / P23087WO01configured to receive and process position sensor measurements to verify the loading point and thedumping point for at least one identified material moving cycle.

[0063] Multiple mining machines 12a-12d may be operated in the mining site 10 for haulingmaterial such as ore from draw points or locations to one or more dumping points where the ore isdeposited. Thus, a mining machine may be operated in an operating or operational material movingcycle that may comprise a sequence of operating states comprising loading the mining machine withmaterial at a loading point, moving the material by the mining machine from the loading point to adumping point, unloading the material at the dumping point, and moving or translating back from thedumping point to the loading point, without a load. The mining machine may also be in a standby modeor standby, when it is idling such that it is operating but not currently moving or performing any work.The material moving cycle may include another sequence of operating states. The material movingcycle may be repeated multiple times during, for example, a shift of the operation of the miningmachine. The mining machine may perform a certain number of cycles during the shift. For example, inexamples in which the mining machine is autonomous, the shift may be performed until, e.g., the miningmachine runs out of fuel or power or until a planned production for a certain zone is met.

[0064] In some examples, a kit 18 may be installed and deployed on a mining machine out of theplurality of mining machines 12a-12d in the mining site 10. The kit may comprise at least one inertialsensor and computer program product comprising computer-executable instructions which, whenexecuted by processing circuitry e.g. at least one processor, cause the processor to perform a methodin accordance with examples of the present disclosure. The computer-executable instructions may bestored in a memory e.g. in a memory of the field computer device that may be included as part of thekit. In some examples, the computer-executable instructions may be installed on the mining machinewithout the use of a separate field computer device, e.g., on a controller unit of the mining machine.

[0065] FIG. 1A illustrates very schematically, for the mining machines 12a and 12b, that themachines may comprise respective kits 18a and 18b, shown by a dot-dashed line. The kit 18a maycomprise the inertial sensor 14a and computer program product that may be stored in the fieldcomputer device 16a. Similarly, the kit 18b may comprise the inertial sensor 14b and computer programproduct that may be stored in the field computer device 16b.

[0066] As shown in FIG.1A, operation of mining machines in the mining site 10, shown by way ofexample as the mining machines 12a-12d, may be controlled by the central control system 22 such ase.g. a fleet controller or fleet control system. The central control system 22, which is communicativelycoupled with one or more of the mining machines 12a-12d, may track locations or positions of each ofthe mining machines 12a-12d, may receive data from the mining machines, and may send commandsor instructions to the mining machines. The fleet control system may also be referred to as a trafficDocket No.: PS56142PC00 / P23087WO01control system, and it may be any type of a control system that monitors multiple machines e.g.receives information from the mining machines, and sends commands or instructions to the miningmachines. The central control system 22 comprises processing circuitry 24 that is configured to executecomputer-executable instructions which thereby cause the processing circuitry 24 to perform a methodfor controlling operation of one or more mining machines, in accordance with examples of the presentdisclosure. The central control system 22 may have various other components not shown herein forsimplicity, e.g., memory, user interface, communication interface, etc.

[0067] FIG.1B illustrates an example of a mining machine 32, such as e.g., any of the miningmachines 12a-12d shown in FIG.1A, or another type of a mining machine. FIG.1B illustratesschematically example locations or positions 34p1 and 34p2 of where an inertial sensor may becoupled to a movable implement 25 of the mining machine 32. The inertial sensor may be configured toacquire accelerometer, magnetometer, and gyroscope measurements. The measurements may beacquired at a desired frequency and may be used to infer when the mining machine lifts the movableimplement 25 such as a hydraulic arm with a bucket or another tool attached thereto. The inertialsensor may be installed on the hydraulic arm of the movable implement 25 in proximity to the bucket,as shown in FIG.1B. In the example illustrated in FIG.1B, the example positions 34p1 and 34p2 maybe evaluated for ease of installation and data quality of obtained sensor measurements. It should beappreciated that the mining machine 32 is shown in FIG.1B as a non-limiting example only, as theinertial sensor may be associated with a mining machine in any suitable location. In some examples,two or more positions on the mining machine may be evaluated for placement of an inertial sensor. Insome examples, the evaluation may not be performed.

[0068] The approach described herein, in accordance with examples of the present disclosure,comprises two workflows also referred to as pipelines, which in turn comprise multiple stages. One ofthe pipelines is a training pipeline, which is developed using historical or batch data, and includes pre-analysis stages such as e.g. sensor description and position, exploratory data analysis (EDA), andmodeling stages such as data capture and pre-processing, feature extraction, modeling, and validation.Another pipeline is a production pipeline, which is developed with data generated in real time, as themining machine is operating in a mining environment or site, and includes such stages as e.g, queuingdata, preprocessing, feature extraction, generating predictions, performing validation logic whichincludes confirming veracity or correctness of identification of a material moving cycle, determiningoperation indicators, and generating results representation.

[0069] FIG.2 illustrates an example of a system 100 in which example embodiments of thepresent disclosure may be implemented. The system 100 may be employed, at least in part, in themining environment or site such as e.g. mining site 10 shown schematically in FIG.1A. The system 100Docket No.: PS56142PC00 / P23087WO01may comprise one or more mining machines, and a mining machine 12 is shown in FIG.2 as arepresentative mining machine. The system 100 may also comprise a central controller or controlsystem 22 such as, e.g. a traffic control system which may be a fleet controller configured to performcoordinated control of machines in the mining site. It should be noted however that the central controlsystem 22 may be remote and it may be positioned outside of the mining site 10. The mining machinesmay be e.g. load-haul-dump (LHD) trucks, shovels, etc.

[0070] As shown in FIG.2, the mining machine 12 may comprise a control system or miningmachine controller 28 and a field computer device 16 which may be configured to perform the methodin accordance with examples of the present disclosure. The field computer device 16 may be inoperable communication with the mining machine controller 28. It should be noted that, even thoughthe field computer device 16 is shown as a separate device in FIG.2, in some examples, the fieldcomputer device 16 may be part of the mining machine controller 28. Thus, the functionality performedby the field computer device 16 may be performed by the mining machine controller 28. In someexamples, this functionality may be imparted to the mining machine controller 28 by installing on themining machine controller 28 computer-executable instructions which may be part of a kit as describedherein. In some examples, this functionality may be imparted to the mining machine controller 28 byinstalling on the mining machine controller 28 computer-executable instructions and at least oneprocessor, which may be part of a kit as described herein.

[0071] The mining machine 12 may comprise or may be associated with a display 33 which maybe configured render a graphical user interface that is configured to display a representation of theidentified one or more material moving cycles, one or more operation indicators, as well as otherinformation related to operation of the mining machine 12 and analysis of the operation using thetechniques described herein. The display 33 may be part of a dashboard display or another type of abuilt-in display. In some examples, the display 33 may be associated e.g. coupled to a suitable locationwithin an operator compartment of the mining machine 12. Furthermore, in some examples, the display33 may be part of a remote device communicatively coupled to mining machine 12 and / or the miningmachine controller 28. For example, the display 33 may be presented in a smartphone, a tablet, oranother computer device which may be positioned remotely relative to the mining machine.

[0072] The machine controller 28 may be located onboard the machine 12. The machinecontroller 28 may be a main controller of the mining machine 12 which is configured to controloperations of the mining machine 12. The machine controller 28 may comprise processing circuitry 30,such as at least one processor, and memory 31 which may comprise one or more memory units. Thememory 31 comprises computer-executable instructions executable by the processing circuitry 30 ofthe machine controller 28. The memory 31 may be configured to store information, data, etc., and theDocket No.: PS56142PC00 / P23087WO01computer-executable instructions to perform, when executed by the processing circuitry 30, variousprocesses related to monitoring and control operation of the mining machine 12. The machine controller28 may comprise or may be associated with various other components not shown herein.

[0073] The field computer device 16 may be adapted to execute computer-executable instructionsto perform the functions or processes described herein. The field computer device 16 may beconnected (e.g., networked) to other machines in a local area network (LAN), an intranet, an extranet,or the Internet. The field computer device 16 may comprise processing circuitry 210 and a memorydevice or memory 220 which may comprise one or more memory units. The memory 220 comprisescomputer-executable instructions which may be executed by the processing circuitry 210 to cause theprocessing circuitry 210 to perform the method in accordance with examples of the present disclosure.

[0074] The memory 220 may store various data related to the method in accordance withexamples of the present disclosure. As shown in FIG.2, the memory 220 may store inertial sensormeasurements 221 which may be acquired by the IMU sensor 14 also shown in FIG.2. As also shownin FIG.2, the memory 220 may store position sensor measurements 222 which may be acquired by atleast one position sensor 17 also shown in FIG.2. The position sensor 17 may be configured tocommunicate with one or more position beacons located in the mining site. In some implementations,the position sensor 17 may be included in the machine controller 28. In some cases, the inertial sensormeasurements and the position sensor measurements may be sent to an external system, e.g., to thecontrol system 22 and / or to another system.

[0075] The memory 220 may comprise a material moving cycles registry 224 storing one or morematerial moving cycles predicted or identified for the mining machine 12.

[0076] The memory 220 may also comprise machine-learning (ML) model units that store at leastone trained ML model as well as various information associated with the trained ML model e.g. dataused to train the model, extracted / constructed and / or selected features, and other information. The atleast one ML model unit may comprise a first ML model unit 226 storing a trained ML model, referred toherein as a first ML model, that can be applied to the inertial sensor measurements to identify or predictat least one material moving cycle from the inertial sensor measurements. The at least one ML modelunit may also comprise a second ML model unit 228 storing a trained ML model, referred to herein as asecond ML model, that can be applied to the position sensor measurements to verify a position of theloading and / or dumping points for the at least one material moving cycle that involves moving materialfrom the loading point to the dumping point.

[0077] The ML model units 226, 228 may comprise one or more subunits or modules not shownhere, e.g., exploratory data analysis (EDA) unit, a feature construction and selection unit, amodel building unit, a model evaluation unit, and other units or subunits.Docket No.: PS56142PC00 / P23087WO01

[0078] The memory 220 may comprise an operation indicators registry 232 storing one or moreoperation indicators 234. The one or more operation indicators 234 may be determined from one ormore cycles predicted or identified for the mining machine. Non-limiting examples of the operationindicators 234 comprise a duration of a certain material moving cycle e.g. an ore hauling cycle; anumber of material moving cycles performed by the mining machine 12 during a certain time period e.g.a shift, a day, a month, a quarter, etc.; a total operating time determined for the mining machine; a totalnon-productive time representing a time during which the machine is stopped and / or is not contributingto production; operational efficiency of the mining machine, an unplanned downtime experienced by themining machine; one or more production delays, and various other operation indicators etc. Theoperation indicators 234 may be stored in association with other information such as e.g. an operatoridentifier identifying an operator of the mining machine 12, a machine identifier identifying the miningmachine 12, locations in the mine where the one or more identified material moving cycles wereidentified, and other information.

[0079] As shown in FIG.2, the processing circuitry 210 may comprise at least one ML modelexecution unit 240 that is configured to execute the trained ML models stored in the ML model units226, 228. The first and second trained ML models may be executed by the processing circuitry 210 toperform the method in accordance with examples of the present disclosure. The processing circuitry210 may also comprise model retraining unit 230 that allows the processing circuitry 210 toautomatically train and retrain the first ML model for identifying and / or predicting a material movingcycle and the second ML model for identifying and / or verifying a loading point and a dumping pointbetween which the mining machine has moved the material during the identified material moving cycle.In some examples, the same module or unit may perform ML model processing and retraining, e.g. theML model execution unit 240. The processing circuitry 210 may include various other modulesand / units configured to perform actions and methods in accordance with examples herein.

[0080] The processing circuitry 210 may include any number of hardware components forconducting data or signal processing or for executing computer code such as computer-executableinstructions stored in the memory 220. The processing circuitry 210 may include a general-purposeprocessor, an application specific processor, a Digital Signal Processor (DSP), an Application SpecificIntegrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a circuit containing processingcomponents, or other programmable logic device, discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functions described herein.

[0081] The memory 220 may be one or more devices for storing data and / or computer code suchas computer-executable instructions for completing or facilitating methods described herein. Thememory 220 may comprise random access memory (RAM), read-only memory (ROM), erasableDocket No.: PS56142PC00 / P23087WO01programmable read-only memory (EPROM), hard drive storage, temporary storage, non-volatilememory, flash memory, optical memory, or any other suitable memory for storing software objectsand / or computer instructions. The memory 604 may include database components, object codecomponents, script components, and / or any other type of information structures for supporting thevarious processes and information structures described in the present disclosure. The memory 220may be communicably connected to the processing circuitry 210, e.g., via a circuit or any other wired orwireless connection.

[0082] The field computer device 16 may also include a communications interface 242 that mayinclude wired and / or wireless communications interfaces, e.g., jacks, antennas, transmitters, receivers,transceivers, wire terminals, etc., for conducting data communications external systems or devices. Invarious examples, the communications may be direct, e.g., local wired or wireless communications, orvia a communications network, e.g., a WAN, the Internet, a cellular network, etc. The field computerdevice 16 may include various other components not shown herein.

[0083] It should be appreciated that any of the data and computer-executable instructions storedin the memory 220 may be loaded onto the processing circuity 210 or used by the processing circuitry210, to perform the method in accordance with examples of the present disclosure.

[0084] Those skilled in the art will appreciate that the units of the field computer device 16described herein may refer to a combination of analogue and digital circuits, and / or one or moreprocessors configured with software and / or firmware, e.g., stored in the field computer device 16, that,when executed by respective one or more processors, may perform the methods in accordance withexamples of the present disclosure. One or more of these processors, as well as the other digitalhardware, may be included in a single Application-Specific Integrated Circuitry (ASIC), or severalprocessors and various digital hardware may be distributed among several separate components,whether individually packaged or assembled into a system-on-a-chip.

[0085] It should be appreciated that the field computer device 16, the machine controller 28, andthe mining machine 12 may comprise various other components not shown in FIG.2 for the sake ofsimplicity. For example, the field computer device 16 may comprise a communication interface forcommunication with the IMU sensor 14 to receive one or more inertial sensor measurements, and forcommunication with remote systems such as e.g. the control system 22.

[0086] FIG. 2 illustrates that the central control system 22 comprises processing circuitry 24 andmemory 260. The central control system 22 may be a central controller or fleet controller that may bepositioned remotely from a plurality of mining machines operating in the mining site. In some examples,the central control system 22 may be positioned in the mining environment or site, e.g., it may be part ofor associated with one of the mining machines in the mining site. Regardless of its specificDocket No.: PS56142PC00 / P23087WO01implementation and location, the central control system 22 is configured to receive information from andto send information and control commands to one or more mining machines out of the plurality ofmining machines in the mining site. The central control system 22 may be configured to control, in acoordinated manner, movements and / or other functions of the mining machines. The mining machinesmay be fully autonomous, semi-autonomous, or manually controlled machines, and various type ofsignals and instructions may be received by the machines from the control system 22.

[0087] The memory 260 of the central control system 22 may store computer-executableinstructions that can be executed by the processing circuitry 24 to cause the processing circuitry 24 toperform monitoring and controlling of operations of the mining machine 12. The memory 260 of thecentral control system 20 may receive various information regarding the mining machine 12 e.g. fromthe field computer device 16 and / or the machine controller 28, such as one or more identified machinemoving cycles, one or more operator indicators, and various other information. Based on the receivedinformation, the control system 22 may generate and send commands to the mining machine 12, aswell as to one or more of other mining machines in the mining site.

[0088] As also shown in FIG.2, the control system 22 may comprise and / or may becommunicatively coupled to a display that is configured to render a graphical user interface 280. Theuser interface 280 may display various information related to mining machines controlled via the controlsystem 22. The user interface 280 may also be configured to receive user input e.g. with respect to thedisplayed information. The control system 22 may include an input and output device interface (notshown) such as e.g. a circuit for controlling input and output from and to peripheral devices includingdevices such as a mouse, a keyboard, joystick, touch-sensitive surface or pad, touch-sensitive screen,etc.

[0089] The user interface 280 of the control system 22 may be configured to present variousinformation based on the identified material moving cycles and the operation indicators. The informationallows visualizing and assessing a status of mining machines in the fleet, as well as a status of theentire fleet. For example, at any point in time, one or more material moving cycles identified for one ormore mining machines currently located in the mining site may be visualized. A representation of themachines in the mining site by material moving cycles may also be visualized such that it is possible toaccess a number of material moving cycles performed by the one or more machines. It may be alsopossible to determine whether a mining machine in the site is currently performing a cycle which hasnot yet been identified. It may be, for example, visualized how many mining machines are currentlyoperative, delayed, reserved or out of service. The one or more representations of the status of theindividual or groups of the machines, and of the entire fleet, may be generated and displayed in realtime, such that a real time monitoring, assessment, and control of the fleet of mining machines may beDocket No.: PS56142PC00 / P23087WO01performed. In this way, a user such as e.g. a fleet operator may be provided with information that canbe used to assess the status of the mining machines and to make decisions regarding operation of themining machines. The information may in some cases be analyzed automatically. Maintenancedecisions, machine repositioning, task or job assignments, and other actions may be performed usingthe identified cycles and the operation indicators identified or predicted in accordance with examples ofthe present disclosure. Instructions or commands may be generated by the control system 22 and sentto the mining machine 12 and / or other machines, instructing the machines to initiate actions related tomaintenance, repositioning, task or job assignments, and other types of actions.

[0090] The approach described herein, in accordance with examples of the present disclosure,may comprise two or more workflows also referred to as pipelines, which in turn may comprise multiplestages. One of the pipelines is a training pipeline, which is developed using historical or batch data, andincludes pre-analysis stages such as e.g. sensor description and position, exploratory data analysis(EDA), and modeling stages such as data capture and processing, feature extraction, modeling, andvalidation. Another pipeline is a production pipeline, which is developed with data generated in realtime, as the mining machine is operating in a mining environment or site, and includes such stages ase.g, queuing data, preprocessing, feature extraction, generating predictions, performing validation logicwhich includes confirming veracity of a state transition, determining operation indicators, and generatingresults representation. Other pipelines may be implemented as well.

[0091] FIG.3 illustrates an example of a computer-implemented process or method 300 formonitoring and controlling operation of a mining machine in a mining site. The mining machine may bee.g. machine 12 such as an LHD machine configured to move or haul material such as ore from onelocation to another. The method 300 may be performed by a computer device, e.g. the field computerdevice 16 or another suitable computer device which may be positioned in the mining machine or maybe otherwise associated with the mining machine including remotely. In some examples, the method300 may be performed by one or both the field computer device 16 and / or the mining machinecontroller 28. The actions at blocks of FIG.3 do not have to be taken in the order stated below, but maybe taken in any suitable order. Dashed boxes indicate optional features.

[0092] At block 302, the method 300 comprises, as the mining machine is operating in the miningsite, receiving inertial sensor measurements acquired by at least one IMU sensor associated with themining machine, wherein the inertial sensor measurements are acquired over a first period of time. Insome examples, the IMU sensor comprises a three-axis accelerometer, a three-axis gyroscope, and athree-axis magnetometer, and the received sensor measurements may comprise sensor data in 9 axes.Thus, patterns indicative of material moving cycles performed by the mining machine may be detectedin the 9-axis inertial sensor data or signal.Docket No.: PS56142PC00 / P23087WO01

[0093] In some examples, data acquired from one or more of the three-axis accelerometer, thethree-axis gyroscope, and the three-axis magnetometer may be used to identify a material movingcycle. For example, the IMU sensor may be positioned on the mining machine, e.g. on a movableimplement of the mining machine, at a location where readings acquired by the IMU sensor areinformative enough such that data from fewer than nine axes may be sufficient to accurately identify ordetect or predict a material moving cycle.

[0094] The first period of time may be a shift comprising several hours, a day i.e.24 hours, amonth, a quarter, or any other period of time. Any period of time may be selected to identify materialmoving cycles for the machine and to determine a number of the cycles performed by the machineduring that period of time.

[0095] At block 312, the method 300 comprises applying a first trained machine-learning (ML)model to the inertial sensor measurements to identify at least one material moving cycle performed bythe mining machine within the first period of time. The at least one material moving cycle out of aplurality of material moving cycles may be identified in real time, as the mining machine is operating inthe mining site. In some examples, the material moving cycle may be identified at a later time, e.g.,after the machine has completed the cycle.

[0096] It should be noted that the material moving cycle may be identified with some probability ofthe identification because the cycle is identified as a pattern recognized in the inertial sensormeasurements. The accuracy of the inertial sensor measurements and other factors may affect thecorrectness of the identification of the material moving cycle.

[0097] In some examples, the material moving cycle may comprise a sequence of operatingstates that the mining machine performs during the material moving cycle. The sequence of theoperating states performed by the mining machine may comprise (i) loading the mining machine withmaterial at a loading point, (ii) moving the material by the mining machine from the loading point to adumping point, (iii) unloading the material at the dumping point, and (iv) moving from the dumping pointto the loading point or to another loading point. For example, an LHD machine may pick a load of thematerial, e.g. in its bucket or other attachment, at the loading point and move the load to the dumpingpoint where the load is deposited. The mining machine, without a load, may then return to the loadingpoint to pick up another load, or the mining machine may move to a different loading point to pick upthe different load. The material moving cycle may be considered completed once the machine returnsto the same or different loading point in a state ready to pick another load. It should be noted that othersequence of the operating states may be considered to constitute the material moving cycle.

[0098] In examples herein, separate operating states are not identified and the material movingcycle is identified and reported as a whole. The first ML model may be trained to recognize a pattern inDocket No.: PS56142PC00 / P23087WO01the inertial sensor measurements that is indicative of a completed material moving cycle. The first MLmodel may be trained using known sub-patterns and time windows in IMU data corresponding tooperating states forming the material moving cycle, but the separate operating states may not bedetected for the purposes of the techniques described herein.

[0099] In some examples, the IMU sensor comprises a three-axis accelerometer, a three-axisgyroscope and a three-axis magnetometer. The inertial sensor measurements may comprise dataacquired in the nine corresponding axes, three per each of the accelerometer, gyroscope, andmagnetometer. Data or signal from each axis may be pre-processed to generate new variables, e.g.,one or more of signal enveloping, low-pass or high-pass filtering to eliminate noise, and movingaverage filtering to smooth the signal may be applied to the data. Feature extraction may be performedusing these new variables.

[0100] In some examples, the trained first ML model may be agnostic to one or more out of amodel of the mining machine, a brand of the mining machine, and a manufacturer of the miningmachine.

[0101] In some examples, the first ML model may be trained in dependency on at least oneproperty of the worksite e.g. mining environment, e.g. a layout of the mining environment, distances tobe traveled by the mining machine, etc.

[0102] In some examples, the first ML model may be trained as described in connection with FIG.4. In some examples, a training pipeline, described in more detail in connection with FIG.4, may beexecuted automatically, whereby the first ML model may be generated and trained, to be suitable foridentifying material moving cycles in measurements acquired by the IMU sensor. In examples herein,IMU sensor data may be sufficient to identify material moving cycles, though position sensor data mayadditionally be used, as described below. In any case, no specific input from a user, such as e.g. theoperator of the mining machine or another user, may be required to identify at least one materialmoving cycle. The accuracy of the identification or detection of the material moving cycle is improved,which improves the way in which the operation of the mining machine is assessed and controlled.

[0103] At block 313, the method 300 may optionally comprise determining a correctness of theidentified at least one material moving cycle. The at least one material moving cycle may be determinedto be correctly identified when the at least one material moving cycle has a duration that is within athreshold range and / or when the at least one material moving cycle is an operational cycle. Otherfactors may be used to determine whether the material moving cycle that has been identified is indeedan actual material moving cycle performed by the mining machine and whether the cycle is completed.

[0104] At decision block 315, it may be determined whether the correctness of the identification ofthe at least one material moving cycle has been confirmed or verified. Responsive to determining thatDocket No.: PS56142PC00 / P23087WO01the correctness of the identification of the at least one material moving cycle has not been verified, theprocess 300 may proceed to block 317. Alternatively, responsive to determining that the correctness ofthe identification of the at least one material moving cycle has been verified, i.e. that the miningmachine is estimated to have performed this cycle, the process 300 may proceed to block 320.

[0105] At block 317, the at least one identified material moving cycle may be removed fromfurther analysis and the process 300 may return to block 302 to receive further inertial sensormeasurements acquired by the IMU sensor. It should be noted that more than one material movingcycles is typically identified for further analysis.

[0106] At block 320, the method 300 comprises determining at least one operation indicator fromthe identified at least one material moving cycle. The at least one material moving cycle may comprisetwo or more material moving cycles. The operation indicator may be or may indicate a duration of theidentified material moving cycle, a number of material moving cycles performed by the mining machineduring the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any othertime period. The operation indicator may comprise any suitable one or more performance metricsindicating a status, a need for service, and other characteristics of the mining machine. The operationindicator may be expressed as a quantitative value, a qualitative value, or a combination thereof.

[0107] Machine utilization times may be useful for establishing proactive maintenance plans forthe mining machine and other machines in the mining environment. In some examples, the operationindicators may comprise a total operating time representing a total time during which the miningmachine is operational and productive e.g. moves the material such as ore from one or more loadingpoints to one or more dumping points. The total operating time may be determined by summing allperiods of time in which the mining machine is active and has been identified as generating value.

[0108] In some examples, the operation indicators may comprise a use of the mining machine pera certain duration of time. For example, a use U of the mining machine per shift may be calculated bydividing the total operating time by a duration of the shift:

[0110] A use of the mining machine during any other period of time may also be determined in asimilar manner.

[0111] In some examples, the operation indicators may comprise an overall operational efficiencyE which may be calculated by dividing the total operating time of the mining machine by a totalavailable time indicating a duration of time period during which the mining machine is or was available.It shows what percentage of the available time is used for production:Docket No.: PS56142PC00 / P23087WO01

[0112] ^ = ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 100% (2)downtime indicating a time during which the mining machine is down due to unplanned failures. Thisindicator can help identify sources of faults that may occur in the mining machine.

[0114] In some examples, the operation indicators may comprise unexpected delays duringproduction, such as one or more of a lack of electrical power, quality problems, or unforeseeninterruptions. Identifying and reducing such delays can improve efficiency of the operation of the miningmachine, a group of the mining machines e.g. in a fleet, and of the entire mine.

[0115] The operation indicators may be determined per shift, month, quarter, or per any otherperiod of time, whereby a fleet performance over that period of time may be determined.

[0116] The operation indicators may be represented on a user interface of a display, e.g., of themining machine and / or a fleet control system, in the manner that allows assessing the fleetperformance. The operation indicators may also include an overall equipment effectiveness (OEE)indicators determined for the mining machine. Real-time monitoring of the machines in the fleet may beperformed, which allows e.g. making decisions during a shift, to ensure that the performance of the fleetduring the shift conforms to target performance. The target performance may be defined as, e.g., oneor more of a number of material moving cycles per a certain time duration, utilization of the miningmachine such as an amount of time during which the machine is not used is minimized, etc. The targetperformance may be set for a shift, a day, a month, a quarter, a year, and / or any other period of time.

[0117] At block 322, the method 300 comprises initiating an action in dependence on determiningof the at least one operation indicator and / or in dependence on the identifying of the material movingcycle. The at least one operation indicator may comprise a plurality of operating indicators. In someexamples, initiating the action may comprise prompting a display of a representation of the identifiedmaterial moving cycle and / or a representation of the operation indicator. The representation of theidentified material moving cycle may be presented on a display such as e.g. a display of the centralcontrol system 22, a display associated with the mining machine and / or with the field computer device,and / or on a display associated with any other device or system.

[0118] In some examples, information on the material moving cycle and / or the at least oneoperation indicator may be displayed on user interface 280 rendered on the display communicativelycoupled to the central control system 22. In this way, a fleet operator or another person may be enabledto access performance of the mining machine, as well as of other machines operating in the mine, andto determine if any actions need to be taken. In some examples, additionally or alternatively,information on the material moving cycle and / or the at least one operation indicator may be displayedDocket No.: PS56142PC00 / P23087WO01on a user interface rendered on the display 33 associated with the mining machine 12. The driver oranother operator of the mining machine 12 may be informed, in real time, about a current status of themining machine and a location of the mining machine in the mine, a number of material moving cyclesthat have been performed by the mining machine, a time that it took the machine to perform each of thecycles, and / or any other suitable information. The driver may then adjust operation of the miningmachine and / or initiate other actions based on the information on the material moving cycle and / or theat least one operation indicator.

[0119] In some examples, initiating the action in dependence on the determining of the at leastone operation indicator comprises one or more of obtaining an instruction to control the mining machinein dependence on the at least one operation indicator, providing the instruction to the mining machine,and adjusting a target requirement for performance of the mining machine.

[0120] In some examples, the instruction to control the mining machine may be obtained by thefield computer device 16 and / or the mining machine controller 28, which may perform functionality ofthe field computer device 16. The instruction may be received from e.g. the central control system 22.In some examples, the instruction may be generated the field computer device 16 and / or the miningmachine controller 28. In some examples, the field computer device 16 may generate and / or receivethe instruction and to provide this instruction to the mining machine controller 28.

[0121] Regardless of the specific way in which the field computer device 16 and / or the miningmachine controller 28 generates or receives or obtains the instruction, such instructions may be acontrol instruction that is used to adjust one or more operating characteristics of the mining machine 12.Non-limiting examples of the operating characteristics comprise one or more of a speed of materialloading and / or unloading by the mining machine, a speed of movement of the mining machine, aposition of the mining machine, specific loading and dumping points at which the mining machine canoperate, timing and duration of idling by the mining machine, etc. For example, the mining machine maybe instructed to stop operation if it is determined that it is not currently capable of performing materialmoving e.g. due to a malfunction. As another example, the mining machine may be instructed to moveto another location in the mine. The mining machine may be instructed to pick up a load of materialfrom an alternative loading point and / or to carry the material to an alternative dumping point.

[0122] In some examples, the instruction to control the mining machine may be generated independence on the at least one operation indicator and additionally in dependence on other dataregarding operation of the mining machine and / or other types of data.

[0123] The adjustment of the target requirement for performance of the mining machine mayinvolve adjusting operational parameters for the mining machine such that the machine is controlled tooperate in dependence on its previously determined performance. For example, if the machine isDocket No.: PS56142PC00 / P23087WO01determined to be underperforming, its target performance, e.g. a number of material moving cyclescompleted during a shift, a frequency of use of the machine, a total length of continuous periods of useof the machine, and / or other target performance parameters, may be increased or otherwise adjusted.It should be noted that the target performance may be adjusted if it is additionally determined that themachine does not experience excessive downtime caused by unplanned failures or by other factors thataffect machine performance.

[0124] If the machine is determined to be overused, its target performance, e.g. a number ofmaterial moving cycles completed during a shift, a frequency of use of the machine, a total length ofcontinuous periods of use of the machine, and / or other target performance parameters, may bedecreased or otherwise adjusted.

[0125] Another type of adjustment for the mining machine, based on the identified material movingcycle and the at least one operation indicator, may include a location of the mining machine in theworksite such as a mine. For example, the mining machine with higher performance metrics may bemoved to a more critical location in the mine. As a related example, the mining machine with lowerperformance metrics may be moved to a location in the mine where less work would be required fromthe mining machine e.g. fewer material moving cycles are expected to be performed during a certaintime period.

[0126] In some examples, one or more reasons of underperformance of the mining machine maybe identified when the machine is determined to be underperforming e.g. due to component failures.For example, it may be determined that the machine is underperforming due to excessive delayscaused by mechanical failures, in which case a measure related to machine maintenance may betaken.

[0127] In some examples, if the mining machine is determined to be underperforming due toexcessive delays caused by environmental conditions, a plan can be implemented to improveconditions in the work area. Some examples include power supply or water supply failures, which canextend machine downtime.

[0128] In some examples, a reliability ranking or another similar comparison measure may beused to rank the mining machines in the worksite based on their performance such as completion ofone or more material moving cycles as identified in accordance with embodiments of the presentdisclosure. For example, a mining machine may be considered to have a higher reliability when it has ahigher utilization efficiency and a lower downtime than one or more of other mining machines. In someexamples, more reliable mining machines, such as e.g. LHD machines, may be assigned to morecritical production points in the worksite. Less reliable mining machines, such as e.g. LHD machines,can be assigned to less critical points in the mine, to ensure planned production.Docket No.: PS56142PC00 / P23087WO01

[0129] FIG.4 illustrates an example of a training pipeline or process 400 for training a ML modele.g. the first ML model, in accordance with embodiments of the present disclosure. The training process400 is performed to obtain an ML model that can be used to detect or predict or identify material movingcycles, also referred to herein as ore hauling cycles, that can be performed by the mining machine. Themachine-learning model may be selected from one or more candidate machine-learning models. Thematerial moving cycles may be identified and defined from patterns detected in signals acquired by oneor more IMU sensor units. The training process 400 may be performed in advance, and a resultingtrained machine-learning model may be provided, for example, as part of a kit provided in accordancewith examples of the present disclosure. In some examples, the training may be performed by a centralcontrol system e.g. central control system 22, and / or by another external control system.

[0130] In some examples, the training may be performed by the field computer device 16 and / orthe mining machine controller 28. Thus, the execution of the training pipeline may be part of the methodin accordance with examples of the present disclosure as performed by the field computer device 16and / or the mining machine controller 28. The training pipeline may be executed automatically.

[0131] At block 402, the process 400 comprises obtaining training inertial sensor measurementsacquired by one or more IMU sensors. The IMU sensors may be coupled to a mining machine e.g. to amovable implement comprising a controllable bucket or another similar tool. The inertial sensormeasurements may be acquired directly or indirectly, from IMU sensors associated with respective oneor more mining machines operating in a mining environment or mining site. The inertial sensormeasurements, which may be referred to as training data, are acquired as the one or more miningmachines are operating in the mining environment. The one or more mining machines may be anysuitable types of mining machines configured to load and upload material and to move with the materialfrom one location to another, e.g., LHD machines. In some examples, each of the mining machinesused to acquire training data comprises one i.e. single IMU sensor unit coupled thereto, e.g., on themovable implement of the mining machine.

[0132] In some examples, the training inertial sensor data may be updated as more sensormeasurements are acquired from the IMU sensors coupled to mining machines in the miningenvironment. In some examples, the one or more inertial sensor measurements used for the trainingstage may be simulated data, or a combination of simulated data and actual inertial sensormeasurements.

[0133] At block 405, exploratory data analysis (EDA) may be performed on the acquired inertialsensor measurements. The EDA may be performed to ensure that the resulting model is agnostic to atype of the mining machine and a mining environment. The EDA stage may include pattern analysis,statistical analysis, frequency analysis, and other types of processing. The objective of the exploratoryDocket No.: PS56142PC00 / P23087WO01analysis of the inertial sensor measurements is to find patterns that allow identifying a time window inwhich the mining machines perform one or more material moving cycles. A material moving cycle maycomprise sequence of operating states comprising loading the mining machine with material at aloading point, moving the material by the mining machine from the loading point to a dumping point,unloading the material at the dumping point, and moving from the dumping point to the loading point orto another loading point.

[0134] At block 407, the process 400 may comprise data preprocessing, which may involvevarious techniques to prepare the inertial sensor measurements for training. During this phase or stage,different data cleaning methods may be applied, such as e.g. outlier filtering, elimination of null orcorrupted values, and signal smoothing using different techniques such as rolling, low-pass filtering orFourier transform. In some examples, calibration of the IMU sensor unit may be performed. Also,secondary signals and / or additional transforms may be obtained that provide relevant information for afeature extraction stage, such as, e.g., envelope calculation, Fourier transform, logarithmic transform,wavelets, and empirical mode decomposition, among others. Preprocessed IMU sensor data may begenerated as a result of preprocessing of the one or more inertial sensor measurements.

[0135] At block 409, the process 400 may comprise performing feature construction and / orextraction to identity one or more features to be used in the ML model. For example, the preprocessedIMU sensor data may be labeled according to observed and / or identified patterns, distinguishingbetween different material moving cycles. The most relevant signals may be selected to form variables,and sliding time windows may be designed or generated with overlapping, adjusted to a duration of thematerial moving cycles. In some examples, for each of the axes e.g.9 axes of an IMU sensor, statisticalindicators may be calculated using the time windows as a basis, and a class is assigned with a label orname of the material moving cycles based on the previously labeled pattern. In some examples, abinary classification may be used, with windows that correspond to a cycle and with windows that donot correspond to a cycle. In some examples, a multiclass classification may be used such that morethan two classes can be assigned to a material moving cycle e.g., a productive cycle, a non-productivecycle, a loaded cycle, an unloaded cycle, etc. The calculated characteristics and the class constitutethe variables to be used for modeling. Non-limiting examples of the statistical indicators may compriseone or more of a mean, a standard deviation, a slope, a median, a maximum, a minimum, polynomialcoefficients, kurtosis, skewness, and frequency response. Non-limiting examples of features obtainedfrom the frequency response are gain, energy, and cutoff frequency.

[0136] At block 411, at a training and model selection stage, the process 400 may comprisetraining one or more candidate machine-learning models and selecting a model from the candidatemodels, which may be performed using any one or more of various approaches. An ML model may beDocket No.: PS56142PC00 / P23087WO01selected from a set of the candidate ML models. For example, training data may be fitted to differentsupervised classification models. In some examples, the machine-learning models comprise decisiontree models, which use a class variable as a target. One or more ML models may be trained andassessed, and a model that gives a most accurate performance and is less computationally expensivethan other models may be selected. For the evaluation, evaluation metrics such as e.g., one or moreout of f1-score, recall, precision, accuracy, area under the curve (AUC), and others, may be used. Also,techniques such as e.g. confusion matrices, classification reports, and others related to an importanceof the variables may be used. In some examples, an ML model may be selected which may deliverresults above 90% for all evaluation metrics used. In some examples, an ML model may be selectedwhich may deliver results above 95% for all evaluation metrics used. The ML model may be selectedbased on other criteria.

[0137] At block 413, the process 400 may comprise validating the trained ML model. Thevalidation may be performed using one or more of various validation techniques. For example, toensure appropriate performance of the selected ML model, predictions may be made with unlabeleddata, and metrics may be generated to evaluate and validate performance of the ML model using theunlabeled data. The validation state is used to access whether the model, selected at the training andmodel selection stage at block 411, is able to perform, i.e. recognize or identify material moving cyclesin inertial sensor measurements acquired by an IMU sensor or IMU sensor unit coupled to the miningmachine, with precision similar to that exhibited by that model at the training and model selection stage.

[0138] It should be noted that a choice of a final ML model is an iterative process, subject tochanges in the training data, so that monitoring, readjustment, and / or retraining may be performed inorder to ensure that the ML model reflects an updated, most recent representation of reality.Accordingly, as shown schematically in FIG.4, by arrow a1, the process 400 may return, from block413, to block 411, to continue training the model, which may be performed iteratively.

[0139] Further, as shown in FIG.4, at decision block 415, it may be determined whether thevalidation of the trained ML model is complete. If this is the case, the trained ML model may be outputat block 417. Otherwise, as shown in FIG.4, the process 400 may return to block 413 to continue thevalidation process of the ML model.

[0140] To perform the method steps of the method for monitoring and controlling operation of amining machine out of a plurality of mining machines in a mining site, the field computer device and / orthe mining machine controller may be configured to perform the processing described in connectionwith FIG.3. In some examples, the field computer device and / or the mining machine controller may beconfigured to perform, at least in part, the processing described in connection with FIG.4 whichDocket No.: PS56142PC00 / P23087WO01involves training a machine-learning model that is configured to be applied to inertial sensormeasurements to predict or identify material moving cycles performed by the mining machine.

[0141] In some examples, in addition to obtaining or receiving inertial sensor measurementsacquired by at least one IMU sensor coupled to the mining machine, the mining machine also obtains orreceives position sensor measurements acquired from at least one position beacon or tag positioned inthe mining site externally to the mining machine. The position tag may be stationary at a predeterminedlocation in the mine. For example, FIG.1A shows schematically an example of the position tags orbeacons 11a-11c, though it should be appreciated that multiple position beacons may be located in themine, to e.g. mark locations of loading areas or points, dumping areas or points, and other locations inthe mine. The position sensor measurements may be used to determine or verify locations of theloading point and the dumping point for one or more material moving cycles identified using the inertialsensor measurements data. In the mine, the mining machine may be moving in a tight environmentwhere multiple loading points and dumping points may be located, and the position sensormeasurements may assist in determining with improved precision an origin and destination of thematerial. Accordingly, for each identified material moving cycle, it may be known where the materialcame from and where it was moved to. This advantageously allows monitoring processes in the minewith improved accuracy, which allows managing and controlling operations of the mining machines andof the entire mine with improved performance. The material moving cycles performed by the miningmachine as well as related information may be determined automatically, without any input from adriver or another person, which improves efficiency of the mine operation and reduces a risk of errors inassessment and control of the mine operations.

[0142] FIG.5 shows a computer-implemented process or method 500 for monitoring andcontrolling operation of a mining machine in a mining site, in accordance with some embodiments of thepresent disclosure. The mining machine may be e.g. machine 12 such as an LHD machine configuredto move or haul material such as ore from one location to another. The method 500 may be performedby a computer device, e.g. the field computer device 16 and / or the mining machine controller 28, or byanother suitable computer device which may be positioned in the mining machine or may be otherwiseassociated with the mining machine. The actions at blocks of FIG.5 do not have to be taken in theorder stated below, but may be taken in any suitable order. Processing at some acts or blocks of FIG.5is similar to corresponding acts or blocks shown in FIG.3 in connection with the method 300, and theirdescription is therefore not repeated in connection with FIG.5.

[0143] At block 502, the method 500 comprises, as the mining machine is operating in the miningsite, receiving inertial sensor measurements acquired by at least one IMU sensor associated with themining machine The inertial sensor measurements may be acquired over a first period of time.Docket No.: PS56142PC00 / P23087WO01

[0144] At block 503, the method 500 comprises, as the mining machine is operating in the miningsite, receiving, over the first period of time, position sensor measurements acquired from at least oneposition beacon positioned in the mining site. The position sensor measurements may be acquired astime series data that may be stored in memory e.g. of the field computer device 16 and / or the miningmachine controller 28.

[0145] The position beacon or tag may be positioned at a known location in the mine. Thecomputer device may receive the position sensor measurements from at least one sensor that isconfigured to communicate with the at least one, typically multiple position tags. The at least oneposition sensor, e.g. position sensor 17 shown in FIG.2, may be installed in or otherwise associatedwith the field computer device and / or mining machine controller 28. The position sensor measurementsmay be acquired as the mining machine is moved through the mine and as the mining machinesreceiving the inertial sensor measurements. In other words, the position sensor measurements may bereceived simultaneously or substantially simultaneously with receiving the inertial sensormeasurements. Thus, the processing at block 503 may be performed simultaneously or substantiallysimultaneously with the processing at block 502.

[0146] At block 505, the method 500 comprises preprocessing of the inertial sensormeasurements. The preprocessing may use known techniques for preparation raw sensormeasurements data to further analysis, including smoothing and other techniques. The preprocessingat block 505 matches the preprocessing performed on the data in the model training pipeline. As data isreceived from the sensors, the data may be continuously processed in real time and stored in thememory of the field computer device 16 and / or the mining machine controller 28.

[0147] At block 507, the method 500 comprises preprocessing of the position sensormeasurements or data. The position sensor data may be time series data that may be preprocessedusing one or more of a moving average filter, a median filter, and Kalman filter, to smooth the signals inthe data and eliminate noise. The preprocessing of the position sensor measurements matches thepreprocessing performed on the data in the training pipeline of the second model.

[0148] In some examples, the position sensor data may be preprocessed in accordance withcharacteristics of material moving cycles identified using embodiments of the present disclosure. Amaterial moving cycle may have a respective duration, different from duration of other cycles, and maydepend on factors such as e.g. machine operator behavior, zones in the mine, distances betweenloading and / or dumping points in the mine, a speed of the mining machine, etc. Thus, the time series ofthe position sensor data may be preprocessed by one or more scaling and dimensionality reductionmethods so that the time series corresponding to a certain material moving cycle, i.e. acquired duringthe performance of that cycle by the mining machine, have the same size.Docket No.: PS56142PC00 / P23087WO01

[0149] It should be noted that the data preprocessing at blocks 505 and 507 may be performed inany suitable order or simultaneously. In some examples, the preprocessing of the position sensormeasurements may be performed after applying the first trained ML model to the inertial sensormeasurements to identify at least one material moving cycle as shown in connection with block 512.

[0150] At block 510, the method 500 comprises performing feature extraction from thepreprocessed inertial sensor measurements and position sensor measurements. The same featuresmay be extracted as those used to train the model. The preprocessed data may be accumulated in realtime, e.g., in the memory of the field computer device or in another memory device, until a window sizenecessary to calculate the characteristics is met, and the data is then used to obtain the inference ofthe model.

[0151] The feature extraction may involve applying a dimensionality reduction technique to extractmeaningful features from the preprocessed inertial sensor measurements and position sensormeasurements. The features which may be extracted from the preprocessed inertial sensormeasurements are suitable for application to these features of the first ML model, to identify at leastone material moving cycle. Similarly, the features which may be extracted from the preprocessedposition sensor measurements are suitable for application to these features of the second ML model, toverify the loading point and the dumping point for the identified at least one material moving cycle.

[0152] At block 512, the method 500 comprises applying the first trained ML model to the inertialsensor measurements to identify at least one material moving cycle. The processing at block 512 maybe performed similar to the processing at block 312 of FIG.3.

[0153] At block 514, the method 500 comprises applying a second trained ML model to theposition sensor measurements to verify the loading point and the dumping point for the identifiedmaterial moving cycle. The second ML model may be trained similarly to the first ML model, e.g. asdescribed in connection with FIG.4, but the training data would comprise position sensormeasurements, actual and / or simulated, received from position beacons or tag positioned in the mineor in a simulator environment.

[0154] In some examples, once the material moving cycle is identified and its correctness isvalidated or confirmed, e.g. as shown in connection with blocks 313 and 315 of FIG.3, the positionsensor measurements or tag signals acquired during the same time period, referred to herein as thefirst time period, may be processed e.g. as shown at blocks 507 and 510. In some examples, theposition sensor measurements or tag signals may be processed before the material moving cycle isidentified.

[0155] The tag signals may be processed and the processed data e.g. features may be input tothe trained second ML model. The second ML model may provide an output such as a value indicatingDocket No.: PS56142PC00 / P23087WO01a probability that a given signal comes from a certain loading point. More than one loading point may beidentified, each assigned a corresponding probability. In some examples, a loading point identified witha highest probability may be selected as an actual loading point from which the material was takenduring the identified material moving cycle. In some examples, the dumping point may be identifiedand / or verified, for the material moving cycle, in a similar manner in which the loading point may beidentified. The loading point may be identified and / or verified based on a portion of the position sensormeasurements that were acquired during a loading time window such as time window during which themining machine is expected, from the training data, to perform loading of the material. The dumpingpoint may be identified and / or verified based on a portion of the position sensor measurements thatwere acquired during a dumping time window such as time window during which the mining machine isexpected, from the training data, to perform dumping or unloading of the material.

[0156] The second ML model may be trained to select the loading and unloading points using timeseries data acquired of each tag signal. Previously acquired time series data may be labeled to indicatea correct source signal for a plurality of material moving cycles, and the second ML model is trained todifferentiate and classify the time series data, to provide an output indicating a probability that positionsensor data acquired from the positions beacons is indicative of actual material origin or loading pointsand / or of material unloading or dumping points.

[0157] At block 520, the method 500 comprises determining at least one operation indicator fromthe identified at least one material moving cycle. The at least one material moving cycle may comprisetwo or more material moving cycles. The operation indicator may be or may indicate a duration of theidentified material moving cycle, a number of material moving cycles performed by the mining machineduring the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any othertime period. The operation indicator may comprise any suitable one or more performance metricsindicating a status, a need for service, and other characteristics of the mining machine. The processingat block 520 may be performed similarly to the processing at block 320 of FIG.3 and the description atblock 320 is applicable herein.

[0158] At block 522, the method 500 comprises initiating an action in dependence on determiningof the at least one operation indicator and / or in dependence on the identifying of the at least onematerial moving cycle. In some examples, initiating the action may comprise prompting a display of arepresentation of the identified at least one material moving cycle and / or a representation of the at leastone operation indicator. In some examples, initiating the action in dependence on the determining ofthe at least one operation indicator comprises one or more of obtaining an instruction to control themining machine in dependence on the at least one operation indicator, providing the instruction to themining machine, and adjusting a target requirement for performance of the mining machine. TheDocket No.: PS56142PC00 / P23087WO01processing at block 522 may be performed similarly to the processing at block 322 of FIG.3 and thedescription at block 322 is applicable herein.

[0159] In some examples, position sensor measurements may not be available, or acquiredposition sensor measurements may not be sufficient to verify positions of loading and / or dumping pointsfor a corresponding material moving cycle.

[0160] FIG.6 illustrates an example of a computer-implemented process or method 600 formonitoring and controlling operation of a mining machine in a mining site, in accordance with someembodiments of the present disclosure. The mining machine may be e.g. machine 12 such as an LHDmachine configured to move or haul material such as ore from one location to another. The method 600may be performed by a computer device, e.g. the field computer device 16 and / or the mining machinecontroller 28, or by another suitable computer device which may be positioned in the mining machine ormay be otherwise associated with the mining machine. The actions at blocks of FIG.6 do not have tobe taken in the order stated below, but may be taken in any suitable order. Processing at some acts orblocks of FIG.6 is similar to processing at corresponding acts or blocks shown in FIGs.3 and 5. Also,blocks in FIG.6 have numerical references that are similar to numerical references of correspondingblocks of FIGs.3 and 5 at which similar processing is performed. The description of processing stepsdescribed in connection with FIGs.3 and 5 is not repeated in connection with FIG.6, but it should beappreciated that the description applies to the method 600 of FIG.6.

[0161] At block 602, the method 600 may comprise, as the mining machine is operating in themining site, receiving inertial sensor measurements acquired by at least one IMU sensor associatedwith the mining machine. The method 600 may also comprise, similar to processing at block 503 of FIG.5, as the mining machine is operating in the mining site, receiving, over the first period of time, positionsensor measurements acquired from at least one position beacon positioned in the mining site.

[0162] At block 612, the method 600 may comprise applying a first trained ML model to theinertial sensor measurements to identify at least one material moving cycle performed by the miningmachine within the first period of time.

[0163] At block 613, the method 600 may comprise determining a correctness of the identified atleast one material moving cycle. The at least one material moving cycle may be determined to becorrectly identified when the at least one material moving cycle has a duration that is within a thresholdrange and / or when the at least one material moving cycle is an operational cycle. Other factors may beused to determine whether the material moving cycle that has been identified is indeed an actualmaterial moving cycle performed by the mining machine and whether the cycle is completed.

[0164] At decision block 615, the method 600 may determine whether the correctness of theidentification of the at least one material moving cycle has been confirmed or verified. Responsive toDocket No.: PS56142PC00 / P23087WO01determining that the correctness of the identification of the at least one material moving cycle has notbeen verified, the process 600 may proceed to block 617. Responsive to determining that thecorrectness of the identification of the at least one material moving cycle has been verified, the process600 may proceed to block 618.

[0165] At block 617, the at least one identified material moving cycle may be removed fromfurther analysis and the process 600 may return to block 602 to receive further inertial sensormeasurements acquired by the IMU sensor. It should be noted that more than one material movingcycles is typically identified for further analysis.

[0166] At decision block 618, the method 600 may determine whether a required amount ofposition sensor measurements or data have been received. The required amount may be defined as anamount of data that is sufficient to verity locations of a loading point and / or a dumping point for thematerial moving cycle that involves transfer of material from the loading point to the dumping point. Insome examples, the position sensor measurements may not be available, e.g., due to absence ormalfunction of a position sensor configured to communicate with one or more position beacons or tagsthat are external to the mining machine. The position sensor may be e.g. position sensor 17 shown inFIG.2, which may be part of or associated with the field computer device 16 and / or the machinecontroller 28. Such position sensor may be absent from some mining machines.

[0167] As another possible reason for which sufficient position sensor measurements may not beacquired is that one or more one or more position beacons or tags may be absent or not workingproperly. Various other factors may contribute to the lack of required amount of position sensor data,such as issues with transmission of communication signals between the position sensor on the miningmachine and the one or more position tags, and others.

[0168] At block 607, responsive to determining that the required amount of the position sensormeasurements is received, the method 600 may perform preprocessing of the position sensormeasurements, similar to the preprocessing of the position sensor measurements at block 507 of FIG.5.

[0169] At block 614, the method 600 comprises applying a second trained ML model to theposition sensor measurements to verify the loading point and the dumping point for the identifiedmaterial moving cycle.

[0170] Referring back to decision block 618, regardless of the specific one or more reasons for thelack of the required amount of position sensor data, responsive to determining that the required amountof the position sensor measurements is not received, the process 609 proceeds to block 620 todetermine at least one operation indicator from the identified at least one material moving cycle. In thiscase, a loading point and / or dumping point have not been verified for the material moving site. OtherDocket No.: PS56142PC00 / P23087WO01techniques may be used to verify the loading point and / or dumping point, without the use of positionsensor measurements. For example, cycles can be manually assigned to a load point by a fleetoperator or mine coordinator, or can be manually selected by the operator e.g. on a screen of acomputing device.

[0171] The process 600 also follows to block 620 responsive to determining that the requiredamount of the position sensor measurements is received, in which case the loading point and / or thedumping point have been verified for the material moving site.

[0172] At block 620, the method 600 comprises determining at least one operation indicator fromthe identified at least one material moving cycle. The at least one material moving cycle may comprisetwo or more material moving cycles. The operation indicator may be or may indicate a duration of theidentified material moving cycle, a number of material moving cycles performed by the mining machineduring the first period of time or another period of time, e.g., a shift, a day, a month, a year, or any othertime period. The operation indicator may comprise any suitable one or more performance metricsindicating a status, a need for service, and other characteristics of the mining machine. The processingat block 620 may be performed similarly to the processing at block 320 of FIG.3 and the description atblock 620 is applicable herein.

[0173] At block 622, the method 600 comprises initiating an action in dependence on determiningof the at least one operation indicator and / or in dependence on the identifying of the at least onematerial moving cycle. In some examples, initiating the action may comprise prompting a display of arepresentation of the identified at least one material moving cycle and / or a representation of the at leastone operation indicator. In some examples, initiating the action in dependence on the determining ofthe at least one operation indicator comprises one or more of obtaining an instruction to control themining machine in dependence on the at least one operation indicator, providing the instruction to themining machine, and adjusting a target requirement for performance of the mining machine. Theprocessing at block 622 may be performed similarly to the processing at block 322 of FIG.3 and thedescription at block 322 is applicable herein.

[0174] FIG.7 illustrates an example of a method 700 that may be performed by a control systemsuch as e.g. central control system 22 shown in FIGs.1A and 2. The central control system 22, whichmay be a traffic control system, comprises processing circuitry 24 e.g. at least one processor 24 that isconfigured to coordinate movements of one or more mining machines out of a plurality of miningmachines in a mining site. The one or more of the mining machines e.g. a machine 12 may comprise acorresponding mining machine controller 28 and a field computer device 16 which may be part of themining machine controller 28 or a separate hardware device. The field computer device 16 or themining machine controller 28 may be configured to perform the method in accordance with examples ofDocket No.: PS56142PC00 / P23087WO01the present disclosure. The field computer device 16 and / or the mining machine controller 28 isconfigured to communicate with the central control system 22 via a communication interface, e.g. tosend various data to the central control system 22 and to receive control instructions and informationfrom the central control system 22.

[0175] At block 702, the central control system may receive, from the mining machine,information on the identified at least one material moving cycle and / or on the at least one operationindicator determined based on the identified at least one material moving cycle.

[0176] At block 704, the central control system may generate at least one command or instructionto the mining machine in dependence on the received representation. The instruction may be to controlthe mining machine in dependence on the at least one operation indicator. In some examples, theinstruction may instruct the mining machine to adjusting a target requirement for performance of themining machine.

[0177] At block 706, the control system may send the at least one instruction to the miningmachine. The generating and sending of the instruction may be performed in the same processing stepor block.

[0178] FIGs.8, 9, 10, and 11 illustrate examples of information that can be presented on a userinterface of a computer device, in accordance with examples of the present disclosure. The userinterface may be rendered on a display associated with the mining machine e.g. the field computerdevice and / or the mining machine controller, and / or on a display associated with the central controldevice, e.g., a traffic controller device or system. The information shown in FIGs.8, 9, 10, and 11 maybe generated and presented as a result of identifying of one or more material moving cycle anddetermining at least one operator indicator based on the identified material moving cycle(s). Theinformation may be displayed as part of initiating an action in dependence on determining of the at leastone operation indicator in accordance with any methods of embodiments of the present disclosure. Inthe examples of FIGs.8-11, during each material moving cycle, a bucket of the material is moved.

[0179] FIG.8 illustrates, for a day and for shifts A, B, and C of the day, a number of buckets withthe material that have been moved during a particular shift. In this and other examples, a bucket refersto a completed material moving cycle. The amount of the moved material is also shown in FIG.8, intons.

[0180] FIG.9 illustrates, for a shift A in this example, a total productivity in a number of buckets,per hours. FIG.9 also shows expected production as a line 901 and cumulative production as a dottedline 903.

[0181] FIG.10 illustrates, for a shift A, production per an operator of a corresponding miningmachine in a mine, expressed as a number of buckets moved per a specific operator.Docket No.: PS56142PC00 / P23087WO01

[0182] FIG.11 illustrates details on performance, by time, of a mining machine, as variousparameters and information. Thus, FIG.11 shows a date, equipment, operator, extraction point,dumping or dump point, cycle time, and detail.

[0183] The information that can be displayed on the user interface of the computer device, inaccordance with examples of the present disclosure, may be used to assess performance of the miningmachine, determine further actions to be taken, control the mining machine, adjust various features ofthe mine, and perform any other suitable actions. It should be appreciated that various other informationmay be displayed as examples of FIGs.8-11 are shown for illustration purposes only.

[0184] In an aspect, a kit for installation on a mining machine is provided, the mining machinebeing configured to operate in a mining site. The kit, e.g. kit 18 shown in FIG.1A, may comprise at leastone IMU sensor configured to be associated with mining machine, and computer program productcomprising computer-executable instructions configured to, when executed by at least one processor,to perform any of the methods in accordance with embodiments of the present disclosure. The at leastone processor may be a processor of a field computer device, a processor of a mining machinecontroller, or any other suitable processor.

[0185] The kit may be installed or deployed on the mining machine such that the at least one IMUsensor may be coupled to the mining machine e.g. on a movable implement of the mining machineand / or in another location where the IMU sensor may record inertial sensor measurements indicative ofthe mining machine performing a material moving cycle. In some examples, the at least one IMUsensor comprises one i.e. single IMU sensor unit. Each IMU sensor unit may comprise anaccelerometer, a gyroscope, and a magnetometer. In some examples, the kit may comprise one ormore IMU sensor unit each comprising a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer.

[0186] In some examples, the computer program product of the kit may be installed on the fieldcomputer device and / or the mining machine controller. Thus, the computer-executable instructions maybe stored in memory of a field computer device e.g. field computer device 16 that may be provided aspart of the kit. In some examples, the computer-executable instructions may be stored in memory of acontroller e.g. mining machine controller 28. In some examples, the computer-executable instructionsmay be implemented as an application or app that can be installed in the memory of the miningmachine controller 28.

[0187] In some examples, the computer program product of the kit may be implemented as thefield computer device such that the field computer device may be installed and deployed on the miningmachine in addition to existing one or more controllers of the mining machine.Docket No.: PS56142PC00 / P23087WO01

[0188] The computer-executable instructions, when executed by at least one processor, maycause the at least one processor to, as the mining machine is operating in the mining site, receiveinertial sensor measurements acquired by the at least one IMU sensor associated with the miningmachine, wherein the inertial sensor measurements are acquired over a first period of time; apply a firsttrained machine-learning model to the inertial sensor measurements to identify at least one materialmoving cycle performed by the mining machine within the first period of time; determine at least oneoperation indicator from the identified at least one material moving cycle; and initiate an action independence on determining of the at least one operation indicator.

[0189] In some examples, the material moving cycle comprises a sequence of operating statesthat the mining machine performs during the material moving cycle, the sequence of operating statescomprising loading the mining machine with material at a loading point, moving the material by themining machine from the loading point to a dumping point, unloading the material at the dumping point,and moving from the dumping point to the loading point or to another loading point.

[0190] In some examples, the computer-executable instructions, when executed by at least oneprocessor, further cause the at least one processor to receive, over the first period of time, positionsensor measurements acquired from at least one position beacon positioned in the mining site; andapply a second trained machine-learning model to the position sensor measurements to verify theloading point and the dumping point for the identified at least one material moving cycle. The secondtrained machine-learning model may be trained in dependency on one or more features selected fromtime series data previously acquired by measuring signals from one or more position beacons. The oneor more position beacons may comprise the at least one position beacon.

[0191] In some examples, the computer-executable instructions, when executed by at least oneprocessor, further cause the at least one processor to determine a correctness of the identified at leastone material moving cycle, wherein the at least one material moving cycle is determined to be correctlyidentified when the at least one material moving cycle has a duration that is within a threshold rangeand / or when the at least one material moving cycle is an operational cycle.

[0192] In some examples, initiating the action in dependence on the determining of the at leastone operation indicator comprises prompting a display of a representation of the identified at least onematerial moving cycle and / or a representation of the at least one operation indicator.

[0193] In some examples, initiating the action in dependence on the determining of the at leastone operation indicator comprises one or more of obtaining an instruction to control the mining machinein dependence on the at least one operation indicator; providing the instruction to the mining machine,and adjusting a target requirement for performance of the mining machine.Docket No.: PS56142PC00 / P23087WO01

[0194] In an aspect, a mining machine comprising the kit in accordance with examples of thepresent disclosure is provided.

[0195] In an aspect, a mining machine for operation in a mining site is provided. The miningmachine comprises a movable implement comprising a tool configured to load and unload material; atleast one IMU sensor associated with mining machine and configured to acquire inertial sensormeasurements as the mining machine is operating in the mining site; and at least one processor and amemory comprising computer-executable instructions. The computer-executable instructions, whenexecuted by the at least one processor, cause the at least one processor to, as the mining machine isoperating in the mining site, receive the inertial sensor measurements acquired by the at least one IMUsensor associated with the mining machine, wherein the inertial sensor measurements are acquiredover a first period of time; apply a first trained machine-learning model to the inertial sensormeasurements to identify at least one material moving cycle performed by the mining machine withinthe first period of time; determine at least one operation indicator from the identified at least onematerial moving cycle; and initiate an action in dependence on determining of the at least one operationindicator.

[0196] In some examples, the material moving cycle comprises a sequence of operating statesthat the mining machine performs during the material moving cycle, the sequence of operating statescomprising loading the mining machine with material at a loading point, moving the material by themining machine from the loading point to a dumping point, unloading the material at the dumping point,and moving from the dumping point to the loading point or to another loading point.

[0197] In some examples, the computer-executable instructions, when executed by the at leastone processor, further cause the at least one processor to determine a correctness of the identified atleast one material moving cycle, wherein the at least one material moving cycle is determined to becorrectly identified when the at least one material moving cycle has a duration that is within a thresholdrange and / or when the at least one material moving cycle is an operational cycle.

[0198] In some examples, the computer-executable instructions, when executed by the at leastone processor, further cause the at least one processor to receive, over the first period of time, positionsensor measurements acquired from at least one position beacon positioned in the mining site, andapply a second trained machine-learning model to the position sensor measurements to verify theloading point and the dumping point for the identified at least one material moving cycle.

[0199] In some examples, initiating the action in dependence on the determining of the at leastone operation indicator comprises prompting a display of a representation of the identified at least onematerial moving cycle and / or a representation of the at least one operation indicator.Docket No.: PS56142PC00 / P23087WO01

[0200] In some examples, initiating the action in dependence on the determining of the at leastone operation indicator comprises one or more of obtaining an instruction to control the mining machinein dependence on the at least one operation indicator, providing the instruction to the mining machine,and adjusting a target requirement for performance of the mining machine.

[0201] Operational steps described in any of the exemplary aspects herein are described toprovide examples and discussion. The steps may be performed by hardware components, may beembodied in machine-executable instructions to cause a processor to perform the steps, or may beperformed by a combination of hardware and software. Although a specific order of method steps maybe shown or described, the order of the steps may differ. In addition, two or more steps may beperformed concurrently or with partial concurrence.

[0202] The terminology used herein is for the purpose of describing particular aspects only and isnot intended to be limiting of the disclosure. As used herein, the singular forms “a,” “an,” and “the” areintended to include the plural forms as well, unless the context clearly indicates otherwise. As usedherein, the term “and / or” includes any and all combinations of one or more of the associated listeditems. It will be further understood that the terms “comprises,” “comprising,” “includes,” and / or“including” when used herein specify the presence of stated features, integers, steps, operations,elements, and / or components, but do not preclude the presence or addition of one or more otherfeatures, integers, steps, operations, elements, components, and / or groups thereof.

[0203] It will be understood that, although the terms first, second, etc., may be used herein todescribe various elements, these elements should not be limited by these terms. These terms are onlyused to distinguish one element from another. For example, a first element could be termed a secondelement, and, similarly, a second element could be termed a first element without departing from thescope of the present disclosure.

[0204] Relative terms such as “below” or “above” or “upper” or “lower” may be used herein todescribe a relationship of one element to another element as illustrated in the Figures. It will beunderstood that these terms and those discussed above are intended to encompass differentorientations of the device in addition to the orientation depicted in the Figures. It will be understood thatwhen an element is referred to as being “connected” or “coupled” to another element, it can be directlyconnected or coupled to the other element, or intervening elements may be present. In contrast, whenan element is referred to as being “directly connected” or “directly coupled” to another element, thereare no intervening elements present.

[0205] Unless otherwise defined, all terms (including technical and scientific terms) used hereinhave the same meaning as commonly understood by one of ordinary skill in the art to which thisdisclosure belongs. It will be further understood that terms used herein should be interpreted as havingDocket No.: PS56142PC00 / P23087WO01a meaning consistent with their meaning in the context of this specification and the relevant art and willnot be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0206] It is to be understood that the present disclosure is not limited to the aspects describedabove and illustrated in the drawings; rather, the skilled person will recognize that many changes andmodifications may be made within the scope of the present disclosure and appended claims. In thedrawings and specification, there have been disclosed aspects for purposes of illustration only and notfor purposes of limitation, the scope of the inventive concepts being set forth in the following claims.

Claims

Docket No.: PS56142PC00 / P23087WO01CLAIMSWhat is claimed is:

1. A computer-implemented method (300) for monitoring and controlling operation of a mining machinein a mining site, the method comprising, by a processor:as the mining machine is operating in the mining site, receiving (302) inertial sensormeasurements acquired by at least one inertial measurement unit, IMU, sensor associated with themining machine, wherein the inertial sensor measurements are acquired over a first period of time;applying (312) a first trained machine-learning model to the inertial sensor measurements toidentify at least one material moving cycle performed by the mining machine within the first period oftime; determining (320) at least one operation indicator from the identified at least one materialmoving cycle; andinitiating (322) an action in dependence on determining of the at least one operation indicator.

2. The method according to claim 1, wherein the material moving cycle comprises a sequence ofoperating states that the mining machine performs during the material moving cycle, the sequence ofoperating states comprising loading the mining machine with material at a loading point, moving thematerial by the mining machine from the loading point to a dumping point, unloading the material at thedumping point, and moving from the dumping point to the loading point or to another loading point.

3. The method according to any of claims 1 to 2, comprising:determining (315) a correctness of the identified at least one material moving cycle, whereinthe at least one material moving cycle is determined to be correctly identified when the at least onematerial moving cycle has a duration that is within a threshold range and / or when the at least onematerial moving cycle is an operational cycle.

4. The method according to any of claims 2 to 3, further comprising receiving (503), over the first periodof time, position sensor measurements acquired from at least one position beacon positioned in themining site.

5. The method according to claim 4, comprising applying (514) a second trained machine-learningmodel to the position sensor measurements to verify the loading point and the dumping point for theidentified at least one material moving cycle.Docket No.: PS56142PC00 / P23087WO016. The method according to claim 5, wherein the second trained machine-learning model is trained independency on one or more features selected from time series data previously acquired by measuringsignals from one or more position beacon.

7. The method according to any of claims 1 to 6, wherein the first trained machine-learning model istrained using inertial sensor measurements previously acquired by one or more IMU sensors.

8. The method according to any of claims 1 to 7, wherein initiating (322) the action in dependence ondetermining of the at least one operation indicator comprises, by the processor, prompting a display ofa representation of the identified at least one material moving cycle and / or a representation of the atleast one operation indicator.

9. The method according to any of claims 1 to 8, wherein initiating (322) the action in dependence onthe determining of the at least one operation indicator comprises one or more of:obtaining an instruction to control the mining machine in dependence on the at least oneoperation indicator;providing the instruction to the mining machine; andadjusting a target requirement for performance of the mining machine.

10. A computer device (16, 28) comprising at least one processor (210, 30) that is configured toperform the method according to any one of claims 1 to 9.

11. A computer program product comprising computer-executable instructions, which, when executedby at least one processor, cause the at least one processor to perform the method according to any ofclaims 1 to 9.

12. A tangible computer-readable storage medium, having stored thereon a computer program productcomprising computer-executable instructions which, when executed by at least one processor, causethe at least one processor to perform the method according to any one of claims 1 to 9.

13. A mining machine (12) for operation in a mining site, the mining machine (12) comprising:a movable implement (15) comprising a tool configured to load and unload material;at least one inertial measurement unit, IMU, (14) sensor associated with mining machine andconfigured to acquire inertial sensor measurements as the mining machine is operating in the miningsite; andat least one processor (210, 30) and a memory (220, 31) comprising computer-executableinstructions that, when executed by the at least one processor, cause the at least one processor to:Docket No.: PS56142PC00 / P23087WO01as the mining machine is operating in the mining site, receive the inertial sensormeasurements acquired by the at least one IMU sensor, wherein the inertial sensormeasurements are acquired over a first period of time;apply a first trained machine-learning model to the inertial sensor measurements toidentify at least one material moving cycle performed by the mining machine within the firstperiod of time;determine at least one operation indicator from the identified at least one materialmoving cycle; andinitiate an action in dependence on determining of the at least one operation indicator.

14. The mining machine (12) according to claim 13, wherein the material moving cycle comprises asequence of operating states that the mining machine performs during the material moving cycle, thesequence of operating states comprising loading the mining machine with material at a loading point,moving the material by the mining machine from the loading point to a dumping point, unloading thematerial at the dumping point, and moving from the dumping point to the loading point or to anotherloading point.

15. The mining machine (12) according to any of claims 13 to 14, wherein the computer-executableinstructions, when executed by the at least one processor, further cause the at least one processor todetermine a correctness of the identified at least one material moving cycle, wherein the at least onematerial moving cycle is determined to be correctly identified when the at least one material movingcycle has a duration that is within a threshold range and / or when the at least one material moving cycleis an operational cycle.

16. The mining machine (12) according to any of claims 14 to 15, wherein the computer-executableinstructions, when executed by the at least one processor, further cause the at least one processor to:receive, over the first period of time, position sensor measurements acquired from atleast one position beacon positioned in the mining site; andapply a second trained machine-learning model to the position sensor measurements toverify the loading point and the dumping point for the identified at least one material moving cycle.

17. The mining machine (12) according to any of claims 13 to 16, wherein initiating the action independence on the determining of the at least one operation indicator comprises prompting, by the atleast one processor, a display of a representation of the identified at least one material moving cycleand / or a representation of the at least one operation indicator.Docket No.: PS56142PC00 / P23087WO0118. The mining machine (12) according to any of claims 13 to 17, wherein initiating the action independence on the determining of the at least one operation indicator comprises one or more of:obtaining an instruction to control the mining machine in dependence on the at least oneoperation indicator;providing the instruction to the mining machine; andadjusting a target requirement for performance of the mining machine.

19. A kit (18) for installation on a mining machine (12) configured to operate in a mining site, the kitcomprising: at least one inertial measurement unit, IMU, (14) sensor configured to be associated withmining machine;computer program product comprising computer-executable instructions configured to beinstalled on the mining machine, the computer-executable instructions, when executed by at least oneprocessor, cause the at least one processor to:as the mining machine is operating in the mining site, receive inertial sensormeasurements acquired by the at least one IMU sensor associated with the mining machine,wherein the inertial sensor measurements are acquired over a first period of time;apply a first trained machine-learning model to the inertial sensor measurements toidentify at least one material moving cycle performed by the mining machine within the firstperiod of time;determine at least one operation indicator from the identified at least one materialmoving cycle; andinitiate an action in dependence on determining of the at least one operation indicator.

20. The kit (18) according to claim 19, wherein the material moving cycle comprises a sequence ofoperating states that the mining machine performs during the material moving cycle, the sequence ofoperating states comprising loading the mining machine with material at a loading point, moving thematerial by the mining machine from the loading point to a dumping point, unloading the material at thedumping point, and moving from the dumping point to the loading point or to another loading point.

21. The kit (18) according to any of claims 19 to 20, wherein the computer-executable instructions,when executed by the at least one processor, further cause the at least one processor to:Docket No.: PS56142PC00 / P23087WO01receive, over the first period of time, position sensor measurements acquired from atleast one position beacon positioned in the mining site; andapply a second trained machine-learning model to the position sensor measurements toverify the loading point and the dumping point for the identified at least one material moving cycle.

Images