Autonomous robotic system and method for identifying, tracking and impacting rocks for mineral grinding

Abstract

The present invention relates to the mining field and, in particular, to an autonomous robotic system and a method for identifying, tracking and selectively impacting rocks that improves mineral grinding efficiency. In particular, the present invention relates to a kit that can be coupled to rock drill machines, enabling them to operate autonomously in low-visibility or limited-visibility mining environments.

Description

[0001] AUTONOMOUS ROBOTIC SYSTEM AND METHOD FOR IDENTIFICATION, TRACKING AND IMPACT OF ROCKS FOR MINING GRINDING

[0002] FIELD OF INVENTION

[0003] The present invention relates to the field of mining, and in particular, to an autonomous robotic system and method for the identification, tracking, and selective impact of rocks, which improves the efficiency of mining operations. Specifically, the present invention relates to a kit attachable to standard rock-breaking hammers or rock-breaking equipment, enabling them to operate autonomously in mining environments with low visibility or poor visibility.

[0004] BACKGROUND

[0005] For several years, large-scale mining has been successfully managing operating costs, focusing on two crucial aspects: increasing productivity in both labor and capital, and improving processes, where innovation has been a key factor. Crushing and grinding play a significant role in operational costs due to their high energy consumption and the inputs required for crushing and grinding media, representing between 30% and 50% of a mining project's operational costs. In particular, a lack of autonomy has been identified in the processes prior to grinding, specifically in the material crushing processes, leading to downtime, accumulations, and blockages caused by rocks that have not been adequately reduced.

[0006] In the processes prior to grinding, a human operator typically controls an industrial piece of equipment called a "rock-breaker arm" or "rock pick." The operator manually observes the stationary or moving rocks and, using a console with buttons and joysticks, positions the arm, assigning the hammer the correct orientation for impact. However, the operation is hampered by the operator's difficulty in visualizing the cohesive solids in environments with suspended particles or inadequate lighting conditions, causing excessive heat and wear on the equipment's steel tip. Furthermore, there is a lack of autonomy in the processes prior to grinding, especially in crushing, leading to downtime and blockages that negatively impact operational efficiency.

[0007] The accuracy of the rock-picking hammer's point position and its angle of incidence are significantly affected by the operator's difficulty in visualizing the cohesive solids due to various environmental factors such as dust, fog, shadows, sunlight, and darkness. Consequently, this manual procedure must be repeated until the solid fractures, with periods of inactivity, generating excessive heat and wear on the rock-picking hammer's point (energy consumption). This presents a challenge to improve and enhance comminution processes in the mining industry.

[0008] Data fusion is defined as the process of combining information from diverse sources and times to facilitate decision-making, whether manual or automated (T. Meng, X. Jing, Z. Yan, and W. Pedrycz, “A survey on machine learning for data fusion,” Information Fusion, vol. 57, pp. 115-129, 2020. [Online], Available: https: / / www.sciencedirect.com / science / article / pii / S1566253519303902). These approaches often involve combining datasets with different spatial resolutions, with the goal of creating a unified dataset that possesses the finest resolution (P. Ghamisi, B. Rasti, N. Yokoya, Q. Wang, B. Hofle, L. Bruzzone, F. Bovolo, M. Chi, K. Anders, R. Gloaguen, PM Atkinson, and JA Benediktsson, “Multisource and multitemporal data fusion in remote sensing: A comprehensive review of the state of the art,” IEEE Geoscience and Remote Sensing Magazine, vol. 7, pp. 6-39, 2019).In multisensor systems, the integration of data from multiple sensors is crucial to reducing detection uncertainty, something that cannot be achieved with individual sensors. This strategy is vital for developing robust models that can perceive the environment under diverse environmental conditions, and is essential for applications such as autonomous vehicles and other automated systems (DJ Yeong, G. Velasco-Hernandez, J. Barry, and J. Walsh, “Sensor and sensor fusion technology in autonomous vehicles: A review,” Sensors, vol. 21, no. 6, 2021. [Online], Available: https: / / wv^j;nd. / 1424- 8220 / 21 / / 2140, and Y. Zhang, D. Sidib'e, O. Morel, and F. M'eriaudeau, “Deep multimodal fusion for semantic image segmentation: A survey,” Image and Vision Computing, vol. 105, p. 104042, 2021. [Online], Available: https: / / www.sciencedirect.com / science / article / pii / S0262885620301748).

[0009] In particular, when a data fusion system considers different types of sensors, it is known as multimodal data fusion. A modality refers to the acquisition of information from a given system through a specific mode or channel. Each modality is characterized by having a temporality and a dimensionality. Dimensionality is classified as 1D (one-dimensional data), 2D (two-dimensional data, for example, an image), or 3D (three-dimensional data, such as a point cloud). Regarding temporality, data can be classified as static (data without a temporal designation) or time series (for example, samples taken at specific time intervals, as in a video) (Y. Zhang, D. Sidib'e, O. Morel, and F. M'eriaudeau, “Deep multimodal fusion for semantic image segmentation: A survey,” Image and Vision Computing, vol. 105, p. 104042, 2021. [Online], Available: https: / / ). R. Bokade, A.

[0010] Navato, R. Ouyang, X. Jin, C.-A. Chou, S. Ostadabbas, and AV Mueller, “A cross-disciplinary comparison of multimodal data fusion approaches and applications: Accelerating learning through trans-disciplinary information sharing,” Expert Systems with Applications, vol. 165, p. 113885, 2021. [Online], Available:

[0011] Representing between 30 and 50% of mining operational costs, comminution (the process of reducing solid materials through crushing, grinding, cutting, among others) has been a significant economic challenge for mining projects due to its high costs, including energy expenditure, and the critical importance of grinding media.

[0012] In the crushing stage, a crucial step in comminution, an operator manually positions a rock-breaker hammer on the rocks to be fragmented before grinding. Precise hammer placement and the correct angle of impact are critical and are significantly affected by factors such as the operator's location, the presence of airborne particles, and variations in ambient light.

[0013] In the prior art, bottomhole systems are known, such as the system proposed in USNo. 2022 / 003059A1 (Schlumberger Technology Corp.), which corresponds to a system that automates a hydraulic fracturing process by automating the pump rate, seeking to reduce downtime. This system comprises: processors and memories; a data interface coupled to pumps and receiving data from sensors, including a pump discharge pressure transducer and a pump rate sensor; a control interface for the pumps; a modeling component, coupled to the processors, that predicts the pressure in a well, a predicted pump rate, and at least some indicative data of the actual pump rate and an estimate of the wellhead pressure, where the well is fluidly coupled to the pumps; and a pump rate adjustment component, coupled to the processors, which, in a predicted pressure mode,It generates a pumping rate control signal that is transmitted through the control interface, using the predicted pressure from the modeling component and a pressure threshold. The data indicative of the actual pumping rate comprises the pumping rate sensor data. The system also includes a head pressure estimation component coupled to the processors, which receives, via the data interface, the data acquired by the pump discharge pressure transducer to generate a head pressure estimate; an interface that receives pressure and pumping rate data, where the pumping rate adjustment component, in an alternative mode,It generates the pumping rate control signal using pressure and pumping rate data without using predicted pressure; a model update component receives input data for the modeling component and receives the pumping rate control signal and uses it to determine the accuracy of the pumping rate control signal and updates the model based at least in part on the determined accuracy. The modeling component comprises a pressure friction model that predicts a friction pressure that is a function of the pumping rate and the fluid's friction property, and that receives the pumping pressure data and updates the pressure friction model.where the pumping pressure data corresponds to an operation pumping a drilling unit into an underground tubular. The pressure friction model uses the instantaneous “shut-in” pressures for one or more hydraulic fracturing stages to determine one or more friction pressures. The system further comprises a component that uses the friction pressures to fit a friction pressure curve and estimate: a friction pressure for a subsequent hydraulic fracturing stage or a bottomhole pressure; a component that analyzes the bottomhole pressure to determine treatment abnormalities and another to determine an indication of screenout; a component that uses the indication of screenout to generate a pumping rate control signal transmitted via a control interface to change a pumping rate; and a pressure change rate component.operationally coupled to at least one of the processors that generates a pressure change rate using wellhead pressure and historical pressure data and pressure change rate outputs for the modeling component that generates the predicted pressure using the pressure change rate; a cluster component that generates fracture cover estimates that depend on one or more operational parameters. The pump rate control signal generated by the pump rate tuning component is implemented so that it depends on the fracture cover estimates or run-in test data. The cluster component analyzes pressure and flow rate to estimate at least one perforation dominance or tortuosity dominance of a fracturing operation. The fracture cover estimate comprises estimates for releasing fluid via perforations in a single-stage, multi-stage hydraulic fracturing operation.

[0014] WO2012039666A1 (Spc Technology Ab) refers to a method for monitoring percussion (hammer) drilling in a well, comprising: detecting or estimating a control frequency or frequencies representative of the hammer percussion frequency; and creating a distribution representation thereof to produce a response to an adjustment of at least one drilling parameter, which may be made, for example, in the direction of the narrowest width of the distribution; and where parameter is one of: bit feed force / weight, feed rate, rotational speed, rotational torque, percussion fluid flow pressure, flushing fluid flow pressure, where the control frequency or frequency is / are sampled to form a representation covering a desired or predetermined time period, and where the adjustment results in widening or narrowing the width of the distribution, and is maintained or prolonged as required by the adjustment.The control frequency(ies) is / are detected outside the borehole, on the drilling rig, in a drill string, or in the ground or air adjacent to it. The percussion frequency spread can be visualized to aid manual tuning. The method further comprises creating a representation of the amplitude of this frequency distribution to produce a response to drilling parameter adjustment as a change in amplitude magnitude. The amplitude is sampled to form a representation covering a desired or predetermined time period. The resulting percussion frequency and / or amplitude data are recorded and stored so that they can be read later as drilling characteristics.The device associated with the method described above comprises: a sensor for detecting or estimating the frequency(ies) that is / are representative of the hammer percussion frequency and a representation means for creating the representation of a frequency distribution spread or frequency control to produce a response to an adjustment of at least one drilling parameter.

[0015] US20220307365A1 (King FAHD University of Petroleum & Minerals) refers to a method for automatic optimization of a rate of penetration, comprising: obtaining, by a computational processor, a plurality of drill surface parameters for a field of interest; identifying, by a computational processor, undefined compressive strength (UCS) data for a target formation based on well logs; calculating, by a computational processor, mechanical specific energy (MSE) data based on the identified UCSs; filtering, by a computational processor, the calculated MSE data based on the identified UCSs data with a small range for the target formation of interest; training, by a computational processor, a machine learning model that uses the drill surface parameters as inputs; and generating an output, by a computational processor,to determine, by a computational processor, a plurality of weights for drilling parameters in a ROP equation derived from the use of a machine learning model, where the drilling surface parameters are used as inputs; to determine, by a computational processor, a plurality of weights for drilling parameters in a Teale's MSE equation for the field of interest, where the drilling surface parameters are used as inputs; to generate, by a computational processor, a plurality of weights for drilling parameters in the Teale's MSE equation for the field of interest, where the drilling surface parameters are used as inputs; to combine, by a computational processor, the machine learning ROP equation with the Teale's MSE equation to form a set of 2 equations; to determine, by a computational processor,A plurality of optimal drilling parameters are determined by simultaneously solving the ROP (machine learning) equation and Teale's MSE equation. A work order is then generated by a computational processor to adjust the drilling parameters based on the determined optimum drilling parameters and previous drilling parameters. Finally, the work order and the determined optimum drilling parameters are displayed on a client device's user interface. The drilling surface parameter is selected from UCS (Unified Drilling System) and MSE data, including torque, revolutions per minute (RPM), weight on bit (WOB), pumping rate (GPM), and standpipe pressure (SPP). The machine learning algorithm is selected from the following options: linear regression, logistic regression, support vector regression, random forest, and decision tree-driven regression.Multilayer perceptron and convolutional neural network. The output weight in the machine learning ROP equation is a value for a selected drilling surface parameter from UCS, MSE, torque, RPM, WOB, GPM, and SPP. The model's output weight in the MSE equation is a value for a selected drilling surface parameter from UCS, MSE, torque, RPM, WOB, GPM, and SPP. It also refers to a system for automatic ROP optimization, comprising an Al module. W02001021927A2 (Vermeer Manufacturing Company) refers to an excavation system, comprising: a cutting tool propelled by a drill pipe that excavates a well; a mud system that pumps drilling fluid through the drill pipe,transporting the drilling fluid and the excavated material out of the well; and a return mud sensor that detects a property of the drilling fluid exiting the well. The property of the drilling fluid exiting the well comprises the particle size in the excavated material, the viscosity of the drilling fluid exiting the well, the density of the drilling fluid exiting the well, or the composition of the drilling fluid exiting the well. Additionally, the system comprises a controller communicatively coupled to the return mud sensor, the mud system, and the drive apparatus, the controller modifying the movement of the cutting tool in response to the drilling fluid property detected by the return mud sensor.and where the controller modifies the movement of the cutting tool based on: the transport rate of the excavated material out of the well; a percentage of solids in the drilling fluid leaving the well; the excavation rate of the cutting tool not exceeding the transport rate of the excavated material out of the well; or a property of the drilling fluid delivered to the well. The controller can modify a property of the drilling fluid pumped through the drill pipe in response to a property of the drilling fluid detected by the return mud sensor. The property of the drilling fluid pumped through the drill pipe comprises one or more of a flow rate, pressure, temperature, viscosity,The controller modifies the rate at which the drilling fluid is pumped through the drill pipe based on changes in the volume of the drilled hole. Optional systems and associated drilling methods are also taught.

[0016] Furthermore, in modern mining, rock crushers constitute the technical and / or administrative boundary between the mine process and the processing plant. In some cases, the rock crushers are assigned to the mine process, while in others, they are assigned to the processing plant. The mine has a responsibility boundary regarding the quality of the rock entering the plant, specifically the particle size distribution of the ore it sends to the processing plant. From this perspective, the first stage of reduction, the rock crusher, belongs to the mine. In other cases, this first stage of reduction is assigned to the processing plant, which is best positioned to understand and define the particle size distribution requirements for the ore entering primary crushing.

[0017] In any scenario, the rock crusher is the first and most crucial piece of equipment that performs initial size reduction and is received by the primary crusher in the plant if it meets the required particle size distribution. Therefore, the performance of the rock crusher directly impacts the production of the mine or the mine-plant system and is a key factor in increasing system productivity. Continuous optimization of its efficiency, productivity, energy consumption, and ultimately, its mechanical reliability—which extends to the entire system—is essential. It is important to note that if oversized rock enters the crushing system, it could cause damage and, consequently, unscheduled shutdowns, resulting in significant economic losses.

[0018] The use of collaborative robots in mining is expected to assist in solving problems that previously required high economic costs and posed risks to personnel. In fact, the mining industry is currently implementing various types of machinery, such as remotely controlled grinding processes using neural networks; trucks that, through Machine Learning (ML), optimize travel times when transporting materials and have information about potential obstacles along the route to the desired destination; and others. In this way, robotization in the mining industry is generating multiple benefits: reduced operator exposure risk, as they work in a safe zone; minimized unscheduled plant shutdowns, enabling continuous operation; and reduced costs, since the same work is done in less time, transforming these operators into process supervisors.

[0019] The article “Energetically Optimal Trajectory for a Redudant Planar Robot by Means of a Nested Loop Algorithm”, by H. Potter et al., April 28, 2022, DOI: 10.5755 / j001.eie.30397, Ref: Web of Science (Wos), proposes an energetically optimal calculation algorithm for generating robotic arm trajectories, considering the robot's starting and ending positions. It proposes an optimal point-to-point trajectory based on four-degree polynomials to reduce the energy consumption of a redundant planar robot in the XY plane. The algorithm optimizes the optimal weight vector, which determines the influence of each point on the total energy consumption of the robotic equipment; the optimal starting and ending configurations of the trajectory; and the vector corresponding to the fifth coefficient of the trajectory generator polynomial.Based on the objective function associated with energy consumption, the best combination is determined to achieve the minimum possible computation time to save energy. The energy performance is compared with its non-redundant version, which limits operation to only two degrees of freedom, in all variables.

[0020] The article “Data fusion in Wireless Sensor Network: An Overview”, Quintero et al., September 8, 2022, Journal of Engineering and Research, This article presents a review of various multimodal Data Fusion (DF) applications and highlights the types of methods used in perturbation-intensive developments. Due to the massive increase in datasets in current applications and the difficulty of processing them in various approximations, data fusion is the predominant path to achieving outstanding results in terms of reliability, efficiency, and accuracy. This review of wireless sensor networks applies various concepts and provides a comprehensive and up-to-date overview of models or architectures, techniques, methodologies, or algorithms, and a review of their applications in different areas. “Fuzzy Control Strategies Development for a 3-Daf Robotic Manipulation in Trajectory Tracking.” Processes 11, No. 12 (November 2023): 3267, https: / / das.org / IO.3390 / grll123267, J. Kern et al., 22 November 2023, DOI: https: / / doi.org / 10.3390 / pr11123267, Ref: Web of Science (Wos), which is incorporated into this application by reference, refers to the development and evaluation of two different controllers for a robotic arm and presents two primary control strategies. The first strategy is a Fuzzy Logic Controller that uses the position error and its derivative as inputs, employing a set of nine control knowledge rules. The second strategy is an adaptive Neuro-fuzzy Inference System Controller, trained to learn the inverse dynamic model of the robot through a structured dataset. The importance of careful parameter tuning and data acquisition to achieve optimal control system performance is highlighted. Experiments were conducted to evaluate the controllers' performance in trajectory tracking and response to external disturbances, such as load variations.The controllers exhibited outstanding accuracy and performance in tracking reference paths with minimal deviations, overshoots, or oscillations. A quantitative analysis using performance indices such as root mean square error (RMSE) and integral of the absolute value of the time-weighted error (ITAE) further confirmed the controllers' effectiveness. The ANFIS controller significantly outperformed the Diffusion Logic Controller, demonstrating superior path-tracking accuracy. This highlights the importance of selecting the direct control method and obtaining high-quality training data.

[0021] “Development of an Algorithm for Volumetric Reconstruction and Estimation of the Center of Mass of Solid Cohesive in Environments with Suspended Particles.” International Journal of Electrical and Electronics Engineering, Volume 10(12), V 0Í 2P 01, December 23, 2023, Ref.: Scopus, which is incorporated by reference, refers to volumetric reconstruction and techniques for extracting a point cloud from a scene, with stereo vision being the most widely recognized. Stereo vision uses two images of a set to calculate the distance of objects from the cameras; however, it has been used in clean environments with controlled lighting. In summary, an algorithm is presented that can estimate a point cloud of an object in a cloudy environment with suspended particles to subsequently approximate the entity's center of mass for mining comminution applications.

[0022] “Fast Rock Detection in Visually Contaminated Mining Environments Using Machine Learning and Deep Learning Techniques”, R, Rodríguez-Guillen et al, Appl. Sci. 2024, 14, 731. The referenced document mentions that advances in machine learning algorithms have led to a boom in object detection and classification. Object detection, such as rock detection, in mining operations is affected by fog, snow, suspended particles, and bright light. These environmental conditions can halt mining operations, resulting in a considerable increase in operating costs. It is therefore noted that selecting a machine learning algorithm that is careful, fast, and helps reduce operating costs due to the aforementioned environmental factors is vital. The document analyzes the Viola-Jones, Aggregate Channel Features (ACF), Faster Regions with Convolutional Neural Networks (Faster R-CNN), Single-Shot Detector (SSD), and You Only Look Once (YOLO) version 4 algorithms, considering metric accuracy, AP50, and average detection times.It indicates that, in preliminary tests, differences were observed between YOLO v4 and the latest versions that proved substantial for the specific problem of rock detection. YOLO v4 proved to be an appropriate and representative selection for evaluating the effectiveness of the methods tested, being the best algorithm, while the SSD algorithm was the fastest. The YOLO v4 algorithm stood out as a promising candidate for detecting rocks with visual contamination in mining operations.

[0023] “A Novel Robotic Controller Using Neural Engineering Framework- Based Spiking Neural Networks”, Marrero D. et al, Sensors 2024, 24, 491. fes.. / / dpiprg / 10 .^ January 12, 2024, https; This work, which is incorporated by reference, reviews Spiking Neural Networks (SNNs) for robotic controllers with the aim of improving the accuracy of trajectory tracking. By emulating the operation of the human brain through the incorporation of temporal encoding mechanisms, SNNs offer greater adaptability and efficiency in information processing and significant advantages in representing the temporal control information of the robotic arm compared to conventional neural networks. Specific implementations of SNNs in robot control are explored, and neural models and learning mechanisms inherent to SNNs are analyzed. Based on the principles of the Neural Engineering Framework (NEF), a spiking PID controller for a 3-DoF robotic arm is designed and simulated using Ñengo and MATLAB R2022b. The controller demonstrated good accuracy and efficiency in trajectory design, showing minimal deviations, overshoots, or oscillations.A comprehensive quantitative evaluation, using metrics such as root mean square error (RMSE) and integral of the time-weighted absolute error (ITAE), validated the effectiveness of the SNN-based controller. Competitive performance was observed, surpassing a fuzzy controller by 5% in terms of ITAE and a conventional PID controller by 6% in ITAE and 30% in RMSE performance. The usefulness of NEF and SNN in the development of effective robotic controllers is highlighted, and the adaptability of SNNs in dynamic environments and advanced robotics applications is confirmed. “Enhancing 3d rock localization in mining environments using bird's-eye view images from the time-offlight blaze 101 camera”, J. Kern et al., Technologies, vol. 12, no. 9, 2024. [Online], Available: https: / / www.mdpi.eom / 2227-7080 / 12 / 9 / 162, incorporated by reference, presents the design and implementation of a robust rock centroid localization system for robotic mining applications, particularly rock breakers. The system comprises three phases: assembly, data acquisition, and data processing. Environmental detection was achieved using a Basler Blaze 101 time-of-flight (ToF) three-dimensional (3D) camera. The data processing phase incorporated advanced algorithms, including Bird's-Eye View (BEV) image conversion and You Only Look Once (YOLO) v8x-Seg instance segmentation. System performance was evaluated using a comprehensive dataset of 627 point clouds, including samples from real-world mining environments. The system achieved efficient processing times of approximately 5 seconds. Segmentation accuracy was evaluated using intersection over union (loU), achieving 95.10%.Localization accuracy was measured using the Euclidean distance in the XY plane (EDXY), reaching 0.0128 m. The normalized error (enorm) on the X and Y axes did not exceed 2.3%. Furthermore, the system demonstrated high reliability with R² values ​​close to 1 for the X and Y axes, maintaining performance under various lighting conditions and in the presence of airborne particles. The mean absolute error (MAE) on the Z axis was 0.0333 m, addressing the challenges in depth estimation. A sensitivity analysis was performed to assess the model's robustness, revealing consistent performance across brightness and contrast variations, with a loU ranging from 92.88% to 96.10%, while also showing increased sensitivity to rotations.

[0024] But there is still a need for a robotic system or kit that can be attached to a standard rock-picking hammer and method that allows autonomously: the identification, tracking and selective impact of rocks to improve operations in mining mills, being able to calculate and track trajectories on fixed and moving rock surfaces, identifying edges and critical points to impact on said rock generating the corresponding fracture.

[0025] BRIEF DESCRIPTION OF THE INVENTION

[0026] The present invention relates to an autonomous robotic system that, through machine learning algorithms, sensor fusion, and volumetric reconstruction, performs the identification, tracking, and selective impact of a rock-breaking hammer or rock-breaking equipment on isolated rocks, stacked rocks, or separated and adjacent rocks, improving operations in mining mills. This system is coupled to the rock-breaking equipment and associated method. The present autonomous robotic system is capable of calculating and tracking trajectories on fixed and moving surfaces or objects (rocks), identifying edges and critical points for impact on said object or rock, generating the corresponding fracture.The present autonomous robotic system is installed / attached to a "rock-picking" machine or equipment, increasing the accuracy of the position and angle of incidence of the equipment to impact properly in complex visibility conditions of the environment: suspended dust, fog, shadows, sun glare, darkness, among others.

[0027] The present advanced sensing system or attachable autonomous robotic system comprises a multimodal vision subsystem that includes a fusion of vision sensors, where the signals from the vision sensors are fused in the data processing to generate the volumetric reconstruction of a rock or cohesive / stacked rocks or separated and close rocks, even in a complex environment that may include one or more of the following characteristics: poor lighting, suspended particles, high humidity, among others, where in addition an intelligent control algorithm implemented in a high-performance computer allows reconfiguration in line, of an articulated electro-mechanical subsystem attached to a rock-breaker hammer, which by means of actuators or hydraulic cylinders drives the arm of the rock-breaker hammer moving it to an impact position where the hammer is placed on the surface of the rock to fracture it.The control systems receive both processed information about the volumetric reconstruction of the rock from the high-performance computer where machine learning is developed, and direct information about the position of the "rock-picking" arm, which comes from the information received from the joint sensors attached to the "rock-picking" arm.

[0028] Volumetric reconstruction of isolated rocks, stacked cohesive rocks, or separated but adjacent rocks is performed based on information received from the fusion of vision sensors, comprising two or more 3D vision sensors or 3D cameras, thermal imaging equipment, particle sensors, radar sensors such as RADAR, optical sensors such as LiDAR, and others. The center of mass of a single rock, stacked rock, or separated but adjacent rock can also be estimated even in dust-clouded environments using a volumetric reconstruction algorithm, preferably a stereoscopic vision volumetric reconstruction algorithm.

[0029] An algorithm for detecting the optimal impact point for rock fracture performs the operation of detecting the optimal impact point, preferably being an Optimal Load Distribution Polyhedron (PDOC) algorithm based on the support points, which have been previously estimated for said rocks.

[0030] The present autonomous robotic system for the identification, tracking, and selective impact of rocks in real time, installable / attachable to a rock-picking equipment using machine learning algorithms, sensor fusion, and volumetric reconstruction, allows the autonomous operation of rock-picking hammer equipment in low-visibility environments and is useful in mining activities related to rock operations such as mining grinding, for example, in primary crushing operations.

[0031] The present autonomous robotic system is installed on a "rock-picking" machine, increasing the accuracy of the position and angle of incidence of the machine to impact properly in complex environmental visibility conditions: suspended dust, fog, shadows, sunlight, darkness, among others.

[0032] This autonomous robotic system is capable of calculating and tracking trajectories on stationary or moving rock surfaces in real time, identifying edges and critical points for impact and fracturing. BRIEF DESCRIPTION OF THE FIGURES

[0033] Figure 1 shows a standard operating scheme of a “rock-picking” machine in mining.

[0034] Figures 2A-2D show a rock comminution scheme using the present autonomous robotic system with machine learning-based control via volumetric reconstruction. In a low-visibility environment due to airborne particles, the system uses environmental features gathered by a fusion of vision sensors to control the joints of the rock-breaker arm. This control is achieved through a controller that actuates an actuator, enabling the impact of the rock-breaker hammer on the rock. The system takes into account information from the joint sensors located on the rock-breaker arm. See Figures 2A-2C. Figure 2D shows a schematic of the location of the joint sensors (or position sensors) and hydraulic cylinders on the rock-breaker arm.

[0035] Figures 3A-3F show impacts of the hydraulic hammer equipment, on rock, (Figs. 3C-3F) from the autonomous robotic system of the present invention, in a mining environment without visibility difficulties (Figs. 3A-3B).

[0036] Figures 4A-4E show impacts of the hydraulic hammer equipment, on rock, (Figs. 4C-4E) from the autonomous robotic system of the present invention, in a mining environment of low visibility due to suspended particles (Figs. 4A-4B).

[0037] Figures 5A-5F show arrangements of stacked rocks (a), rocks together (b), and rocks alone (c) reconstructed by the multimodal data fusion system that detects cohesive solids in adverse conditions using machine learning techniques (see Fig. 5A). Figure 5B shows the interconnection network with a mining environment communication protocol between the sensors and a central computer (see Fig. 5B). Figure 5C shows the data captured by four sensors: (a) BLAZE 101 ToF camera (BASLER), (b) OSO LiDAR (OUSTER), (c) MRS6000 LiDAR (SICK), and (d) MRS1000 LiDAR (SICK). Figure 5D shows an array of six sensors, a switch, and a computer forming a centralized network (see Fig. 5E). Figure 5F shows the array's autonomy.

[0038] Figures 6A-6D show an image composed of a point cloud captured from stacked cohesive solids or piled rocks (Fig. 6A). These points are obtained using two industrial 3D cameras in a dual stereoscopic configuration. The image itself is constructed by converting the point cloud into a BEV (Bird's-Eye View) image. Finally, deep learning algorithms are applied to segment the image, that is, to detect the rocks and their overlap. A table is also shown that quantifies the overlap levels of the rocks that make up the rock pile (Figures 6B-6D). This quantification uses values ​​between 0 and 1, where 0 indicates no overlap (for example, a rock with reference to itself) and 1 indicates complete overlap (for example, one rock completely on top of another).The table numerically establishes the overlap levels of the rocks (belonging to the pile), to inform the intelligent algorithm that operates the "rock-picking" machine and, in this way, generate the orders or commands to unstack or disassemble the pile of rocks to continue with the fracturing process.

[0039] Figures 7A and 7B show comminuted rock with the present autonomous robotic system.

[0040] Figure 8 shows a screen display of the images collected by the image-collecting sensors.

[0041] DETAILED DESCRIPTION OF THE INVENTION

[0042] This system improves the crushing stage, which is essential for adjusting the size of the rocks before they enter the grinding mill. By refining this process, the system ensures that the rocks reach the necessary dimensions for efficient mineral liberation during grinding. Through the identification, tracking, and selective impact of rocks, downtime caused by accumulations and blockages of rocks that have not been effectively fractured during crushing is minimized. Furthermore, energy consumption associated with repetitive impacts in manual operations can be reduced.

[0043] The present autonomous robotic system attachable to a rock-picking hammer equipment allows, in real time, the identification, tracking and selective impact of rocks through machine learning algorithms, sensor fusion, and volumetric reconstruction, improving operations in mining mills.

[0044] The present autonomous robotic system attachable to a rock-picking hammer unit, which allows, in real time, the identification, tracking, and selective impact of rocks, whether isolated, coupled, stacked, or separated and close together, and to fracture them efficiently and accurately, comprising: a) a vision subsystem comprising two or more vision sensors selected from the group consisting of 3D vision sensors or 3D cameras, infrared vision cameras, thermal vision equipment, particle vision equipment, radar sensors such as RADAR, optical sensors such as LiDAR, among others, and metallic structures, preferably metallic stands, on which said vision sensors are fixed at a high level by means of conventional fixing, and thus achieving visualization from different angles of the rock, stacked cohesive rocks, or separated and close together rocks,and such metallic structures being located outside the reach of the rock-picking hammer equipment; b) a data processing subsystem comprising a high-performance computer that communicates with said vision sensors of the vision subsystem by means of mining environment or embedded communication protocols and comprising software for the pre-treatment and fusion of the data from the vision sensors and a library that stores the pre-treated and fused data, and a Deep Learning algorithm performs the volumetric reconstruction of the rock, stacked rocks or separated and close rocks, preferably performing a stereoscopic vision reconstruction; c) an articulated sensing subsystem or articulated electro-mechanical subsystem that allows the continuous evaluation of the position of the rock-picking equipment arm,comprising four positional or joint sensors which, in addition to the aforementioned adverse environmental conditions, can withstand the vibration of the rock-picking equipment and the impacts of rocks projected at high speeds after the hammer strike, where a first positional sensor is located at the base of the motor-first link joint of the rock-picking equipment arm; a second positional sensor is located at the joint between the first and second links or intermediate link of the rock-picking equipment arm; and a third positional sensor is located at the joint between the intermediate link and the third link or free end of the rock-picking equipment arm; and a fourth positional sensor is located at the base of the rock-picking equipment motor, c.1) providing said first, second, and third positional sensors, in conjunction,the joint position data of the “rock picker” equipment established by direct measurement of the displacement of the links of the arm of the “rock picker” equipment and measurement of the tilt angle at the base of the motor-first link joint of the arm of the “rock picker” equipment, wherein the first, second, and third position sensors are linear sensors and each is located inside a hydraulic cylinder or actuator half, protecting it, and each actuator half acts on a link of the arm of the “rock picker” equipment while the fourth hydraulic cylinder with a rotational sensor acts on the end of the arm of the “rock picker” equipment that is joined to the motor of the “rock picker” equipment or base, rotating the arm of the “rock picker” equipment around said base, or c.2) providing said first, second, third, and fourth position sensors, in conjunction,The data on the joint position of the "rock pick" equipment is established by measuring the angle of inclination of each joint of the arm of the "rock pick" equipment, where the first, second, third, and fourth positional sensors are rotational positional sensors and each is located inside a protective metal cabinet or casing, which protects each positional sensor from blows of rocks that are projected at high speeds after the hammer percussion is executed, and where said positional sensors are connected to each other by means of electrical connection cables, preferably by means of electrical connection cables of high mechanical resistance, where each metal cabinet or casing is joined directly to the metal structure of the arm of the rock pick machinery,and the electrical connection cables of each cabinet or metal housing are connected to a main cabinet or metal housing located on the side of the rock-picking equipment; and on each link of the arm of the rock-picking equipment and joining the base end of the arm of the rock-picking equipment and the motor of the rock-picking equipment, there is located a hydraulic cylinder or actuator half, where the first, second and third actuator half are each a linear hydraulic cylinder, which acts on a link of the arm of the rock-picking equipment displacing it while a fourth hydraulic cylinder is a rotational hydraulic cylinder which acts on the end of the arm of the rock-picking equipment, making it rotate around said base, and where said hydraulic cylinders which are joined to each joint of the arm of the rock-picking equipment and to the end of the arm of the rock-picking equipment, are joined by conventional fastening means, said actuator half are controlled by an intelligent control algorithm,integrated into the data processing subsystem comprising a high-performance computer, where in addition to receiving the data from each positional sensor, which are stored in a library, software establishes the new position of the rock-picking equipment arm that coincides with the data of the center of mass of the rock, stacked rocks or nearby rocks, which comes from the volumetric reconstruction of the rock, and which is stored in the data library of arm position and center of mass of the rock, and a Deep Learning algorithm establishes the movement trajectory that the rock-picking equipment arm will perform to reach the position of the center of mass of the rocks and the energy required to fracture it, controlling said actuating means and moving the rock-picking equipment arm to said rock fracture position; and optionally, d) a computational monitoring platform or application, local or remote,which is displayed on the screen of a user device and allows the status of the autonomous robotic system to be established based on one or more of the following system states: status of each vision sensor, status of each positional sensor, status of the hammer of the rock picker, status of each hydraulic cylinder, status of the arm of the rock picker, status of each joint of the arm of the rock picker, status of the motor of the rock picker, status of the mining or embedded communication, status of the volumetric reconstruction of the rock, in real time, among others, where the system state is established based on data from one or more of the following data: temperature of one or more of: hammer, motor, vision sensor, positional sensor, hydraulic cylinders or joints of the arm of the rock picker; percentage displacement capacity of the links of the arm of the rock picker,percentage rotation capacity of each joint of the rock-picking equipment arm; among others, and select on the computational monitoring platform the execution of operating actions that include one or more changes of operating mode from automatic to manual and vice versa, changes of operating mode to calibration mode and vice versa, among others, if necessary, where the user equipment is selected from wireless data transmission and reception equipment, configured to communicate on the local network, preferably fixed or portable computers, smartphones or tablets.

[0045] This autonomous robotic system increases the accuracy of the rock-chipping hammer's position and angle of incidence, enabling it to effectively impact rock even in challenging or low-visibility environments caused by factors such as dust, fog, shadows, sunlight, and darkness. This reduces downtime due to such conditions, shortens rock fracture times, and minimizes overheating of the hammer's tip. Each articulated position sensor is housed within a protective metal enclosure, allowing it to withstand adverse environmental conditions and potential impacts from rock fragments during comminution. The sensors are connected to each other via high-strength electrical cables.Each of the four metal cabinets or enclosures is directly attached to the metal structure of the rock-breaking machine's arm. The joint position sensors are connected via wired communication to a switch, which in turn communicates via wired communication with the high-performance computer. Both the switch and the high-performance computer are housed in a main metal cabinet or enclosure located on the side of the rock-breaking machine.

[0046] The vision sensors are mounted on stands, preferably industrial metal stands, located around the comminution area and out of reach of the rock-breaking machine's arm. These robust industrial stands are strategically positioned to provide a complete field of view of the work area and to capture the main rock piles or clusters that need to be comminuted or reduced. These stands also facilitate the electrical connection of each of the vision sensors, which are mounted high up or on the tops of the stands.

[0047] Firmly mounted at the base of the stands is an industrial metal cabinet or enclosure containing Ethernet communication equipment. This equipment collects and packages the data from the vision sensor array installed on the stands. It enables the secure, high-speed transmission of data directly (via wired connection) to the high-performance computer. The high-performance computer is located inside the system's main metal cabinet or enclosure, situated next to the rock-breaking machine.

[0048] The data captured from the sensor fusion provides a panoramic view, ensuring a clear field of vision of the comminution workspace. The subsystem automatically performs volumetric reconstruction within the workspace to identify rocks and rock piles. The system then automatically activates the rock-breaker to break up the identified rock pile. Finally, it tracks and selects individual rocks to estimate the optimal impact point for each one.

[0049] The machine learning algorithm enables the volumetric reconstruction of the rock being reduced in size using the rock-breaker. It processes data received from the vision subsystem and then merges this data. The algorithm for volumetric rock reconstruction also estimates the center of mass of cohesive solids or rocks in low-visibility environments, such as dust clouds. It relies on stereoscopic vision, allowing the volumetric reconstruction of a particular object from at least two different photographs of the same object. Rock reconstruction is performed on isolated rocks, stacked cohesive rocks, or separated but adjacent rocks. Estimating the center of mass of the rocks or cohesive solids allows the rock-breaker's hammer to deliver a precise blow to fragment the rock or rocks.

[0050] The vision subsystem's sensors capture images—for example, they take photographs—in an environment where visibility is reduced by dust. From these images, the center of mass is estimated by segmenting the rock, and its centroid is calculated. This centroid is then projected onto the rock surface where the rock-breaker's hammer will impact. The vision subsystem also includes sensors that measure the amount of particles, such as airborne particles, in the environment where the image is captured.

[0051] The volumetric reconstruction algorithm reliably performs volumetric reconstruction of rocks of varying sizes and shapes with low estimation error. While several volumetric reconstruction methods exist, most utilize artificial intelligence (AI), and some even monocular vision. However, until now, no volumetric reconstruction based on captured rock image datasets existed that could be used as a training set for an AI algorithm. The algorithm is capable of estimating the center of mass of isolated rocks, cohesive rocks, or rocks separated but close to one another.

[0052] The vision or sensor fusion subsystem allows for greater range, increasing data communication compatibility with various industrial protocols used in rock-cutting equipment in mining. The machine vision subsystem comprises two or more 3D vision sensors or cameras, infrared cameras, thermal imaging equipment, particle imaging equipment, radar, LiDAR, and other technologies.

[0053] When the vision subsystem comprises machine vision sensors, two or more machine vision sensors, or sensor fusion, these are connected to a computer via a switch, forming a centralized network. In this network, the machine vision sensors, switch, and high-performance computer can communicate with each other using mining environment communication protocols, such as IPv4 (RFC 791), where each sensor transmits its data to a central node (A. Barde and S. Jain, “A Survey of Multi-Sensor Data Fusion in Wireless Sensor Networks,” SSRN Electronic Journal, pp. 398–405, 2018). The sensors and the high-performance computer can communicate via IP on the appropriate network segment, for example, 192.168.0.0 / 24. Static addressing can also be used. Data acquisition is implemented programmatically using a programming language and a text editor, and the data for the algorithm is obtained from libraries.

[0054] The image captures are sequential and preferably comprise at least five consecutive images, where the average number of points captured by each vision sensor can vary; for example, they can be 200,000 for the BLAZE 101 ToF camera, 32,768 for the OSO LiDAR, 22,128 for the MRS6000 LiDAR, and 4,404 for the MRS 1000 LiDAR. An even more preferred configuration comprises six sensors and a computer connected via a switch, for example, a Layer 2 switch, using a mining environment protocol, for example, IPv4 (RFC 791), on an appropriate network segment, for example, the 192.168.0.0 / 24 network segment: See Fig. 6D. The devices / computers can communicate using static IP addresses, which is suitable for a small network.If the number of devices is greater than 30, a server with Dynamic Host Configuration Protocol (DHCP) can be implemented for dynamic IR address assignment. If it is necessary to connect more sensors, several switches can be used, maintaining the centralized network structure.

[0055] The switch offers efficient connectivity without requiring advanced configuration. For example, Ethernet connectivity provides a good cost-benefit ratio and the ability to deliver fast transmission speeds, enabling efficient communication between connected devices. Furthermore, the use of a Layer 2 switch ensures that each port has its own collision domain, allowing sensors to utilize the full available bandwidth (100 Mbps) without interference, if needed.

[0056] The vision subsystem can have a global autonomy architecture that establishes the complete framework for autonomous operation (the first architecture), or a second architecture that is integrated within the first architecture. Figure 6E illustrates the workflow of the first architecture. In the "Data Acquisition" stage, information is collected from the vision sensors in various modalities or multimodality, in three dimensions: 1D, for example, radar sensors (RMS1000), particle concentration sensor (SDS011), and light sensor (BH1750); 2D, for example, cameras (2 BLAZE 101 and CERES V 640); and 3D, for example, 2 BLAZE 101.The data obtained are processed in the "Planning and Decision" stage, where two key decision-making strategies are implemented. These strategies are based on identifying the operational scenario (simple or complex, such as a mining environment) and are applied to different rock configurations (single rock, rocks together or close together, and stacked rocks). In this phase, the (x, y, z) coordinates are generated, which are essential for accurately determining the location of impact points. For the "Planning and Decision - Strategy 2" stage, the following was considered: R. Rodríguez-Guillen, J. Kern, and C. Urrea, "Fast rock detection in visually contaminated mining environments using machine learning and deep learning techniques," Applied Sciences, vol. 14, no. 2, 2024. [Online], Available: y J- Kern, R. Rodríguez-Guillen, C. Urrea, and Y. Garcia-Garcia, “Enhancing 3d rock localization in mining environments using bird’s-eye view images from the time-offlight blaze 101 camera,” Technologies, vol. 12, no. 9, 2024. [Online], Available: t s: / / w . m pL conV222?-7Q80 / 12 / 9 / 62. Mientras que en la fase de ’’Motion and Control” se consideró H. Potter, J. Kern, G. Gonzalez, and C. Urrea, “Energetically optimal trajectory for a redundant planar robot by means of a nested loop algorithm,” Elektronika ir Elektrotechnika, vol. 28, no. 2, pp. 4-17, Apr. 2022. [Online], Available: h tps:ffgeiou aíkty^^^ D. Marrero, J. Kern, and C. llrrea, “A novel robotic controller using neural engineering framework-based spiking neural networks,” Sensors, vol. 24, no. 2, 2024. [Online], Available: h tps: / / ^,mdprc y J.

[0057] Kern, D. Marrero, and C. llrrea, “Fuzzy control strategies development for a 3-dof robotic manipulator in trajectory tracking,” Processes, vol. 11 , no. 12, 2023. [Online], Available: https: / / www.mdpi.com / 2227-9717 / 11 / 12 / 3267. Posteriormente, en la etapa de ’’Motion and Control”, se realiza el seguimiento de trayectorias mediante algoritmos de control avanzados, que utilizan dichas coordenadas en concordancia con la estrategia seleccionada (G. Velasco- Hernandez, D. J. Yeong, J. Barry, and J. Walsh, “Autonomous driving architectures, perception and data fusion: A review,” in 2020 IEEE 16th International Conference on Intelligent Computer Communication and Processing (ICCP), 2020, pp. 315-321 , y R. Rodríguez-Guillen, J. Kern, and C. llrrea, “Fast rock detection in visually contaminated mining environments using machine learning and deep learning techniques,” Applied Sciences, vol. 14, no. 2, 2024. [Online], Available: https: / / www.mdpi.eom / 2076-3417 / 14 / 2 / 731).

[0058] Regarding the second architecture, due to the inherent complexity of a data fusion system, the terms used to specify the characteristics and functionalities of each involved part are standardized. Therefore, architectures are implemented that allow for the hierarchical management of data manipulation from diverse sources. Multiple models exist in the scientific literature, such as: T. Meng, X. Jing, Z. Yan, and W. Pedrycz, “A survey on machine learning for data fusion,” Information Fusion, vol. 57, pp. 115-129, 2020. [Online], Available: https: / / www.sciencedirect.com / science / article / pii / S1566253519303902, BPL Lau, SH Marakkalage, Y. Zhou, NU Hassan, C. Yuen, M. Zhang, and U.-X. Tan, “A survey of data fusion in smart city applications,” Information Fusion, vol. 52, pp. 357-374, 2019. [Online], Available: Baroudi,

[0059] AA Al-Roubaiey, and A. Devendiran, “Pipeline leak detection systems and data fusion: A survey,” IEEE Access, vol. 7, pp. 97 426-97 439, 2019. Based on DJ Yeong, G. Velasco-Hernandez, J. Barry, and J. Walsh, “Sensor and sensor fusion technology in autonomous vehicles: A review,” Sensors, vol. 21, no. 6, 2021. [Online], Available: httgg^^w^rndgLcom¿lá24^^0^W / 2140 and X. Wang, K. Li, and A. Chehri, “Multi-sensor fusion technology for 3d object detection in autonomous driving: A review,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 2, pp. 1148-1165, 2024, in Figure 6G shows a model, as part of Strategy 1, of data fusion.

[0060] Figure 6G clearly shows the stages involved in the data fusion architecture. A programming language and a library can be used for data capture. Software is used for processing the information at each stage.

[0061] In the "Data Acquisition" phase, information is obtained at a set time interval from various sources (vision sensors) which, due to their inherent characteristics, provide a specific dimensionality. The information is either 3D or a static point cloud, thus obtaining complete information about the region of interest within the point clouds, a process known as complementary data fusion (K. Mo et al., StructureNet: Hierarchical Graph Networks for 3D Shape Generation, 2019). To reduce the number of points in the point cloud, preprocessing is performed based on various criteria, with the aim of improving the efficiency of the algorithms, including the elimination of data that may be considered unreliable.

[0062] The movement path of the rock-breaking hammer is optimized by minimizing multiple criteria, including: 1) execution time, 2) energy consumption or mechanical work, 3) maximum operating power, and 4) maximum forces and moments. It is known that to optimize the performance of rock-breaking hammers in construction, the following is generally sought: 1) minimum time path planning, 2) minimum energy or effort path planning, 3) minimum shock path planning, 4) minimum time and energy, and 5) minimum time and shock.The algorithm of the present invention generates energy-optimal trajectories and is based on two nested optimization loops. These loops leverage different aspects of the rock-chipping hammer's operation, contributing to an improved search for the global minimum of the function describing energy consumption during a given work trajectory. The nested algorithm finds an energy-efficient, point-to-point trajectory of the kinematically redundant, three-degree-of-freedom planar type. Energy savings in the trajectories are also considered due to the kinematic redundancy of the rock-chipping hammer, resulting from the greater availability of usable joint combinations for a given point. These redundancies contribute to more energy-efficient alternative trajectories.The algorithm establishes the influence of joint friction on the total energy consumption of the rock-chipping hammer. The algorithm develops a system that provides adaptive functions to the dynamics of the rock-chipping hammer. The intelligent learning method applicable to the energy optimization procedure is implemented through a system for generating energy-optimal trajectories, based on a neural network. The neural network is trained with data obtained from different trajectories previously developed by the algorithm. Once the neural network system is trained, this module is tested as a real-time generator of energy-optimal trajectories, considering the average calculation time of these trajectories, thus transforming it into a real-time system.The developed methods were integrated into a single functional structure: 1) trajectory generation using a nested algorithm, 2) identification of dynamic parameters, and 3) generation of energy-optimal trajectories based on a neural network system. The data are used through a methodology called the inverse kinematics model of the rock pick hammer structure. This transforms the information from the Cartesian space composed of the XYZ spatial components (impact point on the selected rock) into the joint components of each of the rock pick hammer's joints, enabling the precise positioning of the impact tool on the detected cohesive solids.

[0063] The sensor fusion system integrates information from multiple sensor sources with diverse operating modes to detect cohesive solid objects in adverse conditions using machine learning techniques. The multimodal data fusion system processes and combines sensor information according to established criteria. This integration improves the robustness and reliability of impact point identification in rocks during mining comminution, enabling a targeted approach to impact point detection that considers the maximum load distribution (PDOC). This, in turn, improves the energy efficiency of the crushing process and reduces wear on the equipment involved.

[0064] Data from both sensor fusion and joint sensors are processed locally, as data transmission in mining facilities is often complex. Therefore, the training and tuning of the artificial intelligence computational models were developed to enable their local deployment and integration within the system.

[0065] Data processing can be described as follows: data capture from various sensors, filtering and preprocessing, dimensionality reduction, fusion of information from different devices, preprocessing for volumetric rock reconstruction using machine learning, and estimation of the center or point of impact for each rock. Subsequently, the Cartesian coordinates of the point of impact are transformed into joint coordinates for actuating the rock-picking hammer's joints.

[0066] The machine learning models designed and implemented in the system process information to automatically detect rocks and segment each cohesive solid / rock. They then perform volumetric reconstruction and subsequently estimate the center or point of impact. These models have been trained with data from various scenarios, ranging from simple to complex, including low-light conditions, occlusions, different perspectives, and the presence of suspended particles.

[0067] Optionally, a user, through a local or remote monitoring platform or computer application displayed on a user device, can monitor the characteristics of the mining operation's data infrastructure and verify the system's status and performance by tracking critical parameters such as sensor status, impact tool status, hydraulic system status, mechanical system status, and volumetric rock reconstruction in real time. In other words, thanks to advanced sensor technology, the user can observe, using thermal imaging, the real-time temperature of the tool, components of the Cartesian space (impact point), and the machinery's joint variables.Using the screen displayed on a computer, users can select options on the platform to perform operational actions, such as changing the operating mode from automatic to manual, calibrating the system, and others. The information can be viewed not only on fixed devices like computers but also on mobile devices such as smartphones or tablets, using wireless data transmission tools and configuring a network to achieve this.

[0068] The present method allows, in real time, the identification, tracking, and selective impact of rocks, whether isolated, coupled, stacked, or separated and close together, through machine learning algorithms, sensor fusion, and volumetric reconstruction, improving operations in mining mills by enabling the precise fracturing of rocks in a rock-breaker, comprising the following steps: a) the capture of images from a vision subsystem comprising two or more vision sensors selected from the group consisting of 3D vision sensors or 3D cameras, infrared vision cameras, thermal vision equipment, particle vision equipment, radar sensors such as RADAR, optical sensors such as LiDAR, among others, located above and fixed on metallic structures, to achieve a visualization from different angles of a rock,cohesive stacked rocks or separate and close rocks; b) process the image data or 3D information received by a high-performance computer from said vision sensors, where said image data are received by means of mining environment communication protocols or embedded communication, and are filtered and processed as received from each vision sensor, and then fused by software and created a library that allows Deep Learning of artificial intelligence that is developed locally, and also the volumetric reconstruction of the rock, preferably, the reconstruction by stereoscopic vision, and determine the center of mass of the rock, cohesive stacked rocks or separate and close rocks; c) continuously evaluate the position of the arm of the “rock picker” equipment by means of four positional sensors located so that a first positional sensor is located at the base of the motor-first link joint of the arm of the “rock picker” equipment,A second positional sensor is located at the joint between the first and second links or intermediate links of the rock-picking equipment arm; a third positional sensor is located at the joint between the intermediate link and the third link or free end of the rock-picking equipment arm; and a fourth positional sensor is located at the base of the rock-picking equipment motor. These sensors monitor the joint position of the rock-picking equipment, reporting a displacement at each link of the rock-picking equipment arm and an angle of inclination at the base of the motor-first link joint of the rock-picking equipment arm. The first, second, and third positional sensors are linear sensors, preferably located inside hydraulic cylinders that correspond to these actuating means. These actuating means consist of three hydraulic cylinders with linear sensors.where each hydraulic cylinder with a linear sensor acts on a link of the rock-picking equipment arm while the fourth hydraulic cylinder is selected from a hydraulic cylinder with a rotational sensor and acts on the base of the rock-picking equipment link that is attached to the rock-picking equipment motor, c) continuously receiving, in the high-performance computer of the data processing subsystem, the position of the rock-picking equipment arm and then, by means of software, calculating an energy-optimized displacement trajectory of the rock-picking equipment arm, which includes a displacement for each link of the rock-picking equipment arm and a rotation angle for the base of the rock-picking equipment arm, towards the center of mass coordinates resulting from the volumetric reconstruction of the rock, creating a library that enables locally developed artificial intelligence Deep Learning,(yd) to move the arm of the rock-chipping equipment by means of actuators that receive signals from the high-performance computer for the movement of the arm of the rock-chipping equipment, which allows the accurate fracturing of the rock, comprising the communication between the actuators and the high-performance computer, mining environment communication protocols or embedded communication, and optionally (e) to monitor locally or remotely, the status of the autonomous robotic system by means of a monitoring platform or computer application that is displayed on the screen of a user device, one or more of: status of each vision sensor, status of each positional sensor, status of the hammer of the rock-chipping equipment, status of each hydraulic cylinder, status of the arm of the rock-chipping equipment, status of each joint of the arm of the rock-chipping equipment, status of the motor of the rock-chipping equipment, status of the mining or embedded communication,real-time status of the volumetric rock reconstruction, among others, by monitoring one or more of: temperature of one or more hammers, motors, vision sensors, positional sensors, or hydraulic cylinders; displacement or rotation angle of each joint of the rock-picking equipment arm; among others, and selecting on the computational monitoring platform the execution of operating actions, including changing the operating mode from automatic to manual and vice versa, changing the operating mode to calibration mode, among others, where the user equipment is selected from wireless data transmission and reception equipment, configured to communicate on the local network, preferably desktop or laptop computers, smartphones, or tablets.

Claims

CLAIMS 1. An autonomous robotic system attachable to a rock-picking hammer, which allows, in real time, the identification, tracking, and selective impact of rocks, whether isolated, coupled, stacked, or separated and close together, and to fracture them efficiently and accurately, characterized in that it comprises: a) a vision subsystem comprising two or more vision sensors selected from the group consisting of 3D vision sensors or 3D cameras, infrared vision cameras, thermal vision equipment, particle vision equipment, radar sensors such as RADAR, optical sensors such as LiDAR, and metallic structures, in which said vision sensors are fixed at a high level by conventional fixing means, and thus achieve visualization from different angles of the rock, stacked cohesive rocks, or separated and close together rocks, and such metallic structures being located outside the reach of the rock-picking hammer;b) a data processing subsystem comprising a high-performance computer that communicates with said vision sensors of the vision subsystem by means of mining environment or embedded communication protocols and comprising software for pre-processing and fusing the data from the vision sensors and a library that stores the pre-processed and fused data, and a Deep Learning algorithm that performs the volumetric reconstruction of the rock, stacked rocks or separated and adjacent rocks;c) an articulated sensing subsystem or electro-mechanical articulated subsystem that allows continuous evaluation of the position of the arm of the “rock pick” equipment, comprising four positional sensors or articulated sensors that, in addition to the adverse environmental conditions already mentioned, can withstand the vibration of the “rock pick” equipment and the impacts of rocks that are projected at high speeds after the hammer percussion is executed, where a first positional sensor is located at the base of the motor-first link joint of the arm of the “rock pick” equipment; a second positional sensor is located at the joint between the first link and the second link or intermediate link of the arm of the “rock pick” equipment; and a third positional sensor is located at the joint between the intermediate link and the third link or free end of the arm of the “rock pick” equipment;and a fourth positional sensor is located at the base of the motor of the “rock pick” equipment, c.1) providing said first, second and third positional sensors, together, with the data of the joint position of the “rock pick” equipment which are established by direct measurement of the displacement of the links of the arm of the “rock pick” equipment and the measurement of the angle of inclination at the base of the motor-first link joint of the arm of the “rock pick” equipment, wherein the first, second and third positional sensors are linear sensors and each is located inside a hydraulic cylinder or actuator means, protecting it, and each means; The actuator acts on a link of the rock-picking equipment arm while the fourth hydraulic cylinder with a rotational sensor acts on the end of the rock-picking equipment arm that is attached to the rock-picking equipment motor or base, rotating the rock-picking equipment arm around said base, or c.2) providing said first, second, third and fourth positional sensors, in conjunction, with data on the joint position of the rock-picking equipment established by measuring the angle of inclination of each joint of the rock-picking equipment arm, wherein the first, second, third and fourth positional sensors are rotational positional sensors and each is located inside a protective metal cabinet or housing, which protects each positional sensor from blows of rocks that are projected at high speeds after the hammer percussion is executed, and wherein said positional sensors are connected to each other by means of electrical connection cables,Preferably, by means of high mechanical resistance electrical connection cables, where each metal cabinet or housing is attached directly to the metal structure of the rock-breaking machine's arm, and the electrical connection cables of each metal cabinet or housing are connected to a main metal cabinet or housing located on the side of the rock-breaking equipment; and in each link of the rock-breaking equipment's arm, and joining the base end of the rock-breaking equipment's arm and the rock-breaking equipment's motor, a hydraulic cylinder or actuator half is located, where the first, second, and third actuators are each a linear hydraulic cylinder, which acts on a link of the rock-breaking equipment's arm, displacing it, while a fourth hydraulic cylinder is a rotary hydraulic cylinder that acts on the end of the rock-breaking equipment's arm, making it rotate around said base.and where said hydraulic cylinders that are joined to each joint of the arm of the “rock pick” equipment and to the end of the arm of the “rock pick” equipment, are joined by conventional means of fixing, said actuating means are controlled by an intelligent control algorithm, integrated in the data processing subsystem comprising a high-performance computer, where in addition to receiving the data from each positional sensor, which are stored in a library, a software establishes the new position of the arm of the “rock pick” equipment that coincides with the data of the center of mass of the rock, stacked rocks or nearby and close rocks, which comes from the volumetric reconstruction of the rock, and which is stored in the library of data of the position of the arm and center of mass of the rock,and a Deep Learning algorithm establishes the movement trajectory that the arm of the "rock pick" equipment will follow to reach the position of the center of mass of the rocks and the energy required to fracture them, controlling said actuating means and moving the arm of the "rock pick" equipment to said rock fracturing position.

2. The autonomous robotic system of claim 1 characterized in that said volumetric reconstruction is a stereoscopic volumetric reconstruction.

3. The autonomous robotic system of claim 2 characterized in that said volumetric or stereoscopic vision reconstruction requires capture data from at least two photographs or images of the rock.

4. The autonomous robotic system of claim 3 characterized in that said image capture comprises consecutive image capture.

5. The autonomous robotic system of claim 1 characterized in that said metal structure is a lectern.

6. The autonomous robotic system of claim 5 characterized in that said stand is located to one side of the “rock pick” equipment.

7. The autonomous robotic system of claim 6 characterized in that said metal structure is a lectern with a metal cabinet or casing.

8. The autonomous robotic system of claim 7 characterized in that said lectern with metal cabinet or casing contains both the communication equipment that receives data from said vision sensors and sends data at high speed directly or via wired communication to the high-performance computer, as well as the high-performance computer.

9. The autonomous robotic system of claim 8 characterized in that said communication equipment is an Ethernet-type switch.

10. The autonomous robotic system of claim 1 characterized in that it further comprises a local or remote monitoring computing platform or application that is displayed on the screen of a user device and allows the establishment of the state of the autonomous robotic system based on one or more of the following system states: state of each vision sensor, state of each positional sensor, state of the hammer of the rock picker, state of each hydraulic cylinder, state of the arm of the rock picker, state of each joint of the arm of the rock picker, state of the motor of the rock picker, state of the mining or embedded communication, state of the volumetric reconstruction of the rock, in real time, where the system state is established based on data from one or more of the following data: temperature of one or more of: hammer, motor, vision sensor, positional sensor, hydraulic cylinders or joints of the arm of the rock picker;percentage displacement capacity of the links of the rock-picking equipment arm, percentage rotation capacity of each joint of the rock-picking equipment arm; and select on the computational monitoring platform the execution of operating actions that include one or more changes of operating mode from automatic to manual and vice versa, changes of operating mode to calibration mode and vice versa, if necessary.; 11. The autonomous robotic system of claim 10 characterized in that said user equipment is selected from wireless data transmission and reception equipment, configured to communicate on the local network.

12. The autonomous robotic system of claim 11 characterized in that said wireless data transmission and reception equipment is selected from desktop or laptop computers, smartphones or tablets.

13. The autonomous robotic system of claim 1 characterized in that the vision subsystem comprises at least one sensor that measures the amount of particles in the environment where the image is captured or the photograph is obtained.

14. The autonomous robotic system of claim 13 characterized in that said sensor that measures the amount of particles is a sensor that measures the amount of suspended particles.

15. The autonomous robotic system of claim 1 characterized in that the vision subsystem comprises six vision sensors, a switch, and a high-performance computer.

16. The autonomous robotic system of claim 1 characterized in that the high-performance computer communicates with vapor switches which in turn communicate with vapor vision sensors.

17. A method that allows, in real time, the identification, tracking, and selective impact of rocks, whether isolated / alone, coupled / cohesive, stacked, or separated and close together, using machine learning algorithms, sensor fusion, and volumetric reconstruction, improving operations in mining mills by enabling the accurate fracturing of rocks, characterized in that it comprises the following steps: a) capturing images from a vision subsystem comprising two or more vision sensors selected from the group consisting of 3D vision sensors or 3D cameras, infrared vision cameras, thermal vision equipment, particle vision equipment, radar sensors such as RADAR, optical sensors such as LiDAR, located above and fixed on metallic structures, to achieve a visualization from different angles of a rock, cohesive rocks stacked, or separated and close together rocks;b) process the image data or 3D information received by a high-performance computer from said vision sensors, where said image data are received by means of mining environment communication protocols or embedded communication, and are filtered and processed as received from each vision sensor, and then merged by software and created a library that allows Deep Learning of artificial intelligence that is developed locally, and also the volumetric reconstruction of the rock, preferably, the reconstruction by stereoscopic vision, and determine the center of mass of the rock, cohesive rocks stacked or separated and close rocks; c) continuously evaluate the position of the arm of the “rock pick” equipment by means of four positional sensors located so that a first positional sensor is located at the base; From the motor-first link joint of the rock picker arm, a second positional sensor is located at the joint between the first and second links or intermediate link of the rock picker arm; and a third positional sensor is located at the joint between the intermediate link and the third link or free end of the rock picker arm, and a fourth positional sensor is located at the base of the rock picker motor, and they monitor the joint position of the rock picker, reporting a displacement in each link of the rock picker arm and an angle of inclination at the base of the motor-first link joint of the rock picker arm, where the first, second and third positional sensors are linear sensors, preferably linear sensors located inside hydraulic cylinders that correspond to said actuating means, and said actuating means are 3 hydraulic cylinders with linear sensors,where each hydraulic cylinder with a linear sensor acts on a link of the rock-picking equipment arm while the fourth hydraulic cylinder is selected from a hydraulic cylinder with a rotational sensor and acts on the base of the rock-picking equipment link that is attached to the rock-picking equipment motor, c) continuously receiving, in the high-performance computer of the data processing subsystem, the position of the rock-picking equipment arm and then, by means of software, calculating an energy-optimized displacement trajectory of the rock-picking equipment arm, which includes a displacement for each link of the rock-picking equipment arm and a rotation angle for the base of the rock-picking equipment arm, towards the center of mass coordinates resulting from the volumetric reconstruction of the rock, creating a library that enables locally developed artificial intelligence Deep Learning,(yd) moving the arm of the rock-chipping equipment by means of actuators that receive signals from the high-performance computer for the movement of the arm of the rock-chipping equipment, which allows the accurate fracturing of the rock, including the communication between the actuators and the high-performance computer, mining environment communication protocols or embedded communication.

18. The method of claim 17, further comprising: (e) monitoring, locally or remotely, the status of the autonomous robotic system by means of a computational monitoring platform or application displayed on a user device screen, one or more of the following: status of each vision sensor, status of each positional sensor, status of the hammer of the rock picker, status of each hydraulic cylinder, status of the arm of the rock picker, status of each joint of the arm of the rock picker, status of the motor of the rock picker, status of the mining or embedded communication, status of the volumetric rock reconstruction, in real time, by monitoring one or more of the following: temperature of one or more hammers, motors, vision sensors, positional sensors, or hydraulic cylinders; displacement or rotation angle of each joint of the arm of the rock picker; and selecting on the computational monitoring platform the execution of actions. operation, including changing the operating mode from automatic to manual and vice versa, changing the operating mode to calibration mode, among others, where the user equipment is selected from wireless data transmission and reception equipment, configured to communicate on the local network, preferably desktop or laptop computers, smartphones or tablets.

Images