Underground shovel autonomous shovel loading operation control method, device and system

Abstract

The application provides a kind of underground shovel loader autonomous shovel loading operation control method, device and system, it is related to mine machinery control technical field, method includes: from the multi-source vision data collected from the surface of the to-be-shovel loading material pile Extract the multi-modal physical characteristics of the surface of the to-be-shovel loading material pile, and determine the shovelability index of the to-be-shovel loading material pile;According to the shovelability index, a three-dimensional space distribution model representing the shovelability of the surface of the to-be-shovel loading material pile is constructed;According to the spatial boundary of underground and the kinematic feasibility boundary of the shovel loader, the constraint condition of the shovel loader operation trajectory is constructed, and based on the constraint condition and the three-dimensional space distribution model, the operation trajectory search is carried out in the three-dimensional pose space of the shovel loader bucket, to obtain the optimal operation trajectory of the shovel loader, for controlling the shovel loader to perform shovel loading operation on the to-be-shovel loading material pile.The present application is used to solve the problem that the existing autonomous shovel loading operation ignores the physical characteristics of the material pile and is difficult to adapt to the narrow and closed environment of underground.

Description

Technical Field

[0001] This invention relates to the field of mining machinery control technology, specifically to a method, device, and system for controlling the autonomous loading and unloading operations of underground loaders. Background Technology

[0002] Underground loaders are core equipment in trackless mining of underground metal mines. Loading operations are the most technically challenging part of their work cycle. Operators must rely on experience to judge the shovelability of material piles, control the bucket insertion depth and lifting / turning rhythm in dusty, confined underground tunnels, while simultaneously preventing machine slippage and bucket jamming. With the advancement of unmanned mine construction, achieving autonomous loading operations for underground loaders has become a key technological bottleneck that the industry urgently needs to overcome.

[0003] Currently, autonomous loading technologies are mainly concentrated in the field of open-pit construction machinery. Existing technologies propose a method for autonomous material movement: first, determine the geometric midpoint of the material pile; then, generate multiple potential loading paths around the midpoint; finally, select the optimal path based on cost function analysis and execute the loading. However, this approach has two drawbacks. First, it models the material pile as a geometrically uniform entity, completely ignoring the spatial non-uniformity of physical properties such as particle size distribution, density, and crack development within the pile. The loading resistance in different areas of an underground blasting material pile can vary by several times. Paths planned solely based on the surface geometric midpoint may guide the bucket towards high-resistance dense areas or areas with large, concentrated particles, leading to jamming or empty loading. Second, this approach is based on the assumption of an open-pit environment, while underground environments present constraints such as dust and water mist obstructing sensors, narrow tunnels limiting loading approach angles, and communication delays—constraints not considered in this approach. Therefore, the transferability of this approach is questionable. Summary of the Invention

[0004] In view of this, it is necessary to provide a method, device and system for controlling the autonomous loading operation of underground loaders, so as to solve the problem that existing autonomous loading operations ignore the physical characteristics of the material pile and are difficult to adapt to the narrow and confined environment of underground mines.

[0005] To address the aforementioned problems, this invention provides a method for controlling the autonomous loading operation of an underground loader, characterized by comprising: Acquire multi-source visual data collected from the surface of the material pile to be shoveled underground, and extract multimodal physical features of the surface of the material pile to be shoveled from the multi-source visual data; The shovelability index of the stockpile to be shoveled is determined based on the aforementioned multimodal physical characteristics; A three-dimensional spatial distribution model characterizing the shovelability of the surface of the material pile to be shoveled is constructed based on the shovelability index. The constraints of the loader's operating trajectory are constructed based on the underground space boundary and the kinematic feasibility boundary of the loader. Based on the constraints and the three-dimensional spatial distribution model, the operating trajectory is searched in the three-dimensional pose space of the loader bucket to obtain the optimal operating trajectory of the loader. Based on the optimal operating trajectory, the loader is controlled to perform loading operations on the stockpile to be loaded.

[0006] In one possible implementation, the multi-source visual data includes point cloud data acquired by lidar, thermal imaging data acquired by a thermal imaging camera, and visible light images acquired by a visible light vision camera. Extracting multimodal physical features from the surface of the material pile to be loaded from the multi-source visual data includes: Dust attenuation compensation is performed on the point cloud data, and the geometric texture features of the material pile to be shoveled are extracted from the compensated point cloud data. Remove the thermal radiation background from the underground roadway wall in the thermal imaging data, extract the effective temperature data from the background-removed thermal imaging data, and calculate the surface temperature decay rate of the effective temperature data as a thermophysical feature. The density of crack edges and the direction of the main cracks in the visible light image of the material to be loaded are extracted as texture structure features. Align the geometric texture features, the thermophysical features, and the texture structure features to the same vehicle coordinate system to obtain the multimodal physical features of the surface of the material pile to be loaded.

[0007] In one possible implementation, determining the shovelability index of the stockpile based on the multimodal physical characteristics includes: The multimodal physical features are input into a pre-trained sparse Gaussian process regression model for regression processing to obtain the predicted values ​​of loading resistance, density, and full bucket rate for each three-dimensional spatial coordinate point inside the material pile to be loaded. The predicted values ​​of loading resistance, compaction, and bucket fullness are weighted according to preset weights, and the weighted results are summed to obtain the shovelability index of the stockpile to be shoveled.

[0008] In one possible implementation, the step of searching for the optimal working trajectory of the loader bucket in the three-dimensional pose space based on the constraints and the three-dimensional spatial distribution model to obtain the optimal working trajectory of the loader includes: Based on the shovelability index in the three-dimensional spatial distribution model, a trajectory sampling region is determined from the three-dimensional pose space. The trajectory sampling region includes a high shovelability region, a normal region, and a low shovelability region. The high shovelability region is the spatial region in the three-dimensional pose space where the shovelability index is greater than a first index threshold. The low shovelability region is the spatial region in the three-dimensional pose space where the shovelability index is less than a second index threshold. The normal region is the spatial region in the three-dimensional pose space where the shovelability index is between the second index threshold and the first index threshold, where the second index threshold is less than the first index threshold. Based on the aforementioned shovelability index, the smoothness of the work trajectory, and the loading resistance, a target cost function for work trajectory planning is constructed. Using the minimum value of the target cost function as the search objective, the bidirectional optimal fast expanding random tree algorithm is invoked to search for the working trajectory in the trajectory sampling area to obtain the optimal working trajectory of the loader. The optimal working trajectory includes the depth curve of the bucket insertion into the material pile to be shoveled, the bucket lifting height curve, and the bucket tilt angle curve. The trajectory sampling density of the high shovelability area is greater than that of the ordinary area, and the trajectory sampling density of the ordinary area is greater than that of the low shovelability area.

[0009] In one possible implementation, when the loader is controlled to perform a loading operation on the stockpile based on the optimal operating trajectory, the method further includes: Based on the rodless chamber pressure data and rod chamber pressure data collected from the boom cylinder and bucket cylinder of the loader, the actual loading resistance experienced by the bucket during operation is calculated. The difference between the actual loading resistance and the expected loading resistance corresponding to the optimal working trajectory is input into the impedance controller to obtain the bucket position adjustment amount, and the actual working trajectory during bucket operation is corrected based on the bucket position adjustment amount. The impedance controller is used to employ a first-order impedance model in the bucket insertion direction to control the bucket to automatically decelerate and retract when it encounters resistance. The impedance controller is used to employ a second-order impedance model in the bucket lifting direction to forcibly control the bucket lifting trajectory. The impedance controller is used to employ a second-order impedance model in the bucket tilting direction to adaptively adjust the bucket tilt angle when the bucket is inserted and to forcibly control the bucket retraction angle when the bucket is retracted.

[0010] In one possible implementation, when the loader is controlled based on the optimal work trajectory to perform a loading operation on the stockpile to be loaded, the method further includes: Real-time acquisition of actual loading resistance data at each position coordinate in the three-dimensional pose space of the loader bucket; and construction of a training dataset based on the actual loading resistance data and the position coordinates. The training dataset is input into a sparse Gaussian process regression model used to determine the shovelability index, and the recursive Bayesian update algorithm is invoked to update the parameters of the sparse Gaussian process regression model.

[0011] In one possible implementation, when the loader is controlled based on the optimal work trajectory to perform a loading operation on the stockpile to be loaded, the method further includes: Obtain the longitudinal dynamic model of the loader; The longitudinal acceleration, mass, traction force, and ground rolling resistance of the loader during the loading operation are input into the longitudinal dynamic model to calculate the current total loading resistance of the loader. The maximum adhesion force between the loader and the ground is calculated based on the preset adhesion coefficient between the loader tires and the ground and the normal load of the loader drive wheel; When the ratio of the current total loading resistance to the maximum adhesion force is greater than a first threshold, the insertion angle and insertion speed of the loader bucket are controlled to be reduced. When the ratio of the current total loading resistance to the maximum adhesion is less than a second threshold, the insertion depth of the bucket is controlled to increase, where the second threshold is less than the first threshold.

[0012] In one possible implementation, when the loader is controlled based on the optimal work trajectory to perform a loading operation on the stockpile to be loaded, the method further includes: The real-time operating status data of the loader when it performs shoveling operation on the pile of material to be shoveled is obtained, and the real-time operating status data is input into the pre-trained reinforcement learning model to generate the output action strategy of the impedance controller to control the shoveling operation. The execution actions of the loading operation are adjusted based on the aforementioned output action strategy; The reward function of the reinforcement learning model is as follows:

[0013] Where R represents the reward function, This represents the upper limit of energy consumption for the loader to perform loading operations, and E represents the actual energy consumption of the loader to perform loading operations. This indicates the set loading cycle time for the loader to perform one loading operation. This indicates the actual loading cycle time of a loader performing one loading operation. This indicates the slip ratio of the loader's tires. This indicates the bucket fullness rate when a loader performs loading operations. , , , All of these represent preset weighting coefficients.

[0014] The present invention also provides an autonomous loading and unloading control device for an underground loader, comprising: The physical feature extraction module is used to acquire multi-source visual data collected from the surface of the material pile to be shoveled and loaded underground, and to extract multi-modal physical features of the surface of the material pile to be shoveled and loaded from the multi-source visual data. A shovelability determination module is used to determine the shovelability index of the stockpile to be shoveled based on the multimodal physical characteristics. The three-dimensional spatial modeling module constructs a three-dimensional spatial distribution model representing the shovelability of the surface of the material pile to be shoveled based on the shovelability index. The operation trajectory planning module is used to construct the constraints of the loader's operation trajectory based on the underground space boundary and the kinematic feasibility boundary of the loader, and to search for the operation trajectory in the three-dimensional pose space of the loader's bucket based on the constraints and the three-dimensional spatial distribution model to obtain the optimal operation trajectory of the loader. The bucket operation control module is used to control the loader to perform shoveling operations on the stockpile to be shoveled based on the optimal operation trajectory.

[0015] The present invention also provides an autonomous loading and unloading operation control system for an underground loader, including sensors, communication equipment, and electronic equipment. The electronic equipment communicates with the sensors through the communication equipment. The electronic equipment includes a memory and a processor. The memory is used to store programs. The processor is coupled to the memory and is used to execute the programs stored in the memory to implement the steps of the autonomous loading and unloading operation control method for the underground loader described above. The sensors are used to collect multi-source visual data from the surface of the underground material pile to be loaded. The communication equipment is used to upload the multi-source visual data to the memory of the electronic equipment.

[0016] The beneficial effects of adopting the above-mentioned implementation method are as follows: The autonomous loading operation control method, device, and system for underground loaders provided by this invention first analyze the multimodal physical characteristics of the surface of the material pile to be loaded through multi-source visual perception. Based on this, the loadability of the material pile to be loaded is determined. Compared with the existing technology that plans the loading path based on the geometric midpoint of the surface, analyzing the physical characteristics and loadability of the material pile in advance is beneficial to actively plan the working trajectory of the loader in combination with the actual loading conditions, so as to achieve targeted autonomous loading operations, rather than passively adjusting in real time according to the actual loading conditions during loading operations. Furthermore, in the specific working trajectory planning, the underground space boundary and the kinematic feasibility boundary of the loader are used as search constraints, so that the trajectory planning of the loader can adapt to the narrow working environment of the underground roadway, ensuring the feasibility of the working trajectory planning. The three-dimensional spatial distribution model constructed by the shovelability index is used as the basis for trajectory planning to carry out the shoveling operation. This realizes the operation process from perception to analysis to planning, which enables the shoveling operation to actively avoid high resistance dense areas or large rich areas of material piles and avoid areas with abnormal resistance, thus achieving efficient and stable autonomous shoveling operation, while also ensuring the safety and reliability of unmanned operation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the autonomous loading operation control method for underground loader provided by the present invention; Figure 2 A schematic diagram of the structure of the autonomous loading and unloading control device for an underground loader provided by the present invention; Figure 3 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of the embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0021] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0022] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0023] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0024] This invention provides a method for controlling autonomous loading operations of underground loaders, applicable to autonomous loading operations in confined underground environments. The executing entity can be various servers, terminals, or cloud devices deployed within or remotely communicating with the loader. When the loader begins autonomous loading operations in a confined underground environment, multi-source visual data is collected from the surface of the material pile to be loaded using multi-source sensors deployed within the loader and uploaded. Then, the autonomous loading operation control method provided by this invention is invoked to plan the optimal operating trajectory of the loader, and the loader is controlled to perform the loading operation on the material pile according to the optimal operating trajectory.

[0025] The following section details the autonomous loading and unloading control method for downhole shovels provided by this invention.

[0026] Figure 1 The flowchart illustrates the autonomous loading operation control method for underground loader provided by this invention, as shown below. Figure 1 As shown, the autonomous loading operation control method of the underground loader can be achieved through the following steps 101 to 105, which are explained in detail below.

[0027] Step 101: Acquire multi-source visual data collected from the surface of the material pile to be shoveled underground, and extract multi-modal physical features from the surface of the material pile to be shoveled from the multi-source visual data.

[0028] In actual loading operations, a LiDAR, a long-wave thermal imaging camera, and a visible light vision camera are integrated on a fixed mounting bracket at the front end of the loader, forming a multimodal sensing unit. In this embodiment, the LiDAR scanning frequency is 20Hz, the angular resolution is 0.05°, and the maximum detection distance is 15m; the long-wave thermal imaging camera has a thermal sensitivity of 50mK and a working wavelength of 8-14μm; and the visible light vision camera has a resolution of 1920×1080 and a frame rate of 30fps. The three types of sensors are calibrated by hand and eye, and uniformly aligned to the vehicle coordinate system. After calibration, the translation error of each sensor coordinate system is no greater than 5mm, and the rotation error is no greater than 0.2°. At the same time, the lever arm conversion calibration of the hydraulic system pressure sensor is completed, establishing the mapping relationship between the pressure of the boom cylinder and bucket cylinder and the force on the bucket end, with a calibration error of no more than 3%.

[0029] The underground material piles awaiting loading are generally composed of two types of ores: sphalerite and sulfide ore. The height of the piles is typically controlled between 1.8m and 2.2m. When the loader autonomously travels underground to a preset stopping distance (e.g., 3m) in front of the material pile, it simultaneously triggers three types of sensors to collect multi-source visual data of the surface of the material pile and uploads it in real time.

[0030] Given the narrow and harsh environment underground, multi-source visual data typically requires preprocessing to remove errors caused by factors such as dust and heat radiation from the tunnel walls. After preprocessing, data analysis is performed to extract the multimodal physical features of the material pile to be loaded.

[0031] In one possible implementation, the multi-source visual data includes point cloud data acquired by lidar, thermal imaging data acquired by thermal imaging cameras, and visible light images acquired by visible light cameras. The extraction of multimodal physical features from the surface of the material pile to be loaded from the multi-source visual data can be achieved in the following ways.

[0032] Dust attenuation compensation is performed on the point cloud data, and the geometric texture features of the material pile to be shoveled are extracted from the compensated point cloud data. Remove the thermal radiation background from the underground roadway wall in the thermal imaging data, extract the effective temperature data from the background-removed thermal imaging data, and calculate the surface temperature decay rate of the effective temperature data as a thermophysical feature. The density of crack edges and the direction of the main cracks in the visible light image of the material to be loaded are extracted as texture structure features. By aligning geometric texture features, thermophysical features, and texture structure features to the same vehicle coordinate system, multimodal physical features of the surface of the material pile to be loaded are obtained.

[0033] Here, for point cloud data acquired by lidar, dust attenuation compensation mainly utilizes the density inversion space of local point cloud to determine the dust concentration, identifying areas obscured by dust (e.g., areas where the dust concentration exceeds a threshold). Then, a cubic spline interpolation algorithm is used to complete the missing point cloud data in the dust-obscured areas, achieving a point cloud completion rate of no less than 92%. Local curvature, normal vector variance, and fast point feature histograms are extracted from the compensated point cloud data as geometric texture features of the material to be loaded.

[0034] For thermal imaging data acquired by thermal imaging cameras, the first step is to remove the thermal radiation background from the tunnel walls, as the tunnel walls are not the objects of the loading operations and therefore require background removal. After background removal, a 5×5 neighborhood background difference algorithm can be used to extract the effective temperature data. Furthermore, the surface temperature decay rate of the effective temperature data is calculated as a thermophysical characteristic using active thermal excitation.

[0035] For visible light images captured by a visible light camera, edge detection algorithms can be used to extract all edge contours on the surface of the material pile to be shoveled. Then, morphological closing operations are used to connect the broken, fine cracks. Further, crack edge density calculation is performed, and the total pixel length of the crack skeleton within the effective area of ​​the material pile is calculated and divided by the total area of ​​the material pile to obtain the crack edge density value per unit area. This value directly reflects the looseness of the material pile; the higher the density, the easier the material pile is to shovel. For the main crack direction, principal component analysis can be used to analyze the identified cracks. The vertical direction corresponding to the minimum eigenvalue is taken as the direction of the main crack. The calculated direction vector is then converted into an angle value to obtain the main crack direction on the material pile surface, thereby extracting texture structure features.

[0036] Finally, the geometric texture features, thermophysical features, and texture structure features are aligned to the same vehicle coordinate system, and multimodal physical features of the material pile surface to be loaded are generated with uniform resolution through feature overlay. The vehicle coordinate system is a virtual coordinate system established with the loader as the center.

[0037] In this embodiment of the invention, by preprocessing multi-source visual data, environmental influencing factors such as dust and tunnel wall heat radiation can be effectively eliminated, improving the accuracy of multimodal physical features. Furthermore, by extracting multimodal physical features, the physical characteristics of the material pile to be shoveled can be analyzed, which is beneficial for assessing its shovelability and providing a basis for subsequent operation trajectory planning.

[0038] Step 102: Determine the shovelability index of the stockpile to be shoveled based on multimodal physical characteristics.

[0039] Here, in this embodiment of the invention, a sparse Gaussian process regression model is pre-designed when analyzing the shovelability of the material pile to be shoveled. This model is trained based on training samples composed of multimodal physical characteristics and predicted values ​​of shoveling resistance, density, and bucket fullness. It can output normalized predicted values ​​of shoveling resistance, density, and bucket fullness for each spatial coordinate point within the three-dimensional space of the material pile to be shoveled, based on the input multimodal physical characteristics. The shovelability index of the material pile to be shoveled is then calculated by weighted summation of the three predicted values ​​output by the sparse Gaussian process regression model.

[0040] In one possible implementation, the shovelability index of the stockpile to be shoveled is determined based on multimodal physical characteristics. This can be achieved in the following ways, which are explained in detail below.

[0041] Multimodal physical features are input into a pre-trained sparse Gaussian process regression model for regression processing to obtain the predicted values ​​of loading resistance, density, and full bucket rate for each three-dimensional spatial coordinate point inside the material pile to be loaded. The predicted values ​​of loading resistance, density, and bucket fullness are weighted according to preset weights, and the weighted results are summed to obtain the shovelability index of the stockpile to be shoveled.

[0042] Specifically, the pre-trained sparse Gaussian process regression model can use the Marting kernel function, specifically the 5 / 2 type kernel function, with the following expression:

[0043] in, This is a hyperparameter, with a value of 0.8m. The value represents the signal variance hyperparameter, with a value of 0.16, and r represents the feature points. With feature points The Euclidean distance between them Representing feature points With feature points The covariance between them.

[0044] The sparse Gaussian process regression model selects 100 induced points as feature points for sparse approximation, with a single-frame inference time controlled within 80ms, meeting the requirements for real-time on-board computing. Here, the feature points correspond to the coordinates within the three-dimensional space of the material pile to be loaded.

[0045] In specific calculations, three sparse Gaussian process regression models can be designed separately for the predicted values ​​of loading resistance, compaction, and bucket fullness, or a single sparse Gaussian process regression model combined with three output kernels can be designed for calculation. Based on the multimodal physical characteristics of the induction point, the sparse Gaussian process regression model predicts the posterior distribution of the induction point through regression prediction, thereby predicting the loading resistance, compaction, and bucket fullness. In the post-processing stage, these predicted values ​​are uniformly normalized and denoted as follows: , as well as .

[0046] Furthermore, the predicted values ​​of loading resistance, compaction, and full bucket rate are weighted and summed according to preset weights to obtain the shovelability index of the stockpile to be shoveled, denoted as S.

[0047] For each three-dimensional spatial coordinate point inside the material pile to be shoveled The formula for calculating the spallability index is as follows:

[0048] in, , , as well as Representing three-dimensional spatial coordinate points respectively The corresponding shovelability index, predicted loading resistance, predicted compaction, and predicted bucket full rate are as follows: , , The weights are preset, and the sum of the three is 1. They can be set as needed according to the actual scenario requirements.

[0049] In this embodiment of the invention, the shovelability of the material pile to be shoveled is determined by calculating the shovelability index. Compared with the existing technology that plans the loading path based on the surface geometric midpoint, the advance analysis of the physical characteristics and shovelability of the material pile is conducive to the proactive planning of the loader's working trajectory in combination with the actual shoveling conditions, so as to achieve targeted autonomous shoveling operations, rather than passively adjusting in real time according to the actual shoveling conditions during the shoveling operation. This is more conducive to achieving efficient and stable autonomous shoveling operations.

[0050] Step 103: Construct a three-dimensional spatial distribution model representing the shovelability of the surface of the material pile to be shoveled based on the shovelability index.

[0051] For each three-dimensional spatial coordinate point inside the material pile to be shoveled, a sparse Gaussian process regression model can be used in the same way to perform regression prediction and calculate the corresponding shovelability index. Therefore, in 3D modeling, the three-dimensional space inside the material pile to be shoveled can be used as a three-dimensional space to characterize the shovelability of the surface of the material pile to be shoveled. Combined with the shovelability index of each coordinate point, this three-dimensional space forms a three-dimensional spatial distribution model characterizing the shovelability of the surface of the material pile to be shoveled.

[0052] Step 104: Construct constraints on the working trajectory of the loader based on the underground space boundary and the kinematic feasibility boundary of the loader. Based on the constraints and the three-dimensional spatial distribution model, search for the working trajectory in the three-dimensional pose space of the loader bucket to obtain the optimal working trajectory of the loader.

[0053] Here, during the operation trajectory search process, the 3D spatial distribution model serves as a cost map for the trajectory search, while ensuring that the loader's operation trajectory is constrained by the underground space boundary and the kinematic feasibility boundary. Specifically, the underground space boundary constraint limits the space of the loader's operation trajectory; the minimum safe distance between the loader's outline and the roadway sidewall is no less than 0.3m. The kinematic feasibility boundary, on the other hand, restricts the loader's motion state by setting the loader's maximum articulated turning angle and the upper and lower limit positions of the bucket movement as hard constraints to satisfy the loader's kinematic feasibility.

[0054] The three-dimensional pose space of a loader bucket specifically includes the three-dimensional position coordinates and three-dimensional attitude angles of the bucket's end. The work trajectory search can employ a bidirectional optimal fast expanding random tree algorithm. Alternatively, both bidirectional fast expanding random tree and optimal fast expanding random tree algorithms can be used, but their search efficiency is lower.

[0055] In one possible implementation, the optimal working trajectory of the loader is obtained by searching the three-dimensional pose space of the loader bucket based on constraints and a three-dimensional spatial distribution model. This can be achieved in the following ways, which are explained in detail below.

[0056] The trajectory sampling region is determined from the shovelability index in the 3D spatial distribution model. The objective cost function for operation trajectory planning is constructed based on the shovelability index, the smoothness of the operation trajectory, and the loading resistance. The search objective is to find the minimum value of the objective cost function. The bidirectional optimal fast expanding random tree algorithm is called to search for the operation trajectory in the trajectory sampling area to obtain the optimal operation trajectory of the loader.

[0057] Because the difficulty of loading operations varies across areas with different shovelability indices, the planned operation trajectories also differ in areas of varying difficulty, resulting in different loading operation costs. Based on a three-dimensional spatial distribution model as a cost map, this embodiment of the invention divides the trajectory sampling area of ​​the material pile to be loaded into three regions: high shovelability areas, normal areas, and low shovelability areas. A high shovelability area is a spatial region in three-dimensional pose space where the shovelability index is greater than a first index threshold. The first index threshold can be 0.7; that is, an area composed of three spatial positions where the shovelability index S is greater than 0.7 is defined as a high shovelability area. A low shovelability area is a spatial region in three-dimensional pose space where the shovelability index is less than a second index threshold. The second index threshold is less than the first index threshold and can be 0.3; that is, an area composed of three spatial positions where the shovelability index S is less than 0.3 is defined as a low shovelability area. The ordinary region is the spatial region in the three-dimensional pose space where the shovelability index is between the second index threshold and the first index threshold. In other words, the region composed of three spatial positions with a shovelability index S greater than or equal to 0.3 and less than or equal to 0.7 is defined as the ordinary region.

[0058] The basis for operation trajectory planning is the calculation of the objective cost function. In this embodiment of the invention, the shovelability index, shoveling resistance, and smoothness of the operation trajectory are comprehensively considered as optimization objectives during operation trajectory planning, and the objective cost function, denoted as J, is calculated based on these three indicators, and expressed as:

[0059] in, This indicates that line integrals are performed along the path of the operation trajectory. This represents the arc length of the infinitesimal element along the path of the task trajectory. In other words, the cost of the task trajectory is cumulative. Cost calculation is performed on the entire task trajectory to measure the global planning performance, rather than the instantaneous state of a single point in the task trajectory path. This represents the sowing feasibility index. The higher the sowing feasibility index, the lower the value of the objective cost function. This represents the variance of the predicted loading resistance value calculated by the sparse Gaussian process regression model. It is used to characterize the uncertainty of the regression prediction of the sparse Gaussian process regression model. Loading resistance is used as a penalty term; the smaller the loading resistance, the lower the value of the objective cost function. This represents the smoothness of the work trajectory, and its value is a negative value. Its function is to constrain the work trajectory, avoid sudden changes in sharp turns, and ensure the smoothness of motion and the feasibility of dynamics. The higher the smoothness, the lower the cost function value. , , These are the set weights, and the sum of the three values ​​is 1. For example, they are 0.5, 0.3, and 0.2 respectively. They can be adjusted adaptively according to the actual situation.

[0060] During the work trajectory search, the search objective is to minimize the cost function, i.e., min(J), ensuring that the cost of the work trajectory is minimized. A bidirectional optimal fast expanding random tree algorithm is used to search for the work trajectory within the trajectory sampling area, ultimately finding the work trajectory with the minimum cost value as the optimal work trajectory for the loader.

[0061] Since there are various trajectory sampling areas, operation trajectory planning can be performed more frequently in high-shovelability areas, while in low-shovelability areas, operation trajectory planning should be minimized to reduce the difficulty of loading operations. The trajectory sampling density in high-shovelability areas is greater than that in ordinary areas; for example, the trajectory sampling density in high-shovelability areas can be set to 3 times that of ordinary areas. The trajectory sampling density in ordinary areas is greater than that in low-shovelability areas; for example, the trajectory sampling density in low-shovelability areas can be set to 1 / 3 times that of ordinary areas.

[0062] The optimal operating trajectory includes the depth curve of the bucket insertion into the material pile to be loaded, the bucket lifting height curve, and the bucket tilt angle curve, which are used to characterize the insertion depth, lifting height, and tilt angle of the bucket. Of course, depending on the distribution of the material pile to be loaded underground and the limitations of the underground surface path, the loader must perform turning and position adjustment in some operating positions. Therefore, the final generated optimal operating trajectory is not continuous, but is formed by the combination of multiple operating trajectories. The planning time of a single trajectory is generally limited to no more than 200ms.

[0063] In this embodiment of the invention, during specific operation trajectory planning, the underground space boundary and the kinematic feasibility boundary of the loader are used as search constraints. This allows the loader's trajectory planning to adapt to the narrow working environment of underground roadways, ensuring the feasibility of the operation trajectory planning. Furthermore, by dividing the trajectory sampling area using the shovelability index in the three-dimensional spatial distribution model, high-resistance dense areas or large-block-rich areas can be selectively avoided, preventing bucket jamming or empty shoveling, reducing the difficulty of shoveling operations, and achieving efficient and stable autonomous shoveling operations.

[0064] Step 104: Control the loader to perform loading operations on the stockpile based on the optimal working trajectory.

[0065] After planning the work trajectory in the trajectory sampling area of ​​the three-dimensional pose space, the control command of the loader can be generated based on the optimal work trajectory. The movement of the loader, the rotation and lifting of the bucket are controlled according to the insertion depth, lifting height and tilting angle of the bucket corresponding to each trajectory sampling point in the work trajectory, so as to perform the loading operation on the pile of material to be loaded.

[0066] In one possible implementation, when controlling the loader to perform the loading operation on the stockpile based on the optimal working trajectory, the following is also included: Based on the rodless chamber pressure data and rod chamber pressure data collected from the boom cylinder and bucket cylinder of the loader, the actual loading resistance experienced by the bucket during operation is calculated. The difference between the actual loading resistance and the expected loading resistance corresponding to the optimal working trajectory is input to the impedance controller to obtain the bucket position adjustment amount, and the actual working trajectory during bucket operation is corrected based on the bucket position adjustment amount.

[0067] Here, during the loading operation, pressure sensors installed in the hydraulic system of the loader can collect real-time pressure data of the rodless and rod-side chambers of the boom cylinder and bucket cylinder, with a sampling frequency set to 100Hz. Based on this pressure data, the actual loading resistance experienced by the bucket can be obtained through lever arm conversion. Furthermore, according to the optimal operating trajectory, the expected loading resistance under real-time loading operation can be determined. This expected loading resistance is a theoretical loading resistance value determined based on the operating trajectory search. Comparing this theoretical loading resistance value with the actual loading resistance measured in real time can determine the difference between the two, and the difference can be calculated as the loading resistance deviation.

[0068] In this embodiment of the invention, the insertion depth, lifting height and tilting angle of the shovel bucket are controlled by an impedance controller according to the control command of the shovel. The shoveling resistance deviation is input into the impedance controller to calculate the bucket position adjustment amount. Based on the bucket position adjustment amount, the actual working trajectory of the bucket during operation is corrected so that the actual working trajectory is consistent with the theoretical working trajectory.

[0069] The impedance controller decouples configuration parameters across three degrees of freedom in the bucket coordinate system to control the movement of the loader and bucket, thereby performing the loading operation. Based on the optimal operating trajectory, the key parameters for controlling the loader are the insertion depth into the material pile, the bucket lifting height, and the bucket tilt angle. Specifically, the impedance controller uses a first-order impedance model in the bucket insertion direction to automatically decelerate and retract the bucket when encountering resistance. It also uses a second-order impedance model in the bucket lifting direction to forcefully control the bucket lifting trajectory. Finally, it uses a second-order impedance model in the bucket tilt direction to adaptively adjust the bucket tilt angle during insertion and forcefully control the bucket retraction angle during retraction.

[0070] For controlling the depth of insertion into the material pile to be loaded, an impedance controller employs a first-order impedance model in the bucket insertion direction to control the bucket to automatically decelerate and retract when encountering resistance. The first-order impedance model implements low-stiffness control and adaptive damping. Adaptive damping dynamically adjusts the damping coefficient to prevent bucket oscillation and ensure contact stability under low stiffness. Low-stiffness control refers to actively reducing the bucket's resistance to elastic deformation, allowing it to quickly retract through controllable elastic deformation when encountering external resistance, rather than directly resisting external forces, thus avoiding rigid impacts. This enables the bucket to automatically decelerate and retract when encountering resistance, protecting the mechanical structure and achieving compliant loading in complex material pile operations.

[0071] For controlling the bucket lifting height, an impedance controller is used to employ a second-order impedance model in the bucket lifting direction to force control of the bucket lifting trajectory. The second-order impedance model implements high-stiffness control to ensure the accuracy of the lifting trajectory, that is, it forces the bucket lifting height to conform to the bucket lifting height set in the optimal operating trajectory to guarantee the accuracy of the lifting trajectory.

[0072] For controlling the bucket tilt angle, the impedance controller still uses a second-order impedance model in the bucket tilt direction. This second-order impedance model is configured with piecewise time-varying stiffness, and its control process is divided into two cases. The first is the bucket insertion stage, which still uses a low-stiffness control strategy, allowing the bucket to tilt when it is inserted into the material pile to be loaded, and adaptively fine-tuning the tilt angle. The second is the bucket retraction stage, which uses a high-stiffness control strategy, forcibly controlling the bucket tilt angle to follow the bucket tilt angle set in the optimal operating trajectory when the bucket is retracted after being inserted into the material pile to ensure accurate bucket retraction angle.

[0073] For example, the stiffness set in the bucket insertion direction is 5×10. 4 N / m, foundation damping is taken as 2×10 N / m 4 N·s / m, adaptive damping coefficient is taken as 0.5, maximum reference resistance is taken as 8×10 4 N. In the bucket lifting direction, the bucket's inertial mass is taken as 500 kg, and the stiffness is set to 5 × 10⁻⁶. 5 N / m, damping is taken as 3×10 4 N·s / m. The moment of inertia in the bucket tilting direction is taken as 120 kg·m², the damping is taken as 800 N·m·s / rad, the stiffness set during the bucket insertion stage is taken as 2000 N·m / rad, and the stiffness set during the bucket retraction stage is taken as 3000 N·m / rad.

[0074] In this embodiment of the invention, an impedance controller is designed to control the loading operation of the loader, ensuring that the actual loading resistance experienced by the bucket during operation strictly matches the theoretically planned optimal operating trajectory, thus guaranteeing efficient execution of the loading operation. Furthermore, a three-stage decoupled segmented variable stiffness control strategy achieves compliant control of the loading force in the bucket insertion direction, bucket lifting direction, and bucket tilting direction.

[0075] In one possible implementation, when the loader is controlling the loader to perform the loading operation on the stockpile based on the optimal working trajectory, it also includes: real-time acquisition of actual loading resistance data of each position coordinate in the three-dimensional pose space of the loader bucket, and constructing a training dataset based on the actual loading resistance data and the position coordinates. The training dataset is input into a sparse Gaussian process regression model used to determine the shovelability index, and the recursive Bayesian update algorithm is invoked to update the parameters of the sparse Gaussian process regression model.

[0076] Here, during each loading operation, within the three-dimensional pose space of the loader bucket, actual loading resistance data at each position coordinate can be collected in real time using devices such as pressure sensors. This data represents the magnitude of the resistance experienced by the bucket at the corresponding actual position during the loading operation. This actual loading resistance data is strictly bound to the corresponding position coordinate to form a training dataset. Thus, each position coordinate and its bound actual loading resistance data constitute a training sample. Then, the actual loading resistance data of the corresponding position coordinates in the training sample are used as labels, and combined with the multimodal physical features extracted during the current loading operation to further form a complete training sample, thereby constructing the training dataset.

[0077] The training dataset is used to train a sparse Gaussian process regression model for determining the shovelability index, in order to update the hyperparameters (such as signal variance hyperparameter, hyperparameter ...) corresponding to the kernel function in the sparse Gaussian process regression model. The loss function of the sparse Gaussian process regression model uses variational free energy or deterministic training conditions as a lower bound for the approximate marginal likelihood. The objective is to minimize the negative log-marginal likelihood while simultaneously minimizing the prediction error. The model parameters are iteratively updated using the loss function, employing a recursive Bayesian update algorithm with an update cycle of 0.1 seconds. During a single update, no more than 50 new samples are added to the training dataset to optimize the model's hyperparameters online. Training stops when the change in the loss function falls below a set threshold or when the maximum number of iterations is reached.

[0078] In this embodiment of the invention, during actual loading operations, the actual loading resistance at each position of the loader bucket is collected in real time as training data, and the sparse Gaussian process regression model that determines the loadability index is updated in real time. As a result, the loadability index and prediction variance of the corresponding region in the three-dimensional spatial distribution model of the loadability are also updated synchronously, so that the model prediction accuracy continues to improve with the increase of the number of loading operations.

[0079] Furthermore, in real-time loading operations, for empty areas in the stockpile that have already been loaded, the shovelability index is directly set to zero and removed from the trajectory sampling area of ​​subsequent trajectory planning, thus realizing a closed-loop learning mechanism of prior inference, subsequent correction, and dynamic iteration.

[0080] In one possible implementation, when controlling the loader to perform the loading operation on the stockpile based on the optimal working trajectory, the following is also included: Obtain the longitudinal dynamic model of the loader; The longitudinal acceleration, mass, traction force, and ground rolling resistance of the loader during the loading operation are input into the longitudinal dynamic model to calculate the current total loading resistance of the loader. The maximum adhesion force between the loader and the ground is calculated based on the preset adhesion coefficient between the loader tires and the ground and the normal load of the loader drive wheel; When the ratio of the current total loading resistance to the maximum adhesion is greater than the first threshold, the insertion angle and insertion speed of the loader bucket are reduced. When the ratio of the current total loading resistance to the maximum adhesion is less than the second threshold, the insertion depth of the bucket of the transporter is increased, and the second threshold is less than the first threshold.

[0081] Here, in this embodiment of the invention, a control mechanism for coordinated anti-slip during actual loading operations is implemented for the loader. Based on the principles of dynamics and physics, a longitudinal dynamic model of the loader is pre-constructed, and the model expression is as follows:

[0082] in, This indicates the total mass of the loader, for example, 18 tons (t). This represents the longitudinal acceleration of the loader during operation. This indicates the traction force that drives the loader. The rolling resistance experienced by the loader during its movement is expressed as f·mg, where g represents the acceleration due to gravity, mg is the weight of the loader, and f represents the rolling resistance coefficient, with a value of 0.25. This indicates the current total resistance to loading by the loader.

[0083] In the expression of the above longitudinal dynamics model, only the current total loading resistance is considered. Since the longitudinal acceleration, mass, traction force, and ground rolling resistance of the loader are unknowns, the total current loading resistance of the loader can be calculated by inputting these parameters into the longitudinal dynamic model. .

[0084] Furthermore, based on the preset adhesion coefficient between the loader tires and the ground, and the normal load on the loader's drive wheels, the maximum adhesion force between the loader and the ground is calculated and expressed as: The preset coefficient of adhesion between the loader tires and the ground is denoted as... Specifically, this value is determined based on the material of the granite floor slab in the underground tunnel, and is taken as 0.6. The normal load on the loader's drive wheel can be measured in advance and denoted as N, for example, 10 tons (t). The maximum adhesion force is then calculated as follows: .

[0085] Furthermore, calculate the total resistance of the current loading process. With maximum adhesion The ratio is determined by several factors. If the ratio is greater than the first threshold, it indicates that the total resistance of the shoveling is too high, resulting in poor safety for the shovel operator. In this case, control commands can be generated to reduce the insertion angle and speed of the shovel bucket, decreasing the insertion angle by 3° and the insertion speed by 20% to reduce shoveling resistance. Simultaneously, the drive force output of the walking system is matched to maintain the total shoveling resistance within the safe margin of maximum ground adhesion. If the ratio is less than the second threshold, it indicates that the bucket still has spare capacity. Therefore, control commands are generated to increase the insertion depth of the shovel bucket by 0.1m to shovel more material and improve single-cycle shoveling efficiency. The first and second thresholds can be set according to actual needs; for example, the first threshold could be 0.75, and the second threshold could be less than the first threshold, for example, 0.4.

[0086] In this embodiment of the invention, during actual bucket operation, the current total resistance of the loader is calculated in real time based on the longitudinal dynamics model of the loader, and the loading operation is adjusted accordingly. This not only ensures the movement safety of the loader, but also appropriately improves the working efficiency of the loader's loading operation.

[0087] In one possible implementation, when controlling the loader to perform the loading operation on the stockpile based on the optimal operating trajectory, the method further includes: The real-time operating status data of the loader when it performs shoveling operations on the material pile to be shoveled is obtained, and the real-time operating status data is input into the pre-trained reinforcement learning model to generate the output action strategy of the impedance controller to control the shoveling operation. The execution actions of the loading operation are adjusted based on the output action strategy.

[0088] Here, this embodiment of the invention also designs a reinforcement learning mechanism that can update and adjust the actions of performing the shoveling operation based on the real-time working status data of the material pile to be shoveled.

[0089] Real-time operating condition data serves as the state space input to the reinforcement learning model, including a 30-dimensional physical feature vector during the loading process, specifically the bucket pose, bucket speed, hydraulic cylinder pressure, loader's travel motor torque, the current load capacity index, and tire slippage rate. The output action strategy serves as the action space output for the reinforcement learning model, including the 6-dimensional impedance parameter adjustment of the impedance controller, denoted as... These represent the virtual moment of inertia adjustment of the bucket around its own Z-axis (vertical direction), the virtual damping adjustment of the bucket along the horizontal forward X-axis, the virtual damping adjustment of the bucket along the Z-axis (vertical direction), the virtual damping adjustment of the bucket around the boom hinge point in the pitch direction, the virtual stiffness adjustment of the bucket along the horizontal forward X-axis, and the virtual stiffness adjustment of the bucket around the boom hinge point in the pitch direction. These impedance parameter adjustments are all output in real time by the pre-trained reinforcement learning model, serving as the basic impedance parameters for dynamically correcting the impedance controller according to the output action strategy, thereby adjusting the execution actions of the loading operation.

[0090] The reinforcement learning model's specific network structure consists of two parts: a policy network and an evaluation network. Both the policy network and the evaluation network are 4-layer fully connected neural networks, each containing 256 neurons. The activation function is a leaky linear rectified function with a negative slope of 0.01. The learning rate of the policy network is... The learning rate of the network is used to evaluate its performance. The capacity of the experience replay pool corresponding to the output action strategy is set to 30,000.

[0091] The reinforcement learning model iterates online during continuous operation, updating the network parameters every 10 loading cycles. It automatically adjusts the stiffness and damping parameters of the impedance controller based on the current real-time working condition data, so that the loading performance continuously converges to the optimal state of multi-objective integration.

[0092] During the network parameter update process, the reward function of the reinforcement learning model is as follows:

[0093] Where R represents the reward function value calculated by the reward function. This represents the upper limit of energy consumption for the loader to perform loading operations, and E represents the actual energy consumption of the loader to perform loading operations. This indicates the set loading cycle time for the loader to perform one loading operation. This indicates the actual loading cycle time of a loader performing one loading operation. This indicates the slip ratio of the loader's tires. This indicates the bucket fullness rate when a loader performs loading operations. , , , All of these represent preset weighting coefficients.

[0094] The tire slip ratio is calculated by comparing the travel motor speed with the laser odometer speed. The actual loading cycle time and actual energy consumption can all be measured or estimated in real time during a single loading operation. The bucket fullness rate can be calculated during a single loading operation by measuring the volume of the material to be loaded and then calculating the ratio of that volume to the total bucket volume. The upper limit for loading energy consumption can be preset to 1.2 kWh, and the set loading cycle time can be 25 seconds. (The text also mentions preset weighting coefficients.) , , , The sum of the values ​​of the four is 1, for example , , , It can be set reasonably according to actual needs. For example , , A balance should be maintained as much as possible to avoid increasing energy consumption in pursuit of full hopper rate, and to avoid sacrificing energy consumption and full hopper rate in pursuit of operating speed. For hard rocks or slippery materials, The size should be increased accordingly to protect the hardware of the loader.

[0095] The specific training method for the reinforcement learning model here can refer to the conventional training method of reinforcement learning models in existing technologies, and will not be elaborated further.

[0096] In this embodiment of the invention, the basic impedance parameters of the impedance controller are optimized in real time by using a reinforcement learning model under the real-time working condition of the loading operation. This can effectively enable the loading performance of the loader to continuously converge to reach the optimal state as much as possible. Under the premise of ensuring operational safety, it further improves the adaptive capability under complex working conditions, and is especially suitable for underground operation scenarios with large fluctuations in ore and rock properties and poor floor conditions.

[0097] In summary, this invention first analyzes the multimodal physical characteristics of the surface of the material pile to be loaded through multi-source visual perception. Based on this, it determines the loadability of the material pile. Compared to existing technologies that plan loading paths based on the geometric midpoint of the surface, analyzing the physical characteristics and loadability of the material pile in advance allows for proactive planning of the loader's operating trajectory in conjunction with actual loading conditions, enabling targeted autonomous loading operations, rather than passively adjusting in real time according to actual loading conditions during the loading operation. Furthermore, in the specific operation trajectory planning, the underground space boundary and the kinematic feasibility boundary of the loader are used as search constraints, ensuring that the loader's trajectory planning can adapt to the narrow working environment of underground roadways and guaranteeing the feasibility of the operation trajectory planning. The three-dimensional spatial distribution model constructed by the shovelability index is used as the basis for trajectory planning to carry out the shoveling operation. This realizes the operation process from perception to analysis to planning, which enables the shoveling operation to actively avoid high resistance dense areas or large rich areas of material piles and avoid areas with abnormal resistance, thus achieving efficient and stable autonomous shoveling operation, while also ensuring the safety and reliability of unmanned operation.

[0098] The following is a detailed description of the autonomous loading and unloading control device for underground loader provided by this invention.

[0099] like Figure 2 As shown, the autonomous loading operation control device for underground loader specifically includes: a physical feature extraction module 201, a shovelability determination module 202, a three-dimensional space modeling module 203, an operation trajectory planning module 204, and a bucket operation control module 205.

[0100] Specifically, the physical feature extraction module 201 is used to acquire multi-source visual data collected from the surface of the material pile to be shoveled underground, and extract multi-modal physical features of the surface of the material pile to be shoveled from the multi-source visual data; the shovelability determination module 202 is used to determine the shovelability index of the material pile to be shoveled based on the multi-modal physical features; the three-dimensional space modeling module 203 constructs a three-dimensional space distribution model characterizing the shovelability of the surface of the material pile to be shoveled based on the shovelability index; the operation trajectory planning module 204 is used to construct constraints on the operation trajectory of the shoveler based on the underground space boundary and the kinematic feasibility boundary of the shoveler, and search for the operation trajectory in the three-dimensional pose space of the shoveler bucket based on the constraints and the three-dimensional space distribution model to obtain the optimal operation trajectory of the shoveler; and the bucket operation control module 205 is used to control the shoveler to perform shoveling operations on the material pile to be shoveled based on the optimal operation trajectory.

[0101] The autonomous loading and unloading operation control device for underground loaders provided in the above embodiments can realize the technical solutions described in the embodiments of the autonomous loading and unloading operation control method for underground loaders. The specific implementation principles of each module or unit can be found in the corresponding content in the embodiments of the autonomous loading and unloading operation control method for underground loaders, and their technical effects can also be referred to each other. They will not be repeated here.

[0102] This invention also provides an autonomous loading and unloading control system for an underground loader, including sensors, communication equipment, and electronic equipment 300, wherein the electronic equipment 300 communicates with the sensors through the communication equipment.

[0103] like Figure 3 As shown, the electronic device 300 includes a processor 301, a memory 302, and a display 303. Figure 3 Only some components of the electronic device 300 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0104] In some embodiments, memory 302 may be an internal storage unit of electronic device 300, such as a hard disk or memory of electronic device 300. In other embodiments, memory 302 may also be an external storage device of electronic device 300, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 300.

[0105] Furthermore, the memory 302 may include both internal storage units of the electronic device 300 and external storage devices. The memory 302 is used to store application software and various types of data installed on the electronic device 300.

[0106] In some embodiments, processor 301 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 302 or process data, such as the autonomous loading and unloading operation control method for underground loader in this invention.

[0107] In some embodiments, display 303 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 303 is used to display information from electronic device 300 and to display a visual user interface. Components 301-303 of electronic device 300 communicate with each other via a system bus.

[0108] In some embodiments of the present invention, when the processor 301 executes the computer program in the memory 302, the following steps can be implemented: acquiring multi-source visual data collected from the surface of the material pile to be shoveled underground, and extracting multimodal physical features of the surface of the material pile to be shoveled from the multi-source visual data; determining the shovelability index of the material pile to be shoveled based on the multimodal physical features; constructing a three-dimensional spatial distribution model characterizing the shovelability of the surface of the material pile to be shoveled based on the shovelability index; constructing constraints on the working trajectory of the shoveler based on the underground space boundary and the kinematic feasibility boundary of the shoveler, and performing a working trajectory search in the three-dimensional pose space of the shoveler bucket based on the constraints and the three-dimensional spatial distribution model to obtain the optimal working trajectory of the shoveler; and controlling the shoveler to perform shoveling operations on the material pile to be shoveled based on the optimal working trajectory.

[0109] It should be understood that when the processor 301 executes the computer program in the memory 302, in addition to the functions described above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0110] Furthermore, the embodiments of the present invention do not specifically limit the type of electronic device 300 mentioned. Electronic device 300 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, electronic device 300 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0111] Sensors are used to collect multi-source visual data from the surface of the underground material pile to be shoveled.

[0112] A communication device for uploading multi-source visual data to the memory 302 of an electronic device 300, so that the processor 301 can execute the computer program in the memory 302.

[0113] In some embodiments, sensors may include scientific instruments for capturing images, industrial cameras, depth cameras, thermal imaging cameras, visible light vision cameras, lidar, long-range radar, high-end scanners, camera lenses, inertial navigation devices, inertial measurement units (IMUs), etc. Communication equipment in some embodiments may be network communication devices deployed with Internet or Ethernet network links, such as switches, routers, firewalls, gateways, bridges, repeaters, wireless access points (APs), modems, optical transceivers, fiber optic transceivers, etc.

[0114] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0115] The above provides a detailed description of the autonomous loading and unloading control method, device, and system for underground loader provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for controlling autonomous loading operations of an underground loader, characterized in that, include: Acquire multi-source visual data collected from the surface of the material pile to be shoveled underground, and extract multimodal physical features of the surface of the material pile to be shoveled from the multi-source visual data; The shovelability index of the stockpile to be shoveled is determined based on the aforementioned multimodal physical characteristics; A three-dimensional spatial distribution model characterizing the shovelability of the surface of the material pile to be shoveled is constructed based on the shovelability index. The constraints of the loader's operating trajectory are constructed based on the underground space boundary and the kinematic feasibility boundary of the loader. Based on the constraints and the three-dimensional spatial distribution model, the operating trajectory is searched in the three-dimensional pose space of the loader bucket to obtain the optimal operating trajectory of the loader. Based on the optimal operating trajectory, the loader is controlled to perform loading operations on the stockpile to be loaded.

2. The method for controlling autonomous loading and unloading operations of an underground loader according to claim 1, characterized in that, The multi-source visual data includes point cloud data acquired by lidar, thermal imaging data acquired by a thermal imaging camera, and visible light images acquired by a visible light vision camera. Extracting multimodal physical features from the surface of the material pile to be loaded from the multi-source visual data includes: Dust attenuation compensation is performed on the point cloud data, and the geometric texture features of the material pile to be shoveled are extracted from the compensated point cloud data. Remove the thermal radiation background from the underground roadway wall in the thermal imaging data, extract the effective temperature data from the background-removed thermal imaging data, and calculate the surface temperature decay rate of the effective temperature data as a thermophysical feature. The density of crack edges and the direction of the main cracks in the visible light image of the material to be loaded are extracted as texture structure features. Align the geometric texture features, the thermophysical features, and the texture structure features to the same vehicle coordinate system to obtain the multimodal physical features of the surface of the material pile to be loaded.

3. The method for controlling autonomous loading and unloading operations of an underground loader according to claim 1, characterized in that, The determination of the shovelability index of the stockpile based on the multimodal physical characteristics includes: The multimodal physical features are input into a pre-trained sparse Gaussian process regression model for regression processing to obtain the predicted values ​​of loading resistance, density, and full bucket rate for each three-dimensional spatial coordinate point inside the material pile to be loaded. The predicted values ​​of loading resistance, compaction, and bucket fullness are weighted according to preset weights, and the weighted results are summed to obtain the shovelability index of the stockpile to be shoveled.

4. The method for controlling autonomous loading and unloading operations of an underground loader according to claim 1, characterized in that, The process of searching for the optimal working trajectory of the loader bucket in the three-dimensional pose space based on the constraints and the three-dimensional spatial distribution model includes: Based on the shovelability index in the three-dimensional spatial distribution model, a trajectory sampling region is determined from the three-dimensional pose space. The trajectory sampling region includes a high shovelability region, a normal region, and a low shovelability region. The high shovelability region is the spatial region in the three-dimensional pose space where the shovelability index is greater than a first index threshold. The low shovelability region is the spatial region in the three-dimensional pose space where the shovelability index is less than a second index threshold. The normal region is the spatial region in the three-dimensional pose space where the shovelability index is between the second index threshold and the first index threshold, where the second index threshold is less than the first index threshold. Based on the aforementioned shovelability index, the smoothness of the work trajectory, and the loading resistance, a target cost function for work trajectory planning is constructed. Using the minimum value of the target cost function as the search objective, the bidirectional optimal fast expanding random tree algorithm is invoked to search for the working trajectory in the trajectory sampling area to obtain the optimal working trajectory of the loader. The optimal working trajectory includes the depth curve of the bucket insertion into the material pile to be shoveled, the bucket lifting height curve, and the bucket tilt angle curve. The trajectory sampling density of the high shovelability area is greater than that of the ordinary area, and the trajectory sampling density of the ordinary area is greater than that of the low shovelability area.

5. The method for controlling autonomous loading and unloading operations of an underground loader according to claim 1, characterized in that, When the loader is controlled to perform loading operations on the stockpile based on the optimal operating trajectory, the method further includes: Based on the rodless chamber pressure data and rod chamber pressure data collected from the boom cylinder and bucket cylinder of the loader, the actual loading resistance experienced by the bucket during operation is calculated. The difference between the actual loading resistance and the expected loading resistance corresponding to the optimal working trajectory is input into the impedance controller to obtain the bucket position adjustment amount, and the actual working trajectory during bucket operation is corrected based on the bucket position adjustment amount. The impedance controller is used to employ a first-order impedance model in the bucket insertion direction to control the bucket to automatically decelerate and retract when it encounters resistance. The impedance controller is used to employ a second-order impedance model in the bucket lifting direction to forcibly control the bucket lifting trajectory. The impedance controller is used to employ a second-order impedance model in the bucket tilting direction to adaptively adjust the bucket tilt angle when the bucket is inserted and to forcibly control the bucket retraction angle when the bucket is retracted.

6. The method for controlling autonomous loading and unloading operations of an underground loader according to claim 1, characterized in that, When the loader is controlled to perform loading operations on the stockpile based on the optimal operating trajectory, the method further includes: Real-time acquisition of actual loading resistance data at each position coordinate in the three-dimensional pose space of the loader bucket; and construction of a training dataset based on the actual loading resistance data and the position coordinates. The training dataset is input into a sparse Gaussian process regression model used to determine the shovelability index, and the recursive Bayesian update algorithm is invoked to update the parameters of the sparse Gaussian process regression model.

7. The method for controlling autonomous loading and unloading operations of an underground loader according to claim 1, characterized in that, When the loader is controlled to perform loading operations on the stockpile based on the optimal operating trajectory, the method further includes: Obtain the longitudinal dynamic model of the loader; The longitudinal acceleration, mass, traction force, and ground rolling resistance of the loader during the loading operation are input into the longitudinal dynamic model to calculate the current total loading resistance of the loader. The maximum adhesion force between the loader and the ground is calculated based on the preset adhesion coefficient between the loader tires and the ground and the normal load of the loader drive wheel; When the ratio of the current total loading resistance to the maximum adhesion force is greater than a first threshold, the insertion angle and insertion speed of the loader bucket are controlled to be reduced. When the ratio of the current total loading resistance to the maximum adhesion is less than a second threshold, the insertion depth of the bucket is controlled to increase, where the second threshold is less than the first threshold.

8. The method for controlling autonomous loading and unloading operations of an underground loader according to claim 1, characterized in that, When the loader is controlled to perform loading operations on the stockpile based on the optimal operating trajectory, the method further includes: The real-time operating status data of the loader when it performs shoveling operation on the pile of material to be shoveled is obtained, and the real-time operating status data is input into the pre-trained reinforcement learning model to generate the output action strategy of the impedance controller to control the shoveling operation. The execution actions of the loading operation are adjusted based on the aforementioned output action strategy; The reward function of the reinforcement learning model is as follows: Where R represents the reward function, This represents the upper limit of energy consumption for the loader to perform loading operations, and E represents the actual energy consumption of the loader to perform loading operations. This indicates the set loading cycle time for the loader to perform one loading operation. This indicates the actual loading cycle time of a loader performing one loading operation. This indicates the slip ratio of the loader's tires. This indicates the bucket fullness rate when a loader performs loading operations. , , , All of these represent preset weighting coefficients.

9. A control device for autonomous loading and unloading operations of an underground loader, characterized in that, include: The physical feature extraction module is used to acquire multi-source visual data collected from the surface of the material pile to be shoveled and loaded underground, and to extract multi-modal physical features of the surface of the material pile to be shoveled and loaded from the multi-source visual data. A shovelability determination module is used to determine the shovelability index of the stockpile to be shoveled based on the multimodal physical characteristics. The three-dimensional spatial modeling module constructs a three-dimensional spatial distribution model representing the shovelability of the surface of the material pile to be shoveled based on the shovelability index. The operation trajectory planning module is used to construct the constraints of the loader's operation trajectory based on the underground space boundary and the kinematic feasibility boundary of the loader, and to search for the operation trajectory in the three-dimensional pose space of the loader's bucket based on the constraints and the three-dimensional spatial distribution model to obtain the optimal operation trajectory of the loader. The bucket operation control module is used to control the loader to perform shoveling operations on the stockpile to be shoveled based on the optimal operation trajectory.

10. A control system for autonomous loading and unloading operations of an underground loader, characterized in that, It includes sensors, communication devices, and electronic devices, wherein the electronic devices communicate with the sensors through the communication devices; The electronic device includes a memory and a processor. The memory is used to store a program. The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of the autonomous loading operation control method for downhole loaders as described in any one of claims 1 to 8. The sensor is used to collect multi-source visual data from the surface of the underground material pile to be shoveled; The communication device is used to upload the multi-source visual data to the memory of the electronic device.

Images