An intelligent electric mining shovel system for autonomous operation based on remote monitoring

Abstract

An intelligent electric mining shovel system for autonomous operation based on remote monitoring belongs to the technical field of excavators. First, a lidar is added to the existing structure of the electric mining shovel and a remote control system is installed in the control cabin. The lidar is used to obtain the point cloud data of the ore, and the multi-point cloud modeling accuracy is used to extract the inherent features. Secondly, according to the position and attitude information of the excavator, the planning layer composed of the task planning layer and the motion planning layer can generate the excavation trajectory, and the planned optimal excavation trajectory is converted into a reference trajectory in the upper control center and fed back to the PLC and the control center. Finally, the scale prototype of the modified mining excavator is integrated, and the working condition of the prototype is monitored in real time through 5G communication technology to verify the superiority and applicability of the algorithm. The present invention can fully consider the influence of energy consumption and excavation volume on the excavation work on the basis of a remote excavator, improve the excavation efficiency, increase the full bucket rate, reduce the energy consumption, and can monitor in real time and handle emergencies in a timely manner to ensure the safety of personnel.

Description

Technical Field

[0001] The present invention belongs to the technical field of excavators, and relates to an intelligent electric mining shovel system for autonomous operation based on remote monitoring. Background Art

[0002] The mechanical front shovel excavator for mining (also known as an electric mining shovel) is a large and complex mechanical equipment that integrates "excavation" and "loading" in open-pit mining, and its comprehensive performance directly determines the mining efficiency and safety of the entire mine. The electric mining shovel has the characteristics of high system integration, large bucket capacity, and high technical added value. Moreover, due to its strong excavation adaptability, high operation reliability, and low maintenance cost, it has been widely used in open-pit mining.

[0003] The electric mining shovel generally plays a core role in open-pit mining and is widely used for unloading overlying rock layers or loading minerals. At present, the operation of the electric mining shovel relies on manual operation. Due to the huge volume of the equipment, the driver's line of sight is limited. At the same time, due to the complex and changeable ore and rock, uneven topography and other problems, the driver can only rely on his own experience, and there are obvious deficiencies in aspects such as excavation environment assessment and excavation decision control. In addition, due to the harsh excavation environment and random operation of the driver, its working efficiency and energy consumption largely depend on the proficiency of the operator, often resulting in high energy consumption, underloading or overloading during the excavation process, and even problems such as mine accidents caused by human operation errors. Therefore, there are mainly the following problems during the excavation operation: (1) Low excavation efficiency and high energy consumption: Due to reasons such as experience and the driver's own state, some drivers often "deep dig" or "shallow dig" during the excavation process, resulting in equipment overloading or underloading, seriously affecting the excavation efficiency and energy consumption; (2) Frequent failures: Due to insufficient experience of some drivers, the excavation process is random, and the working device bears strong impact loads, often resulting in accidents such as broken axles and pins, and tipping of the boom; (3) Poor safety: Due to the relatively harsh excavation environment and severe vibration in the cab, the driver is extremely prone to fatigue, resulting in operation errors and causing major safety accidents. In addition, the cultivation of a qualified driver often takes several years. At present, the reduction of the working-age labor force and the decrease in the willingness of young people to work have gradually begun to show an impact on the labor-intensive mining industry.

[0004] To solve the above practical problems, the development of intelligent and remote-controlled mining electric shovels has become an inevitable choice to improve the operation quality and cope with the shortage of labor. Through advanced intelligent technologies and communication technologies, the manual excavation operation with low efficiency, high power consumption and poor safety can be changed to realize intelligent excavation. Among them, the characteristics of high speed, low latency and ultra-large bandwidth of communication technology can meet the remote control requirements of operators to monitor the working conditions in real time and operate with a handle in real time. Combining the two can enable operators to set work tasks in the remote control room and the electric shovel to dig autonomously. A remote control operation room can switch to control multiple intelligent mining electric shovels back and forth, and only a small number of operation rooms are needed to meet the needs of mine development.

[0005] The current research on the trajectory design / trajectory planning of unmanned mining electric shovels has just started, mainly focusing on the design of a single structure or trajectory parameters under manual operation mode, and has not comprehensively considered the impact of the two on the excavation performance, so it cannot meet the performance requirements of high efficiency and energy saving in later autonomous excavation. The existing unmanned excavator technology is not yet mature, and there are still problems such as too slow earthmoving operations, high fuel consumption, expensive accessories, and high maintenance costs of the machine. The actual work is not as flexible as manual operation. Unmanned excavators need to set work tasks in advance and work strictly according to the settings, resulting in the inability to properly handle many special situations existing in the actual working environment. When encountering these situations, professional personnel need to deal with them in a timely manner. Therefore, the existing unmanned excavators cannot completely get rid of manual operation, and the actual working environment is harsh, prone to accidents and causing casualties. Remote excavators are the best solutions to these problems, avoiding direct contact between personnel. Professional personnel can understand the on-site situation in detail through the remote control system and solve problems, which can effectively avoid the threat of the harsh environment to personnel and ensure the timely handling of emergencies. However, remote excavators still have some of the same drawbacks as traditional excavators. Their working efficiency and energy consumption largely depend on the proficiency of operators, and operators need to complete some repetitive work for a long time. Therefore, in order to further improve the excavation efficiency during the operation of mining electric shovels under the premise of safety and economic benefits, it is necessary to propose an intelligent mining electric shovel system for autonomous operation based on remote monitoring. Summary of the Invention

[0006] Aiming at the problems existing in the prior art, the present invention provides an autonomous operation intelligent electric mining shovel system based on remote monitoring, which can fully consider the influence of energy consumption and excavation volume on the excavation work on the basis of a remote excavator, so as to further improve the excavation efficiency of the electric mining shovel. At the same time, the invention conducts trajectory planning based on the material model established from the obtained material point cloud data, and can obtain the excavation trajectory with the largest excavation volume. And the invention predicts the excavation force by using a hybrid method combining a physics-based analysis method and a data-driven method, and modifies the excavation angle according to the prediction result, which can reduce the energy consumption during the operation of the electric shovel. The invention utilizes the characteristics of high speed, low latency and ultra-large bandwidth of 5G communication technology to monitor the working conditions of the intelligent electric mining shovel in real time.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] An autonomous operation intelligent electric mining shovel system under remote monitoring, the autonomous operation intelligent electric mining shovel system (UES) includes an upper control center and a lower control center. The WK-55 type mining excavator prototype was modified according to the requirements of this system. The specific modification is as follows: a camera and a tension sensor are installed on the top of the boom; an inclination sensor is installed on the dipper stick; a 3D lidar and an inertial sensor are installed in the control room; a radio device, an electric control cabinet and a sensor data collection box are installed behind the control room. Specifically as follows:

[0009] In the first step, the electric shovel of the excavator prototype first rotates to the ore truck. At this time, the lidar rotates in front of the material, and the lidar starts to scan the material multiple times to generate material point cloud data. After the lidar scanning is completed, the generated material point cloud data is sent to the upper control center, and the electric shovel rotates in front of the material. The upper control center extracts the inherent features of the data by using a sparse-promoting polynomial response surface (SPPRS), and conducts modeling according to the extracted inherent features.

[0010] Further, formally, assume that the sample set for training consists of samples where \(l\) represents the total number of point clouds, \(i\) represents the number of samples of each point cloud, and \(x,y,z\) represent the three-dimensional coordinates of the point cloud in the lidar. The material surface reconstruction of the \(l\) point clouds is:

[0011]

[0012] In the formula is the feature mapping function, represents the coefficient of the feature, and can be solved by minimizing the optimization target as:

[0013]

[0014] Among them, U is a K-order matrix of learning features, and λ≥0 is the regularization parameter of the regularization term for feature selection.

[0015] In the second step, a Cartesian coordinate system is established on the material model in the first step, and its coordinate system is as Figure 2 shown, O′ represents the bottom of the material model, and S x represents the horizontal direction towards the inside of the material, and S y represents the direction vertically upward perpendicular to the ground. The upper control center uses a hybrid method combining a physics-based analysis method and a data-driven method on the material model to predict the digging force, and the prediction result is shown in Figure 3. The digging force prediction curve is consistent with the ground truth curve. According to the material model, the digging force prediction result, and the position and attitude information of the excavator, the upper control center generates a polynomial function digging trajectory with a large digging volume, low energy consumption, and short digging time in the above coordinate system.

[0016] Furthermore, a point-to-point (PTP) trajectory generation strategy based on polynomials is defined here to describe the entire digging trajectory. In the above coordinate system, the digging trajectory can be expressed as:

[0017]

[0018] where a n (n is a positive integer) represents the polynomial coefficient, and t represents time. The digging trajectory planning problem of the UES is to generate a digging trajectory with a large digging volume, low energy consumption, and short digging time, cut into the ore pile from the initial point to the final point, achieving the minimum energy consumption and time, and the maximum mining volume.

[0019] Furthermore, in generating the digging trajectory, the upper control center first generates an initial trajectory, and then gradually adjusts the polynomial coefficient according to the magnitude of the total objective function J. After multiple iterations, the total objective function J reaches the optimal value. The total objective function J is the sum of the weights of each sub-objective:

[0020] J = ω1J1 + ω2J2 + ω3J3 #(4)

[0021] where J1 is the digging time, J2 is the energy consumption, and J3 is the excavation volume. ω1 is the weight coefficient of the digging time, ω2 is the weight coefficient of the energy consumption, and ω3 is the weight coefficient of the digging volume. The weight coefficients are determined according to the manual operation data of expert operators. By minimizing the objective function through optimization-based and learning-based algorithms, the value of a n can be determined.

[0022] In the third step, to improve the accuracy of the excavation trajectory tracking, a tracking control strategy enhanced by deep learning is used in the upper control center to convert the excavation trajectory planned in the second step into a reference trajectory and then feedback it to the PLC in the electric control cabinet. The PLC controls the electric shovel to work according to the reference trajectory through the frequency converter, inverter and related actuators. After the excavation is completed, the electric shovel rotates to the position of the ore truck and dumps the material onto the ore truck. At this time, the lidar rotates in front of the material and scans the material again to establish a model. The system works in a cycle according to the sequence of the first three steps. At the same time, the tension sensor detects the tension of the rope connecting the boom. Excessive tension will affect the stability of the boom; the tilt sensor detects the angle of the dipper stick. Too large or too small an angle of the dipper stick may cause collisions; the inertial sensor detects the rotation speed of the electric shovel. Excessive speed changes are likely to damage the components; the sensor data acquisition box feeds the sensor data back to the PLC and the upper control center. When the data is abnormal, the PLC controls the entire prototype to stop working.

[0023] In the fourth step, after receiving the abnormal data, the upper control center transmits the status information of the boom, dipper stick and bucket to the remote control room through the radio device. The operator analyzes in the remote control room based on the status information and the camera images and conducts remote control of the excavator to solve the problem. After the problem is solved, the system will start working in a cycle from the first step.

[0024] Advantages of the present invention: The present invention provides an intelligent mining electric shovel system for autonomous operation based on remote monitoring, which can, on the basis of a remote excavator, fully consider the influence of energy consumption and excavation volume on the excavation work, thereby further improving the excavation efficiency of the mining electric shovel. At the same time, the present invention realizes real-time monitoring of the intelligent mining electric shovel in the remote control operation room through the characteristics of high speed, low latency and ultra-large bandwidth of the communication technology, and can timely handle emergencies to ensure personnel safety. The system conducts trajectory planning for the material model and predicts the excavation force, which can improve the full bucket rate and reduce energy consumption. The present invention realizes the autonomous operation of the electric shovel through the PLC, frequency converter, inverter and related actuators, replacing the long-term repetitive work of the operator and reducing the labor intensity of the personnel. Brief Description of the Drawings

[0025] Figure 1 It is a schematic flow chart of an intelligent mining electric shovel system for autonomous operation under remote monitoring according to the present invention.

[0026] Figure 2 It is the UES excavation trajectory of the present invention. The meanings of the letters in the figure are as follows: O′ represents the bottom of the material model, S x represents the horizontal direction towards the inside of the material, S y represents the direction vertically upward perpendicular to the ground.

[0027] Comparison of predicted and actual excavation forces in Fig. 3. (a) Comparison of actual and predicted lifting forces, (b) comparison of actual and predicted pushing forces.

[0028] Comparison of excavation trajectories and manual operations in Fig. 4. (a) Comparison of tracked and predicted excavation trajectories, (b) comparison of automatic and manual operations. Detailed implementation mode

[0029] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0030] The present invention proposes an autonomous mining unmanned excavator system (UES) capable of remote monitoring for handling different working conditions, and its process is as Figure 1 shown. The intelligent electric shovel system structure of the mining excavator includes an upper control center and a lower control center. The UES has a hierarchical structure, combining multi-modal perception, motion planning, and trajectory control. The prototype of the WK-55 type mining excavator was modified according to the requirements of this system; the specific modification is as follows: a camera and a tension sensor are installed on the top of the boom; an inclination sensor is installed on the dipper stick; a 3D lidar and an inertial sensor are installed in the control room; a radio device, an electric control cabinet, and a sensor data collection box are installed behind the control room. Compared with manual operation, under the condition of basically the same excavation volume, the distribution of the excavation trajectory under autonomous operation is more consistent and the power consumption is lower.

[0031] (1) First, the electric shovel of the excavator prototype rotates to the ore truck. At this time, the lidar rotates to the front of the material, and the lidar starts to scan the material multiple times without being affected by the electric shovel during scanning, generating material point cloud data. After the lidar scanning is completed, the generated material point cloud data is sent to the upper control center, and the electric shovel rotates to the front of the material. The upper control center uses a sparsity-promoting polynomial response surface (SPPRS) to extract the inherent features of the data and builds a model based on the extracted inherent features. Formally, assume that the sample set for training consists of samples composed of, where l represents the total number of point clouds, i represents the number of samples of each point cloud, and x, y, z represent the three-dimensional coordinates of the point cloud in the lidar. The material surface of the l point cloud is reconstructed as:

[0032]

[0033] In the formula is the feature mapping function, represents the coefficient of the feature, which can be solved by minimizing the optimization objective as:

[0034]

[0035] where U is the K-order matrix of the learned features, and λ ≥ 0 is the regularization parameter of the regularization term for selecting features.

[0036] (2) Establish a Cartesian coordinate system on the material model in the first step. The coordinate system is as Figure 2 shown, where O′ represents the bottom of the material model, and S x represents the horizontal direction towards the interior of the material, and S y represents the direction vertically upward perpendicular to the ground. The upper control center used a hybrid method combining a physics-based analysis method and a data-driven method on the material model to predict the digging force. The prediction result is shown in Figure 3, and the digging force prediction curve is consistent with the ground truth curve. According to the material model, the digging force prediction result, and the position and attitude information of the excavator, the upper control center generates a polynomial function digging trajectory with a large digging volume, low energy consumption, and short digging time in the above coordinate system. Here, a polynomial-based point-to-point (PTP) trajectory generation strategy is defined to describe the entire digging trajectory. In the above coordinate system, the digging trajectory can be expressed as:

[0037]

[0038] where a n (n is a positive integer) represents the polynomial coefficient, and t represents time. The digging trajectory planning problem of the UES is to generate a digging trajectory with a large digging volume, low energy consumption, and short digging time, cutting into the ore pile from the initial point to the final point, achieving the minimum energy consumption and time, and the maximum mining volume. In generating the digging trajectory, the upper control center first generates an initial trajectory, and then gradually adjusts the polynomial coefficient according to the magnitude of the total objective function J. After multiple iterations, the total objective function J reaches the optimal value. The total objective function J is the sum of the weights of each sub-objective:

[0039] J = ω1J1 + ω2J2 + ω3J3 #(4)

[0040] where J1 is the digging time, J2 is the energy consumption, and J3 is the excavation volume. ω1 is the digging time weight coefficient, ω2 is the energy consumption weight coefficient, and ω3 is the digging volume weight coefficient. The weight coefficients are determined based on the manual operation data of expert operators. By minimizing the objective function through optimization-based and learning-based algorithms, the value of a n can be determined.

[0041] (3) To improve the accuracy of the excavation trajectory tracking, a tracking control strategy enhanced by deep learning is used in the upper control center. The excavated trajectory planned in the second step is converted into a reference trajectory and then fed back to the PLC in the electric control cabinet. The PLC controls the electric shovel to work according to the reference trajectory through the frequency converter, inverter and related actuators. After the excavation is completed, the electric shovel rotates to the position of the ore truck and dumps the material onto the ore truck. At this time, the lidar rotates in front of the material and scans the material again to establish a model. The system works in a cycle according to the sequence of the first three steps. At the same time, the tension sensor detects the tension of the rope connecting the boom, and excessive tension will affect the stability of the boom; the inclination sensor detects the angle of the dipper stick, and too large or too small an angle of the dipper stick may cause collisions; the inertial sensor detects the rotation speed of the electric shovel, and components are prone to damage when the speed changes too much; the sensor data acquisition box feeds the sensor data back to the PLC and the upper control center. When the data is abnormal, the PLC controls the entire prototype to stop working.

[0042] (4) After receiving the abnormal data, the upper control center transmits the status information of the boom, dipper stick and bucket to the remote control room through the radio device. The operator analyzes in the remote control room based on the status information and the camera images, and remotely controls the excavator to solve the problem. After the problem is solved, the system will start working in a cycle from the first step.

[0043] The comparison between the planned excavation trajectory and the actual excavation trajectory is shown in Figure 4(a). It can be found that the actual excavation trajectory almost completely follows the planned curve. The comparison between the automatic excavation trajectory and the manual operation trajectory is shown in Figure 4(b). When the mining excavator is manually operated, the excavation trajectories are discretely distributed, while the trajectories are more consistent during autonomous excavation. At the same time, the consistency of the trajectory distribution will reduce the mining energy consumption. To improve the intelligent level of autonomous mining, the present invention designs an intelligent mining electric shovel system (UES) for autonomous operation based on remote monitoring. The system adopts a hierarchical structure, encodes the knowledge of expert operators, physics-based analysis models and data-driven methods, reduces the calculation time, and realizes the real-time monitoring of the structural performance.

[0044] The above embodiments only represent the implementation modes of the present invention, but should not be construed as limiting the scope of the patent of the present invention. It should be noted that for those skilled in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention.

Claims

1. An intelligent electric mining shovel system for autonomous operation based on remote monitoring, characterized in that, The autonomous operation intelligent mining electric shovel system UES includes an upper control center and a lower control center; it is realized based on a modified prototype of a mining excavator. The modification is as follows: install a camera and a tension sensor on the top of the boom; install an inclination sensor on the dipper arm; install a 3D lidar and an inertial sensor in the control room; install a radio device, an electric control cabinet and a sensor data collection box behind the control room. Specifically as follows: In the first step, the electric shovel of the excavator prototype first rotates to the ore truck. At this time, the lidar rotates in front of the material, and the lidar starts to scan the material multiple times to generate material point cloud data. After the lidar scanning is completed, the generated material point cloud data is sent to the upper control center, and the electric shovel rotates in front of the material. The upper control center uses the sparse-promoting polynomial response surface SPPRS to extract the inherent features of the data and performs modeling based on the extracted inherent features. Step 2: Establish a Cartesian coordinate system on the material model in the first step. O′ represents the bottom of the material model, and S x represents the horizontal direction towards the interior of the material, and S y represents the direction vertically upward perpendicular to the ground; the upper control center uses a hybrid method combining a physics-based analysis method and a data-driven method on the material model to predict the excavation force, and the excavation force prediction curve is consistent with the ground truth curve; According to the material model, the excavation force prediction result, and the position and attitude information of the excavator, the upper control center generates a polynomial function excavation trajectory with a large excavation volume, low energy consumption, and short excavation time in the above coordinate system. Specifically: Define a point-to-point PTP trajectory generation strategy based on polynomials to describe the entire excavation trajectory; in the coordinate system, the excavation trajectory can be expressed as: Among them, a n (where n is a positive integer) represents the polynomial coefficient, and t represents time; the problem of the UES mining trajectory planning is to generate a mining trajectory with a large mining volume, low energy consumption, and short mining time, cutting into the ore pile from the initial point to the final point, achieving the minimum energy consumption and time, and the maximum mining volume; In the process of generating the excavation trajectory, the upper control center first generates an initial trajectory, and then gradually adjusts the polynomial coefficients according to the magnitude of the total objective function J. After multiple iterations, the total objective function J reaches the optimal value. The total objective function J is the sum of the weights of each sub-objective: J = ω1J1 + ω2J2 + ω3J3#(4) Among them, J1 is the excavation time, J2 is the energy consumption, J3 is the excavation volume; ω1 is the excavation time weight coefficient, ω2 is the energy consumption weight coefficient, ω3 is the excavation volume weight coefficient, and the weight coefficients are determined according to the manual operation data of expert operators. In the third step, in order to improve the accuracy of excavation trajectory tracking, a deep learning enhanced tracking control strategy is used in the upper control center to convert the excavation trajectory planned in the second step into a reference trajectory, and then feedback it to the PLC in the electric control cabinet. The PLC controls the electric shovel to work according to the reference trajectory through a frequency converter, an inverter and related actuators. After the excavation is completed, the electric shovel rotates to the position of the ore truck and dumps the material on the ore truck. At this time, the lidar rotates in front of the material and scans the material again to establish a model. The electric shovel system works in a cycle according to the sequence of the first three steps. At the same time, the tension sensor detects the rope tension connecting the boom, the inclination sensor detects the angle of the dipper arm, the inertial sensor detects the rotation speed of the electric shovel, and the sensor data collection box feeds the sensor data back to the PLC and the upper control center. When the data is abnormal, the PLC controls the entire prototype to stop working. In the fourth step, after receiving the abnormal data, the upper control center transmits the status information of the boom, dipper arm, and bucket to the remote control room through the radio device. The operator analyzes in the remote control room according to the status information and the camera image and remotely controls the excavator.

2. The intelligent electric mining shovel system for autonomous operation based on remote monitoring according to claim 1, wherein, In the first step, formally, assume that the sample set for training consists of samples where l represents the total number of point clouds, i represents the number of samples in each point cloud, and x, y, z represent the three-dimensional coordinates of the point cloud in the lidar; the material surface of the l point clouds is reconstructed as: where is the feature mapping function, represents the coefficient of the feature, and is solved by minimizing the optimization objective as follows: Among them, U is the K-order matrix of learning features, and λ ≥ 0 is the regularization parameter of the regularization term for selecting features.

3. The intelligent electric mining shovel system for autonomous operation based on remote monitoring according to claim 1, wherein Determine the value of a by minimizing the objective function through optimization-based and learning-based algorithms n value

Images