Intelligent remote shovel control method, system, and shovel

Abstract

This application relates to the field of engineering machinery control technology, and discloses an intelligent remote electric shovel control method, system, and shovel. The method includes: real-time acquisition and processing of the electric shovel's boom posture and spatial parameters to obtain shovel pose data; predicting the bucket's three-dimensional spatial trajectory within a preset future time period based on the current and preset historical shovel pose data, historical state data, control input signals, and environmental constraints; performing three-dimensional data perception and acquisition of the electric shovel's operating environment, establishing a three-dimensional obstacle model, and dynamically updating the environmental map; performing spatial intersection judgment between the bucket's three-dimensional spatial trajectory and the obstacle three-dimensional model, and providing early warning of potential collision risks; and executing corresponding intervention operations based on the early warning results according to a preset control strategy and the shovel's working status. This application improves the safety of electric shovel operations, enhances the level of operational intelligence and work efficiency, and possesses excellent adaptability to harsh environments.

Description

Technical Field

[0001] This application belongs to the field of engineering machinery control technology, specifically relating to an intelligent remote electric shovel control method, system, and electric shovel. Background Technology

[0002] During operation, remote-controlled electric shovels have a limited field of vision and lack a three-dimensional perspective. When the bucket, as the working component, moves up, down, left, and right, the following safety hazards exist: 1. Lack of a mechanism to predict the movement trajectory of the bucket end, making it easy to accidentally touch it in confined working spaces or near obstacles; 2. Difficulty in responding to obstacles in real time, such as suddenly appearing equipment, people, or foreign objects; 3. Poor ability for operators to perceive the real-time position of the bucket in remote or semi-blind situations; 4. Some misoperations, such as "exceeding the working range" or "overextending," can easily cause the bucket to collide with structural components.

[0003] Therefore, remote-controlled electric shovels urgently need a control method that can both predict the bucket position and intelligently avoid obstacles to prevent injuries to people, damage to the machine, or equipment damage. Summary of the Invention

[0004] To address the aforementioned issues, this application provides an intelligent remote electric shovel control method, system, and electric shovel to overcome or at least partially overcome the shortcomings of the prior art.

[0005] In a first aspect, this application provides an intelligent remote electric shovel control method, comprising: Bucket position detection steps: Real-time acquisition and processing of the posture and spatial parameters of the electric shovel boom to obtain electric shovel posture data, which includes: the angle values ​​of each connecting joint of the electric shovel boom, and the position coordinates and direction vector of the bucket in three-dimensional space; Trajectory prediction steps: Based on the electric shovel's pose data, historical status data, control input signals, and environmental constraints within the current and preset historical time periods, predict the three-dimensional spatial trajectory of the bucket within the preset future time period; Obstacle perception steps: Perform 3D data perception and collection on the electric shovel's operating environment, establish a 3D model of obstacles, and dynamically update the environmental map; Collision detection steps: Perform spatial intersection detection between the three-dimensional trajectory of the bucket and the three-dimensional model of the obstacle to provide early warning of potential collision risks between the bucket and the obstacle; Control and intervention steps: According to the preset control strategy, based on the working status of the electric shovel, execute the corresponding intervention operation based on the early warning result.

[0006] Secondly, this application also provides an intelligent remote electric shovel control system, the system comprising: The bucket position detection module is used to collect and process the posture and spatial parameters of the electric shovel boom in real time to obtain electric shovel posture data. The electric shovel posture data includes: the angle values ​​of each connecting joint of the electric shovel boom, and the position coordinates and direction vector of the bucket in three-dimensional space. The trajectory prediction module is used to predict the three-dimensional spatial trajectory of the bucket within a preset future time period based on the electric shovel's pose data, historical state data, control input signals, and environmental constraints within the current and preset historical time periods. The obstacle perception module is used to perceive and collect three-dimensional data of the electric shovel's operating environment, build a three-dimensional model of obstacles, and dynamically update the environmental map. The collision detection module is used to make spatial intersection judgments between the three-dimensional spatial trajectory of the bucket and the three-dimensional model of the obstacle, and to provide early warning of potential collision risks between the bucket and the obstacle. The control and intervention module is used to perform corresponding intervention operations based on the working status of the electric shovel and the early warning results, according to the preset control strategy.

[0007] Thirdly, this application also provides an electric shovel, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described intelligent remote electric shovel control method.

[0008] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described intelligent remote electric shovel control method.

[0009] The above-mentioned at least one technical application used in the embodiments of this application can achieve the following beneficial effects: 1. Improved safety of electric shovel operation: By real-time monitoring of the three-dimensional position and attitude of the boom and bucket, combined with the environmental perception module to accurately capture obstacle information in the working environment, the potential collision risk between the bucket's movement trajectory and obstacles can be predicted in advance. In dangerous working conditions, it can also trigger early braking, path adjustment or alarm operations in a timely manner to effectively avoid collision accidents and ensure the safety of equipment, personnel and working area.

[0010] 2. Improved operational intelligence and work efficiency: This application calculates future trajectory points based on current joint angular velocity and other data to avoid collision risks in advance; the obstacle avoidance strategy can guide the electric shovel to automatically adjust its working posture, reduce manual intervention costs, and lower the difficulty of operation; at the same time, it supports feedforward control for continuous operation, optimizes the work process, effectively improves loading efficiency per unit time, and reduces the labor intensity of operators.

[0011] 3. Excellent adaptability to harsh environments: By adopting a multi-modal sensor fusion method, it can effectively adapt to complex operating environments such as poor lighting and unstable signals in mining areas, ensuring the stability of environmental perception and trajectory detection. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic diagram of the structure of an intelligent remote electric shovel control system according to an embodiment of this application is shown; Figure 2 A flowchart illustrating an intelligent remote electric shovel control method according to an embodiment of this application is shown. Figure 3 A schematic diagram illustrating the prediction of a three-dimensional spatial trajectory of a bucket according to an embodiment of this application is shown; Figure 4 A schematic diagram of an OBB collisionbox overlap determination algorithm according to an embodiment of this application is shown; Figure 5 A schematic diagram of the result of an electric shovel according to an embodiment of this application is shown. Detailed Implementation

[0013] To make the objectives, technical claims, and advantages of this application clearer, the technical application of this application will be clearly and completely described below with reference to specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] Figure 1 A schematic diagram of the structure of an intelligent remote electric shovel control system according to an embodiment of this application is shown. Figure 1 As can be seen, the intelligent remote electric shovel control system communicates with the remote control terminal of the electric shovel. The intelligent remote electric shovel control system can be deployed as an edge terminal on the electric shovel body (not shown in the figure), and the remote control terminal of the electric shovel acts as the cloud. The intelligent remote electric shovel control system has necessary communication or electrical connections with the necessary related hardware on the electric shovel body. The necessary related hardware includes, but is not limited to, sensors, communication components, etc.

[0015] The intelligent remote electric shovel control system mainly includes: a bucket position detection module, a trajectory prediction module, an obstacle perception module, a collision judgment module, and a control intervention module.

[0016] The bucket position detection module is used to collect and process the posture and spatial parameters of the electric shovel boom in real time to obtain electric shovel posture data. The electric shovel posture data includes: the angle values ​​of each connecting joint of the electric shovel boom, and the position coordinates and direction vector of the bucket in three-dimensional space. The trajectory prediction module is used to predict the three-dimensional spatial trajectory of the bucket within a preset future time period based on the electric shovel's pose data, historical state data, control input signals, and environmental constraints within the current and preset historical time periods. The obstacle perception module is used to perceive and collect three-dimensional data of the electric shovel's operating environment, build a three-dimensional model of obstacles, and dynamically update the environmental map. The collision detection module is used to make spatial intersection judgments between the three-dimensional spatial trajectory of the bucket and the three-dimensional model of the obstacle, and to provide early warning of potential collision risks between the bucket and the obstacle. The control and intervention module is used to perform corresponding intervention operations based on the working status of the electric shovel and the early warning results, according to the preset control strategy.

[0017] The detailed functions of each module are discussed in the following description of the methods, and will not be repeated here.

[0018] This application also provides an intelligent remote electric shovel control method, which can be based on... Figure 1 The example of the intelligent remote electric shovel control system is provided, but it is not limited to this. Any system or architecture that can implement the business logic of the intelligent remote electric shovel control method of this application is acceptable.

[0019] Figure 2 A flowchart illustrating an embodiment of an intelligent remote electric shovel control method according to this application is shown. Figure 2 As can be seen, this embodiment includes steps S210 to S260: Bucket position detection step S210: Real-time acquisition and processing of the posture and spatial parameters of the electric shovel boom to obtain electric shovel posture data. The electric shovel posture data includes: the angle values ​​of each connecting joint of the electric shovel boom, and the position coordinates and direction vector of the bucket in three-dimensional space.

[0020] The main purpose of this step is to acquire and process the attitude and spatial parameters of key components in the electric shovel boom system in real time. These key components include, but are not limited to, the boom, the lifting boom, and the bucket. The attitude and spatial parameters include, but are not limited to, joint angle data, attitude data, and position data. This data forms the foundation for bucket trajectory prediction and obstacle avoidance functions.

[0021] Various sensors are installed on the electric shovel body, including but not limited to angle sensors, attitude sensors, position sensors, and torque sensors. These sensors are communicatively or electrically connected to the bucket position detection module of the intelligent remote electric shovel control system, which can acquire the relevant attitude and spatial parameters of the electric shovel arm during operation in real time and transmit them to the bucket position detection module of the intelligent remote electric shovel control system.

[0022] Among them, the angle sensor can be a rotary encoder or a potentiometer, preferably a rotary encoder; the angle sensor is installed at key joints such as boom-boom and boom-bucket, and is used to measure the rotation angle of the shaft in real time.

[0023] An attitude sensor (IMU, inertial measurement unit) can be installed at the end of the bucket and boom to acquire local acceleration and angular velocity, which can be used to calculate attitude changes.

[0024] Position sensor, which can be RTK-GNSS, laser rangefinder or visual locator, etc., is used to obtain the three-dimensional absolute position (X, Y, Z) of the bucket end in the world coordinate system.

[0025] Torque sensors, placed in hydraulic cylinders or joints, are used to assess load conditions and help determine motion trends.

[0026] The sensors used in electric shovels, such as IMU, rotary encoder, GNSS, or visual positioning, all have their drawbacks when used individually: for example, IMU can measure acceleration and angular velocity in real time, but it is subject to "drift," and the data deviation becomes larger and larger over time; rotary encoders are accurate in measuring joint angles, but they can only measure relative angles and cannot directly calculate the three-dimensional position of the bucket; GNSS / visual positioning can measure absolute position, but it is easily affected by the obstruction of the mining area and dust, and the data is occasionally inaccurate.

[0027] Therefore, in this application, the data from the above-mentioned multiple sensors are selectively fused together. Data fusion is to "take the best and make up for the worst" and integrate the data from these sensors to obtain more accurate and stable bucket status data than a single sensor.

[0028] It should be noted that multiple sensors support a redundant design. This redundancy can be achieved by using a unified hardware system with primary and backup replacements, or by utilizing only a portion of the multimodal data to execute this application. If multiple sensor types are used, and one fails, other sensors can continue to collect data. In this case, computational accuracy may decrease, but basic obstacle avoidance capabilities are still maintained. This localized redundancy design ensures basic obstacle avoidance capabilities even when critical modules fail, guaranteeing operational continuity and adapting to the complex field mining scenarios of electric shovels.

[0029] In some embodiments of this application, if multiple sensors simultaneously include a rotary encoder and an IMU, GNSS, or a visual locator, then after removing abnormal noise from the acceleration and angular velocity data collected by the IMU at the shovel's boom and bucket, and the joint angle data collected by the rotary encoder at each connecting joint, an extended Kalman filter (EKF) algorithm is used for fusion. The IMU's drift error is compensated based on the rotary encoder's angle data, resulting in precise angle values ​​of each connecting component of the shovel and bucket motion state data, denoted as the first fused data. The three-dimensional position data of the bucket tip collected by the GNSS or visual locator is preprocessed to filter out jump data caused by environmental occlusion. A sliding window optimization algorithm is then introduced to perform a second fusion of the fused precise angle values ​​and bucket motion state data with the GNSS / visual locator position data. The sliding window performs optimal calculations on the multi-source data within a preset time period to obtain the second fused data, thereby improving the position estimation accuracy of the bucket tip in the world coordinate system. The second fused data is then standardized and parsed to output the angle values ​​of each connecting joint of the shovel's boom in real time. Coordinates of the bucket tip in three-dimensional space These data, along with the direction vector, are recorded as electric shovel pose data, used for electric shovel trajectory prediction and obstacle avoidance judgment.

[0030] In some embodiments of this application, the bucket position detection module can also perform error correction and redundancy protection. Specifically, the bucket position detection module has a built-in initialization self-test program and a periodic calibration mechanism. When powered on, it automatically performs hardware on / off checks, initial zero position checks, and data validity checks on each sensor. It automatically corrects sensors with initial zero drift and performs benchmark calibration on angle sensors, IMUs, visual positioners, etc., according to a preset time period or operation duration threshold to compensate for cumulative sensor errors and aging deviations, thereby improving the long-term stability of data detection.

[0031] Furthermore, the bucket position detection module supports a multi-sensor redundancy strategy, configuring primary and backup signal sources for the same detected physical quantity and monitoring the working status and data validity of all sensors in real time. When any primary sensor experiences hardware failure, data jump, or communication interruption, it can seamlessly switch to the backup signal source in milliseconds, ensuring the continuity of bucket status data acquisition and improving the overall robustness of the system.

[0032] In some embodiments of this application, the bucket position detection module integrates a hardware interface adapted to industrial communication, supports CAN, EtherCAT, and Modbus industrial communication protocols, and can transmit the pre-processed and fused standardized bucket status data to the trajectory prediction module and visualization module through a dedicated interface at a 100Hz millisecond-level refresh rate, thereby realizing real-time data interaction with the trajectory prediction module and visualization module.

[0033] The trajectory prediction step S220 predicts the three-dimensional spatial trajectory of the bucket within a preset future time period based on the current and preset historical shovel pose data, historical state data, control input signals, and environmental constraints.

[0034] This step can be performed by the trajectory prediction module, which is mainly used to predict the three-dimensional spatial trajectory of the bucket in the future based on data such as the current and historical bucket position, boom posture and control signals, so as to provide timely decision-making basis for collision judgment and path intervention.

[0035] When predicting the three-dimensional spatial trajectory of the bucket, the input data mainly includes, but is not limited to: the current electric shovel pose data and the electric shovel pose data recorded in the past time (such as the past 0.5 to 2 seconds), which can be provided by the bucket position detection module; historical state data, such as the real-time acceleration, angular velocity, and movement speed of the bucket and boom in the past 0.5 to 2 seconds, and even historical movement trends, such as continuous lifting, uniform rotation, and accelerated lowering; control input signals, including but not limited to the control lever angle, speed command, and hydraulic signal; and environmental constraints, such as the maximum allowable extension angle of the bucket and the limit strategy.

[0036] The main idea behind trajectory extrapolation based on dynamic models is as follows: using the kinematic model of the electric shovel arm, such as DH parameters or inverse kinematics solutions, and jointly considering multiple factors such as the arm's inertia, current acceleration, velocity vector, and bucket load, a recursive formula is used to predict the trajectory in a short time (e.g., within 1-2 seconds). Further support is provided for dynamic refreshing and continuous prediction to maintain system real-time performance; even further, support is provided for expanding the prediction window length (typically 1-3 seconds); this can be used to detect risk areas in the path in advance and generate suggested obstacle avoidance paths, etc.

[0037] The following uses the Denavi-Hartenberg convention (DH) parametric model method as an example to illustrate the prediction process of the bucket's three-dimensional spatial trajectory.

[0038] In some embodiments of this application, the kinematic model is a three-degree-of-freedom serial manipulator model of an electric shovel boom established based on the DH parameter method. In this three-degree-of-freedom serial manipulator model, the shovel chassis rotation joint, boom pitch joint, and boom / bucket slewing joint are defined as link joints in sequence. By setting the joint angle, displacement, length, and torsion angle of each link, the transformation matrix of each link joint is constructed. The total transformation matrix is ​​obtained by multiplying the transformation matrices of each link joint. The calculation formula for the position of the bucket tip is obtained by combining the total transformation matrix with the fixed offset vector of the bucket tip relative to the last link. The future state data of each link joint in the future time is predicted based on the historical state data of each link joint in the historical time, and substituted into the calculation formula for the position of the bucket tip to obtain the three-dimensional spatial position coordinate sequence of the bucket tip in the future time, generating a continuous set of predicted trajectory points of the bucket, that is, the three-dimensional spatial trajectory of the bucket.

[0039] Historical and future state data include, but are not limited to, joint rotation angles (also known as joint angles), angular velocity, and angular acceleration, all of which can be acquired by the aforementioned sensors.

[0040] For details, please refer to Figure 3 , Figure 3 This diagram illustrates a three-dimensional spatial trajectory prediction of a bucket according to an embodiment of the present application. Figure 3 As can be seen, the embodiments of this application abstract the excavator boom as a three-degree-of-freedom serial robotic arm system, which can also be extended to more degrees of freedom as needed, as shown in Table 1 below (where Joint2 is in...). Figure 3 (Not shown in the image).

[0041] Table 1

[0042] The DH (Halving and Deutsche Grammar) model is used to establish coordinate transformations, starting from the base (chassis) coordinate system and ending at the bucket end coordinate system. The DH model describes the relative coordinate transformation relationships between the links. In this embodiment, the following four parameters are used to define the transformation of each joint, as shown in Table 2: Table 2

[0043] Specifically, in this embodiment, the DH parameters for each link are set as shown in Table 3 below: Table 3

[0044] Where: L1 represents the boom length; L2 represents the forearm length (including bucket offset).

[0045] Assume that the tip of the bucket (end point, bucket tip) is offset forward by a fixed length relative to the center of the last link. Then the transformation matrix T for each joint is obtained. i for: ; By transforming the transformation matrix T of each joint i Performing a series of multiplications, we obtain the total transformation matrix as follows: T 03 = T1 T2 T3.

[0046] Multiplying the total transformation matrix by the offset vector of the bucket tip relative to the last link, we obtain the formula for calculating the bucket tip position: ; Where L3 is the fixed offset length of the bucket tip relative to the center of the last connecting rod.

[0047] Furthermore, based on kinematic prediction: taking the current angular velocity as a linear estimate, and using data such as joint rotation angle, angular velocity, and angular acceleration of the linkage joint over historical time periods, the joint rotation angle at a series of future times t+Δt is predicted. The prediction formula is as follows: ; Substituting the predicted joint angle into the aforementioned formula for calculating the bucket tip position, we obtain the bucket tip position at time t+Δt: ; Through iteration, a set of future trajectories can be obtained, which represents the three-dimensional spatial trajectory of the bucket within a predetermined future timeframe: .

[0048] Please refer to this again. Figure 3 ,exist Figure 3 The dashed line represents the predicted three-dimensional spatial trajectory of the bucket over a certain period of time in the future.

[0049] In the obstacle perception step S230, three-dimensional data perception and collection are performed on the electric shovel's operating environment to establish a three-dimensional obstacle model and dynamically update the environment map.

[0050] Specifically, in some embodiments of this application, step S230 includes: acquiring three-dimensional point cloud data and environmental perception of the electric shovel's working environment using at least one of a 3D LiDAR, a depth camera, and a TOF sensor mounted on the shovel body; sequentially performing noise reduction, ground segmentation, and clustering processing on the acquired environmental point cloud data; classifying and identifying targets using a target classification model, with target categories including but not limited to: personnel, vehicles, and rocks; constructing a three-dimensional obstacle model using at least one of voxel mesh, AABB bounding box, OBB bounding box, or 3D Mesh, and uniformly representing all obstacles in the working environment as an obstacle set; dynamically updating the global environmental map at a preset frequency, and unifying the spatial coordinate reference between the environmental map and trajectory prediction.

[0051] Various sensors can be installed on the top of the electric shovel or in front of the bucket. In this embodiment, the following types can be used, but are not limited to: (1) 3D LiDAR for acquiring high-precision point clouds; (2) Depth camera / stereo camera (Stereo / ToF) for acquiring RGB-D images; (3) Preferably, millimeter-wave radar is added to enhance detection capabilities in adverse weather conditions. Furthermore, this embodiment supports multi-sensor fusion to enhance robustness. For example, VLP combined with TOF, or radar combined with a depth camera / stereo camera.

[0052] When modeling obstacles, the collected point cloud data undergoes denoising, ground segmentation, and clustering. A pre-trained target classification model, such as the YOLO AI model, is recommended for obstacle classification and recognition, including people, vehicles, and rocks. Voxel grids, bounding boxes (AABB / OBB), or 3D meshes are used to model obstacles. Finally, the obstacle set is represented as follows: ; Among them, each O i (i=1,2......m) are obstacle objects with volume information.

[0053] In the above modeling process, multi-sensor off-site arrangement and multi-source data fusion processing can be used to reduce the sensing occlusion area.

[0054] In areas where sensors cannot perceive data, historical environmental maps can be used as static backgrounds for reference to compensate for the lack of field of view.

[0055] When the main radar or vision camera fails, the system automatically switches to low-precision radar or inertial navigation for backup perception, ensuring continuous and reliable obstacle avoidance.

[0056] Furthermore, the environmental map containing obstacles is periodically refreshed. In some embodiments of this application, the preferred update frequency is 20Hz. This application does not limit this frequency and it can be set according to the accuracy requirements and computing power. The output format is uniformly converted to the ROS-TF Frame standard format in the coordinate system and shares the reference coordinate origin and attitude reference with the trajectory prediction module.

[0057] The collision judgment step S240 performs a spatial intersection judgment between the three-dimensional spatial trajectory of the bucket and the three-dimensional model of the obstacle, and provides an early warning of the potential collision risk between the bucket and the obstacle.

[0058] The main purpose of this step is to make spatial cross-judgment based on the future bucket position sequence output by the trajectory prediction module and the three-dimensional obstacle model generated by the obstacle perception module, to assess the potential collision risk between the bucket and objects in the environment in real time, and to transmit the results to the control intervention module to execute corresponding actions.

[0059] Specifically, in some embodiments of this application, the collision judgment step S240 includes: interpolating or sampling the three-dimensional spatial trajectory of the bucket at equal time intervals to generate a uniform trajectory key point set; expanding the three-dimensional model of the obstacle by a preset safety factor to introduce a fault tolerance space; using the minimum distance method, OBB (directed bounding box) collision box overlap judgment algorithm, or a combination of the two to perform spatial cross-judgment; recording the timestamp of the first occurrence of collision risk; combining the bucket speed and acceleration to determine whether the remaining time meets the braking and response requirements; and issuing an intervention signal.

[0060] When performing spatial intersection judgment, the inputs include: the three-dimensional spatial trajectory of the bucket output by the trajectory prediction module, the set of three-dimensional obstacle models, and system parameters and safety thresholds.

[0061] The three-dimensional spatial trajectory of the bucket is represented as follows: ; The above formula represents the spatial position of the bucket at N future moments.

[0062] As mentioned earlier, the set of 3D obstacle models is represented as follows: ; Each O in the above formula i (i=1,2......m) represents a spatial model of a point cloud voxel grid obstacle.

[0063] System parameters and safety thresholds include, but are not limited to, the following information: warning distance, emergency stop distance, prediction time window, and current bucket speed. The system uses the following parameters: D_warn: warning distance, e.g., 0.5 meters; D_stop: emergency stop distance, e.g., 0.2 meters; T_window: prediction time window, e.g., 1.5 seconds; V_max: current bucket speed, used to dynamically adjust the safety distance. This application does not limit the size of the thresholds; they can be set according to accuracy and actual needs.

[0064] When making spatial intersection judgments, the three-dimensional spatial trajectory of the bucket is first sampled, and the three-dimensional model of the obstacle is expanded.

[0065] Specifically, the three-dimensional spatial trajectory of the bucket is interpolated or sampled, and the predicted three-dimensional spatial trajectory of the bucket is sampled or interpolated at equal time intervals. A uniformly distributed set of trajectory key points is generated at fixed time intervals (e.g., one frame every 100ms), so that the trajectory points are evenly distributed on the time axis.

[0066] The obstacle 3D model is expanded according to a safety factor. For example, in order to improve the reliability of collision detection and the safety of the system, the constructed obstacle 3D model is expanded according to a preset safety factor. For example, the bounding box size of the obstacle is uniformly expanded by 10% along each dimension to introduce a reasonable fault tolerance space and avoid the missed detection of collision risk due to detection error, motion jitter or model deviation.

[0067] Then, the minimum distance method, the OBB collision method, or a combination of both are used to perform spatial cross-trajectory judgment in order to predict whether there is a risk of collision.

[0068] For the minimum distance method, each trajectory key point obtained above can be sequentially subjected to spatial intersection detection and distance calculation with the three-dimensional model of the obstacle to achieve a full-domain collision risk assessment of the future movement path of the bucket.

[0069] Specifically, for each trajectory key point P i The formula for calculating the minimum distance between it and the 3D models of all obstacles is as follows: If d ij If the value is less than or equal to D_warn, then there is a potential collision risk; otherwise, there is no potential collision risk.

[0070] In some embodiments of this application, it is recommended to use the KD-Tree data structure, and based on this data structure, use the fast nearest point search algorithm to realize the fast minimum distance calculation between trajectory points and obstacle point clouds, which can significantly improve computational efficiency.

[0071] Furthermore, the collision risk level can be determined based on the minimum distance between the trajectory key points and the obstacles. For example, in some embodiments of the present application, three levels are divided, specifically: high risk (red): there is dij ≤ D_stop, and an immediate emergency stop is required; medium risk (orange): D_stop < dij ≤ D_warn, and the system needs to limit the speed or give a warning; low risk (green): all dij > D_warn, no intervention is required, only record.

[0072] In order to simplify the collision judgment, in some embodiments of the present application, the OBB (Oriented Bounding Box) collision box overlap judgment algorithm can also be used. Please refer to Figure 4 , Figure 4 shows a schematic diagram of the OBB collision box overlap judgment algorithm according to an embodiment of the present application. From Figure 4 it can be seen that to calculate whether two OBBs collide, only need to calculate whether the projections of them on the 4 coordinate axes in the figure overlap. If there is an overlap, the two polygons are in contact. In actual calculation, the judgment method is converted to: if the projections of the two polygons on all axes overlap, it is determined that there is a collision risk; otherwise, there is no risk of collision.

[0073] Specifically in this embodiment, the OBB of the obstacle can be directly generated by clustering the environmental 3D point cloud and then fitting; the OBB of the bucket can inherit the calculated position and attitude of the bucket, and then be generated according to a fixed size. For the bucket and each obstacle, it can be calculated whether the projections of the bucket and the obstacle on all axes overlap. If the projections on one or more axes do not overlap, it is determined that there is a collision risk; otherwise, there is no risk of collision.

[0074] Preferably, in some embodiments of the present application, these two methods can be combined and used. First, efficiently judge and then accurately quantify: The first step: quickly screen with the OBB collision box overlap algorithm to first determine whether the bucket and the obstacle "may collide". If there is no overlap in the projection, it is directly determined as "no collision" without subsequent calculation; The second step: if there is an overlap in the projection, then use the minimum distance calculation to accurately quantify, calculate the specific minimum distance, and combine the safety factor and the evaluation risk level to provide a basis for subsequent obstacle avoidance intervention.

[0075] Perform time sorting and reaction prediction according to the risk judgment result. Specifically, record the timestamp t of the trajectory point when the collision risk first appears k , and combine the current bucket speed and acceleration to judge whether the remaining time from the current moment to t k meets the requirements of the braking distance and the system response time; if it is predicted that the response lag will cause a collision, an intervention signal can be sent in advance.

[0076] Finally, the warning results are output, which include, but are not limited to, the following: Risk Level ∈ {None, Low, Medium, High}; the number and location of the nearest obstacle; suggested control actions, such as deceleration, changing direction, and emergency stop; and visual auxiliary marker data that can be used to display the interface.

[0077] In the control intervention step S250, according to the preset control strategy, the corresponding intervention operation is executed based on the working status of the electric shovel and the early warning result.

[0078] Intervention actions include, but are not limited to: speed limits, directional guidance, forced stops, etc.

[0079] The main functions of the control intervention module include, but are not limited to: receiving warning results from the collision judgment module, including risk level and suggested actions; determining the current control mode type, which is either operational or idle, and manual, remote, or autonomous; automatically generating intervention commands such as speed limit, emergency stop, path deviation, and attitude restriction according to the pre-configured control strategy; and sending the intervention commands to the actuators or transmitting them to the remote terminal via the control bus for execution.

[0080] In some embodiments of this application, the control strategy sets response levels according to risk level and current operating state, as detailed in Table 4: Table 4

[0081] Specifically, in some embodiments of this application, the control intervention includes: obtaining the current working status of the electric shovel, which is either in operation or idle, and whether it is a manual, remote, or autonomous working state; confirming whether automatic control is allowed based on user permission configuration; if automatic control is allowed, parsing the collision risk level and its timing, directly invoking the emergency stop logic in high-risk situations, and generating progressive control intervention instructions in medium-risk situations, including but not limited to: setting the maximum hydraulic opening limit, dynamically scaling the operating lever signal, and introducing a slow-in / slow-out coefficient in the control curve; sending the control intervention instructions to actuators such as hydraulic controllers, PLCs, or servo units, obtaining successful instruction execution feedback using a write confirmation mechanism, and transmitting execution confirmation information back via a 5G or LoRa communication module if it is remote control.

[0082] The control and intervention module can be divided into three parts according to business logic: state judgment machine, intervention decision-maker, and command issuance and confirmation mechanism.

[0083] When implementing control intervention, the current status of the electric shovel is first determined, specifically by obtaining its current working status, such as: working / idle, manual / remote / autonomous. Then, based on the user's permission configuration, it is confirmed whether automatic system control is allowed. If not, the process is stopped, and manual intervention is required. If allowed, the system proceeds to the intervention decision-making unit.

[0084] The intervention decision-maker first analyzes the risk level and its timing. If it is high risk, i.e., in an emergency, it directly invokes the emergency stop logic. If it is medium risk, i.e., under controllable risk, it generates progressive control intervention commands, including but not limited to: ① setting the maximum hydraulic opening limit; ② dynamically scaling the control lever signal; ③ introducing a slow-in / slow-out coefficient into the control curve.

[0085] Finally, the command issuance and confirmation mechanism interfaces with the execution system to issue control intervention commands. Actuators include, but are not limited to, hydraulic controllers, PLCs, servo units, etc., and the actuators execute the control intervention commands.

[0086] Furthermore, in some embodiments of this application, a "write confirmation mechanism" is also supported, that is, a success flag must be fed back when the instruction is issued.

[0087] In some embodiments of this application, if the electric shovel is in a remote control state, it is also necessary to send back execution confirmation information through a communication channel, such as 5G / LoRa.

[0088] In summary, this application improves the safety of electric shovel operations: by real-time monitoring of the three-dimensional position and attitude of the boom and bucket, combined with the environmental perception module to accurately capture obstacle information in the working environment, it can predict the potential collision risk between the bucket's movement trajectory and obstacles in advance, and can promptly trigger early braking, path adjustment or alarm operations in dangerous working conditions, effectively avoiding collision accidents and ensuring the safety of equipment, personnel and working area.

[0089] This application improves the level of intelligent operation and work efficiency: by calculating future trajectory points based on current joint angular velocity and other data, this application can avoid collision risks in advance; the obstacle avoidance strategy can guide the electric shovel to automatically adjust its working posture, reduce manual intervention costs and reduce the difficulty of operation; at the same time, it supports feedforward control for continuous operation, optimizes the work process, effectively improves the loading efficiency per unit time, and reduces the labor intensity of operators.

[0090] This application has excellent adaptability to harsh environments: by adopting a multimodal sensor fusion method, it can effectively adapt to complex operating environments such as poor lighting and unstable signals in mining areas, ensuring the stability of environmental perception and trajectory detection.

[0091] Figure 5 A schematic diagram of the structure of an electric shovel according to an embodiment of this application is shown. Figure 5As shown, the electric shovel includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external devices via a network connection. When the computer program is executed by the processor, it implements the functions or steps of an intelligent remote electric shovel control method.

[0092] In one embodiment, the electric shovel provided in this application includes a memory and a processor. The memory stores a database and a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the aforementioned intelligent remote electric shovel control method.

[0093] The above is as stated in this application. Figure 1 The method executed by the intelligent remote electric shovel control system disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0094] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the aforementioned intelligent remote electric shovel control method.

[0095] It should be noted that the functions or steps that can be achieved by the electric shovel or computer-readable storage medium described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0096] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0098] The above-described embodiments are only used to illustrate the technical application of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical applications described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical applications to deviate from the spirit and scope of the technical applications of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for controlling an intelligent remote electric shovel, characterized in that, include: Bucket position detection steps: Real-time acquisition and processing of the posture and spatial parameters of the electric shovel boom to obtain electric shovel posture data, which includes: the angle values ​​of each connecting joint of the electric shovel boom, and the position coordinates and direction vector of the bucket in three-dimensional space; Trajectory prediction steps: Based on the electric shovel's pose data, historical status data, control input signals, and environmental constraints within the current and preset historical time periods, predict the three-dimensional spatial trajectory of the bucket within the preset future time period; Obstacle perception steps: Perform 3D data perception and collection on the electric shovel's operating environment, establish a 3D model of obstacles, and dynamically update the environmental map; Collision detection steps: Spatial intersection detection is performed between the three-dimensional spatial trajectory of the bucket and the three-dimensional model of the obstacle to provide early warning of potential collision risks between the bucket and the obstacle; Control and intervention steps: According to the preset control strategy, based on the working status of the electric shovel, execute the corresponding intervention operation based on the early warning result.

2. The method according to claim 1, characterized in that, In the bucket position detection step, if multiple sensors are included simultaneously: Rotary encoders and IMUs, GNSS or visual positioners; after removing abnormal noise from the acceleration and angular velocity data collected by the IMUs at the boom and bucket of the electric shovel, as well as the joint angle data collected by the rotary encoders at each connecting joint, the extended Kalman filter algorithm is used to fuse them. The angle data of the rotary encoders is used as a reference to compensate for the drift error of the IMU, so as to obtain the accurate angle values ​​of each connecting component of the electric shovel and the movement status data of the bucket, which is recorded as the first fused data. The three-dimensional position data of the bucket tip collected by GNSS or visual locator is preprocessed to filter out the jump data caused by environmental occlusion. Then, a sliding window optimization algorithm is introduced to perform secondary fusion of the fused precise angle value and the bucket motion state data GNSS or visual locator position data. The second fused data is obtained by performing the best calculation on the multi-source data within a preset time through the sliding window. The second fused data is standardized and analyzed to output the angle values ​​of each connecting joint of the electric shovel boom, as well as the three-dimensional spatial coordinates and direction vector of the bucket tip in real time.

3. The method according to claim 1, characterized in that, In the trajectory prediction step, a three-degree-of-freedom serial manipulator model of the electric shovel boom is established based on the DH parameter method. In this three-degree-of-freedom serial manipulator model of the electric shovel boom, the rotating joint of the electric shovel chassis, the pitching joint of the boom, the forearm and the slewing joint of the bucket are defined as linkage joints in sequence. By setting the joint angle, displacement, length and torsion angle of each link, the transformation matrix of each link joint is constructed; The total transformation matrix is ​​obtained by multiplying the transformation matrices of each link joint. The formula for calculating the position of the bucket tip is obtained by combining the total transformation matrix with the fixed offset vector of the bucket tip relative to the last link. Based on the historical state data of each link joint in the historical time period, the future state data in the future time period is predicted, and then substituted into the calculation formula of the bucket tip position to obtain the three-dimensional spatial trajectory of the bucket tip in the future time period.

4. The method according to claim 1, characterized in that, The obstacle perception steps include: acquiring three-dimensional point cloud data and perceiving the working environment of the electric shovel by using at least one of the 3D LiDAR, depth camera and TOF sensor mounted on the electric shovel body; and performing noise reduction, ground segmentation and clustering processing on the acquired environmental point cloud data in sequence. A target classification model is used for classification and identification. The obstacle 3D model is constructed using at least one of voxel mesh, AABB bounding box, OBB bounding box or 3D Mesh, and all obstacles in the working environment are uniformly represented as an obstacle set; The global environment map is dynamically updated at a preset frequency, and the spatial coordinate benchmarks of the environment map and trajectory prediction are unified.

5. The method according to claim 1, characterized in that, The collision determination steps include: The three-dimensional spatial trajectory of the bucket is interpolated or sampled at equal time intervals to generate a uniform set of key points of the trajectory, and the three-dimensional model of the obstacle is expanded according to a preset safety factor to introduce a fault tolerance space. Spatial intersection judgment is performed using the minimum distance method, the OBB collision box overlap judgment algorithm, or a combination of the two. Record the timestamp of the first collision risk, combine bucket speed and acceleration to determine whether the remaining time meets the braking and response requirements, and issue an intervention signal.

6. The method according to claim 5, characterized in that, When performing spatial intersection judgment, the OBB collision box overlap algorithm is used to first determine whether the OBB of the bucket and the obstacle overlaps on all axes. If so, it is determined that there is a collision risk; otherwise, there is no collision risk. If there is a risk of collision, then the minimum distance between the tip of the bucket and the 3D model of the obstacle is calculated again using the minimum distance calculation. Collision risk level is determined based on minimum distance and preset safety threshold.

7. The method according to claim 5 or 6, characterized in that, A fast nearest-point search algorithm is implemented using the KD-Tree data structure to calculate the minimum distance between a trajectory point and a 3D model of an obstacle.

8. The method according to claim 1, characterized in that, Control intervention steps include: Receive early warning results, which include risk levels and recommended actions; Determine the current control mode type, which is either: operational or idle, or manual, remote, or autonomous. Intervention commands are automatically generated based on pre-configured control strategies. The intervention commands are one of the following: speed limit, emergency stop, path deviation, and attitude restriction. Intervention commands are sent to the actuators or transmitted to remote terminals via the control bus for execution.

9. A smart remote electric shovel control system, characterized in that, The system includes: The bucket position detection module is used to collect and process the posture and spatial parameters of the electric shovel boom in real time to obtain electric shovel posture data. The electric shovel posture data includes: the angle values ​​of each connecting joint of the electric shovel boom, and the position coordinates and direction vector of the bucket in three-dimensional space. The trajectory prediction module is used to predict the three-dimensional spatial trajectory of the bucket within a preset future time period based on the electric shovel's pose data, historical state data, control input signals, and environmental constraints within the current and preset historical time periods. The obstacle perception module is used to perceive and collect three-dimensional data of the electric shovel's operating environment, build a three-dimensional model of obstacles, and dynamically update the environmental map. The collision detection module is used to make spatial intersection judgments between the three-dimensional spatial trajectory of the bucket and the three-dimensional model of the obstacle, and to provide early warning of potential collision risks between the bucket and the obstacle. The control and intervention module is used to perform corresponding intervention operations based on the working status of the electric shovel and the early warning results, according to the preset control strategy.

10. An electric shovel, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The steps of the above-described intelligent remote electric shovel control method are implemented when the processor executes the computer program.

Images