Electric shovel remote semi-automatic digging and loading control method and system based on 5G private network

Abstract

The invention provides an electric shovel remote semi-automatic digging and loading control method and system based on a 5G private network, and relates to the technical field of remote control, and the method comprises the steps: carrying out the panoramic environment scanning, and obtaining laser point cloud data and an original video stream; carrying out sensing verification by fusing the double-GPS positioning information, and positioning the relative pose of the mine card; executing dynamic optical flow splicing to obtain a panoramic view; after obstacle avoidance element recognition, obstacle avoidance coordinate space modeling is carried out, and a space obstacle avoidance coordinate array is obtained; after the unloading position is received, excavating action dynamic fitting is carried out, and an automatic operation instruction sequence is generated; and in the process of executing semi-automatic rotary unloading, a panoramic control video stream is fed back through a 5G private network, and high-authority digging and loading intervention is conducted. According to the invention, the technical problems of equipment damage, operation failure or safety accidents caused by untimely remote control response due to certain delay of data transmission and system response during remote control of mining equipment in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of remote control technology, specifically to a remote semi-automatic excavation and loading control method and system for electric shovels based on a 5G private network. Background Technology

[0002] With the continuous development of automation and intelligent technologies in the mining industry, especially the increasing demand for automation and digitalization, remote control technology for mining equipment such as electric shovels is becoming a key technology for improving operational efficiency, ensuring safe production, and reducing labor costs. Traditional mining equipment often experiences delays during remote control due to slow data transmission and sensor response speeds. For example, data acquisition and processing from sensors such as lidar and cameras require a certain amount of time, especially in complex environments, where obstacle recognition and obstacle avoidance decisions are frequently affected by these delays. This delay makes remote control less responsive in complex operating environments, potentially causing the electric shovel to fail to avoid obstacles in time or fail to execute tasks according to the predetermined path, resulting in equipment damage, operational failure, or safety accidents. Summary of the Invention

[0003] This application provides a remote semi-automatic excavation and loading control method and system for electric shovels based on a 5G private network. It aims to solve the technical problem that when mining equipment is remotely controlled, the delay in data transmission and system response makes the remote control response untimely, resulting in equipment damage, operation failure or safety accidents.

[0004] The first aspect disclosed in this application provides a remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network. The method includes: driving a laser radar and camera array pre-installed on the target electric shovel to perform a panoramic environmental scan, obtaining laser point cloud data and a raw video stream; fusing dual GPS positioning information to perform sensor verification of the laser point cloud data and locate the relative pose of the mining truck; performing dynamic optical flow stitching on the raw video stream to obtain a panoramic view; identifying obstacle avoidance elements based on the panoramic view, and then performing obstacle avoidance coordinate space modeling based on the laser point cloud data to obtain a spatial obstacle avoidance coordinate array; receiving the unloading position uploaded by a remote manual platform, using the relative pose of the mining truck and the unloading position as the path planning target, dynamically fitting the excavation action according to the spatial obstacle avoidance coordinate array to generate an automatic operation command sequence; during the process of controlling the target electric shovel to perform semi-automatic slewing and unloading using the automatic operation command sequence, feeding back the panoramic control video stream to the remote manual platform via the 5G private network for high-level excavation and loading intervention.

[0005] The second aspect of this application discloses a remote semi-automatic excavation and loading control system for electric shovels based on a 5G private network. This system is used in the aforementioned remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network. The system includes: a panoramic environment scanning module for driving a pre-installed lidar and camera array on the target electric shovel to perform panoramic environment scanning, obtaining lidar point cloud data and raw video stream; a sensor verification module for fusing dual GPS positioning information to perform sensor verification of the lidar point cloud data and locate the relative pose of the mining truck; a dynamic optical flow stitching module for performing dynamic optical flow stitching on the raw video stream to obtain a panoramic view; and an obstacle avoidance coordinate space modeling module. The system is used to identify obstacle avoidance elements based on the panoramic view, and then combine the laser point cloud data to model the obstacle avoidance coordinate space to obtain a spatial obstacle avoidance coordinate array; the excavation action dynamic fitting module is used to receive the unloading position uploaded by the remote manual platform, and then use the relative pose of the mining truck and the unloading position as the path planning target, and perform excavation action dynamic fitting based on the spatial obstacle avoidance coordinate array to generate an automatic operation instruction sequence; the high-authority excavation and loading intervention module is used to feed back the panoramic control video stream to the remote manual platform through the 5G private network during the process of controlling the target electric shovel to perform semi-automatic rotary unloading using the automatic operation instruction sequence, and to perform high-authority excavation and loading intervention.

[0006] One or more technical solutions provided in this application have at least the following beneficial effects:

[0007] By installing a LiDAR and camera array on the electric shovel, the working environment can be scanned from all angles, acquiring laser point cloud data and raw video streams. This provides both three-dimensional spatial information and two-dimensional visual information of the environment. Dynamic optical flow stitching of the raw video stream generates a seamless panoramic view, providing comprehensive and real-time environmental awareness for subsequent obstacle detection, path planning, and work instruction generation. Using the GPS positioning information of the electric shovel and the mining truck, sensor verification can be performed to ensure the accuracy of the laser point cloud data and calculate the relative pose of the mining truck. This process improves positioning accuracy, enabling the electric shovel to accurately determine the relative position between the mining truck and the shovel in complex working environments. Combining the panoramic view and laser point cloud data, obstacles in the environment are identified and modeled, generating a spatial obstacle avoidance coordinate array to ensure obstacle avoidance during excavation. Obstacles are eliminated to ensure the safety and smooth operation of the electric shovel during tasks. Based on the relative posture of the mining truck, the unloading position, and the spatial obstacle avoidance coordinate array, an automatic operation command sequence adapted to the working environment is generated to ensure that the electric shovel can accurately execute tasks, avoid collisions, and achieve efficient operation. The automatic operation command sequence controls the electric shovel to perform semi-automatic slewing unloading, improving the efficiency and accuracy of electric shovel operation while reducing the burden of manual operation. Through a 5G private network, real-time panoramic control video streams are fed back to a remote control platform, allowing remote operators to monitor the operation process in real time. If abnormalities occur or adjustments are needed during execution, the remote control platform can intervene with high-level authority in digging and loading. This intervention authority ensures that operators can intervene and make adjustments at any time during automatic operation, improving the safety and flexibility of the operation.

[0008] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network, provided in an embodiment of this application.

[0010] Figure 2 A schematic diagram of the structure of a remote semi-automatic excavation and loading control system for electric shovels based on a 5G private network, provided in an embodiment of this application.

[0011] Figure labeling: Panoramic environment scanning module 10, sensor verification module 20, dynamic optical flow stitching module 30, obstacle avoidance coordinate space modeling module 40, excavation action dynamic fitting module 50, high-authority excavation and loading intervention module 60. Detailed Implementation

[0012] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0013] Example 1, as Figure 1 As shown in the figure, this application provides a remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network. The method includes:

[0014] The system drives the pre-installed lidar and camera array on the target electric shovel to perform a panoramic environmental scan, obtaining laser point cloud data and raw video streams.

[0015] A pre-installed lidar on the target electric shovel is used for panoramic environmental scanning. The lidar generates laser point cloud data of the surrounding environment by emitting and receiving reflected laser light. This is a technology that uses laser ranging to obtain the position of objects in space. The laser point cloud data contains the coordinate information of the objects in three-dimensional space. A camera array is used to capture raw video streams of the target electric shovel's working environment. These video streams provide visual information about the shovel's surroundings, including obstacles, mining equipment, and other objects. Through the combined operation of the lidar and camera array, the target electric shovel can generate a detailed 3D environment model and video stream within its working environment, providing foundational data for subsequent obstacle avoidance, localization, and path planning.

[0016] By integrating dual GPS positioning information to perform sensing verification of the laser point cloud data, the relative pose of the mining truck is determined.

[0017] The electric shovel is equipped with a GPS device to obtain its position in the global coordinate system, and the mining truck is also equipped with a GPS device to obtain its position in the global coordinate system. The GPS coordinates of the electric shovel and the mining truck are combined, and the accuracy of the laser point cloud data is verified by comparing the spatial relationship between the two positions. Dynamic registration of the laser point cloud data using multiple sensors is performed to ensure that the coordinate system of the laser point cloud data is consistent with the coordinate systems of the electric shovel and the mining truck. Based on the GPS positioning information of the electric shovel and the mining truck, the initial relative pose of the mining truck relative to the electric shovel, i.e., the relative position and orientation between the two, is calculated. This information will be used for subsequent path planning and operation command generation.

[0018] Dynamic optical flow stitching is performed on the original video stream to obtain a panoramic view.

[0019] The obtained raw video stream is processed using a dynamic optical flow stitching method. Optical flow is a technique for calculating the velocity and direction of objects moving between video frames. In this process, motion features in the video stream are analyzed, and multiple video frames are stitched together by calculating motion vectors to construct a seamless panoramic view. The stitched panoramic view provides a panoramic visual effect of the environment surrounding the electric shovel, assisting in subsequent obstacle recognition and path planning.

[0020] After identifying obstacle avoidance elements based on the panoramic view, obstacle avoidance coordinate space modeling is performed in combination with the laser point cloud data to obtain a spatial obstacle avoidance coordinate array.

[0021] A panoramic view is used to detect obstacles in the working environment of the electric shovel. By analyzing the panoramic image, dynamic and static obstacles in the environment, such as mining tools and rocks, are identified. After identifying obstacle avoidance elements, the identified obstacle information is combined with laser point cloud data. The laser point cloud data provides spatial location and shape information of obstacles for obstacle avoidance. By mapping the obstacle information to a three-dimensional coordinate system, a spatial obstacle avoidance coordinate array is constructed. This coordinate array contains the spatial positions of all obstacles for subsequent path planning and obstacle avoidance algorithms.

[0022] After receiving the unloading position uploaded by the remote control platform, the system uses the relative pose of the mining truck and the unloading position as the path planning target, and dynamically fits the excavation action according to the spatial obstacle avoidance coordinate array to generate an automatic operation instruction sequence.

[0023] The unloading location, uploaded remotely by a human operator via a 5G private network, is located in a designated unloading area within the mining area. The electric shovel needs to excavate and transport the ore from the mining point to this location. Based on the obtained relative pose of the ore carrier, the current position of the ore carrier relative to the electric shovel is determined. The ore carrier's pose information serves as the basis for path planning, ensuring that the electric shovel can accurately dock with the ore carrier. Using the relative pose of the ore carrier and the uploaded unloading location as targets, the electric shovel's digging path is determined. This path planning not only ensures the completion of the digging task but also considers obstacles and obstacle avoidance requirements in the surrounding environment. Obstacle avoidance path planning is performed based on a spatial obstacle avoidance coordinate array. This array contains the spatial coordinate information of obstacles, which is used to avoid collisions between the electric shovel and obstacles. While ensuring obstacle avoidance, dynamic fitting of the digging actions is performed. That is, based on the relative position of the ore carrier, the unloading location, and the obstacle avoidance path, the electric shovel's digging actions are adjusted in real time. The fitting process is based on the electric shovel's action capability, digging accuracy, and operating speed to generate an action sequence that meets the task requirements. Finally, an automatic operation instruction sequence is generated based on the dynamic fitting results. These instructions include actions such as the rotation, digging, and unloading of the electric shovel to ensure the smooth completion of the task.

[0024] During the process of using the automatic operation instruction sequence to control the target electric shovel to perform semi-automatic slewing and unloading, the panoramic control video stream is fed back to the manual remote platform through the 5G private network for high-authority excavation and loading intervention.

[0025] After receiving the automatic operation command sequence, the electric shovel performs a semi-automatic slewing and unloading operation. During this process, the shovel rotates according to the instructions and unloads the ore into the ore truck. While the shovel is performing the slewing and unloading, a panoramic control video stream is fed back in real time via a 5G private network. This video stream, provided by the shovel's camera, offers a complete operational perspective, helping remote operators monitor the operation in real time. Upon receiving the panoramic video stream, the remote operator can comprehensively monitor the surrounding working environment, operation progress, and any obstacles or emergencies. If any abnormalities or dangerous situations occur during operation, the remote operator can immediately intervene with high-level access. This means that the remote operator can directly intervene in the shovel's actions, adjusting its operating path or pausing the operation to ensure safety and accuracy. High-level intervention includes, but is not limited to, forcibly stopping the operation, changing the parameters of the digging action, and adjusting the work plan. Such interventions can improve operational efficiency while ensuring safety.

[0026] Furthermore, after identifying obstacle avoidance elements based on the panoramic view, obstacle avoidance coordinate space modeling is performed using the laser point cloud data to obtain a spatial obstacle avoidance coordinate array. The method includes:

[0027] The panoramic view is input in parallel into the dynamic and static detection channels of the obstacle detection model to perform dual-thread obstacle avoidance element recognition, resulting in dynamic obstacle bounding boxes and static obstacle masks. Based on the panoramic view, the dynamic obstacle bounding boxes and static obstacle masks are spatiotemporally aligned to obtain fused obstacle avoidance element labels. The laser point cloud data is mapped to the electric shovel coordinate system using LiDAR SLAM, and obstacle clustering is performed based on the point cloud density to obtain three-dimensional obstacle clusters. The fused obstacle avoidance element labels are then back-projected onto the three-dimensional obstacle clusters, and obstacle attribute weighting association is performed to generate the spatial obstacle avoidance coordinate array.

[0028] The dynamic detection channel is used to detect dynamic obstacles in the video stream, such as moving mining trucks or people; the static detection channel is used to detect constant obstacles in the scene, such as rocks and fixed structures. Dynamic and static detection are performed in parallel, meaning that multi-threading technology is used to process the panoramic view simultaneously in both channels, identifying dynamic and static obstacles separately. Specifically, a motion-based target detection algorithm identifies the bounding boxes of dynamic obstacles in the panoramic view; these bounding boxes are rectangular regions used to mark the position and size of the dynamic obstacles. A semantic segmentation algorithm is used to identify the regions of static obstacles based on the texture features and geometry of the panoramic view, generating a static obstacle mask. This mask is a binary image that marks the pixel regions of the obstacles.

[0029] Spatiotemporal alignment is performed on the two detection results. This involves matching the dynamic obstacle bounding boxes and static obstacle masks based on their time series and spatial location. Since the dynamic obstacle bounding boxes and static obstacle masks may appear in the same spatial region, alignment using timestamps and spatial coordinates is necessary to fuse their obstacle information at the same time and in the same space. After spatiotemporal alignment, fused obstacle avoidance element labels are generated. These labels represent the location, shape, and type of all obstacles and are used for subsequent spatial modeling and path planning.

[0030] In electric shovels, the laser point cloud data generated by LiDAR describes the three-dimensional spatial information of the surrounding environment. To interface with the coordinate system of the shovel itself, LiDAR SLAM (Simultaneous Localization and Mapping) is used to transform the point cloud data from the LiDAR coordinate system to the shovel coordinate system. The LiDAR SLAM algorithm combines the ranging data of the LiDAR and the motion information of the shovel to generate a map of the surrounding environment in real time. The density of the point cloud data can reflect the distribution of obstacles in different areas. Through clustering algorithms, such as DBSCAN, obstacles are clustered according to the density of the point cloud to identify the three-dimensional position of each obstacle. Each cluster represents an obstacle or a part of an obstacle, and the size, shape, and density of the clusters reflect the characteristics of the obstacles.

[0031] The obstacle information from the integrated obstacle avoidance element labels is back-projected onto 3D obstacle clusters. This back-projection process combines obstacle information extracted from the panoramic view with 3D point cloud data from LiDAR to ensure data consistency and matching. Attribute weights are assigned to obstacles within each cluster, meaning each obstacle is assigned a weight based on its type, size, and location. For example, dynamic obstacles have a greater impact than static obstacles and are therefore given higher weights. The location information and attributes of all obstacles are combined to generate a spatial obstacle avoidance coordinate array. This array is a list of 3D spatial coordinates containing the precise location, size, type, and attributes of all obstacles in the electric shovel's working environment. This coordinate array provides the foundational data for subsequent path planning and obstacle avoidance.

[0032] Furthermore, the panoramic view is input in parallel into the dynamic and static detection channels of the obstacle detection model, and dual-thread obstacle avoidance element recognition is performed to obtain dynamic obstacle bounding boxes and static obstacle masks. The method includes:

[0033] The panoramic view is divided into dynamic frame sequences and static keyframes. The dynamic frame sequences are input into the dynamic detection channel built on the YOLOv5 object detection network, and dynamic obstacles are detected in real time based on motion feature temporal analysis, and the dynamic obstacle bounding boxes are output. The static keyframes are input into the static detection channel built on the U-Net semantic segmentation network, and static obstacles are segmented at the pixel level based on texture geometric structure features, and the static obstacle mask is output.

[0034] A panoramic view is a complete sequence of environmental images captured by a camera. To more effectively process and identify dynamic and static obstacles, the panoramic video stream images are first split into two categories: dynamic frame sequences, which contain frames with moving objects in the video, primarily focusing on frames with dynamic objects such as mining trucks, people, and construction vehicles; these frames are used for dynamic obstacle detection. Static keyframes are stable, unchanging frames in the panoramic view; these frames are used to identify static obstacles such as pits, ore piles, or equipment supports. The splitting of dynamic frame sequences and static keyframes mainly relies on temporal changes and image differences in the video stream. Objects in dynamic frame sequences shift over time, while static keyframes are less affected by changes, including the scene background and fixed obstacles.

[0035] The dynamic frame sequence extracted from the panoramic view is input into a YOLOv5 (You Only Look Once Version 5) object detection network. YOLOv5 is a deep learning algorithm specifically designed for real-time object detection in images, particularly suitable for detecting objects in dynamic scenes. The YOLOv5 network not only identifies obstacles through single-frame analysis but also combines temporal analysis to extract the motion features of objects. For example, dynamic obstacles such as mining trucks, personnel, or construction vehicles appear and move across multiple consecutive frames. YOLOv5 can accurately identify and track these dynamic objects based on these motion features. For each dynamic obstacle, YOLOv5 outputs a bounding box, a rectangle that defines the object's position and size in the image. The output rectangle coordinates are used to determine the exact location of the dynamic obstacle.

[0036] The static keyframes extracted from the panoramic view are input into the static detection channel of the U-Net semantic segmentation network. U-Net is a convolutional neural network widely used for image segmentation, particularly suitable for pixel-level classification tasks. The U-Net network performs pixel-level obstacle segmentation based on the texture and geometric features of the static keyframe images. Static obstacles, such as pits, ore piles, and equipment supports, have obvious texture or shape features, allowing U-Net to accurately segment obstacle regions in the image. The output of U-Net is a static obstacle mask, a binary image where obstacle regions are marked as 1 and background regions as 0. The mask defines the specific location and shape of the obstacles and is used for path planning and obstacle avoidance.

[0037] Furthermore, after receiving the unloading position uploaded by the remote control platform, the method uses the relative pose of the mining truck and the unloading position as the path planning target, and dynamically fits the excavation action according to the spatial obstacle avoidance coordinate array to generate an automatic operation instruction sequence. The method includes:

[0038] Starting from the relative pose of the mining truck and ending at the unloading position, a three-dimensional path search space is constructed in the spatial obstacle avoidance coordinate array. Based on the safety distance associated with obstacle type, the shortest collision-free path is fitted in the three-dimensional path search space to generate a global obstacle avoidance path. The global obstacle avoidance path is segmented according to the tangent direction of the obstacles along the way to obtain multiple obstacle avoidance path segments. Dynamic fitting of action parameters is performed on the multiple obstacle avoidance path segments to obtain multiple sets of atomic-level mining actions. The multiple sets of atomic-level mining actions are closed-loop compliant screening and splicing to output the automatic operation instruction sequence.

[0039] Based on the relative pose of the mining truck, the starting point for path planning is determined. The unloading position uploaded by the remote control platform is taken as the target endpoint; this is the location where the ore needs to be unloaded after the electric shovel completes its digging task. The spatial obstacle avoidance coordinate array provides the spatial coordinates of obstacles around the electric shovel. Based on this coordinate information, a three-dimensional path search space is constructed. This search space is calibrated in a three-dimensional coordinate system, where obstacle information and open areas are used for path planning.

[0040] In the spatial obstacle avoidance coordinate array, each obstacle has different physical characteristics and influence range. Based on the characteristics of each obstacle, a safety distance is set for each type. For example, dynamic obstacles, such as moving mining trucks, require a larger safety distance, while static obstacles, such as equipment supports or ore piles, require a smaller safety distance. Based on the obstacle type and safety distance, the shortest collision-free path is calculated in the three-dimensional path search space. This path not only ensures that the electric shovel reaches the destination from the starting point but also avoids interference from obstacles. The method for calculating the shortest path can use traditional path planning algorithms, such as the A* algorithm and Dijkstra's algorithm, ensuring that the path minimizes the travel distance while avoiding obstacles.

[0041] In the global obstacle avoidance path, the electric shovel may traverse multiple obstacle areas. To ensure the accuracy and safety of path planning, the tangential direction of each obstacle is analyzed. The tangential direction refers to the direction in which the electric shovel contacts the obstacle surface as it passes by. Segmenting the path along the tangential direction ensures that the electric shovel does not get too close to the obstacle during obstacle avoidance, thus preventing collisions. Based on the tangential direction of the obstacles, the global obstacle avoidance path is divided into multiple obstacle avoidance path segments. Each path segment corresponds to a specific digging or moving action performed by the electric shovel within a certain period. The purpose of segmentation is to dynamically adjust the electric shovel's path and actions according to different obstacle conditions, achieving more flexible and precise operation.

[0042] Within each segmented obstacle avoidance path, dynamic fitting of motion parameters is performed based on the current obstacle and shovel status. This process includes real-time adjustments to parameters such as the shovel's working state, digging action, slewing angle, and feed rate. The dynamic fitting process considers the different characteristics of each path segment; for example, when the shovel traverses a narrow area, its motion parameters are adjusted to avoid collisions with obstacles. The fitted motion for each path segment is broken down into atomic-level digging actions. These atomic-level actions include the shovel's slewing angle, digging depth, feed rate, and tilt angle—the specific instructions for the shovel to execute each small step.

[0043] The generated atomic-level digging actions are screened, spliced, and optimized. Specifically, by combining and enumerating different atomic-level digging actions, multiple action sequences are generated. Closed-loop compliant splicing simulation is performed on multiple action sequences, and multiple evaluation indicators, such as accuracy, safety, and time, are calculated. Through weighted evaluation and descending order, the optimal action sequence is selected to generate the final automatic operation instruction sequence. This ensures that the electric shovel can perform efficient, accurate, and safe digging tasks in complex environments, maximizing operational efficiency and reducing potential risks.

[0044] Furthermore, the method involves performing closed-loop compliant filtering and splicing on the multiple sets of atomic-level mining actions to output the automatic operation instruction sequence, the method comprising:

[0045] The multiple sets of atomic-level mining actions are combined and enumerated to obtain multiple atomic-level mining action sequences; closed-loop compliant splicing simulation is performed on the multiple atomic-level mining action sequences to quantify the evaluation indicators of the multiple action sequences; the weighted evaluation results of the multiple action sequence evaluation indicators are sorted in descending order to locate and extract the automatic operation instruction sequence from the multiple atomic-level mining action sequences.

[0046] Atomic-level digging actions are the most basic operational units when an electric shovel performs a task. These include digging depth, slewing angle, and bulldozing direction. From multiple atomic-level actions, different sequences of atomic-level digging actions are combined and enumerated. Each sequence represents a different action combination to address different work scenarios and requirements. These combined digging action sequences represent multiple different plans executed by the electric shovel during operation. The purpose of combination enumeration is to provide the electric shovel with multiple options through different action combinations, enabling the system to select the most suitable action sequence for subsequent evaluation and optimization.

[0047] In multiple atomic-level digging action sequences, closed-loop compliant splicing simulations are performed. This process simulates the actual actions of the electric shovel within the combined action sequences, ensuring smooth transitions between each action and conforming to the shovel's physical properties, such as response speed and rotation angle. "Closed-loop" refers to real-time monitoring and adjustment of the shovel's actions, ensuring continuous execution of each digging action without stuttering or unnatural movements. "Compliance" refers to the smoothness of action transitions, ensuring no abrupt jumps between actions and preventing operational instability or malfunctions due to uncoordinated movements. During the splicing simulation, multiple evaluation metrics are calculated. These metrics measure the efficiency, smoothness, safety, and accuracy of the action sequences. For example, evaluation metrics include: action execution accuracy, measuring the error range of each action; obstacle avoidance safety margin, measuring whether the path can avoid obstacles and the safety margin for obstacle avoidance operations; and command execution time, measuring the time required to complete each action. Quantifying these evaluation metrics helps determine the quality of each action sequence and provides a basis for subsequent action sequence selection.

[0048] Different weights are assigned to each evaluation metric based on different task requirements. For example, in some scenarios, the accuracy of action execution is more important than the time taken; therefore, different weights are assigned to the evaluation metrics according to actual needs. The evaluation results of all atomic-level digging action sequences are weighted and sorted in descending order, that is, the optimal action sequence is placed first. The evaluation results determine which sequences best meet the target requirements, such as highest efficiency, lowest error, and safest obstacle avoidance. Finally, the best-performing sequence is selected from multiple atomic-level digging action sequences and used as the final automated operation instruction sequence. This instruction sequence is then passed to the electric shovel for actual digging operations.

[0049] Furthermore, the method also includes:

[0050] By performing a self-check on the target electric shovel, the shovel mechanism status data is obtained; the maintenance cycle is updated based on the shovel mechanism status data to obtain a dynamic maintenance time threshold; during the process of controlling the target electric shovel to perform rotary unloading using the automatic operation instruction sequence, maintenance is forcibly interrupted based on the dynamic maintenance time threshold.

[0051] Before executing a task, a self-inspection is performed on all components of the target electric shovel, such as the slewing mechanism, bucket, and transmission system, to ensure that each component is within its normal operating range. Based on the self-inspection results, electric shovel mechanism status data is generated, including the health status, operational status, and potential failure risks of each component. This self-inspection of the electric shovel mechanism is fundamental to ensuring the normal operation of the equipment, enabling the timely detection of potential problems and preventing operational interruptions due to equipment failure.

[0052] Based on the status data of the electric shovel mechanism, the maintenance cycle is recalculated. The maintenance cycle refers to the period of stable operation of the equipment before a comprehensive maintenance is required. The maintenance cycle is updated based on the following factors: component wear level (if a component has significant wear or damage, the maintenance cycle will be shortened); failure risk (if potential failures are found in certain components during self-inspection, a shorter maintenance cycle is recommended); and usage frequency (if the equipment operates under high load for extended periods, the maintenance cycle will be shortened accordingly). Based on the updated maintenance cycle, a dynamic maintenance time threshold is calculated. This threshold represents the maximum time the electric shovel can continue to operate stably from the current moment. Exceeding this threshold will force maintenance. Setting a dynamic maintenance time threshold helps prevent damage to the electric shovel due to overload operation, extends equipment lifespan, and reduces unexpected failures.

[0053] When the electric shovel executes its automatic operation sequence, especially during critical operations such as rotary unloading, its status is monitored in real time. If the shovel's operating time exceeds the dynamic maintenance time threshold, a forced maintenance interruption is automatically initiated, pausing the current operation and stopping the shovel's operation for necessary inspection and repair. This mechanism prevents the equipment from continuing to operate under overload conditions, reducing the risk of equipment damage. By forcibly interrupting operation, it ensures that the equipment undergoes sufficient maintenance before each operation, guaranteeing safety during operation and long-term stable operation of the equipment.

[0054] Furthermore, the method involves updating the maintenance cycle based on the electric shovel mechanism status data to obtain a dynamic maintenance time threshold, and includes:

[0055] Based on the status association attributes of electric shovel components, the status data of the electric shovel mechanism is decomposed to obtain multiple health status vectors of multiple electric shovel components; according to the multiple health status vectors, multiple predicted maintenance time points are located at multiple maintenance threshold baselines of the multiple electric shovel components; the multiple predicted maintenance time points are normalized and sorted in descending order, and the earliest time node is extracted as the dynamic maintenance time threshold.

[0056] Each component of an electric shovel has multiple attributes related to its health status. These attributes can be used to describe the component's working condition, potential failure risks, and maintenance needs. Types of electric shovel components include pushing mechanisms, rotary motors, etc. The working status of each component can be monitored by multiple sensors.

[0057] The corresponding status data are extracted from each component of the electric shovel to form a health status vector. Each component's health status vector consists of different monitoring indicators, as follows: Pushing mechanism vector = [Temperature, Vibration Amplitude, Peak Pressure], where temperature refers to the operating temperature of the pushing mechanism; vibration amplitude refers to the vibration amplitude of the pushing mechanism, with a larger amplitude indicating wear or malfunction; peak pressure refers to the maximum pressure the pushing mechanism can withstand. Slewing motor vector = [Temperature, Current Harmonics, Vibration Frequency], where temperature refers to the operating temperature of the slewing motor, with excessively high temperatures indicating motor overload or heat dissipation problems; current harmonics refer to the harmonic components in the current, with abnormal current harmonics indicating motor malfunction; vibration frequency refers to the vibration frequency of the slewing motor, with abnormal vibration frequencies indicating damage to the motor or its related components.

[0058] Each component's health status vector has an associated maintenance threshold baseline. These baselines, derived from manufacturer specifications or historical data analysis, represent the component's normal operating range under different working conditions. If a health status vector exceeds this maintenance threshold baseline, it indicates that the component has failed or is about to fail, requiring maintenance. Based on each component's health status vector, a predicted maintenance time point is calculated. This time point represents the moment when the component is likely to fail or experience performance degradation. By comparing the health status vector with the maintenance threshold baseline, the timing of maintenance required for the component is predicted. For example, by analyzing the trend of health status vector changes over time, it is possible to predict when a component will reach a failure threshold. Accurately predicting maintenance time points can provide early warning of equipment failures, reduce downtime and repair costs caused by sudden failures, and ensure that equipment is maintained at the appropriate time.

[0059] Because different components have different health status vectors and maintenance threshold baselines, the range of predicted maintenance time points for each component may also differ. To facilitate comparison, multiple predicted maintenance time points are normalized. Normalization means converting the maintenance time points of different components into relative time values, ensuring they are compared under the same standard. After normalization, the predicted maintenance time points of all components are sorted in descending order, prioritizing the components most urgently requiring maintenance. This allows for the priority handling of components closest to failure. From the descending-ordered predicted maintenance time points, the earliest time node is extracted as the dynamic maintenance time threshold. This time node represents the earliest moment when the electric shovel needs maintenance. After this moment, operation is forcibly interrupted, and maintenance is performed to ensure timely maintenance of the equipment during operation and avoid production interruptions due to equipment failure.

[0060] Furthermore, by fusing dual GPS positioning information to perform sensing verification of the laser point cloud data and locating the relative pose of the mining truck, the method includes:

[0061] The global coordinates of the target electric shovel are obtained through the electric shovel's GPS; the global coordinates of the mining truck to be unloaded are obtained through the mining truck's GPS; the initial relative pose of the mining truck to be unloaded relative to the target electric shovel is calculated based on the global coordinates of the electric shovel and the global coordinates of the mining truck; after converting the laser point cloud data from the radar coordinate system to the electric shovel's body coordinate system, multi-source sensor dynamic registration is performed based on the initial relative pose, and the relative pose of the mining truck is output.

[0062] The electric shovel is equipped with a GPS device to monitor its location in real time. The GPS device receives satellite signals and calculates the shovel's current longitude, latitude, and altitude as its global coordinates.

[0063] Mining trucks are also equipped with GPS devices to monitor their location. Similar to electric shovels, the GPS devices on mining trucks obtain their longitude, latitude, and altitude through satellite signals, providing global coordinates for the mining trucks.

[0064] Based on the global coordinates of the electric shovel and the mining truck, the initial relative pose of the mining truck relative to the electric shovel is calculated. The relative pose includes two aspects: relative position, which is the displacement vector of the mining truck relative to the electric shovel, represented by the coordinate difference between the mining truck and the electric shovel, such as the difference in X, Y, and Z coordinates; and relative orientation, which represents the angular difference between the mining truck and the electric shovel, indicating their relative direction. By performing differential calculations on the global coordinates of the electric shovel and the mining truck, the relative position and orientation of the mining truck relative to the electric shovel are obtained. For example, assuming the position of the electric shovel is A and the position of the mining truck is B, then the relative pose is the vector BA, representing the direction and distance from the electric shovel to the mining truck. After calculating the initial relative pose of the mining truck, it is possible to determine the relative position of the mining truck in front of, to the side of, or behind the electric shovel, and to perform path planning and operation control accordingly.

[0065] The laser point cloud data is generated based on the radar coordinate system. In this step, the laser point cloud data is transformed from the radar coordinate system to the electric shovel body coordinate system to ensure that the point cloud data is consistent with the position and orientation of the electric shovel. The transformation process involves coordinate transformation techniques, such as rotation matrices or homogeneous transformation matrices, to map the data from the radar coordinate system to the electric shovel body coordinate system. Based on the transformed laser point cloud data, multi-source sensor dynamic registration technology is used, combined with the initial relative pose of the electric shovel, to further optimize the calculation of the relative pose of the mining truck. The registration process fuses data from different sensors (including LiDAR and GPS) to improve positioning accuracy. Through dynamic registration, the estimation of the relative pose of the mining truck can be continuously adjusted according to the real-time changes in sensor data. After dynamic registration, the final output is the relative pose of the mining truck, that is, the precise position and orientation of the mining truck relative to the electric shovel. This information is used for subsequent path planning and operation command generation to ensure that the electric shovel can accurately dock with the mining truck and successfully complete the unloading operation.

[0066] Furthermore, the action sequence evaluation metrics include the action execution accuracy range, the obstacle avoidance safety margin range, and the instruction execution time.

[0067] The motion execution accuracy range refers to the error range between the actual execution result and the target instruction when the electric shovel performs digging actions. This range can be measured by detecting the difference between the actual movement path of the electric shovel and the expected path. Accuracy ensures that the electric shovel can accurately execute tasks according to the predetermined path, avoiding resource waste or equipment damage caused by misoperation. The obstacle avoidance safety margin range represents the minimum safe distance between the electric shovel and obstacles during path planning. This range is used to measure the safety margin between the electric shovel and surrounding obstacles during path planning and actual digging. A larger safety margin helps avoid the risk of collision due to environmental changes, ensuring operational safety. Instruction execution time refers to the time required for the electric shovel to perform a specific operation. This indicator is used to measure task efficiency and ensure that the operation can be completed on time. For large-scale operations, reducing the execution time of each action can improve overall operational efficiency and productivity.

[0068] Example 2 is based on the same inventive concept as the remote semi-automatic excavation and loading control method for electric shovels based on 5G private networks in the previous examples, such as... Figure 2 As shown in the figure, this application embodiment provides a remote semi-automatic excavation and loading control system for electric shovels based on a 5G private network. The system includes:

[0069] The panoramic environment scanning module 10 drives the lidar and camera array pre-installed on the target electric shovel to perform panoramic environment scanning, obtaining lidar point cloud data and raw video stream; the sensor verification module 20 fuses dual GPS positioning information to perform sensor verification of the lidar point cloud data and locate the relative pose of the mining truck; the dynamic optical flow stitching module 30 performs dynamic optical flow stitching on the raw video stream to obtain a panoramic view; the obstacle avoidance coordinate space modeling module 40 identifies obstacle avoidance elements based on the panoramic view and combines the lidar point cloud data to perform obstacle avoidance coordinate space modeling, obtaining a spatial obstacle avoidance coordinate array; the excavation action dynamic fitting module 50 receives the unloading position uploaded by the manual remote platform, uses the relative pose of the mining truck and the unloading position as the path planning target, performs excavation action dynamic fitting based on the spatial obstacle avoidance coordinate array, and generates an automatic operation instruction sequence; the high-authority excavation and loading intervention module 60, during the process of controlling the target electric shovel to perform semi-automatic slewing unloading using the automatic operation instruction sequence, feeds back the panoramic control video stream to the manual remote platform through the 5G private network to perform high-authority excavation and loading intervention.

[0070] Furthermore, the obstacle avoidance coordinate space modeling module 40 is used to perform the following operation steps:

[0071] The panoramic view is input in parallel into the dynamic and static detection channels of the obstacle detection model to perform dual-thread obstacle avoidance element recognition, resulting in dynamic obstacle bounding boxes and static obstacle masks. Based on the panoramic view, the dynamic obstacle bounding boxes and static obstacle masks are spatiotemporally aligned to obtain fused obstacle avoidance element labels. The laser point cloud data is mapped to the electric shovel coordinate system using LiDAR SLAM, and obstacle clustering is performed based on the point cloud density to obtain three-dimensional obstacle clusters. The fused obstacle avoidance element labels are then back-projected onto the three-dimensional obstacle clusters, and obstacle attribute weighting association is performed to generate the spatial obstacle avoidance coordinate array.

[0072] Furthermore, the obstacle avoidance coordinate space modeling module 40 is used to perform the following operation steps:

[0073] The panoramic view is divided into dynamic frame sequences and static keyframes. The dynamic frame sequences are input into the dynamic detection channel built on the YOLOv5 object detection network, and dynamic obstacles are detected in real time based on motion feature temporal analysis, and the dynamic obstacle bounding boxes are output. The static keyframes are input into the static detection channel built on the U-Net semantic segmentation network, and static obstacles are segmented at the pixel level based on texture geometric structure features, and the static obstacle mask is output.

[0074] Furthermore, the dynamic fitting module 50 for the mining action is used to perform the following operation steps:

[0075] Starting from the relative pose of the mining truck and ending at the unloading position, a three-dimensional path search space is constructed in the spatial obstacle avoidance coordinate array. Based on the safety distance associated with obstacle type, the shortest collision-free path is fitted in the three-dimensional path search space to generate a global obstacle avoidance path. The global obstacle avoidance path is segmented according to the tangent direction of the obstacles along the way to obtain multiple obstacle avoidance path segments. Dynamic fitting of action parameters is performed on the multiple obstacle avoidance path segments to obtain multiple sets of atomic-level mining actions. The multiple sets of atomic-level mining actions are closed-loop compliant screening and splicing to output the automatic operation instruction sequence.

[0076] Furthermore, the dynamic fitting module 50 for the mining action is used to perform the following operation steps:

[0077] The multiple sets of atomic-level mining actions are combined and enumerated to obtain multiple atomic-level mining action sequences; closed-loop compliant splicing simulation is performed on the multiple atomic-level mining action sequences to quantify the evaluation indicators of the multiple action sequences; the weighted evaluation results of the multiple action sequence evaluation indicators are sorted in descending order to locate and extract the automatic operation instruction sequence from the multiple atomic-level mining action sequences.

[0078] Furthermore, the high-privilege mining intervention module 60 is used to perform the following operation steps:

[0079] By performing a self-check on the target electric shovel, the shovel mechanism status data is obtained; the maintenance cycle is updated based on the shovel mechanism status data to obtain a dynamic maintenance time threshold; during the process of controlling the target electric shovel to perform rotary unloading using the automatic operation instruction sequence, maintenance is forcibly interrupted based on the dynamic maintenance time threshold.

[0080] Furthermore, the high-privilege mining intervention module 60 is used to perform the following operation steps:

[0081] Based on the status association attributes of electric shovel components, the status data of the electric shovel mechanism is decomposed to obtain multiple health status vectors of multiple electric shovel components; according to the multiple health status vectors, multiple predicted maintenance time points are located at multiple maintenance threshold baselines of the multiple electric shovel components; the multiple predicted maintenance time points are normalized and sorted in descending order, and the earliest time node is extracted as the dynamic maintenance time threshold.

[0082] Furthermore, the sensor verification module 20 is used to perform the following operation steps:

[0083] The global coordinates of the target electric shovel are obtained through the electric shovel's GPS; the global coordinates of the mining truck to be unloaded are obtained through the mining truck's GPS; the initial relative pose of the mining truck to be unloaded relative to the target electric shovel is calculated based on the global coordinates of the electric shovel and the global coordinates of the mining truck; after converting the laser point cloud data from the radar coordinate system to the electric shovel's body coordinate system, multi-source sensor dynamic registration is performed based on the initial relative pose, and the relative pose of the mining truck is output.

[0084] Furthermore, the action sequence evaluation metrics include the action execution accuracy range, the obstacle avoidance safety margin range, and the instruction execution time.

[0085] Through the foregoing detailed description of the remote semi-automatic excavation and loading control method for electric shovels based on 5G private networks, those skilled in the art can clearly understand the remote semi-automatic excavation and loading control system for electric shovels based on 5G private networks in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and relevant parts can be referred to in the method section.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network, characterized in that, The method includes: The system drives the pre-installed lidar and camera array on the target electric shovel to perform a panoramic environmental scan, obtaining laser point cloud data and raw video streams. The relative pose of the mining truck is determined by integrating dual GPS positioning information to perform sensor verification of the laser point cloud data. Dynamic optical flow stitching is performed on the original video stream to obtain a panoramic view; After identifying obstacle avoidance elements based on the panoramic view, obstacle avoidance coordinate space modeling is performed in combination with the laser point cloud data to obtain a spatial obstacle avoidance coordinate array. After receiving the unloading position uploaded by the remote manual platform, the system uses the relative pose of the mining truck and the unloading position as the path planning target, performs dynamic fitting of the excavation action based on the spatial obstacle avoidance coordinate array, and generates an automatic operation instruction sequence. During the process of using the automatic operation instruction sequence to control the target electric shovel to perform semi-automatic slewing and unloading, the panoramic control video stream is fed back to the manual remote platform through the 5G private network for high-authority excavation and loading intervention.

2. The remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in claim 1, characterized in that, After identifying obstacle avoidance elements based on the panoramic view, obstacle avoidance coordinate space modeling is performed in conjunction with the laser point cloud data to obtain a spatial obstacle avoidance coordinate array. The method includes: The panoramic view is input in parallel into the dynamic and static detection channels of the obstacle detection model to perform dual-thread obstacle avoidance element recognition, thereby obtaining the dynamic obstacle bounding box and the static obstacle mask. Based on the panoramic view alignment, the dynamic obstacle bounding box and static obstacle mask are spatiotemporally aligned to obtain the fused obstacle avoidance element labels; The laser point cloud data is mapped to the electric shovel coordinate system using LiDAR SLAM, and obstacle clustering is performed based on the point cloud density to obtain three-dimensional obstacle clusters. The fused obstacle avoidance element labels are back-projected onto the obstacle 3D cluster, and obstacle attribute weighting association is performed to generate the spatial obstacle avoidance coordinate array.

3. The remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in claim 2, characterized in that, The panoramic view is input in parallel into the dynamic and static detection channels of the obstacle detection model, and dual-thread obstacle avoidance element recognition is performed to obtain dynamic obstacle bounding boxes and static obstacle masks. The method includes: The panoramic view is divided into dynamic frame sequences and static keyframes; The dynamic frame sequence is input into the dynamic detection channel constructed based on the YOLOv5 target detection network, and dynamic obstacles are detected in real time based on motion feature temporal analysis, and the dynamic obstacle bounding boxes are output. The static keyframes are input into the static detection channel constructed based on the U-Net semantic segmentation network, and the static obstacles are segmented at the pixel level based on the texture geometric structure features to output the static obstacle mask.

4. The remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in claim 1, characterized in that, After receiving the unloading position uploaded by the remote control platform, the method uses the relative pose of the mining truck and the unloading position as the path planning target, dynamically fits the excavation action according to the spatial obstacle avoidance coordinate array, and generates an automatic operation instruction sequence. The method includes: Starting from the relative pose of the mining truck and ending at the unloading position, a three-dimensional path search space is constructed in the spatial obstacle avoidance coordinate array; Based on the safety distance associated with obstacle type, the shortest collision-free path fitting is performed in the three-dimensional path search space to generate a global obstacle avoidance path; The global obstacle avoidance path is divided into multiple obstacle avoidance path segments based on the tangential direction of obstacles along the way; Dynamic fitting of motion parameters is performed on the multiple obstacle avoidance path segments to obtain multiple sets of atomic-level mining actions; The multiple sets of atomic-level mining actions are subjected to closed-loop compliant screening and splicing to output the automatic operation instruction sequence.

5. The remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in claim 4, characterized in that, The method involves performing closed-loop compliant filtering and splicing on the multiple sets of atomic-level mining actions to output the automatic operation instruction sequence. By combining and enumerating the multiple sets of atomic-level mining actions, multiple atomic-level mining action sequences are obtained; Closed-loop compliant splicing simulations were performed on the multiple atomic-level mining action sequences to quantify the evaluation metrics of the multiple action sequences; The weighted evaluation results of the multiple action sequence evaluation indicators are arranged in descending order to extract the automatic operation instruction sequence from the multiple atomic-level mining action sequences.

6. The remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in claim 1, characterized in that, The method further includes: By performing a self-inspection of the target electric shovel, the status data of the electric shovel mechanism can be obtained. The operation and maintenance cycle is updated based on the electric shovel mechanism status data to obtain a dynamic operation and maintenance time threshold. During the process of controlling the target electric shovel to perform rotary unloading using the automatic operation instruction sequence, maintenance is forcibly interrupted based on the dynamic operation and maintenance time threshold.

7. The remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in claim 6, characterized in that, The method involves updating the maintenance cycle based on the electric shovel mechanism status data to obtain a dynamic maintenance time threshold, and includes: Based on the state association attributes of electric shovel components, the state data of the electric shovel mechanism is decomposed to obtain multiple health state vectors of multiple electric shovel components; Based on the multiple health status vectors, multiple predicted maintenance time points are located at multiple maintenance threshold baselines of the multiple electric shovel components; The multiple predicted maintenance time points are normalized and sorted in descending order, and the earliest time point is extracted as the dynamic operation and maintenance time threshold.

8. The remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in claim 1, characterized in that, The method involves fusing dual GPS positioning information to perform sensor verification of the laser point cloud data and locating the relative pose of the mining truck. The global coordinates of the target electric shovel are obtained using the electric shovel's GPS. Obtain the global coordinates of the mining truck to be unloaded using the mining truck's GPS; The initial relative pose of the mining truck to be unloaded relative to the target electric shovel is calculated based on the global coordinates of the electric shovel and the global coordinates of the mining truck. After converting the laser point cloud data from the radar coordinate system to the electric shovel body coordinate system, multi-source sensor dynamic registration is performed based on the initial relative pose to output the relative pose of the mining truck.

9. The remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in claim 5, characterized in that, The evaluation metrics for the action sequence include the action execution accuracy range, the obstacle avoidance safety margin range, and the instruction execution time.

10. A remote semi-automatic excavation and loading control system for electric shovels based on a 5G private network, characterized in that: The system is used to implement the remote semi-automatic excavation and loading control method for electric shovels based on a 5G private network as described in any one of claims 1-9, the system comprising: The panoramic environment scanning module is used to drive the lidar and camera array pre-installed on the target electric shovel to perform panoramic environment scanning, and obtain lidar point cloud data and raw video stream; The sensor verification module is used to fuse dual GPS positioning information to perform sensor verification of the laser point cloud data and locate the relative pose of the mining truck. The dynamic optical flow stitching module is used to perform dynamic optical flow stitching on the original video stream to obtain a panoramic view. The obstacle avoidance coordinate space modeling module is used to identify obstacle avoidance elements based on the panoramic view, and then combine the laser point cloud data to perform obstacle avoidance coordinate space modeling to obtain a spatial obstacle avoidance coordinate array. The excavation action dynamic fitting module is used to receive the unloading position uploaded by the manual remote platform, and then use the relative pose of the mining truck and the unloading position as the path planning target. Based on the spatial obstacle avoidance coordinate array, it performs dynamic fitting of the excavation action to generate an automatic operation instruction sequence. The high-authority excavation and loading intervention module is used to perform high-authority excavation and loading intervention when the target electric shovel is controlled to perform semi-automatic rotary unloading using the automatic operation instruction sequence. The panoramic control video stream is fed back to the manual remote platform via a 5G private network.

Images