Unmanned excavator

Abstract

The application discloses an unmanned excavator, which comprises a vehicle-mounted hardware subsystem, a remote cockpit subsystem and an integrated software subsystem. The vehicle-mounted hardware subsystem is arranged on the body of the unmanned excavator and is used for collecting environment sensing data and body attitude data of a working environment in which the unmanned excavator is located in real time, driving an actuator of the unmanned excavator to complete a working action according to a received control instruction, and the like. The remote cockpit subsystem is used for providing a remote monitoring and instruction input interface, sending a manually controlled instruction inputted from outside to the vehicle-mounted hardware subsystem, receiving and displaying environment and state information returned by the vehicle-mounted hardware subsystem, and the like. The integrated software subsystem is used for processing the environment sensing data and the body attitude data, generating a working control instruction containing a working decision and a motion track, sending the working control instruction to the vehicle-mounted hardware subsystem, providing a man-machine interaction interface, and the like. The integrated software subsystem is configured with three working modes, including a remote control mode, a semi-automatic loading and unloading mode and a full-automatic excavating and loading mode.

Description

Technical Field

[0001] This application relates to the field of automation and intelligent control technology for construction machinery, and in particular to an unmanned excavator. Background Technology

[0002] In critical operational scenarios such as earthwork engineering, mining, and disaster relief, excavators serve as core equipment, undertaking heavy earthwork excavation and loading tasks. However, traditional manual operation faces multiple challenges: operators must perform repetitive actions for extended periods, easily leading to fatigue and a significant decrease in work efficiency, making it difficult to maintain high-intensity, high-precision continuous operations; simultaneously, in hazardous conditions such as tunnel excavation, slope construction, landslide response, or toxic gas environments, operators are directly exposed to high risks, seriously threatening their personal safety; furthermore, due to varying skill levels among operators, work quality fluctuates greatly, making it impossible to achieve standardization and consistency in the construction process, affecting the overall progress and quality of the project.

[0003] Although some excavator products with remote control or basic automation functions have been launched on the market, these systems still have obvious limitations: most only support simple remote control command transmission or mechanical reproduction of fixed trajectories, lacking real-time perception and intelligent decision-making capabilities for complex and ever-changing on-site environments; the level of system intelligence is insufficient, failing to effectively integrate the three-dimensional recognition function of work objects such as dump truck beds and material accumulation states, and unable to dynamically update the work environment model and generate optimal path planning; more importantly, existing systems are rigid in terms of switching operation modes, unable to achieve a smooth transition between remote manual intervention, semi-automatic assisted operation and fully automatic operation, resulting in poor adaptability in sudden working conditions and difficulty in meeting the dual requirements of flexibility and reliability in actual engineering.

[0004] The aforementioned defects severely restrict the excavator's potential for efficient, safe, and precise operation, necessitating a new system architecture to overcome existing technological bottlenecks. Summary of the Invention

[0005] The purpose of this application is to provide an unmanned excavator to solve the technical problems in related technologies, such as the excavator operating system having single function, low level of intelligence, and inability to flexibly switch operating modes according to complex working conditions, resulting in high labor costs, low operating efficiency, and safety hazards. It has the advantages of improving operating efficiency, ensuring operator safety, and supporting flexible switching of operating modes.

[0006] In a first aspect, embodiments of this application provide an unmanned excavator, including: The vehicle-mounted hardware subsystem is deployed on the unmanned excavator body and is used to collect environmental perception data and body posture data of the working environment in real time, and drive the actuator of the unmanned excavator to complete the working action according to the received control commands. The remote cockpit subsystem is connected to the vehicle hardware subsystem via a wireless communication network. It provides a remote monitoring and command input interface, sends externally input manual control commands to the vehicle hardware subsystem, and receives and displays environmental and status information returned by the vehicle hardware subsystem. An integrated software subsystem is deployed in the vehicle computing unit of the vehicle hardware subsystem and the cockpit computing unit of the remote cockpit subsystem. It is used to process the environmental perception data and the body posture data, generate operation control commands that include operation decisions and motion trajectories, send the operation control commands to the vehicle hardware subsystem, and provide a human-machine interaction interface. The integrated software subsystem is configured with three operating modes: remote control mode, semi-automatic loading and unloading mode, and fully automatic excavation and loading mode.

[0007] This application provides an unmanned excavator, including an onboard hardware subsystem deployed on the excavator body. This subsystem collects real-time environmental perception data and body posture data of the excavator's operating environment and drives the excavator's actuators to complete operational actions based on received control commands. A remote cockpit subsystem, connected to the onboard hardware subsystem via a wireless communication network, provides a remote monitoring and command input interface, sends externally input manual control commands to the onboard hardware subsystem, and receives and displays environmental and status information transmitted back from the onboard hardware subsystem. An integrated software subsystem, deployed in the onboard computing unit of the onboard hardware subsystem and the cockpit-side computing unit of the remote cockpit subsystem, processes the environmental perception data and body posture data, generates operation control commands containing operation decisions and motion trajectories, sends these commands to the onboard hardware subsystem, and provides a human-machine interface. The integrated software subsystem is configured with three operating modes: remote control mode, semi-automatic loading and unloading mode, and fully automatic excavation and loading mode. Through remote monitoring and automatic control mechanisms, operator fatigue and exposure to risky environments are reduced, while supporting flexible switching between three operating modes, thereby improving operational efficiency and ensuring operator safety. Attached Figure Description

[0008] Figure 1 This is a block diagram of an unmanned excavator provided in an embodiment of this application; Figure 2 This is a schematic diagram of the operation process of an unmanned excavator provided in an embodiment of this application; Figure 3This is a schematic diagram of the structural architecture of an unmanned excavator provided in an embodiment of this application; Figure 4 This is a schematic diagram of the hardware deployment of the vehicle-mounted hardware subsystem provided in this application embodiment on an unmanned excavator; Figure 5 This is a schematic diagram of a human-machine interface for the cockpit control software provided in an embodiment of this application; The system includes: 1 tilt sensor, 2 auxiliary lidar, 3 wireless communication module, 4 blind spot lidar, 5 rotation angle measurement device, 6 camera, 7 main lidar, 8 vehicle-mounted computing unit, 9 RTK antenna, and 10 wired control device. Detailed Implementation

[0009] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0010] It should be understood that the steps described in the method embodiments disclosed in this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0011] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0012] In related technologies, traditional manually operated excavators have inherent drawbacks such as operator fatigue leading to limited work efficiency, high personal safety risks when operating in dangerous and complex environments, and difficulty in standardizing work accuracy due to differences in operator skills. Existing remote-controlled or single-automatic excavator systems generally lack real-time three-dimensional perception capabilities of the work object and environment, cannot make autonomous decisions based on complex site conditions, and have rigid work mode switching mechanisms, making it difficult to achieve seamless transitions between remote manual intervention, semi-automatic assistance, and fully automatic operation. This affects the overall adaptability and reliability of the system, specifically manifested in work decisions relying on manual input, delayed environmental model updates, and the risk of operational interruption during mode switching.

[0013] To address the technical problems existing in related technologies, this application provides an unmanned excavator. Please refer to [link / reference]. Figure 1 , Figure 1 This is a block diagram of an unmanned excavator provided in an embodiment of this application.

[0014] like Figure 1 As shown, the unmanned excavator 100 includes: an onboard hardware subsystem 101, deployed on the unmanned excavator body, used to collect environmental perception data and body posture data of the working environment in real time, and drive the actuators of the unmanned excavator to complete the operation according to the received control commands; a remote cockpit subsystem 102, connected to the onboard hardware subsystem through a wireless communication network, used to provide remote monitoring and command input interface, send externally input manual control commands to the onboard hardware subsystem, and receive and display the environmental and status information returned by the onboard hardware subsystem; an integrated software subsystem 103, deployed in the onboard computing unit of the onboard hardware subsystem and the cockpit computing unit of the remote cockpit subsystem, used to process the environmental perception data and body posture data, generate operation control commands including operation decisions and motion trajectories, send the operation control commands to the onboard hardware subsystem, and provide a human-machine interface; wherein, the integrated software subsystem is configured with three operation modes, including remote control mode, semi-automatic loading and unloading mode, and fully automatic excavation and loading mode.

[0015] In one embodiment, reference is made to Figure 1 The provided unmanned excavator 100 consists of an on-board hardware subsystem 101, a remote cockpit subsystem 102, and an integrated software subsystem 103. The unmanned excavator 100 supports a variety of different operating modes, including remote control mode, semi-automatic loading and unloading mode, and fully automatic excavation and loading mode.

[0016] For example, the vehicle-mounted hardware subsystem 101 is deployed on the unmanned excavator body to collect environmental perception data and body attitude data of the working environment in which the unmanned excavator 100 is located in real time. For example, environmental perception data can be obtained through multiple sensors installed on the excavator, such as cameras and lidar, or body attitude data can be obtained through inertial measurement units, encoders, etc. In addition, the vehicle-mounted hardware subsystem 101 also drives the actuators of the unmanned excavator to complete the working actions according to the received control commands, wherein the received control commands come from the remote cockpit subsystem and the integrated software subsystem. This controls the movement of the actuators of the unmanned excavator 100, such as the boom, stick, bucket, slewing mechanism, and traveling mechanism.

[0017] The remote cockpit subsystem 102 is connected to the vehicle-mounted hardware subsystem 101 via a wireless communication network, which can be a cellular network, satellite communication network, or local area wireless network. The remote cockpit subsystem 102 provides a remote monitoring and command input interface. For example, the operator can input manual control commands using input devices such as joysticks or pedals. Externally input manual control commands can also be sent to the vehicle-mounted hardware subsystem, while simultaneously receiving and displaying environmental and status information transmitted back from the vehicle-mounted hardware subsystem. For example, the transmitted information may include video streams from the excavator's surroundings, sensor data, and equipment operating status.

[0018] The integrated software subsystem 103 is deployed in the onboard computing unit of the vehicle-mounted hardware subsystem and the cockpit-side computing unit of the remote cockpit subsystem, and is used to process environmental perception data and body attitude data. For example, it can perform operations such as filtering, fusion, and target recognition on sensor data. Based on the processing results, the integrated software subsystem 103 generates operation control commands that include operation decisions and motion trajectories. For example, it can plan the excavation path and unloading point according to the operation task and environmental information. Subsequently, the operation control commands are sent to the onboard hardware subsystem 101 to drive the actuators of the unmanned excavator 100. In addition, the integrated software subsystem 103 provides a human-machine interface, enabling the operator to intuitively monitor the excavator status and perform operations.

[0019] For the three operating modes of the unmanned excavator, the switching between operating modes is realized based on the integrated software subsystem 103. These operating modes include remote control mode, semi-automatic loading and unloading mode, and fully automatic digging and loading mode. In remote control mode, the operator directly controls every movement of the excavator through the remote cockpit subsystem, similar to traditional manual operation, but the operator does not need to be in a dangerous environment. In semi-automatic loading and unloading mode, some operational steps are completed automatically by the system; for example, the system can automatically plan the unloading path and execute the unloading action, while the digging action is still manually controlled by the operator. In fully automatic digging and loading mode, the system can autonomously complete the entire operation process from digging to loading, including environmental perception, operation decision-making, path planning, and motion control; the operator only needs to set tasks and monitor the process.

[0020] Furthermore, the onboard hardware subsystem 101 on the unmanned excavator 100 is used to collect environmental perception data and body posture data in real time, and drive the actuators to complete the operation. However, in actual working environments, it is necessary to ensure the comprehensiveness of environmental perception data and the accuracy of body posture data. Therefore, the onboard hardware subsystem also includes: The environmental perception module consists of multiple cameras and / or lidar arranged in the cab and boom of the unmanned excavator, used to collect images of the surrounding environment and three-dimensional point cloud data of the unmanned excavator in the working environment. The attitude feedback module includes a tilt sensor, a slewing angle measuring device, and an RTK high-precision positioning device, which are used to detect the attitude angle, slewing angle, and absolute ground coordinates of the unmanned excavator's actuators in real time. The actuators include the boom, stick, bucket, slewing mechanism, and traveling mechanism. The on-board computing unit is used to generate motion control commands for the actuators based on the data collected by the environmental perception module and the attitude feedback module. The drive-by-wire module is used to receive motion control commands and respond to them to drive the actuator to perform corresponding actions. The wireless communication module is used for data interaction with the remote cockpit subsystem.

[0021] In one embodiment, the environmental perception module comprises multiple cameras and / or lidar units deployed on the cab and boom of the unmanned excavator to acquire images and 3D point cloud data of the surrounding environment in the working environment. The environmental perception module is a key component for acquiring information about the environment surrounding the unmanned excavator. Cameras, by capturing visible light images, provide rich texture, color, and semantic information, aiding in the identification of target objects, obstacles, and features of the working area. Lidar units, by emitting laser beams and measuring reflection time, generate high-precision 3D point cloud data, enabling accurate distance measurement, construction of a 3D geometric model of the environment, and contour extraction of obstacles. Deploying these sensors on the cab and boom ensures effective coverage of the working area, the excavator's own range of motion, and potentially hazardous areas, thereby providing comprehensive and multi-dimensional environmental perception capabilities.

[0022] The attitude feedback module includes a tilt sensor, a slewing angle measuring device, and an RTK (Real-Time Kinematic) high-precision positioning device. It is used to detect the attitude angles, slewing angles, and absolute geodetic coordinates of the excavator's actuators in real time. The actuators include the boom, stick, bucket, slewing mechanism, and traveling mechanism. The attitude feedback module monitors the motion status and precise position of the excavator and its core actuators in real time. The tilt sensor detects the pitch and roll angles of the excavator body or its actuators, which is crucial for maintaining vehicle balance and correcting operating posture. The slewing angle measuring device accurately measures the rotation angle of the excavator's upper slewing mechanism to determine the bucket's orientation relative to the body. The RTK high-precision positioning device, by receiving differential GPS signals, provides centimeter-level or even millimeter-level absolute geodetic coordinates of the excavator body, ensuring precise positioning in the global coordinate system. These actuators (boom, stick, bucket, slewing mechanism, and traveling mechanism) are the core components for excavator operation; accurate detection of their attitude and position is fundamental to achieving refined control and operation planning.

[0023] The onboard computing unit (PCU) generates motion control commands for the actuators based on data collected by the environmental perception module and the attitude feedback module. As the core processing unit on the unmanned excavator, the PCU is responsible for data fusion, decision-making, and command generation. It receives image and point cloud data from the environmental perception module, as well as attitude and positioning data from the attitude feedback module, and performs real-time fusion, analysis, and processing of this multi-source heterogeneous data. Based on these processing results, the PCU can understand the current working environment, accurately grasp the unmanned excavator's own motion state, and, in conjunction with preset tasks or received remote commands, generate refined motion control commands for each actuator.

[0024] The drive-by-wire module receives motion control commands and responds to them to drive the actuators to perform corresponding actions. It is responsible for converting the motion control commands generated by the onboard computing unit into actual mechanical movements to drive the various actuators of the unmanned excavator, enabling them to accurately complete tasks such as digging, loading, and moving according to the instructions.

[0025] The wireless communication module is used for data interaction with the remote cockpit subsystem. It ensures seamless data exchange between the onboard hardware subsystem and the remote cockpit subsystem. While transmitting environmental perception data, vehicle attitude data, and the unmanned excavator's status information collected by the onboard hardware subsystem to the remote cockpit subsystem for display in real time, it also receives control commands (including manual control commands and operation control commands) sent by the remote cockpit subsystem. This module can use a communication module based on a 4G / 5G cellular network, or a communication module based on Wi-Fi or a dedicated radio link.

[0026] For example, the onboard hardware subsystem 101 comprehensively and accurately acquires information about the working environment and the attitude data of the unmanned excavator itself through multi-source heterogeneous sensors (camera, lidar, tilt sensor, slewing angle measuring device, and RTK high-precision positioning device). This data, after effective processing by the onboard computing unit, generates highly precise motion control commands, which directly drive the actuators through the drive-by-wire module, achieving refined control of the boom, stick, bucket, slewing mechanism, and traveling mechanism.

[0027] Based on this processing method, the perception capability, positioning accuracy, and operation execution accuracy of unmanned excavators in complex operation scenarios can be effectively improved. It effectively solves the problems of incomplete environmental perception, inaccurate body posture feedback, and inaccurate control command conversion of traditional unmanned excavators, thereby ensuring the efficiency, safety, and reliability of operations.

[0028] Furthermore, the unmanned excavator system 100 described above can achieve remote monitoring and command input by the operator through the remote cockpit subsystem 102. To improve operational efficiency and accuracy, the remote cockpit subsystem includes: a cockpit-end computing unit for running the cockpit-end control software within the integrated software subsystem; human-machine interface hardware for displaying multi-view videos of the unmanned excavator transmitted from the vehicle-mounted hardware subsystem, the 3D environment model uploaded by the integrated software subsystem, and system status information, and for receiving manual control commands input by the operator; and a cockpit communication module, paired with the wireless communication module of the vehicle-mounted hardware subsystem to establish a two-way communication link.

[0029] In one embodiment, the cockpit communication module establishes a stable bidirectional communication link with the wireless communication module of the vehicle-mounted hardware subsystem 101, laying the foundation for data exchange between the remote cockpit subsystem 102 and the unmanned excavator 100. After the communication link is established, the multi-view video of the unmanned excavator 100 transmitted back by the vehicle-mounted hardware subsystem 101, the 3D environment model uploaded by the integrated software subsystem 103, and the system status information can be transmitted to the remote cockpit subsystem 102 through the cockpit communication module. After receiving this data, the cockpit-end computing unit runs the cockpit-end control software in the integrated software subsystem 103 deployed on it, processes and integrates the received data, and drives the human-machine interaction hardware to present this information to the operator in an intuitive and comprehensive manner.

[0030] Meanwhile, the operator inputs manual control commands through human-machine interface hardware, such as joysticks and buttons. These commands are then processed by the cockpit-side computing unit and sent back to the onboard hardware subsystem 103 via the cockpit communication module. This allows the operator to monitor the operating status and environmental information of the unmanned excavator 100 in real time and issue control commands accurately and promptly.

[0031] Through the collaborative work of various components, operators can gain a comprehensive understanding of the unmanned excavator's operation on an integrated platform and perform precise control, thereby significantly improving the efficiency, safety, and operating experience of remote operations. This effectively solves the problems of incomplete information acquisition and untimely or inaccurate command input in remote operations.

[0032] Furthermore, in the above embodiments, the integrated software subsystem 103 is capable of processing environmental perception data and body posture data, and generating operation control commands. Therefore, in order to improve the accuracy of autonomous planning and processing, the integrated software subsystem includes: a perception and recognition module, used to identify the position and posture of the dump truck associated with the unmanned excavator, and to generate a three-dimensional environmental model of the working environment; an autonomous operation module, used to generate excavation information and motion trajectory, and to generate operation control commands based on the excavation information and motion trajectory; and a cockpit-side control software module, used to provide a human-machine interface, and to perform mode switching, command issuance, status monitoring, and information display.

[0033] In one embodiment, the perception and recognition module is a key component of the integrated software subsystem 103. Its main function is to identify and model key targets in the unmanned excavator's operating environment. Specifically, the perception and recognition module is responsible for processing environmental perception data from the onboard hardware subsystem to identify the position and orientation of the dump truck bed associated with the unmanned excavator in real time, i.e., its position and orientation in three-dimensional space. Simultaneously, it is also responsible for constructing and continuously updating a three-dimensional environmental model of the operating environment, which includes information such as terrain, obstacles, and material accumulation status. The implementation of the perception and recognition module may include, but is not limited to: using deep learning algorithms to fuse data from multiple sensors (such as cameras and LiDAR) to achieve accurate detection and pose estimation of the dump truck bed; or employing SLAM (Simultaneous Localization and Mapping) based technology, combining visual and LiDAR data to construct a high-precision three-dimensional point cloud map and mesh model.

[0034] The autonomous operation module is the core of the integrated software subsystem 103, enabling autonomous operation. It performs intelligent decision-making and path planning based on environmental information and task details provided by the perception and recognition module. Specifically, the autonomous operation module generates detailed excavation information, including but not limited to determining the excavation sequence, specific excavation points, and unloading points. Based on this, the module plans the unmanned excavator's trajectory from its current position to the target point, ensuring a smooth and efficient operation. Finally, the module calculates and generates specific operation control commands based on the generated excavation information and trajectory, which are then sent to the onboard hardware subsystem 103 to drive the actuators. The implementation of the autonomous operation module can include, but is not limited to: using rule-based expert systems or reinforcement learning algorithms for operation decision-making; or utilizing path planning techniques such as the A* algorithm and the RRT (Rapidly-exploring Random Tree) algorithm to generate optimal or suboptimal trajectories.

[0035] The cockpit-side control software module is the specific embodiment of integrated software subsystem 103 within the remote cockpit subsystem, primarily providing human-machine interaction functions for the operator. It offers an intuitive human-machine interface, enabling operators to easily switch modes (such as remote control mode, semi-automatic loading / unloading mode, and fully automatic digging and loading mode), issue commands (such as starting / pausing operations and specifying digging areas), monitor status (such as excavator real-time attitude, operation progress, and system health status), and display information (such as multi-view video, 3D environment model, and dump truck position). The implementation of the cockpit-side control software module can include, but is not limited to: developing desktop applications based on frameworks such as Qt and Electron, providing rich graphical interfaces and interactive controls; or using Web technologies (such as React and Vue) to build browser-based applications, enabling cross-platform access and deployment.

[0036] For example, a perception and recognition module, an autonomous operation module, and a cockpit-side control software module are introduced into the integrated software subsystem. The perception and recognition module first uses environmental perception data collected by the onboard hardware subsystem to construct and update a three-dimensional environmental model of the working environment in real time, and accurately identifies the position and orientation of the dump truck bed associated with the unmanned excavator. Then, the autonomous operation module receives the three-dimensional environmental model and dump truck bed position and orientation information provided by the perception and recognition module, and intelligently generates detailed excavation information, including excavation points, unloading points, and operation sequences, in conjunction with preset work tasks or instructions issued by the operator through the cockpit-side control software module. It also plans the motion trajectory required for the unmanned excavator to perform these operations. This excavation information and motion trajectory are then converted into specific operation control instructions and sent to the drive-by-wire execution module of the onboard hardware subsystem to drive the unmanned excavator's actuators to complete the corresponding work actions. Meanwhile, the cockpit-side control software module provides the operator with a comprehensive human-machine interface within the remote cockpit subsystem. This interface displays real-time multi-view videos from the onboard hardware subsystem, a 3D environment model and dump truck pose model generated by the integrated software subsystem, and the real-time status of the unmanned excavator. This allows the operator to easily switch operating modes, issue high-level commands, and monitor the entire operation process, as well as handle anomaly alarms. This solves the problems of insufficient environmental understanding, limited autonomous decision-making capabilities, and low human-machine interaction efficiency in complex operating scenarios of traditional integrated software systems, enabling unmanned excavators to perform semi-automatic and fully automatic tasks more efficiently and safely.

[0037] Furthermore, in the above embodiments, the unmanned excavator 100 has three different operating modes, specifically, the integrated software subsystem 103 is configured with three operating modes, including remote control mode, semi-automatic loading and unloading mode, and fully automatic digging and loading mode. The integrated software subsystem further includes a perception and recognition module, an autonomous operation module, and a cockpit-side control software module. Considering the actual operating conditions of the unmanned excavator 100, efficient and accurate digging and unloading are required. Therefore, how to accurately and in real-time perceive the position and attitude of the dump truck bed, and how to accurately and dynamically acquire three-dimensional information of the operating environment (especially the material accumulation state), are key challenges to achieving efficient, safe, and autonomous operation.

[0038] To address this, a top-sensing recognition module is proposed, comprising: a dump truck pose recognition unit, which uses a deep learning model to identify and calculate the 3D position and orientation of the dump truck bed associated with the unmanned excavator in the excavator coordinate system based on camera and LiDAR data, and generates a dump truck pose model; and a 3D environment reconstruction and update unit, which integrates color image data, LiDAR point cloud data, and RTK positioning data to generate and update a 3D environment model of the working environment and material accumulation status in real time, and displays it.

[0039] In one embodiment, the dump truck pose recognition unit is used to identify and calculate the three-dimensional position and orientation of the dump truck bed associated with the unmanned excavator in the excavator coordinate system in real time based on camera and LiDAR data, using a deep learning model, and generate a dump truck pose model. In implementation, specific markers placed on the dump truck bed can be used to capture the 3D pose using a camera and image processing algorithms. Alternatively, dense point cloud data of the dump truck bed can be obtained using LiDAR, and the geometry of the bed can be identified and its pose estimated using point cloud registration algorithms or shape feature extraction methods. The combined use of camera and LiDAR data can compensate for the limitations of a single sensor, improving the robustness and accuracy of the recognition. Simultaneously, the deep learning model can automatically learn and extract complex features from a large amount of sensor data, thereby achieving high-precision target recognition and pose estimation.

[0040] The 3D environment reconstruction and update unit is used to fuse color image data, LiDAR point cloud data, and RTK positioning data to generate and display a 3D environment model that updates the working environment and material accumulation status in real time. By constructing and maintaining an accurate 3D model of the working environment and reflecting environmental changes, especially changes in material accumulation status, it provides comprehensive spatial information for the autonomous operation of the unmanned excavator 100, enabling it to understand the working scenario, plan paths, identify obstacles, and assess work progress. Data fusion can be performed at the feature level or decision level after sensor calibration and time synchronization. For example, pixels of color images can be associated with LiDAR point clouds to assign color information to the point clouds. Alternatively, probabilistic graphical models or optimization-based methods can be used to fuse data from different sensors to obtain a more accurate environmental state estimate. Color image data provides visual details and texture information of the environment, which helps to identify object categories and surface features; LiDAR point cloud data provides high-precision 3D geometric information, which is the basis for building the 3D model; RTK positioning data provides high-precision absolute position information of the unmanned excavator itself, used to convert local sensor data to a global coordinate system, ensuring the global consistency and accuracy of the map. Real-time updates ensure the model reflects dynamic changes at the work site, such as changes in the shape of material piles caused by excavation. The model's display function provides intuitive visual feedback for human-computer interaction, allowing operators to understand work progress and environmental conditions through the 3D model.

[0041] Furthermore, an autonomous operation module is proposed, comprising: an operation decision unit, used to generate excavation information based on the 3D environment model of the excavator's position and the operating environment, combined with the received operation task; the excavation information includes the excavation operation sequence, excavation points, and unloading points; a trajectory planning unit, used to plan the motion trajectory of the unmanned excavator from its current position to the target point; and a motion control unit, used to generate opening control commands for each joint mechanism on the unmanned excavator based on the motion trajectory using a dual closed-loop motion control algorithm of speed and position, and send the opening control commands to the drive-by-wire execution module.

[0042] In one embodiment, the operation decision-making unit is the core intelligent component of the autonomous operation module. It intelligently formulates operation strategies and plans based on current environmental information and operational objectives. The operation decision-making unit can be implemented using a rule engine or expert system-based approach. It presets a series of operation logics and priorities to make judgments and decisions based on the truck bed pose and 3D environment model data provided by the perception and recognition module. Alternatively, it can employ a decision-making model based on reinforcement learning or deep learning. Through learning and training on a large amount of operation data, it autonomously generates optimal mining strategies to adapt to different operation scenarios and material states.

[0043] The trajectory planning unit is responsible for calculating the optimal path for the unmanned excavator 100 from its current position to the target position. Path planning can employ graph search algorithms, such as A* or Dijkstra's algorithm, or sampling algorithms, such as Rapid Random Tree Exploration (RRT) or Probabilistic Route Map (PRM). During the planning process, factors such as obstacle avoidance, path smoothness, energy consumption, and the excavator's own kinematic and dynamic constraints are comprehensively considered.

[0044] The motion control unit is responsible for converting the motion trajectory planned by the trajectory planning unit into specific action commands for each joint mechanism of the unmanned excavator. It can employ a classic PID controller or a more advanced LQR controller, adjusting joint openings based on real-time feedback attitude data to achieve precise tracking of the target trajectory. Alternatively, it can use a model predictive control (MPC) algorithm to predict and generate optimal control commands based on the excavator's dynamic model and future trajectory information.

[0045] Through the collaborative working mechanism, the autonomous operation module can decompose high-level or relatively complex tasks into specific action sequences and accurately guide the unmanned excavator 100 to complete complex tasks, significantly improving the operation efficiency and accuracy of the unmanned excavator in fully automatic mode.

[0046] Furthermore, the cockpit-side control software module includes: a remote control interface for receiving control signals input by the operator and converting them into motion control commands for the unmanned excavator; a perception information interaction interface for distributing and displaying the multi-view video of the unmanned excavator, the dump truck's pose model, the real-time attitude model of the unmanned excavator, and the 3D environment model transmitted back by the vehicle-mounted hardware subsystem on the interactive interface; a mode switching and command issuance interface, providing graphical buttons for the operator to switch between three operating modes with one click; and a status monitoring and alarm unit for centrally displaying system status, work progress, and excavator status abnormality alarm information.

[0047] In one embodiment, the remote control interface is used to receive physical manipulation input from the operator and convert it into motion control commands that the system can recognize. This interface can be based on physical input devices such as joysticks or pedals, acquiring analog or digital signals to convert the operator's intentions into desired speed, position, or torque commands for each joint of the unmanned excavator. Alternatively, it can be based on a virtual control interface on a touchscreen or virtual reality device, generating corresponding control commands through gesture recognition or click operations.

[0048] The perception information interaction interface is used to integrate and display various perception data and model information from the vehicle hardware subsystem 101 and the integrated software subsystem 103. It can use graphical user interface (GUI) technology to display multi-view video streams, 3D environment models, dump truck pose models, and real-time attitude models in a regional and hierarchical layout. Alternatively, it can employ augmented reality (AR) or virtual reality (VR) technology to overlay or integrate this information into the operator's field of vision, providing an immersive interactive experience.

[0049] The mode switching and command issuance interface provides a convenient operating method, allowing operators to switch between different operating modes and issue advanced commands. Operators can intuitively select remote control mode, semi-automatic loading and unloading mode, or fully automatic excavation and loading mode through graphical buttons or drop-down menus, or quickly switch modes and issue specific commands through voice recognition or preset shortcut keys.

[0050] The status monitoring and alarm unit is responsible for monitoring the system's operating status, work progress, and potential anomalies in real time, and issuing timely warnings. It can display key parameters of the excavator, such as oil level, temperature, pressure, communication status, and workload, through dashboards, indicator lights, or text messages. Parameters exceeding safety thresholds will be highlighted or flashed. It can also integrate audible alarms, pop-up notifications, or SMS notifications, immediately alerting the operator when a fault, collision risk, or deviation from the preset path is detected.

[0051] Through the collaborative work of interfaces and units, operators can efficiently perform remote monitoring, issue commands, and switch modes on a unified and intuitive interface, significantly improving the remote control experience and operational efficiency of unmanned excavators.

[0052] Furthermore, the integrated software subsystem 103 is configured with three operating modes: remote control mode, semi-automatic loading and unloading mode, and fully automatic excavation and loading mode. Therefore, the integrated software subsystem is configured to: in remote control mode, convert real-time operator commands into motion control commands for the unmanned excavator and send these commands to the onboard hardware subsystem; display in real-time the multi-view video, real-time attitude model of the unmanned excavator, and 3D environment model transmitted back from the onboard hardware subsystem; monitor the pitch angle, communication quality, and operational safety status of the unmanned excavator, and issue alarm information when an anomaly is detected.

[0053] In one embodiment, in remote control mode, the integrated software subsystem 103 is responsible for converting operator commands, such as joystick or pedal signals, input by the operator through the remote cockpit subsystem into motion control commands required by the unmanned excavator's actuators. This conversion can be based on a preset mapping relationship, inverse kinematics algorithm, or dynamic model to ensure that the operator's intentions can be accurately translated into precise movements of the unmanned excavator's joints. The converted motion control commands are then sent to the onboard hardware subsystem via a wireless communication network, which drives the actuators to complete the corresponding work actions.

[0054] Meanwhile, the integrated software subsystem 103 is also responsible for displaying various types of information transmitted back from the vehicle-mounted hardware subsystem in real time. Multi-view video can be captured by multiple cameras deployed on the unmanned excavator itself, providing real-time image streams from different angles, allowing the operator to comprehensively observe the work site. The real-time attitude model can dynamically present the current attitude and position information of the unmanned excavator based on data provided by attitude feedback modules (such as tilt sensors and slewing angle measuring devices) in the vehicle-mounted hardware subsystem, combined with the unmanned excavator's 3D model. The 3D environment model can be reconstructed by fusing data collected by environmental perception modules (such as LiDAR and cameras) with RTK high-precision positioning data, providing a stereoscopic view of the work environment, including terrain, obstacles, and material accumulation status. This real-time display of information greatly enhances the operator's perception of the remote work environment and the status of the unmanned excavator.

[0055] In addition, the integrated software subsystem 103 monitors key operating parameters of the unmanned excavator 100. Pitch angle monitoring, based on data from tilt sensors, assesses the unmanned excavator's stability on slopes or uneven ground to prevent tipping. Communication quality monitoring includes real-time evaluation of indicators such as data transmission rate, latency, packet loss rate, and signal strength to ensure reliable transmission of remote control commands and integrity of returned information. Monitoring of operational safety status is more comprehensive, including assessments of potential collision risks, intrusion into the work area, and abnormal actuator movements. These assessments can be based on environmental perception data, attitude data, and preset safety rules. Upon detecting any anomalies, the integrated software subsystem immediately issues alarm information, such as audible and visual alarms or warning messages via the human-machine interface hardware of the remote cockpit subsystem, to remind the operator to take timely action.

[0056] Furthermore, the integrated software subsystem 103 is configured with three operating modes, including remote control mode, semi-automatic loading and unloading mode, and fully automatic excavation and loading mode. Therefore, the integrated software subsystem is configured to: in semi-automatic loading and unloading mode, identify the position and orientation of the dump truck associated with the unmanned excavator and feed it back to the interactive interface; respond to the operator's loading / unloading button instruction after completing the excavation operation, determine the unloading point and unloading path based on the real-time updated 3D environment model and the dump truck's position and orientation model associated with the unmanned excavator; and generate control commands based on the unloading point and unloading path and send them to the onboard hardware subsystem to control the unmanned excavator to perform the unloading operation.

[0057] In one embodiment, in semi-automatic loading and unloading mode, when the operator issues a loading / unloading button command through the human-machine interface, the integrated software subsystem immediately initiates the automated unloading process. During this process, the perception and recognition module acquires and updates the 3D environmental model of the operating environment in real time, and accurately identifies the position and orientation of the dump truck's bucket associated with the unmanned excavator, generating a dump truck orientation model. Based on this real-time and accurate environmental and target information, the autonomous operation module in the integrated software subsystem can intelligently determine the optimal unloading point and path. For example, it considers the current position, orientation, capacity, and material accumulation state of the dump truck's bucket, planning a safe, efficient, and uniformly loaded trajectory. Subsequently, these determined unloading points and paths are converted into specific motion control commands and sent to the onboard hardware subsystem. Upon receiving these commands, the drive-by-wire execution module in the onboard hardware subsystem drives the unmanned excavator's boom, stick, bucket, and other actuators to precisely complete the unloading operation according to the predetermined trajectory and orientation.

[0058] In this way, the unmanned excavator can seamlessly transition from manual digging to automated unloading in semi-automatic loading and unloading mode. The integrated software subsystem, by recognizing the dump truck's bucket position in real time and combining it with an updated 3D environment model, can accurately determine the unloading point and path, thereby generating precise control commands to drive the unmanned excavator to perform the unloading operation. This significantly improves the automation and accuracy of the unloading operation, and also enhances the overall safety and stability of the operation.

[0059] Furthermore, the integrated software subsystem 103 is configured with three operating modes, including remote control mode, semi-automatic loading and unloading mode, and fully automatic digging and loading mode. Therefore, the integrated software subsystem is configured to: in fully automatic digging and loading mode, generate a three-dimensional environment model of the operating environment of the unmanned excavator and identify the position of the dump truck's bucket associated with the unmanned excavator; in response to a selection operation in the three-dimensional environment model, determine the digging area and digging depth information of the unmanned excavator; generate digging information based on the three-dimensional environment model, the material accumulation state in the operating environment, the bucket position, the digging area, and the digging depth information, including digging points, digging strategies, and movement trajectories; calculate the digging information to obtain the opening control commands for each joint of the unmanned excavator and send them to the onboard hardware subsystem to drive the unmanned excavator to operate until a preset workload is achieved, the preset workload being determined based on the dump truck's full earthwork information or digging depth information.

[0060] In one embodiment, the integrated software subsystem 103 is capable of constructing and maintaining a digital representation of the work area surrounding the unmanned excavator. This three-dimensional environment model forms the basis for autonomous operation planning, providing key information such as terrain, obstacles, and material accumulation. It can also accurately detect and determine the position and orientation of the dump truck bed working in conjunction with the unmanned excavator 100 in three-dimensional space. The dump truck bed's orientation is crucial information for achieving precise unloading and loading. Simultaneously, the integrated software subsystem 103 provides a human-machine interaction method, allowing the operator to intuitively specify the range and depth of the excavation operation on the visualized three-dimensional environment model. This enables the operator to flexibly define work tasks without requiring complex parameter input.

[0061] Furthermore, the integrated software subsystem 103 will comprehensively consider various environmental and task parameters to intelligently plan specific excavation operation details. Among these, excavation information is the core instruction guiding the unmanned excavator to execute operations. Based on a 3D environmental model, it can analyze the shape, volume, and distribution of material accumulation, and combine this with the excavation area and depth. Using path planning algorithms and operation strategy algorithms (such as layered excavation, spiral excavation, etc.), it can generate the optimal sequence of excavation points, excavation order, and corresponding excavation strategies. It can also utilize rule-based expert systems or reinforcement learning models to dynamically adjust the excavation points, excavation order, and excavation strategies based on the current material accumulation state, the dump truck's bucket posture, and the set excavation area and depth, in order to maximize excavation efficiency and loading accuracy. The excavation information includes the excavation point, excavation strategy, and movement trajectory. The excavation point refers to the specific three-dimensional coordinate position of the unmanned excavator bucket in the material pile during the excavation operation. The excavation strategy refers to a series of action parameters and sequences, such as the bucket's posture, entry angle, excavation path, and bucket lifting speed, when performing excavation operations at each excavation point. The movement trajectory refers to the continuous movement path of the unmanned excavator from its current position to the excavation point, from the excavation point to the unloading point, and during the operation, the movement paths of each joint (such as the boom, stick, bucket, and slewing mechanism).

[0062] After determining the excavation information and movement path, further analysis and processing are needed to determine how to control the unmanned excavator 100. Specifically, the integrated software subsystem 103 transforms the high-level excavation information into precise control signals that the unmanned excavator's underlying actuators can understand and execute. This is a crucial step in achieving autonomous operation. For example, using inverse kinematics algorithms, the excavation points, excavation strategies, and movement trajectories defined in the excavation information are transformed into target angles or positions of various joints such as the boom, stick, bucket, and slewing mechanism at different points in time. Then, using a PID controller or other advanced control algorithms, these target values ​​are converted into opening control commands required to drive the hydraulic valves or motors of each joint. Alternatively, a model predictive control (MPC) approach can be used, combined with the dynamic model of the unmanned excavator, to calculate and generate opening control commands for each joint that meet the requirements of the excavation information in real time, while considering system constraints and optimization objectives.

[0063] Finally, the integrated software subsystem 103 ensures that the unmanned excavator 100 accurately completes its tasks by monitoring the work progress in real time. It should be noted that the work status at each excavation point can be determined based on a preset workload, which is based on the dump truck's full earthwork information or digging depth information. For example, the unmanned excavator will stop working at that excavation point after digging to the set depth, or it will stop working after the dump truck's load reaches the load corresponding to the full earthwork information.

[0064] Furthermore, referring to Figures 2 to 5 ,in, Figure 2 This is a schematic diagram of the operation process of an unmanned excavator provided in an embodiment of this application.

[0065] like Figure 2 As shown, the workflow for the three modes is as follows: 1. System startup and self-test.

[0066] After the system is powered on and the communication connection self-test is successful, the operator selects the operating mode according to the operational requirements on the control software interface in the cockpit.

[0067] 2. Remote control mode.

[0068] When the "remote control" mode is selected, the human-machine interface of the cockpit control software (see reference) Figure 5 The system will display real-time video footage from multiple perspectives (video latency can be reduced to less than 120ms through a low-latency video push-pull streaming program), a real-time 3D model of the excavator's posture, and a point cloud fusion 3D scene map of the current field of view (i.e., the 3D environment model in the above embodiments) as a perception feedback of the on-site working environment. The operator can remotely control the excavator in real time through the cockpit joystick and other means based on the above perception feedback information. At the same time, the system monitors the status of the excavator during operation and will issue alarm feedback when abnormal conditions such as excessive pitch angle of the excavator, reduced communication quality, or detection of personnel in the work area are detected.

[0069] 3. Semi-automatic loading and unloading mode.

[0070] When the "semi-automatic loading and unloading" mode is selected, the human-machine interface of the cockpit control software displays real-time video footage from multiple perspectives, a real-time 3D model of the excavator's posture, a point cloud fusion 3D scene map under the current field of view, and the dump truck's position. When the dump truck arrives in the excavator's working area, the system can automatically identify and locate the dump truck's bucket posture and feed it back to the human-machine interface. After the operator manually digs soil using the cockpit control lever and presses the "loading and unloading button," the system's autonomous operation module, based on the real-time updated point cloud fusion 3D scene map under the current field of view and the dump truck's bucket posture information, autonomously selects the optimal unloading point and obstacle avoidance unloading path, and precisely controls the excavator to achieve the unloading cycle of "lifting-slewing-unloading-slewing-resetting."

[0071] 4. Automatic excavation and loading mode.

[0072] After selecting the "fully automatic excavation and loading" mode and issuing the start command, the specific processing flow includes environmental perception feedback, excavation area selection, autonomous decision-making path planning, precise control, and cycle monitoring.

[0073] The environmental perception feedback includes the following steps: First, the excavator's boom, stick, and bucket are fully extended, meaning each joint of the boom, stick, and bucket moves to its maximum limit angle according to the set opening commands, reducing the impact of the excavator's upper structure on the LiDAR point cloud acquisition. Then, the excavator's slewing mechanism rotates one revolution from the initial angle at a set speed. At this time, the on-board perception and recognition module starts working simultaneously, with the LiDAR and camera continuously scanning. The 3D environment reconstruction unit generates a real-time updated map of the area surrounding the excavator and feeds it back to the cockpit to display a 3D map of the 360° working area with a radius of 20 meters around the excavator. Simultaneously, the dump truck pose recognition unit determines the target dump truck's bucket posture and feeds it back to the cockpit for display. The dump truck's bucket posture information is also saved in an intermediate file for use by the autonomous operation module. Once the current dump truck is full and has moved, and the next dump truck is in place, the system automatically identifies the next dump truck's bucket posture and updates the intermediate file.

[0074] Secondly, the selection of the excavation area includes: the operator selects the excavation area and sets the excavation depth in the 3D map display window on the cockpit according to the actual operation requirements, which is then passed to the perception and recognition module to generate the status information of the excavation area and sent to the autonomous operation module.

[0075] Next, autonomous decision-making includes: the operation decision-making unit of the autonomous operation module, which autonomously selects the optimal excavation point and excavation strategy (flat excavation or high platform drag excavation) based on the real-time updated 3D map of the area surrounding the excavator, the material accumulation status information of the excavation area, the attitude information of the dump truck bucket, and the excavation depth control instructions issued by the operator, and determines the unloading point based on the soil and rock elevation information of the dump truck bucket.

[0076] Furthermore, trajectory planning includes: the trajectory planning unit calculates a complete, smooth, collision-free motion trajectory from the current position to the excavation point, then back to the unloading point, and then to the next excavation point.

[0077] Then, precise control includes: the motion control unit calculates the trajectory into opening motion commands for each joint, and drives the excavator to execute a complete action cycle of "digging-lifting-slewing-unloading-slewing-resetting" through the wire-controlled controller.

[0078] Finally, the cycle and monitoring include: the system continuously executes the steps of "environmental perception feedback, excavation area selection, autonomous decision-making path planning, and precise control" until the preset workload is reached (e.g., the dump truck is fully loaded with earth and rock or the required excavation depth is achieved). Throughout the process, the operator can use the human-machine interface at the cockpit (see reference). Figure 5 It can monitor the operation process in real time and can intervene in the emergency stop state or switch modes at any time via the joystick.

[0079] Furthermore, the system is configured with three operating modes, which can be switched between. Specifically, during fully automatic operation, if the system encounters an unidentified obstacle (such as a large rock) or personnel entering the work area, it will stop operating and send an alarm to the cockpit. The operator can immediately switch to "remote control" mode to manually operate the excavator to resolve the abnormality, and then switch back to "fully automatic" mode, and the system will continue operating. Simultaneously, when the excavation area contains a lot of earth and rock debris, or when the condition of the lower layer of earth and rock is highly variable and inconvenient for automatic excavation, the operator can switch to "semi-automatic loading and unloading" mode to maximize efficiency. This eliminates the need to select an excavation area; the operator can manually operate the excavator to complete the earth and rock excavation, and then press the "loading and unloading" button next to the control lever, and the system will automatically complete the loading and unloading actions.

[0080] Additionally, refer to Figure 3 , Figure 3 This is a schematic diagram of the structural architecture of an unmanned excavator provided in an embodiment of this application. The overall architecture of the unmanned excavator includes an on-board hardware subsystem installed on the excavator, a remote cockpit subsystem deployed in the control center, and an integrated software subsystem deployed in the computing units of the two subsystems.

[0081] At the same time, refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the hardware deployment of the vehicle-mounted hardware subsystem provided in this application on an unmanned excavator. Specifically, a camera with a field of view of at least 120° is installed at the top front, sides, and rear of the excavator's cab. An area-array LiDAR is integrated at the location of the camera at the top front of the excavator's cab as the main LiDAR to collect point cloud information about the excavator's surrounding environment. Two area-array LiDARs are installed on each side of the area connecting the excavator's boom and stick as auxiliary LiDARs to collect point cloud information below the excavator's tracks and inside the dump truck's bed. A mechanical LiDAR is mounted on a bracket in the middle of the excavator's cab as a blind spot LiDAR to collect point cloud information from the sides and rear of the excavator, forming a 360° circumferential and 180° vertical environmental perception capability.

[0082] Tilt sensors and slewing angle measuring devices are installed at key locations on the excavator's boom, stick, bucket, cab, and slewing center.

[0083] The RTK base station was set up at a high point near the work area, and the mobile station antenna was installed on both sides of the rear of the excavator cab.

[0084] The wire-controlled device (i.e., the wire-controlled execution module in the above embodiments) is connected in parallel with the original manually operated mechanical hydraulic pilot valve of the excavator, so that it can receive CAN signals through the wire-controlled controller and directly control the hydraulic oil circuit, while ensuring that the original manual operation branch is not affected.

[0085] The on-board computing unit is an industrial-grade rugged computer installed in the excavator cab and connected to all sensors and actuators via interfaces such as CAN bus and Ethernet.

[0086] High-power wireless communication modules (such as 5G CPE or wireless self-organizing network base stations) are installed on the vehicle body to ensure a stable connection with the remote control cockpit.

[0087] In summary, the above embodiments provide an unmanned excavator, including an onboard hardware subsystem deployed on the excavator body, used to collect environmental perception data and body posture data of the working environment in real time, and drive the excavator's actuators to complete the operation according to the received control commands; a remote cockpit subsystem, connected to the onboard hardware subsystem through a wireless communication network, used to provide remote monitoring and command input interface, send externally input manual control commands to the onboard hardware subsystem, and receive and display environmental and status information returned by the onboard hardware subsystem; an integrated software subsystem, deployed in the onboard computing unit of the onboard hardware subsystem and the cockpit-end computing unit of the remote cockpit subsystem, used to process the environmental perception data and body posture data, generate operation control commands including operation decisions and motion trajectories, send the operation control commands to the onboard hardware subsystem, and provide a human-machine interface; wherein, the integrated software subsystem is configured with three operation modes, including remote control mode, semi-automatic loading and unloading mode, and fully automatic excavation and loading mode.

[0088] The unmanned excavator provided in the embodiments of this application has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application. Moreover, those skilled in the art can make several improvements and modifications without departing from the principles of this application, and these improvements and modifications are also considered to be within the protection scope of this application.

Claims

1. An unmanned excavator, characterized in that, include: The vehicle-mounted hardware subsystem is deployed on the unmanned excavator body and is used to collect environmental perception data and body posture data of the working environment in real time, and drive the actuator of the unmanned excavator to complete the working action according to the received control commands. The remote cockpit subsystem is connected to the vehicle hardware subsystem via a wireless communication network. It provides a remote monitoring and command input interface, sends externally input manual control commands to the vehicle hardware subsystem, and receives and displays environmental and status information returned by the vehicle hardware subsystem. An integrated software subsystem is deployed in the vehicle computing unit of the vehicle hardware subsystem and the cockpit computing unit of the remote cockpit subsystem. It is used to process the environmental perception data and the body posture data, generate operation control commands that include operation decisions and motion trajectories, send the operation control commands to the vehicle hardware subsystem, and provide a human-machine interaction interface. The integrated software subsystem is configured with three operating modes: remote control mode, semi-automatic loading and unloading mode, and fully automatic excavation and loading mode.

2. The unmanned excavator according to claim 1, characterized in that, The vehicle-mounted hardware subsystem includes: The environmental perception module consists of multiple cameras and / or lidar arranged in the cab and boom of the unmanned excavator, used to collect images and three-dimensional point cloud data of the surrounding environment of the unmanned excavator in the working environment. The attitude feedback module includes a tilt sensor, a slewing angle measuring device, and an RTK high-precision positioning device, used to detect in real time the attitude angle, slewing angle, and absolute ground coordinates of the actuator of the unmanned excavator. The actuator includes a boom, stick, bucket, slewing mechanism, and traveling mechanism. The on-board computing unit is used to generate motion control commands for the actuator based on the data collected by the environmental perception module and the data collected by the attitude feedback module. A drive-by-wire actuator module is used to receive the motion control command and respond to the motion control command to drive the actuator to perform corresponding actions; The wireless communication module is used for data interaction with the remote cockpit subsystem.

3. The unmanned excavator according to claim 1, characterized in that, The remote cockpit subsystem includes: A cockpit-side computing unit is used to run the cockpit-side control software in the integrated software subsystem; The human-machine interaction hardware is used to display the multi-view video of the unmanned excavator transmitted back by the vehicle-mounted hardware subsystem, the three-dimensional environment model and system status information uploaded by the integrated software subsystem, and to receive the manual control commands input by the operator. The cockpit communication module is paired with the wireless communication module of the vehicle hardware subsystem to establish a two-way communication link.

4. The unmanned excavator according to claim 1, characterized in that, The integrated software subsystem includes: The perception and recognition module is used to identify the position and posture of the dump truck bed associated with the unmanned excavator, and to generate a three-dimensional environment model of the working environment. An autonomous operation module is used to generate excavation information and movement trajectory, and to generate operation control instructions based on the excavation information and movement trajectory; The cockpit-side control software module is used to provide a human-machine interface, as well as to switch modes, issue commands, monitor status, and display information.

5. The unmanned excavator according to claim 4, characterized in that, The perception and recognition module includes: The dump truck pose recognition unit is used to identify and calculate the three-dimensional position and orientation of the dump truck bed associated with the unmanned excavator in the excavator coordinate system in real time based on camera and lidar data and through a deep learning model, and generate the dump truck pose model of the dump truck. The three-dimensional environment reconstruction and update unit is used to fuse color image data, lidar point cloud data and RTK positioning data to generate and update the three-dimensional environment model of the working environment and material accumulation status in real time and display it.

6. The unmanned excavator according to claim 4, characterized in that, The autonomous operation module includes: The operation decision unit is used to generate excavation information based on the truck bed position and the three-dimensional environment model of the operation environment, combined with the received operation task. The excavation information includes the excavation operation sequence, excavation point and unloading point. The trajectory planning unit is used to plan the motion trajectory of the unmanned excavator from its current position to the target point. The motion control unit is used to generate opening control commands for each joint mechanism on the unmanned excavator based on the motion trajectory using a dual closed-loop motion control algorithm of speed and position, and sends the opening control commands to the drive-by-wire execution module.

7. The unmanned excavator according to claim 5, characterized in that, The cockpit-side control software module includes: The remote control interface is used to receive the operation signals input by the operator and convert the operation signals into motion control commands for the unmanned excavator; The perception information interaction interface is used to display the multi-view video of the unmanned excavator, the pose model of the dump truck, the real-time attitude model of the unmanned excavator, and the three-dimensional environment model transmitted back by the vehicle hardware subsystem on the interactive interface. The mode switching and command issuance interface provides a graphical button for operators to switch between three operating modes with one click; The status monitoring and alarm unit is used to centrally display system status, work progress, and alarm information for excavator status abnormalities.

8. The unmanned excavator according to claim 1, characterized in that, The integrated software subsystem is configured as follows: In remote control mode, the operator's instructions received in real time are converted into motion control instructions for the unmanned excavator, and the motion control instructions are sent to the vehicle-mounted hardware subsystem. The multi-view video transmitted back by the vehicle-mounted hardware subsystem, the real-time attitude model of the unmanned excavator transmitted back by the vehicle-mounted hardware subsystem, and the three-dimensional environment model are displayed in real time. The pitch angle, communication quality, and operational safety status of the unmanned excavator are monitored, and an alarm is issued when an abnormality is detected.

9. The unmanned excavator according to claim 1, characterized in that, The integrated software subsystem is configured as follows: In semi-automatic loading and unloading mode, the position and posture of the dump truck bed associated with the unmanned excavator are identified and fed back to the interactive interface; In response to the operator's instruction to press the loading / unloading button after completing the excavation work, the unloading point and unloading path are determined based on the real-time updated three-dimensional environment model and the dump truck pose model associated with the unmanned excavator. Control commands are generated based on the unloading point and the unloading path and sent to the vehicle-mounted hardware subsystem to control the unmanned excavator to perform unloading operations.

10. The unmanned excavator according to claim 1, characterized in that, The integrated software subsystem is configured as follows: In fully automatic excavation and loading mode, a three-dimensional environment model of the working environment of the unmanned excavator is generated, and the position and posture of the dump truck associated with the unmanned excavator are identified. In response to a selection operation in the three-dimensional environment model, the excavation area and digging depth information of the unmanned excavator are determined; Based on the three-dimensional environment model, the material accumulation state in the working environment, the truck bed position, the excavation area, and the excavation depth information, excavation information is generated, which includes excavation points, excavation strategies, and movement trajectories. The excavation information is processed to obtain the opening control commands of each joint of the unmanned excavator and sent to the vehicle-mounted hardware subsystem to drive the unmanned excavator to work until a preset workload is achieved. The preset workload is determined based on the dump truck's full earthwork information or the excavation depth information.

Images