Control methods, devices, systems, and loaders for autonomous digging devices on loaders
Patent Information
- Application Number
- CN202611635533.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-22
- Publication Date
- 2026-11-17
Smart Images

Figure CN123109230A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent control technology for excavating construction machinery, and in particular to a control method, device, system, and loader for an autonomous digging device of a loader. Background Technology
[0002] Loaders, as widely used construction machinery, operate in cycles including digging, transporting, and unloading. The digging phase is the core of the interaction between the loader and the material, and this process accounts for the majority of the machine's energy consumption. Furthermore, the digging process requires coordinating forward travel with the lifting or rotating motion of the bucket, demanding extremely high operator skills. Therefore, autonomous digging has become a key technology for the intelligentization of loaders. To achieve autonomous digging, research must be conducted on environmental perception and intelligent decision optimization to improve the accuracy of autonomous digging strategies.
[0003] Current research on autonomous control strategies for loader shoveling hydraulic systems mainly includes numerical simulations using discrete element methods and multibody dynamics, as well as theoretical modeling of bucket-material interactions using soil mechanics and multibody dynamics. However, numerical simulation and theoretical modeling methods abstract the shoveling process into an optimization problem within a closed, deterministic model, and are considered embodied intelligence paradigms. In contrast, embodied intelligence paradigms emphasize continuous interaction and learning between the agent and the environment to achieve autonomous decision-making and adaptive capabilities. This mainly includes: imitation learning, which trains the controller to reproduce manual shoveling strategies by collecting expert demonstration data, and reinforcement learning methods, which autonomously optimize strategies through trial-and-error interaction with the environment. However, traditional reinforcement learning training processes face a dilemma between simulated and real environments. Specifically, interaction with simulated environments often fails to capture the complex mechanical behavior of bulk materials in real environments, leading to a significant "reality gap" in shoveling strategy optimization; trial-and-error learning with real environments, through reinforcement learning training in real environments, is time-consuming and resource-intensive, while also facing challenges such as equipment damage and safety risks.
[0004] To overcome these challenges, a novel solution based on world model reinforcement learning is proposed. This method utilizes historical and online operational data of the loader's digging process in a real environment to construct a reasonable internal representation world model. This model enables the loader agent to quickly explore strategies and adaptively optimize within the model, ultimately applying the learned digging strategies to the real environment. However, constructing a purely data-driven world model using the loader's own operational data results in a "black box" model, lacking explicit constraints on the bucket-material coupling mechanism, making it difficult to guarantee the accuracy and reliability of the internal representation.
[0005] Therefore, there is a need for a control method and device for an autonomous digging device of a loader based on model and data fusion, which can integrate the bucket-material coupling mechanism and the changing characteristics of the operating data during the digging process, and improve the accuracy of the autonomous digging strategy. Summary of the Invention
[0006] To address the issue of simulated and real-world environments in traditional reinforcement learning training processes, embodiments of this invention provide a control method, device, medium, and loader for an autonomous digging device driven by model and data fusion, effectively improving the accuracy of the autonomous digging strategy.
[0007] To achieve the above objectives, a control method for an autonomous digging device of a loader is provided, comprising the following steps: Step S1: Obtain the loader's digging operation data and the material pile surface information, and construct a physical theoretical control model for the loader's autonomous digging process. The digging operation data includes: current state parameter s t and excavation control parameter a t And according to the current state parameter s t and excavation control parameter a t The optimal digging trajectory and the corresponding theoretical digging performance are obtained through the physical theory control model. and theoretical digging performance error Where t represents the current time step; the physical theory control model includes: kinematic model, dynamic model, bucket force model, bucket-material coupling model, digging resistance simulation model, and digging trajectory optimization model; Step S2, using the current state parameter s t , Excavation control parameter a t and theoretical digging performance error Using this as input, a data compensation model for digging performance error based on Transformer and Hybrid Density Network (MDN) is constructed to predict the output digging performance error. and the state parameters at the next moment and will Compared with theoretical digging performance Error compensation is performed to obtain the corrected digging performance. Thus, a world model is obtained based on the excavation performance error data compensation model and the physical theory control model. Step S3: Based on the world model, construct an autonomous digging agent for the loader using a reinforcement learning algorithm, and train the autonomous digging agent for the loader; the agent uses the state parameters of the next time step. As input, output the shoveling control parameters for the next moment. The training of the loader's autonomous digging agent includes: constructing a reward function based on the prediction results of the world model, and driving the agent's policy network to update with the goal of maximizing the cumulative reward, until the policy converges, thereby obtaining an autonomous digging controller. Step S4: Deploy the autonomous digging controller in the actual digging operation environment and output digging control commands to the loader actuator.
[0008] Preferably, the control method for the loader's autonomous digging device is characterized in that step S1 includes: The surface information of the material pile before the digging operation is obtained by using binocular vision and / or lidar. The depth information of lidar is matched and fused with the color and texture information of the image to construct a three-dimensional model of the material pile surface. Material characteristic parameters are obtained, and kinematic, dynamic, bucket force, bucket-material coupling, and digging resistance simulation models of the loader's digging process are constructed using the digging operation data and material characteristic parameters, respectively, to simulate and output the digging trajectory and its corresponding digging resistance. The bucket filling rate is calculated based on the shovel trajectory and the three-dimensional model of the material pile surface. And calculate the digging efficiency based on the digging operation time. ; By integrating the three-dimensional information of the material pile surface and the generated digging trajectory information, a digging trajectory optimization model is constructed to output the optimal digging trajectory and its corresponding theoretical digging performance. The theoretical digging performance Due to the excavation resistance Digging efficiency and shovel full bucket rate The results were obtained through comprehensive calculations.
[0009] Preferably, the control method for the loader's autonomous digging device calculates the theoretical digging performance. The steps include: The digging resistance was respectively... Full rate and digging efficiency Normalize the mapping to the [0, 1] interval; Obtained by linear weighted summation : , in, Let be the weighting coefficient, satisfying , and They are respectively The maximum and minimum values; and They are respectively The maximum and minimum values; and They are respectively The maximum and minimum values.
[0010] Preferably, the control method for the loader's autonomous digging device includes: , in, , These are the pressure signals of the large and small cylinders of the boom's autonomous digging device for the loader; , These are the pressure signals of the large and small chamber hydraulic cylinders of the loader's autonomous digging device; , , These are the displacement signals of the boom cylinder, bucket cylinder, and vehicle forward displacement of the loader's autonomous digging device; This is the torque signal for the drive shaft; , These are the horizontal and vertical material penetration depths of the loader's autonomous digging device.
[0011] Preferably, the control method for the loader's autonomous digging device includes: A data compensation model for digging performance errors is constructed using a 3-layer Transformer and a 3-layer hybrid density network (MDN). The Transformer is used to capture the long-range dependencies of the time series data of the digging process, and the MDN is used to output the probability distribution of digging performance errors. The current state parameters are then used... Excavation control parameters and simulation performance error A time-series dataset is constructed as input to the digging performance error data compensation model, and the digging performance error is predicted as output. and the state parameters at the next moment ; The world model is expressed in probability density form as follows: , Where T represents the total number of time steps in the complete time series of the digging process, h t This represents the hidden layer state parameters of the Transformer model at time t; The training loss function of the world model is: ,in, This indicates the excavation process at the end of the complete time series. The digging performance is adjusted in real time.
[0012] Preferably, the control method for the autonomous digging device of the loader, wherein the steps of constructing a reward function based on the prediction results of the world model and driving the policy network of the agent to update with the objective of maximizing the cumulative reward until the policy converges to obtain the autonomous digging controller include: The agent is constructed using the Actor-Critic algorithm in reinforcement learning; the agent includes a policy network and a value network. During model training, the policy network generates initial shoveling control parameters based on the current state parameters. The parameters are then input into the world model to predict the state parameters for the next time step. Model parameters and the corresponding modified digging performance ; A reward function is constructed based on the prediction results, input into the value network to guide optimization, and the policy network is updated using the policy gradient method to minimize the temporal difference error in updating the value network. The policy network and the value network are alternately updated and iterated until the network converges, thus obtaining the autonomous digging controller.
[0013] Preferably, in the control method of the loader's autonomous digging device, the final reward is constructed by combining dense and sparse rewards. The dense reward is: at each time step Calculate real-time progress rewards ,in To achieve the maximum digging performance, The initial digging performance; the sparse reward is: when the digging process is completed... After a certain time step, a terminal reward will be given. The final reward The weighted sum of the dense reward and the sparse reward: ,in, The weighting coefficient represents the weighting percentage of the intensive reward in the overall reward, and its preferred value range is 0.6 to 0.9.
[0014] On the other hand, a control device is also provided, including a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the control method of the loader autonomous digging device as described in any of the above.
[0015] On the other hand, a control system for an autonomous digging device for a loader is also provided, including: The multi-source sensor data acquisition module is used to collect multi-source parameter data of the loader during the digging operation; the multi-source parameter data includes: hydraulic cylinder pressure signal, hydraulic cylinder displacement signal, vehicle forward displacement signal, drive shaft torque signal of the loader's autonomous digging device boom and / or bucket, and point cloud data of the material pile surface perceived by binocular vision and / or lidar. The physical mechanism modeling module is used to construct a physical theoretical control model of the loader's digging process based on the multi-source parameter data. This physical theoretical control model includes: a kinematic model, a dynamic model, a bucket force model, a bucket-material coupling model, a digging resistance simulation model, and a digging trajectory optimization model. The simulation outputs the optimal digging trajectory and the corresponding theoretical digging performance. and theoretical digging performance error ; The error data-driven compensation module has a built-in Transformer-MDN hybrid model. It is used to predict the state and shovel performance error at the next moment by taking the current state parameters, shovel control parameters and shovel performance error parameters of the shovel process as inputs, and to perform error compensation on the output of the physical mechanism modeling module, so as to construct a world model of the shovel performance error data compensation model and the physical theory control model. The control strategy optimization module uses a reinforcement learning algorithm to construct an autonomous digging agent for the loader, uses the world model to iteratively optimize and train the strategy, and outputs the optimal digging control strategy. The execution module is used to deploy the trained autonomous digging agent in the vehicle-mounted controller of the actual digging operation environment, receive real-time sensor data and output digging control commands to the loader's execution mechanism.
[0016] On the other hand, a loader is also provided, including the control device as described above.
[0017] The above technical solution has the following technical effects: (1) The present invention innovatively incorporates the bucket-material coupling mechanism in the digging process into the construction of the world model. By constructing a physical theory control model to provide physical constraints and prior knowledge, and using an error data compensation model to adaptively compensate for the unmodeled dynamics and errors of the physical theory control model, a dual-drive architecture that complements theory and data is formed, overcoming the limitations of a single model in terms of representation accuracy or generalization.
[0018] (2) The embodiments of the present invention construct an internal representation world model based on real operating data, enabling the agent to complete efficient strategy exploration and optimization within the model, and then deploy it to the real environment. This fundamentally decouples the learning process from physical trial and error, taking into account both the accuracy of environmental representation and the safety of training, solving the dilemma of simulated environment and real environment in the traditional reinforcement learning training process in the prior art, and improving the accuracy of autonomous digging strategy. Attached Figure Description
[0019] Figure 1 This is a flowchart illustrating the control method of a loader's autonomous digging device according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the construction process of the physical theory control model for the autonomous digging process of a loader, as shown in one embodiment of the present invention. Figure 3 This is a schematic diagram of the control system of an autonomous digging device for a loader according to an embodiment of the present invention. Detailed Implementation
[0020] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.
[0021] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.
[0022] Example 1: Figure 1 This is a flowchart illustrating the control method of a loader's autonomous digging device according to an embodiment of the present invention. Figure 1 As shown, the method of this embodiment includes the following steps: Step (1): By integrating shoveling operation data and material pile surface information, a physical theory control model for the loader's autonomous shoveling process is constructed. The physical theory control model includes a kinematic model, a dynamic model, a bucket force model, a bucket-material coupling model, a shoveling resistance simulation model, and a shoveling trajectory optimization model, which are used to solve for the optimal shoveling trajectory and its corresponding theoretical shoveling performance. , This indicates the current time step; the specific process for constructing the physical theory control model is detailed in [reference needed]. Figure 2 As shown.
[0023] Step (2): Utilize the current status parameters in the excavation operation data. and excavation control parameters The physical theory control model of the loader's autonomous digging process in step (1) is adopted to output the theoretical digging performance. and the corresponding theoretical digging performance error ; Step (3): Using the current state parameters Excavation control parameters and simulation performance error As input, a data compensation model for digging performance errors is constructed based on a Transformer-MDN hybrid model. This error data compensation model is then fused with a physical theory control model to construct a world model driven by model and data fusion. This world model is used to predict digging performance errors. and with Error compensation is performed to obtain the corrected digging performance. and the state at the next moment ; Step (4): Construct an autonomous digging agent for the loader using reinforcement learning algorithms; the agent uses state parameters For input, where This represents the network parameters of the Transformer model at the next time step and outputs the digging control parameters. The world model uses control parameters and state parameters as inputs to predict the corrected digging performance. and the state at the next moment Based on the prediction results, a reward function is constructed, and the policy network of the agent is driven to be updated iteratively with the goal of maximizing the cumulative reward. The above process is repeated until the policy converges, and an autonomous digging controller with the optimal digging policy is obtained. Step (5): Deploy the autonomous digging controller in the actual digging operation environment to realize autonomous digging decision-making and control.
[0024] Specifically, such as Figure 2 As shown, step (1) involves solving for the optimal digging trajectory and its corresponding theoretical digging performance. The specific steps are as follows: Step (1.1): Use binocular vision and / or lidar to acquire the surface information of the material pile before the digging operation, and perform feature matching and fusion of the depth information of lidar and image color and texture information to construct a three-dimensional model of the material pile surface. Step (1.2) utilizes the digging operation data and material characteristic parameters to construct the kinematic model, dynamic model, bucket force model, bucket-material coupling model, and digging resistance simulation model of the loader's digging process, respectively, to simulate and output the digging trajectory and its corresponding digging resistance. The bucket filling rate is calculated based on the digging trajectory and the three-dimensional model of the material pile surface. And calculate the digging efficiency based on the digging operation time. ; In one specific implementation, the specific digging operation data includes current state parameters and digging control parameters, such as parameters related to oil pressure, speed, and displacement; in another specific implementation, the construction of each of the above physical theory control models is based on existing technology and will not be elaborated here.
[0025] Step (1.3) integrates the three-dimensional information of the material pile surface and the generated digging trajectory information to construct a digging trajectory optimization model, so as to output the optimal digging trajectory and its corresponding theoretical digging performance. Theoretical digging performance Due to excavation resistance Digging efficiency and shovel full bucket rate The results were obtained through comprehensive calculations.
[0026] Specifically, such as Figure 2 As shown, the theoretical digging performance in step (1.3) The evaluation was conducted comprehensively using the following methods: First, the resistance to digging was assessed separately. Full rate and digging efficiency Normalization is performed to map each indicator to the [0, 1] interval; then, a comprehensive performance index is obtained through linear weighted summation. ,in, These are the weighting coefficients for each performance indicator, satisfying... Furthermore, it can adaptively adjust according to different working conditions.
[0027] Specifically, such as Figure 1 As shown, the current state parameters mentioned in step (2) Represented as: ,in, , These are the pressure signals from the large and small chamber hydraulic cylinders of the boom, respectively. , These are the pressure signals from the large and small chamber hydraulic cylinders of the rotating bucket, respectively. , , These are respectively the boom cylinder displacement, bucket cylinder displacement, and vehicle forward displacement signals; This is the torque signal for the drive shaft; , These are the horizontal and vertical material penetration depths, respectively.
[0028] Specifically, such as Figure 1 As shown, the specific steps for constructing the world model based on model and data fusion in step (3) are as follows: Step (3.1) involves constructing a data compensation model for digging performance errors using a 3-layer Transformer and a 3-layer MDN. The Transformer module is used to capture the long-range dependencies of the time series data during the digging process, while the MDN module is used to output the probability distribution of digging performance errors. The current state parameters are then used... Excavation control parameters and simulation performance error Construct a time series dataset as input to the model to predict the output state at the next time step. and predicted digging performance error ; In this specific embodiment, the network structure parameters of the error data compensation model are set as follows: The encoder layer of the Transformer module adopts a multi-head self-attention mechanism, with the number of attention heads preferably being 4 to 8, and the hidden layer dimension preferably being 128 to 512. In this embodiment, it is preferably set to 8 attention heads and a hidden layer dimension of 256. The MDN module consists of 3 hidden layers, with the number of neurons decreasing sequentially (e.g., from 512 to 128), used to map the output of the Transformer to a Gaussian mixture distribution parameter. The number of Gaussian mixture components (i.e., the number of mixture components) of the MDN is... The optimal value is 3 to 5, and in this embodiment, it is preferably set to 3. The ReLU function is used as the activation function to prevent gradient vanishing. The time series dataset constructed using the above structural parameters is iteratively trained using the Adam optimization algorithm until the model converges, thereby obtaining a trained error data compensation model.
[0029] Step (3.2) involves predicting the digging performance error. Theoretical digging performance Error compensation is performed to obtain the corrected digging performance. and the state at the next moment Thus, a world model based on model and data fusion is constructed; the world model is expressed in probability density form as follows: ,in, This indicates that the Transformer model at time... The hidden state parameters of the network; the training loss function of this world model is: ,in, This indicates the excavation process at the end of the complete time series. The digging performance is adjusted in real time.
[0030] Specifically, such as Figure 1As shown, the specific steps for obtaining the autonomous digging controller with the optimal digging strategy in step (4) are as follows: Step (4.1) uses the Actor-Critic algorithm in reinforcement learning to construct the autonomous control strategy of the loader's digging hydraulic system. The algorithm consists of a policy network (Actor) and a value network (Critic). In step (4.2), during model training, the policy network generates initial digging control parameters. And combined with the current state parameters As input to the world model, to predict the output state at the next time step. Model parameters and the corresponding modified digging performance ; Step (4.3) involves constructing the final reward using a combination of dense and sparse rewards. The dense reward is: at each time step The new time-state is predicted based on the world model. and the corrected digging performance Calculate real-time progress rewards ,in To achieve the maximum digging performance, The initial digging performance; the sparse reward is: when the digging process is completed... After a certain time step, a terminal reward will be given. This is used to evaluate the overall performance of the entire digging trajectory; the final reward It is a weighted sum of dense and sparse rewards, i.e.: ,in, The weighting coefficient represents the proportion of intensive rewards in the overall reward, and its preferred value is between 0.6 and 0.9. In this specific embodiment, it is determined by trial-and-error or grid search. In this embodiment, the value of is preferably set to 0.8; in practical engineering applications, It can make adaptive adjustments based on the stability and convergence speed requirements of different operating conditions of the loader. For example, when the initial network training is difficult, the dense reward weight can be appropriately increased to guide the agent to quickly explore the target state.
[0031] Final reward The policy network parameters are passed as input to guide its optimization. At the same time, the policy gradient method is used to update the policy network parameters, and the value network parameters are updated by minimizing the temporal difference error to ensure its convergence and stability. In step (4.4), the policy network and the value network are iteratively optimized by alternating updates until the network converges and outputs the optimized digging control decision.
[0032] Example 2: Figure 3 This is a schematic diagram of the control system of a loader's autonomous digging device according to an embodiment of the present invention. Figure 3 The model- and data fusion-driven autonomous loader digging device control system of this embodiment of the invention includes: The multi-source sensor data acquisition module 201 is used to collect multi-source parameter data of the loader during the actual digging operation. The multi-source parameter data includes: hydraulic cylinder pressure signal of boom and / or bucket, hydraulic cylinder displacement signal, vehicle forward displacement signal, drive shaft torque signal, and point cloud data of the material pile surface perceived by binocular vision and / or lidar. The physical mechanism modeling module 202 is used to construct kinematic models, dynamic models, bucket force models, bucket-material coupling models, and digging resistance simulation models of the loader digging process based on multi-source parameter data, and to simulate and output candidate digging trajectories and their corresponding digging performance. The error data-driven compensation module 203 has a built-in Transformer-MDN hybrid model. It is used to predict the state and shovel performance error at the next moment by taking the current state parameters, shovel control parameters and shovel performance error parameters of the shovel process as inputs, and to perform error compensation on the output of the physical mechanism modeling module in order to build a world model based on model and data fusion. The control strategy optimization module 204 constructs an autonomous digging agent based on the Actor-Critic reinforcement learning algorithm, uses a world model to iteratively optimize and train the strategy, and outputs the optimal digging control strategy. The vehicle-mounted controller deployment and execution module 205 is used to deploy the trained autonomous digging agent in the vehicle-mounted controller in the actual digging operation environment, receive real-time sensor data and output digging control commands to the hydraulic actuator of the loader.
[0033] Example 3: The present invention also provides a control device, which includes a processor and a memory; the memory stores at least one program, which is executed by the processor to implement the control method embodiment of the loader autonomous digging device as described above.
[0034] Furthermore, as an executable solution, the device can be a computer unit, which may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer unit may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the computer unit is merely an example and does not constitute a limitation on the computer unit. It may include more or fewer components, or combine certain components, or use different components. For example, the computer unit may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.
[0035] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, connecting various parts of the entire computer unit via various interfaces and lines.
[0036] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer unit by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0037] Example 4: The present invention also provides a loader that includes the control device described above.
[0038] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.
Claims
1. A control method for an autonomous digging device of a loader, characterized in that, Includes the following steps: Step S1: Obtain the loader's digging operation data and the material pile surface information, and construct a physical theoretical control model for the loader's autonomous digging process. The digging operation data includes: current state parameter s t and excavation control parameter a t And according to the current state parameter s t and excavation control parameter a t The optimal digging trajectory and the corresponding theoretical digging performance are obtained through the physical theory control model. and theoretical digging performance error Where t represents the current time step; the physical theory control model includes: kinematic model, dynamic model, bucket force model, bucket-material coupling model, digging resistance simulation model, and digging trajectory optimization model; Step S2, using the current state parameter s t , Excavation control parameters a t and theoretical digging performance error Using this as input, a data compensation model for digging performance error based on Transformer and Hybrid Density Network (MDN) is constructed to predict the output digging performance error. and the state parameters at the next moment and will Compared with theoretical digging performance Error compensation is performed to obtain the corrected digging performance. Thus, a world model is obtained based on the excavation performance error data compensation model and the physical theory control model. Step S3: Based on the world model, construct an autonomous digging agent for the loader using a reinforcement learning algorithm, and train the autonomous digging agent for the loader; the agent uses the state parameters of the next time step. As input, output the shoveling control parameters for the next moment. The training of the loader's autonomous digging agent includes: constructing a reward function based on the prediction results of the world model, and driving the agent's policy network to update with the goal of maximizing the cumulative reward, until the policy converges, thereby obtaining an autonomous digging controller. Step S4: Deploy the autonomous digging controller in the actual digging operation environment and output digging control commands to the loader actuator.
2. The control method for the autonomous digging device of a loader according to claim 1, characterized in that, Step S1 includes: The surface information of the material pile before the digging operation is obtained by using binocular vision and / or lidar. The depth information of lidar is matched and fused with the color and texture information of the image to construct a three-dimensional model of the material pile surface. Material characteristic parameters are obtained, and kinematic, dynamic, bucket force, bucket-material coupling, and digging resistance simulation models of the loader's digging process are constructed using the digging operation data and material characteristic parameters, respectively, to simulate and output the digging trajectory and its corresponding digging resistance. The bucket filling rate is calculated based on the shovel trajectory and the three-dimensional model of the material pile surface. And calculate the digging efficiency based on the digging operation time. ; By integrating the three-dimensional information of the material pile surface and the generated digging trajectory information, a digging trajectory optimization model is constructed to output the optimal digging trajectory and its corresponding theoretical digging performance. The theoretical digging performance Due to the excavation resistance Digging efficiency and shovel full bucket rate The results were obtained through comprehensive calculations.
3. The control method for the autonomous digging device of a loader according to claim 2, characterized in that, Calculate the theoretical digging performance The steps include: The digging resistance was respectively... Full rate and digging efficiency Normalize the mapping to the [0, 1] interval; Obtained by linear weighted summation : , in, For the weighting coefficients, satisfying , and They are respectively The maximum and minimum values; and They are respectively The maximum and minimum values; and They are respectively The maximum and minimum values.
4. The control method for the autonomous digging device of a loader according to claim 1, characterized in that: , in, , These are the pressure signals of the large and small cylinders of the boom's autonomous digging device for the loader; , These are the pressure signals of the large and small chamber hydraulic cylinders of the loader's autonomous digging device; , , These are the displacement signals of the boom cylinder, bucket cylinder, and vehicle forward displacement of the loader's autonomous digging device. This is the torque signal for the drive shaft; , These are the horizontal and vertical material penetration depths of the loader's autonomous digging device.
5. The control method for the autonomous digging device of a loader according to claim 1, characterized in that: A data compensation model for digging performance errors is constructed using a 3-layer Transformer and a 3-layer hybrid density network (MDN). The Transformer is used to capture the long-range dependencies of the time series data of the digging process, and the MDN is used to output the probability distribution of digging performance errors. The current state parameters are then used... Excavation control parameters and simulation performance error A time-series dataset is constructed as input to the digging performance error data compensation model, and the digging performance error is predicted as output. and the state parameters at the next moment ; The world model is expressed in probability density form as follows: , Where T represents the total number of time steps in the complete time series of the digging process, h t This represents the hidden layer state parameters of the Transformer model at time t; The training loss function of the world model is: ,in, This indicates the excavation process at the end of the complete time series. The digging performance is adjusted in real time.
6. The control method for the autonomous digging device of a loader as described in claim 1, characterized in that, The steps of constructing a reward function based on the prediction results of the world model, and driving the policy network update of the agent with the goal of maximizing the cumulative reward until the policy converges, to obtain an autonomous digging controller include: The agent is constructed using the Actor-Critic algorithm in reinforcement learning; the agent includes a policy network and a value network. During model training, the policy network generates initial digging control parameters based on the current state parameters. The parameters are then input into the world model to predict the state parameters for the next time step. Model parameters and the corresponding modified digging performance ; A reward function is constructed based on the prediction results, input into the value network to guide optimization, and the policy network is updated using the policy gradient method to minimize the temporal difference error in updating the value network. The policy network and the value network are alternately updated and iterated until the network converges, thus obtaining the autonomous digging controller.
7. The control method for the autonomous digging device of a loader according to claim 6, characterized in that, The final reward is constructed by combining dense and sparse rewards. The dense reward is: at each time step Calculate real-time progress rewards ,in To achieve the maximum digging performance, The initial digging performance; the sparse reward is: when the digging process is completed... After a certain time step, a terminal reward will be given. The final reward The weighted sum of the dense reward and the sparse reward: ,in, The weighting coefficient represents the weighting percentage of the intensive reward in the overall reward, and its preferred value range is 0.6 to 0.
9.
8. A control device, characterized in that, It includes a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the control method for the loader's autonomous digging device as described in any one of claims 1 to 7.
9. A control system for an autonomous digging device of a loader, characterized in that, include: The multi-source sensor data acquisition module is used to collect multi-source parameter data of the loader during the digging operation; The multi-source parameter data includes: hydraulic cylinder pressure signals, hydraulic cylinder displacement signals, vehicle forward displacement signals, and drive shaft torque signals of the loader's autonomous digging device, as well as point cloud data of the material pile surface perceived by binocular vision and / or lidar. The physical mechanism modeling module is used to construct a physical theoretical control model of the loader's digging process based on the multi-source parameter data. This physical theoretical control model includes: a kinematic model, a dynamic model, a bucket force model, a bucket-material coupling model, a digging resistance simulation model, and a digging trajectory optimization model. The simulation outputs the optimal digging trajectory and the corresponding theoretical digging performance. and theoretical digging performance error ; The error data-driven compensation module has a built-in Transformer-MDN hybrid model. It is used to predict the state and shovel performance error at the next moment by taking the current state parameters, shovel control parameters and shovel performance error parameters of the shovel process as inputs, and to perform error compensation on the output of the physical mechanism modeling module, so as to construct a world model of the shovel performance error data compensation model and the physical theory control model. The control strategy optimization module uses a reinforcement learning algorithm to construct an autonomous digging agent for the loader, uses the world model to iteratively optimize and train the strategy, and outputs the optimal digging control strategy. The execution module is used to deploy the trained autonomous digging agent in the vehicle-mounted controller of the actual digging operation environment, receive real-time sensor data and output digging control commands to the loader's execution mechanism.
10. A loader, characterized in that, Includes the control device as described in claim 8.
