An excavator control method and control system for unstructured working conditions

Abstract

This application provides an excavator control method and control system for unstructured working conditions. For the excavator control method and control system for unstructured working conditions provided by this application, on the one hand, in view of the dynamic reshaping characteristics of the soil during the excavation process, the excavation action trajectory is planned through the soil state acquisition model and the excavation action planning model, which can adapt to unstructured working conditions and ensure stable operation effects; on the other hand, through real-time perception and data feedback, the excavation action trajectory can be adjusted in real time during the operation and optimized according to the soil state, which can improve the flexibility and effect of the operation; on the third hand, by autonomously planning the excavation action trajectory, the dependence on manual operators can be reduced, the need for manual intervention can be reduced, and the autonomous operation ability can be improved; on the fourth hand, by optimizing the excavation action trajectory through the excavation action planning model, the energy consumption during the excavation process can be reduced, the operation efficiency can be improved, and the operation cost can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of excavators, and in particular, to an excavator control method and a control system for unstructured working conditions. Background Art

[0002] Most of the existing automated operation methods decompose tasks step by step on the premise of making a structured assumption about the environment. However, in actual operations, the working conditions often exhibit unstructured characteristics. For example, in the continuous earthwork excavation link, during the interactive process between the bucket and the plastic environment of the earthwork, the dynamic reshaping characteristics shown by the earthwork often do not change according to the ideal situation. In this way, on the one hand, it may lead to a decrease in the accuracy of the excavation forming surface; on the other hand, during the excavation process, it is difficult to accurately control the filling degree of the bucket. Further, the inaccuracy in controlling the full bucket rate will lead to an increase in excavation energy consumption, resulting in a deterioration of the autonomous operation effect of the task. Summary of the Invention

[0003] In view of this, the present application provides an excavator control method and a control system for unstructured working conditions to solve the problems of the decrease in the accuracy of the excavation forming surface and the increase in excavation energy consumption caused by the dynamic reshaping characteristics of the earthwork, improve the autonomous operation effect of the task, improve the operation efficiency, and reduce the operation cost.

[0004] Specifically, the present application is implemented through the following technical solutions:

[0005] The first aspect of the present application provides an excavator control method, and the method includes:

[0006] The sensing device continuously collects the environmental state of the working area of the excavator and the self-state of the excavator, and feeds back the environmental state and the self-state to the client;

[0007] The client models the excavator and the working area based on the environmental state and the self-state, and displays the obtained modeling result to the user;

[0008] The client responds to the operation of the user to set an operation task based on the modeling result, and feeds back the operation task to the control device;

[0009] When receiving the operation task, the control device obtains the current dense point cloud data of the working area from the sensing device;

[0010] The control device inputs the dense point cloud data and the excavation action trajectory of the previous moment into the earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein, the current earthwork state includes a description of the state of the earthwork and its transition;

[0011] The control device inputs the current earthwork state into the excavation action planning model to plan the excavation action trajectory in the current earthwork state by the excavation action planning model;

[0012] The control device controls the operation of the excavator according to the excavation action trajectory, and after the operation is completed, the step of obtaining the current dense point cloud data of the operation area from the sensing device is executed again until the operation task is completed.

[0013] A second aspect of the present application provides a control system for controlling an excavator facing unstructured working conditions. The control system includes a client, a sensing device provided on the excavator, and a control device provided on the excavator; wherein,

[0014] The sensing device is used to collect the environmental state of the operation area of the excavator and the self-state of the excavator in real time, and feed back the environmental state and the self-state to the client;

[0015] The client is used to model the excavator and the operation area based on the environmental state and the self-state, and display the obtained modeling result to the user;

[0016] The client is used to, in response to the operation of the user setting an operation task based on the modeling result, feed back the operation task to the control device;

[0017] The control device is used to, when receiving the operation task, obtain the current dense point cloud data of the operation area from the sensing device;

[0018] The control device is used to input the dense point cloud data and the excavation action trajectory of the previous moment into the earthwork state acquisition model to determine the current earthwork state by the earthwork state acquisition model according to the dense point cloud data and the excavation action trajectory; wherein, the current earthwork state includes a description of the state of the earthwork and its transfer;

[0019] The control device is used to input the current earthwork state into the excavation action planning model to plan the excavation action trajectory in the current earthwork state by the excavation action planning model;

[0020] The control device is used to control the operation of the excavator according to the excavation action trajectory, and after the operation is completed, the step of obtaining the current dense point cloud data of the operation area from the sensing device is executed again until the operation task is completed.

[0021] The excavator control method and control system provided by this application for unstructured working conditions. Firstly, aiming at the dynamic reshaping characteristics of the soil during excavation, the excavation action trajectory is planned through the soil state acquisition model and the excavation action planning model, which can adapt to unstructured working conditions and ensure stable operation effects. Secondly, through real-time perception and data feedback, the excavation action trajectory can be adjusted in real time during operation and optimized according to the soil state, which can improve the flexibility and effect of operation. Thirdly, by autonomously planning the excavation action trajectory, the dependence on manual operators can be reduced, the need for manual intervention can be lowered, and the autonomous operation ability can be improved. Fourthly, by optimizing the excavation action trajectory through the excavation action planning model, the energy consumption during excavation can be reduced, the operation efficiency can be improved, and the operation cost can be lowered.

[0022] Furthermore, for the excavator control method and control system provided by this application for unstructured working conditions, firstly, by establishing the first empirical knowledge base, the policy evaluation model can be trained based on the first empirical knowledge base, so that the policy evaluation model can accurately evaluate the value of excavation actions.

[0023] Secondly, through the trained policy evaluation model, the excavation action value of the data pair composed of the soil state and the excavation action trajectory can be accurately obtained, and then based on this, the second empirical knowledge base can be constructed, so that the second empirical knowledge base can accumulate a large amount of empirical data to cover various different excavation working conditions. In this way, subsequently, when training the excavation action planning model based on the second empirical knowledge base, not only can the generalization ability of the model be enhanced, enabling it to operate effectively in different environments and conditions, but also the model can perform autonomous learning and evolution based on the second empirical knowledge base, gradually improving the accuracy of decision-making and control, so that the excavation action planning model can more accurately plan the action trajectory in future excavation operations, improving the overall operation accuracy and effect.

[0024] Thirdly, by establishing the second empirical knowledge base, when training the excavation action planning model subsequently, the excavation action value can be used as a supervision signal to train the excavation action planning model. This can not only enable the excavation action planning model to quickly identify and learn the optimal excavation action trajectory, reduce the training time, and improve the learning efficiency, but also guide the model training through the supervision signal, optimize the operation path of the excavator, and improve the overall operation effect and efficiency. In addition, through continuous feedback and optimization, the model can continuously improve the excavation action planning ability, reduce the dependence on manual intervention, and improve the autonomous operation level of the excavator. Description of the Drawings

[0025] Figure 1 It is a flowchart of the first embodiment of the excavator control method provided by this application for unstructured working conditions;

[0026] Figure 2 Schematic diagram of a control system shown in an exemplary embodiment of the present application;

[0027] Figure 3 Schematic diagram of a modeling result shown in an exemplary embodiment of the present application;

[0028] Figure 4 Schematic diagram of the implementation principle of setting the excavation starting coordinates shown in an exemplary embodiment of the present application;

[0029] Figure 5 Flowchart of the second embodiment of the excavator control method for unstructured working conditions provided by the present application;

[0030] Figure 6 Flowchart of the third embodiment of the excavator control method for unstructured working conditions provided by the present application;

[0031] Figure 7 Schematic diagram of a strategy evaluation model shown in an exemplary embodiment of the present application. Detailed Description of the Invention

[0032] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application.

[0033] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms "a", "the" and "said" used in the present application are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0035] Specific embodiments are given below to introduce the technical solutions of the present application in detail.

[0036] Figure 1 Flowchart of the first embodiment of the excavator control method for unstructured working conditions provided by the present application. Please refer toFigure 1 , the method provided in this embodiment is applied to a control system; the control system includes a client, a sensing device disposed on an excavator, and a control device disposed on the excavator; the method includes:

[0037] S101. The sensing device collects the environmental state of the working area of the excavator and the self-state of the excavator in real time, and feeds back the environmental state and the self-state to the client.

[0038] Specifically, Figure 2 is a schematic diagram of the control system shown in an exemplary embodiment of the present application. Please refer to Figure 2 , the control system is used to control the excavator, and the control system may include a client, a sensing device, and a control device.

[0039] The client may be other devices independent of the excavator or devices disposed on the excavator. In this embodiment, it is not limited. For example, in a possible implementation, the client may be a mobile phone, a computer, etc. independent of the excavator; for another example, in another possible implementation, the client may be an on-vehicle terminal disposed on the excavator.

[0040] The sensing device is used to collect the environmental state of the working area of the excavator and the self-state of the excavator in real time. The sensing device may include a binocular camera disposed on the excavator (collecting the environmental state through the binocular camera) and a sensor group disposed on the excavator (collecting the self-state of the excavator through the sensor group). It should be noted that the sensor group may include a lidar, a pose sensor, an inclination sensor, a displacement sensor, a rotation speed sensor, a pressure sensor, etc. In addition, it can be understood that the control device may be a controller disposed on the excavator.

[0041] Specifically, the environmental state of the working area generally refers to the external conditions and factors of the working area where the excavator is located, which may include topographic and geomorphic information, obstacle information, the surrounding environment of the working area, etc. For example, in a possible implementation, the environmental state is characterized by dense point cloud data.

[0042] Furthermore, the self-state of the excavator refers to the attributes and states of the excavator itself, which may include position, attitude, and mechanical structure state, etc.

[0043] It should be noted that the sensing device collects the environmental state and the self-state in real time, and feeds back the collected environmental state and self-state to the client.

[0044] S102. The client models the excavator and the working area based on the environmental state and the self-state, and displays the obtained modeling result to the user.

[0045] Optionally, in one possible implementation, the modeling can be performed in the following manner:

[0046] (1) Preprocess the collected data (refer to the previous description, the collected data includes environmental status and its own status) to clean, calibrate, and align the data, and remove outliers from the data to ensure the accuracy and consistency of the data.

[0047] (2) Extract modeling features from the processed data; among them, the modeling features may include the action features of the excavator (such as digging depth, steering angle, etc.), the terrain features of the environment (such as slope, obstacle position, etc.), the soil features (such as hardness, humidity, etc.), etc.

[0048] (3) Based on the modeling method, establish a model of the excavator and the working area according to the extracted modeling features to obtain the modeling result.

[0049] Specifically, Figure 3 is a schematic diagram of the modeling result shown in an exemplary embodiment of the present application. Please refer to Figure 3 , through modeling, the modeling result can be displayed to the user (the user may be an operator) through a graphical user interface, so that the user can understand the environmental information of the current working area and the real-time status of the excavator based on the modeling result, so that the operator can perform operation planning and set operation tasks based on this.

[0050] S103. The client responds to the operation of the user setting an operation task based on the modeling result, and feeds back the operation task to the control device.

[0051] Specifically, refer to the previous description. After the client performs modeling and obtains the modeling result, the modeling result can be displayed on the interface, and the operator can set an operation task based on the displayed modeling result.

[0052] In specific implementation, when the operator sets an operation task, the task type and the task parameters corresponding to the task type can be set. Specifically, the task type may include a leveling task, a foundation pit task, a ditch task, etc. It should be noted that the task parameters corresponding to different task types are different. For example, when the task type is a leveling task, Table 1 shows the task parameters corresponding to the leveling task in an exemplary embodiment of the present application. Referring to Table 1, the task parameters corresponding to the leveling task may include the starting coordinate, the ending angle, the trajectory type, the cycle time, the maximum leveling angle, etc.

[0053] Table 1 Task parameters corresponding to the leveling task

[0054] Type Parameter Quantity float Excavation starting coordinate 3 float Termination angle 1 usint Trajectory type 1 float Time 1 float Maximum leveling angle 1

[0055] For another example, Table 2 shows the task parameters corresponding to the foundation pit task in an exemplary embodiment of the present application. Referring to Table 2, when the task type is the foundation pit task, the corresponding task parameters include the starting coordinate for excavation, cutting angle, trajectory type, cycle time, cutting-in angle (trimming angle), foundation pit depth, foundation pit length, foundation pit width, etc.

[0056] Table 2 Task Parameters Corresponding to Foundation Pit Task

[0057] Type Parameter Quantity float Excavation starting coordinate 3 float Cutting angle 1 usint Trajectory type 1 float Cycle time 1 float Cutting-in angle (trimming angle) 1 float Foundation pit depth 1 float Foundation pit length 1 float Foundation pit width 1

[0058] It should be noted that when setting the starting coordinate for excavation, it can be set through graphical interaction or parameter interaction. Figure 4 The schematic diagram of the implementation for setting the starting coordinate for excavation shown in an exemplary embodiment of the present application. Please refer to Figure 4 , when setting the starting coordinate for excavation, through graphical interaction, the starting and ending points of excavation can be selected on the map in the modeling result by using the mouse. It should be noted that the image interaction method can be used in the occasion of manual grid division of the terrain. Further, please continue to refer to Figure 4 , when setting the starting coordinate for excavation, it can also be set through parameter interaction by inputting the coordinates in the pre-determined world coordinate system and inputting the starting coordinate for excavation. It should be noted that the parameter interaction method is used in the occasion where digital modeling and coordinate planning have been carried out for a large range of terrain.

[0059] S104. When the control device receives the operation task, it obtains the current dense point cloud data of the operation area from the sensing device.

[0060] Referring to the previous description, the sensing device includes a binocular camera, and the binocular camera will obtain the dense point cloud data of the operation area in real time. When the control device receives the operation task, it will obtain the current dense point cloud data of the operation area from the sensing device, and then plan the excavation action trajectory based on the dense point cloud data, and then convert the excavation action trajectory into an electrical signal, and control the excavator based on the electrical signal to enable the excavator to complete the operation task.

[0061] It should be noted that the dense point cloud data refers to a large number of discrete point data collected in three-dimensional space, and these points densely cover the surface of the object or the surface of the scene. Therefore, detailed geometric shape and texture information of the operation area can be captured through the dense point cloud data.

[0062] S105. The control device inputs the dense point cloud data and the excavation action trajectory at the previous moment into the soil state acquisition model, so that the soil state acquisition model determines the current soil state according to the dense point cloud data and the excavation action trajectory; wherein, the current soil state includes the description of the state of the soil and its transition.

[0063] Specifically, the earthwork state acquisition model is a model used to acquire the description (parametric representation) of the state and transfer of earthwork. The function of this model is to provide a representation form that can be used to understand and predict the change of earthwork state through the analysis and processing of input data. The input of the earthwork state acquisition model is dense point cloud data and the excavation action trajectory of the previous moment, and the output is the current earthwork state. It should be noted that the output of this model is specifically the parametric representation of the state and transfer of earthwork.

[0064] It can be understood that initially, if there is no excavation action trajectory of the previous moment, the earthwork state acquisition model directly outputs the current earthwork state based on the dense point cloud data.

[0065] It should be noted that in theory, the earthwork state matrix can be directly obtained based on the dense point cloud data to realize the observation of the earthwork state. However, in actual construction, due to the deformation problem of the earthwork, the near-end area may be blocked by the earthwork under the binocular camera view, resulting in blank sensing information. Therefore, simply relying on the dense point cloud data will not be able to accurately obtain the earthwork state.

[0066] For this reason, in this application, the earthwork state is acquired based on the dense point cloud data and the excavation action trajectory of the previous moment to accurately obtain the earthwork state.

[0067] The specific implementation process and implementation principle of the earthwork state acquisition model for acquiring the earthwork state will be introduced in detail in the following embodiments and will not be elaborated here.

[0068] S106. The control device inputs the current earthwork state into the excavation action planning model so that the excavation action planning model plans the excavation action trajectory in the current earthwork state.

[0069] It should be noted that the excavation action planning model is a model used to determine and optimize the operation path and action sequence of the excavator when performing the excavation task. By analyzing the current earthwork state, the excavation action planning model can generate the optimal excavation action trajectory. Planning the excavation action trajectory based on the earthwork state through this excavation action planning model can improve work efficiency, reduce energy consumption and ensure the safety of the operation.

[0070] It can be understood that the excavation action trajectory includes an action trajectory composed of multiple points and a reference point for the excavation position.

[0071] In specific implementation, the current earthwork state (the current earthwork state is identified in the form of a matrix) is dimension-reduced and transformed into a vector input through state encoding, thereby avoiding the convolution operation of the network and reducing the complexity of the state.

[0072] It should be noted that the core part of the excavation action planning model is two deep policy networks, which are used to approximate the policy of continuous actions. After receiving the soil state, it predicts the next excavation action trajectory according to the current action output. In addition, since the excavation action is a real-valued vector, in actual use, the output layer size of the excavation action planning model is set to the action dimension to match the multi-dimensional action space.

[0073] S107. The control device controls the excavator to operate according to the excavation action trajectory, and after the operation is completed, it executes again the step of obtaining the current dense point cloud data of the operation area from the sensing device until the operation task is completed.

[0074] Specifically, after obtaining the excavation action trajectory, the control device controls the excavator to operate according to the excavation action trajectory, and after the operation is completed, it repeats the above steps to indicate that the operation task is completed.

[0075] For the method provided in this embodiment, on the one hand, aiming at the dynamic reshaping characteristics of the soil during excavation, the excavation action trajectory is planned through the soil state acquisition model and the excavation action planning model, which can adapt to unstructured working conditions and ensure stable operation effects; on the other hand, through real-time perception and data feedback, the excavation action trajectory can be adjusted in real time during the operation and optimized according to the soil state, which can improve the flexibility and effect of the operation; on the third hand, by autonomously planning the excavation action trajectory, the dependence on manual operators can be reduced, the need for manual intervention can be reduced, and the ability of autonomous operation can be improved; on the fourth hand, by optimizing the excavation action trajectory through the excavation action planning model, the energy consumption during excavation can be reduced, the operation efficiency can be improved, and the operation cost can be reduced.

[0076] Figure 5 This is the flowchart of the second embodiment of the excavator control method for unstructured working conditions provided by this application. Please refer to Figure 5 , for the method provided in this embodiment, on the basis of the above embodiment, the determining the current soil state according to the dense point cloud data and the excavation action trajectory includes:

[0077] S501. For any dense point cloud data, clip the dense point cloud data according to the boundary constraint conditions indicated by the operation task to obtain target point cloud data.

[0078] In specific implementation, in a possible implementation manner, through the boundary constraint conditions, any dense point cloud data in the dense point cloud data can be clipped according to the following formula:

[0079] ,

[0080] where is the initial coordinate of the constraint parameter boundary, is the boundary length, Width is the boundary width, and Deepth is the boundary depth.

[0081] S502. According to the target point cloud data, parameterize the earthwork state based on the voxel method to obtain an earthwork state matrix.

[0082] Specifically, the voxel method divides the three-dimensional space into uniform voxels, and then describes the earthwork state according to the point cloud density or other attributes within each voxel.

[0083] When specifically implemented, the earthwork state matrix can be obtained according to the following method:

[0084] (1) Voxelization: Divide the three-dimensional space into a uniform voxel grid. Each voxel represents a discrete area.

[0085] (2) Point cloud data conversion: Map the target point cloud data onto the voxel grid. According to the distribution of the target point cloud data in each voxel, the number of point clouds within each voxel can be calculated.

[0086] (3) Parameterize the earthwork state: Parameterize the earthwork state according to the number of point clouds within the voxel.

[0087] (4) Construct the earthwork state matrix: Organize the parameterized earthwork state into a matrix. Among them, each row or each element of the matrix corresponds to a voxel, and the columns correspond to different features or attributes.

[0088] When specifically implemented, referring to the previous description, the target point cloud data can be projected onto the voxel matrix, the earthwork surface can be estimated through the filling rate, and it is reasonably assumed that the target area under the earthwork surface is also an earthwork entity. Then, the earthwork state matrix parameterized based on the voxel method can be constructed by the following formula: ,

[0089] Among them, is the earthwork state matrix, is the number of point clouds under the corresponding voxel, is the judgment threshold.

[0090] Specifically, if the number of point clouds under the corresponding voxel is greater than or equal to the judgment threshold, it is determined that the earthwork is an entity, and the corresponding matrix element is set to 1; if the number of point clouds under the corresponding voxel is less than the judgment threshold, it is determined that the earthwork is a non-entity, and the corresponding matrix element is set to 0.

[0091] S503. Define a two-dimensional trajectory matrix according to the operation task, and map each trajectory point in the excavation action trajectory into the two-dimensional trajectory matrix to obtain the target two-dimensional trajectory matrix corresponding to the excavation action trajectory.

[0092] In specific implementation, the bucket trajectory can be processed on the x-z plane of the excavation robot reference system, and the trajectory sequence in the joint space can be calculated through the oil cylinder motion information collected in each excavation cycle. Then the excavation action trajectory at the previous moment (i.e., the excavation action trajectory in this embodiment) can be calculated based on the kinematic formula of the excavation robot.

[0093] Specifically, in a possible implementation manner, the specific implementation process of this step may include:

[0094] (1) Define the two-dimensional trajectory matrix as where the is the excavation length of the operation task; the is the task accuracy of the operation task.

[0095] (2) Traverse the trajectory points in the excavation action trajectory and map the currently traversed trajectory point into the two-dimensional trajectory matrix according to the following formula to obtain the target two-dimensional trajectory matrix:

[0096] ,

[0097] ,

[0098] where is the constraint parameter, is the length of the excavation action trajectory, is the number of trajectory points in the th interval, is the target area boundary margin.

[0099] It should be noted that during the mapping process, the trajectory part in the earthwork interaction stage should be strictly segmented to prevent the influence of the aerial motion trajectory points on the actual excavation trajectory calculation.

[0100] S504. Fill the blank areas in the target two-dimensional trajectory matrix through an interpolation method to obtain a complete spatial trajectory matrix.

[0101] Specifically, since the target two-dimensional trajectory matrix is non-uniform and there are blanks in some areas of the target two-dimensional trajectory matrix, interpolation is required to fill the missing blank areas to obtain a complete spatial trajectory matrix.

[0102] In specific implementation, in order to reasonably extrapolate while maintaining the accuracy of the original data, a linear interpolation method is used to fill the blank area in the target two-dimensional trajectory matrix to obtain a complete spatial trajectory matrix.

[0103] It should be explained that the linear interpolation method estimates between known data points by assuming that the change between data points is linear, that is, the value of the unknown point between two known points changes along a straight line.

[0104] S505. Expand the spatial trajectory matrix in the width direction according to the density of the dense point cloud data to obtain a sequence of trajectory matrices.

[0105] It should be explained that in order to fuse with the earthwork state matrix obtained based on the dense point cloud data, so as to realize the construction of the excavation trajectory surface of the target area with point cloud accuracy, the spatial trajectory matrix is expanded in the width direction according to the density of the dense point cloud to obtain a sequence of trajectory matrices.

[0106] S506. Determine the current earthwork state according to the earthwork state matrix and the sequence of trajectory matrices.

[0107] In specific implementation, by traversing the sequence of trajectory matrices, it is fused with the earthwork state matrix obtained based on the dense point cloud data to obtain the final earthwork state. In specific implementation, the fusion can be carried out according to the following formula:

[0108] ,

[0109] where, is the current earthwork state;

[0110] is the sequence of trajectory matrices The element value at the index position;

[0111] is the earthwork state matrix at the index position The element value of;

[0112] is an undefined or non-representable numerical value.

[0113] It should be noted that in order to maximize the preservation of the original terrain, in the specific fusion process, only in the target area where the surface data is missing (that is, the area), the sequence of trajectory matrices and the earthwork state matrix are fused and constructed. At the same time, introduce To establish the final earthwork state.

[0114] The method provided in this embodiment presents a method for determining the earthwork state. This method is based on dense point cloud data and the excavation action trajectory at the previous moment to obtain the earthwork state, which can avoid the problem that the sensing information may be blank due to the possible occlusion of the earthwork in the proximal area from the perspective of binocular cameras, and accurately obtain the earthwork state.

[0115] Figure 6 It is a flowchart of the third embodiment of the excavator control method for unstructured working conditions provided in this application. Please refer to Figure 6 In the method provided in this embodiment, on the basis of the above embodiment, the method further includes:

[0116] S601. After the control device controls the excavator to operate according to the excavation action trajectory, it obtains the effect parameters of the excavator during this operation.

[0117] Specifically, the effect parameters include, but are not limited to, operation time, fluctuation range of excavation force, energy loss, friction loss of the working arm system, and bucket fill rate, etc.; hereinafter, the effect parameters including operation time, energy loss, friction loss of the working arm coefficient, and bucket fill rate are taken as examples for illustration.

[0118] Specifically, the energy loss is the total energy consumed by the excavator during operation, which can reflect the fuel or power usage efficiency of the excavator; the friction loss of the working arm system refers to: during the movement of the working arm system of the excavator (including the boom, bucket, etc.), the energy loss caused by friction, which has a certain impact on the working efficiency and service life of the excavator; the bucket fill rate refers to the degree of filling of the excavator bucket during one excavation action, which can be used to measure the loading efficiency of the excavator.

[0119] In specific implementation, during the process of the excavator executing the above excavation action trajectory, the power consumption of the excavator during this period can be obtained, and then the energy consumption loss of the excavator can be calculated by using the power consumption of the excavator during this period. Further, when obtaining the friction loss of the working arm system, the friction loss of the working arm system can be estimated based on the temperature rise of the hydraulic cylinder lubricating oil. In addition, when obtaining the bucket fill rate, the actual loading amount of the bucket and the maximum capacity of the bucket can be obtained, and then the bucket fill rate can be calculated based on the actual loading amount and the maximum capacity of the bucket.

[0120] S602. The control device evaluates the reward feedback value of the excavation action trajectory according to the effect parameters; wherein, the magnitude of the reward feedback value of the excavation action trajectory represents the quality of taking the excavation action trajectory under the current earthwork state.

[0121] Specifically, when calculating the reward feedback value, factors such as operation time, energy consumption, total joint travel, and full bucket rate need to be considered simultaneously. These factors are those affecting the performance of earthwork operations. In actual calculations, the task objective is expressed as completing the established operation task at the lowest fuel consumption and system loss cost within the shortest time.

[0122] In summary, in one possible implementation, the reward feedback value of the excavation action trajectory can be calculated according to the following formula:

[0123] ,

[0124] wherein, the is the energy loss;

[0125] the is the weight coefficient of the energy loss;

[0126] the is the friction loss of the working arm system;

[0127] the is the weight coefficient of the friction loss of the working arm system;

[0128] the is the full bucket rate;

[0129] the is the weight coefficient of the full bucket rate;

[0130] the is the operation time;

[0131] the is the weight coefficient of the operation time.

[0132] It should be noted that the weight coefficients of each effect parameter are set according to actual needs and are not limited in this embodiment. For example, they can be fitted based on the habits of skilled operators and construction experience. In addition, in one possible implementation, the weight coefficients of each effect parameter can be determined according to specific application objectives, operating costs, and equipment characteristics.

[0133] Specifically, for example, in one possible implementation, in a scenario where energy efficiency is emphasized, energy consumption should be minimized to improve fuel efficiency. At this time, when selecting the weight coefficients, 𝑤1 should be set relatively large, and 𝑤2 and 𝑤3 should be set relatively small. For example, in one embodiment, let 𝑤1 be equal to 0.6, let 𝑤2 be equal to 0.2, and let 𝑤3 be equal to 0.2.

[0134] It should be noted that the magnitude of the reward feedback value of the excavation motion trajectory represents the quality of adopting the excavation motion trajectory under the current soil condition. That is, the larger the reward feedback value of the excavation motion trajectory, the more appropriate it is to adopt the excavation motion trajectory under the current soil condition, which is beneficial to reducing energy loss and friction loss of the working arm system and improving the full bucket rate; the smaller the reward feedback value of the excavation motion trajectory, the less appropriate it is to adopt the excavation motion trajectory under the current soil condition, which is not conducive to reducing energy loss and friction loss of the working arm system and improving the full bucket rate.

[0135] S603. The control device determines the effect of the excavation motion trajectory on the environment and the degree of completion of the excavation motion trajectory for the operation task; wherein, the effect is characterized by the soil condition after the operation of executing the excavation motion trajectory; the degree of completion is determined based on the soil condition after the operation and the task parameters of the operation task.

[0136] Specifically, the effect of the excavation motion trajectory on the environment refers to the impact on the surrounding natural environment and ecosystem after the excavator executes the excavation motion trajectory. In a possible implementation manner, the effect of the excavation motion trajectory on the environment is characterized by the soil condition after the excavator executes the excavation motion trajectory (for the convenience of distinction, this soil condition is denoted as the soil condition after the operation). In other words, for example, at time t, the excavator executes the excavation motion trajectory at this moment. After executing this motion trajectory, S t+1 can be used to characterize the effect of the excavation motion trajectory on the environment, where S t+1 is the soil condition at time t + 1.

[0137] In specific implementation, after the excavator executes the excavation motion trajectory at time t, it can further obtain the dense point cloud data at time t + 1. Refer to Figure 5 for the description. Based on the dense point cloud data at time t + 1 and the excavation motion trajectory at time t, and based on the soil condition acquisition model, the soil condition at time t + 1 can be obtained.

[0138] Specifically, the degree of completion of the excavation motion trajectory for the operation task is determined based on the soil condition after the operation and the task parameters of the operation task. Specifically, the soil condition after the operation can be compared with the task parameters of the operation task, and the degree of completion of the excavation motion trajectory for the operation task can be determined based on the comparison result. For example, in an embodiment, the operation task requires excavating a foundation pit with a depth of 3 meters in the operation area. After executing the excavation motion trajectory, based on the soil condition after the operation, it is determined that the depth of the foundation pit after executing the excavation motion trajectory is only 2.5 meters. At this time, it can be determined that the degree of completion of this excavation motion trajectory for the operation task is 85%.

[0139] S604. The control device stores the current earthwork state, the excavation action trajectory, the reward feedback value of the excavation action trajectory, the action effect, and the completion degree as an experience in the first experience knowledge base.

[0140] Specifically, for example, in one embodiment, for the convenience of description, the current earthwork state is denoted as The excavation action trajectory is denoted as The reward feedback value of this excavation action trajectory is denoted as The action effect of this excavation action trajectory on the environment is denoted as The completion degree of this excavation action trajectory for the operation task is denoted as After executing this excavation trajectory action, As an experience It is stored in the first experience knowledge base.

[0141] It can be understood that the first experience knowledge base includes multiple experiences, and the format of each experience is as follows:

[0142] .

[0143] S605. The control device uses the experiences in the first experience knowledge base to train a policy evaluation model, and obtains a trained policy evaluation model; wherein, the trained policy evaluation model is used to estimate the excavation action value of taking this excavation action trajectory in this earthwork state according to the data composed of the earthwork state and the excavation action trajectory; wherein, the excavation action value characterizes the quality of this excavation action trajectory.

[0144] Specifically, the trained policy evaluation model is used to estimate the excavation action value of taking this excavation action trajectory in this earthwork state according to the data composed of the earthwork state and the excavation action trajectory. Its input is the data pair composed of the earthwork state and the excavation action trajectory, and the output is the excavation action value. It should be noted that the output excavation action value characterizes the quality of taking this excavation action trajectory in this earthwork state. For example, the larger the excavation action value, the better it means to take this excavation action trajectory in this earthwork state, and the smaller the excavation action value, the worse it means to take this excavation action trajectory in this earthwork state.

[0145] Optionally, in one possible implementation manner, Figure 7 This is a schematic diagram of the policy evaluation model shown in an exemplary embodiment of the present application. Please refer to Figure 7 , in one possible implementation manner, the policy evaluation model includes an encoder, a dual evaluation network, and a dual objective evaluation network; wherein,

[0146] The encoder is used to encode the input data pair to obtain an encoded matrix;

[0147] The double evaluation network is used to estimate the excavation action value of the excavation action trajectory according to the encoding matrix;

[0148] The double-objective evaluation network is used to determine the evaluation value of the excavation action trajectory according to the encoding matrix, so as to train the double evaluation network based on the evaluation value.

[0149] Specifically, the encoder combines the input earthwork state and the excavation action trajectory. Further, the double evaluation network and the double-objective evaluation network established based on the double Q-value estimation technology receive the input data pair and output the estimated value of the corresponding excavation action value.

[0150] Optionally, in a possible implementation manner, the step of training the policy evaluation model by using the experience in the first experience knowledge base to obtain the trained policy evaluation model includes:

[0151] (1) For each piece of experience in the first experience knowledge base, calculate the temporal difference TD error and the policy proximity coefficient of this piece of experience, and calculate the weight of this piece of experience according to the TD error and the policy proximity coefficient of this piece of experience.

[0152] Specifically, for a certain piece of experience, the weight of this piece of experience can be calculated according to the following formula:

[0153] Calculate the weight of this piece of experience according to the following formula:

[0154] ,

[0155] where, the is the weight of the i-th piece of experience;

[0156] the is the TD error of the i-th piece of experience;

[0157] the is the gradient sampling intensity coefficient;

[0158] the is the policy proximity coefficient.

[0159] For the specific implementation principle and implementation process of calculating the TD error of a piece of experience, reference can be made to the description in the related technology, which will not be elaborated here.

[0160] (2) Sampling is performed according to the specified sampling method to obtain sampling data; where, the specified sampling method is the is the probability density function constructed by using the weights of each piece of experience in the first experience knowledge base.

[0161] In specific implementation, for each piece of experience, the weight of the piece of experience is used as the probability of being collected during the sampling process, and then based on the first experience knowledge base, the probability density is constructed using the weights of each piece of experience. Then, samples are drawn from the probability density function P(t) through a sampling method to obtain sampling data. .

[0162] (3) Use the sampling data to train the policy evaluation model.

[0163] In specific implementation, after obtaining the sampling data, the sampling data can be used to train the policy evaluation model.

[0164] Specifically, the objective function of the dual evaluation network in the policy evaluation model is:

[0165] ,

[0166] ,

[0167] where, is the current state; is the mining action in the previous state; is the parameter of the network; is the value function; is the batch processing scale; is the range of the m nearest pieces of experience; is the distribution function using

[0168]

[0169] ,

[0170] where, is the parameter of the i-th target network.

[0171] S606. The control device constructs a data pair composed of an earthwork state and a mining action trajectory using one piece of experience in the first experience knowledge base, and inputs the data pair into the trained policy evaluation model, so that the trained policy evaluation model outputs the mining action value of the data pair.

[0172] It should be noted that after training the policy evaluation model, a data pair composed of an earthwork state and a mining action trajectory can be used, and through the trained policy evaluation model, the mining action value of the data pair can be output, that is, the mining action value of adopting the mining action trajectory in the earthwork state can be obtained.

[0173] During specific implementation, for any piece of experience in the first experience knowledge base, such as Experience 1, select the earthwork state in Experience 1 and the excavation action trajectory to form a data pair . Input this data pair into the trained policy evaluation model. The trained policy evaluation model evaluates this data pair and finally outputs the excavation action value. For example, in one embodiment, for the sake of convenience of explanation, the excavation action value is denoted as .

[0174] S607. The control device stores a data pair and the excavation action value of this data pair as a piece of experience in the second experience knowledge base.

[0175] Combined with the above example, for example, in a possible implementation manner, is stored in the second experience knowledge base as a piece of experience.

[0176] It can be understood that the second experience knowledge base contains multiple pieces of experience. Each piece of experience consists of a data pair composed of the earthwork state and the excavation action trajectory, and the excavation action value of this data pair. By gradually accumulating the historical data of the excavation operation, it provides rich data support for the excavation action planning, making the decision-making process more scientific and efficient, so as to achieve a better operation decision.

[0177] S608. The control device uses the experience in the second experience knowledge base to train the excavation action planning model, so that the excavation action planning model learns and improves the ability of the excavation action trajectory planning.

[0178] During specific implementation, the objective function of the excavation action planning model is:[[]]

[0179] ,

[0180] where is the information entropy function; is the regularization coefficient, used to control the learning intensity; is the current state; is the excavation action in the previous state, is the parameter of the first network; is the parameter of the second network; is the value function; is the batch scale.

[0181] During specific implementation, a variable α value is adopted, and a minimum entropy constraint method is established. The α value is calculated by the dynamic programming method:[[]]

[0182] ,

[0183] Among them, is the adaptive regularization coefficient. It should be noted that referring to the previous description, it can be understood that the control device integrates two modes. One is to try to autonomously learn the strategy of a given task through operations, which is mainly carried out at the initial stage of the operation. Since there is not enough experience in the experience knowledge base, the excavation action planning model has not learned any operation strategies yet. The other is to autonomously excavate through the learned excavation action planning model for a given operation task. This mode requires the control device to carry out a large number of operation attempts and store rich experience. At this time, the learning effect of the excavation action planning model on the given task determines the performance of the excavator's autonomous operation.

[0184] For the method provided in this embodiment, on the one hand, by establishing the first experience knowledge base, the policy evaluation model can be trained based on the first experience knowledge base, so that the policy evaluation model can accurately evaluate the excavation action value.

[0185] On the other hand, through the trained policy evaluation model, the excavation action value of the data pair composed of the earthwork state and the excavation action trajectory can be accurately obtained. Then, based on this, the second experience knowledge base can be constructed, so that the second experience knowledge base can accumulate a large amount of experience data to cover various different excavation working conditions. In this way, when training the excavation action planning model based on the second experience knowledge base later, it can not only enhance the generalization ability of the model, enabling it to operate effectively in different environments and conditions, but also enable the model to perform autonomous learning and evolution based on the second experience knowledge base, gradually improving the accuracy of decision-making and control, so that the excavation action planning model can more accurately plan the action trajectory in future excavation operations, improving the overall operation accuracy and effect.

[0186] On the third hand, by establishing the second experience knowledge base, when training the excavation action planning model later, the excavation action value can be used as a supervision signal to train the excavation action planning model. This can not only enable the excavation action planning model to quickly identify and learn the optimal excavation action trajectory, reduce the training time, and improve the learning efficiency, but also guide the model training through the supervision signal, optimize the operation path of the excavator, and improve the overall operation effect and efficiency. In addition, through continuous feedback and optimization, the model can continuously improve the excavation action planning ability, reduce the dependence on manual intervention, and improve the autonomous operation level of the excavator.

[0187] Corresponding to the foregoing embodiment of a control method for an excavator facing unstructured working conditions, the present application also provides an embodiment of a control system.

[0188] Please continue to refer to Figure 2, this application also provides a control system, which is used to control an excavator facing unstructured working conditions. The control system includes a client, a sensing device installed on the excavator, and a control device installed on the excavator; among them,

[0189] The sensing device is used to collect the environmental state of the working area of the excavator and the self-state of the excavator in real time, and feedback the environmental state and the self-state to the client;

[0190] The client is used to model the excavator and the working area based on the environmental state and the self-state, and display the obtained modeling result to the user;

[0191] The client is used to, in response to the operation of the user setting a working task based on the modeling result, feedback the working task to the control device;

[0192] The control device is used to, when receiving the working task, obtain the current dense point cloud data of the working area from the sensing device;

[0193] The control device is used to input the dense point cloud data and the excavation action trajectory of the previous moment into the soil state acquisition model, so that the soil state acquisition model determines the current soil state according to the dense point cloud data and the excavation action trajectory; among them, the current soil state includes a description of the state of the soil and its transfer;

[0194] The control device is used to input the current soil state into the excavation action planning model, so that the excavation action planning model plans the excavation action trajectory in the current soil state;

[0195] The control device is used to control the operation of the excavator according to the excavation action trajectory, and after the operation is completed, execute the step of obtaining the current dense point cloud data of the working area from the sensing device again until the working task is completed.

[0196] The control system of this embodiment can be used to execute Figure 1 the steps of the method embodiment shown, and the specific implementation principle and implementation process are similar, which will not be elaborated here.

[0197] The above are only the preferred embodiments of this application, and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included within the scope of protection of this application.

Claims

1. A control method for an excavator in non-structural working conditions, characterized in that: The method is applied to the control system of the excavator control method for non-structural working conditions; The control system includes a client, a sensing device arranged on the excavator, and a control device arranged on the excavator; the method includes: The sensing device collects the environmental status of the working area of ​​the excavator and the self-status of the excavator in real time, and feeds back the environmental status and the self-status to the client; The client models the excavator and the working area based on the environmental state and the client's own state, and displays the obtained modeling results to the user; The client, in response to the user's operation of setting a job task based on the modeling result, feeds back the job task to the control device; When receiving the operation task, the control device acquires the current dense point cloud data of the operation area from the sensing device; The control device inputs the dense point cloud data and the excavation action trajectory at the last moment into an earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein the current earthwork state includes a description of the state of the earthwork and its transfer; The control device inputs the current earthwork state into an excavation motion planning model so that the excavation motion planning model plans an excavation motion trajectory under the current earthwork state; The control device controls the excavator to operate according to the excavation motion trajectory, and after the operation is completed, again executes the step of acquiring the current dense point cloud data of the operation area from the sensing device until the operation task is completed; The method further comprises: After the control device controls the excavator to operate according to the excavation motion trajectory, the control device obtains effect parameters of the excavator during the current operation; The control device evaluates the reward feedback value of the excavation action trajectory according to the effect parameter; wherein the magnitude of the reward feedback value of the excavation action trajectory represents the degree of superiority or inferiority of taking the excavation action trajectory in the current earthwork state; The control device determines the effect of the excavation action trajectory on the environment and the degree of completion of the excavation action trajectory on the work task; wherein the effect is represented by the post-work earthwork state after the excavation action trajectory is executed; and the degree of completion is determined based on the post-work earthwork state and the task parameters of the work task; The control device stores the current earthwork state, the excavation action trajectory, the reward feedback value of the excavation action trajectory, the action effect, and the completion degree as an experience in a first experience knowledge base; The control device uses the experience in the first experience knowledge base to train a strategy evaluation model to obtain a trained strategy evaluation model; wherein the trained strategy evaluation model is used to estimate the value of an excavation action of taking the excavation action trajectory under the earthwork state according to data consisting of the earthwork state and the excavation action trajectory; wherein the excavation action value represents the quality of the excavation action trajectory; The control device constructs a data pair consisting of an earthwork state and an excavation action trajectory using an experience in the first experience knowledge base, and inputs the data pair into the trained strategy evaluation model so that the trained strategy evaluation model outputs the excavation action value of the data pair; The control device stores a data pair and the mining action value of the data pair as an experience in a second experience knowledge base; The control device trains the mining motion planning model using the experience in the second experience knowledge base, so that the mining motion planning model learns and improves the ability of mining motion trajectory planning.

2. The method according to claim 1, characterized in that The determining of the current earthwork state according to the dense point cloud data and the excavation action trajectory includes: For any dense point cloud data, the dense point cloud data is cropped according to the boundary constraints indicated by the task to obtain the target point cloud data: According to the target point cloud data, the earthwork state is parameterized based on the voxel method to obtain an earthwork state matrix; A two-dimensional trajectory matrix is ​​defined according to the operation task, and each trajectory point in the mining action trajectory is mapped to the two-dimensional trajectory matrix to obtain a target two-dimensional trajectory matrix corresponding to the mining action trajectory; Filling the blank areas in the target two-dimensional trajectory matrix by an interpolation method to obtain a complete spatial trajectory matrix; Expanding the spatial trajectory matrix in a width direction according to the density of the dense point cloud data to obtain a trajectory matrix sequence; The current earthwork state is determined according to the earthwork state matrix and the trajectory matrix sequence.

3. The method according to claim 2, characterized in that A two-dimensional trajectory matrix is ​​defined according to the operation task, and each trajectory point in the mining action trajectory is mapped to the two-dimensional trajectory matrix to obtain a target two-dimensional trajectory matrix of the mining action trajectory, including: Define the two-dimensional trajectory matrix as ;in, , is the excavation length of the operation task; The task accuracy of the task; Traverse the mining action trajectory The trajectory points in , and map the currently traversed trajectory points to the two-dimensional trajectory matrix according to the following formula to obtain the target two-dimensional trajectory matrix: ， ， in, is the constraint parameter, is the length of the excavation action trajectory, For the The number of trajectory points on the interval, is the target area boundary margin.

4. The method according to claim 2, characterized in that: The determining the current earthwork state according to the earthwork state matrix and the trajectory matrix sequence comprises: ， in, is the current earthwork status; is the trajectory matrix sequence The value of the element at the index position; is the earthwork status matrix at index position The element value of An undefined or unrepresentable value.

5. The method according to claim 1, characterized in that The effect parameters include operation time, energy loss, friction loss of the working arm system and full bucket rate; the reward feedback value of the excavation action trajectory is evaluated according to the effect parameters, including: The reward feedback value of the mining action trajectory is evaluated according to the following formula: ， Among them, the is the energy loss; Said is the weight coefficient of the energy loss; Said is the friction loss of the working arm system; Said is the weight coefficient of the friction loss of the working arm system; Said is the full bucket rate; Said is the weight coefficient of the full bucket rate; Said is the time of the operation; Said is the weight coefficient of the operation time.

6. The method according to claim 1, characterized in that The strategy evaluation model includes an encoder, a dual evaluation network and a dual target evaluation network; wherein, The encoder is used to encode the input data pair to obtain a coding matrix; The dual evaluation network is used to estimate the mining action value of the mining action trajectory according to the encoding matrix; The dual-objective evaluation network is used to determine the evaluation value of the mining action trajectory according to the encoding matrix, so as to train the dual evaluation network based on the evaluation value.

7. The method according to claim 6, characterized in that The step of using the experience in the first experience knowledge base to train the strategy evaluation model to obtain a trained strategy evaluation model includes: For each experience in the first experience knowledge base, calculate the time difference TD error and the strategy proximity coefficient of the experience, and calculate the weight of the experience according to the TD error and the strategy proximity coefficient of the experience; Sampling is performed according to a specified sampling method to obtain sampling data; wherein the specified sampling method is Said A probability density function constructed by using the weights of each experience in the first experience knowledge base; The strategy evaluation model is trained using the sampled data; wherein the objective function of the dual evaluation network in the strategy evaluation model is: ， ， in, is the current state; Digging action for the previous state; For the The parameters of the network; is the value function; is the batch processing scale; is the m nearest experience ranges; For use as the distribution function of its probability density; The gradient update method of the dual evaluation network is: ， in, are the parameters of the i-th target network.

8. The method according to claim 7, characterized in that The step of calculating the weight of the experience according to the TD error of the experience and the strategy proximity coefficient includes: The weight of this experience is calculated according to the following formula: ， Among them, the is the weight of the i-th experience; Said is the TD error of the i-th experience; Said is the gradient sampling intensity coefficient; Said is the strategy proximity coefficient.

9. A control system, characterized in that: The control system is used to control an excavator facing non-structural working conditions, and the control system includes a client, a sensing device arranged on the excavator, and a control device arranged on the excavator; wherein, The sensing device is used to collect the environmental status of the working area of ​​the excavator and the self status of the excavator in real time, and feed back the environmental status and the self status to the client; The client is used to model the excavator and the working area based on the environmental state and the self-state, and present the obtained modeling results to the user; The client is configured to respond to the user's operation of setting a job task based on the modeling result and feed back the job task to the control device; The control device is used to obtain the current dense point cloud data of the working area from the sensing device when receiving the working task; The control device is used to input the dense point cloud data and the excavation action trajectory at the previous moment into an earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein the current earthwork state includes a description of the state of the earthwork and its transfer; The control device is used to input the current earthwork state into an excavation action planning model so that the excavation action planning model plans an excavation action trajectory under the current earthwork state; The control device is used to control the excavator operation according to the excavation action trajectory, and after the operation is completed, again execute the step of acquiring the current dense point cloud data of the operation area from the sensing device until the operation task is completed; The control device is also used for: After controlling the excavator to operate according to the excavation motion trajectory, obtaining effect parameters of the excavator during the operation; According to the effect parameter, the reward feedback value of the excavation action trajectory is evaluated; wherein the magnitude of the reward feedback value of the excavation action trajectory represents the pros and cons of taking the excavation action trajectory under the current earthwork state; Determine the effect of the excavation action trajectory on the environment and the degree of completion of the excavation action trajectory on the work task; wherein the effect is characterized by the post-work earthwork state after the excavation action trajectory is executed; and the degree of completion is determined based on the post-work earthwork state and the task parameters of the work task; The current earthwork state, the excavation action trajectory, the reward feedback value of the excavation action trajectory, the action effect, and the completion degree are stored as an experience in a first experience knowledge base; Using the experience in the first experience knowledge base to train the strategy evaluation model, a trained strategy evaluation model is obtained; wherein the trained strategy evaluation model is used to estimate the value of an excavation action of taking the excavation action trajectory under the earthwork state according to data consisting of the earthwork state and the excavation action trajectory; wherein the excavation action value represents the quality of the excavation action trajectory; Using an experience in the first experience knowledge base to construct a data pair consisting of an earthwork state and an excavation action trajectory, and inputting the data pair into the trained strategy evaluation model so that the trained strategy evaluation model outputs the excavation action value of the data pair; storing a data pair and the mining action value of the data pair as an experience in a second experience knowledge base; The mining motion planning model is trained using the experience in the second experience knowledge base, so that the mining motion planning model learns and improves the ability of mining motion trajectory planning.

Images