Excavator control method and system oriented to non-structural working conditions

Abstract

The invention provides an excavator control method and system oriented to non-structural working conditions. According to the excavator control method and system oriented to the non-structural working condition, on one hand, according to the dynamic remodeling characteristic of earthwork in the excavation process, the excavation action track is planned through the earthwork state obtaining model and the excavation action planning model, the excavator control method and system can adapt to the non-structural working condition, and it is ensured that the operation effect is stable; on the second aspect, through real-time sensing and data feedback, the excavation action track can be adjusted in real time in the operation process, optimization is conducted according to the earthwork state, and the operation flexibility and effect can be improved; thirdly, by autonomously planning an excavation action track, the dependence on a manual operator can be reduced, the requirement of manual intervention is reduced, and the autonomous operation capability is improved; and fourthly, the mining action track is optimized through the mining action planning model, so that the energy consumption in the mining process can be reduced, the operation efficiency can be improved, and the operation cost can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of excavators, and in particular to an excavator control method and control system for non-structural working conditions. Background Art

[0002] Most existing automated operation methods decompose tasks step by step under the premise of making structured assumptions about the environment. However, in actual operations, working conditions often exhibit unstructured characteristics. For example, in the continuous earth excavation process, during the plastic interaction between the bucket and the environment, the dynamic remodeling characteristics of the earth often do not change according to the ideal situation. This may lead to a decrease in the accuracy of the excavation surface; on the other hand, it is difficult to accurately control the filling degree of the bucket during the excavation process. Furthermore, the inaccuracy of the control of the full bucket rate will lead to an increase in excavation energy consumption and a deterioration in the autonomous operation effect of the task. Summary of the invention

[0003] In view of this, the present application provides an excavator control method and control system for non-structural working conditions, which is used to solve the problems of decreased excavation surface accuracy and increased excavation energy consumption caused by the dynamic reshaping characteristics of earthwork, improve the task autonomy effect, improve work efficiency, and reduce work costs.

[0004] Specifically, the present application is implemented through the following technical solutions: A first aspect of the present application provides an excavator control method, the method comprising: The sensing device collects the environmental status of the working area of ​​the excavator and the self-status of the excavator in real time, and feeds back the environmental status and the self-status to the client; The client models the excavator and the working area based on the environmental state and the client's own state, and displays the obtained modeling results to the user; The client, in response to the user's operation of setting a job task based on the modeling result, feeds back the job task to the control device; When receiving the operation task, the control device acquires the current dense point cloud data of the operation area from the sensing device; The control device inputs the dense point cloud data and the excavation action trajectory at the last moment into an earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein the current earthwork state includes a description of the state of the earthwork and its transfer; The control device inputs the current earthwork state into an excavation motion planning model so that the excavation motion planning model plans an excavation motion trajectory under the current earthwork state; The control device controls the excavator operation according to the excavation motion trajectory, and after the operation is completed, again executes the step of acquiring the current dense point cloud data of the operation area from the sensing device until the operation task is completed.

[0005] A second aspect of the present application provides a control system, which is used to control an excavator facing non-structural working conditions, and the control system includes a client, a sensing device arranged on the excavator, and a control device arranged on the excavator; wherein, The sensing device is used to collect the environmental status of the working area of ​​the excavator and the self status of the excavator in real time, and feed back the environmental status and the self status to the client; The client is used to model the excavator and the working area based on the environmental state and the self-state, and present the obtained modeling results to the user; The client is configured to respond to the user's operation of setting a job task based on the modeling result and feed back the job task to the control device; The control device is used to obtain the current dense point cloud data of the working area from the sensing device when receiving the working task; The control device is used to input the dense point cloud data and the excavation action trajectory at the previous moment into an earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein the current earthwork state includes a description of the state of the earthwork and its transfer; The control device is used to input the current earthwork state into an excavation action planning model so that the excavation action planning model plans an excavation action trajectory under the current earthwork state; The control device is used to control the excavator operation according to the excavation action trajectory, and after the operation is completed, again execute the step of obtaining the current dense point cloud data of the operation area from the sensing device until the operation task is completed.

[0006] The excavator control method and control system for non-structured working conditions provided in the present application, on the one hand, according to the dynamic reshaping characteristics of earth during the excavation process, the excavation motion trajectory is planned through the earth state acquisition model and the excavation motion planning model, which can adapt to non-structured working conditions and ensure stable working results; on the other hand, through real-time perception and data feedback, the excavation motion trajectory can be adjusted in real time during the operation process, and optimized according to the earth state, which can improve the flexibility and effect of the operation; on the third hand, by autonomously planning the excavation motion trajectory, the dependence on manual operators can be reduced, the need for manual intervention is reduced, and the ability of autonomous operation is improved; on the fourth hand, by optimizing the excavation motion trajectory through the mining motion planning model, the energy consumption in the excavation process can be reduced, the working efficiency can be improved, and the working cost can be reduced.

[0007] Furthermore, the excavator control method and control system for non-structural working conditions provided in the present application, on the first aspect, by establishing a first experience knowledge base, a strategy evaluation model can be trained based on the first experience knowledge base so that the strategy evaluation model can accurately evaluate the value of the excavation action.

[0008] Secondly, through the trained strategy evaluation model, the mining action value of the data pair composed of earthwork status and excavation action trajectory can be accurately obtained, and then the second experience knowledge base can be constructed based on this, so that the second experience knowledge base can accumulate a large amount of experience data to cover various different excavation conditions. In this way, the subsequent training of the mining action planning model based on the second experience database can not only enhance the generalization ability of the model so that it can work effectively in different environments and conditions, but also enable the model to learn and evolve autonomously based on the second experience knowledge base, gradually improve the accuracy of decision-making and control, and enable the mining action planning model to plan the action trajectory more accurately in future excavation operations, thereby improving the overall operation accuracy and effect.

[0009] Thirdly, by establishing a second experience knowledge base, the mining action value can be used as a supervisory signal to train the mining action planning model in the future. This not only enables the mining action planning model to quickly identify and learn the optimal mining action trajectory, reduce training time, and improve learning efficiency, but also guides model training through supervisory signals, optimizes the excavator's operating path, and improves overall operating results and efficiency. In addition, through continuous feedback and optimization, the model can continuously improve the ability of mining action planning, reduce dependence on manual intervention, and improve the excavator's autonomous operation level. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A flowchart of a first embodiment of an excavator control method for non-structural working conditions provided by the present application; Figure 2A schematic diagram of a control system shown as an exemplary embodiment of the present application; Figure 3 A schematic diagram of a modeling result shown for an exemplary embodiment of the present application; Figure 4 This is a schematic diagram of an exemplary embodiment of the present application showing the implementation principle of setting the starting coordinates of excavation; Figure 5 A flowchart of Embodiment 2 of the excavator control method for non-structural working conditions provided by the present application; Figure 6 A flowchart of Embodiment 3 of the excavator control method for non-structural working conditions provided by the present application; Figure 7 A schematic diagram of a policy evaluation model shown as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0011] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0012] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more associated listed items.

[0013] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0014] Specific embodiments are given below to introduce the technical solution of the present application in detail.

[0015] Figure 1 This is a flowchart of the first embodiment of the excavator control method for non-structural working conditions provided by this application. Figure 1 The method provided in this embodiment is applied to a control system; the control system includes a client, a sensing device arranged on an excavator, and a control device arranged on the excavator; the method includes: S101, the sensing device collects the environmental status of the working area of ​​the excavator and the excavator's own status in real time, and feeds back the environmental status and the excavator's own status to the client.

[0016] Specifically, Figure 2 This is a schematic diagram of a control system according to an exemplary embodiment of the present application. Figure 2 , the control system is used to control the excavator, and the control system may include a client, a sensing device and a control device.

[0017] The client may be other equipment independent of the excavator, or may be equipment installed on the excavator, which is not limited in this embodiment. For example, in one possible implementation, the client may be a mobile phone, computer, etc. independent of the excavator; for another example, in another possible implementation, the client may be a vehicle-mounted terminal installed on the excavator.

[0018] The sensing device is used to collect the environmental status of the working area of ​​the excavator and the excavator's own status in real time. The sensing device may include a binocular camera set on the excavator (through which the environmental status is collected) and a sensor group set on the excavator (through which the excavator's own status is collected). It should be noted that the sensor group may include a laser radar, a posture sensor, an inclination sensor, a displacement sensor, a speed sensor, a pressure sensor, etc. In addition, it is understood that the control device may be a controller set on the excavator.

[0019] Specifically, the environmental state of the working area generally refers to the external conditions and factors of the working area where the excavator is located, which may include topographic information, obstacle information, the surrounding environment of the working area, etc. For example, in a possible implementation, the environmental state is represented by dense point cloud data.

[0020] Furthermore, the excavator's own state refers to the excavator's own properties and states, which may include position, posture, and mechanical structure state, etc.

[0021] It should be noted that the sensing device collects the environmental status and its own status in real time, and feeds back the collected environmental status and its own status to the client.

[0022] S102: The client models the excavator and the working area based on the environmental state and the client's own state, and displays the obtained modeling results to the user.

[0023] Optionally, in a possible implementation, modeling can be performed in the following manner: (1) Preprocess the collected data (see the previous description, the collected data includes environmental status and its own status) to clean, calibrate and align the data and remove outliers to ensure the accuracy and consistency of the data.

[0024] (2) Extracting modeling features from the processed data; the modeling features may include the motion features of the excavator (such as digging depth, steering angle, etc.), the terrain features of the environment (such as slope, obstacle location, etc.), and the soil features (such as hardness, humidity, etc.).

[0025] (3) Based on the modeling method, a model of the excavator and the working area is established according to the extracted modeling features to obtain the modeling results.

[0026] Specifically, Figure 3 This is a schematic diagram of the modeling results shown in an exemplary embodiment of the present application. Figure 3 Through modeling, the modeling results can be displayed to the user (the user can be an operator) through a graphical user interface, so that the user can understand the environmental information of the current working area and the real-time status of the excavator through the modeling results, so that the operator can plan the work and set the work tasks based on this.

[0027] S103: In response to the user's operation of setting a job task based on the modeling result, the client feeds back the job task to the control device.

[0028] Specifically, referring to the previous description, after the client performs modeling and obtains the modeling results, the modeling results can be displayed on the interface, and the operator can set the job task based on the displayed modeling results.

[0029] In specific implementation, when setting the work task, the operator can set the task type and the task parameters corresponding to the task type. Specifically, the task type may include leveling tasks, foundation pit tasks, ditch tasks, etc. It should be noted that the task parameters corresponding to different task types are different. For example, when the task type is a leveling task, Table 1 is the task parameters corresponding to the leveling task shown in an exemplary embodiment of the present application. Referring to Table 1, the task parameters corresponding to the leveling task may include starting coordinates, end angles, trajectory types, cycle time, maximum leveling angles, etc.

[0030] Table 1 Task parameters corresponding to the leveling task type parameter quantity float Mining start coordinates 3 float End Angle 1 usint Track Type 1 float time 1 float Maximum trimming angle 1 For another example, Table 2 shows the task parameters corresponding to the foundation pit task in an exemplary embodiment of the present application. Please refer to Table 2, when the task type is a foundation pit task, the corresponding task parameters include the starting coordinates for excavation, cutting angle, trajectory type, cycle time, cut-in angle (trimming angle), foundation pit depth, foundation pit length, foundation pit width, etc.

[0031] Table 2 Task parameters corresponding to the foundation pit task type parameter quantity float Mining start coordinates 3 float Cutting Angle 1 usint Track Type 1 float Cycle time 1 float Cut-in angle (trimming angle) 1 float Depth of foundation pit 1 float Length of foundation pit 1 float Width of foundation pit 1 It should be noted that when setting the starting coordinates of mining, it can be set through graphical interaction or parameter interaction. Figure 4 This is a schematic diagram of the implementation principle of setting the starting coordinates of mining according to an exemplary embodiment of the present application. Figure 4 When setting the starting coordinates of the excavation, you can select the starting and ending points of the excavation on the map in the modeling results through the graphical interaction method. It should be noted that the graphical interaction method can be used in situations where the terrain is manually meshed. For further information, please continue to refer to Figure 4 When setting the starting coordinates of excavation, you can also use parameter interaction to input coordinates in a predetermined world coordinate system and input the starting coordinates of excavation. It should be noted that the parameter interaction method is used in situations where digital modeling and coordinate planning have been performed on a large range of terrain.

[0032] S104: When receiving the operation task, the control device obtains the current dense point cloud data of the operation area from the sensing device.

[0033] As described above, the perception device includes a binocular camera, which acquires dense point cloud data of the working area in real time. When receiving a work task, the control device acquires the current dense point cloud data of the working area from the perception device, and then plans the mining action trajectory based on the dense point cloud data, and then converts the mining action trajectory into an electrical signal, and controls the excavator based on the electrical signal so that the excavator completes the work task.

[0034] It should be noted that dense point cloud data refers to a large amount of discrete point data collected in three-dimensional space. These points densely cover the surface of an object or scene. Therefore, dense point cloud data can capture detailed geometric shape and texture information of the working area.

[0035] S105. The control device inputs the dense point cloud data and the excavation action trajectory at the previous moment into the earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein the current earthwork state includes a description of the state of the earthwork and its transfer.

[0036] Specifically, the earthwork state acquisition model is a model used to obtain the description (parametric representation) of the state and transfer of the earthwork. The role of this model is to provide a representation form that can be used to understand and predict the change of the earthwork state through the analysis and processing of the input data. The input of the earthwork state acquisition model is dense point cloud data and the excavation action trajectory at the previous moment, and the output is the current earthwork state. It should be noted that the output of this model is specifically a parametric representation of the state and transfer of the earthwork.

[0037] It is understandable that, initially, if there is no excavation action trajectory at the previous moment, the earthwork state acquisition model directly outputs the current earthwork state based on the dense point cloud data.

[0038] It should be noted that, in theory, the earthwork state matrix can be directly obtained based on dense point cloud data to achieve observation of earthwork state. However, in actual construction, due to the deformation of earthwork, the near-end area of ​​the binocular camera may be blocked by earthwork, resulting in blank sensor information. Therefore, relying solely on dense point cloud data will not accurately obtain the earthwork state.

[0039] To this end, in this application, the earthwork status is obtained based on dense point cloud data and the excavation action trajectory at the previous moment to accurately obtain the earthwork status.

[0040] The specific implementation process and implementation principle of the earthwork status acquisition model for acquiring the earthwork status will be introduced in detail in the following embodiments and will not be repeated here.

[0041] S106: The control device inputs the current earthwork state into an excavation motion planning model so that the excavation motion planning model plans an excavation motion trajectory under the current earthwork state.

[0042] It should be noted that the excavation action planning model is a model used to determine and optimize the operation path and action sequence of the excavator when performing excavation tasks. The excavation action planning model can generate the optimal excavation action trajectory by analyzing the current earthwork state. By using the excavation action planning model to plan the excavation action trajectory based on the earthwork state, work efficiency can be improved, energy consumption can be reduced, and the safety of the operation can be ensured.

[0043] It can be understood that the excavation motion trajectory includes an motion trajectory composed of multiple points and an excavation position reference point.

[0044] In the specific implementation, the current earthwork state (the current earthwork state is identified in the form of a matrix) is reduced in dimension through state encoding and converted into vector input, thereby avoiding the convolution operation of the network and reducing the complexity of the state.

[0045] It should be noted that the core part of the mining action planning model is two deep policy networks, which are used to approximate the strategy of continuous actions. After receiving the earthwork state, the prediction of the next mining action trajectory is output according to the current action. In addition, since the mining action is a real-valued vector, in actual use, the output layer size of the mining action planning model is set to the action dimension to match the multivariate action space.

[0046] S107, the control device controls the excavator operation according to the excavation motion trajectory, and after the operation is completed, again executes the step of obtaining the current dense point cloud data of the operation area from the sensing device until the operation task is completed.

[0047] Specifically, after obtaining the excavation motion trajectory, the control device controls the excavator to operate according to the excavation motion trajectory, and after the operation is completed, repeats the above steps to indicate that the operation task has been completed.

[0048] The method provided in this embodiment, on the one hand, plans the excavation motion trajectory through the earthwork state acquisition model and the excavation motion planning model in view of the dynamic reshaping characteristics of the earthwork during the excavation process, which can adapt to unstructured working conditions and ensure stable working results; on the other hand, through real-time perception and data feedback, the excavation motion trajectory can be adjusted in real time during the operation process, and optimized according to the earthwork state, which can improve the flexibility and effect of the operation; on the third hand, through autonomous planning of the excavation motion trajectory, it can reduce dependence on manual operators, reduce the need for manual intervention, and improve the ability of autonomous operation; on the fourth hand, through the mining action planning model to optimize the excavation motion trajectory, it can reduce energy consumption in the excavation process, improve working efficiency, and reduce working costs.

[0049] Figure 5 This is a flowchart of the second embodiment of the excavator control method for non-structural working conditions provided by this application. Figure 5 The method provided in this embodiment, based on the above embodiment, determines the current earthwork state according to the dense point cloud data and the excavation action trajectory, including: S501. For any dense point cloud data, the dense point cloud data is cropped according to the boundary constraint conditions indicated by the operation task to obtain target point cloud data.

[0050] In a specific implementation, in one possible implementation, through the boundary constraints, the dense point cloud data can be calculated according to the following formula: Crop any dense point cloud data in: , in, are the initial coordinates of the constraint parameter boundaries, is the border length, Width is the border width, and Deepth is the border depth.

[0051] S502: According to the target point cloud data, the earthwork state is parameterized based on the voxel method to obtain an earthwork state matrix.

[0052] Specifically, the voxel method divides the three-dimensional space into uniform voxels, and then describes the earthwork status based on the point cloud density or other attributes within each voxel.

[0053] In specific implementation, the earthwork state matrix can be obtained according to the following method: (1) Voxelization: Divide the three-dimensional space into a uniform voxel grid. Each voxel represents a discrete area.

[0054] (2) Point cloud data conversion: The target point cloud data is mapped to a voxel grid. Based on the distribution of the target point cloud data in each voxel, the number of point clouds in each voxel can be calculated.

[0055] (3) Parametric earthwork status: The earthwork status is parametrically represented according to the number of point clouds within the voxel.

[0056] (4) Constructing the earthwork state matrix: The parameterized earthwork state is organized into a matrix, where each row or element of the matrix corresponds to a voxel, and the columns correspond to different features or attributes.

[0057] In specific implementation, referring to the previous description, the target point cloud data can be projected onto the voxel matrix, the earthwork surface can be estimated by the filling rate, and it is reasonably assumed that the target area under the earthwork surface is also an earthwork entity. Then, the earthwork state matrix parameterized by the voxel method can be constructed by the following formula: , in, is the earthwork state matrix, is the number of point clouds under the corresponding voxel, is the judgment threshold.

[0058] Specifically, if the number of point clouds under the corresponding voxel is greater than or equal to the judgment threshold, the earthwork is judged to be a solid and the corresponding matrix element is set to 1; if the number of point clouds under the corresponding voxel is less than the judgment threshold, the earthwork is judged to be a non-solid and the corresponding matrix element is set to 0.

[0059] S503 , defining a two-dimensional trajectory matrix according to the operation task, and mapping each trajectory point in the mining action trajectory to the two-dimensional trajectory matrix, to obtain a target two-dimensional trajectory matrix corresponding to the mining action trajectory.

[0060] In specific implementation, the bucket trajectory can be processed on the xz plane of the excavator robot reference system, and the trajectory sequence in the joint space can be calculated by collecting the cylinder motion information during each excavation cycle. Then the excavation action trajectory at the last moment (that is, the excavation action trajectory in this embodiment) It can be calculated based on the kinematic formula of the mining robot.

[0061] Specifically, in a possible implementation manner, the specific implementation process of this step may include: (1) Define the two-dimensional trajectory matrix as in Said is the excavation length of the task; is the task accuracy of the task.

[0062] (2) Traversing and mining action trajectories The trajectory points in And map the currently traversed trajectory points into the two-dimensional trajectory matrix according to the following formula to obtain the target two-dimensional trajectory matrix: , , in, is the constraint parameter, is the length of the excavation action trajectory, For the The number of trajectory points on the interval, is the target area boundary margin.

[0063] It should be noted that during the mapping process, the trajectory part in the interaction phase with the earthwork should be strictly segmented to prevent the influence of aerial motion trajectory points on the actual excavation trajectory calculation.

[0064] S504 , filling the blank areas in the target two-dimensional trajectory matrix by an interpolation method to obtain a complete spatial trajectory matrix.

[0065] Specifically, since the target two-dimensional trajectory matrix is ​​non-uniform, the target two-dimensional trajectory matrix has blanks in some areas, so the missing blank areas need to be filled by interpolation to obtain a complete spatial trajectory matrix.

[0066] In the specific implementation, in order to reasonably extrapolate while maintaining the accuracy of the original data, the linear interpolation method is used to fill the blank areas in the target two-dimensional trajectory matrix to obtain a complete spatial trajectory matrix.

[0067] It should be explained that the linear interpolation method makes an estimate between known data points by assuming that the changes between data points are linear, that is, the value of the unknown point changes along a straight line between two known points.

[0068] S505 , expanding the spatial trajectory matrix in a width direction according to the density of the dense point cloud data to obtain a trajectory matrix sequence.

[0069] It should be explained that in order to merge with the earthwork state matrix obtained based on dense point cloud data, so as to realize the construction of the excavation trajectory surface of the target area with point cloud accuracy, the spatial trajectory matrix is ​​expanded in the width direction according to the density of the dense point cloud to obtain the trajectory matrix sequence

[0070] S506: Determine the current earthwork state according to the earthwork state matrix and the trajectory matrix sequence.

[0071] In specific implementation, the trajectory matrix sequence is traversed and merged with the earthwork state matrix obtained based on dense point cloud data to obtain the final earthwork state. In specific implementation, the fusion can be performed according to the following formula: , in, is the current earthwork status; is the trajectory matrix sequence The value of the element at the index position; is the earthwork status matrix at index position The element value of An undefined or unrepresentable value.

[0072] It should be noted that in order to maximize the preservation of the original terrain, in the specific fusion process, only the target area where surface data is missing (i.e. The trajectory matrix sequence is constructed by merging the earthwork state matrix. Establish the final earthwork state.

[0073] The method provided in this embodiment provides a method for determining the state of earthwork. This method obtains the state of earthwork based on dense point cloud data and the excavation motion trajectory at the previous moment. It can avoid the problem of blank sensor information caused by the obstruction of earthwork in the near-end area under the perspective of the binocular camera, and accurately obtain the state of earthwork.

[0074] Figure 6 This is a flowchart of the third embodiment of the excavator control method for non-structural working conditions provided by this application. Figure 6The method provided in this embodiment, based on the above embodiment, further includes: S601. After controlling the excavator to operate according to the excavation motion trajectory, the control device obtains effect parameters of the excavator during the operation.

[0075] Specifically, the effect parameters include but are not limited to operation time, excavation force fluctuation amplitude, energy loss, working arm system friction loss and full bucket rate; the following is an example of effect parameters including operation time, energy loss, working arm coefficient friction loss and full bucket rate.

[0076] Specifically, energy loss is the total energy consumed by the excavator during operation, which can reflect the efficiency of the excavator's use of fuel or electricity; the friction loss of the working arm system refers to: during the movement of the excavator's working arm system (including arm, bucket, etc.), the energy loss caused by friction has a certain impact on the working efficiency and service life of the excavator; the full bucket rate refers to the degree to which the excavator bucket is filled in an excavation action, which can be used to measure the loading efficiency of the excavator.

[0077] In specific implementation, when the excavator executes the above excavation action trajectory, the power consumption of the excavator during this period can be obtained, and then the energy consumption of the excavator during this period can be used to calculate the energy loss of the excavator. Further, when obtaining the friction loss of the working arm system, the friction loss of the working arm system can be estimated based on the temperature rise of the lubricating oil of the hydraulic cylinder. In addition, when obtaining the full bucket rate, the actual load of the bucket and the maximum capacity of the bucket can be obtained, and then the full bucket rate can be calculated based on the actual load and the maximum capacity of the bucket.

[0078] S602. The control device evaluates the reward feedback value of the excavation action trajectory according to the effect parameter; wherein the magnitude of the reward feedback value of the excavation action trajectory represents the pros and cons of taking the excavation action trajectory in the current earthwork state.

[0079] Specifically, when calculating the reward feedback value, it is necessary to consider factors such as operation time, energy consumption, total joint travel and full bucket rate, which are factors affecting earthwork performance. In actual calculations, the task goal is expressed as completing the given task in the shortest time with the least fuel consumption and system loss.

[0080] In summary, in a possible implementation, the reward feedback value of the mining action trajectory can be calculated according to the following formula: , Among them, the is the energy loss; Said is the weight coefficient of the energy loss; Said is the friction loss of the working arm system; Said is the weight coefficient of the friction loss of the working arm system; Said is the full bucket rate; Said is the weight coefficient of the full bucket rate; Said is the time of the operation; Said is the weight coefficient of the operation time.

[0081] It should be noted that the weight coefficient of each effect parameter is set according to actual needs and is not limited in this embodiment. For example, it can be fitted by skilled operator habits and construction experience. In addition, in a possible implementation, the weight coefficient of each effect parameter can be determined according to specific application goals, operating costs and equipment characteristics.

[0082] Specifically, for example, in a possible implementation, in a scenario focusing on energy efficiency, energy consumption should be minimized to improve fuel efficiency. At this time, in the selection of weight coefficients, 𝑤1 should be set relatively large, and 𝑤2 and 𝑤3 should be set relatively small. For example, in one embodiment, 𝑤1 is set equal to 0.6, 𝑤2 is set equal to 0.2, and 𝑤3 is set equal to 0.2.

[0083] It should be noted that the reward feedback value of the excavation action trajectory represents the pros and cons of taking the excavation action trajectory in the current earthwork state. That is, the larger the reward feedback value of the excavation action trajectory, the more appropriate it is to take the excavation action trajectory in the current earthwork state, which is beneficial to reduce energy loss and friction loss of the working arm system and improve the full bucket rate; the smaller the reward feedback value of the excavation action trajectory, the more inappropriate it is to take the excavation action trajectory in the current earthwork state, which is not beneficial to reduce energy loss and friction loss of the working arm system and improve the full bucket rate.

[0084] S603, the control device determines the effect of the excavation action trajectory on the environment and the degree of completion of the excavation action trajectory on the work task; wherein the effect is characterized by the post-work earthwork state after executing the excavation action trajectory; the degree of completion is determined based on the post-work earthwork state and the task parameters of the work task.

[0085] Specifically, the effect of the excavation action trajectory on the environment refers to the impact of the excavator on the surrounding natural environment and ecosystem after executing the excavation action trajectory. In one possible implementation, the effect of the excavation action trajectory on the environment is characterized by the earthwork state after the excavator completes the excavation action trajectory (for the convenience of distinction, the earthwork state is recorded as the earthwork state after the operation). In other words, for example, at time t, the excavator executes the excavation action trajectory at that time. After executing the action trajectory, S t+1 Characterizes the effect of mining action trajectory on the environment, where S t+1 is the earthwork status at time t+1.

[0086] In specific implementation, after the excavator executes the excavation action trajectory at time t, it can further obtain the dense point cloud data at time t+1, see Figure 5 The description of can be based on the dense point cloud data at time t+1 and the excavation action trajectory at time t, and based on the earthwork state acquisition model, the earthwork state at time t+1 can be obtained.

[0087] Specifically, the degree of completion of the excavation action trajectory for the work task is determined based on the earthwork state after the work and the task parameters of the work task. Specifically, the earthwork state after the work can be compared with the task parameters of the work task, and the degree of completion of the excavation action trajectory for the work task can be determined based on the comparison result. For example, in one embodiment, the work task requires digging a foundation pit with a depth of 3 meters in the work area. After executing the excavation action trajectory, based on the earthwork state after the work, it is determined that the depth of the foundation pit after executing the excavation action trajectory is only 2.5 meters. At this time, it can be determined that the degree of completion of the excavation action trajectory for the work task is 85%.

[0088] S604: The control device stores the current earthwork state, the excavation action trajectory, the reward feedback value of the excavation action trajectory, the action effect, and the completion degree as an experience in a first experience knowledge base.

[0089] Specifically, for example, in one embodiment, for the convenience of explanation, the current earthwork state is recorded as The excavation action trajectory is recorded as The reward feedback value of the mining action trajectory is recorded as The effect of the mining action trajectory on the environment is recorded as The degree of completion of the mining action trajectory for the task is recorded as After executing the excavation trajectory action, As an experience Stored in the first experience knowledge base.

[0090] It can be understood that the first experience knowledge base includes multiple experiences, and the format of each experience is as follows: .

[0091] S605. The control device uses the experience in the first experience knowledge base to train a strategy evaluation model to obtain a trained strategy evaluation model; wherein the trained strategy evaluation model is used to estimate the excavation action value of taking the excavation action trajectory under the earthwork state based on data consisting of the earthwork state and the excavation action trajectory; wherein the excavation action value represents the quality of the excavation action trajectory.

[0092] Specifically, the trained strategy evaluation model is used to estimate the value of the excavation action of taking the excavation action trajectory in the earthwork state based on the data pair consisting of the earthwork state and the excavation action trajectory. Its input is the data pair consisting of the earthwork state and the excavation action trajectory, and the output is the excavation action value. It should be noted that the output excavation action value represents the degree of pros and cons of taking the excavation action trajectory in the earthwork state. For example, the larger the excavation action value, the better it is to take the excavation action trajectory in the earthwork state, and the smaller the excavation action value, the worse it is to take the excavation action trajectory in the earthwork state.

[0093] Optionally, in a possible implementation, Figure 7 This is a schematic diagram of a strategy evaluation model shown in an exemplary embodiment of the present application. Figure 7 In a possible implementation, the strategy evaluation model includes an encoder, a dual evaluation network, and a dual-objective evaluation network; wherein, The encoder is used to encode the input data pair to obtain a coding matrix; The dual evaluation network is used to estimate the mining action value of the mining action trajectory according to the encoding matrix; The dual-objective evaluation network is used to determine the evaluation value of the mining action trajectory according to the encoding matrix, so as to train the dual evaluation network based on the evaluation value.

[0094] Specifically, the encoder combines the input earthwork state and the excavation action trajectory. Further, the dual evaluation network and the dual target evaluation network established based on the dual Q value estimation technology receive the input data pairs and output the estimated value of the corresponding excavation action value.

[0095] Optionally, in a possible implementation, the using the experience in the first experience knowledge base to train the strategy evaluation model to obtain a trained strategy evaluation model includes: (1) For each experience in the first experience knowledge base, the time difference TD error and the strategy proximity coefficient of the experience are calculated, and the weight of the experience is calculated according to the TD error and the strategy proximity coefficient of the experience.

[0096] Specifically, for a certain experience, the weight of the experience can be calculated according to the following formula: The weight of this experience is calculated according to the following formula: , Among them, the is the weight of the i-th experience; Said is the TD error of the i-th experience; Said is the gradient sampling intensity coefficient; Said is the strategy proximity coefficient.

[0097] For the specific implementation principle and process of calculating the TD error of an experience, please refer to the description in the relevant technology, which will not be repeated here.

[0098] (2) Sampling is performed according to a specified sampling method to obtain sampling data; wherein the specified sampling method is Said A probability density function is constructed using the weights of each experience in the first experience knowledge base.

[0099] In the specific implementation, for each experience, the weight of the experience is used as the probability of the experience being collected during the sampling process, and then based on the first experience knowledge base, the probability density is constructed using the weights of each experience , and then extract samples from the probability density function P(t) through sampling method to obtain sampling data .

[0100] (3) Using the sampled data to train the strategy evaluation model.

[0101] In specific implementation, after obtaining the sampled data, the sampled data can be used to train the strategy evaluation model.

[0102] Specifically, the objective function of the dual evaluation network in the strategy evaluation model is: , , in, is the current state; Digging action for the previous state; For the The parameters of the network; is the value function; is the batch processing scale; is the m nearest experience ranges; For use as the distribution function of its probability density.

[0103] Furthermore, the gradient updating method of the dual evaluation network is: , in, are the parameters of the i-th target network.

[0104] S606. The control device constructs a data pair consisting of earthwork status and excavation action trajectory using an experience in the first experience database, and inputs the data pair into the trained strategy evaluation model so that the trained strategy evaluation model outputs the excavation action value of the data pair.

[0105] It should be noted that after the strategy evaluation model is trained, the data pair consisting of the earthwork state and the excavation action trajectory can be used to output the excavation action value of the data pair through the trained strategy evaluation model, that is, the excavation action value of using the excavation action trajectory under the earthwork state is obtained.

[0106] In the specific implementation, for any experience in the first experience database, such as experience 1, select the earthwork state in experience 1 and mining motion trajectories Pair to form a data pair , input this data pair into the trained strategy evaluation model, the trained strategy evaluation model evaluates the data pair, and finally outputs its mining action value. For example, in one embodiment, for the convenience of explanation, the mining action value is recorded as .

[0107] S607: The control device stores a data pair and the mining action value of the data pair as an experience in a second experience database.

[0108] Combined with the above example, for example, in a possible implementation, The experience is stored in the second experience database as an experience.

[0109] It is understandable that the second experience database contains multiple experiences, each of which consists of a data pair consisting of earthwork status and excavation action trajectory, as well as the excavation action value of the data pair. By gradually accumulating historical data of excavation operations, rich data support is provided for excavation action planning, making the decision-making process more scientific and efficient, thereby achieving better operation decisions.

[0110] S608: The control device trains the mining action planning model using the experience in the second experience knowledge base, so that the mining action planning model learns and improves the ability of mining action trajectory planning.

[0111] In specific implementation, the objective function of the mining action planning model is: , in, is the information entropy function; is the regularization coefficient, which is used to control the learning intensity; is the current state; For the excavation action in the previous state, are the parameters of the first network; are the parameters of the second network; is the value function; is the batch processing scale.

[0112] In the specific implementation, a variable α value is used, and a minimum entropy constraint method is established to calculate the α value through a dynamic programming method: , in, is the adaptive regularization coefficient. It should be noted that, referring to the previous description, it can be understood that the control device integrates two modes. One is to autonomously learn the strategy of a given task through operation attempts. This is mainly carried out in the early stage of the operation. Since there is not enough experience in the experience knowledge base, the mining action planning model has not yet learned any operation strategy. The other is to autonomously mine a given operation task through the learned mining action planning model. This mode requires the control device to perform after a large number of operation attempts and storage of rich experience. At this time, the learning effect of the mining action planning model on the given task determines the performance of the excavator's autonomous operation.

[0113] The method provided in this embodiment, in a first aspect, by establishing a first experience knowledge base, can train a strategy evaluation model based on the first experience knowledge base, so that the strategy evaluation model can accurately evaluate the value of the mining action.

[0114] Secondly, through the trained strategy evaluation model, the mining action value of the data pair composed of earthwork status and excavation action trajectory can be accurately obtained, and then the second experience knowledge base can be constructed based on this, so that the second experience knowledge base can accumulate a large amount of experience data to cover various different excavation conditions. In this way, the subsequent training of the mining action planning model based on the second experience database can not only enhance the generalization ability of the model so that it can work effectively in different environments and conditions, but also enable the model to learn and evolve autonomously based on the second experience knowledge base, gradually improve the accuracy of decision-making and control, and enable the mining action planning model to plan the action trajectory more accurately in future excavation operations, thereby improving the overall operation accuracy and effect.

[0115] Thirdly, by establishing a second experience knowledge base, the mining action value can be used as a supervisory signal to train the mining action planning model in the future. This not only enables the mining action planning model to quickly identify and learn the optimal mining action trajectory, reduce training time, and improve learning efficiency, but also guides model training through supervisory signals, optimizes the excavator's operating path, and improves overall operating results and efficiency. In addition, through continuous feedback and optimization, the model can continuously improve the ability of mining action planning, reduce dependence on manual intervention, and improve the excavator's autonomous operation level.

[0116] Corresponding to the aforementioned embodiment of an excavator control method for non-structural working conditions, the present application also provides an embodiment of a control system.

[0117] Please continue to refer to Figure 2 The present application also provides a control system, which is used to control an excavator facing non-structural working conditions, and the control system includes a client, a sensing device arranged on the excavator, and a control device arranged on the excavator; wherein, The sensing device is used to collect the environmental status of the working area of ​​the excavator and the self status of the excavator in real time, and feed back the environmental status and the self status to the client; The client is used to model the excavator and the working area based on the environmental state and the self-state, and present the obtained modeling results to the user; The client is configured to respond to the user's operation of setting a job task based on the modeling result and feed back the job task to the control device; The control device is used to obtain the current dense point cloud data of the working area from the sensing device when receiving the working task; The control device is used to input the dense point cloud data and the excavation action trajectory at the previous moment into an earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein the current earthwork state includes a description of the state of the earthwork and its transfer; The control device is used to input the current earthwork state into an excavation action planning model so that the excavation action planning model plans an excavation action trajectory under the current earthwork state; The control device is used to control the excavator operation according to the excavation action trajectory, and after the operation is completed, again execute the step of obtaining the current dense point cloud data of the operation area from the sensing device until the operation task is completed.

[0118] The control system of this embodiment can be used to execute Figure 1 The steps, specific implementation principles and implementation processes of the method embodiment shown are similar and will not be repeated here.

[0119] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A control method for an excavator in non-structural working conditions, characterized in that: The method is applied to the control system of the excavator control method for non-structural working conditions; The control system includes a client, a sensing device arranged on the excavator, and a control device arranged on the excavator; the method includes: The sensing device collects the environmental status of the working area of ​​the excavator and the self-status of the excavator in real time, and feeds back the environmental status and the self-status to the client; The client models the excavator and the working area based on the environmental state and the client's own state, and displays the obtained modeling results to the user; The client, in response to the user's operation of setting a job task based on the modeling result, feeds back the job task to the control device; When receiving the operation task, the control device acquires the current dense point cloud data of the operation area from the sensing device; The control device inputs the dense point cloud data and the excavation action trajectory at the last moment into an earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein the current earthwork state includes a description of the state of the earthwork and its transfer; The control device inputs the current earthwork state into an excavation motion planning model so that the excavation motion planning model plans an excavation motion trajectory under the current earthwork state; The control device controls the excavator operation according to the excavation motion trajectory, and after the operation is completed, again executes the step of acquiring the current dense point cloud data of the operation area from the sensing device until the operation task is completed.

2. The method according to claim 1, characterized in that The method further comprises: After the control device controls the excavator to operate according to the excavation motion trajectory, the control device obtains effect parameters of the excavator during the current operation; The control device evaluates the reward feedback value of the excavation action trajectory according to the effect parameter; wherein the magnitude of the reward feedback value of the excavation action trajectory represents the degree of superiority or inferiority of taking the excavation action trajectory in the current earthwork state; The control device determines the effect of the excavation action trajectory on the environment and the degree of completion of the excavation action trajectory on the work task; wherein the effect is represented by the post-work earthwork state after the excavation action trajectory is executed; and the degree of completion is determined based on the post-work earthwork state and the task parameters of the work task; The control device stores the current earthwork state, the excavation action trajectory, the reward feedback value of the excavation action trajectory, the action effect, and the completion degree as an experience in a first experience knowledge base; The control device uses the experience in the first experience knowledge base to train a strategy evaluation model to obtain a trained strategy evaluation model; wherein the trained strategy evaluation model is used to estimate the value of an excavation action of taking the excavation action trajectory under the earthwork state according to data consisting of the earthwork state and the excavation action trajectory; wherein the excavation action value represents the quality of the excavation action trajectory; The control device constructs a data pair consisting of an earthwork state and an excavation action trajectory using an experience in the first experience database, and inputs the data pair into the trained strategy evaluation model so that the trained strategy evaluation model outputs the excavation action value of the data pair; The control device stores a data pair and the mining action value of the data pair as an experience in a second experience database; The control device trains the mining motion planning model using the experience in the second experience knowledge base, so that the mining motion planning model learns and improves the ability of mining motion trajectory planning.

3. The method according to claim 1, characterized in that The determining of the current earthwork state according to the dense point cloud data and the excavation action trajectory includes: For any dense point cloud data, the dense point cloud data is cropped according to the boundary constraints indicated by the task to obtain the target point cloud data: According to the target point cloud data, the earthwork state is parameterized based on the voxel method to obtain an earthwork state matrix; A two-dimensional trajectory matrix is ​​defined according to the operation task, and each trajectory point in the mining action trajectory is mapped to the two-dimensional trajectory matrix to obtain a target two-dimensional trajectory matrix corresponding to the mining action trajectory; Filling the blank areas in the target two-dimensional trajectory matrix by an interpolation method to obtain a complete spatial trajectory matrix; Expanding the spatial trajectory matrix in a width direction according to the density of the dense point cloud data to obtain a trajectory matrix sequence; The current earthwork state is determined according to the earthwork state matrix and the trajectory matrix sequence.

4. The method according to claim 3, characterized in that A two-dimensional trajectory matrix is ​​defined according to the operation task, and each trajectory point in the mining action trajectory is mapped to the two-dimensional trajectory matrix to obtain a target two-dimensional trajectory matrix of the mining action trajectory, including: Define the two-dimensional trajectory matrix as ;in, , is the excavation length of the task; The task accuracy of the task; Traverse the mining action trajectory The trajectory points in , and map the currently traversed trajectory points to the two-dimensional trajectory matrix according to the following formula to obtain the target two-dimensional trajectory matrix: ， ， in, is the length of the excavation action trajectory, For the The number of trajectory points in an interval.

5. The method according to claim 3, characterized in that: The determining the current earthwork state according to the earthwork state matrix and the trajectory matrix sequence comprises: ， Among them, the It is a parametric representation of the current earthwork status; Said is the trajectory matrix sequence; Said is the earthwork state matrix.

6. The method according to claim 2, characterized in that The effect parameters include operation time, energy loss, friction loss of the working arm system and full bucket rate; the reward feedback value of the excavation action trajectory is evaluated according to the effect parameters, including: The reward feedback value of the mining action trajectory is evaluated according to the following formula: ， Among them, the is the energy loss; Said is the weight coefficient of the energy loss; Said is the friction loss of the working arm system; Said is the weight coefficient of the friction loss of the working arm system; Said is the full bucket rate; Said is the weight coefficient of the full bucket rate; Said is the time of the operation; Said is the weight coefficient of the operation time.

7. The method according to claim 2, characterized in that The strategy evaluation model includes an encoder, a dual evaluation network and a dual target evaluation network; wherein, The encoder is used to encode the input data pair to obtain a coding matrix; The dual evaluation network is used to estimate the mining action value of the mining action trajectory according to the encoding matrix; The dual-objective evaluation network is used to determine the evaluation value of the mining action trajectory according to the encoding matrix, so as to train the dual evaluation network based on the evaluation value.

8. The method according to claim 7, characterized in that The step of using the experience in the first experience knowledge base to train the strategy evaluation model to obtain a trained strategy evaluation model includes: For each experience in the first experience knowledge base, a time difference TD error and a strategy proximity coefficient of the experience are calculated, and a weight of the experience is calculated according to the TD error and the strategy proximity coefficient of the experience; Sampling is performed according to a specified sampling method to obtain sampling data; wherein the specified sampling method is Said A probability density function constructed by using the weights of each experience in the first experience knowledge base; The strategy evaluation model is trained using the sampled data; wherein the objective function of the dual evaluation network in the strategy evaluation model is: ， ， The gradient update method of the dual evaluation network is: 。 9. The method according to claim 8, characterized in that The step of calculating the weight of the experience according to the TD error of the experience and the strategy proximity coefficient includes: The weight of this experience is calculated according to the following formula: ， Among them, the is the weight of the i-th experience; Said is the TD error of the i-th experience; Said is the gradient sampling intensity coefficient; Said is the strategy proximity coefficient.

10. A control system, characterized in that: The control system is used to control an excavator facing non-structural working conditions, and the control system includes a client, a sensing device arranged on the excavator, and a control device arranged on the excavator; wherein, The sensing device is used to collect the environmental status of the working area of ​​the excavator and the self status of the excavator in real time, and feed back the environmental status and the self status to the client; The client is used to model the excavator and the working area based on the environmental state and the self-state, and present the obtained modeling results to the user; The client is configured to respond to the user's operation of setting a job task based on the modeling result and feed back the job task to the control device; The control device is used to obtain the current dense point cloud data of the working area from the sensing device when receiving the working task; The control device is used to input the dense point cloud data and the excavation action trajectory at the previous moment into an earthwork state acquisition model, so that the earthwork state acquisition model determines the current earthwork state according to the dense point cloud data and the excavation action trajectory; wherein the current earthwork state includes a description of the state of the earthwork and its transfer; The control device is used to input the current earthwork state into an excavation action planning model so that the excavation action planning model plans an excavation action trajectory under the current earthwork state; The control device is used to control the excavator operation according to the excavation action trajectory, and after the operation is completed, again execute the step of obtaining the current dense point cloud data of the operation area from the sensing device until the operation task is completed.

Images