Unmanned excavator control method, device and equipment and unmanned excavator

Abstract

Embodiments of the present application are devoted to providing an unmanned excavator control method, device, equipment and unmanned excavator, so as to realize that the unmanned excavator controls itself to work according to different working conditions. The working mode of the unmanned excavator is divided into a slope excavation mode, a flat land soil raking mode and a flat land soil piling mode according to the working conditions, and a pre-trained action classification network is used to classify the working conditions of the unmanned excavator to obtain the corresponding action mode. The three working modes summarized in the present application can cope with the working conditions of the unmanned excavator in most cases, and the present application further proposes a relatively simple control logic, that is, the control logic of obtaining the corresponding action mode through the action classification network, so as to realize the control of the unmanned excavator.

Description

Technical Field

[0001] This application relates to the field of construction machinery, specifically to a control method, device, equipment, and unmanned excavator for an unmanned excavator. Background Technology

[0002] Excavators are common construction machinery widely used in construction operations. Currently, all excavators are manually controlled by operators. To reduce operator injury in harsh working conditions and increase the reliability of excavators in such environments, the development of unmanned excavators is an important direction for excavator development.

[0003] However, how to control the operation of unmanned excavators remains a problem that has not yet been fully solved. For unmanned excavators, if the control logic is too simple, it will be unable to cope with complex working environments; if the control logic is too complex, its implementation will consume a lot of system resources, and complex control logic is also not conducive to the subsequent maintenance and updates of the unmanned excavator's control logic. Summary of the Invention

[0004] In view of this, embodiments of the present invention aim to provide an unmanned excavator control method, device, equipment, and unmanned excavator, so as to enable the unmanned excavator to control itself to work according to different working conditions.

[0005] In a first aspect, embodiments of the present invention provide a control method for an unmanned excavator, the method comprising:

[0006] Acquire terrain data, which is data obtained by detecting the working environment of the unmanned excavator, including the materials that the unmanned excavator needs to process;

[0007] The terrain data is input into a pre-trained action classification network to obtain action patterns. The unmanned excavator is then controlled to work according to these action patterns. The action patterns include one of the following: slope excavation mode, flat ground raking mode, and flat ground piling mode. The slope excavation mode has an arc excavation trajectory and is used when the amount of material to be processed by the unmanned excavator is greater than a first preset material amount. The flat ground raking mode has a straight excavation trajectory and is used when the amount of material to be processed by the unmanned excavator is no greater than the first preset material amount but greater than a second preset material amount. The flat ground piling mode collects scattered materials and is used when the amount of material to be processed by the unmanned excavator is less than the second preset material amount.

[0008] Optionally, acquiring terrain data includes:

[0009] The working environment of the unmanned excavator is detected to obtain raw point cloud data;

[0010] The original point cloud data is cropped to separate data related to the material that the unmanned excavator needs to process from the original point cloud data, thereby obtaining cropped point cloud data;

[0011] The cropped point cloud data is converted into a two-dimensional height map;

[0012] The two-dimensional height map is converted into a single-channel grayscale image and used as the terrain data.

[0013] Optionally, the step of cropping the original point cloud data to obtain cropped point cloud data includes:

[0014] A transformed coordinate system is established with the boom hinge point of the unmanned excavator as the origin, the facing direction of the unmanned excavator as the positive X-axis, the left side of the unmanned excavator as the positive Y-axis, and the top of the unmanned excavator as the positive Z-axis. The original point cloud data is placed in the transformed coordinate system to obtain coordinate point cloud data.

[0015] The point cloud clipping range on the X-axis is determined based on the nearest and farthest distances that the unmanned excavator can reach in the X-axis direction of the transformed coordinate system, and the point cloud clipping range on the Y-axis of the transformed coordinate system is determined based on the bucket width of the unmanned excavator.

[0016] The coordinate point cloud data is clipped according to the point cloud clipping range on the X-axis and the point cloud clipping range on the Y-axis to obtain the clipped point cloud data.

[0017] Optionally, converting the cropped point cloud data into a two-dimensional height map includes:

[0018] The coverage area of ​​the cropped point cloud data is divided into multiple grids;

[0019] For each of the multiple grids, the following processing is performed: when the amount of point cloud data in the coverage area of ​​the grid is greater than a preset threshold, the average height of all point cloud data in the grid is taken as the height value of the grid; or, when the amount of point cloud data in the coverage area of ​​the grid is not greater than the preset threshold, the coverage area of ​​the grid is expanded outward from the X-axis and Y-axis directions until the amount of point cloud data in the coverage area of ​​the grid is greater than the preset threshold, and the average height of all point cloud data in the expanded grid is taken as the height value of the grid.

[0020] All processed grids are height-normalized to obtain the two-dimensional height map. The upper limit of the height normalization range is the highest position that the unmanned excavator can dig, and the lower limit of the height normalization range is the smaller value between the lowest position that the unmanned excavator can dig and the lowest position that can be excavated on the ground.

[0021] Optionally, the pre-trained action classification network is a convolutional neural network;

[0022] The structure of the convolutional neural network includes an input layer, an output layer, and multiple hidden layers. The types of hidden layers include at least one of batch normalization layer, convolutional layer, pooling layer, ReLU layer, fully connected layer, and softmax layer.

[0023] Optionally, the method for obtaining training data for training the convolutional neural network includes:

[0024] Collect pre-training terrain data for unmanned excavators when they should be working in slope excavation mode, flat ground raking mode, and flat ground piling mode.

[0025] The pre-trained terrain data is augmented to obtain the training data used to train the convolutional neural network. The data augmentation methods include adding data noise, calculating the average value, and mirroring at least one of these.

[0026] Optionally, the data augmentation of the pre-trained terrain data includes:

[0027] For unmanned excavators, add two-dimensional Gaussian noise to the pre-trained terrain data corresponding to the slope excavation mode where they should work.

[0028] or,

[0029] For the pre-training terrain data corresponding to the unmanned excavator's work in the leveling and piling modes, at least two pre-training terrain data corresponding to the same work mode should be selected and averaged.

[0030] or,

[0031] Mirror all the pre-trained terrain data along the Y-axis.

[0032] Secondly, embodiments of the present invention provide a control device for an unmanned excavator, comprising:

[0033] The acquisition unit acquires terrain data, which is data obtained by detecting the working environment of the unmanned excavator, including the materials that the unmanned excavator needs to process.

[0034] A classification and control unit is used to input the terrain data into a pre-trained action classification network, obtain action patterns, and control the unmanned excavator to work according to the action patterns. The action patterns include one of a slope excavation mode, a level ground raking mode, and a level ground piling mode. The slope excavation mode has an arc excavation trajectory and is used when the amount of material to be processed by the unmanned excavator is greater than a first preset material amount. The level ground raking mode has a straight excavation trajectory and is used when the amount of material to be processed by the unmanned excavator is no greater than the first preset material amount but greater than a second preset material amount. The level ground piling mode collects scattered materials and is used when the amount of material to be processed by the unmanned excavator is less than the second preset material amount.

[0035] Thirdly, embodiments of the present invention provide an unmanned excavator control device, including a memory and a processor;

[0036] The memory is connected to the processor and is used to store programs;

[0037] The processor is used to implement the unmanned excavator control method as described in the first aspect by running the program in the memory.

[0038] Fourthly, embodiments of the present invention provide an unmanned excavator, including a controller and a terrain data acquisition device;

[0039] The controller is used to execute the unmanned excavator control method as described in the first aspect.

[0040] This application provides a control method for an unmanned excavator. The method categorizes the excavator's working modes into slope excavation mode, level ground raking mode, and level ground piling mode based on the working conditions. A pre-trained action classification network is used to classify the excavator's working conditions and obtain the corresponding action modes. The three working modes summarized in this application can handle the work of the excavator in most situations. This application also proposes a relatively simple control logic, namely, obtaining the control logic corresponding to the action mode through the action classification network, thus realizing the control of the unmanned excavator. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0042] Figure 1 The diagram shown is a flowchart illustrating a control method for an unmanned excavator provided in an embodiment of this application.

[0043] Figure 2 The diagram shown is a schematic diagram of the convolutional neural network structure provided in an embodiment of this application.

[0044] Figure 3 This is a schematic diagram illustrating the training results of establishing and training an action classification network according to an embodiment of this application.

[0045] Figure 4 This is a schematic diagram illustrating the verification results of the established action classification network provided in an embodiment of this application.

[0046] Figure 5 This is a schematic diagram of the structure of an unmanned excavator control device according to an embodiment of this application.

[0047] Figure 6 This is a schematic diagram of the structure of an unmanned excavator control device according to an embodiment of this application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] This application primarily studies the control logic of unmanned excavators. Excavators are widely used in engineering construction scenarios such as mines, ports, and factories. Researching intelligent and unmanned excavators can avoid excavator operators working in harsh or dangerous environments. Unmanned excavators can also work continuously around the clock, effectively improving work efficiency and reducing labor costs.

[0050] The first embodiment of the present invention provides a control method for an unmanned excavator, such as... Figure 1 As shown, the method may include the following steps:

[0051] Step 101: Obtain terrain data. The terrain data is the data obtained by detecting the working environment of the unmanned excavator. The working environment of the unmanned excavator includes the materials that need to be processed by the unmanned excavator.

[0052] The current technical challenge for unmanned excavators lies in controlling them to perform excavation work. For non-excavation tasks, such as controlling the unmanned excavator to travel to a construction site, the control logic is almost identical to that of controlling a regular vehicle.

[0053] Analogous to a conventional excavator, the first step for an operator is to observe the surrounding environment to determine how to proceed with the work. Similarly, for unmanned excavators, it is also necessary to detect and acquire data about their working environment. Since this application primarily focuses on how to control an excavator to perform digging work, it requires that the working environment of the unmanned excavator include the materials that the unmanned excavator needs to handle.

[0054] Specific methods for detecting the working environment of unmanned excavators include acquiring images of the working environment through image acquisition equipment, or obtaining point cloud data of the working environment through radar equipment.

[0055] Step 102: Input the terrain data into the pre-trained action classification network to obtain the action patterns, and control the unmanned excavator to work according to the action patterns. The action patterns include one of the following: slope excavation mode, flat ground raking mode, and flat ground piling mode. The excavation trajectory of the slope excavation mode is an arc, which is used when the amount of material to be processed by the unmanned excavator is greater than the first preset material amount. The excavation trajectory of the flat ground raking mode is a straight line, which is used when the amount of material to be processed by the unmanned excavator is no greater than the first preset material amount but greater than the second preset material amount. The flat ground piling mode will collect scattered materials, which is used when the amount of material to be processed by the unmanned excavator is less than the second preset material amount.

[0056] The control logic of an unmanned excavator greatly affects its automated operation efficiency. This application proposes selecting different preset action modes based on the amount of material within the unmanned excavator's operating area. Quantitative analysis of the material quantity can be achieved specifically through the material's volume and shape.

[0057] The preset action modes in this application are three main action modes obtained by the technical personnel through analysis and summary of the operating behavior of ordinary excavators. Specifically, they include slope excavation, leveling and raking, and leveling and piling. Slope excavation is suitable for situations where there is a large amount of material in front of the excavator, and the excavation trajectory is an arc. Leveling and raking is suitable for situations where the amount of material is moderate, and the excavation trajectory is a straight line. Leveling and piling is suitable for situations where the amount of material is small, mainly used to pile scattered material to the edge of the slope for the next excavation, thereby improving work efficiency. Furthermore, to standardize the amount of material, this application defines a first preset material quantity and a second preset material quantity. A material quantity greater than the first preset material quantity is considered a large amount; a material quantity between the first and second preset material quantities is considered a moderate amount; and a material quantity less than the second preset material quantity is considered a small amount.

[0058] A pre-trained action classification network analyzes terrain data to identify the amount of material the unmanned excavator needs to handle and outputs the most suitable action pattern for that working environment. The action classification network can be one of three types: recurrent neural networks, convolutional neural networks, or long short-term memory networks.

[0059] This embodiment provides a process for acquiring terrain data once and inputting it into an action classification network to obtain a working mode of an unmanned excavator. For a complete excavation operation, the unmanned excavator needs to use the above three different action modes sequentially to complete the excavation operation. Therefore, it can be set to continuously or periodically acquire terrain data and input it into the action classification network during the operation of the unmanned excavator. When the output result of the action classification network changes, the unmanned excavator can be controlled to change its working mode in a timely manner.

[0060] The second embodiment of this application further specifies the unmanned excavator control method in the first embodiment in a more detailed and specific way. Some or all of the technical features in the second embodiment can be combined with or replaced by the first embodiment, either individually or in combination, to obtain more feasible unmanned excavator control methods.

[0061] The unmanned excavator control method in the second embodiment of this application is described in detail below:

[0062] Optionally, acquiring terrain data includes: detecting the working environment of the unmanned excavator to acquire raw point cloud data; cropping the raw point cloud data to separate data related to the materials to be processed by the unmanned excavator from the raw point cloud data to acquire cropped point cloud data; converting the cropped point cloud data into a two-dimensional height map; and converting the two-dimensional height map into a single-channel grayscale image as terrain data.

[0063] To obtain more accurate terrain data, given that radar can detect distant targets both day and night, is less affected by weather conditions, and has all-weather, all-time characteristics, as well as a certain degree of penetration capability, this embodiment proposes a method for acquiring terrain data using radar.

[0064] Since radar acquires point cloud data, it cannot be directly input into the action classification network. This embodiment proposes to obtain terrain data that can be input into the action classification network through the following steps: After acquiring the raw point cloud data, the raw point cloud data is first preprocessed by cropping and coordinate transformation; then, the cropped point cloud data is converted into two dimensions to make the data conform to the input data format of the action classification network, that is, the cropped point cloud is converted into a height map; finally, the height map is normalized and converted into a grayscale image to reduce errors in the data analysis process, thus obtaining the final terrain data.

[0065] Optionally, the original point cloud data is cropped to obtain cropped point cloud data, including: establishing a transformed coordinate system with the boom hinge point of the unmanned excavator as the origin, the facing direction of the unmanned excavator as the positive X-axis, the left side of the unmanned excavator as the positive Y-axis, and the top of the unmanned excavator as the positive Z-axis; placing the original point cloud data in the transformed coordinate system to obtain coordinate point cloud data; determining the point cloud cropping range on the X-axis based on the nearest and farthest distances that the unmanned excavator can reach in the X-axis direction of the transformed coordinate system, and determining the point cloud cropping range on the Y-axis based on the bucket width of the unmanned excavator; cropping the coordinate point cloud data based on the point cloud cropping ranges on the X-axis and Y-axis to obtain cropped point cloud data.

[0066] During the operation of an unmanned excavator, the excavator's action mode actually depends only on the volume and shape information of the material at the excavation location. Therefore, this embodiment provides a method to separate the volume and shape information of the material at the excavation location by rotating and cropping the original point cloud data.

[0067] This embodiment uses a common excavator, model SY215E, as an example to illustrate the specific steps for point cloud cropping of this model of excavator:

[0068] 1) Perform coordinate transformation on the original point cloud data. The transformed point cloud takes the excavator boom hinge point as the origin of the coordinate system, with the front as the positive X-axis direction, the left as the positive Y-axis direction, and the upward as the positive Z-axis direction.

[0069] 2) Determine the point cloud clipping range on the X-axis based on the nearest and farthest distances that the excavator can reach in the X-axis direction. According to the dimensions of the boom, stick, and bucket of the SY215E model excavator, the point cloud clipping range in the X-axis direction can be set to [2,8]m;

[0070] 3) Determine the point cloud clipping range on the Y-axis based on the width of the excavator bucket. According to the SY215E model bucket width of 1.125m, the point cloud clipping range in the Y direction can be set to [-0.6, 0.6]m.

[0071] 4) Crop the original point cloud data according to the set point cloud clipping range in the X and Y directions.

[0072] Optionally, converting the clipped point cloud data into a two-dimensional height map includes: dividing the coverage area of ​​the clipped point cloud data into multiple grids; performing the following processing on each of the multiple grids: when the amount of point cloud data in the coverage area of ​​the grid is greater than a preset threshold, taking the average height of all point cloud data in the grid as the height value of the grid; or, when the amount of point cloud data in the coverage area of ​​the grid is not greater than the preset threshold, expanding the coverage area of ​​the current grid outward from the X-axis and Y-axis directions until the amount of point cloud data in the coverage area of ​​the grid is greater than the preset threshold, taking the average height of all point cloud data in the expanded grid as the height value of the grid; performing height normalization on all processed grids to obtain a two-dimensional height map, wherein the upper limit of the height normalization range is the highest position that the unmanned excavator can dig, and the lower limit of the height normalization range is the smaller value between the lowest position that the unmanned excavator can dig and the lowest position that can be excavated on the ground.

[0073] Because point cloud data has an irregular and disordered data format, it is not suitable as input for action classification networks. This embodiment provides a method to organize point cloud data through spatial grid block indexing technology, transform 3D point cloud data into 2D height maps, and then normalize it to obtain data that can be used as input for action classification networks.

[0074] First, set the grid resolution, determine the number of grids in length and width, and divide the coverage area of ​​the cropped point cloud data into multiple grids.

[0075] Considering the varying density of point cloud data, if the amount of point cloud data in a particular grid exceeds a preset threshold, it is considered that the amount of data within that grid is sufficient to obtain its height value, and the average height of all point clouds in that grid is taken as the current grid's height value. Otherwise, it is considered that the amount of data within that grid is insufficient to obtain its height value, and the current grid is expanded outwards in the X and Y directions until the amount of point cloud data in that grid exceeds the preset threshold, and then the height value of that grid is obtained.

[0076] Normalization is used to convert data into a uniform scale range to eliminate dimensional differences between different variables. In this embodiment, normalization maps height values ​​to a specific range, such as [0,1] or [-1,1], so that height values ​​have similar numerical ranges. To normalize the height of the height map, it is mainly necessary to consider the highest point that the excavator can dig (ExcaDigHigh), the lowest point that the excavator can dig (ExcaDigLow), and the lowest point of ground available for excavation (GroundDepth). The normalized height value is determined by the following three formulas:

[0077] ZMin=min(ExcaDigLow,GroundDpth)

[0078] ZMax = ExcaDigHigh

[0079]

[0080] Optionally, the pre-trained action classification network is a convolutional neural network; the structure of the convolutional neural network includes an input layer, an output layer and multiple hidden layers, and the types of hidden layers include at least one of batch normalization layer, convolutional layer, pooling layer, ReLU layer, fully connected layer and softmax layer.

[0081] Determining which action pattern is suitable for a given terrain based on terrain data is a multi-class classification problem. This embodiment selects a convolutional neural network (CNN) as the classifier. A convolutional network is essentially an input-to-output mapping; it can learn a large number of input-output mapping relationships without requiring any precise mathematical expressions between inputs and outputs. As long as the convolutional network is trained with known patterns, it possesses the ability to map between input-output pairs. The network structure designed in this invention is as follows: Figure 2 As shown, it consists of input and output layers as well as multiple hidden layers. The hidden layers can be divided into batch normalization layers, convolutional layers, pooling layers, ReLU layers, fully connected layers, and softmax layers.

[0082] like Figure 3 and Figure 4 As shown, this embodiment provides a specific process for establishing and validating the action classification network. A total of 252 sets of terrain data were collected and manually labeled, including 178 sets of excavated terrain, 64 sets of piled terrain, and 44 sets of raking terrain. 89 sets of excavated terrain, 32 sets of piled terrain, and 22 sets of raking terrain were randomly selected and augmented, resulting in 6143 training images, which were then input into the convolutional neural network for training. The accuracy of the classifier was verified using the remaining 143 sets of real data. The training accuracy is as follows: Figure 3 As shown.

[0083] Then, network validation was performed. Point cloud data was collected again and manually labeled, resulting in 82 sets of mining data, 51 sets of pile-building data, and 57 sets of raking data. These were then input into the trained action classification network to obtain the classification accuracy as shown below. Figure 4 As shown in the figure, the overall classification accuracy rate is 98%. There were misclassifications between piling and raking, but no misclassifications between digging and raking. This meets the requirements for accurate switching of action modes in real-world scenarios.

[0084] Optionally, the method for acquiring training data for training the convolutional neural network includes: collecting pre-training terrain data corresponding to when the unmanned excavator should be working in slope excavation mode, flat ground raking mode, and flat ground piling mode; performing data augmentation on the pre-training terrain data to obtain training data for training the convolutional neural network, wherein the data augmentation methods include adding data noise, calculating the average value, and mirroring at least one of these.

[0085] High-quality training data is a prerequisite for obtaining an action classification network with good classification performance. This embodiment provides a method for obtaining high-quality training data. Multiple sets of terrain data, including excavated terrain, piled terrain, and raked terrain, are collected and manually labeled. Then, a portion of the excavated terrain, piled terrain, and raked terrain are randomly selected for data augmentation to increase the amount of training data. Specifically, the original terrain data can be modified by adding data noise, calculating the average, and mirroring at least one of these methods to obtain new training data.

[0086] Optionally, data augmentation can be performed on the pre-training terrain data, including: adding two-dimensional Gaussian noise to the pre-training terrain data corresponding to the unmanned excavator's operation in the slope excavation mode; or, averaging at least two pre-training terrain data points corresponding to the unmanned excavator's operation in the leveling and piling modes; or, mirroring all the pre-training terrain data points according to the Y-axis.

[0087] Data augmentation refers to generating more training data based on existing training samples. Its purpose is to make the augmented training data as close as possible to the true distribution of data, thereby improving detection accuracy. Furthermore, data augmentation forces the model to learn more robust features, effectively improving the model's generalization ability. For the application requirement of classifying the action patterns of unmanned excavators, this embodiment employs the following methods for data augmentation.

[0088] 1) Superimposed two-dimensional Gaussian noise

[0089] In actual excavation, adding convex mounds to terrain with abundant soil does not affect the result of classifying it as a slope excavation pattern. Therefore, adding two-dimensional Gaussian noise to the slope excavation samples increases sample diversity and enriches the sample quantity.

[0090] The formula for a two-dimensional Gaussian distribution is as follows:

[0091]

[0092] Simplifying the above equation by letting σ1 = σ2 and ρ = 0, we get:

[0093]

[0094] For the terrain of the high platform excavation, a two-dimensional Gaussian distribution with a distribution of σ is added at (μ1,μ2). After selecting appropriate parameters and superimposing N Gaussian distributions, the data-enhanced slope excavation terrain sample is obtained.

[0095] 2) Calculate the average of multiple graphs

[0096] For flat, piled-up terrain, considering that the differences between different samples are not significant and the material distribution is relatively uniform, multiple samples of the same type can be summed and averaged. This operation generates new samples that do not affect the original classification results and can also increase the richness of the samples. Based on this, data-enhanced samples of flat, piled-up terrain can be obtained.

[0097] 3) Mirror

[0098] During the action classification process, the distribution of materials along the Y-axis does not affect the final classification result. Therefore, all training samples are mirrored along the Y-axis to obtain double the number of training samples.

[0099] Accordingly, the third embodiment of this application also provides a control device for an unmanned excavator, such as... Figure 5 As shown, the device includes:

[0100] Acquisition unit 501 acquires terrain data, which is data obtained by detecting the working environment of the unmanned excavator, including the materials that the unmanned excavator needs to process;

[0101] The classification and control unit 502 is used to input the terrain data into a pre-trained action classification network, obtain action patterns, and control the unmanned excavator to work according to the action patterns. The action patterns include one of the following: slope excavation mode, flat ground raking mode, and flat ground piling mode. The slope excavation mode has an arc excavation trajectory and is applied when the amount of material to be processed by the unmanned excavator is greater than a first preset material amount. The flat ground raking mode has a straight excavation trajectory and is applied when the amount of material to be processed by the unmanned excavator is no greater than the first preset material amount but greater than a second preset material amount. The flat ground piling mode collects scattered materials and is applied when the amount of material to be processed by the unmanned excavator is less than the second preset material amount.

[0102] The unmanned excavator control device provided in this embodiment belongs to the same application concept as the unmanned excavator control method provided in the above embodiments of this application. It can execute the unmanned excavator control method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment can be found in the specific processing content of the unmanned excavator control method provided in the above embodiments of this application, and will not be repeated here.

[0103] The functions implemented by the acquisition unit 501 and the classification and control unit 502 described above can be implemented by the same or different processors, and this application embodiment does not limit them.

[0104] It should be understood that the units in the above device can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above units. All units in the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0105] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0106] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0107] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0108] The fourth embodiment of this application also proposes a control device for an unmanned excavator, see [link]. Figure 6 As shown, the device includes:

[0109] Memory 600 and processor 610;

[0110] The memory 600 is connected to the processor 610 and is used to store programs;

[0111] The processor 610 is used to implement the unmanned excavator control method disclosed in any of the above embodiments by running the program stored in the memory 600.

[0112] Specifically, the aforementioned unmanned excavator control equipment may also include: a bus, a communication interface 620, an input device 630, and an output device 640.

[0113] The processor 610, memory 600, communication interface 620, input device 630, and output device 640 are interconnected via a bus. Among them:

[0114] A bus can include a pathway for transmitting information between various components of a computer system.

[0115] The processor 610 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0116] The processor 610 may include a main processor, as well as a baseband chip, modem, etc.

[0117] The memory 600 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 600 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0118] Input device 630 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0119] Output device 640 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0120] The communication interface 620 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0121] The processor 610 executes the program stored in the memory 600 and calls other devices, and can be used to implement the various steps of any of the unmanned excavator control methods provided in the above embodiments of this application.

[0122] The fifth embodiment of this application also proposes an unmanned excavator, including a controller and a terrain data acquisition device;

[0123] A controller for performing the unmanned excavator control method as described in the Method Embodiments section of this application.

[0124] The unmanned excavator provided in this embodiment belongs to the same application concept as the unmanned excavator control method provided in the above embodiments of this application. It can execute the unmanned excavator control method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects. Technical details not described in detail in this embodiment can be found in the specific processing content of the unmanned excavator control method provided in the above embodiments of this application, and will not be repeated here.

[0125] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0126] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0127] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0128] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0129] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0130] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0131] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0132] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0134] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A control method for an unmanned excavator, characterized in that, The method includes: Acquire terrain data, which is data obtained by detecting the working environment of the unmanned excavator, including the materials that the unmanned excavator needs to process; The terrain data is input into a pre-trained action classification network to obtain action patterns. The unmanned excavator is then controlled to work according to these action patterns. The action patterns include one of the following: slope excavation mode, flat ground raking mode, and flat ground piling mode. The slope excavation mode has an arc excavation trajectory and is used when the amount of material to be processed by the unmanned excavator is greater than a first preset material amount. The flat ground raking mode has a straight excavation trajectory and is used when the amount of material to be processed by the unmanned excavator is no greater than the first preset material amount but greater than a second preset material amount. The flat ground piling mode collects scattered materials and is used when the amount of material to be processed by the unmanned excavator is less than the second preset material amount.

2. The method according to claim 1, characterized in that, The acquisition of terrain data includes: The working environment of the unmanned excavator is detected to obtain raw point cloud data; The original point cloud data is cropped to separate data related to the material that the unmanned excavator needs to process from the original point cloud data, thereby obtaining cropped point cloud data; The cropped point cloud data is converted into a two-dimensional height map; The two-dimensional height map is converted into a single-channel grayscale image and used as the terrain data.

3. The method according to claim 2, characterized in that, The step of cropping the original point cloud data to obtain cropped point cloud data includes: A transformed coordinate system is established with the boom hinge point of the unmanned excavator as the origin, the facing direction of the unmanned excavator as the positive X-axis, the left side of the unmanned excavator as the positive Y-axis, and the top of the unmanned excavator as the positive Z-axis. The original point cloud data is placed in the transformed coordinate system to obtain coordinate point cloud data. The point cloud clipping range on the X-axis is determined based on the nearest and farthest distances that the unmanned excavator can reach in the X-axis direction of the transformed coordinate system, and the point cloud clipping range on the Y-axis of the transformed coordinate system is determined based on the bucket width of the unmanned excavator. The coordinate point cloud data is clipped according to the point cloud clipping range on the X-axis and the point cloud clipping range on the Y-axis to obtain the clipped point cloud data.

4. The method according to claim 2, characterized in that, The step of converting the cropped point cloud data into a two-dimensional height map includes: The coverage area of ​​the cropped point cloud data is divided into multiple grids; For each of the multiple grids, the following processing is performed: when the amount of point cloud data in the coverage area of ​​the grid is greater than a preset threshold, the average height of all point cloud data in the grid is taken as the height value of the grid; or, when the amount of point cloud data in the coverage area of ​​the grid is not greater than the preset threshold, the coverage area of ​​the grid is expanded outward from the X-axis and Y-axis directions until the amount of point cloud data in the coverage area of ​​the grid is greater than the preset threshold, and the average height of all point cloud data in the expanded grid is taken as the height value of the grid. All processed grids are height-normalized to obtain the two-dimensional height map. The upper limit of the height normalization range is the highest position that the unmanned excavator can dig, and the lower limit of the height normalization range is the smaller value between the lowest position that the unmanned excavator can dig and the lowest position that can be excavated on the ground.

5. The method according to claim 1, characterized in that, The pre-trained action classification network is a convolutional neural network; The structure of the convolutional neural network includes an input layer, an output layer, and multiple hidden layers. The types of hidden layers include at least one of batch normalization layer, convolutional layer, pooling layer, ReLU layer, fully connected layer, and softmax layer.

6. The method according to claim 5, characterized in that, A method for obtaining training data for training the convolutional neural network includes: Collect pre-training terrain data for unmanned excavators when they should be working in slope excavation mode, flat ground raking mode, and flat ground piling mode. The pre-trained terrain data is augmented to obtain the training data used to train the convolutional neural network. The data augmentation methods include adding data noise, calculating the average value, and mirroring at least one of these.

7. The method according to claim 6, characterized in that, The data augmentation of the pre-trained terrain data includes: For unmanned excavators, add two-dimensional Gaussian noise to the pre-trained terrain data corresponding to the slope excavation mode where they should work. or, For the pre-training terrain data corresponding to the unmanned excavator's work in the leveling and piling modes, at least two pre-training terrain data corresponding to the same work mode should be selected and averaged. Mirror all the pre-trained terrain data along the Y-axis.

8. A control device for an unmanned excavator, characterized in that, include: The acquisition unit acquires terrain data, which is data obtained by detecting the working environment of the unmanned excavator, including the materials that the unmanned excavator needs to process. A classification and control unit is used to input the terrain data into a pre-trained action classification network, obtain action patterns, and control the unmanned excavator to work according to the action patterns. The action patterns include one of a slope excavation mode, a level ground raking mode, and a level ground piling mode. The slope excavation mode has an arc excavation trajectory and is used when the amount of material to be processed by the unmanned excavator is greater than a first preset material amount. The level ground raking mode has a straight excavation trajectory and is used when the amount of material to be processed by the unmanned excavator is no greater than the first preset material amount but greater than a second preset material amount. The level ground piling mode collects scattered materials and is used when the amount of material to be processed by the unmanned excavator is less than the second preset material amount.

9. A control device for an unmanned excavator, characterized in that, Including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the unmanned excavator control method as described in any one of claims 1 to 7 by running the program in the memory.

10. An unmanned excavator, characterized in that, Includes controllers and terrain data acquisition equipment; The controller is used to execute the unmanned excavator control method as described in any one of claims 1 to 7.

Images