Automatic excavation control method and system based on three-dimensional model

Abstract

The application discloses an automatic excavation control method and system based on a three-dimensional model, which extracts dynamic change features of a three-dimensional model of a working face at multiple predetermined time points within a predetermined time period in a time dimension by using a first convolutional neural network model of a space attention mechanism and a second convolutional neural network model using a three-dimensional convolution kernel; and extracts multi-scale dynamic change features of a pitch angle value in the time dimension through a multi-scale neighborhood feature extraction module, and then estimates the responsiveness of the three-dimensional model change of the working face to the pitch angle change to represent the responsiveness correlation features, so as to control the pitch angle at the current time point. In this way, the pitch angle at the current time point can be accurately controlled in real time, so as to improve the efficiency and quality of the excavator excavation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control, and more particularly, to an automatic excavation control method and system based on a three-dimensional model. BACKGROUND

[0002] An excavator is one of the most typical functions, the most complex structure, and the most widely used engineering machinery, which plays an extremely important role in industrial and civil construction, transportation, water conservancy and power engineering, mining and military engineering construction.

[0003] At present, in the working process of the excavator, the angle of the bucket, the dipper arm, the swing arm and the vehicle body is generally measured by the inclination sensor, and the three-dimensional working posture control of the excavator is realized by relying on real-time adjustment by manual operation. However, since this control method needs to be operated manually, it is not only time-consuming and laborious, but also difficult to ensure the accuracy of the control, and the use of the inclination sensor for three-dimensional measurement of the excavator in this process will result in that the measurement result is difficult to accurately meet the actual working situation of the excavation, thereby reducing the work efficiency and work quality, and even may cause serious safety accidents. With the continuous development of automation technology and video recognition technology, it is of great significance to realize automatic excavation of the excavator.

[0004] Therefore, an optimized automatic excavation control scheme based on a three-dimensional model is expected.

[0005] The information disclosed in this BACKGROUND section is only for the purpose of increasing the understanding of the background of the present application and should not be taken as an acknowledgment or any form of suggestion that this information forms prior art of this application. SUMMARY

[0006] The present application aims to provide an automatic excavation control method and system based on a three-dimensional model, thereby overcoming the defects in the prior art.

[0007] In order to achieve the above-mentioned purpose, the present application provides an automatic excavation control method and system based on a three-dimensional model, which extracts the dynamic change characteristics of the three-dimensional model of the working face in the time dimension of a plurality of predetermined time points within a predetermined time period by using a first convolutional neural network model using a spatial attention mechanism and a second convolutional neural network model using a three-dimensional convolution kernel; extracts the multi-scale dynamic change characteristics of the pitch angle value in the time dimension by a multi-scale neighborhood feature extraction module, and represents the responsiveness correlation characteristics of the three-dimensional model change of the working face to the pitch angle change by the responsiveness estimation of the two, so as to control the pitch angle of the current time point. In this way, the pitch angle of the current time point can be accurately controlled in real time, so as to improve the efficiency and quality of the excavation of the excavator.

[0008] According to an aspect of the present application, a three-dimensional model-based automatic mining control method is provided, which comprises:

[0009] obtaining a three-dimensional model of a working face at a plurality of predetermined time points within a predetermined time period and pitch angle values at the plurality of predetermined time points;

[0010] inputting the three-dimensional model of the working face at the plurality of predetermined time points into a first convolutional neural network model using a spatial attention mechanism respectively to obtain a plurality of working face feature matrices;

[0011] arranging the plurality of working face feature matrices into a three-dimensional input tensor and then passing through a second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature vector;

[0012] arranging the pitch angle values at the plurality of predetermined time points into a pitch angle input vector and then passing through a multi-scale neighborhood feature extraction module to obtain a pitch angle feature vector;

[0013] calculating a responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector to obtain a classification feature matrix; and

[0014] passing the classification feature matrix through a classifier to obtain a classification result, the classification result being used to indicate whether the pitch angle at the current time point should be increased or decreased.

[0015] According to another aspect of the present application, a three-dimensional model-based automatic mining control system is provided, which comprises:

[0016] a data acquisition module configured to obtain a three-dimensional model of a working face at a plurality of predetermined time points within a predetermined time period and pitch angle values at the plurality of predetermined time points;

[0017] a working face feature extraction module configured to input the three-dimensional model of the working face at the plurality of predetermined time points into a first convolutional neural network model using a spatial attention mechanism respectively to obtain a plurality of working face feature matrices;

[0018] a working face change feature extraction module configured to arrange the plurality of working face feature matrices into a three-dimensional input tensor and then pass through a second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature vector;

[0019] a pitch angle feature extraction module configured to arrange the pitch angle values at the plurality of predetermined time points into a pitch angle input vector and then pass through a multi-scale neighborhood feature extraction module to obtain a pitch angle feature vector;

[0020] a responsiveness estimation module configured to calculate a responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector to obtain a classification feature matrix; and

[0021] The pitch angle control result generation module is configured to pass the classification feature matrix through a classifier to obtain a classification result, which is used to indicate whether the pitch angle at the current time point should be increased or decreased.

[0022] Compared with the prior art, the present application has the following beneficial effects:

[0023] The automatic excavation control method based on a three-dimensional model provided in the present application extracts the dynamic change features of the three-dimensional model of the working face at multiple predetermined time points within a predetermined time period in the time dimension through a first convolutional neural network model using a spatial attention mechanism and a second convolutional neural network model using a three-dimensional convolution kernel; extracts the multi-scale dynamic change features of the pitch angle value in the time dimension through a multi-scale neighborhood feature extraction module, and represents the responsiveness correlation features of the change of the three-dimensional model of the working face to the change of the pitch angle through the responsiveness estimation of the two, so as to control the pitch angle at the current time point. In this way, the pitch angle at the current time point can be accurately controlled in real time, so as to improve the efficiency and quality of the excavation of the excavator. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The application scenario diagram of the automatic excavation control method based on a three-dimensional model of the embodiments of the present application;

[0025] Figure 2 The flowchart of the automatic excavation control method based on a three-dimensional model of the embodiments of the present application;

[0026] Figure 3 The architecture schematic diagram of the automatic excavation control method based on a three-dimensional model of the embodiments of the present application;

[0027] Figure 4 The flowchart of the automatic excavation control method based on a three-dimensional model of the embodiments of the present application, in which the pitch angle values at the multiple predetermined time points are arranged into a pitch angle input vector and then passed through a multi-scale neighborhood feature extraction module to obtain a pitch angle feature vector;

[0028] Figure 5 The flowchart of the automatic excavation control method based on a three-dimensional model of the embodiments of the present application, in which the classification feature matrix is passed through a classifier to obtain a classification result;

[0029] Figure 6 The flowchart of the training step in the automatic excavation control method based on a three-dimensional model of the embodiments of the present application;

[0030] Figure 7 The block diagram of the automatic excavation control system based on a three-dimensional model of the embodiments of the present application. DETAILED DESCRIPTION

[0031] The specific embodiments of the present application will be described in detail below, but it should be understood that the scope of protection of the present application is not limited by the specific embodiments.

[0032] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.

[0033] Scenario overview

[0034] As described above, at present, in the working process of the excavator, the angles of the bucket, the dipper arm, the swing arm and the vehicle body of the excavator are generally measured by the inclination sensor, and the three-dimensional working posture control of the excavator is realized by relying on real-time adjustment by manual operation. However, since this control method needs to be operated manually, it not only consumes time and effort, but also makes it difficult to ensure the accuracy of the control, and the use of the inclination sensor to measure the three-dimensional excavator in this process will make it difficult for the measurement results to accurately meet the actual working situation of the excavation, thereby reducing the work efficiency and work quality, and even may cause serious safety accidents. With the continuous development of automation technology and video recognition technology, it is of great significance to realize automatic excavation of the excavator. Therefore, an optimized automatic excavation control scheme based on a three-dimensional model is expected.

[0035] At present, deep learning and neural networks have been widely used in computer vision, natural language processing, speech signal processing and other fields. In addition, deep learning and neural networks have also shown a level close to or even surpassing that of humans in the fields of image classification, object detection, semantic segmentation, text translation and the like.

[0036] In recent years, the development of deep learning and neural networks has provided new solutions and schemes for intelligent control of automatic excavation.

[0037] It can be understood that automatic excavation is to research the automatic operation control system of the wheel bucket excavator, use mature technologies such as laser scanners, high-precision positioning, video recognition, research automatic recognition technologies such as working face width, material properties, flatness, coordinates, form real-time 3D modeling, and build a virtual three-dimensional coordinate space with the same proportion. Develop an automatic operation control system, dynamically adjust the walking, pitching, footage, rotation and other actions of the excavator, and realize automatic excavation operation.

[0038] Correspondingly, since considering the existing wheel bucket excavator operation control scheme, most of them rely on the angle sensor to measure the angle of the excavator's bucket, stick, boom and vehicle body, and rely on real-time adjustment by artificial to realize the working posture control of the excavator. This not only consumes a lot of manpower, but also makes it difficult for the excavator to meet the actual working environment in the actual working process, thereby reducing the working efficiency and quality. The automatic excavation technology can realize the automatic excavation control of the wheel bucket excavator by forming real-time 3D modeling and constructing a virtual three-dimensional coordinate space with the same proportion, so as to meet the actual excavation conditions.

[0039] Based on this, in the technical scheme of the present application, it is expected to extract the dynamic change characteristics of the working surface in the time dimension and the multi-scale dynamic change characteristics of the pitch angle in the time dimension based on the three-dimensional model of the working surface using artificial intelligence control technology based on deep learning, and to represent the responsiveness of the three-dimensional model of the working surface to the change of the pitch angle. The responsiveness of the correlation characteristics of the pitch angle is estimated by the pitch angle feature vector and the working surface change feature vector, so as to control the pitch angle at the current time point, so as to realize the automatic excavation control based on the three-dimensional model. That is, by applying artificial intelligence technology to the automatic excavation of the excavator, the automatic excavation control scheme based on the three-dimensional model is constructed to improve the efficiency and quality of excavation.

[0040] Specifically, in the technical scheme of the present application, first, the three-dimensional model of the working surface at multiple predetermined time points in a predetermined time period and the pitch angle value at the multiple predetermined time points are obtained. In particular, in one specific example of the present application, the three-dimensional model of the working surface can be obtained by scanning with a laser scanner. Then, in order to accurately control the pitch angle of the excavator in real time according to the actual working surface during the operation of the excavator, the real-time dynamic change feature information of the operating surface in contact with the excavator needs to be focused on in the dynamic change feature extraction of the three-dimensional model of the working surface, and the remaining irrelevant to the pitch angle control. Therefore, in the technical scheme of the present application, the three-dimensional model of the working surface at the multiple predetermined time points is further input into the first convolutional neural network model using spatial attention mechanism for feature mining to extract the feature information of the operating surface in contact with the excavator in the three-dimensional model at the multiple predetermined time points, thereby obtaining multiple working surface feature matrices.

[0041] Then, considering that the plurality of working face feature matrices are feature distribution information of the working face focusing on the excavator operation surface at a plurality of predetermined time points within the predetermined time period of the three-dimensional model of the working face, in order to be able to extract its dynamic feature distribution information within the predetermined time period, it is necessary to further arrange the plurality of working face feature matrices into a three-dimensional input tensor and then process it in the second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature vector. In particular, here, the convolution kernel of the second convolutional neural network model is a three-dimensional convolution kernel, which has W (width), H (height) and C (channel dimension), and in the technical solution of the present application, the channel dimension of the three-dimensional convolution kernel corresponds to the time dimension of the three-dimensional input tensor, so that when three-dimensional convolution coding is performed, the dynamic distribution features of the working face of the three-dimensional model can be extracted along with the time dimension.

[0042] Further, since the pitch angle value has volatility and uncertainty in the time dimension, and has different mode change features within the predetermined time period, in order to accurately extract the dynamic change feature information of the pitch angle value, the pitch angle values at the plurality of predetermined time points are arranged into a pitch angle input vector and then coded in a multi-scale neighborhood feature extraction module to extract multi-scale neighborhood correlation features of the pitch angle value at different time spans, thereby obtaining a pitch angle feature vector.

[0043] Next, the responsiveness estimate of the pitch angle feature vector with respect to the working face change feature vector is calculated, which represents the responsiveness correlation feature information between the dynamic change features of the three-dimensional model of the working face and the multi-scale change features of the pitch angle, and is used as a classification feature matrix for classification processing in the classifier to obtain a classification result indicating whether the pitch angle at the current time point should be increased or decreased. In this way, real-time control of the pitch angle at the current time point can be achieved.

[0044] In particular, in the technical solution of the present application, when calculating the responsiveness estimate of the pitch angle feature vector with respect to the working face change feature vector, since the working face change feature vector represents the channel dimension distribution of the working face change feature map, its feature distribution deviates from the distribution of the initial three-dimensional model of the working face along the time sequence direction, while the pitch angle feature vector still represents the multi-scale time sequence correlation of the pitch angle values at the plurality of predetermined time points.

[0045] Therefore, in order to improve the calculation accuracy of the responsiveness estimate of the pitch angle feature vector with respect to the working face change feature vector, it is necessary to restore the causal relationship between the pitch angle feature vector and the working face change feature vector.

[0046] Based on this, the sequence-to-sequence response rule internalization learning loss function is introduced, denoted as:

[0047]

[0048]

[0049]

[0050] wherein V1 and V2 are the training pitch angle feature vector and the training working face change feature vector respectively, and W1 and W2 are the weight matrix of the classifier for the training pitch angle feature vector and the training working face change feature vector respectively, V1 + denotes the first activation feature vector, V2 + denotes the second activation feature vector, denotes the loss function value, ReLU(·) denotes the ReLU activation function, Sigmoid(·) denotes the Sigmoid activation function, denotes matrix multiplication, and d(·,·) denotes the Euclidean distance between two vectors.

[0051] That is, through the squeeze-and-excitation channel attention mechanism of the different weight matrices of the classifier for the pitch angle feature vector V1 and the working face change feature vector V2, the enhanced discriminative ability between the sequences of the feature vectors is obtained. By training the network with this loss function, the recovery of the causal relationship features with better discriminability between the response sequences can be realized, so as to internalize the learning of the cause-and-effect response rule between the sequences of the pitch angle feature vector V1 and the working face change feature vector V2, thereby enhancing the accuracy of the response calculation between the pitch angle feature vector V1 and the working face change feature vector V2 as the feature sequence. In this way, the pitch angle at the current time point can be accurately controlled in real time, so as to improve the efficiency and quality of the excavator excavation, while ensuring the safety of the excavation.

[0052] Based on this, this application proposes an automatic mining control method based on a 3D model, which includes: acquiring 3D models of working faces at multiple predetermined time points within a predetermined time period and the pitch angle values ​​of the multiple predetermined time points; inputting the 3D models of the working faces at the multiple predetermined time points into a first convolutional neural network model using a spatial attention mechanism to obtain multiple working face feature matrices; arranging the multiple working face feature matrices into a 3D input tensor and then using a second convolutional neural network model with 3D convolutional kernels to obtain a working face change feature vector; arranging the pitch angle values ​​at the multiple predetermined time points into a pitch angle input vector and then using a multi-scale neighborhood feature extraction module to obtain a pitch angle feature vector; calculating the responsiveness estimate of the pitch angle feature vector relative to the working face change feature vector to obtain a classification feature matrix; and passing the classification feature matrix through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the pitch angle at the current time point should increase or decrease.

[0053] Figure 1 This is an application scenario diagram of the automatic mining control method based on a 3D model according to an embodiment of this application. For example... Figure 1 As shown, in this application scenario, firstly, three-dimensional models of the working surfaces at multiple predetermined time points within a predetermined time period are obtained (e.g., such as...). Figure 1 The C1 shown in the figure) and the pitch angle values ​​at the plurality of predetermined time points (e.g., as shown in the figure) Figure 1 (as shown in C2); then, the obtained 3D model of the working face and the pitch angle values ​​are input into a server that has deployed an automatic mining control algorithm based on the 3D model (e.g., as shown in C2); Figure 1 As shown in S), the server processes the 3D model and the pitch angle value using an automatic mining control algorithm based on the 3D model to generate a classification result indicating whether the pitch angle should increase or decrease at the current time point.

[0054] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0055] Exemplary methods

[0056] Figure 2 This is a flowchart of an automatic mining control method based on a 3D model according to an embodiment of this application. Figure 2As shown, according to the automatic excavation control method based on a three-dimensional model of the embodiment of the present application, the method comprises: S110, obtaining a three-dimensional model of a working face at a plurality of predetermined time points in a predetermined time period and pitch angle values at the plurality of predetermined time points; S120, inputting the three-dimensional model of the working face at the plurality of predetermined time points into a first convolutional neural network model using a spatial attention mechanism respectively to obtain a plurality of working face feature matrices; S130, arranging the plurality of working face feature matrices into a three-dimensional input tensor and then passing through a second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature vector; S140, arranging the pitch angle values at the plurality of predetermined time points into a pitch angle input vector and then passing through a multi-scale neighborhood feature extraction module to obtain a pitch angle feature vector; S150, calculating a responsiveness estimation of the pitch angle feature vector relative to the working face change feature vector to obtain a classification feature matrix; and S160, passing the classification feature matrix through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the pitch angle at the current time point should be increased or decreased.

[0057] Figure 3 The architecture schematic diagram of the automatic excavation control method based on a three-dimensional model according to the embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, in the network architecture of the automatic excavation control method based on a three-dimensional model, first, a three-dimensional model of a working face at a plurality of predetermined time points in a predetermined time period and pitch angle values at the plurality of predetermined time points are obtained; then, the three-dimensional model of the working face at the plurality of predetermined time points is inputted into a first convolutional neural network model using a spatial attention mechanism respectively to obtain a plurality of working face feature matrices; next, the plurality of working face feature matrices are arranged into a three-dimensional input tensor and then passed through a second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature vector; then, the pitch angle values at the plurality of predetermined time points are arranged into a pitch angle input vector and then passed through a multi-scale neighborhood feature extraction module to obtain a pitch angle feature vector; next, a responsiveness estimation of the pitch angle feature vector relative to the working face change feature vector is calculated to obtain a classification feature matrix; and finally, the classification feature matrix is passed through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the pitch angle at the current time point should be increased or decreased. Figure 3

[0058] ​Specifically, in step S110, a three-dimensional model of the working face at a plurality of predetermined time points within a predetermined time period and a pitch angle value at the plurality of predetermined time points are obtained. As described above, at present, in the working process of the excavator, the angles of the bucket, the dipper arm, the swing arm and the vehicle body of the excavator are generally measured by the inclination sensor, and the three-dimensional working posture control of the excavator is realized by relying on manual real-time adjustment. However, since this control method needs to be operated manually, it not only consumes a lot of time and effort, but also makes it difficult to guarantee the accuracy of control, and in this process, using the inclination sensor to measure the three-dimensional excavator will make it difficult for the measurement result to accurately meet the actual working situation of excavation, thereby reducing the work efficiency and work quality, and even may cause serious safety accidents. With the continuous development of automation technology and video recognition technology, it is of great significance to realize automatic excavation of the excavator. Therefore, an optimized automatic excavation control scheme based on three-dimensional model is expected.

[0059] At present, deep learning and neural networks have been widely used in computer vision, natural language processing, speech signal processing and other fields. In addition, deep learning and neural networks have also shown a level close to or even surpassing human level in image classification, object detection, semantic segmentation, text translation and other fields.

[0060] In recent years, the development of deep learning and neural networks has provided new solutions and schemes for intelligent control of automatic excavation.

[0061] It can be understood that automatic excavation is to study the automatic operation control system of wheel bucket excavator, use mature technologies such as laser scanner, high-precision positioning and video recognition, study automatic recognition technologies such as working face width, material properties, flatness and coordinates, form real-time 3D modeling, and build a virtual three-dimensional coordinate space with the same proportion. Develop an automatic operation control system to dynamically adjust the walking, pitch, footage and rotation of the excavator, and realize automatic excavation operation.

[0062] Correspondingly, since most of the existing operation control schemes of wheel bucket excavators rely on inclination sensors to measure the angles of the bucket, the dipper arm, the swing arm and the vehicle body of the excavator, and rely on manual real-time adjustment to realize the working posture control of the excavator. This not only consumes a lot of manpower, but also makes it difficult for the working posture of the excavator to meet the actual working environment in the actual working process, thereby leading to a decrease in work efficiency and quality. Automatic excavation technology can realize automatic excavation control of wheel bucket excavator by forming real-time 3D modeling and building a virtual three-dimensional coordinate space with the same proportion to meet the actual excavation conditions.

[0063] Based on this, in the technical solution of the present application, it is expected to extract the dynamic change features of the working face in the time dimension and the multi-scale dynamic change features of the pitch angle in the time dimension based on the three-dimensional model of the working face using artificial intelligence control technology based on deep learning, and to represent the responsiveness of the three-dimensional model change of the working face to the pitch angle change with the responsiveness estimation of the pitch angle feature vector and the working face change feature vector, so as to control the pitch angle at the current time point and realize automatic excavation control based on the three-dimensional model. That is, by applying artificial intelligence technology to automatic excavation of the excavator, an automatic excavation control scheme based on the three-dimensional model is constructed to improve the efficiency and quality of excavation.

[0064] Specifically, in the technical solution of the present application, first, the three-dimensional model of the working face at multiple predetermined time points in a predetermined time period and the pitch angle values at the multiple predetermined time points are obtained. In particular, in one specific example of the present application, the three-dimensional model of the working face can be obtained by scanning with a laser scanner.

[0065] Specifically, in step S120, the three-dimensional model of the working face at the multiple predetermined time points is respectively input into a first convolutional neural network model using a spatial attention mechanism to obtain multiple working face feature matrices. Then, in order to accurately control the pitch angle of the excavator in real time according to the actual working face during the operation of the excavator, it is necessary to focus on the real-time dynamic change feature information of the operating surface in contact with the excavator in the extraction of the dynamic change features of the three-dimensional model of the working face, and to filter out the remaining irrelevant feature information for pitch angle control. Therefore, in the technical solution of the present application, the three-dimensional model of the working face at the multiple predetermined time points is further input into the first convolutional neural network model using the spatial attention mechanism for feature mining to extract feature information focusing on the operating surface in contact with the excavator in the three-dimensional model at the multiple predetermined time points, thereby obtaining multiple working face feature matrices.

[0066] It should be understood that attention mechanism is a data processing method in machine learning, which is widely used in various types of machine learning tasks such as natural language processing, image recognition and speech recognition. On the one hand, attention mechanism is to enable the network to automatically learn the places that need to be paid attention to in the picture or text sequence; on the other hand, attention mechanism generates a mask through the operation of the neural network, and the weights of the values on the mask. Generally speaking, the spatial attention mechanism takes the average of different channels of the same pixel point, and then obtains the spatial features through some convolution and upsampling operations. The pixel points of each layer of the spatial features are assigned different weights.

[0067] More specifically, in the embodiments of the present application, each layer of the first convolutional neural network model using a spatial attention mechanism respectively performs the following operations on the input data during the forward propagation of the layer: performing convolution processing on the input data to generate a convolution feature map; performing pooling processing on the convolution feature map to generate a pooled feature map; performing nonlinear activation on the pooled feature map to generate an activated feature map; calculating the mean of each position of the activated feature map along the channel dimension to generate a spatial feature matrix; calculating the class Softmax function value of each position in the spatial feature matrix to obtain a spatial score matrix; and calculating the point-by-point multiplication of the spatial feature matrix and the spatial score matrix to obtain a feature matrix; wherein the feature matrix output by the last layer of the first convolutional neural network model using a spatial attention mechanism is the plurality of work surface feature matrices.

[0068] The first convolutional neural network model using a spatial attention mechanism extracts feature information focused on the operating surface in contact with the working of the excavator in the three-dimensional model at a plurality of predetermined time points, and filters out the remaining useless interference feature information irrelevant to the luffing angle control, which is conducive to improving the accuracy of the classification structure.

[0069] Specifically, in step S130, the plurality of work surface feature matrices are arranged into a three-dimensional input tensor and then processed by a second convolutional neural network model using a three-dimensional convolution kernel to obtain a work surface change feature vector. Then, considering that the plurality of work surface feature matrices are feature distribution information of the three-dimensional model of the work surface focused on the operating surface of the excavator at a plurality of predetermined time points within the predetermined time period, in order to extract the dynamic feature distribution information thereof within the predetermined time period, the plurality of work surface feature matrices need to be further arranged into a three-dimensional input tensor and then processed by a second convolutional neural network model using a three-dimensional convolution kernel to obtain a work surface change feature vector.

[0070] In particular, here, the convolution kernel of the second convolutional neural network model is a three-dimensional convolution kernel, which has W (width), H (height), and C (channel dimension). In the technical solution of the present application, the channel dimension of the three-dimensional convolution kernel corresponds to the time dimension of the three-dimensional input tensor, so that when three-dimensional convolution coding is performed, the dynamic distribution feature of the work surface of the three-dimensional model along the time dimension can be extracted. It should be understood that by increasing the time dimension in the input of the neural network, the neural network can simultaneously extract time and spatial features for behavior recognition and video processing.

[0071] Further, the second convolutional neural network model using a three-dimensional convolution kernel is used to obtain the working face change feature vector after the plurality of working face feature matrices are arranged into a three-dimensional input tensor, comprising: inputting the three-dimensional input tensor into the second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature map; and performing global mean pooling on each feature matrix of the working face change feature map along the channel dimension to obtain a working face change feature vector.

[0072] That is, the plurality of working face feature matrices extracted by the second convolutional neural network model using a three-dimensional convolution kernel are the feature distribution information of the excavator operating surface at a plurality of predetermined time points within the predetermined time period.

[0073] Specifically, in step S140, the pitch angle values at the plurality of predetermined time points are arranged into a pitch angle input vector, and then a multi-scale neighborhood feature extraction module is used to obtain a pitch angle feature vector. Further, since the pitch angle values have volatility and uncertainty in the time dimension, and have different mode change features within the predetermined time period, in order to accurately extract the dynamic change feature information of the pitch angle values, the pitch angle values at the plurality of predetermined time points are further arranged into a pitch angle input vector, and then encoded in the multi-scale neighborhood feature extraction module to extract multi-scale neighborhood correlation features of the pitch angle values at different time spans, thereby obtaining a pitch angle feature vector.

[0074] More specifically, in the embodiments of the present application, Figure 4 For the automatic excavation control method based on a three-dimensional model according to the embodiments of the present application, a flow chart of arranging the pitch angle values at the plurality of predetermined time points into a pitch angle input vector and then using a multi-scale neighborhood feature extraction module to obtain a pitch angle feature vector is shown as Figure 4 The pitch angle values at the plurality of predetermined time points are arranged into a pitch angle input vector, and then a multi-scale neighborhood feature extraction module is used to obtain a pitch angle feature vector, comprising: S210, inputting the pitch angle input vector into a first convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain a first scale pitch angle feature vector, wherein the first convolutional layer has a first one-dimensional convolution kernel with a first length; S220, inputting the pitch angle input vector into a second convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain a second scale pitch angle feature vector, wherein the second convolutional layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and S230, concatenating the first scale pitch angle feature vector and the second scale pitch angle feature vector to obtain the pitch angle feature vector.

[0075] Further, a first convolution layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolution layers is used to perform one-dimensional convolution coding on the pitch angle input vector to obtain the first scale pitch angle feature vector according to the following formula:

[0076]

[0077] wherein a is the width of the first convolution kernel in the x direction, F(a) is the first convolution kernel parameter vector, G(x-a) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the pitch angle input vector.

[0078] Further, a second convolution layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolution layers is used to perform one-dimensional convolution coding on the pitch angle input vector to obtain the second scale pitch angle feature vector according to the following formula:

[0079]

[0080] wherein b is the width of the second convolution kernel in the x direction, F(b) is the second convolution kernel parameter vector, G(x-b) is the local vector matrix operated with the convolution kernel function, m is the size of the second convolution kernel, and X represents the pitch angle input vector.

[0081] In this way, the multi-scale neighborhood correlation features of the pitch angle value at different time spans can be extracted, so that the classification result is more accurate.

[0082] Specifically, in step S150, the responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector is calculated to obtain a classification feature matrix. Then, the responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector is calculated again, so as to represent the responsiveness correlation feature information between the dynamic change features of the three-dimensional model of the working face and the multi-scale change features of the pitch angle, and to use the same as a classification feature matrix for classification processing in a classifier to obtain a classification result indicating whether the pitch angle at the current time point should be increased or decreased. In this way, the pitch angle at the current time point can be controlled in real time.

[0083] Further, the responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector is calculated according to the following formula to obtain a classification feature matrix:

[0084]

[0085] wherein V1 represents the pitch angle feature vector, V2 represents the working face change feature vector, and M represents the classification feature matrix, represents matrix multiplication.

[0086] Specifically, in step S160, the classification feature matrix is passed through a classifier to obtain a classification result, which is used to indicate that the pitch angle at the current time point should be increased or should be decreased. In this way, the pitch angle at the current time point can be accurately controlled in real time, so as to improve the efficiency and quality of the excavator's excavation while ensuring the safety of the excavation.

[0087] More specifically, in the embodiments of the present application, Figure 5 For the automatic excavation control method based on a three-dimensional model according to the embodiments of the present application, a flowchart of the process of passing the classification feature matrix through a classifier to obtain a classification result is shown in Figure 5 As shown, the process of passing the classification feature matrix through a classifier to obtain a classification result, which is used to indicate that the pitch angle at the current time point should be increased or should be decreased, includes: S310 expanding the classification feature matrix into a classification feature vector according to a row vector or a column vector; S320 using multiple fully connected layers of the classifier to perform fully connected coding on the classification feature vector to obtain an encoded classification feature vector; and S330 passing the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

[0088] In a specific embodiment of the present application, the classification feature matrix is processed by the classifier using the following formula to obtain the classification result;

[0089] wherein the formula is: softmax{(W n ,B n ):…:(W1,B1)|Project(F)} where Project(F) represents projecting the classification feature matrix into a vector, W1 to W n are weight matrices, and B1 to B n are bias vectors.

[0090] Further, the automatic excavation control method based on a three-dimensional model further includes a training step: training the first convolutional neural network model using a spatial attention mechanism, the second convolutional neural network model using a three-dimensional convolution kernel, and the classifier. Figure 6 For the automatic excavation control method based on a three-dimensional model according to the embodiments of the present application, a flowchart of the training step is shown in Figure 6As shown, the training step includes: S410, obtaining training data, the training data including a training three-dimensional model of the working face at a plurality of predetermined time points within a predetermined time period, training pitch angle values at the plurality of predetermined time points, and a real value of whether the pitch angle at the current time point should be increased or decreased; S420, inputting the training three-dimensional model of the working face at the plurality of predetermined time points into the first convolutional neural network model using a spatial attention mechanism respectively to obtain a plurality of training working face feature matrices; S430, arranging the plurality of training working face feature matrices into a training three-dimensional input tensor and then passing through the second convolutional neural network model using a three-dimensional convolution kernel to obtain a training working face change feature vector; S440, arranging the training pitch angle values at the plurality of predetermined time points into a training pitch angle input vector and then passing through the multi-scale neighborhood feature extraction module to obtain a training pitch angle feature vector; S450, calculating a responsiveness estimation of the training pitch angle feature vector with respect to the training working face change feature vector to obtain a training classification feature matrix; S460, passing the training classification feature matrix through the classifier to obtain a classification loss function value; S470, calculating a sequence-to-sequence response rule internalization learning loss function value based on a distance between the training pitch angle feature vector and the training working face change feature vector; and S480, calculating a weighted sum of the classification loss function value and the sequence-to-sequence response rule internalization learning loss function value as a loss function value to train the first convolutional neural network model using a spatial attention mechanism, the second convolutional neural network model using a three-dimensional convolution kernel, and the classifier.

[0091] In particular, in the technical solution of the present application, when calculating the responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector, since the working face change feature vector represents the channel dimension distribution of the working face change feature map, the distribution of the initial three-dimensional model of the working face in the time sequence direction deviates on its feature distribution, and the pitch angle feature vector still represents the multi-scale time sequence correlation of the pitch angle values at the plurality of predetermined time points.

[0092] Therefore, in order to improve the calculation accuracy of the responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector, it is necessary to restore the causal relationship between the pitch angle feature vector and the working face change feature vector.

[0093] Based on this, a sequence-to-sequence response rule internalization learning loss function is introduced, that is, the sequence-to-sequence response rule internalization learning loss function value is calculated based on the distance between the training pitch angle feature vector and the training working face change feature vector as follows: wherein the formula is:

[0094]

[0095]

[0096]

[0097] wherein V1 and V2 are the training pitch angle feature vector and the training working face change feature vector respectively, and W1 and W2 are the weight matrix of the classifier for the training pitch angle feature vector and the training working face change feature vector respectively, V1 + represents the first activation feature vector, V2 + represents the second activation feature vector, represents the loss function value, ReLU(·) represents the ReLU activation function, Sigmoid(·) represents the Sigmoid activation function, represents the matrix multiplication, d(·,·) represents the Euclidean distance between two vectors.

[0098] That is, through the squeeze-and-excitation channel attention mechanism of the weight matrix of the classifier for the different sequences of the pitch angle feature vector V1 and the working face change feature vector V2, the enhanced discriminative ability between the sequences of the feature vectors is obtained. By training the network with this loss function, the recovery of the cause-and-effect feature with better discriminability between the response sequences can be realized, so as to internalize the learning of the cause-and-effect response rule between the sequences of the pitch angle feature vector V1 and the working face change feature vector V2, thereby enhancing the accuracy of the responsiveness calculation between the pitch angle feature vector V1 and the working face change feature vector V2 as the feature sequence. In this way, the pitch angle at the current time point can be accurately controlled in real time, so as to improve the efficiency and quality of the excavator excavation, while ensuring the safety of the excavation.

[0099] In summary, the three-dimensional model-based automatic excavation control method based on the embodiments of the present application is illustrated, which extracts the dynamic change features of the three-dimensional model of the working face at multiple predetermined time points within a predetermined time period in the time dimension through the first convolutional neural network model using the spatial attention mechanism and the second convolutional neural network model using the three-dimensional convolution kernel; the multi-scale dynamic change features of the pitch angle value in the time dimension are extracted through the multi-scale neighborhood feature extraction module, and the responsiveness estimation of the two is used to represent the responsiveness correlation feature of the change of the three-dimensional model of the working face to the change of the pitch angle, so as to control the pitch angle at the current time point. In this way, the pitch angle at the current time point can be accurately controlled in real time, so as to improve the efficiency and quality of the excavator excavation.

[0100] Exemplary system

[0101] Figure 7A block diagram of an automatic excavation control system based on a three-dimensional model according to an embodiment of the present application. As shown in Figure 7 The automatic excavation control system based on a three-dimensional model 100 according to an embodiment of the present application includes: a data acquisition module 110 configured to acquire a three-dimensional model of a working face at a plurality of predetermined time points within a predetermined time period and an inclination angle value at the plurality of predetermined time points; a working face feature extraction module 120 configured to input the three-dimensional model of the working face at the plurality of predetermined time points into a first convolutional neural network model using a spatial attention mechanism respectively to obtain a plurality of working face feature matrices; a working face change feature extraction module 130 configured to arrange the plurality of working face feature matrices into a three-dimensional input tensor and then pass the three-dimensional input tensor through a second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature vector; an inclination angle feature extraction module 140 configured to arrange the inclination angle value at the plurality of predetermined time points into an inclination angle input vector and then pass the inclination angle input vector through a multi-scale neighborhood feature extraction module to obtain an inclination angle feature vector; a responsiveness estimation module 150 configured to calculate a responsiveness estimation of the inclination angle feature vector relative to the working face change feature vector to obtain a classification feature matrix; and an inclination angle control result generation module 160 configured to pass the classification feature matrix through a classifier to obtain a classification result, where the classification result is used to indicate whether the inclination angle at a current time point should be increased or decreased.

[0102] In an embodiment of the present application, in the automatic excavation control system based on a three-dimensional model 100 described above, the three-dimensional model of the working face is obtained by a laser scanner.

[0103] In an embodiment of the present application, in the automatic excavation control system based on a three-dimensional model 100 described above, the working face feature extraction module is configured to: in a forward propagation process of each layer of the first convolutional neural network model using a spatial attention mechanism, perform the following operations on input data respectively: perform convolution processing on the input data to generate a convolution feature map; perform pooling processing on the convolution feature map to generate a pooled feature map; perform nonlinear activation on the pooled feature map to generate an activated feature map; calculate a mean value of each position along a channel dimension of the activated feature map to generate a spatial feature matrix; calculate a Softmax function value of each position in the spatial feature matrix to obtain a spatial score matrix; and calculate a point-by-point multiplication of the spatial feature matrix and the spatial score matrix to obtain a feature matrix; and the feature matrix output by the last layer of the first convolutional neural network model using a spatial attention mechanism is the plurality of working face feature matrices.

[0104] In an embodiment of the present application, in the automatic excavation control system based on three-dimensional model 100 described above, the working face change feature extraction module comprises: a working face change feature extraction unit, configured to input the three-dimensional input tensor into the second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature map; and a global mean pooling unit, configured to perform global mean pooling on each feature matrix along the channel dimension of the working face change feature map to obtain a working face change feature vector.

[0105] In an embodiment of the present application, in the automatic excavation control system based on three-dimensional model 100 described above, the pitch angle feature extraction module comprises: a first scale pitch angle feature extraction unit, configured to input the pitch angle input vector into a first convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain a first scale pitch angle feature vector, wherein the first convolutional layer has a first one-dimensional convolution kernel with a first length; a second scale pitch angle feature extraction unit, configured to input the pitch angle input vector into a second convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain a second scale pitch angle feature vector, wherein the second convolutional layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and a concatenation unit, configured to concatenate the first scale pitch angle feature vector and the second scale pitch angle feature vector to obtain the pitch angle feature vector.

[0106] In an embodiment of the present application, in the automatic excavation control system based on three-dimensional model 100 described above, the first scale pitch angle feature extraction unit is configured to perform one-dimensional convolutional coding on the pitch angle input vector using the first convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain the first scale pitch angle feature vector according to the following formula: wherein the formula is:

[0107]

[0108] wherein a is the width of the first convolution kernel in the x direction, F(a) is the first convolution kernel parameter vector, G(x-a) is the local vector matrix operated with the convolution kernel function, w is the size of the first convolution kernel, and X represents the pitch angle input vector.

[0109] The second scale pitch angle feature extraction unit is configured to perform one-dimensional convolutional coding on the pitch angle input vector using the second convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain the second scale pitch angle feature vector according to the following formula: wherein the formula is:

[0110]

[0111] In an embodiment of the present application, in the automatic excavation control system 100 based on the three-dimensional model, the responsiveness estimation module is further configured to calculate the responsiveness estimation of the pitch angle feature vector relative to the working face change feature vector to obtain a classification feature matrix according to the following formula:

[0112]

[0113] wherein V1 represents the pitch angle feature vector, V2 represents the working face change feature vector, and M represents the classification feature matrix, represents matrix multiplication.

[0114] In an embodiment of the present application, in the automatic excavation control system 100 based on the three-dimensional model, the pitch angle control result generation module comprises: a feature matrix unfolding unit configured to unfold the classification feature matrix into a classification feature vector according to a row vector or a column vector; a fully connected coding unit configured to use a plurality of fully connected layers of the classifier to perform fully connected coding on the classification feature vector to obtain a coded classification feature vector; and a classification result generation unit configured to pass the coded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

[0115] In an embodiment of the present application, in the automatic excavation control system based on three-dimensional model 100 described above, a training module is further included for training the first convolutional neural network model using a spatial attention mechanism, the second convolutional neural network model using a three-dimensional convolution kernel, and the classifier; wherein the training module comprises: a training data acquisition module for acquiring training data, the training data comprising training three-dimensional models of the working face at a plurality of predetermined time points within a predetermined time period, training pitch angle values at the plurality of predetermined time points, and real values of whether the pitch angle at the current time point should be increased or decreased; a training working face feature extraction module for inputting the training three-dimensional models of the working face at the plurality of predetermined time points into the first convolutional neural network model using a spatial attention mechanism to obtain a plurality of training working face feature matrices; a training working face change feature extraction module for arranging the plurality of training working face feature matrices into a training three-dimensional input tensor and then passing through the second convolutional neural network model using a three-dimensional convolution kernel to obtain a training working face change feature vector; a training pitch angle feature extraction module for arranging the training pitch angle values at the plurality of predetermined time points into a training pitch angle input vector and then passing through the multi-scale neighborhood feature extraction module to obtain a training pitch angle feature vector; a training responsiveness estimation module for calculating the responsiveness estimation of the training pitch angle feature vector with respect to the training working face change feature vector to obtain a training classification feature matrix; a training pitch angle control result generation module for passing the training classification feature matrix through the classifier to obtain a classification loss function value; a calculation module for calculating a sequence-to-sequence response rule internalization learning loss function value based on the distance between the training pitch angle feature vector and the training working face change feature vector; and a training module for calculating the weighted sum of the classification loss function value and the sequence-to-sequence response rule internalization learning loss function value as a loss function value to train the first convolutional neural network model using a spatial attention mechanism, the second convolutional neural network model using a three-dimensional convolution kernel, and the classifier.

[0116] In an embodiment of the present application, in the automatic excavation control system based on three-dimensional model 100 described above, the training module is further configured to calculate the sequence-to-sequence response rule internalization learning loss function value according to the following formula:

[0117]

[0118]

[0119]

[0120] wherein V1 and V2 are the training pitch angle feature vector and the training working face change feature vector respectively, and W1 and W2 are the weight matrix of the classifier for the training pitch angle feature vector and the training working face change feature vector respectively, V1 + represents a first activation feature vector, V2 + represents a second activation feature vector, represents a loss function value, ReLU(·) represents a ReLU activation function, Sigmoid(·) represents a Sigmoid activation function, represents matrix multiplication, and d(·,·) represents the Euclidean distance between two vectors.

[0121] Here, those skilled in the art can understand that the specific functions and operations of each unit and module in the above-described three-dimensional model-based automatic excavation control system 100 have been described in detail above with reference to the three-dimensional model-based automatic excavation control method described above, and therefore repeated descriptions thereof will be omitted. Figures 1 to 6

[0122] As described above, the three-dimensional model-based automatic excavation control system 100 according to an embodiment of the present application can be implemented in various terminal devices, such as a server for a three-dimensional model-based automatic excavation control system, etc. In one example, the three-dimensional model-based automatic excavation control system 100 according to an embodiment of the present application can be integrated into a terminal device as a software module and / or a hardware module. For example, the three-dimensional model-based automatic excavation control system 100 can be a software module in an operating system of the terminal device, or can be an application program developed for the terminal device; of course, the three-dimensional model-based automatic excavation control system 100 can also be one of many hardware modules of the terminal device.

[0123] Alternatively, in another example, the three-dimensional model-based automatic excavation control system 100 and the terminal device can also be separate devices, and the three-dimensional model-based automatic excavation control system 100 can be connected to the terminal device through a wired and / or wireless network, and transmit interactive information in an agreed data format.

[0124] The above describes the basic principles of the present application in connection with specific embodiments, but it should be noted that the advantages, advantages, effects, etc. mentioned in the present application are only examples and are not limiting, and these advantages, advantages, effects, etc. cannot be considered as the various embodiments of the present application must have. In addition, the above specific details disclosed are only for the purpose of example and for the purpose of understanding, and are not limiting, and the above details do not limit the present application to the above specific details.

[0125] ​The block diagrams of the devices, apparatuses, equipment, systems referred to in this application are only illustrative examples and are not intended to require or imply that the connection, arrangement, configuration must be as shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include," "contain," "have," and the like are open-ended words that are intended to mean "including but not limited to," and are to be used interchangeably. The words "or" and "and" as used herein are intended to mean "and / or," and are to be used interchangeably. The word "such as" as used herein is intended to mean "such as but not limited to," and is to be used interchangeably.

[0126] It is also important to note that the devices, apparatuses, and methods of the present application can be embodied in a variety of other forms, including devices, apparatuses, and methods that are not specifically disclosed herein. Thus, the present application is not limited to the aspects described herein, but rather the general principles defined herein can be applied to other aspects as well without departing from the scope of the present application.

[0127] The above description of disclosed aspects is given for illustrative purposes only and is not intended to limit the scope of the application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0128] The above description has been given for illustrative and descriptive purposes. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for automatic excavation control based on a three-dimensional model, characterized by, The method comprises the following steps: obtaining a three-dimensional model of a working face at a plurality of predetermined time points within a predetermined time period and pitch angle values at the plurality of predetermined time points; inputting the three-dimensional model of the working face at the plurality of predetermined time points into a first convolutional neural network model using a spatial attention mechanism respectively to obtain a plurality of working face feature matrices; arranging the plurality of working face feature matrices into a three-dimensional input tensor and then inputting the three-dimensional input tensor into a second convolutional neural network model using a three-dimensional convolution kernel to obtain a working face change feature vector; arranging the pitch angle values at the plurality of predetermined time points into a pitch angle input vector and then inputting the pitch angle input vector into a multi-scale neighborhood feature extraction module to obtain a pitch angle feature vector; calculating a responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector to obtain a classification feature matrix; and inputting the classification feature matrix into a classifier to obtain a classification result, wherein the classification result is used to indicate whether the pitch angle at a current time point should be increased or decreased.

2. The automatic excavation control method based on a three-dimensional model according to claim 1, characterized by, The three-dimensional model of the working face is obtained by scanning with a laser scanner.

3. The automatic excavation control method based on a three-dimensional model according to claim 2, characterized by, The inputting the three-dimensional model of the working face at the plurality of predetermined time points into the first convolutional neural network model using the spatial attention mechanism to obtain the plurality of working face feature matrices comprises that each layer of the first convolutional neural network model using the spatial attention mechanism respectively performs the following operations on the input data in a forward propagation process of the layer: performing convolution processing on the input data to generate a convolution feature map; performing pooling processing on the convolution feature map to generate a pooling feature map; performing nonlinear activation on the pooling feature map to generate an activation feature map; calculating a mean value of each position of the activation feature map along a channel dimension to generate a spatial feature matrix; calculating a Softmax function value of each position in the spatial feature matrix to obtain a spatial score matrix; and calculating a point-by-point multiplication of the spatial feature matrix and the spatial score matrix to obtain a feature matrix; wherein the feature matrix output by the last layer of the first convolutional neural network model using the spatial attention mechanism is the plurality of working face feature matrices.

4. The automatic excavation control method based on a three-dimensional model according to claim 3, characterized by, The inputting the plurality of working face feature matrices into the second convolutional neural network model using the three-dimensional convolution kernel to obtain the working face change feature vector comprises: inputting the three-dimensional input tensor into the second convolutional neural network model using the three-dimensional convolution kernel to obtain a working face change feature map; and performing global mean pooling on each feature matrix of the working face change feature map along the channel dimension to obtain the working face change feature vector.

5. The automatic excavation control method based on a three-dimensional model according to claim 4, characterized by, The inputting the pitch angle values at the plurality of predetermined time points into the multi-scale neighborhood feature extraction module to obtain the pitch angle feature vector comprises: inputting the pitch angle input vector into a first convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolution layers to obtain a first-scale pitch angle feature vector, wherein the first convolutional layer has a first one-dimensional convolution kernel with a first length; inputting the pitch angle input vector into a second convolutional layer of a multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain a second scale pitch angle feature vector, wherein the second convolutional layer has a second one-dimensional convolutional kernel of a second length, and the first length is different from the second length; and concatenating the first scale pitch angle feature vector and the second scale pitch angle feature vector to obtain the pitch angle feature vector.

6. The automatic excavation control method based on a three-dimensional model according to claim 5, characterized in that, the inputting the pitch angle input vector into the first convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain a first scale pitch angle feature vector comprises using the first convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to one-dimensionally convolutionally encode the pitch angle input vector according to a first formula to obtain the first scale pitch angle feature vector; wherein the first formula is: ； wherein, a is a width of the first convolution kernel in the x direction, is a first convolution kernel parameter vector, is a local vector matrix to be operated with the convolution kernel function, w is a size of the first convolution kernel, X denotes the pitch angle input vector; the inputting the pitch angle input vector into the second convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to obtain a second scale pitch angle feature vector comprises using the second convolutional layer of the multi-scale neighborhood feature extraction module comprising a plurality of parallel one-dimensional convolutional layers to one-dimensionally convolutionally encode the pitch angle input vector according to a second formula to obtain the second scale pitch angle feature vector; wherein the second formula is: ； wherein, b is a width of the second convolution kernel in the x direction, is a second convolution kernel parameter vector, is a local vector matrix to be operated with the convolution kernel function, m is a size of the second convolution kernel, X denotes the pitch angle input vector.

7. The automatic excavation control method based on a three-dimensional model according to claim 6, characterized by, the calculating the responsiveness estimate of the pitch angle feature vector with respect to the working face change feature vector to obtain a classification feature matrix further comprises calculating the responsiveness estimate of the pitch angle feature vector with respect to the working face change feature vector according to a third formula to obtain a classification feature matrix; wherein the third formula is: ； wherein denotes the pitch angle feature vector, denotes the working face change feature vector, denotes the classification feature matrix, denotes matrix multiplication.

8. The automatic excavation control method based on a three-dimensional model according to claim 7, characterized by, the passing the classification feature matrix through a classifier to obtain a classification result, the classification result being used to indicate whether the pitch angle at the current time point should be increased or decreased, comprises: unfolding the classification feature matrix into a classification feature vector according to a row vector or a column vector; fully connecting the classification feature vector using a plurality of fully connected layers of the classifier to obtain an encoded classification feature vector; and passing the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

9. The automatic excavation control method based on a three-dimensional model according to claim 8, characterized by, Further comprising a training step of training the first convolutional neural network model using a spatial attention mechanism, the second convolutional neural network model using a three-dimensional convolutional kernel, and the classifier; wherein the training step comprises: obtaining training data, the training data comprising training three-dimensional models of working faces at a plurality of predetermined time points within a predetermined time period, training pitch angle values at the plurality of predetermined time points, and true values of whether the pitch angle at the current time point should be increased or decreased; inputting the training three-dimensional models of working faces at the plurality of predetermined time points into the first convolutional neural network model using a spatial attention mechanism respectively to obtain a plurality of training working face feature matrices; arranging the plurality of training working face feature matrices into a training three-dimensional input tensor to obtain a training working face change feature vector through the second convolutional neural network model using a three-dimensional convolution kernel; arranging the training pitch angle values at the plurality of predetermined time points into a training pitch angle input vector to obtain a training pitch angle feature vector through the multi-scale neighborhood feature extraction module; calculating a responsiveness estimation of the training pitch angle feature vector with respect to the training working face change feature vector to obtain a training classification feature matrix; passing the training classification feature matrix through a classifier to obtain a classification loss function value; calculating a sequence-to-sequence response rule internalization learning loss function value based on a distance between the training pitch angle feature vector and the training working face change feature vector; and calculating a weighted sum of the classification loss function value and the sequence-to-sequence response rule internalization learning loss function value as a loss function value to train the first convolutional neural network model using a spatial attention mechanism, the second convolutional neural network model using a three-dimensional convolution kernel, and the classifier.

10. The automatic excavation control method based on a three-dimensional model according to claim 9, characterized by, the sequence-to-sequence response rule internalization learning loss function value is calculated based on a distance between the training pitch angle feature vector and the training working face change feature vector according to a fourth formula; wherein the fourth formula is: ； wherein, and are the training pitch angle feature vector and the training face change feature vector, respectively, and and are the weight matrices of the classifier for the training pitch angle feature vector and the training face change feature vector, respectively, denotes a first activation feature vector, denotes a second activation feature vector, denotes a loss function value, denotes an activation function, denotes an activation function, denotes a matrix multiplication, denotes the Euclidean distance between two vectors.

11. An automatic excavation control system based on a three-dimensional model, characterized by, comprising: a data acquisition module configured to acquire a three-dimensional model of a working face at a plurality of predetermined time points within a predetermined time period and pitch angle values at the plurality of predetermined time points; a working face feature extraction module configured to input the three-dimensional model of the working face at the plurality of predetermined time points into a first convolutional neural network model using a spatial attention mechanism to obtain a plurality of working face feature matrices; a working face change feature extraction module configured to arrange the plurality of working face feature matrices into a three-dimensional input tensor to obtain a working face change feature vector through a second convolutional neural network model using a three-dimensional convolution kernel; a pitch angle feature extraction module configured to arrange the pitch angle values at the plurality of predetermined time points into a pitch angle input vector to obtain a pitch angle feature vector through a multi-scale neighborhood feature extraction module; a responsiveness estimation module configured to calculate a responsiveness estimation of the pitch angle feature vector with respect to the working face change feature vector to obtain a classification feature matrix; and a pitch angle control result generation module configured to pass the classification feature matrix through a classifier to obtain a classification result, the classification result being used to indicate whether the pitch angle at the current time point should be increased or decreased.

Images