Deep learning based shield muck three-dimensional point cloud segmentation and volume calculation method

Abstract

The application provides a shielded spoil three-dimensional point cloud segmentation and volume calculation method based on deep learning, and belongs to the technical fields of deep learning and shield construction, and the method comprises the following steps: S1, point cloud data acquisition, preprocessing and sample labeling; S2, deep learning segmentation model construction and training; S3, spoil point cloud segmentation; and S4, spoil volume calculation. The application realizes efficient interaction of point cloud attention in four serialization modes, and avoids the influence of point cloud disorder on calculation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and shield technology, and particularly relates to a shield muck three-dimensional point cloud segmentation and volume calculation method based on deep learning. BACKGROUND

[0002] The shield method has the advantages of high safety, high construction efficiency and small environmental impact, and has become an important construction technology for modern urban rail transit construction. However, when passing through complex geological environments such as water-rich sand layers and uneven hard and soft strata, it is easy to cause over-excavation of the shield, cause the lining wall to be empty behind, and ultimately cause ground subsidence. Ground subsidence directly affects urban traffic and may even damage existing building structures, which has a very adverse impact on urban construction. The existing shield muck volume measurement method mainly relies on manual visual inspection of the loading degree of the muck bucket to estimate the muck volume. This method highly depends on manual operation and judgment, and has high subjectivity and uncertainty.

[0003] Three-dimensional point cloud data usually contains a large number of noise points and background points, which will introduce errors into volume calculation. High-precision volume calculation requires obtaining pure muck surface point cloud and removing background points and noise points. However, the distribution of the muck surface is diverse, and there is no general distribution rule or spatial feature. Existing clustering and regression algorithms cannot accurately segment the muck surface point cloud and the background point cloud.

[0004] Early point cloud computing ideas are to voxelize point clouds into dense grids and use 3D convolution on voxel grids. However, due to the cubic growth of the number of voxels with resolution, this strategy brings huge computational and memory overhead. Drawing on the achievements of convolutional neural networks in visual tasks, some point cloud segmentation methods follow this approach, projecting 3D point clouds onto different image planes, and then using 2D convolutional neural networks to extract features. However, point clouds have the characteristics of unordered and non-structured distribution, and the standard convolutional neural network does not work well for them. With the remarkable achievements of Transformer in natural language processing, it is found that the attention mechanism has a unique fit with the structure of point cloud data. The calculation principle of self-attention lies in obtaining the correlation between elements in the set, and point cloud is a set of elements with location attributes. However, the amount of point cloud data is huge, and direct global attention calculation on the dataset exceeds the processing capacity of general-purpose computers, and some methods successfully improve the performance of point cloud models by converting global attention to local attention. However, the muck point cloud has a wide range of scales and a dense point distribution, which requires the content of the attention window to be flexible enough and the interaction between the windows to be efficient enough to expand the receptive field of local attention and maintain the global relevance of point cloud features. However, there are still two challenges in applying existing deep learning point cloud segmentation methods to shield construction. First, the construction site usually does not have large-scale computing servers, and the GPU performance of the computer is low, which limits the size of the local attention window. Second, the muck point cloud has a wide range of spatial scales and a diverse spatial distribution. This requires the content of the attention window to be rich enough, and the attention windows need to have efficient interaction capabilities to expand the receptive field of local attention and maintain the global relevance of point cloud features. SUMMARY

[0005] The application provides a shield muck three-dimensional point cloud segmentation and volume calculation method based on deep learning, which realizes efficient interaction of point cloud attention by using four serialization methods, avoiding the influence of point cloud disorder on calculation.

[0006] To achieve the above purpose, the application adopts the following technical solutions:

[0007] The shield muck three-dimensional point cloud segmentation and volume calculation method based on deep learning comprises:

[0008] S1. Point cloud data acquisition, preprocessing and sample labeling; the three-dimensional point cloud data containing the muck bucket and the muck surface is collected by scanning the muck transport track car with a line laser radar; the three-dimensional point cloud data is automatically determined and stored for processing to obtain initial muck point cloud data; based on the initial muck point cloud data, the muck surface points, background points and noise points are manually labeled to obtain the labeled muck point cloud sample dataset;

[0009] S2. Deep learning segmentation model construction and training; a deep learning-based slag segmentation model containing an encoder-decoder structure is constructed, the encoder contains a down-sampling module and a Transformer block, the Transformer block contains a normal vector relative position encoding module and a serialized point attention interaction module, and the decoder contains an up-sampling module and an Upsample block; the deep learning-based slag segmentation model is trained using the labeled slag point cloud sample data set, and a trained deep learning-based slag segmentation model is obtained;

[0010] S3. Slag point cloud segmentation; the initial slag point cloud data is input into the trained deep learning-based slag segmentation model, and is sequentially subjected to data initialization, feature extraction, down-sampling, and up-sampling processing to obtain slag surface point cloud data with background points and noise points removed;

[0011] S4. Slag volume calculation; taking a single slag hopper as a calculation unit, the slag surface point cloud data with background points and noise points removed is divided into unit cubes and the volume is calculated, and the volume result is corrected in combination with the actual size of the slag hopper to obtain the actual loading volume of the slag.

[0012] In the specification, the automatic determination storage process in S1 is specifically: by comparing the z-axis coordinates of each point in the point cloud with the z-axis coordinates of the top of the lining, it is determined whether it is a track car loaded with slag, and only the three-dimensional point cloud data corresponding to the track car loaded with slag is stored.

[0013] In the specification, the manual labeling of slag surface points, background points and noise points in S1 is specifically: based on the point cloud coordinates and spatial distribution characteristics, the point cloud is divided into slag surface points, background points and noise points, and class labels are added to different categories of points.

[0014] In the specification, the encoder-decoder structure in S2 is specifically: the encoder contains 4 down-sampling stages, 2 Transformer blocks are used for the first two down-sampling stages, 3 Transformer blocks are used for the last two down-sampling stages, and the feature channels are 16, 32, 64, 128, and 256 in turn; the decoder contains 4 up-sampling stages, 2 Upsample blocks are used for each up-sampling stage, the feature channels are 256, 128, 64, and 32 in turn, and finally a SubMConv3d block module and a fully connected layer are used to output the segmentation result.

[0015] In the specification, the specific processing process of the normal vector relative position encoding module in S2 is: receiving the point cloud feature map and the point cloud normal vector, mapping the point cloud normal vector into a feature vector through an MLP layer, fusing the point cloud feature map through a cross attention module, and then sequentially passing through a 3D sparse convolution, a Linear layer, and a LayerNorm layer to form a relative position encoding vector. The normal vector feature and the relative position encoding vector are fused through a jump connection to output the processing result.

[0016] In the specification, the specific processing process of the serialized point attention interaction module in S2 is: adjusting the input feature map through a Linear layer and a LayerNorm layer, serializing the feature channel through four space-filling curves of Z-Index, Hilbert, Spiral, and Cantor, compressing the serialized feature channel to the original channel number, calculating attention through an SMHA layer, and then sequentially passing through a Linear layer, a LayerNorm layer, and an MLP layer to output the processing result.

[0017] In the specification, the down-sampling module of the encoder adopts a grid down-sampling method, specifically: dividing the point cloud space into a three-dimensional grid, aggregating multiple points in the grid into a single representative point through averaging, maximum value taking, or random selection, and then sequentially processing through a Linear layer, a BatchNorm layer, and a GELU layer to realize down-sampling.

[0018] In the specification, the up-sampling module of the decoder adopts a grid up-sampling method, specifically: adjusting the channel of the input feature map through a Linear layer, reducing the grid scale and increasing the number of grids, introducing the saved feature copy in the down-sampling stage, fusing the up-sampling feature and the feature copy through element addition, and then processing through a BatchNorm layer and a GELU layer to realize up-sampling.

[0019] In the specification, the specific process of training the deep learning-based slag soil segmentation model using the labeled slag soil point cloud sample dataset in S2 includes: data augmentation of the sample data, which includes random rotation, scaling, and adding Gaussian noise; dividing the augmented sample data into a training set and a validation set, using a cross-entropy loss function as the optimization objective, and iteratively training until the model converges.

[0020] In the specification, the slag surface point cloud data of S4 after removing background points and noise is divided into unit cubes and the volume is calculated, specifically: the area of the unit cube is determined according to the number of slag surface points, the height of the unit cube is determined according to the coordinates of the points, the volume of all unit cubes is counted and summed to obtain the initial volume of the slag surface point cloud to the construction bottom surface; the virtual plane is constructed according to the actual size of the slag bucket to correct the volume result, specifically: a virtual plane lower than the real bottom surface is constructed according to the actual length, width and volume of the slag bucket, the volume difference between the initial volume and the virtual plane and the real bottom surface is calculated, and the initial volume is subtracted by the volume difference to obtain the actual loading volume of the slag.

[0021] In summary, the present application has at least the following beneficial effects:

[0022] There is no point cloud scanning and data processing method for slag transportation rail cars in the prior art, and the present application fills this technical gap, greatly improving the accuracy and timeliness of the shield slag volume measurement, and the data processing speed is improved by 66 times compared with manual data analysis.

[0023] Compared with the general point cloud segmentation algorithm, the segmentation accuracy of the slag point cloud data is improved by 13%.

[0024] Compared with the commonly used point cloud volume calculation method, such as the triangulation method and the slicing method, the special slag volume calculation method has higher adaptability to scanning data, and the average calculation error is reduced by 3.12% compared with the existing method, and the calculation speed is improved by 167%.

[0025] The NCPE calculation module of the present application introduces the normal vector of the point cloud through cross attention based on the original relative position encoding calculation method, improves the effect of relative position encoding, enhances the spatial information expression of the point cloud, reduces the spatial information loss of the point cloud features at each down-sampling stage, and improves the model precision.

[0026] The SPAI calculation module of the present application applies a point cloud attention calculation method through four kinds of point cloud serialization fusion. Point cloud serialization avoids the influence of the disorder of point cloud on deep learning. Four kinds of sequence order fusion solve the problem that a single sequence order cannot completely adapt to the changing slag surface; in addition, different sequence order fusion can also enrich the receptive field of the local attention window, so that the Transformer module obtains better calculation effect. Based on this idea, when constructing the attention window, two different construction paths are adopted for the four kinds of sequence order, which further improves the interaction efficiency of the attention window, and finally achieves significant accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0028] Figure 1 The schematic diagram of the method for segmenting and calculating the volume of the three-dimensional point cloud of the shield spoil involved in the present application.

[0029] Figure 2 The schematic diagram of the three-dimensional point cloud scanning method for the shield spoil volume involved in the present application.

[0030] Figure 3 The schematic diagram of the point cloud segmentation model network structure based on deep learning involved in the present application.

[0031] Figure 4 The schematic diagram of the relative position encoding module NCPE structure combined with the normal vector involved in the present application.

[0032] Figure 5 The schematic diagram of the point cloud grid downsampling calculation module structure involved in the present application.

[0033] Figure 6 The schematic diagram of the point cloud grid downsampling calculation principle involved in the present application.

[0034] Figure 7 The schematic diagram of the point cloud grid upsampling calculation module structure involved in the present application.

[0035] Figure 8 The schematic diagram of the sparse convolution calculation module structure for point cloud involved in the present application.

[0036] Figure 9 The schematic diagram of the serialized point attention interaction calculation module structure involved in the present application.

[0037] Figure 10 The schematic diagram of the four kinds of space-filling curve paths used by the serialized point attention interaction module involved in the present application.

[0038] Figure 11 The schematic diagram of the point cloud data structure change process in the calculation of the serialized point attention interaction module involved in the present application.

[0039] Figure 12 The schematic diagram of the visualization result of the SMPN for segmenting the surface point cloud of the spoil involved in the present application.

[0040] Figure 13 FIG. 1 is a schematic diagram of a slag surface point cloud volume calculation method involved in the present application.

[0041] FIG. 1 is a schematic diagram of a slag surface point cloud volume calculation method involved in the present application. DETAILED DESCRIPTION

[0042] In the following, only certain exemplary embodiments are simply described. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the spirit or scope of the embodiments of the present application. Therefore, the drawings and the description are considered to be exemplary in nature rather than limiting.

[0043] The following disclosure provides many different embodiments, or examples, for implementing different structures of the embodiments of the present application. For the purpose of simplifying the disclosure of the embodiments of the present application, the components and settings of specific examples are described in the following. Of course, they are only examples, and the purpose is not to limit the embodiments of the present application. In addition, the embodiments of the present application can refer to the same reference numerals and / or reference letters in different examples, and such repetition is for the purpose of simplification and clarity, which does not indicate the relationship between the various embodiments and / or settings discussed.

[0044] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0045] As Figure 1 shown, the present embodiment provides a shield slag three-dimensional point cloud segmentation and volume calculation method based on deep learning, comprising:

[0046] S1. Point cloud data acquisition, preprocessing and sample annotation; the three-dimensional point cloud data containing the slag hopper 4 and the slag surface 5 is collected by scanning the passing slag transportation track vehicle through the line laser radar; the initial slag point cloud data is obtained by automatically determining and storing the three-dimensional point cloud data; based on the initial slag point cloud data, the slag surface points, background points and noise points are manually annotated to obtain the annotated slag point cloud sample data set;

[0047] S2. Deep learning segmentation model construction and training; a deep learning-based slag segmentation model containing an encoder-decoder structure is constructed, the encoder contains a down-sampling module and a Transformer block, the Transformer block contains a normal vector relative position encoding module and a serialized point attention interaction module, the decoder contains an up-sampling module and an Upsample block; the deep learning-based slag segmentation model is trained using the annotated slag point cloud sample data set to obtain the trained deep learning-based slag segmentation model;

[0048] S3. Slag point cloud segmentation; input the initial slag point cloud data into the trained deep learning-based slag segmentation model, sequentially pass through data initialization, feature extraction, down-sampling, and up-sampling processing, to obtain the slag surface point cloud data with background points and noise points removed;

[0049] S4. Slag volume calculation; taking a single slag bucket 4 as a calculation unit, based on the slag surface point cloud data with background points and noise points removed, divide the unit cubic and calculate the volume, combine the actual size of the slag bucket to construct a virtual plane to correct the volume result, and obtain the actual loading volume of the slag.

[0050] In some embodiments, the automatic determination in S1 specifically stores the three-dimensional point cloud data corresponding to the track car loaded with slag by comparing the z-axis coordinates of each point in the point cloud with the z-axis coordinates of the top of the lining.

[0051] In some embodiments, the manual labeling of slag surface points, background points and noise points in S1 specifically divides the point cloud into slag surface points, background points and noise points based on the point cloud coordinates and spatial distribution characteristics, and adds class labels to different categories of points.

[0052] In some embodiments, the encoder-decoder structure in S2 specifically includes 4 down-sampling stages, each of the first two down-sampling stages uses 2 Transformer blocks, each of the last two down-sampling stages uses 3 Transformer blocks, and the feature channels are 16, 32, 64, 128, and 256 in turn; the decoder includes 4 up-sampling stages, each of which uses 2 Upsample blocks, and the feature channels are 256, 128, 64, and 32 in turn, and finally outputs the segmentation result through a SubMConv3d block module and a fully connected layer.

[0053] In some embodiments, the normal vector relative position encoding module in S2 specifically processes as follows: receiving the point cloud feature map and the point cloud normal vector, mapping the point cloud normal vector to a feature vector through an MLP layer, fusing the point cloud normal vector feature and the point cloud feature map through a cross-attention module, and then sequentially passing through a 3D sparse convolution, a Linear layer, and a LayerNorm layer to form a relative position encoding vector, fusing the normal vector feature and the relative position encoding vector through a skip connection, and outputting the processing result.

[0054] In some embodiments, the specific processing procedure of the serialization point attention interaction module in S2 is as follows: after the input feature map is adjusted by a Linear layer and a LayerNorm layer, the feature map is serialized by Z-Index, Hilbert, Spiral and Cantor four kinds of space-filling curves, the serialized feature channels are compressed to the original channel number, the attention is calculated by an SMHA layer, and then the processing result is output by sequentially passing through a Linear layer, a LayerNorm layer and an MLP layer.

[0055] In some embodiments, the down-sampling module of the encoder adopts a grid down-sampling method, specifically: the point cloud space is divided into a three-dimensional grid, multiple points in the grid are aggregated into a single representative point by averaging, taking the maximum value or random selection, and then sequentially processed by a Linear layer, a BatchNorm layer and a GELU layer to realize down-sampling.

[0056] In some embodiments, the up-sampling module of the decoder adopts a grid up-sampling method, specifically: after the input feature map is adjusted in channel by a Linear layer, the grid scale is reduced and the grid number is increased, the feature copy saved in the down-sampling stage is introduced, the up-sampling feature and the feature copy are fused by element addition, and then processed by a BatchNorm layer and a GELU layer to realize up-sampling.

[0057] In some embodiments, the training of the deep learning-based slag segmentation model using the labeled slag point cloud sample dataset in S2 specifically includes: data augmentation is performed on the sample data, the data augmentation includes random rotation, scaling and adding Gaussian noise; the augmented sample data is divided into a training set and a validation set, a cross-entropy loss function is used as the optimization objective, and the model is iteratively trained until convergence.

[0058] In some embodiments, the slag surface point cloud data after removing the background points and the noise in S4 is divided into unit cubes and the volume is calculated, specifically: the area of the unit cube is determined according to the number of slag surface points, the height of the unit cube is determined according to the coordinates of the points, the volume of all unit cubes is counted and summed to obtain the initial volume of the slag surface point cloud to the constructed bottom surface; the volume result is corrected by constructing a virtual plane according to the actual size of the slag bucket, specifically: a virtual plane lower than the real bottom surface is constructed according to the actual length, width and volume of the slag bucket, the volume difference between the initial volume and the virtual plane and the real bottom surface is calculated, and the initial volume is subtracted by the volume difference to obtain the actual loading volume of the slag.

[0059] The technical concept of the present application is as follows:

[0060] The shield muck three-dimensional point cloud segmentation and volume calculation method based on deep learning can realize high-precision real-time calculation of the shield tunneling muck volume, the three-dimensional point cloud of the muck bucket loading condition on the muck transportation track vehicle is obtained through the proposed point cloud scanning method, the background points and noise points are removed by using the proposed muck surface point cloud segmentation model, and the pure muck surface points are calculated by using the proposed point cloud volume calculation method. Through the point sequence attention interaction method, the limitations caused by the disorder and sparsity characteristics of the point cloud in the local attention receptive field are solved. In order to segment the dynamic boundary between the muck surface 5 and the muck bucket 4, a normal vector relative position coding module is designed to enhance the spatial feature representation of the point cloud. The present application can realize automatic data acquisition and data processing without affecting the construction process, and finally obtain high-precision muck volume calculation results. The present application can improve the control accuracy and reliability of shield mucking, reduce the tunneling disturbance, reduce the ground loss, eliminate the ground subsidence hidden danger, and ensure the safety of shield construction and urban construction.

[0061] By using a line laser radar to scan a moving train, a complete shield muck transportation vehicle three-dimensional point cloud is obtained. The developed muck point cloud segmentation model is used to remove the background points and noise points such as muck buckets 4 and inverted arch linings, and the volume is calculated based on the extracted pure muck surface point cloud. A muck segmentation model based on deep learning is developed, which is called SMPN (Shield Muck Point Network). SMPN effectively combines sparse convolution and Transformer, while improving the calculation efficiency and accuracy of the model. In addition, SMPN uses four kinds of space-filling curves to realize the serialization of point cloud, and enhances the representation of point cloud spatial position through normal conditional position encoding (NCPE). The present application contains a serialization point attention interaction (SPAI) technology, which realizes the adaptive adjustment of the perception range and point distribution by dividing the point cloud of different serialization orders into windows, and expands the interaction range of the attention window through the combination of different area paths. Finally, the present application contains a volume calculation method based on muck surface point cloud.

[0062] The network structure of the model is improved on the encoder-decoder framework. The encoder mainly includes four down-sampling stages, each of which is composed of a down-sampling module and a Transformer block. The Transformer block is composed of an NCPE module and an SPAI. The model will first pass through a Transformer block to form the point cloud features before down-sampling. The first two down-sampling stages use two Transformer blocks to refine the features each time, and the last two down-sampling stages use three Transformer blocks to refine the features each time. The feature channels of the five stages in the encoder are 16, 32, 64, 128, and 256, respectively. The decoder is the reverse process of the encoder, but the Upsample block used in each up-sampling stage is 2. Compared with the Transformer block, the Upsample block adds an SCB before the NCPE, making the increase of spatial information more smooth. The feature channels corresponding to each up-sampling stage are 256, 128, 64, and 32, respectively. After up-sampling, the final segmentation result is output through an SCB and a fully connected layer.

[0063] The NCPE is a normal vector relative position encoding module that fuses the normal vector and the feature map to emphasize the representation of the original point cloud spatial information in the point cloud feature map. It is composed of an MLP, a cross-attention module, a 3D sparse convolution, a Linear, and a LayerNorm.

[0064] The down-sampling module applies a grid down-sampling method, which is composed of a Grid Pooling, a Linear, a BatchNorm, and a GELU in turn.

[0065] The up-sampling module applies a grid up-sampling method, which is composed of a Linear, a Grid Pooling, a Linear, an element addition, a BatchNorm, and a GELU in turn.

[0066] The SPAI module is used to form a serialized point cloud and calculate the point cloud attention fused by four serialization orders. This module is the core calculation module of the point cloud segmentation method of the application. The module is composed of a Linear, a LayerNorm, a Serialization, a Linear, an SMHA, a Linear, an element addition, a LayerNorm, and an MLP. The SMHA is a serialized multi-head attention layer. This module uses two different attention window division paths according to the characteristics of the serialized point cloud, enriching the interaction range of the local attention window.

[0067] The volumetric algorithm for waste soil point cloud calculations uses a single waste soil hopper 4 as the computational unit. The space between the waste soil surface 5 and the bottom of the hopper is divided into numerous unit cubes. The number and area of ​​the unit cubes are determined by the number of scan points on the waste soil surface 5, and the height of the unit cubes is determined by the coordinates of the scan points. The volume enclosed by the waste soil surface 5 and the waste soil hopper 4 is obtained by summing all the unit cubes. Considering that the actual bottom surface of the waste soil hopper is an irregular plane, a virtual plane is constructed based on the actual volume of the waste soil hopper during calculation for program execution. The specific calculation method is shown in the following formula.

[0068] ;

[0069] ;

[0070] ;

[0071] in the formula This represents the volume of the construction waste calculated from the point cloud on the surface of the waste. and These represent the Z-coordinate values ​​of points on the surface of the slag and the structural plane, respectively. Represents the area of ​​a unit cube; This represents the volume difference between the constructed plane and the real plane; and These represent the actual length and width of the slag hopper, respectively. This indicates the total number of points on the surface of the slag heap; This indicates the actual loading volume of the slag hopper; This represents the formed structural plane.

[0072] The specific plan is as follows:

[0073] 1. Slag cloud collection:

[0074] like Figure 2 As shown, a lidar mounting bracket 2 is installed at the top of the shield tunnel, and a line-scan lidar 1 is installed below the lidar mounting bracket 2, scanning objects passing by the radar vertically downwards. During shield tunneling, the shield transport railcar 3 travels back and forth between the shield machine and the tunnel entrance. When the shield transport railcar 3 passes the line-scan lidar 1, the line-scan lidar 1 acquires a point cloud of the upper surface of the railcar, including the slag hopper 4 and the slag surface 5. The lidar is connected to the host computer in the construction monitoring room via fiber optic cable, and the scanned data is sent to the host computer in the monitoring room in real time. The host computer automatically determines whether to store the slag point cloud data based on the returned slag point cloud data; that is, data storage begins immediately when a slag transport railcar passes by the radar, and no radar data is stored when an unloaded railcar passes by. The collected slag point cloud data is automatically analyzed according to the invented point cloud segmentation and volume calculation algorithm, and the final volume result is output and uploaded to the subway company's monitoring platform. The criteria for determining whether a railcar passes by the lidar are as follows:

[0075] ;

[0076] wherein represents the z-axis coordinate of each point in the point cloud; represents the z-axis coordinate of the top of the lining in the point cloud; represents the culling point operation; and respectively represent the total number of profile points of each radar scan and the number of points that are culled; is a preset threshold value, which is determined by the height distance from the head of the track car to the radar.

[0077] 2. Slag point cloud segmentation:

[0078] Referring to Figure 3 , the point cloud data of the input model is first subjected to data initialization, and the data initialization module forms six cache data according to the input data, namely point cloud coordinates, point cloud grid, point cloud normal vector, four point cloud serialization paths, point cloud features, and pooling cache results. Among them, the point cloud serialization path is formed by four space-filling curves, and the four space-filling curves can be but are not limited to Z-Index, Hilbert, Spiral, and Cantor. Figure 3 Point initializing in the figure represents the point cloud initialization operation; SCB represents the sparse convolution calculation module, and the calculation process is shown in Figure 8 ; NCPE represents the relative position encoding module of the application, and the detailed calculation process is shown in Figure 4 ; SPAI represents the serialized point attention interaction module of the application, and the calculation process is shown in Figure 9 ; Downsampling represents the grid downsampling module, and the calculation process is shown in Figure 5 ; Linear represents a fully connected layer, which is a commonly used calculation method in deep learning. Parent feat represents the original feature parameters cached in the downsampling process, which are fused with the feature maps in the upsampling process of the model to improve the original spatial information expression of the upsampling point cloud feature map.

[0079] The initialized point cloud is subjected to the SCB (SubMConv3d block) module to preliminarily form point cloud features, and the SCB is composed of 3D sparse convolution, BatchNorm (data regularization), and GELU (activation function).

[0080] Subsequently, the point cloud features are input into the first Transformer block, which contains the invented NCPE and SPAI modules. After that, the results are passed through the Down sampling module to reduce the grid size of the point cloud features, achieving down sampling. Before down sampling, the feature map formed by the Transformer block is saved in the pooling cache vector.

[0081] After the first down sampling, the point cloud features are passed through two consecutive Transformer blocks to gradually condense the point cloud features. Subsequently, the second down sampling is performed, and the feature results before down sampling are retained in the pooling cache vector.

[0082] After the second down sampling, the point cloud features are again passed through two consecutive Transformer blocks to gradually condense the point cloud features. Subsequently, the third down sampling is performed, and the feature results before down sampling are retained in the pooling cache vector.

[0083] After the third down sampling, the point cloud features will be passed through three consecutive Transformer blocks. As the number of down sampling stages increases, the number of feature channels of the point cloud also increases, requiring more Transformer blocks to achieve feature extraction and condensation. Finally, the fourth down sampling is performed, and the feature results before down sampling are retained in the pooling cache vector. After the fourth down sampling, the point cloud features are again passed through three consecutive Transformer blocks. At this point, the feature encoding stage of the model has been completed, forming rich point cloud semantic features.

[0084] The down-sampled feature extraction results are input into the up-sampling decoding network. The point cloud features are first input into the first up-sampling module, and then passed through two consecutive upsample blocks. The upsample block is composed of SCB, NCPE, and SPAI in turn. Compared with the down-sampling stage, the up-sampling process additionally adds an SCB before each Transformer module to better restore the spatial information of the point cloud. After two consecutive upsample blocks, the feature map enters the next up-sampling stage. Before the next up-sampling stage, the feature map is fused with the pooling cache results of the same spatial resolution.

[0085] The process of the four up-sampling stages is the same. After passing through four up-sampling stages, the spatial resolution of the feature map will be consistent with the input point cloud. Finally, the point cloud features are converted into point cloud segmentation results through an SCB and a Linear.

[0086] Referring to Figure 4, NCPE takes two inputs, point cloud feature map and point cloud normal vector. The normal vector is first passed through an MLP layer and then fused with the feature map through a cross-attention module. The MLP layer is used to map the normal vector to a feature vector, and the feature channels of the normal vector are unified with the feature. The feature map after adding the normal vector is sequentially passed through a SubMConv3d layer, a Linear layer, and a LayerNorm layer. These three layers form a relative position encoding vector based on the feature information. Finally, the normal vector feature fusion result and the relative position encoding vector are added element-wise through a skip connection, and the NCPE calculation result is output. It should be noted that the skip connection should be used after the normal vector is fused with the feature, i.e., after the Cross-Attention. This is because the skip connection allows to skip the calculation layers within the link, and the operation of fusing the normal vector in the feature map is necessary, as it supplements the original coordinate information for the subsequent SPAI calculation. Figure 4 In the formula, Normal represents the normal vector input; Feat represents the feature map input; Cross-Attention represents the cross-attention calculation module, which is an improved module of the general Multi-Head Attention, used to calculate the attention results of different input sources; SubMconv3D represents 3D sparse convolution calculation; Linear represents a fully connected layer, which is a common calculation method in deep learning; LayerNorm represents layer normalization, which is a technique used in deep learning to reduce internal covariate shift, stabilize the neural network training process, and improve model convergence speed and stability; and represents element-wise addition.

[0087] The downsampling in SMPN uses grid downsampling, and the calculation module is described in Figure 5 The downsampled feature map is sequentially passed through a Linear, BatchNorm, and GELU layer. First, the feature dimension is transformed and combined through linear transformation, and then the transformed feature distribution is adjusted to a stable state (to avoid falling into the saturation region of the activation function, such as the ends of Sigmoid, causing gradient disappearance) through batch normalization, while accelerating the training. Finally, the nonlinearity is introduced through the activation function to enhance the expression ability of the model. Figure 5 In the formula, Feat represents the feature map input; Grid Pooling represents grid pooling, which divides the point cloud into regular grids and generates high-order features for local regions by aggregating the point cloud features within the grid; BatchNorm represents batch normalization, which is a key technique in deep learning for accelerating training and improving stability, mainly solving the "internal covariate shift" problem in deep network training; and GELU is an activation function with smooth characteristics.

[0088] The grid pooling method is described in Figure 6Grid Pooling divides the point cloud space into several three-dimensional grids. When there are multiple points in a grid, a single point is used to represent all points in the grid to achieve downsampling. Multiple points in a grid can be aggregated by averaging, taking the maximum value, or randomly selecting, etc. By increasing the grid size, downsampling can aggregate points in a larger range. Grid Pooling makes the number of spatial points no longer reduced by a fixed multiple or pattern, but is determined by the distribution of points, thereby achieving more efficient data dimensionality reduction and feature fusion, while preserving key geometric structure information and enhancing the adaptability of the model to complex spatial distribution. In addition, the grid size is a hyperparameter that can be set according to the actual distribution of points to reasonably control the downsampling ratio of samples. Figure 6 In the formula, Grid size represents the grid size; Down represents grid downsampling, and Up represents grid upsampling.

[0089] The calculation process of the upsampling module is shown in Figure 7 Before grid upsampling, the feature map first needs to pass through a Linear layer to adjust the output feature channel, and then reduce the grid scale and increase the number of grids. Then, a Linear layer is used to make the increased points more consistent with the feature expression. At this time, the feature copy saved by the downsampling process is introduced, and the upsampling features are fused through element addition. The feature copy is passed through a Linear layer before fusion, allowing the model to fine-tune the feature copy through training to better achieve feature fusion. Figure 7 In the formula, Linear represents a fully connected layer, which is a common calculation method in deep learning; Grid Pooling represents grid pooling, which divides the point cloud into regular grids (grids) and generates local high-order features by aggregating point cloud features in the grid; BatchNorm represents batch normalization; GELU is an activation function with smoothing properties; and represents element addition.

[0090] The output result of the upsampling module is input into a spatial convolution block (SCB), as shown in Figure 8 This module is composed of three sub-modules connected in sequence: SubMConv3d, BatchNorm, and GELU. The input data is first passed through the SubMConv3d layer, which performs convolution operations on the three-dimensional space of the original point cloud to extract local features. Since it is a sub-manifold convolution, it does not change the spatial distribution of the point cloud (i.e., the position and number of points remain unchanged). Subsequently, BatchNorm and GELU are used to achieve standardization processing and nonlinear transformation. Figure 8 In the formula, SubMconv3D represents 3D sparse convolution calculation; BatchNorm represents batch normalization; and GELU is an activation function with smoothing properties.

[0091] The core computing module of the point cloud segmentation model SMPN is SPAI, and the computing process is shown in Figure 9 The input feature map is first passed through a Linear layer and a LayerNorm, which is to allow the feature map before serialization to adaptively adjust. Then, the point cloud is given different arrangement orders through Z-Index, Hilbert, Spiral, and Cantor four kinds of space-filling curves, and the connection order of the four paths is shown in Figure 10 After the four different sorting methods, the point cloud feature channel is expanded to four times the original, and then the channel number is restored to the input channel through a Linear to avoid excessive expansion of the parameter amount in attention calculation. Then, the feature map participates in SMHA calculation, and SMHA is a proposed serialized multi-head attention module, which is characterized by using different paths to form an attention window. After the feature map is input into SMHA, the channel number is expanded to three times the original through a Linear layer to form q, k, and v matrices, and then q, k, and v matrices are divided into q1, q2, k1, k2, v1, and v2 matrices, and the channel number of each matrix is one sixth of the input channel. q1, q2, k1, k2, v1, and v2 are divided into two groups, and local attention windows are formed by different paths, respectively. Then q1 and q2, k1 and k2, and v1 and v2 are spliced in the channel to form q, k, and v matrices, and then the attention calculation is realized. After the attention calculation is completed, the feature map is input into a Linear layer again to strengthen the expression of the attention result, and then the original input feature through element addition is fused through the jump connection, and finally the result is input into the LayerNorm and MLP layer to realize the standardization of the feature result and enhance the nonlinear expression ability. Figure 9 Linear in the middle represents a fully connected layer, which is a commonly used calculation method in deep learning; LayerNorm represents layer normalization; Linear represents a fully connected layer, which is a commonly used calculation method in deep learning; Z-Index, Hilbert, Spiral, and Cantor are four kinds of space-filling curves used; Serialization Multi-Head Attention (SMHA) represents point cloud multi-head serialized attention calculation. q1, q2, k1, k2, v1, and v2 are two groups of attention matrices for SMHA calculation, and Patch path represents different attention window construction paths; Attention represents an attention calculation module, which is a standard attention calculation unit in transformer.

[0092] The morphological change process of the feature map tensor in SPAI is shown in Figure 11The point cloud feature map is mapped to 4 different sequential orders and stacked in the feature channel, the feature map is evenly divided into two parts in the format of evenly arranging the feature map according to the 4 sequential orders, and the attention window is divided in different paths. The first way is to traverse the first part of the feature map horizontally, at this time the attention window contains different sequential point clouds respectively. The second way is to traverse the second part of the feature map vertically, at this time each attention window will contain 4 different sequential point clouds, and finally the attention calculation is performed on all windows to enrich the interaction results of the attention window.

[0093] The slag point cloud segmentation result of the present application is shown in Figure 12 . From the overall sample, the model has fully learned the division basis of the slag surface 5 and the slag hopper 4, and the segmentation effect meets the expectation. The model clearly defines the boundaries of the slag surface 5 and the slag hopper 4, and the side wall and bottom surface of the box are also clearly divided. Although the contour of the contact between the slag and the side wall is relatively smooth (the two surfaces are not directly tangent, but gradually transitioned), the model can still clearly distinguish the two classifications. For the case that the sample distribution along the X axis is uneven due to the speed fluctuation of the slag transportation rail car, SMPN has sufficient generalization ability to cope with the changes in sample density or spatial distribution. The results prove that SMPN is close to the effect of manual segmentation in slag point segmentation, and has significant engineering value.

[0094] The calculation method of the point cloud volume is referred to Figure 13 . Due to the shielding of the laser scanning by the slag hopper partition, there is an interruption in the middle of the slag surface points. Before calculating the volume, the interruption needs to be filled first. According to the X coordinate distribution of the point cloud, the X coordinate interruption is greater than the L x coordinate of the two sides of the point, and the interruption length L d is further detected, and the points with X coordinates of L d / 2 on both sides of the interruption are extracted and filled into the interruption. When calculating the volume, first count the number of input points, and calculate the area of the rectangular unit cell represented by each point according to the actual size of the hopper car . Then, a virtual slag hopper 4 bottom surface is constructed, that is, a part of the volume is added to the actual volume of the slag hopper 4, and then divided by the actual of the slag hopper to obtain a lower plane than the real bottom surface, and record the plane axis coordinate. Let the Z axis coordinate of each slag surface point minus to obtain the distance from each point to the constructed bottom surface, sum the distance from all points to the constructed bottom surface, and multiply by the area of the rectangular unit cell to obtain the volume of the slag point cloud to the constructed bottom surface. Finally, subtract from the result to obtain the actual loading volume of each hopper.

[0095] The comparison results of the proposed SMPN with other point cloud models are shown in Table 1. Compared with the models with less parameters such as PointNet++ and PTv2, the segmentation accuracy of SMPN is greatly improved, and the mAcc, mIoU and oAcc are increased by 13.6%, 6.5% and 6.6% respectively. Compared with large models such as advanced Swin3D and Octformer, the proposed SMPN still has an accuracy advantage under the condition of shortening the inference time, and the mAcc, mIoU and oAcc of SMPN are increased by 1.9%, 0.7% and 0.4% respectively compared with Swin3D. In Table 1, Methods represents the model or calculation method used for comparison; Para (M) represents the model parameter quantity, with the unit of 10 6 mAcc, mIoU and oAcc represent three evaluation methods of the model; Muck (IoU), Box (IoU) and Rail (IoU) represent the IoU results of the slag surface, slag bucket and track respectively; Training time (h) represents the training time of the model, with the unit of hours; Inference time (s) represents the inference time of the model, with the unit of seconds.

[0096] Table 1. Comparison results of SMPN with other point cloud models

[0097] Methods Para (M) mAcc mIoU oAcc Muck (IoU) Box (IoU) Rail (IoU) Training time (h) Inference time (s) PointNet++ 3.5 0.778 0.791 0.820 0.756 0.813 0.804 8.2 8.8 MinkUNet 37.9 0.851 0.841 0.860 0.834 0.822 0.867 28.5 36.4 PTv2 12.8 0.913 0.906 0.917 0.898 0.921 0.899 20.7 15.6 OctFormer 44.0 0.952 0.947 0.962 0.941 0.944 0.956 33.4 38.6 Swin3D 71.1 0.968 0.964 0.979 0.946 0.965 0.981 34.8 54.7 SMPN 34.46 0.987 0.971 0.983 0.974 0.968 0.975 26.8 32.5

[0098] Table 2. Test results of four kinds of serialized fusion attention methods

[0099]

[0100] The test results of the four proposed attention methods of serialized fusion are shown in Table 2. The test first tests the effect of the four space-filling curves based on the multi-head self-attention mechanism (MHA) respectively. The results show that the serialization of point clouds improves the computational efficiency of the Transformer. It is reflected in the use of Z-Index serialization path, which improves the mAcc, mIoU and oAcc of the model by 4.7%, 6.4% and 5.9% respectively compared with the model without using the serialization path. However, the fixed serialization order leads to a decrease in the stability of the segmentation accuracy, which is reflected in the increase of the interquartile range (IQR). Due to the diversity of the slag surface boundary, any given serialization order is difficult to completely adapt to all samples. To solve these problems, the present application SPAI realizes the attention calculation of point clouds with different serialization orders. The results show that allowing the model to selectively use different space curves can effectively improve the accuracy and stability of the model. For example, the simultaneous use of Z-Index+Hilbert order improves the mAcc, mIoU and oAcc by 6.1%, 5.0% and 4.8% respectively compared with the use of Z-Index order only. However, the test of different curves did not find the optimal curve combination, and the test results of the four times of two curve combinations are very close, which shows that the fusion of the two curves is not the upper limit of the model. The above test results show that the diversity of the order enhances the adaptability and interaction ability of the attention receptive field and improves the performance of the model. Based on these findings, a diversified patch path is introduced to enhance the variation of point cloud ordering, thereby achieving the best experimental performance in the benchmark evaluation. The final implementation method of SPAI further improves the mAcc, mIoU and oAcc by 3.6%, 2.5% and 2.7% compared with the use of Z-Index+Hilbert order. It should be pointed out that if the number of serialization orders is further increased, due to the limitation of the channel, the feature expression of a single order may be incomplete. Therefore, under the current parameters of SMPN, the number of serialization orders is selected as four. The implementation method of the patch path is similar to the increase of the attention head, which needs to consider the number of feature channels and the patch size, and both the two paths have reached the upper limit of the existing equipment. In Table 2, Attention method represents the attention calculation method used by the test model; Serialization represents the point cloud space serialization path used by the test model; Four-curve merge (only path-1) represents the test results of using four path fusion, but only using one serialization path; Four-curve merge (only path-1) represents the test results of using four path fusion, and using two serialization paths; mAcc, mIoU, oAcc, IQR represent the evaluation indicators of the test model.

[0101] The above-described embodiments are intended to illustrate, not to limit, the present application, and thus the modification of the example values or the substitution of equivalent elements should be within the scope of the present application.

[0102] From the above detailed description, it can be seen that the present application can achieve the aforementioned purpose, and has met the requirements of the Patent Law.

[0103] Although the preferred embodiments of the present application have been described, those skilled in the art who have the benefit of the basic inventive concept can make additional changes and modifications to the embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application. The above description is merely the preferred embodiments of the present application, and is not intended to limit the present application. It should be noted that any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

[0104] It should be noted that the above description of the flow is merely for example and illustration, and does not limit the scope of the present application. Those skilled in the art can make various modifications and changes to the flow under the guidance of the present application. However, these modifications and changes are still within the scope of the present application.

[0105] The above has described the basic concept, and it is obvious that the above-mentioned invention disclosure is only as an example and does not constitute a limitation to the present application for those skilled in the art after reading this application. Although it is not explicitly stated here, those skilled in the art can make various modifications, improvements and modifications to the present application. Such modifications, improvements and modifications are suggested in the present application, so such modifications, improvements and modifications are still within the spirit and scope of the exemplary embodiments of the present application.

[0106] Meanwhile, specific words are used in the present application to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" means a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that the "an embodiment" or "one embodiment" or "an alternative embodiment" mentioned in different positions in the present specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of the present application can be properly combined.

[0107] Moreover, as will be appreciated by persons skilled in the art, the present application is capable of being embodied with several different types of categories or circumstances of patentable subject matter including any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof. Accordingly, the various aspects of the present application can be embodied in hardware and / or in software (including firmware, resident software, micro-code, etc.) that runs on a processor or controller to cause a computer or computer- based device to perform various computer-implemented processes. The computer program instructions can be embodied in a computer program product, which can include a computer-readable medium, such as a floppy disk, a CD-ROM, a DVD, a Blu-ray Disc, a flash drive, a memory stick, a magnetic tape, a magnetic disk, a memory chip, a semiconductor, or the like, which can be accessed by a computer or computer- based device. Accordingly, the various aspects of the present application can be embodied in a number of different forms, all of which have been contemplated to be within the scope of the present application. Accordingly, the various aspects of the present application can be embodied in a number of different forms, all of which have been contemplated to be within the scope of the application. For example, while the various aspects of the present application have been described in the context of fully functioning computers, those skilled in the art will appreciate that the various aspects of the present application are capable of being embodied in a number of different forms and that the various aspects of the present application apply equally to mobile computers, embedded computers, servers, computer software, and computer networks. Accordingly, the various aspects of the present application can be embodied in the context of a number of different types of general purpose computers, special purpose computers, computer systems, or computer networks.

[0108] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, conventional procedural programming languages, such as the C programming language, Visual Basic, Fortran 2103, Perl, COBOL 2102, PHP, ABAP, dynamic programming languages, such as Python, Ruby and Groovy, or another programming language. The program code can execute entirely on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider), or in a cloud computing environment, or as a service such as Software as a Service (SaaS).

[0109] In addition, the order of processing elements or sequences, unless specifically stated by a claim, are not intended to be limiting, but are an authorization to the skilled artisan, and that the application can not be implemented in the exact order shown. Although the above disclosure discusses some presently preferred embodiments of the application, it is to be understood that numerous additions, modifications, and depletions of functions and elements can be made by one of ordinary skill in the art without departing from the scope of the application. For example, although the implementation of the various components discussed above can be embodied in hardware devices, it can also be implemented as a pure software solution, for example, as an installation on an existing server or mobile device.

[0110] For similar reasons, it is to be appreciated that the teachings of the present application provided herein can be applied to any embodiment of the present application, and that actual claims applied for or patent granted can be broader than any single, featured embodiment. Accordingly, a patent applicant has constructed and filed examples to particularly point out and distinctly claim those aspects which are regarded as novel and those aspects specifically shown.

Claims

1. A method for segmenting and calculating the volume of a three-dimensional point cloud of shield muck based on deep learning, characterized in that, include: S1. Point cloud data acquisition, preprocessing, and sample labeling; The three-dimensional point cloud data, including the slag bucket and the slag surface, is collected by scanning the passing slag transport railcar with a line lidar. The three-dimensional point cloud data is automatically judged and stored to obtain the initial slag point cloud data. Based on the initial slag point cloud data, the slag surface points, background points and noise points are manually labeled to obtain the labeled slag point cloud sample dataset. S2. Construction and training of deep learning segmentation model; Construct a deep learning-based waste soil segmentation model with an encoder-decoder structure. The encoder includes a downsampling module and a Transformer block. The Transformer block includes a normal vector relative position encoding module and a serialized point attention interaction module. The decoder includes an upsampling module and an Upsample block. Train the deep learning-based waste soil segmentation model using an annotated waste soil point cloud sample dataset to obtain the trained deep learning-based waste soil segmentation model. S3. Slag point cloud segmentation; The initial slag point cloud data is input into the trained deep learning-based slag segmentation model, and then processed sequentially through data initialization, feature extraction, downsampling, and upsampling to obtain slag surface point cloud data after removing background points and noise. S4. Calculation of slag volume: Using a single slag hopper as the calculation unit, the slag surface point cloud data is divided into unit cubes based on the slag surface point cloud data after removing background points and noise, and the volume is calculated. Combined with the actual size of the slag hopper, a virtual plane is constructed to correct the volume result, and the actual loading volume of the slag is obtained. The specific processing procedure of the relative position encoding module for normal vectors in S2 is as follows: receiving point cloud feature maps and point cloud normal vectors, mapping the point cloud normal vectors into feature vectors through an MLP layer, fusing them with the point cloud feature maps through a cross-attention module, and then sequentially passing them through a 3D sparse convolution, a Linear layer, and a LayerNorm layer to form a relative position encoding vector. The normal vector features and the relative position encoding vectors are then fused through a skip connection, and the processing result is output. The specific processing procedure of the serialization point attention interaction module in S2 is as follows: After the input feature map is adjusted by the Linear layer and the LayerNorm layer, it is serialized by four space filling curves: Z-Index, Hilbert, Spiral, and Cantor. The serialized feature channels are compressed to the original number of channels. Attention is calculated by the SMHA layer, and then the result is output after passing through the Linear layer, the LayerNorm layer, and the MLP layer in sequence.

2. The deep learning-based shield muck three-dimensional point cloud segmentation and volume calculation method according to claim 1, characterized in that, The automatic determination and storage process in S1 is as follows: by comparing the z-axis coordinates of each point in the point cloud with the z-axis coordinates of the top of the lining, it is determined whether it is a railcar loaded with slag, and only the three-dimensional point cloud data corresponding to the railcar loaded with slag is stored.

3. The deep learning-based shield muck three-dimensional point cloud segmentation and volume calculation method according to claim 1, characterized in that, The manual labeling of surface points, background points, and noise points in S1 is as follows: based on the point cloud coordinates and spatial distribution characteristics, the point cloud is divided into surface points, background points, and noise points, and category labels are added to different categories of points.

4. The deep learning-based shield muck three-dimensional point cloud segmentation and volume calculation method according to claim 1, characterized in that, The encoder-decoder structure in S2 is as follows: the encoder comprises four down-sampling stages, each of the first two down-sampling stages uses two Transformer blocks, each of the last two down-sampling stages uses three Transformer blocks, and the feature channels are 16, 32, 64, 128 and 256 in turn; the decoder comprises four up-sampling stages, each of which uses two Upsample blocks, and the feature channels are 256, 128, 64 and 32 in turn, and finally a segmentation result is output through a SubMConv3d block module and a fully connected layer.

5. The deep learning-based shield muck three-dimensional point cloud segmentation and volume calculation method according to claim 4, characterized in that, The down-sampling module of the encoder adopts a grid down-sampling method, which specifically comprises: dividing the point cloud space into a three-dimensional grid, aggregating multiple points in the grid into a single representative point by averaging, taking the maximum value or randomly selecting, and then sequentially processing through a Linear layer, a BatchNorm layer and a GELU layer to realize down-sampling.

6. The deep learning-based shield muck three-dimensional point cloud segmentation and volume calculation method according to claim 4, characterized in that, The up-sampling module of the decoder adopts a grid up-sampling method, which specifically comprises: adjusting the channel of the input feature map through a Linear layer, reducing the grid size and increasing the number of grids, introducing a saved feature copy in the down-sampling stage, fusing the up-sampling feature and the feature copy through element addition, and then processing through a BatchNorm layer and a GELU layer to realize up-sampling.

7. The deep learning-based shield muck three-dimensional point cloud segmentation and volume calculation method according to claim 1, characterized in that, The training of the slag segmentation model based on deep learning in S2 using the labeled slag point cloud sample dataset specifically comprises: data augmentation of the sample data, which includes random rotation, scaling and adding Gaussian noise; dividing the augmented sample data into a training set and a validation set, using a cross-entropy loss function as the optimization objective, and iteratively training until the model converges. 8.The deep learning based shield muck three-dimensional point cloud segmentation and volume calculation method according to claim 1, characterized in that, The slag surface point cloud data after removing background points and noise in S4 is divided into unit cubes and the volume is calculated, which specifically comprises: determining the area of the unit cube according to the number of slag surface points, determining the height of the unit cube according to the coordinates of the points, and calculating the volume of all unit cubes and summing them up to obtain the initial volume of the slag surface point cloud to the constructed bottom surface; and combining the actual size of the slag bucket to construct a virtual plane to correct the volume result, which specifically comprises: constructing a virtual plane below the real bottom surface according to the actual length, width and volume of the slag bucket, calculating the volume difference between the initial volume and the virtual plane and the real bottom surface, and subtracting the volume difference from the initial volume to obtain the actual loading volume of the slag.

Images