Automobile crane three-dimensional point cloud segmentation method based on deep learning

Abstract

The invention discloses an automobile crane three-dimensional point cloud segmentation method based on deep learning, and the method comprises the following steps: S1, placing three-dimensional models, which are established by SolidWorks software, of an automobile crane in different working states in a simulation scene constructed by a Gazebo platform, carrying out dense mapping on the three-dimensional model of the automobile crane through a Fast-LIO2 algorithm to obtain three-dimensional dense point cloud data of the automobile crane; s2, on the basis of the three-dimensional dense point cloud data of the automobile crane, constructing a SegNet model of the automobile crane; s3, based on a label smoothing strategy, setting a loss function of the SegNet model; and S4, carrying out training and reasoning on the SegNet model. The method has the advantages that the precision of point cloud segmentation in a complex scene is improved, and the method is suitable for segmentation of key parts of the automobile crane.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of three-dimensional point cloud segmentation of automobile cranes, and particularly relates to a three-dimensional point cloud segmentation method for automobile cranes based on deep learning. BACKGROUND

[0002] Three-dimensional point cloud semantic segmentation is a method for obtaining the semantic label value of each point in a three-dimensional scene by using a three-dimensional point cloud semantic segmentation algorithm, so as to realize segmentation. In recent years, point cloud semantic segmentation methods based on deep learning have made significant progress. These methods construct complex neural network architectures, utilize powerful feature extraction and multi-scale information fusion capabilities, and realize accurate semantic segmentation of three-dimensional point clouds. Compared with traditional methods, deep learning-based techniques can effectively handle large-scale three-dimensional semantic segmentation tasks with significantly reduced labeled data, relying on a small amount of point labels through supervised learning or semi-supervised learning, greatly reducing the time and labor cost of data labeling. According to the processing method of three-dimensional point cloud data and the directness of feature extraction, point cloud semantic segmentation methods are usually divided into direct and indirect methods. Direct methods such as PointNet, which is based on a multi-layer perceptron, directly process raw point cloud data, extract point cloud features using a deep learning model, and perform segmentation without converting point cloud data into a regular grid, overcoming the quantization artifact problem in traditional methods, but without considering the relationship between points. Qi et al. proposed a hierarchical version of PointNet, PointNet++, which learns point features from the neighborhood of each point. Indirect methods usually first convert point clouds into regular data forms such as voxels or multi-view images, and then apply deep learning models for processing. Su et al. proposed a multi-view convolutional neural network (MVCNN) based on two-dimensional images, which projects three-dimensional objects into multiple views and aggregates view features. This method ignores the spatial information of objects, so it is suitable for segmenting single objects but not complex scenes; Maturana et al. proposed VoxNet, which uses a voxel processing network to achieve robust 3D object recognition. Wu et al. proposed a 3D ShapeNet convolutional network for learning the distribution of points in different three-dimensional shapes and establishing a probability distribution of binary variables on a voxel grid. However, with increasing resolution, the above methods still have certain limitations in processing dense three-dimensional point clouds. The present application focuses on automobile cranes, uses a direct method to realize three-dimensional point cloud segmentation, considers the global and local spatial relationships between points, and introduces an attention mechanism to aggregate point cloud features and improve point cloud segmentation accuracy.

[0003] There are many problems in the existing three-dimensional point cloud segmentation of automobile cranes. First, there are few publicly available high-quality automobile crane point cloud datasets, and the sample diversity is insufficient, so the algorithm has limited performance in generalization performance; second, uneven sensor resolution or accuracy reduction during data acquisition can easily cause detail loss, especially in complex environments, point cloud sparsity is intensified, which affects the accurate segmentation of target components. In addition, environmental factors such as occlusion and reflection also increase the difficulty of data processing, which restricts the reliability of the segmentation algorithm. SUMMARY

[0004] The purpose of the present application is to provide a three-dimensional point cloud segmentation method for automobile cranes based on deep learning according to the shortcomings of the prior art. First, a simulated automobile crane dataset is prepared, and then a three-dimensional point cloud segmentation network SegNet for automobile cranes is proposed. The network combines graph convolutional networks and shape attention mechanisms to optimize the encoder structure, thereby improving the accuracy of point cloud segmentation in complex scenes and making it more suitable for the segmentation of key components of automobile cranes.

[0005] The purpose of the present application is achieved by the following technical solutions:

[0006] A three-dimensional point cloud segmentation method for automobile cranes based on deep learning, the method comprising the following steps:

[0007] S1: Place the three-dimensional model of the automobile crane in different working states established by SolidWorks software in the simulation scene built on the Gazebo platform, and perform dense mapping on the three-dimensional model of the automobile crane by the Fast-LIO2 algorithm to obtain three-dimensional dense point cloud data of the automobile crane;

[0008] S2: Based on the three-dimensional dense point cloud data of the automobile crane, a SegNet model of the automobile crane is constructed;

[0009] S3: Based on the label smoothing strategy, set the loss function of the SegNet model;

[0010] S4: Train and infer the SegNet model.

[0011] In step S1, the slewing angle and the pitch angle of the boom of the automobile crane are respectively provided with 7 and 5, the slewing angle of the boom of the automobile crane is respectively 0°, ±30°, ±45°, ±60°, and the pitch angle of the boom of the automobile crane is respectively 0°, 10°, 30°, 45°, 60°, and 35 three-dimensional models of the automobile crane in different working states are obtained by combination.

[0012] In step S1, the method for dense mapping is as follows:

[0013] The input data of the laser radar is transmitted to a feature extraction module to obtain planar features and edge features; the extracted features and inertial measurement unit measurement values are input into a state estimation module for state estimation at a frequency range of 10Hz to 50Hz; the estimated pose registers the feature points into a global coordinate frame and merges them with the currently constructed feature point map; and the updated map adds more new points for the next mapping process.

[0014] In step S2, before constructing the SegNet model of the truck crane, the three-dimensional dense point cloud data of the truck crane is preprocessed, and the method steps of the preprocessing are as follows:

[0015] In view of the sparsity and noise of the three-dimensional dense point cloud data, denoising, filtering and downsampling operations are performed.

[0016] In step S2, the SegNet model is composed of an encoder, a decoder and a segmentation head, the encoder is provided with five, one MLP module and four SAGT modules, the decoder is provided with four, and the segmentation head is provided with one, and the method steps of constructing the SegNet model are as follows:

[0017] The input three-dimensional dense point cloud data is encoded by the MLP module to extract high-dimensional features; four SAGT modules are used to downsample the encoded features multiple times, gradually reducing the number of features while increasing the number of feature channels, so as to more effectively capture depth features at different levels, and in this process, the point cloud features extracted by each encoder are transmitted to the decoder through a jump connection and are fused with the output of the previous layer decoder.

[0018] In the method of constructing the SegNet model, the method steps of the feature fusion are as follows:

[0019] The input three-dimensional dense point cloud data is downsampled by adaptive voxel filtering to obtain key points containing global information as spatial fixed points δ s ; the global feature extraction part of the SAGT module is entered, the three-dimensional dense point cloud data is downsampled by the farthest point sampling method, and a plurality of global key points x i are extracted while the general outline of the object is retained;

[0020] The Euclidean distance of the global key points x i and the spatial fixed points δ s is solved, the similarity between each pair of points is obtained by calculating the Gaussian kernel value of each pair of points, and the similarity formula is calculated as follows:

[0021] g(δ sx i )=∑exp(-γ||δ s -x i || 2 );

[0022] where g(δ s ,x i ) is a spatial weighting function, and γ is a weighting coefficient;

[0023] The similarity is weighted to the original feature point f i , so that the original feature point f i contains global key information, and the feature is weighted in the semantic space. The semantic weighting formula is as follows:

[0024]

[0025] where h(z s ,f i ) is a semantic weighting function, z s is a semantic fixed point, i' is the index of the original feature point, and f i' is the original feature point with index i';

[0026] Combined with the semantic weighting function h(z s ,f i ), the spatial weighting function g(δ s ,x i ), and the original feature point f i , the extracted global feature information is obtained through the cross-attention mechanism:

[0027]

[0028] where f is the global feature;

[0029] Local edge features are extracted through graph convolution. The graph convolution formula is as follows:

[0030] x ij =max j:(i,j)∈ε h Θ (x i ,x j );

[0031] where h Θ is an edge function, x i is a center point, x j is a neighbor key point of x i , θ m and φ m are edge function coefficients;

[0032] Edge function h Θ The specific selection is as follows:

[0033] e ijm = ReLU (theta m * (f j -f i )+ phi m * f i );

[0034] The feature difference (f j -f i ) between the key points is calculated and input into a two-dimensional graph convolution network together with the original feature point f i , and local key features are extracted through an MLP layer and maximum pooling;

[0035] The maximum pooling formula is as follows:

[0036]

[0037] In the formula, is a local feature;

[0038] In combination with the global feature obtained through the global key point, the linear layer and the learnable parameter alpha are used to dynamically adjust the proportion between the global feature and the local feature , so as to give the key point the fused local and global features:

[0039]

[0040] In the formula, f i ^ is the fused feature.

[0041] In step S3, the loss function adopts a cross-entropy loss function, and the cross-entropy loss function is defined as follows:

[0042]

[0043] In the formula, C is the number of target categories, p(x i ) is a label after smoothing processing, and q(x i ) is a probability that the model predicts to belong to the category.

[0044] The advantages of the present application are: the accuracy of point cloud segmentation in a complex scene is improved, and the application is suitable for segmentation of key parts of a truck crane. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a flow chart of the three-dimensional point cloud segmentation method of the truck crane based on deep learning of the present application;

[0046] Figure 2 Three-dimensional model diagram of the automobile crane of the present application (1);

[0047] Figure 3 Three-dimensional model diagram of the automobile crane of the present application (2);

[0048] Figure 4 Three-dimensional dense point cloud diagram of the automobile crane of the present application (1);

[0049] Figure 5 Three-dimensional dense point cloud diagram of the automobile crane of the present application (2);

[0050] Figure 6 SegNet model structure diagram of the automobile crane of the present application;

[0051] Figure 7 Global key point extraction schematic diagram of the present application;

[0052] Figure 8 Local feature extraction structure diagram of the present application;

[0053] Figure 9 Segmentation result diagram of the present application. DETAILED DESCRIPTION

[0054] The features and other related features of the present application will be further described in detail below by way of examples in conjunction with the accompanying drawings, so as to facilitate the understanding of the same by the skilled in the art:

[0055] Embodiment: As shown in the accompanying drawings, the present embodiment relates to a deep learning-based automobile crane three-dimensional point cloud segmentation method, which mainly includes the following steps: Figure 1

[0056] S1: Data set making: first, based on the SolidWorks software, three-dimensional models of the automobile crane in different working states are established, obtaining 35 three-dimensional models; then, a simulation scene is built on the Gazebo platform, and the three-dimensional models are placed in appropriate positions to simulate the real working environment of the automobile crane; then, Fast-LIO2 algorithm is applied for dense mapping to generate three-dimensional dense point cloud data of the automobile crane in different working states, and a simulation automobile crane data set is made for subsequent experimental research.

[0057] Among them, as shown in Figure 2 and Figure 3 ​As shown, the variable state automobile crane model is processed by SolidWorks software, the boom rotation angle and the luffing angle of the automobile crane model are adjusted, the horizontal angle (rotation angle) is set to 0°, ±30°, ±45°, ±60°, and the vertical angle (luffing angle) is set to 0°, 10°, 30°, 45°, 60°, so that a total of 35 automobile crane models in different postures are obtained. As shown in Figure 4 and Figure 5 As shown, based on the Gazebo platform, a dense mapping is performed using the Fast-LIO algorithm to generate a three-dimensional point cloud model. During the mapping process, the input data of the laser radar is first transmitted to the feature extraction module to obtain the plane features and edge features; then, the extracted features and the inertial measurement unit (IMU) measurement values are input into the state estimation module for state estimation at a frequency range of 10 Hz to 50 Hz; subsequently, the estimated pose registers the feature points to the global coordinate frame and merges them with the currently constructed feature point map; the updated map adds more new points for the next mapping process. Through the Fast-LIO dense mapping of the automobile crane model, a high-density three-dimensional point cloud is finally obtained, and the color, texture and other information of the crane are preserved as much as possible.

[0058] S2: Network structure design: The point cloud segmentation process includes data preprocessing, network model construction, training and evaluation. In the data preprocessing stage, in view of the sparsity and noise of point cloud data, denoising, filtering and downsampling operations are performed to improve the efficiency and accuracy of subsequent labeling and model training. The network model (SegNet model of automobile crane) adopts an encoder-decoder (Encoder-Decoder) architecture, which can effectively extract the global and local features of the point cloud. The encoder is used to capture the macrostructure information of the point cloud, and the decoder is used to accurately identify the local geometric details, realizing accurate segmentation of the point cloud semantics.

[0059] Among them, inspired by the SPoTr network model architecture, it is composed of an encoder, a decoder and a segmentation head, and has great advantages in processing unstructured point cloud data using an encoder-decoder architecture. First, this structure effectively captures multi-scale global context information through layer-by-layer downsampling and feature extraction, and the decoder restores local details through upsampling and feature fusion, thereby improving the segmentation accuracy. In addition, the skip connection mechanism allows the encoder to extract features directly to the decoder, enhancing the model's ability to perceive details and avoiding information loss. This structure flexibly adapts to the non-structural features of point cloud data and can be combined with other deep learning modules to further improve the performance and computational efficiency of the model. As shown in Figure 6As shown, in view of the characteristics of the segmentation object and the small number of data sets in this embodiment, the encoder structure is improved, referred to as a SAGT (Shape-Aware Graph Convolution Transformer) module. Specifically, the input point cloud data is first encoded by an MLP (Multilayer Perceptron) module to extract high-dimensional features; then, four SAGT modules are used to perform multiple down-sampling on the encoded features, gradually reducing the number of features while increasing the number of feature channels, so as to more effectively capture deep features at different levels. In this process, the point cloud features extracted by each encoder module are transmitted to the decoder part through a jump connection and are fused with the output of the previous layer decoder. This structure design effectively integrates low-level detail information and high-level abstract features, thereby achieving accurate segmentation of semantic point cloud data and significantly enhancing the recognition ability of complex geometric structures. This method not only improves the segmentation accuracy, but also improves the overall robustness of the model and reduces the computational complexity.

[0060] Specifically, as shown in Figure 7 , the embodiment uses global key points to represent the shape information of the object, and aggregates point cloud features by combining the Transformer mechanism of shape context, helping the model to understand the overall shape and structural features. Attention mechanism has achieved remarkable results in text and image processing fields. In view of the discreteness and disorder of point cloud data, the attention mechanism can effectively establish the association between points and points, and then aggregate key geometric structure features in the global and local range. This embodiment combines global cross-attention and graph convolution, while capturing overall shape information and local details, achieving accurate semantic segmentation of point cloud data. In order to better utilize the global shape context information, this embodiment introduces global key points, focusing on the long-range dependencies in the point cloud, and capturing global geometric patterns. First, the input point cloud is down-sampled by adaptive voxel filtering to obtain key points containing global information as spatial fixed points δ s ; then enter the global feature extraction part of the SAGT module in the encoder, and down-sample the point cloud by the farthest point sampling (FPS) method, while preserving the overall outline of the object, extract several global key points x i . The global key points x i extracted layer by layer are calculated with the Euclidean distance of the spatial fixed points δ s extracted at the beginning of the model, and then the similarity between each pair of points is obtained by calculating the Gaussian kernel value of each pair of points, and the similarity formula is calculated as follows:

[0061] g(δ s ,x i )=∑exp(-γ||δs x i | 2 );

[0062] where g(δ s ,x i ) is a spatial weighting function, and γ is a weighting coefficient.

[0063] This similarity is weighted to the original point f i feature, so that the original point contains global key information. The above is the spatial feature weighting, in addition to the spatial feature weighting, this embodiment also carries out the weighting of the feature in the semantic space, and the semantic weighting formula is as follows:

[0064]

[0065] where h(z s ,f i ) is a semantic weighting function, z s is a semantic fixed point, i' is an original feature point index, and f i' is the original feature point with index i'.

[0066] Combined with the semantic feature information h(z s ,f i ), the spatial feature information g(δ s ,x i ), and the original point feature f i , the extracted global feature information can be obtained through the cross attention mechanism:

[0067]

[0068] where, is the global feature.

[0069] The model updates the feature vector of each point by associating with the features of the global key points. The introduction of global key points enables the model to better capture global information during feature updating and maintain long-distance dependencies in the point cloud. Through the association of these global key points with the features of the local region of the point cloud, the model realizes the calculation of global attention and continuously updates the shape context information of the point cloud, thereby improving the global structure perception ability and semantic segmentation accuracy. In SAGT, compared with SPoTr, the global key point extraction method based on attention mechanism in the global feature aggregation module is updated to the strategy of extracting key points using adaptive voxel filtering. This method can determine fixed key points at the initialization of the model, and is particularly suitable for precise point cloud segmentation tasks in small scenes. This adjustment not only reduces the computational complexity of the model, but also enhances its robustness, avoiding the fluctuations in the results caused by the changes in the weights learned by the attention mechanism each time.

[0070] This embodiment combines graph convolution with the Transformer mechanism to extract local edge features through graph convolution. Unlike traditional convolutional neural networks that perform convolution on regular two-dimensional or three-dimensional grid data, graph convolution can process irregular graph structure data and is suitable for disordered three-dimensional point cloud data. Figure 8 As shown in the figure, in the point cloud segmentation task, due to the lack of explicit connection between points, the edge features play a key role in the semantic understanding of the point cloud. Therefore, this embodiment introduces a graph structure in the local area, associates the points in the neighborhood through feature aggregation, and extracts local edge features. Graph convolution is to learn the center point x i Point x with surrounding neighbors j The relationship between h Θ It is the edge function used to learn edge relationships. The selection of edge function is extremely important for graph convolution to obtain local features.

[0071] x ij =max j:(i,j)∈ε h Θ (x i ,x j );

[0072] Where h Θ is the edge function, the global key point x i is the center point, x j is x i Neighbor key points, θ m and φ m are all marginal function coefficients.

[0073] Similar to edge detection in image processing by calculating the maximum gradient, this embodiment calculates the maximum feature difference between the center point and other points in the neighborhood in point cloud processing to extract the most significant features in the local area to represent edge features. The SAGT module proposed in this embodiment realizes edge feature extraction through local graph convolution. Specifically, the module first uses the ball query method to extract feature points x within a certain radius around the key point. j , and limit the number of feature points. Then, this experiment selected an edge function with better performance to obtain local features, as shown in the formula:

[0074] e ijm =ReLU(θ m ·(f j -f i )+φ m ·f i ).

[0075] Then calculate the feature difference between the key points (f j-f i ), and compare it with the original feature point f i The two layers are input into a two-dimensional graph convolutional network, and local key features are extracted through the MLP layer and maximum pooling. The maximum pooling formula is as follows:

[0076]

[0077] Where, It is a local feature.

[0078] In addition, the global features obtained by global key points are combined and the global features are dynamically adjusted using linear layers and learnable parameters α. With local features The ratio between them is calculated, thus giving the key points the fused local and global features:

[0079]

[0080] Where, f i ^ is the fusion feature.

[0081] This method fully utilizes the global shape information and local gradient information of the object. This method has strong interpretability and has shown satisfactory results in practical applications, effectively improving the performance and accuracy of point cloud segmentation tasks.

[0082] S3: Loss Function Design: To address the challenges of class imbalance, spatial continuity, and multi-scale features, the traditional cross-entropy loss is widely used to measure the deviation between predicted class probabilities and true labels, but this can easily lead to overconfidence in certain classes. To address this, we introduce a label smoothing strategy that converts hard labels into soft labels. By adjusting the target class probability and assigning a small probability to non-target classes, this significantly enhances robustness to noisy data, thereby improving the model's generalization and stability.

[0083] Point cloud data has characteristics such as sparsity, irregularity, spatial structure, and geometric features. Therefore, loss function design usually needs to deal with issues such as class imbalance, maintaining spatial continuity, utilizing geometric distance, incorporating global context, and processing multi-scale features. Cross-entropy loss is widely used in point cloud segmentation tasks. It measures the error of classification results by comparing the predicted class probability distribution with the true class label. Label smoothing reduces the model's overconfidence in a certain category by converting one-hot labels into soft labels, thereby enhancing the model's generalization ability. The cross-entropy loss function is defined as follows:

[0084]

[0085] Where C is the number of target categories, p(x i ) is the smoothed label, q(xi ) is the probability that the model predicts belongs to this class, the probability value of the target class changes from 1 to a smaller value, and the remaining classes also obtain small probability values, which has strong robustness to noise data in the point cloud. In addition, the label smoothing strategy is introduced in this embodiment, which effectively improves the generalization ability and stability of the segmentation model, ensuring that the model can better adapt to various crane components and environmental changes.

[0086] S4: Model training and inference: In the model training and inference stage, the network is trained using the simulated automobile crane point cloud dataset, and the segmentation results are compared with the manually labeled results.

[0087] Among them, the number of simulated crane point cloud data points obtained by radar scanning is about 600k, to remove noise, a simple voxel filtering algorithm (SOR) is used for filtering, and the normal vector of the point cloud is calculated. Then, a small amount of samples are selected for labeling key components, and these labeled data are used for model training and verification, and finally inference is performed on unlabeled crane point cloud dataset. On the validation set, performance indicators such as intersection over union (IoU) are quantitatively evaluated, while on the test set, qualitative inspection is performed to verify the accuracy of the segmentation results.

[0088] In the training stage, an Nvidia A800 80G graphics card is used to train the network, the input dense point cloud is first processed through data augmentation and normalization, then voxel gridding is performed to ensure that the number of points of each sample is fixed at 36000, and the batch size is set to 4. During training, Smooth Cross Entropy is used as the loss function, and a label smoothing technique of 0.2 is applied. The optimizer selects AdamW, the weight decay is set to 1.0e-4, the initial learning rate is 0.01, and the cosine annealing learning rate scheduler is used, the minimum learning rate is set to 1.0e-5. The training process runs for a total of 200 epochs (training rounds), and the best model is obtained on the 78th epoch.

[0089] Using 7 simulated crane point cloud files labeled by hand as the validation set, the model's prediction results and manual labeling achieve high consistency. As shown in Figure 9 , the overall accuracy on the validation set reaches 99.26%, the average accuracy of the class is 97.07%, and the average intersection over union is 95.72%.

[0090] The beneficial technical effects of this embodiment are: improving the accuracy of point cloud segmentation in complex scenarios, suitable for segmentation of key components of automobile cranes.

[0091] Although the above embodiments have been described with reference to the accompanying drawings, it is to be understood that the present application is not limited to the embodiments disclosed herein, but that various modifications and changes can be made thereto without departing from the scope of the application as set forth in the claims.

Claims

1. A 3D point cloud segmentation method for truck crane based on deep learning, characterized by The method comprises the following steps: S1: Place a 3D model of a truck crane in different working states, created using SolidWorks software, in a simulation scene built on the Gazebo platform, and densely map the 3D model of the truck crane using the Fast-LIO2 algorithm to obtain 3D dense point cloud data of the truck crane. S2: constructing a SegNet model of the truck crane based on the three-dimensional dense point cloud data of the truck crane; S3: Based on the label smoothing strategy, set the loss function of the SegNet model; S4: Train and reason about the SegNet model.

2. The deep learning-based 3D point cloud segmentation method for a truck crane according to claim 1, characterized in that In step S1, the jib of the truck crane is provided with seven rotation angles and five pitch angles. The seven rotation angles of the jib of the truck crane are 0°, ±30°, ±45°, and ±60°, respectively. The five pitch angles of the jib of the truck crane are 0°, 10°, 30°, 45°, and 60°, respectively. Three-dimensional models of the truck crane in 35 different working states are obtained by combining the above.

3. The deep learning-based 3D point cloud segmentation method for a truck crane according to claim 1, characterized in that In step S1, the dense mapping method steps are as follows: The input data from the lidar is passed to the feature extraction module to obtain planar features and edge features. The extracted features and inertial measurement unit measurements are input into the state estimation module, which performs state estimation in the frequency range of 10Hz to 50Hz. The estimated pose registers the feature points into the global coordinate frame and merges them with the currently constructed feature point map. The updated map will add more new points for the next step of the mapping process.

4. The deep learning-based 3D point cloud segmentation method for a truck crane according to claim 1, characterized in that In step S2, before constructing the SegNet model of the truck crane, the three-dimensional dense point cloud data of the truck crane is preprocessed. The preprocessing method steps are as follows: To address the sparsity and noise issues of the three-dimensional dense point cloud data, denoising, filtering and downsampling operations are performed.

5. The deep learning-based 3D point cloud segmentation method for a truck crane according to claim 1, characterized in that In step S2, the SegNet model consists of an encoder, a decoder, and a segmentation head. The encoder has five, namely, one MLP module and four SAGT modules, the decoder has four, and the segmentation head has one. The steps of the method for constructing the SegNet model are as follows: The input three-dimensional dense point cloud data is encoded by the MLP module to extract high-dimensional features; the encoded features are downsampled multiple times using the four SAGT modules, gradually reducing the number of features while increasing the number of feature channels, so as to more effectively capture deep features at different levels. In this process, the point cloud features extracted by each encoder are passed to the decoder through a jump connection and feature fused with the output of the decoder of the previous layer.

6. The deep learning-based 3D point cloud segmentation method for a truck crane according to claim 5, characterized in that In the method for constructing the SegNet model, the steps of the feature fusion method are as follows: The input three-dimensional dense point cloud data is downsampled by adaptive voxel filtering to obtain key points containing global information as spatial fixed points δ s Enter the global feature extraction part of the SAGT module, downsample the three-dimensional dense point cloud data through the farthest point sampling method, and extract several global key points x while retaining the general outline of the object i ; For the global key point x i With the spatial fixed point δ s Solve the Euclidean distance and calculate the Gaussian kernel value of each pair of points to obtain the similarity between each pair of points. The similarity formula is as follows: g(δ s ,x i )=∑exp(-γ||δ s -x i || 2 ); In the formula, g(δ s ,x i ) is the spatial weighting function, γ is the weighting coefficient; Similarity weighted to the original feature point f i In the example, the original feature point f i Contains global key information and performs weighting on the semantic space of features. The semantic weighting formula is as follows: In the formula, h(z s ,f i ) is the semantic weighting function, z s is a semantic fixed point, i ' is the original feature point index, f i' For index i ' The original feature points of Combined with the semantic weighting function h(z s ,f i ) and the spatial weighting function g(δ s ,x i ) and the original feature point f i , the extracted global feature information is obtained through the cross attention mechanism: Where, is a global feature; Local edge features are extracted through graph convolution. The graph convolution formula is as follows: x ij =max j:(i,j)∈ε h Θ (x i ,x j ); Where h Θ is the edge function, the global key point x i is the center point, x j is x i Neighbor key points, θ m and φ m All are marginal function coefficients; Marginal function h Θ The specific options are as follows: from ijm =ReLU(θ m ·(f j -f i )+φ m ·f i ); Calculate the feature difference between key points (f j -f i ), and compare it with the original feature point f i They are input into a two-dimensional graph convolutional network, and local key features are extracted through the MLP layer and maximum pooling; The maximum pooling formula is as follows: Where, It is a local feature; Combined with the global features obtained by global key points, the global features are dynamically adjusted using linear layers and learnable parameters α With local features The ratio between them is calculated, thus giving the key points the fused local and global features: Where, f i ^ For fusion features.

7. The deep learning-based 3D point cloud segmentation method for a truck crane according to claim 1, characterized in that In step S3, the loss function adopts the cross entropy loss function, which is defined as follows: Where C is the number of target categories, p(x i ) is the smoothed label, q(x i ) is the probability that the model predicts belongs to that category.

Images