Urban street point cloud semantic segmentation method based on self-attention global feature enhancement

Abstract

The application provides a kind of city street point cloud semantic segmentation method based on self-attention global feature enhancement, comprising: S1, input a city street large scene point data as down-sampling layer, the global feature extracted from down-sampling layer is carried out in the self-attention global feature coding module based on the coding operation of self-attention, and it is remapped to the one-dimensional vector matched with the input feature channel, the channel self-attention value is obtained, and then it is adaptively enhanced with the original global feature;S2, the feature output by the self-attention global feature coding module in step S1 is transposed and fully connected by the global semantic mapping module, the probability that each category exists is output, the multiclass cross-entropy loss is formed by the vector composed of each category probability and the true value, and the cross-entropy loss of point label dimension is combined to further optimize the network parameters.The application focuses on the global feature of city street scene, and the semantic segmentation precision effect is improved by enhancing the global feature channel by channel.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision, and particularly relates to a city street point cloud semantic segmentation method based on self-attention global feature enhancement. BACKGROUND

[0002] The point cloud of a city street scene contains a large amount of valuable information related to human and nature, and semantic segmentation of the point cloud is the basis and key for a series of subsequent important applications. The self-attention mechanism designed in some point cloud segmentation deep learning models often only focuses on local regions, ignoring the global information rich in spatial and semantic features extracted by the down-sampling network. At present, thanks to the rapid development of the field of deep learning, the application of deep learning methods to three-dimensional point cloud semantic segmentation often achieves better results than the classic machine learning segmentation methods of manually extracted features. There are three common methods in the prior art, which are three-dimensional point cloud research based on deep learning, application of attention mechanism to point cloud, and point cloud semantic segmentation of large-scale city street scenes.

[0003] Deep learning is widely used in computer vision and pattern recognition, and three-dimensional point cloud analysis based on deep learning has become a hot issue. Convolutional neural network (CNN) is a representative model of deep learning, which has excellent performance on structured data such as two-dimensional images. However, for irregular point cloud data, CNN cannot be directly used for processing. Point cloud-oriented deep learning methods mainly include projection-based, voxel-based, point-based and graph-based methods. The projection-based method projects 3D point cloud onto a 2D plane to convert the point cloud into a set of images rendered from different views, and uses a traditional convolutional neural network for feature learning. The voxel-based method converts the voxelization of the point cloud into a 3D grid, so as to use 3D convolution to process the point cloud. The point-based method directly performs operations on the point cloud, and the pioneering work PointNet uses a shared multi-layer perceptron (MLP) to extract information from the point cloud in a point-by-point manner, but the defect of PointNet is that it cannot extract local features. PointNet++ overcomes this defect and continuously extracts local features from different scales, making it suitable for complex point cloud scenes. Since graphs have the potential to describe the shape and geometric structure of irregular 3D point clouds, some studies also use graph structures to process point clouds.

[0004] Attention mechanism has been preliminarily applied in the deep learning method of point cloud, and the purpose is to focus on a certain target or a certain scene, and form different attention distribution inside the target and other spatial positions in the scene. The attention model essentially changes the weighting of the target data. GACNet proposes a graph attention convolution, which dynamically adjusts the convolution kernel to adapt to the shape of the target structure by learning the attention weight, and effectively improves the accuracy of point cloud semantic segmentation. TANet forms triple attention by using attention mechanism in point dimension, voxel dimension and channel dimension, so that the model is robust to noise to achieve detection results. Inspired by the great success of the self-attention mechanism-based transformer (Transformer) in natural language processing, the Transformer structure is applied to the three-dimensional spatial data set, and a point Transformer structure that is invariant to point cloud pair arrangement and base number is designed, which performs excellently in point cloud classification and semantic segmentation tasks.

[0005] Point cloud semantic segmentation faces challenges on large-scale urban street scene datasets. Large-scene point cloud datasets such as Semantic3D and Semantic Kitti are characterized by large number of point clouds, wide distribution range and usually composed of multiple semantic classes, thus posing great challenges to deep learning feature extraction models. PointNet++ proposes a multi-level feature extraction structure and uses a multilayer perceptron (MLP) to extract point cloud features, achieving good results in the Semantic Kitti dataset. SnapNet projects the input point cloud into an image at multiple angles and uses a convolutional neural network (CNN) to learn effective features and recover point labels from pixel labels, thus achieving the purpose of large-scene point cloud semantic segmentation. SPG re-expresses point cloud information by using a super point graph, which is rich in edge information and reduces feature loss in the point cloud representation process, and based on this, a deep learning model for large-scale point cloud semantic segmentation is established. RandLA-Net model uses random sampling as a downsampling method to reduce the resolution of input large-scene point cloud, and uses a local feature aggregation module at each downsampling layer to maximize the preservation of feature information lost in random sampling, effectively improving the speed and accuracy of large-scene point cloud segmentation.

[0006] Patent CN110188817B discloses a real-time high-performance street view image semantic segmentation method based on deep learning, including preparing street view image training, verification and test data set; downsampling the data set image to reduce the resolution of the image; modifying the existing lightweight classification network as a basic feature extraction network for semantic segmentation; connecting a discriminative hole space pyramid pooling after the basic feature extraction network to solve the multi-scale problem of semantic segmentation; stacking several convolution layers to form a shallow spatial information preservation network; using a feature fusion network to fuse the obtained feature maps to form a prediction result; comparing the output image with the semantic annotation image in the data set, using the back propagation algorithm for end-to-end training to obtain a real-time high-performance street view image semantic segmentation network model; inputting the street view image to be tested into the real-time high-performance street view image semantic segmentation network model to obtain the semantic segmentation result of the street view image.

[0007] In addition, there are also patents disclosing related network segmentation methods, such as patent CN113642390A discloses a street view image semantic segmentation method based on local attention network, based on the segmentation structure of encoder-decoder, the encoder network extracts features layer by layer, while the decoder restores the resolution of the image step by step through upsampling, achieving the purpose of classifying each pixel in the image, during convolution operation, each feature map is given different weights through the local attention network module, the weight allocated is larger for the feature map with segmentation gain, while the weight is smaller for the redundant feature map, which can significantly improve the discrimination ability of the network model for each class, reduce the inconsistency of intra-class segmentation, and improve the visual smoothness effect of semantic segmentation. And patent CN110188817B proposes a real-time high-performance street view image semantic segmentation method, by adding hole convolution and dense skip layer connection to the existing real-time lightweight classification network, the lightweight classification network is modified into a basic feature extraction network; then, an improved hole space pyramid pooling is connected behind the obtained basic feature extraction network to more effectively solve the multi-scale problem of semantic segmentation; then, a spatial information preservation network is designed to encode sufficient spatial detail information, which is parallel to the above two network branches; finally, a specially designed feature fusion network is used to connect the above two features belonging to different levels.

[0008] In summary, the prior art does not use self-attention mechanism in urban large scene dataset point cloud semantic segmentation, and ignores the weight values of each semantic category. Moreover, most of the current point cloud semantic segmentation work does not pay much attention to and utilize the global feature information in the large scene, but directly predicts the semantic labels of each point through upsampling operation of the global feature information extracted by the downsampling network, which limits the semantic labeling result due to insufficient extraction of point cloud features. Therefore, a deep learning method for large-scale point cloud semantic segmentation of global features in urban street scenes is proposed, which enhances the global features channel by channel to improve the accuracy of semantic segmentation. SUMMARY

[0009] To achieve the above-mentioned purpose, the present application focuses on efficient attention mechanism, uses deep learning method to extract global features, further proposes a global feature encoding module, and applies it to global features to enhance global position features and global semantic features, highlight feature expression ability, and further improve the effect of semantic segmentation.

[0010] The present application provides the following technical solutions:

[0011] A city street point cloud semantic segmentation method based on self-attention global feature enhancement, comprising:

[0012] S1, the global feature extracted from the city street large scene point data is subjected to self-attention based coding operation in the self-attention global feature encoding module, and is remapped to a one-dimensional vector matching the input feature channel to obtain channel self-attention value, and then each channel is adaptively enhanced with the original global feature;

[0013] S2, the features output by the self-attention global feature encoding module in step S1 are transposed and fully connected through the global semantic mapping module to obtain the probability of existence of each category, and the vector composed of the probability of each category and the true value forms a multi-category cross-entropy loss, and the cross-entropy loss of the point label dimension is combined to further optimize the network parameters.

[0014] Preferably, in step S1, the self-attention global feature encoding module comprises a self-attention encoding layer and a feature channel strengthening layer.

[0015] Preferably, S1 is specifically:

[0016] S1.1, the extracted global feature is first taken as an input module in the self-attention encoding layer, and the linear transformation function f i and the activation function are used in formula (1) three times to remap the global feature to 128xr1 dimension, 128xr2 dimension and 128xr3 dimension.

[0017] S1.2, the self-attention value is calculated by a Softmax activation function, and then multiplied with the global feature extracted before in the channel dimension, and then the tensor is accumulated first and then tiled to the same dimension as the original feature;

[0018] S1.3, f raw ,f i and f att perform a connection operation, and the obtained feature map is input into a feature channel enhancement layer after an MLP to obtain a feature map with the same dimension as the input;

[0019] The formula is as follows:

[0020]

[0021] f att =SUM_CP(δ(f i )×f i ) (2).

[0022] Preferably, the feature obtained by the self-attention encoding layer is subjected to the formula operation, to obtain a 1×r-dimensional vector

[0023] Preferably, the 1×r-dimensional vector obtained is subjected to the formula mapped to the range of 0-1 by a Sigmoid function to form a channel attention, and then the channel attention is multiplied with f input channel by channel and then added, so as to enhance the information in the global feature.

[0024] Preferably, S2 is specifically as follows:

[0025] S2.1, the global semantic mapping module takes the output feature of the self-attention global feature encoding module in step S1 as input, and maps it to the required total semantic category by transposition and full connection operation to form a 1×n-dimensional global semantic category vector, and each value in the vector represents the semantic label probability value of each semantic category of the input point cloud scene;

[0026] S2.2, the output is obtained by feature extraction of a down-sampling layer and re-mapping of the output feature of the global feature encoding module, and the semantic label prediction value corresponding to each point of the original point cloud data is obtained. The predicted value and the true value are calculated by multi-label classification cross-entropy loss, and the loss value of the scene dimension is obtained.

[0027] Preferably, the weighted cross-entropy loss is calculated by the semantic label prediction value and the point cloud scene marked by the true value, so as to obtain the point dimension loss (L pt ), and the calculation formula is as follows:

[0028]

[0029] where n is the number of categories, v i is the proportion of the number of samples in the i-th category to the total number of samples, p i (x) is the probability of point x in the i-th category, q i (x) is the real distribution of the label of point x.

[0030] Preferably, the global semantic mapping module maps the encoded global features to each semantic category, and then compares them with the real values of the scene to form a general cross-entropy loss L sce of multi-label classification, which is expressed as follows:

[0031] L sce = -p(x i ) * log(q i (x)) (5).

[0032] Preferably, the point cloud semantic label loss and the scene semantic label loss in the general cross-entropy loss L sce of multi-label classification are often not in the same order of magnitude, so the following method is used for adjustment:

[0033] L global = L pt + m * L sce (6);

[0034] By using the hyperparameter m in formula (6), the two-dimensional loss function can be calculated, where L sce is the general cross-entropy loss, and L pt is the loss of the point dimension.

[0035] The present application has at least one of the following beneficial effects:

[0036] 1) The present application proposes a self-attention global feature encoding module, which is applied to global features, can enhance global position features and global semantic features, and highlight feature expression capability; on the other hand, in order to efficiently utilize the semantic information in the global features, the present application further proposes a global semantic mapping module for scene semantic labeling, thereby realizing accurate segmentation of city street large scene point cloud data.

[0037] 2) Compared with the prior art, directly predicting the semantic labels of each point by upsampling operation of the global feature information extracted by the downsampling network leads to the problem that the semantic labeling result is insufficiently learned by the point cloud feature, and the application remaps the global feature information extracted by the downsampling layer to a one-dimensional vector matched with the input feature channel through the operation of the self-attention global feature encoding module, obtains the channel self-attention value, and then adaptively enhances each channel with the original global feature, so as to achieve the effect of information enhancement in the global feature.

[0038] 3) The self-attention global feature encoding module combines the self-attention mechanism to encode and process the global feature, improves the expression ability of the global feature, and the global semantic mapping module remaps the intermediate layer feature after the self-attention global feature encoding to each semantic category, combines the weight value formed by the number of samples of each semantic category, improves the learning ability of the model to the feature, and effectively improves the segmentation effect. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creating any creative labor. In all the drawings, similar elements or parts are generally denoted by similar reference numerals, identifiers. In the drawings, each element or part is not necessarily drawn according to the actual proportion.

[0040] Figure 1 is a schematic diagram of the global feature encoding layer in the present application;

[0041] Figure 2 is a schematic diagram of the global semantic mapping module in the present application;

[0042] Figure 3 is a technical route block diagram of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. In the following description, specific details such as specific configurations and components are provided only to help understand the embodiments of the present application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, in order to be clear and concise, the description of known functions and structures is omitted in the embodiments.

[0044] It should be understood that the reference herein to“one embodiment” or“the embodiment” means that a particular feature, structure, or characteristic described is included in at least one embodiment of the application. Therefore, appearances of“one embodiment” or“in one embodiment” or“in the embodiment” in various places in the specification are not necessarily referring to the same embodiment. Furthermore, various specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0045] In addition, reference numerals and / or letters can be repeated in different instances throughout the specification. Such repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and / or configurations discussed.

[0046] The term“and / or” herein is merely used to describe associated objects, indicating that there can be three types of relationships, for example, A and / or B can mean that A exists alone, B exists alone, and A and B exist simultaneously. The term“ / and” herein is used to describe another associated object relationship, indicating that there can be two types of relationships, for example, A / and B can mean that A exists alone, and A and B exist simultaneously. In addition, the character“ / ” herein generally indicates that the associated objects before and after are in an“or” relationship.

[0047] The term“at least one” herein is merely used to describe associated objects, indicating that there can be three types of relationships, for example, at least one of A and B can mean that A exists alone, A and B exist simultaneously, and B exists alone.

[0048] It should also be noted that the relationship terms such as first and second, etc. used herein are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms“include”,“contain” or any other variants thereof are intended to cover non-exclusive inclusion.

[0049] Embodiment 1

[0050] The application introduces a city street point cloud semantic segmentation method based on self-attention global feature enhancement, comprising:

[0051] S1, input a city street large scene point data as a down-sampling layer, perform a self-attention coding operation on the global feature extracted from the down-sampling layer in a self-attention global feature coding module, and remap it to a one-dimensional vector matching the input feature channel to obtain a channel self-attention value, and then adaptively enhance it with the original global feature channel by channel;

[0052] S2, the feature output by the self-attention global feature encoding module in step S1 is transposed and fully connected by the global semantic mapping module to obtain the probability of the existence of each category, and a multi-category cross-entropy loss is formed by the vector composed of the probability of each category and the true value, and the cross-entropy loss of the point label dimension is combined to further optimize the network parameters.

[0053] In step S1, the down-sampling layer uses random sampling as the down-sampling method to reduce the resolution of the input large scene point cloud, and a multi-layer perception is used at each down-sampling layer to extract features from the local point cloud aggregated by KNN (K-Nearest Neighbor algorithm). By using the nearest neighbor algorithm (KNN) and the point dimension multi-layer perception (MLP), the features can be effectively extracted from the massive point cloud, and the use of random sampling as the down-sampling method greatly improves the model speed; the self-attention global feature encoding module and the global semantic mapping module pay attention to the global features extracted by the down-sampling layer, encode the global features, and enhance the global features through channel self-attention, so as to adaptively enhance the expression ability of the feature information and the expansibility of the model, improve the segmentation effect and accuracy, and realize the end-to-end of the model.

[0054] Embodiment 2

[0055] Based on embodiment 1, this embodiment further illustrates the self-attention global feature encoding module.

[0056] Combined with the accompanying Figure 1 , Figure 1 is a schematic diagram of the global feature encoding module in the present application.

[0057] The self-attention global feature encoding module includes a self-attention encoding layer and a feature channel strengthening layer.

[0058] Preferably, S1 is specifically:

[0059] S1.1, first, the extracted global feature is taken as the input module in the self-attention encoding layer, and the linear transformation function f i and the activation function are used to remap the global feature to 128xr1, 128xr2 and 128xr3 dimensions.

[0060] S1.2, the self-attention value is calculated by the Softmax activation function, and then multiplied with the previously extracted global feature in the channel dimension, and the tensor is first accumulated and then tiled to the same dimension as the original feature.

[0061] S1.3, f raw , f i and f attThe connection operation is performed, and the obtained feature map enters a feature channel strengthening layer after an MLP to obtain a feature consistent with the input dimension;

[0062] The formula is as follows:

[0063]

[0064] f att =SUM_CP(δ(f i )×f i ) (2).

[0065] The feature obtained by the self-attention encoding layer is subjected to the formula Operation, to obtain a 1xr-dimensional vector

[0066] The obtained 1xr-dimensional vector is mapped to the range of 0-1 through a Sigmoid function, a channel attention is formed, and then a method similar to the residual structure is used to multiply the channel attention by f input , and then add them, so as to achieve the effect of information enhancement in the global feature.

[0067] By applying the self-attention global feature encoding module to the global feature, the global position feature and the global semantic feature can be enhanced, and the feature expression capability can be highlighted.

[0068] Embodiment 3

[0069] On the basis of embodiments 1 and 2, in combination with the attached Figure 1 、 2 , this embodiment describes the global semantic mapping module. Figure 1 is a schematic diagram of the global feature encoding layer of the application, Figure 2 is a schematic diagram of the global semantic mapping module of the application.

[0070] The global semantic mapping module is to transpose and fully connect the output feature of the previous module, output the probability of each category existing, and then form a multi-category cross-entropy loss through the vector composed of the probabilities of each category and the true value, and the cross-entropy loss of the point label dimension is combined to further optimize the network parameters.

[0071] The S2 is specifically:

[0072] S2.1, the global semantic mapping module takes the output feature in the self-attention global feature encoding module in step S1 as input, and maps it to the total semantic category required for classification through transposition and full connection operation, to form a 1xn-dimensional global semantic category vector, and each value in the vector represents the semantic label probability value of each semantic category of the input point cloud scene;

[0073] S2.2, by downsampling the feature extraction and output feature remapping in the global feature encoding module, the output corresponding to the semantic label prediction value of each point of the original point cloud data, the cross entropy loss calculation of multi-label classification of the predicted value and the true value, the loss value of the scene dimension is obtained.

[0074] Further, the predicted value is calculated with the point cloud scene marked with the true value to obtain the weighted cross entropy loss, so as to obtain the point dimension loss (L pt ), the calculation formula is as follows:

[0075]

[0076] In the formula, n is the number of categories, v i is the proportion of the number of i-th category samples to the total number of samples, p i (x) is the probability of point x in i category, q i (x) is the real distribution of the label of point x.

[0077] The global semantic mapping module maps the encoded global features to each semantic category, and then compares them with the scene true value to form a multi-label classification ordinary cross entropy loss L sce , the expression is as follows:

[0078] L sce =-p(x i )×log(q i (x)) (5)。

[0079] Preferably, the point cloud semantic label loss and the scene semantic label loss in the formed multi-label classification ordinary cross entropy loss L sce are often not in the same order of magnitude, so the following method is used for adjustment:

[0080] L global =L pt +m×L sce (6);

[0081] Through formula (6), the hyperparameter m is used to perform normalization calculation on the two dimension loss functions, wherein L sce is the ordinary cross entropy loss, and L pt is the point dimension loss. The final formed loss is not particularly affected by the unevenness of the point dimension or global dimension semantic loss, and a better optimization effect is achieved.

[0082] The global semantic feature extraction of the global contained category can interactively affect the accuracy of the output point label, and the improvement of the point label accuracy can further improve the prediction accuracy of the global label, so as to achieve the effect of mutual promotion. In this way, not only the segmentation effect of the network can be improved, but also the generalization performance of the network is improved.

[0083] Embodiment 4

[0084] Based on the above embodiments 1-3, combined with the attached Figure 3 , Figure 3 is a technical route block diagram of the present application. The implementation of the semantic segmentation method at least includes the following stages:

[0085] The first stage is to input a city street scene point data, wherein the data for training needs to be labeled with true value. The input data is first subjected to scene point cloud feature extraction and down-sampling. In this stage, the input point cloud will complete feature extraction through a local feature aggregation module, and then the extracted features will be reduced in number and point cloud resolution through random sampling. The stage outputs the global features aggregated with the entire input scene point cloud rich in semantic, spatial form and texture information.

[0086] The second stage is based on self-attention global feature channel-by-channel enhancement. In this stage, the global features aggregated in the previous stage are encoded by a self-attention global feature encoding module to obtain one-dimensional channel self-attention values, and then multiplied and added with the original global features to achieve channel-by-channel enhancement of the global features.

[0087] In the last stage of this stage, the one-dimensional channel self-attention values are input as intermediate layer features into a global semantic mapping module to obtain probabilities containing various semantic categories, form a scene dimension semantic loss function for the learning of the scene classifier in the training stage, and output the scene semantic label in the test stage according to the category probability.

[0088] The third stage is point cloud feature up-sampling and point cloud point-by-point labeling. The channel-by-channel enhanced global features are input into an up-sampling layer, and receive the same level down-sampling features through a skip layer connection to enrich the feature information. The up-sampling layer adopts a bilinear interpolation method, and receives the same level down-sampling features through a skip layer connection to retain the loss features. After bilinear interpolation, the semantic category to which each point of the input point cloud scene belongs is finally predicted, and a point dimension semantic loss is generated. In the training stage, the point dimension classifier is optimized, and in the test stage, the prediction value of the point label is obtained.

[0089] The fourth stage enters the precision evaluation link of point cloud segmentation. In this link, the classification results will be qualitatively analyzed and observed in the form of pictures or videos, and the segmentation accuracy statistical results will be output for quantitative observation. The segmentation results of other methods will also be compared synchronously, and finally the segmentation error is analyzed to further optimize the model parameters.

[0090] Example 5

[0091] Based on Example 4, the accuracy evaluation link of the fourth stage entering point cloud segmentation is further illustrated.

[0092] The comparative indicators used in this evaluation method include the intersection over union (IoU) and the overall accuracy (OA). The IoU and the mean intersection over union (mIoU) are calculated as follows:

[0093]

[0094]

[0095] where n is the number of classes.

[0096] The specific calculation formula of OA is as follows:

[0097]

[0098] where TP represents the number of points with consistent predicted and ground truth categories, TN (True Negative) represents the number of points that should be correctly classified but are incorrectly classified by the model, FP (False Positive) represents the number of points that are incorrectly classified into the category, and FN (False Negative) represents the number of positive samples that are incorrectly classified as negative samples.

[0099] The above evaluation method is a comprehensive evaluation method for laser radar point cloud semantic segmentation based on qualitative, quantitative and comparative analysis. This method is not only suitable for point-based segmentation evaluation, but also can evaluate the segmentation effect from the perspective of entity objects.

[0100] Table 1: Comparison of results on the Semantic3D dataset

[0101]

[0102] Table 2: Comparison of results on the CASIC dataset

[0103]

[0104]

[0105] Through the comparison of the data in Tables 1 and 2, it is shown that the present application is superior to most existing model methods, and the mIoU can reach more than 78% (Semantic3D).

[0106] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application in any form or in essence. It should be noted that those skilled in the art can make some changes, modifications and equivalent changes of the above disclosed technical contents without departing from the spirit and scope of the present application, and all equivalent embodiments of the present application are included. Meanwhile, any equivalent changes, modifications and evolution of the above embodiments according to the essential technology of the present application are still within the scope of the technical solutions of the present application.

Claims

1. A city street point cloud semantic segmentation method based on self-attention global feature enhancement, characterized in that, Comprise: S1, input a city street scene point data as a down-sampling layer, encode the global feature extracted from the down-sampling layer in the self-attention global feature encoding module based on self-attention, and remap it to a one-dimensional vector matching the input feature channel to obtain the channel self-attention value, and then adaptively enhance it with the original global feature channel by channel; The self-attention global feature encoding module comprises a self-attention encoding layer and a feature channel strengthening layer; The features obtained from the self-attention encoding layer are subjected to the formula (3) operation to obtain a 1 x r-dimensional vector ; The obtained 1 x r-dimensional vector is mapped to the 0-1 range through a Sigmoid function to form channel attention, and then the channel attention is multiplied by the global feature to obtain a feature vector of the same dimension as the global feature. The feature vector is mapped to the 0-1 range through a Sigmoid function to form channel attention, and then the channel attention is multiplied by the global feature to obtain a feature vector of the same dimension as the global feature. The feature vector is mapped to the 0-1 range through a Sigmoid function to form channel attention, and then the channel attention is multiplied by the S2, the features output by the self-attention global feature encoding module in step S1 are transposed and fully connected by the global semantic mapping module, the probability of the existence of each category is output, and the vector composed of the probability of each category and the true value forms a multi-category cross-entropy loss, and the cross-entropy loss of the point label dimension is combined to further optimize the network parameters; The common cross-entropy loss of the formed multi-label classification The point cloud semantic label loss and the scene semantic label loss in the common cross-entropy loss of the formed multi-label classification are not in the same order of magnitude, so the following method is adopted for adjustment: （6）； By formula (6) using hyperparameters m , can be calculated on two-dimensional loss function, where is the ordinary cross-entropy loss, is the loss of point dimension; S1 specifically is: S1.1 First, the global features extracted by the downsampling layer are used as the input module in the self-attention coding layer, and then the linear transformation function in the cubic formula (1) is applied. and activation function Remap global features to 128× Dimensions, 128× Maintenance and 128× dimension; S1.2, calculate the self-attention value by the Softmax activation function, and then multiply the global feature extracted before by channel dimension, and then perform the accumulation operation on the tensor, and then flatten it to the same dimension as the original feature; S1.3、 With The connection operation is performed, and the obtained feature map enters the feature channel enhancement layer after the MLP to obtain the same dimension as the input. Wherein, the formula is as follows: （1）； （2）。 2. The city street point cloud semantic segmentation method based on self-attention global feature enhancement according to claim 1, characterized in that, S2 specifically is: S2.1, the global semantic mapping module takes the output feature in the self-attention global feature encoding module in step S1 as input, and remaps it to the total semantic category required for classification through transposition and full connection operation, forming a 1×n-dimensional global semantic category vector, and each value in the vector represents the semantic label probability value of each semantic category of the input point cloud scene; S2.2, through the feature extraction of the down-sampling layer and the remapping of the output feature in the global feature encoding module, the predicted value of the semantic label corresponding to each point of the original point cloud data is output, and the predicted value and the true value are calculated by the cross-entropy loss of multi-label classification to obtain the loss value of the scene dimension.

3. The city street point cloud semantic segmentation method based on self-attention global feature enhancement according to claim 2, characterized in that, The semantic label prediction value is calculated with the point cloud scene marked with true value to obtain a weighted cross-entropy loss, so as to obtain a loss in the point dimension L pt , and the calculation formula is as follows: （4）， In the formula n is the number of classes, is the number of classes, i is the proportion of the number of samples of the class in the total number of samples, is the point x In i is the probability of the class, is the point x is the true distribution of the label.

4. The city street point cloud semantic segmentation method based on self-attention global feature enhancement according to claim 2 or 3, characterized in that, The global semantic mapping module maps the encoded global features to each semantic category, and compares with the scene ground truth to form a multi-label classification ordinary cross-entropy loss , and the expression is as follows: （5）。

Images