Airborne point cloud classification method and system considering global-local self-attention mechanism

Abstract

The invention provides an airborne point cloud classification method and system considering a global-local self-attention mechanism. The method comprises the steps of 1, selecting an ISPR-3D airborne point cloud data set; 2, preprocessing the point cloud data; step 3, constructing an edge convolutional network based on the point cloud data, and extracting local features of the point cloud; 4, constructing a local self-attention mechanism network based on the extracted local features of the point cloud; step 5, constructing a global self-attention mechanism network; step 6, constructing a global-local self-attention mechanism module; 7, adopting a U-net network design to complete an overall network structure; and step 8, inputting airborne laser point cloud test data to be classified into the trained overall network structure to obtain a category label of each point. The invention innovatively provides a global-local self-attention mechanism neural network integrating edge convolution for solving the problems of insufficient feature expression and unbalanced sample categories in airborne point cloud large scene classification.

Description

Technical Field

[0001] The present invention relates to the technical field of laser scanning data processing, and in particular to an airborne point cloud classification method and system taking into account a global-local self-attention mechanism. Background Art

[0002] Airborne laser radar (Light Laser Detection and Ranging, LiDAR) uses laser beams to scan the surface and combines with a positioning system to obtain high-resolution three-dimensional map data. It has the advantage of quickly obtaining large-scale, high-precision ground information. Classification is to classify data according to different features and attributes through specific algorithms and processing procedures. Airborne point cloud classification is the basis of three-dimensional scene understanding and plays an important role in terrain mapping, urban planning, forest monitoring, power line detection, and road detection. However, when processing large-scale point cloud data, airborne point cloud classification faces problems such as data noise, incompleteness, uneven sampling, and complex ground objects. Therefore, airborne point cloud classification still has the significance of further exploration.

[0003] At present, most of the airborne point cloud classification methods are implemented through traditional machine learning methods and deep learning methods. Traditional machine learning methods often use methods based on manual feature extraction in point cloud classification. For example, the Support Vector Machine (SVM) proposed by Zhang et al., the Random Forest (RF) proposed by He et al., the Markov Random Field (MRF) proposed by Munoz et al., and the Conditional Random Field (CRF) proposed by Niemeyer et al. This type of method first extracts features from the point cloud, including geometric features (such as normal vectors, curvature), statistical features (such as point density), and local features (such as shape context descriptors), and then uses these features to train classifiers to achieve classification and segmentation of different categories of features or objects in the point cloud. The challenge of traditional machine learning methods is that they need to manually design feature extraction methods, and the processing of the non-structured and high-dimensional nature of point cloud data is relatively complex, and they are easily affected by noise and uneven sampling, resulting in low classification accuracy.

[0004] Compared with traditional machine learning methods, deep learning methods automatically learn the feature expression of data through neural network structures, without the need to manually design feature extractors. They can process large-scale high-dimensional data and achieve end-to-end learning and prediction. In the early research on point cloud classification, most of them were based on voxels or projections. Voxel-based methods such as Rabbani et al. proposed to use smoothness constraint methods to segment point clouds. By voxelizing point cloud data, the modeling and constraints of smoothness in point clouds were achieved, thereby achieving effective segmentation. Projection-based methods such as Chen et al. proposed a multi-view 3D object detection network. First, the point cloud data was projected to multiple viewpoints, and then the semantic segmentation and detection of 3D objects were achieved using deep learning methods. However, both voxel-based and projection-based methods have disadvantages such as information loss, high computational complexity, and difficulty in processing irregular shapes. There are certain limitations in processing large-scale and complex point cloud data. With the emergence of PointNet and PointNet++ models, more and more scholars have paid attention to point-based end-to-end deep learning methods, and a series of improved methods have emerged. For example, Li et al. proposed a 3D MultiscalePointNet method, which improved the performance of PointNet++ through multi-scale feature learning and aggregation, and achieved better results in point cloud processing tasks. Inspired by the self-attention mechanism, the Point Transformer model proposed by Zhao et al. realizes the construction of the network model based on the self-attention mechanism. The model designs a self-attention layer and combines position encoding to construct a local Transformer block to achieve tasks such as semantic segmentation, component segmentation and classification. However, the above-mentioned point-based deep learning models in the field of computer vision only focus on exploring local geometric structures, while ignoring the global contextual relationship between points. In addition, most of the above-mentioned deep learning methods classify indoor scene point clouds, but it is difficult to accurately classify various typical ground objects from discrete, large-scale, complex ground objects, uneven category distribution and noisy airborne point cloud data, which has become one of the research hotspots in related fields in recent years. Summary of the invention

[0005] In view of the problems of insufficient feature expression and unbalanced sample categories in the classification of large scenes of airborne point clouds, the present invention provides an airborne point cloud classification method and system taking into account the global-local self-attention mechanism. The present invention designs an edge convolutional network, a local self-attention mechanism network and a global self-attention mechanism network, and on this basis realizes a large scene classification method of airborne point clouds based on deep learning.

[0006] According to a first aspect of the present invention, the present invention provides an airborne point cloud classification method taking into account a global-local self-attention mechanism, comprising the following steps:

[0007] Step 1: Select the ISPRS-3D airborne point cloud dataset;

[0008] Step 2: Preprocess the point cloud data in the airborne point cloud dataset;

[0009] Step 3: Build an edge convolutional network based on point cloud data to extract local features of the point cloud;

[0010] Step 4: Construct a local self-attention mechanism network based on the local features of the extracted point cloud;

[0011] Step 5: Build a global self-attention mechanism network;

[0012] Step 6: Construct a global-local self-attention mechanism module;

[0013] Step 7: Use U-net network design to complete the overall network structure;

[0014] Step 8: Input the airborne laser point cloud test data to be classified into the trained overall network structure to obtain the category label of each point.

[0015] Based on the above technical solution, the present invention can also make the following improvements.

[0016] Optionally, each point cloud data in the ISPRS-3D airborne point cloud dataset provides spatial XYZ coordinates, intensity, return number and semantic label.

[0017] Optionally, the preprocessing of the point cloud data in the airborne point cloud data set includes:

[0018] The intensity values ​​obtained from the airborne LiDAR data are input into the network to divide the entire scene into rectangular areas of preset sizes. During the model training phase, several rectangular areas are randomly selected, several points are randomly selected from each rectangular area, and a certain proportion of the points are randomly discarded. During the model testing phase, all points from the segmented rectangular areas in the test set are fed into the training model to achieve point-to-point classification. The small fragmented areas that appear at the edge of the scene are merged into the surrounding larger areas to ensure the integrity of each rectangular area.

[0019] Optionally, constructing an edge convolutional network based on point cloud data includes edge feature extraction, deep feature extraction and local feature extraction.

[0020] Optionally, the construction of the local self-attention mechanism network adopts a radius-based ball query method to apply the self-attention network in a local neighborhood around each data point.

[0021] Optionally, the construction of the global self-attention mechanism network includes the following steps:

[0022] For any point P i ∈P, select the global marker point P g ∈P; calculate any point P in the space in turn i To all other points P r ∈P(r≠i, r=0,1,…N), and obtain a distance matrix with dimension N*N;

[0023] Sort each row of the distance matrix D(i, r) in ascending order and return the sorted index D′;

[0024] Select the index of the first g elements of each row of index D′ to get the index of the g neighbor points closest to each point, and select g global marker points P g ;

[0025] The position encoding θ is added to the attention vector γ and the transformed feature α using a subtraction relationship to obtain the output feature.

[0026] Optionally, the construction of the global-local self-attention mechanism module includes:

[0027] The input point cloud feature information is transformed through a linear layer and a global self-attention layer to capture the global context information of the entire point cloud, and the two information are superimposed to obtain features containing global information;

[0028] The features containing global information are sent to the local self-attention layer to obtain further enriched local feature information;

[0029] The obtained local feature information is mapped to a higher dimension through a linear layer and processed by a global self-attention layer to integrate the global context information again;

[0030] The original input features are added to the transformed features to get the final output features.

[0031] Optionally, the overall network structure is completed by using U-net network design, including:

[0032] During downsampling, an edge convolution module is added to capture local features. The global and local features of the point cloud are extracted through the global-local self-attention mechanism module. Skip connections are added between the downsampling and upsampling stages to connect the low-level features from the downsampling stage with the features of the same resolution size in the upsampling stage. The point features of the upsampling layer are input into the fully connected layer, and the semantic labels of all input points are output.

[0033] Optionally, the airborne laser point cloud test data to be classified is input into the trained overall network structure to obtain the category label of each point, including:

[0034] Based on the ISPRS-3D airborne point cloud training sample set, the network model is trained to obtain the training results. The ISPRS-3D airborne point cloud test sample set to be classified is input to automatically classify the test data and evaluate the accuracy.

[0035] According to a second aspect of the present invention, there is provided an airborne point cloud classification system taking into account a global-local self-attention mechanism, comprising:

[0036] Data acquisition module, used to select ISPRS-3D airborne point cloud data set;

[0037] A data preprocessing module, used for preprocessing point cloud data in the airborne point cloud data set;

[0038] The network model construction module is used to construct an edge convolution network based on point cloud data to extract local features of the point cloud; construct a local self-attention mechanism network based on the extracted local features of the point cloud, construct a global self-attention mechanism network, and construct a global-local self-attention mechanism module;

[0039] The point cloud data classification module is used to complete the overall network structure using the U-net network design, input the airborne laser point cloud test data to be classified into the trained overall network structure, and obtain the category label of each point.

[0040] Technical effects and advantages of the present invention:

[0041] The present invention provides an airborne point cloud classification method and system that takes into account the global-local self-attention mechanism. In order to solve the problems of insufficient feature expression and imbalanced sample categories in large-scene classification of airborne point clouds, the present invention innovatively proposes a global-local self-attention mechanism neural network that integrates edge convolution. The network design of the present invention is based on the U-net framework, and an edge convolution layer is designed. The relative distance between points is used to construct a spatial relationship graph, which effectively captures neighboring features in point cloud data, helps to extract and express complex object boundaries and texture information, and enhances the classification accuracy and stability of the model. The present invention introduces a global-local self-attention mechanism module, which can take into account global context information while paying attention to the features of a few categories, improve the problem of category imbalance, and further improve the accuracy of classification.

[0042] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is an ISPRS-3D dataset provided by an embodiment of the present invention;

[0044] Figure 2 It is an ISPRS-3D data set preprocessing diagram provided by an embodiment of the present invention;

[0045] Figure 3 is a flow chart of edge convolution local feature extraction provided by an embodiment of the present invention;

[0046] Figure 4 Schematic diagram of a local self-attention mechanism module provided by an embodiment of the present invention;

[0047] Figure 5 Schematic diagram of a global-local self-attention mechanism module provided by an embodiment of the present invention;

[0048] Figure 6 is an overall network structure diagram provided by an embodiment of the present invention;

[0049] Figure 7 This is the classification result of the ISPRS-3D data set provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] It is understandable that based on the defects in the background technology, the embodiment of the present invention proposes an airborne point cloud classification method taking into account the global-local self-attention mechanism, and the method includes the following steps:

[0052] Step 1: Select the ISPRS-3D airborne point cloud dataset;

[0053] In this embodiment, the ISPRS-3D airborne point cloud dataset is selected, which consists of airborne laser scanning data collected by the Leica ALS 50 system from the Vaihingen area of ​​Germany. The airborne laser scanning data is classified into nine semantic classes: power lines (p_l), low vegetation (l_v), impermeable layer (i_s), vehicles (car), fences (f_h), roofs (r_f), exterior walls (f_e), shrubs (s_b), and trees (t_e). The ISPRS 3D annotation website provides scenes from three different areas: one scene with 753,876 points is used as a training set, and the other two scenes with 411,722 points are used as test sets. Each point cloud data provides spatial XYZ coordinates, intensity, number of returns, number of returns, and semantic labels. As shown in the figure, the ISPRS 3D annotation website provides scenes from three different areas: one scene with 753,876 points is used as a training set, and the other two scenes with 411,722 points are used as test sets. Each point cloud data provides spatial XYZ coordinates, intensity, number of returns, number of returns, and semantic labels. Figure 1 Three scenes from the dataset are shown, with reference to the rendered colors for each category.

[0054] Step 2: Preprocess the point cloud data in the airborne point cloud dataset;

[0055] The original 3D coordinates and optional features of the point cloud (intensity values ​​obtained from airborne LiDAR data are used in this paper) are input into the network. However, due to limited GPU memory, the entire training point set cannot be directly input into the network. In order to solve the above problems, the entire scene is divided into a set of 30m*30m*40m rectangular areas. In the model training stage, in order to improve the robustness of the model and prevent overfitting, several rectangular areas are first randomly selected, 8,192 points are randomly selected from each rectangular area, and 12.5% ​​of the points are randomly discarded. In the model testing stage, all points from the segmented rectangular areas in the test set are fed into the training model to achieve point-to-point classification. As Figure 2 The figure shows the preprocessing diagram of the ISPRS-3D dataset. Taking the ISPRS-3D dataset as an example, due to the different number of points in each rectangular area, the small broken areas that appear at the edge of the scene are merged into the surrounding larger areas to ensure the integrity of each rectangular area.

[0056] Step 3: Build an edge convolutional network based on point cloud data to extract local features of the point cloud;

[0057] In point cloud classification tasks, a common challenge is the lack of an effective description of the topological relationship between points. This means that important local structural information in the point cloud may be lost, which is crucial for classifying objects, scenes, or performing other tasks. In order to solve this problem, an embodiment of the present invention introduces an edge convolution module (EdgeConv). Edge convolution can effectively capture the connection and relative position relationship between points to enhance the local feature expression ability of point cloud data. Through edge convolution, not only the characteristics of each point itself can be taken into account, but also the information of the surrounding neighboring points can be fully utilized, thereby improving the classification effect of subsequent tasks.

[0058] The edge convolution operation includes edge feature extraction, depth feature extraction, and local feature extraction. The specific steps are as follows:

[0059] 1) Edge feature extraction. Figure 3 : is a flow chart of edge convolution local feature extraction provided by an embodiment of the present invention; first, for each input point P i k neighboring points P around (i=0,1,…N) j (j=0, 1, ...k), forming a KNN graph. i ) and its surrounding points P j The feature differences are used to capture local structural information. These feature differences are combined into the features of the center point to form the edge feature e ij The obtained edge features contain the local information of each point. The process is as follows Figure 3 (a) shows the edge feature e ij The expression is:

[0060] e ij =[f(P i ), f(P j -P i )] (1)

[0061] Where: f(P i ) is the feature of the center point; f(P j -P i ) is the neighborhood node feature minus the center point feature.

[0062] 2) Deep feature extraction. Using MLP to extract deep features from edge features aims to further enrich and extract deep features, thereby enhancing the expressiveness of features. The process is as follows: Figure 3 (b) shows the extracted deep edge feature e′ ij The expression is:

[0063] e′ ij =h[f(P i ), f(P j -Pi )] (2)

[0064] Where: h is a multi-layer perceptron.

[0065] 3) Local feature extraction. Finally, through the maximum pooling operation (Maxpooling), the most significant features are extracted from the deep edge features of each point to form the local features of the point cloud. These local features have deeper information, such as Figure 3 (c) shows the local feature P′ i The expression is:

[0066] P′ i =Maxpooling(i, j∈E)h[f(x i ), f(P j -P i )] (3)

[0067] Through the above operations, the local features of the point cloud can be effectively extracted, thus playing an important role in the subsequent point cloud classification tasks.

[0068] Step 4: Construct a local self-attention mechanism network based on the local features of the extracted point cloud;

[0069] Point cloud data is essentially a set of points P={P i}(i=0,1,…N). The attention mechanism can adapt to the irregularity and flexibility of point clouds when processing point cloud data, effectively capture the relationship between points, and thus better handle tasks such as point cloud classification and segmentation. Therefore, the local self-attention module in this paper adopts a radius-based ball query method, that is, applying a self-attention network in the local neighborhood around each data point.

[0070] The local self-attention layer is designed as Figure 4 As shown, for any point P in space i ∈P takes K points within a given radius (rank the first K points in ascending order) to form a point cloud subset P j (j=0,1,…k), its local neighborhood ensures a fixed regional scale, making the local regional features more generalizable in space. i The output feature can be expressed as y i , which is calculated as follows:

[0071]

[0072] in, ψ and α are point-wise feature transformations (linear projections), ρ is the normalization function Softmax; γ is the MLP mapping function; the position encoding function θ is an MLP with two linear layers and one ReLU nonlinearity.

[0073] Step 5: Build a global self-attention mechanism network;

[0074] The local self-attention network only focuses on the k nearest neighboring points and lacks the global contextual relationship between points. Therefore, this paper adds global self-attention on the basis of the local self-attention network. Global self-attention is implemented in a symmetrical way in this paper. In the self-attention mechanism, the query Q, key K, and value V are all generated from the same input sequence. After calculating the attention weight, it is weighted and applied to the value, ensuring the symmetry of the attention calculation. This implementation enables each element to be adjusted and updated based on the information of the entire input sequence, thereby effectively capturing long-distance dependencies.

[0075] For any point P i ∈P, the key to its global self-attention is to select some global marker points P g ∈P, so that information can flow better between points that are not paid attention to by local self-attention. Global self-attention supplements the global context information between all points on the basis of local self-attention. Globally marked point P g The selection rules are as follows:

[0076] First, calculate any point P in the space in turn i To all other points P r ∈P(r≠i, r=0,1,…N) to obtain a distance matrix with dimension N*N, which can be expressed as:

[0077]

[0078] Secondly, sort the elements of each row in the distance matrix D(i, r) from small to large, and return the sorted index D′. Select the index of the first g elements of each row of index D′ to obtain the index of the g neighboring points closest to each point, thereby selecting g global marker points P g The selected global marker point P g Sorting based on the relative distance of the entire point cloud, rather than being limited to a local neighborhood, facilitates the flow of information among all points in the network, thereby effectively extracting global contextual information between all points. Finally, using the subtraction relationship and adding the position encoding θ to the attention vector γ and the transformed feature α, the output feature can be expressed as:

[0079]

[0080] Step 6: Construct a global-local self-attention mechanism module;

[0081] The design of the Global-Local Self-Attention Block (GLAT) combines the advantages of two self-attention mechanisms. The global self-attention mechanism provides the model with rich global context information by capturing long-distance dependencies in the point cloud, allowing the model to recognize and understand the macro structure of the entire scene. At the same time, the local self-attention module focuses on capturing the local structural features of the point cloud. It focuses on the details and neighboring areas around each point, so that the model can understand the local geometry and features in detail. Through residual connections, the GLAT module effectively combines the outputs of the global and local self-attention mechanisms. This combination not only retains the advantages of the two mechanisms, but also enhances the model's ability to express features of different scales, thereby effectively realizing airborne point cloud classification tasks.

[0082] Figure 5 : is a schematic diagram of a global-local self-attention mechanism module provided by an embodiment of the present invention. As shown in the figure, the construction of the global-local self-attention mechanism module includes the following sub-steps:

[0083] Step 6.1: The input point cloud feature information is transformed through a linear layer and a global self-attention layer to capture the global context information of the entire point cloud, and the two information are superimposed to obtain features containing global information;

[0084] In step 6.2, the features containing global information are sent to the local self-attention layer to obtain further enriched local feature information;

[0085] Step 6.3, the obtained local feature information is mapped to a higher dimension through a linear layer and processed by a global self-attention layer to integrate the global context information again;

[0086] In step 6.4, the original input features are added to the transformed features to obtain the final output features. This fusion not only retains the advantages of the two mechanisms, but also enhances the model's ability to express features of different scales, thereby effectively achieving airborne point cloud classification tasks.

[0087] Step 7: Use U-net network design to complete the overall network structure;

[0088] In this embodiment, the overall network structure adopts U-net network design. Figure 6 The figure shows the overall network structure diagram provided by an embodiment of the present invention. The overall network structure is completed by using the U-net network design, which includes the following sub-steps:

[0089] In the traditional downsampling stage, the resolution of the point cloud data is reduced, resulting in information loss. Therefore, the present invention considers adding an edge convolution module to capture local features during downsampling. In addition, the global and local features of the point cloud are effectively extracted through the global-local self-attention mechanism module. In each upsampling stage, the point set size is gradually increased to 128, 512, 2048, and 8192 points. In order to merge low-level information in the downsampling stage, this paper adds jump connections between the downsampling and upsampling stages to connect the low-level features from the downsampling stage with the features of the same resolution size in the upsampling stage. The point features of the last upsampling layer are input into a fully connected layer to output the semantic labels of all input points.

[0090] Step 8: Input the airborne laser point cloud test data to be classified into the trained overall network structure to obtain the category label of each point.

[0091] In this embodiment, the network model is trained based on the ISPRS-3D airborne point cloud training sample set to obtain training results. Figure 7 As shown, ISPRS-3D shows the classification results of the data set, the real label (left) and the classification results of the method described in this embodiment (right). Finally, the ISPRS-3D airborne point cloud test sample set to be classified is input, and the point cloud of the test data is automatically classified and the accuracy is evaluated.

[0092] Finally, the example of the automatic classification result of large-scene airborne laser 3D point cloud by the embodiment of the present invention is as follows: Figure 7 As shown in the figure, it can be seen that the airborne point cloud classification method taking into account the global-local self-attention mechanism described in the embodiment of the present invention can stably and accurately automatically classify airborne laser point clouds with classification targets of different types, quantities and sizes.

[0093] According to a second aspect of the present invention, an embodiment of the present invention provides an airborne point cloud classification system taking into account a global-local self-attention mechanism, comprising:

[0094] Data acquisition module, used to select ISPRS-3D airborne point cloud data set;

[0095] A data preprocessing module, used for preprocessing the point cloud data in the airborne point cloud data set;

[0096] The network model construction module is used to construct an edge convolution network based on point cloud data to extract local features of the point cloud; construct a local self-attention mechanism network based on the extracted local features of the point cloud, construct a global self-attention mechanism network, and construct a global-local self-attention mechanism module;

[0097] The point cloud data classification module is used to complete the overall network structure using the U-net network design, input the airborne laser point cloud test data to be classified into the trained overall network structure, and obtain the category label of each point.

[0098] It can be understood that the airborne point cloud classification system taking into account the global-local self-attention mechanism provided in the embodiment of the present invention corresponds to the airborne point cloud classification method taking into account the global-local self-attention mechanism provided in the aforementioned embodiments. The relevant technical features of the airborne point cloud classification system taking into account the global-local self-attention mechanism can refer to the relevant technical features of the airborne point cloud classification method taking into account the global-local self-attention mechanism, which will not be repeated here.

[0099] In summary, in order to solve the problems of insufficient feature expression and unbalanced sample categories in large-scale classification of airborne point clouds, the present invention creatively proposes a global-local self-attention mechanism neural network integrating edge convolution. It has the following technical effects:

[0100] 1) The network design is based on the U-net framework, and an edge convolution layer is designed. The relative distance between points is used to construct a spatial relationship graph, which effectively captures the neighboring features in the point cloud data, helps to extract and express the boundary and texture information of complex objects, and enhances the classification accuracy and stability of the model;

[0101] 2) The global-local self-attention mechanism module is introduced to take into account the global context information while paying attention to the characteristics of a few categories, thus improving the problem of category imbalance and further improving the accuracy of classification.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0103] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An airborne point cloud classification method taking into account the global-local self-attention mechanism, characterized in that: The following steps are involved: Step 1: Select the ISPRS-3D airborne point cloud dataset; Step 2: Preprocess the point cloud data in the airborne point cloud dataset; Step 3: Build an edge convolutional network based on point cloud data to extract local features of the point cloud; Step 4: Construct a local self-attention mechanism network based on the local features of the extracted point cloud; Step 5: Build a global self-attention mechanism network; Step 6: Construct a global-local self-attention mechanism module; Step 7: Use U-net network design to complete the overall network structure; Step 8: Input the airborne laser point cloud test data to be classified into the trained overall network structure to obtain the category label of each point.

2. The method for airborne point cloud classification taking into account the global-local self-attention mechanism according to claim 1, characterized in that: Each point cloud data in the ISPRS-3D airborne point cloud dataset provides spatial XYZ coordinates, intensity, return number and semantic label.

3. The method for airborne point cloud classification taking into account the global-local self-attention mechanism according to claim 1, characterized in that: The preprocessing of the point cloud data in the airborne point cloud data set includes: The intensity values ​​obtained from the airborne point cloud data are input into the network to divide the entire scene into rectangular areas of preset sizes. During the model training phase, several rectangular areas are randomly selected, several points are randomly selected from each rectangular area, and a certain proportion of points are randomly discarded. During the model testing phase, all points from the segmented rectangular areas in the test set are fed into the training model to complete point-to-point classification. The small fragmented areas that appear at the edge of the scene are merged into the surrounding larger areas to ensure the integrity of each rectangular area.

4. The method for airborne point cloud classification taking into account the global-local self-attention mechanism according to claim 1, characterized in that: The construction of an edge convolutional network based on point cloud data includes edge feature extraction, depth feature extraction and local feature extraction.

5. The method for airborne point cloud classification taking into account the global-local self-attention mechanism according to claim 1, characterized in that: The local self-attention mechanism network constructed based on the local features of the extracted point cloud adopts a radius-based ball query method and applies the self-attention network in the local neighborhood around each data point.

6. The method for airborne point cloud classification taking into account the global-local self-attention mechanism according to claim 1, characterized in that: The construction of the global self-attention mechanism network includes the following steps: For any point P i ∈P, select the global marker point P g ∈P; calculate any point P in the space in turn i To all other points P r ∈P(r≠i, r=0,1,…N), and obtain a distance matrix with dimension N*N; Sort each row of the distance matrix D(i,r) from small to large, and return the sorted index D′; Select the index of the first g elements of each row of index D′ to get the index of the g neighbor points closest to each point, and select g global marker points P g ; The position encoding θ is added to the attention vector γ and the transformed feature α using a subtraction relationship to obtain the output feature.

7. The method for airborne point cloud classification taking into account the global-local self-attention mechanism according to claim 1, characterized in that: The module for constructing the global-local self-attention mechanism includes: The input point cloud feature information is transformed through a linear layer and a global self-attention layer to capture the global context information of the entire point cloud, and the two information are superimposed to obtain features containing global information; The features containing global information are sent to the local self-attention layer to obtain further enriched local feature information; The obtained local feature information is mapped to a higher dimension through a linear layer and processed by a global self-attention layer to integrate the global context information again; The original input features are added to the transformed features to get the final output features.

8. The method for airborne point cloud classification taking into account the global-local self-attention mechanism according to claim 1, characterized in that: The overall network structure completed by using U-net network design includes: During downsampling, an edge convolution module is added to capture local features. The global and local features of the point cloud are extracted through the global-local self-attention mechanism module. Skip connections are added between the downsampling and upsampling stages to connect the low-level features from the downsampling stage with the features of the same resolution size in the upsampling stage. The point features of the upsampling layer are input into the fully connected layer, and the semantic labels of all input points are output.

9. The method for airborne point cloud classification taking into account the global-local self-attention mechanism according to claim 1, characterized in that: The airborne laser point cloud test data to be classified is input into the trained overall network structure to obtain the category label of each point, including: Based on the ISPRS-3D airborne point cloud training sample set, the network model is trained to obtain the training results. The ISPRS-3D airborne point cloud test sample set to be classified is input to automatically classify the test data and evaluate the accuracy.

10. An airborne point cloud classification system taking into account a global-local self-attention mechanism, characterized in that: include: Data acquisition module, used to select ISPRS-3D airborne point cloud data set; A data preprocessing module, used for preprocessing point cloud data in the airborne point cloud data set; A network model building module is used to build an edge convolutional network based on point cloud data, extract local features of the point cloud, build a local self-attention mechanism network based on the extracted local features of the point cloud, build a global self-attention mechanism network, and build a global-local self-attention mechanism module; The point cloud data classification module uses the U-net network design to complete the overall network structure. The airborne laser point cloud test data to be classified is input into the trained overall network structure to obtain the category label of each point.

Images