Self-attention landslide segmentation method based on point cloud neighborhood structure perception

Abstract

The invention discloses a self-attention landslide segmentation method based on point cloud neighborhood structure perception, and relates to the technical field of landslide detection. Acquiring laser point cloud data of the landslide area; using the point cloud data of each point in the laser point cloud data as the input of a pre-trained landslide segmentation model for operation to obtain a landslide classification identification result of each point in the laser point cloud data; the landslide segmentation model comprises a plurality of neighborhood perception and feature coding layers, a plurality of neighborhood perception and feature decoding layers and an output layer. The plurality of neighborhood sensing and feature coding layers are used for performing sampling and feature extraction of different scales on the point cloud data, determining neighborhood sensing coefficients of neighborhood sampling points and updating sampling point features; and the plurality of neighborhood sensing and feature decoding layers are used for decoding and splicing the updated features so as to serve as the input of the output layer to obtain a landslide classification and recognition result. The self-attention landslide segmentation method based on point cloud neighborhood structure perception disclosed by the invention can improve the accuracy of landslide detection.

Description

Technical Field

[0001] The invention belongs to the technical field of landslide detection, and in particular relates to a point cloud neighborhood structure-aware self-attention landslide segmentation method. Background Art

[0002] Landslide is a common natural disaster that poses a serious threat to human life and property. Effective landslide detection is of great significance for post-disaster rescue, post-disaster reconstruction and vegetation restoration planning.

[0003] At present, landslide area detection is often carried out through landslide segmentation based on point cloud data. However, due to the diverse morphology of landslides in natural scenes and the complex perimeter of landslides, landslide segmentation based on point cloud data often lacks the ability to extract features of the overall contour and local details of the landslide, and lacks the utilization of the landslide boundary structure. As a result, landslide segmentation based on point cloud data is difficult to achieve the expected effect, and the accuracy of landslide detection is poor.

[0004] Therefore, how to provide an effective solution to improve the accuracy of landslide detection has become a difficult problem to be solved urgently in the prior art. Summary of the invention

[0005] The purpose of the present invention is to provide a point cloud neighborhood structure-aware self-attention landslide segmentation method to solve the above-mentioned problems existing in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a point cloud neighborhood structure-aware self-attention landslide segmentation method, comprising:

[0008] Obtain laser point cloud data of the landslide area;

[0009] The point cloud data of each point in the laser point cloud data is used as the input of the pre-trained landslide segmentation model to perform calculations to obtain the landslide classification and recognition results of each point in the laser point cloud data;

[0010] The landslide segmentation model includes an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers, and an output layer;

[0011] The multiple neighborhood perception and feature encoding layers are used to perform sampling and feature extraction of input point cloud data at different scales;

[0012] Calculate the slope corresponding to each sampling point in point cloud data of different scales and the slope variation between each sampling point and its neighboring sampling points;

[0013] Extract features of each sampling point in point cloud data of different scales, and obtain features of each sampling point and its neighboring sampling points in point cloud data of different scales;

[0014] Based on the features corresponding to each sampling point in the point cloud data of different scales and the features of the neighborhood sampling points of each sampling point in the point cloud data of different scales, the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales is determined, and the neighborhood perception coefficient is used to characterize the influence of the neighborhood sampling point on the corresponding sampling point;

[0015] Based on the neighborhood perception coefficients of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales, the features corresponding to each sampling point in the point cloud data of different scales are updated to obtain the updated features corresponding to each sampling point in the point cloud data of different scales;

[0016] The multiple neighborhood perception and feature decoding layers are used to decode and splice the updated features corresponding to each sampling point in the point cloud data of different scales to obtain splicing features, so that the splicing features can be used as input of the output layer for calculation to obtain the landslide classification and recognition results of each point in the laser point cloud data.

[0017] Based on the above disclosed content, the present invention obtains laser point cloud data of a landslide area; uses the point cloud data of each point in the laser point cloud data as the input of a pre-trained landslide segmentation model for calculation, and obtains landslide classification and recognition results of each point in the laser point cloud data; wherein the landslide segmentation model includes an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers, and an output layer; the multiple neighborhood perception and feature encoding layers are used to perform sampling and feature extraction of the input point cloud data at different scales; calculates the slope corresponding to each sampling point in the point cloud data at different scales and the slope variation between each sampling point and its neighboring sampling points; extracts features of each sampling point in the point cloud data at different scales, and extracts features of each sampling point and its neighboring sampling points in the point cloud data at different scales; based on the point cloud data at different scales, the landslide segmentation model comprises an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers, and an output layer; the multiple neighborhood perception and feature encoding layers are used to perform sampling and feature extraction of the input point cloud data at different scales; calculates the slope corresponding to each sampling point in the point cloud data at different scales and the slope variation between each sampling point and its neighboring sampling points; extracts features of each sampling point in the point cloud data at different scales, and obtain ... According to the features corresponding to each sampling point in the point cloud data of different scales and the features of the neighborhood sampling points of each sampling point in the point cloud data of different scales, the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales is determined, and the neighborhood perception coefficient is used to characterize the influence of the neighborhood sampling points on the corresponding sampling points; based on the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales, the features corresponding to each sampling point in the point cloud data of different scales are updated to obtain the updated features corresponding to each sampling point in the point cloud data of different scales; multiple neighborhood perception and feature decoding layers are used to decode and splice the updated features corresponding to each sampling point in the point cloud data of different scales to obtain the splicing features, so that the splicing features are used as the input of the output layer for calculation, and the landslide classification and recognition results of each point in the laser point cloud data are obtained. In this way, when detecting landslides, by extracting the features of the sampling points and their neighborhood sampling points, the neighborhood perception coefficient that characterizes the degree of influence of the neighborhood sampling points on the corresponding sampling points is determined and the features of the sampling points are updated, thereby taking into account the influence of the neighborhood structure of the landslide boundary point cloud on the sliding segmentation. At the same time, through sampling and feature extraction at different scales, the ability to extract the overall contour and local details of the landslide can be improved. Therefore, when segmenting the landslide, both the influence of the neighborhood structure of the landslide boundary point cloud on the landslide segmentation and the overall contour and local detail characteristics of the landslide can be taken into account, thereby improving the accuracy of landslide detection.

[0018] In a possible design, the calculation of the slope corresponding to each sampling point in the point cloud data of different scales and the slope variation between each sampling point and its neighboring sampling points includes:

[0019] The slope corresponding to each sampling point in point cloud data of different scales is calculated by fitting the plane using principal component analysis method;

[0020] Based on the slopes corresponding to each sampling point in point cloud data of different scales and the slopes corresponding to the neighborhood sampling points of each sampling point in point cloud data of different scales, the slope variation between each sampling point and its neighborhood sampling points in point cloud data of different scales is calculated.

[0021] In a possible design, the slope corresponding to the sampling point is calculated according to the following formula (1);

[0022]

[0023] Among them, n x 、n y and n z They respectively represent the components of the minimum eigenvector of the point cloud data in the three coordinate directions obtained by the principal component analysis method;

[0024] The slope variation between the sampling point and its neighboring sampling points is calculated according to the following formula (2);

[0025]

[0026] Among them, θ i represents the slope corresponding to the i-th sampling point, θ ij Indicates the slope corresponding to the jth neighborhood sampling point of the i-th sampling point.

[0027] In a possible design, the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales is determined according to the following formula (3);

[0028]

[0029] Among them, α and β represent linear transformations, f i represents the feature corresponding to the i-th sampling point, f ij represents the characteristics of the jth neighboring sampling point of the i-th sampling point, represents a multilayer perceptron, λ represents the position encoding of the i-th sampling point and the j-th neighboring sampling point of the i-th sampling point in three-dimensional space, d k represents the key vector dimension of the neighborhood-aware self-attention module in the neighborhood-aware and feature encoding layer, ξ represents the neighborhood-aware encoding, and Linear represents linear transformation, σ ReLU Represents the ReLU activation function.

[0030] In a possible design, the features corresponding to the sampling points are updated according to the following formula (4):

[0031]

[0032] Among them, ⊙ represents the Hadamard product, p represents the neighborhood sampling point corresponding to the sampling point, represents the set of neighborhood sampling points corresponding to the sampling point, f ij Indicates the characteristics of the neighborhood sampling points corresponding to the sampling point.

[0033] In one possible design, the multiple neighborhood perception and feature coding layers include a first neighborhood perception and feature coding layer, a second neighborhood perception and feature coding layer, a third neighborhood perception and feature coding layer, a fourth neighborhood perception and feature coding layer, and a fifth neighborhood perception and feature coding layer; the multiple neighborhood perception and feature decoding layers include a first neighborhood perception and feature decoding layer, a second neighborhood perception and feature decoding layer, a third neighborhood perception and feature decoding layer, a fourth neighborhood perception and feature decoding layer, and a fifth neighborhood perception and feature decoding layer; the first neighborhood perception and feature coding layer includes a first neighborhood perception layer and a first coding layer; The second neighborhood perception and feature coding layer includes a second neighborhood perception layer and a second coding layer, the third neighborhood perception and feature coding layer includes a third neighborhood perception layer and a third coding layer, the fourth neighborhood perception and feature coding layer includes a fourth neighborhood perception layer and a fourth coding layer, the fifth neighborhood perception and feature coding layer includes a fifth neighborhood perception layer and a fifth coding layer, the first neighborhood perception and feature decoding layer includes a first decoding layer and the first neighborhood perception layer, the second neighborhood perception and feature decoding layer includes a second decoding layer and the second neighborhood perception layer, the third neighborhood perception and feature decoding layer includes a third decoding layer and the third neighborhood perception layer, the fourth neighborhood perception and feature decoding layer includes a fourth decoding layer and the fourth neighborhood perception layer, the fifth neighborhood perception and feature decoding layer includes a fifth decoding layer and the fifth neighborhood perception layer, the first neighborhood perception layer, the first coding layer, the second neighborhood perception layer, the second coding layer, the third neighborhood perception layer, the third coding layer, the fourth neighborhood perception layer, the fourth coding layer, the fifth neighborhood perception layer and the fifth coding layer are connected in sequence, the first decoding layer, the fifth neighborhood perception layer, the second decoding layer, the fourth neighborhood perception layer, the fifth coding layer and the fifth neighborhood perception layer are connected in sequence, The output end of the first coding layer is also connected to the input end of the fifth decoding layer, the output end of the second coding layer is also connected to the input end of the fourth decoding layer, the output end of the third coding layer is also connected to the input end of the third decoding layer, the output end of the fourth coding layer is also connected to the input end of the second decoding layer, and the output end of the fifth coding layer is connected to the input end of the first decoding layer through a multi-layer perception layer.

[0034] In a possible design, the first neighborhood perception and feature encoding layer, the second neighborhood perception and feature encoding layer, the third neighborhood perception and feature encoding layer, the fourth neighborhood perception and feature encoding layer and the fifth neighborhood perception and feature encoding layer sample the input point cloud data at different scales according to sampling numbers of N, N / 4, N / 16, N / 64 and N / 256, respectively, where N represents the total amount of data corresponding to the point cloud data, and the first neighborhood perception and feature decoding layer, the second neighborhood perception and feature decoding layer, the third neighborhood perception and feature decoding layer, the fourth neighborhood perception and feature decoding layer and the fifth neighborhood perception and feature decoding layer upscale the concatenated features according to multiples of 1, 2, 4, 8 and 16, respectively.

[0035] In a second aspect, the present invention provides a point cloud neighborhood structure-aware self-attention landslide segmentation device, comprising:

[0036] An acquisition unit, used for acquiring laser point cloud data of the landslide area;

[0037] A computing unit, used for computing the point cloud data of each point in the laser point cloud data as an input of a pre-trained landslide segmentation model to obtain a landslide classification and recognition result of each point in the laser point cloud data;

[0038] The landslide segmentation model includes an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers, and an output layer;

[0039] The multiple neighborhood perception and feature encoding layers are used to perform sampling and feature extraction of input point cloud data at different scales;

[0040] Calculate the slope corresponding to each sampling point in point cloud data of different scales and the slope variation between each sampling point and its neighboring sampling points;

[0041] Extract features of each sampling point in point cloud data of different scales, and obtain features of each sampling point and its neighboring sampling points in point cloud data of different scales;

[0042] Based on the features corresponding to each sampling point in the point cloud data of different scales and the features of the neighborhood sampling points of each sampling point in the point cloud data of different scales, the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales is determined, and the neighborhood perception coefficient is used to characterize the influence of the neighborhood sampling point on the corresponding sampling point;

[0043] Based on the neighborhood perception coefficients of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales, the features corresponding to each sampling point in the point cloud data of different scales are updated to obtain the updated features corresponding to each sampling point in the point cloud data of different scales;

[0044] The multiple neighborhood perception and feature decoding layers are used to decode and splice the updated features corresponding to each sampling point in the point cloud data of different scales to obtain splicing features, so that the splicing features can be used as input of the output layer for calculation to obtain the landslide classification and recognition results of each point in the laser point cloud data.

[0045] In a third aspect, the present invention provides an electronic device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the point cloud neighborhood structure-aware self-attention landslide segmentation method as described in the first aspect or any possible design of the first aspect.

[0046] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, executes the point cloud neighborhood structure-aware self-attention landslide segmentation method described in the first aspect or any possible design of the first aspect.

[0047] In a fifth aspect, the present invention provides a computer program product comprising instructions, which, when executed on a computer, cause the computer to execute the point cloud neighborhood structure-aware self-attention landslide segmentation method as described in the first aspect or any possible design of the first aspect.

[0048] Beneficial effects:

[0049] The point cloud neighborhood structure-aware self-attention landslide segmentation method provided by the present invention, when detecting landslides, extracts the features of sampling points and their neighborhood sampling points, determines the neighborhood perception coefficient that characterizes the degree of influence of the neighborhood sampling points on the corresponding sampling points, and updates the features of the sampling points, thereby taking into account the influence of the neighborhood structure of the landslide boundary point cloud on the sliding segmentation. At the same time, through sampling and feature extraction at different scales, the ability to extract the overall contour and local details of the landslide can be improved. Therefore, when segmenting the landslide, both the influence of the neighborhood structure of the landslide boundary point cloud on the landslide segmentation and the overall contour and local detail features of the landslide can be taken into account, thereby improving the accuracy of landslide detection and facilitating practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of the point cloud neighborhood structure-aware self-attention landslide segmentation method provided in an embodiment of the present application;

[0051] Figure 2 A network structure diagram of a landslide segmentation model provided in an embodiment of the present application;

[0052] Figure 3A schematic block diagram of a point cloud neighborhood structure-aware self-attention landslide segmentation device provided in an embodiment of the present application;

[0053] Figure 4 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0055] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0056] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0057] In order to realize the detection of landslide areas, an embodiment of the present application provides a point cloud neighborhood structure-aware self-attention landslide segmentation method, which can improve the accuracy of landslide detection.

[0058] The point cloud neighborhood structure-aware self-attention landslide segmentation method provided in the embodiment of the present application can be applied to a user terminal or a server. It is understood that the execution subject does not constitute a limitation on the embodiment of the present application.

[0059] The point cloud neighborhood structure-aware self-attention landslide segmentation method provided in an embodiment of the present application will be described in detail below.

[0060] like Figure 1As shown, it is a flowchart of the point cloud neighborhood structure-aware self-attention landslide segmentation method provided in the first aspect of the embodiment of the present application. The point cloud neighborhood structure-aware self-attention landslide segmentation method may include but is not limited to the following steps S101-S102.

[0061] Step S101: Acquire laser point cloud data of the landslide area.

[0062] The laser point cloud data includes point cloud data of multiple points, and the point cloud data includes three-dimensional space coordinates and may also include color attributes, such as the values ​​of R, G, and B color channels.

[0063] Step S102: using the point cloud data of each point in the laser point cloud data as the input of the pre-trained landslide segmentation model to perform calculations, and obtaining the landslide classification and recognition results of each point in the laser point cloud data.

[0064] In an embodiment of the present application, a landslide segmentation model for landslide detection is pre-trained. After the laser point cloud data of the landslide area is obtained, the point cloud data of each point in the laser point cloud data can be used as the input of the pre-trained landslide segmentation model for calculation to obtain the landslide classification and recognition result of each point in the laser point cloud data. The landslide classification and recognition result can be the probability of each point in the laser point cloud data being a landslide point and a non-landslide point.

[0065] The landslide segmentation model includes an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers, and an output layer, etc. The input layer is connected to the multiple neighborhood perception and feature encoding layers, and the multiple neighborhood perception and feature decoding layers are connected to the output layer.

[0066] Multiple neighborhood perception and feature encoding layers are used to sample and extract features at different scales of the input point cloud data.

[0067] In order to ensure that the point cloud samples more points in the area with larger curvature and control the uniformity of the spatial distribution of the sampling points, the present application can design a curvature-based sampling method. First, for a point cloud with a number of N, the principal component analysis method can be used to fit a plane to each point and calculate the average curvature, and then the point cloud is sorted from large to small according to the curvature. When M points need to be sampled, a threshold η (ranging from 0 to 1) can be set, and the sampling interval is the first (1-η)N+ηM points in the sorted point cloud, and then the farthest point sampling algorithm is used to sample M points in the sampling interval.

[0068] Multiple neighborhood perception and feature encoding layers are also used to calculate the slope corresponding to each sampling point in point cloud data of different scales and the slope variation between each sampling point and its neighboring sampling points.

[0069] In an embodiment of the present application, the principal component analysis (PCA) method can be used to fit the plane to calculate the slope corresponding to each sampling point in the point cloud data of different scales, and then based on the slope corresponding to each sampling point in the point cloud data of different scales and the slope corresponding to the neighborhood sampling points of each sampling point in the point cloud data of different scales, the slope variation between each sampling point in the point cloud data of different scales and its neighborhood sampling points is calculated.

[0070] When calculating the slope corresponding to each sampling point, the slope corresponding to the sampling point can be calculated according to the following formula (1);

[0071]

[0072] Among them, n x 、n y and n z They respectively represent the components of the minimum eigenvector of the point cloud data obtained by the principal component analysis method in the three coordinate directions.

[0073] When calculating the slope variation between a sampling point and its neighboring sampling points, the slope variation between the sampling point and its neighboring sampling points can be calculated according to the following formula (2):

[0074]

[0075] Among them, θ i represents the slope corresponding to the i-th sampling point, θ ij represents the slope corresponding to the jth neighborhood sampling point of the ith sampling point, It represents the slope variation between the i-th sample and its j-th neighboring sample point.

[0076] Multiple neighborhood perception and feature encoding layers are also used to extract features of each sampling point in point cloud data of different scales, thereby extracting features of each sampling point and its neighborhood sampling points in point cloud data of different scales.

[0077] In the embodiment of the present application, for given point cloud data, the characteristics of each sampling point and its neighborhood sampling points in point cloud data of different scales can be obtained through linear transformation.

[0078] Multiple neighborhood perception and feature encoding layers are also used to determine the neighborhood perception coefficients of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales based on the features corresponding to each sampling point in the point cloud data of different scales and the features of the neighborhood sampling points of each sampling point in the point cloud data of different scales.

[0079] In the embodiment of the present application, the attention mechanism can be used to calculate the neighborhood perception coefficient of the neighboring sampling points corresponding to each sampling point in the point cloud data of different scales, and the neighborhood perception coefficient is used to characterize the influence of the neighboring sampling points on the corresponding sampling points. Specifically, the neighborhood perception coefficient of the neighboring sampling points corresponding to each sampling point in the point cloud data of different scales can be determined according to the following formula (3);

[0080]

[0081] Among them, α and β represent linear transformations, f i represents the feature corresponding to the i-th sampling point, f ij represents the characteristics of the jth neighboring sampling point of the i-th sampling point, represents a multilayer perceptron, λ represents the position encoding of the i-th sampling point and the j-th neighboring sampling point of the i-th sampling point in three-dimensional space, d k represents the key vector dimension of the neighborhood-aware self-attention module in the neighborhood-aware and feature encoding layer, ξ represents the neighborhood-aware encoding, and Linear represents linear transformation, σ ReLU Represents the ReLU activation function.

[0082] The multiple neighborhood perception and feature encoding layers are also used to update the features corresponding to each sampling point in the different scale point cloud data based on the neighborhood perception coefficients of the neighborhood sampling points corresponding to each sampling point in the different scale point cloud data, so as to obtain the updated features corresponding to each sampling point in the different scale point cloud data. In the embodiment of the present application, the neighborhood perception coefficients can be weightedly aggregated to update the features of the sampling points. Specifically, the features corresponding to the sampling points can be updated according to the following formula (4);

[0083]

[0084] Among them, ⊙ represents the Hadamard product, p represents the neighborhood sampling point corresponding to the sampling point, represents the set of neighborhood sampling points corresponding to the sampling point, f ij Indicates the characteristics of the neighborhood sampling points corresponding to the sampling point.

[0085] Multiple neighborhood perception and feature decoding layers are used to decode and splice the updated features corresponding to each sampling point in point cloud data of different scales to obtain spliced ​​features, so that the spliced ​​features can be used as input of the output layer for calculation to obtain the landslide classification and recognition results of each point in the laser point cloud data.

[0086] like Figure 2In the embodiment of the present application, the multiple neighborhood perception and feature coding layers include a first neighborhood perception and feature coding layer, a second neighborhood perception and feature coding layer, a third neighborhood perception and feature coding layer, a fourth neighborhood perception and feature coding layer, and a fifth neighborhood perception and feature coding layer. The first neighborhood perception and feature coding layer, the second neighborhood perception and feature coding layer, the third neighborhood perception and feature coding layer, the fourth neighborhood perception and feature coding layer, and the fifth neighborhood perception and feature coding layer can sample the input point cloud data at different scales according to the sampling numbers of N, N / 4, N / 16, N / 64 and N / 256, respectively, where N represents the total amount of data corresponding to the point cloud data. The multiple neighborhood perception and feature decoding layers include a first neighborhood perception and feature decoding layer, a second neighborhood perception and feature decoding layer, a third neighborhood perception and feature decoding layer, a fourth neighborhood perception and feature decoding layer, and a fifth neighborhood perception and feature decoding layer. The first neighborhood perception and feature decoding layer, the second neighborhood perception and feature decoding layer, the third neighborhood perception and feature decoding layer, the fourth neighborhood perception and feature decoding layer, and the fifth neighborhood perception and feature decoding layer can respectively upgrade the dimensions of the spliced ​​features according to multiples of 1, 2, 4, 8, and 16.

[0087] Among them, the first neighborhood perception and feature coding layer includes a first neighborhood perception layer and a first coding layer, the second neighborhood perception and feature coding layer includes a second neighborhood perception layer and a second coding layer, the third neighborhood perception and feature coding layer includes a third neighborhood perception layer and a third coding layer, the fourth neighborhood perception and feature coding layer includes a fourth neighborhood perception layer and a fourth coding layer, the fifth neighborhood perception and feature coding layer includes a fifth neighborhood perception layer and a fifth coding layer, the first neighborhood perception and feature decoding layer includes a first decoding layer and the first neighborhood perception layer, the second neighborhood perception and feature decoding layer includes a second decoding layer and the second neighborhood perception layer, the third neighborhood perception and feature decoding layer includes a third decoding layer and the third neighborhood perception layer, the fourth neighborhood perception and feature decoding layer includes a fourth decoding layer and the fourth neighborhood perception layer, the fifth neighborhood perception and feature decoding layer includes a fifth decoding layer and the fifth neighborhood perception layer, the first neighborhood perception The first coding layer, the second neighborhood perception layer, the second coding layer, the third neighborhood perception layer, the third coding layer, the fourth neighborhood perception layer, the fourth coding layer, the fifth neighborhood perception layer and the fifth coding layer are connected in sequence, the first decoding layer, the fifth neighborhood perception layer, the second decoding layer, the fourth neighborhood perception layer, the third decoding layer, the third neighborhood perception layer, the fourth decoding layer, the second neighborhood perception layer, the fifth decoding layer and the first neighborhood perception layer are connected in sequence, the output end of the first coding layer is also connected to the input end of the fifth decoding layer, the output end of the second coding layer is also connected to the input end of the fourth decoding layer, the output end of the third coding layer is also connected to the input end of the third decoding layer, the output end of the fourth coding layer is also connected to the input end of the second decoding layer, and the output end of the fifth coding layer is connected to the input end of the first decoding layer through a multi-layer perception layer.

[0088] In the encoding stage, by inputting the point cloud data of N points (including 3D coordinates and color attributes), the input feature dimension D = 6 can first be encoded through a layer of multi-layer perceptron (MLP) to abstract the initial 6-dimensional features to a high dimension C = 32. In the subsequent encoding stage, each layer will be embedded in the neighborhood-aware self-attention, and the point data will be sampled according to N→N / 4→N / 16→N / 64→N / 256. The sampled points in each layer are grouped using the k-nearest neighbor algorithm, and finally the feature dimension is upgraded according to C→2C→4C→8C→16C through a shared multi-layer perceptron.

[0089] In the decoding stage, the multi-scale features extracted from each encoding layer are fused and propagated to achieve point cloud classification. In the decoding stage, each layer will fuse low-level and high-level features through residual connections, and embed a neighborhood-aware self-attention module in each layer, and use the inverse distance interpolation method to upsample the point cloud. In the last step, a multi-layer perceptron is used to predict the point cloud label through the normalized exponential (Softmax) function, and its output dimension is 2, representing the probability of predicting a landslide point and a non-landslide point, respectively.

[0090] In summary, the point cloud neighborhood structure-aware self-attention landslide segmentation method provided by the present invention obtains laser point cloud data of the landslide area; uses the point cloud data of each point in the laser point cloud data as the input of a pre-trained landslide segmentation model for calculation, and obtains the landslide classification and recognition results of each point in the laser point cloud data; wherein the landslide segmentation model includes an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers and an output layer; the multiple neighborhood perception and feature encoding layers are used to perform sampling and feature extraction on the input point cloud data at different scales; calculates the slope corresponding to each sampling point in the point cloud data at different scales and the slope variation between each sampling point and its neighboring sampling points; extracts the features of each sampling point in the point cloud data at different scales, and extracts the features of each sampling point and its neighboring sampling points in the point cloud data at different scales; based on Based on the features corresponding to each sampling point in point cloud data of different scales and the features of the neighborhood sampling points of each sampling point in point cloud data of different scales, the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in point cloud data of different scales is determined, and the neighborhood perception coefficient is used to characterize the influence of the neighborhood sampling points on the corresponding sampling points; based on the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in point cloud data of different scales, the features corresponding to each sampling point in point cloud data of different scales are updated to obtain the updated features corresponding to each sampling point in point cloud data of different scales; multiple neighborhood perception and feature decoding layers are used to decode and splice the updated features corresponding to each sampling point in point cloud data of different scales to obtain splicing features, so that the splicing features are used as the input of the output layer for calculation, and the landslide classification and recognition results of each point in the laser point cloud data are obtained. In this way, when detecting landslides, by extracting the features of the sampling points and their neighborhood sampling points, the neighborhood perception coefficient that characterizes the degree of influence of the neighborhood sampling points on the corresponding sampling points is determined and the features of the sampling points are updated, thereby taking into account the influence of the neighborhood structure of the landslide boundary point cloud on the sliding segmentation. At the same time, through sampling and feature extraction at different scales, the ability to extract the overall contour and local details of the landslide can be improved. Therefore, when segmenting the landslide, both the influence of the neighborhood structure of the landslide boundary point cloud on the landslide segmentation and the overall contour and local detail characteristics of the landslide can be taken into account, thereby improving the accuracy of landslide detection and facilitating practical application and promotion.

[0091] See also Figure 3According to a second aspect of an embodiment of the present application, a point cloud neighborhood structure-aware self-attention landslide segmentation device is provided. The point cloud neighborhood structure-aware self-attention landslide segmentation device comprises:

[0092] An acquisition unit, used for acquiring laser point cloud data of the landslide area;

[0093] A computing unit, used for computing the point cloud data of each point in the laser point cloud data as an input of a pre-trained landslide segmentation model to obtain a landslide classification and recognition result of each point in the laser point cloud data;

[0094] The landslide segmentation model includes an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers, and an output layer;

[0095] The multiple neighborhood perception and feature encoding layers are used to perform sampling and feature extraction of input point cloud data at different scales;

[0096] Calculate the slope corresponding to each sampling point in point cloud data of different scales and the slope variation between each sampling point and its neighboring sampling points;

[0097] Extract features of each sampling point in point cloud data of different scales, and obtain features of each sampling point and its neighboring sampling points in point cloud data of different scales;

[0098] Based on the features corresponding to each sampling point in the point cloud data of different scales and the features of the neighborhood sampling points of each sampling point in the point cloud data of different scales, the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales is determined, and the neighborhood perception coefficient is used to characterize the influence of the neighborhood sampling point on the corresponding sampling point;

[0099] Based on the neighborhood perception coefficients of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales, the features corresponding to each sampling point in the point cloud data of different scales are updated to obtain the updated features corresponding to each sampling point in the point cloud data of different scales;

[0100] The multiple neighborhood perception and feature decoding layers are used to decode and splice the updated features corresponding to each sampling point in the point cloud data of different scales to obtain splicing features, so that the splicing features can be used as input of the output layer for calculation to obtain the landslide classification and recognition results of each point in the laser point cloud data.

[0101] The working process, working details and technical effects of the point cloud neighborhood structure-aware self-attention landslide segmentation device provided in the second aspect of this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0102] like Figure 4As shown, the third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the point cloud neighborhood structure-aware self-attention landslide segmentation method as described in the first aspect of the embodiment.

[0103] For specific examples, the memory may include but is not limited to random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out memory (FIFO) and / or first-in-last-out memory (FILO), etc.; the processor may be but is not limited to a microprocessor of the STM32F105 series, an ARM (Advanced RISC-Machines), an X86 or other architecture processor, or a processor with an integrated NPU (neural-network processing units); the transceiver may be but is not limited to a WiFi (Wireless Fidelity) wireless transceiver, a Bluetooth wireless transceiver, a General Packet Radio Service (GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc.

[0104] In a fourth aspect of the present embodiment, there is provided a computer-readable storage medium storing instructions for the point cloud neighborhood structure-aware self-attention landslide segmentation method described in the first aspect of the embodiment, that is, the computer-readable storage medium stores instructions, and when the instructions are run on a computer, the point cloud neighborhood structure-aware self-attention landslide segmentation method described in the first aspect is executed. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive, and / or a memory stick, etc., and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0105] A fifth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the point cloud neighborhood structure-aware self-attention landslide segmentation method as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0106] It should be understood that certain details are provided in the following description to facilitate a complete understanding of the example embodiments. However, it should be understood by those of ordinary skill in the art that the example embodiments can be implemented without these certain details. For example, the system can be shown in a block diagram to avoid obscuring the example with unnecessary details. In other examples, well-known processes, structures, and techniques may not be shown in unnecessary detail to avoid obscuring the example embodiments.

[0107] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A point cloud neighborhood structure-aware self-attention landslide segmentation method, characterized in that: include: Obtain laser point cloud data of the landslide area; The point cloud data of each point in the laser point cloud data is used as the input of the pre-trained landslide segmentation model to perform calculations to obtain the landslide classification and recognition results of each point in the laser point cloud data; The landslide segmentation model includes an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers, and an output layer; The multiple neighborhood perception and feature encoding layers are used to perform sampling and feature extraction of input point cloud data at different scales; Calculate the slope corresponding to each sampling point in point cloud data of different scales and the slope variation between each sampling point and its neighboring sampling points; Extract features of each sampling point in point cloud data of different scales, and obtain features of each sampling point and its neighboring sampling points in point cloud data of different scales; Based on the features corresponding to each sampling point in the point cloud data of different scales and the features of the neighborhood sampling points of each sampling point in the point cloud data of different scales, the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales is determined, and the neighborhood perception coefficient is used to characterize the influence of the neighborhood sampling point on the corresponding sampling point; Based on the neighborhood perception coefficients of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales, the features corresponding to each sampling point in the point cloud data of different scales are updated to obtain the updated features corresponding to each sampling point in the point cloud data of different scales; The multiple neighborhood perception and feature decoding layers are used to decode and splice the updated features corresponding to each sampling point in the point cloud data of different scales to obtain splicing features, so that the splicing features can be used as input of the output layer for calculation to obtain the landslide classification and recognition results of each point in the laser point cloud data.

2. The point cloud neighborhood structure-aware self-attention landslide segmentation method according to claim 1, characterized in that: The step of calculating the slope corresponding to each sampling point in the point cloud data of different scales and the slope variation between each sampling point and its neighboring sampling points includes: The slope corresponding to each sampling point in point cloud data of different scales is calculated by fitting the plane using principal component analysis method; Based on the slopes corresponding to each sampling point in point cloud data of different scales and the slopes corresponding to the neighborhood sampling points of each sampling point in point cloud data of different scales, the slope variation between each sampling point and its neighborhood sampling points in point cloud data of different scales is calculated.

3. The point cloud neighborhood structure-aware self-attention landslide segmentation method according to claim 2, characterized in that: The slope corresponding to the sampling point is calculated according to the following formula (1); Among them, n x 、n y and n z They respectively represent the components of the minimum eigenvector of the point cloud data obtained by the principal component analysis method in the three coordinate directions; The slope variation between the sampling point and its neighboring sampling points is calculated according to the following formula (2); Among them, θ i represents the slope corresponding to the i-th sampling point, θ ij Indicates the slope corresponding to the jth neighborhood sampling point of the i-th sampling point.

4. The point cloud neighborhood structure-aware self-attention landslide segmentation method according to claim 1, characterized in that: According to the following formula (3), the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales is determined; Among them, α and β represent linear transformations, f i represents the feature corresponding to the i-th sampling point, f ij represents the characteristics of the jth neighboring sampling point of the i-th sampling point, represents a multilayer perceptron, λ represents the position encoding of the i-th sampling point and the j-th neighboring sampling point of the i-th sampling point in three-dimensional space, d k represents the key vector dimension of the neighborhood-aware self-attention module in the neighborhood-aware and feature encoding layer, ξ represents the neighborhood-aware encoding, and Linear represents linear transformation, σ ReLU Represents the ReLU activation function.

5. The point cloud neighborhood structure-aware self-attention landslide segmentation method according to claim 4, characterized in that: Update the features corresponding to the sampling points according to the following formula (4); Among them, ⊙ represents the Hadamard product, p represents the neighborhood sampling point corresponding to the sampling point, represents the set of neighborhood sampling points corresponding to the sampling point, f ij Indicates the characteristics of the neighborhood sampling points corresponding to the sampling point.

6. The point cloud neighborhood structure-aware self-attention landslide segmentation method according to claim 1, characterized in that: The multiple neighborhood perception and feature coding layers include a first neighborhood perception and feature coding layer, a second neighborhood perception and feature coding layer, a third neighborhood perception and feature coding layer, a fourth neighborhood perception and feature coding layer, and a fifth neighborhood perception and feature coding layer; the multiple neighborhood perception and feature decoding layers include a first neighborhood perception and feature decoding layer, a second neighborhood perception and feature decoding layer, a third neighborhood perception and feature decoding layer, a fourth neighborhood perception and feature decoding layer, and a fifth neighborhood perception and feature decoding layer; the first neighborhood perception and feature coding layer includes a first neighborhood perception layer and a first coding layer; the second neighborhood perception and The feature coding layer includes a second neighborhood perception layer and a second coding layer, the third neighborhood perception and feature coding layer includes a third neighborhood perception layer and a third coding layer, the fourth neighborhood perception and feature coding layer includes a fourth neighborhood perception layer and a fourth coding layer, the fifth neighborhood perception and feature coding layer includes a fifth neighborhood perception layer and a fifth coding layer, the first neighborhood perception and feature decoding layer includes a first decoding layer and the first neighborhood perception layer, the second neighborhood perception and feature decoding layer includes a second decoding layer and the second neighborhood perception layer, the third neighborhood perception and feature decoding layer includes a third decoding layer and the third The fourth neighborhood perception and feature decoding layer includes a fourth decoding layer and the fourth neighborhood perception layer, and the fifth neighborhood perception and feature decoding layer includes a fifth decoding layer and the fifth neighborhood perception layer. The first neighborhood perception layer, the first coding layer, the second neighborhood perception layer, the second coding layer, the third neighborhood perception layer, the third coding layer, the fourth neighborhood perception layer, the fourth coding layer, the fifth neighborhood perception layer and the fifth coding layer are connected in sequence. The first decoding layer, the fifth neighborhood perception layer, the second decoding layer, the fourth neighborhood perception layer, the third decoding layer, the third neighborhood perception layer, the fourth decoding layer, the second neighborhood perception layer, the fifth decoding layer and the first neighborhood perception layer are connected in sequence. The output end of the first coding layer is also connected to the input end of the fifth decoding layer, the output end of the second coding layer is also connected to the input end of the fourth decoding layer, the output end of the third coding layer is also connected to the input end of the third decoding layer, the output end of the fourth coding layer is also connected to the input end of the second decoding layer, and the output end of the fifth coding layer is connected to the input end of the first decoding layer through a multi-layer perception layer.

7. The point cloud neighborhood structure-aware self-attention landslide segmentation method according to claim 6, characterized in that: The first neighborhood perception and feature encoding layer, the second neighborhood perception and feature encoding layer, the third neighborhood perception and feature encoding layer, the fourth neighborhood perception and feature encoding layer and the fifth neighborhood perception and feature encoding layer sample the input point cloud data at different scales according to the sampling numbers of N, N / 4, N / 16, N / 64 and N / 256 respectively, where N represents the total amount of data corresponding to the point cloud data, and the first neighborhood perception and feature decoding layer, the second neighborhood perception and feature decoding layer, the third neighborhood perception and feature decoding layer, the fourth neighborhood perception and feature decoding layer and the fifth neighborhood perception and feature decoding layer upscale the spliced ​​features according to multiples of 1, 2, 4, 8 and 16 respectively.

8. A point cloud neighborhood structure-aware self-attention landslide segmentation device, characterized in that: include: An acquisition unit, used for acquiring laser point cloud data of the landslide area; A computing unit, used for computing the point cloud data of each point in the laser point cloud data as an input of a pre-trained landslide segmentation model to obtain a landslide classification and recognition result of each point in the laser point cloud data; The landslide segmentation model includes an input layer, multiple neighborhood perception and feature encoding layers, multiple neighborhood perception and feature decoding layers, and an output layer; The multiple neighborhood perception and feature encoding layers are used to perform sampling and feature extraction of input point cloud data at different scales; Calculate the slope corresponding to each sampling point in point cloud data of different scales and the slope variation between each sampling point and its neighboring sampling points; Extract features of each sampling point in point cloud data of different scales, and obtain features of each sampling point and its neighboring sampling points in point cloud data of different scales; Based on the features corresponding to each sampling point in the point cloud data of different scales and the features of the neighborhood sampling points of each sampling point in the point cloud data of different scales, the neighborhood perception coefficient of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales is determined, and the neighborhood perception coefficient is used to characterize the influence of the neighborhood sampling point on the corresponding sampling point; Based on the neighborhood perception coefficients of the neighborhood sampling points corresponding to each sampling point in the point cloud data of different scales, the features corresponding to each sampling point in the point cloud data of different scales are updated to obtain the updated features corresponding to each sampling point in the point cloud data of different scales; The multiple neighborhood perception and feature decoding layers are used to decode and splice the updated features corresponding to each sampling point in the point cloud data of different scales to obtain splicing features, so that the splicing features can be used as input of the output layer for calculation to obtain the landslide classification and recognition results of each point in the laser point cloud data.

9. An electronic device, characterized in that: It includes a memory, a processor and a transceiver which are sequentially communicatively connected, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the point cloud neighborhood structure-aware self-attention landslide segmentation method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or the instruction is executed by a computer, the point cloud neighborhood structure-aware self-attention landslide segmentation method according to any one of claims 1 to 7 is implemented.

Images