A point cloud classification method combining spatial features and spectral features

Abstract

The application relates to a point cloud classification method combining spatial features and spectral features, and specifically comprises the following steps: defining a convolution kernel and convolution operation for point cloud convolution; performing point cloud convolution and pooling operation; establishing a point cloud classification model combining spatial and spectral features; training the classification model for hyperspectral laser radar point cloud data; and classifying the point cloud data of the hyperspectral laser radar by using the point cloud data classification model. The application combines the spatial and spectral information obtained by the hyperspectral laser radar for joint processing and feature extraction, carries out point cloud classification research on the fusion of spatial and spectral features, proposes a point cloud classification model combining spatial and spectral features, and further improves the point cloud classification precision of the laser radar.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral lidar, and particularly relates to a point cloud classification method that combines spatial and spectral features. Background Technology

[0002] LiDAR, as a modern optical remote sensing technology that utilizes active detection, combines traditional radar technology with modern laser technology, becoming a new technological development direction in the field of Earth observation. LiDAR point cloud classification involves acquiring semantic labels for each point in the LiDAR point cloud data, enabling the description and analysis of semantic attributes such as scene structure and object function. Accurate LiDAR point cloud classification is the foundation for complex indoor and outdoor visual interpretation and scene understanding.

[0003] Currently, the research focus on point cloud classification using lidar data both domestically and internationally primarily involves obtaining high-density and high-precision 3D point cloud data and then calculating geometric quantities such as distance variation, normal variation, curvature variation, elevation difference, density, and topological relationships to achieve point cloud classification and feature extraction. Lidar typically acquires very large amounts of point cloud data, making point cloud classification and feature extraction computationally intensive, inefficient, and inaccurate. Furthermore, some algorithms fail to yield accurate results when geometric differences are small. Additionally, traditional lidar systems are generally single-wavelength, limiting their ability to acquire target attribute information. The spectral information in the laser backscattering intensity data is relatively insufficient, resulting in limited detection capabilities for ground object categories. Fusion of passive optical hyperspectral remote sensing data with lidar data can compensate for the lack of spectral information in lidar, effectively improving the accuracy of ground object identification and classification. However, the biggest challenge in jointly interpreting data from different sensors is accurately registering different types of data to the same spatial coordinate system. Simultaneously, hyperspectral imaging is constrained by passive light source illumination conditions.

[0004] Hyperspectral lidar, as a novel remote sensing method developed in recent years, can overcome the limitations of natural lighting conditions and possesses both high spatial detection capabilities and the ability to detect the physical properties of ground objects. Besides acquiring the three-dimensional point cloud geometric information of the target surface, it can also simultaneously obtain the spectral information of ground objects, resulting in lidar point cloud data with hyperspectral information. The hyperspectral lidar system generates a point cloud [x,y,z,I] that combines hyperspectral backscattering intensity. i (λ)](i=1,2,…n),I i (λ) represents the backscattering intensity as a continuous function of wavelength λ. Different target surfaces have different reflectivities for different wavelengths, resulting in differences in the obtained laser intensity values. The echo intensity of hyperspectral lidar characterizes the reflection spectral characteristics of the target for lasers of different wavelengths.

[0005] In recent years, many researchers have studied point cloud classification for LiDAR based on spectral information, which mainly comes from unnormalized raw point cloud intensity information or normalized spectral information. Currently, commonly used methods for point cloud classification in hyperspectral LiDAR include Support Vector Machines (SVMs), Decision Trees, Logistic Regression, Naive Bayes, and Random Forests. These traditional machine learning methods primarily rely on manually designed features as input to the classifier, representing shallow classifier approaches. However, the features extracted by manually designed rules have weak expressive power and struggle to bridge the semantic gap between the spectral data features of the low-level point cloud and the high-level logical information, often resulting in low classification accuracy. Furthermore, specific classification rules and parameter designs have poor adaptability to point clouds in complex scenes, and the model's generalization ability fails to achieve satisfactory results when handling complex scenarios.

[0006] In recent years, deep learning-based methods have achieved a series of breakthroughs in fields such as image classification and object recognition, demonstrating stronger feature learning and representation capabilities compared to traditional machine learning methods. Deep learning-based methods can automatically extract point cloud feature information from data through an end-to-end learning mechanism, without requiring manually defined features. This effectively avoids the influence of subjective factors in feature extraction by traditional machine learning methods, thus adaptively extracting features and classifying them, resulting in strong model generalization capabilities. Given the massive amounts of point cloud data from LiDAR, and with sufficient labeled training samples, deep learning networks can often achieve good classification accuracy.

[0007] LiDAR point cloud classification involves acquiring semantic labels for each point in LiDAR point cloud data, enabling the description and analysis of semantic attributes such as scene structure and object function. Accurate LiDAR point cloud classification is the foundation and prerequisite for visual interpretation and scene understanding. Currently, research methods for LiDAR point cloud classification both domestically and internationally primarily involve obtaining high-density and high-precision 3D point cloud data, calculating the geometric changes of the point cloud, and then achieving point cloud classification and feature extraction. Hyperspectral LiDAR, in addition to acquiring the 3D point cloud geometric information of the target surface, can also simultaneously obtain the spectral information of ground features. Summary of the Invention

[0008] To fully leverage the advantages of high-precision three-dimensional information and high-resolution spectral information acquisition by hyperspectral lidar, this invention performs joint processing and feature extraction on the spatial and spectral information acquired by hyperspectral lidar, conducts research on point cloud classification based on spatial-spectral feature fusion, and proposes a point cloud classification model that combines spatial and spectral features to further improve the accuracy of lidar point cloud classification.

[0009] This invention proposes a point cloud classification method that combines spatial and spectral features, specifically including the following steps:

[0010] Step 1: Define the convolution kernel and convolution operation for point cloud convolution;

[0011] Step 2: Extract spatial features by performing convolution and pooling operations on the point cloud based on the spatial feature convolutional neural network branch.

[0012] Step 3: Extract spectral features by performing convolution and pooling operations on the spectral dimension based on the spectral feature convolutional neural network branch;

[0013] Step 4: Establish a point cloud classification model that combines spatial and spectral features.

[0014] Furthermore, in step one,

[0015] A point cloud generated by a hyperspectral lidar point is a collection of 3D points. It contains a total of M points in the point cloud, specifically described by the following formula (5):

[0016]

[0017] In the formula: It is the m-th point in the point cloud;

[0018] Point P m The attribute f describes its spatial coordinates and spectral reflectance information, expressed as the following formula (6):

[0019] f(P m ) = [X m ,Y m Z m ,ρ m (λ1),ρ m (λ2),…,ρ m (λ n (6)

[0020] In the formula, X m ,Y m Z m They represent point P respectively m The three-dimensional coordinates, ρ m (λ n Point P represents m At wavelength λ n Spectral reflectance at time

[0021] P is defined based on H neighboring points. m 3D sensory field, sensory field Defined as follows (7):

[0022]

[0023] In the formula: Represents "any", ||P m -P h ||Representative point P m and point P h distance, Represented by distance ||P m -P h ||, point P m H nearest neighbors, the nearest neighbors are described as {P} h |h=1,2,…,H}, based on H nearest neighbors, point P m The attribute is represented as

[0024] To perform convolution operations in 3D point clouds, the 3D convolution kernel is defined as follows (8):

[0025] K S ={k0,k1,…,k S-1} (8)

[0026] In the formula: S represents the number of weights in the convolution kernel, the center coordinates of the convolution kernel are k0 = (0,0,0), for each kernel point k in the convolution kernel, the dimension of the weights in the deep learning process is consistent with the dimension of the attributes of each point, and the weight of each kernel point in the convolution kernel is defined as ω(k S Based on H neighborhood points and convolution kernel K S The weight vector is defined by the following formula (9) for the convolution operation in 3D point clouds:

[0027]

[0028] Define function P represents h -P m and K s The cosine similarity, as shown in formula (10), indicates that the higher the similarity, the stronger the influence of the convolution kernel on neighboring points. When the maximum value is reached, we obtain the value s at which the maximum value is reached, denoted as s'.

[0029]

[0030] In the formula, the operation<A,B> The inner product operation of vectors A and B is represented by the following formula (11):

[0031]

[0032] Furthermore, in step two,

[0033] The set P of 3D point clouds is the input of the point cloud convolutional network. The convolutional layer performs the convolution operation of formula (11). The pooling operation adopts the max pooling method to obtain the receptive field of each point. The downsampling rate is r. The spatial feature convolutional neural network branch includes the input layer, convolutional layer, max pooling layer and fully connected layer.

[0034] Furthermore, in step three,

[0035] The input layer of the spectral feature convolutional neural network branch is I. cnn =[ρ(λ1),ρ(λ2),…,ρ(λ) n )],(n=1,2,…,N), where n is the wavelength index, N is the wavelength number, and ρ(λ n ) is the nth reflectance;

[0036] The convolutional layers of the spectral feature convolutional neural network branch perform convolution operations through different convolutional kernels, as shown in formula (12):

[0037]

[0038] Where: matrix It is the i-th feature of the previous layer, i.e., the (l-1)-th layer. It is the j-th feature of the current layer, i.e., the l-th layer, and M is the number of features; and is the convolution kernel parameter of the current layer, i.e., the l-th layer; f is a non-linear function; and * is the convolution operation.

[0039] The spectral feature convolutional neural network branch includes an input layer, convolutional layers, pooling layers, a ReLU layer after each convolutional layer, and a fully connected layer, with two training processes: forward propagation and backward propagation.

[0040] Furthermore, in step four,

[0041] The point cloud classification model network consists of a spectral feature convolutional neural network branch and a spatial feature convolutional neural network branch. The fully connected layers of the two branches are concatenated to generate a new fully connected layer, representing the fused features. The operation process is shown in formula (13), where F 光谱 It is a deep-level spectral eigenvector, F 空间 It is a spatial feature vector, and F represents the fused feature vector.

[0042] F = [F 光谱 ,F 空间 (13).

[0043] This invention offers the following beneficial technical effects: In addition to acquiring the geometric information of point clouds, hyperspectral lidar can also acquire spectral information of targets across multiple bands. It possesses the characteristics of high-dimensionality, heterogeneity, and massive data volume, demonstrating enormous potential and flexibility in target feature extraction and recognition applications. The spectral data of hyperspectral lidar is a collection of spectral vectors from different bands. Each point in the detected point cloud corresponds to a vector in an N-dimensional feature space, and the spectral sequence in the spectral space is essentially an ordered and continuous data structure.

[0044] This invention relates to a point cloud classification method that combines spatial and spectral features, achieving feature-level fusion of spectral and point cloud spatial data. This invention automatically extracts and classifies multiple features from point cloud data, fully utilizing the advantages of hyperspectral lidar in simultaneously acquiring high-precision 3D information and high-resolution spectral information, resulting in high classification accuracy and strong generalization ability. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of a three-dimensional imaging experiment using a hyperspectral lidar.

[0046] Figure 2 A schematic diagram of fitting a plane to the neighborhood set of a point cloud;

[0047] Figure 3 For P m A schematic diagram of the receptive field and convolution kernel;

[0048] Figure 4 A schematic diagram of PointConv-CNN, a point cloud convolutional network for spatial feature extraction;

[0049] Figure 5 A schematic diagram of the network structure for the spectral feature extraction branch;

[0050] Figure 6 This is the model architecture for Spectrum-PointConv-CNN. Detailed Implementation

[0051] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] (1) Hyperspectral lidar point cloud data acquisition and point cloud calculation

[0053] Hyperspectral lidar data acquisition equipment mainly includes a laser emitting unit, a laser receiving unit, and a scanning unit. The laser emitting unit uses a supercontinuum laser as the emission source, continuously emitting white light signals. An acousto-optic tunable filter selects different wavelengths for emission through signal tuning in the time domain. The laser receiving unit mainly consists of a receiving optical system, a photodetector, and a data acquisition system. Its function is to collect the laser echo signal through a telescope optical system and convert it into an electrical signal on a photodiode (APD) photodetector, then record the full waveform echo signal of the hyperspectral lidar through the acquisition module. The system scanning unit mainly achieves the acquisition of high-density point cloud data through the control of a two-dimensional turntable.

[0054] 1) Distance calculation

[0055] The ranging principle of hyperspectral lidar is similar to that of traditional single-wavelength lidar. It mainly calculates the distance from the lidar to the target by calculating the time interval between the emitted laser signal and the received echo signal. To eliminate noise during the measurement process, Gaussian filtering is first applied to the full-waveform lidar echo data. Based on the filtered data, the maximum echo intensity and its position corresponding to the peak are extracted, and the flight time of the laser is calculated. The distance R between the lidar and the target is calculated as follows: (1)

[0056]

[0057] Where Δt is the round-trip time of the laser pulse, and c is the speed of light.

[0058] 2) Three-dimensional coordinate calculation

[0059] To scan a target, the hyperspectral lidar transmitting module needs to rotate in both azimuth and elevation directions using a two-dimensional turntable on the scanning module. This rotation allows the laser beam to scan the target from all angles, thereby acquiring the target's three-dimensional information. The scanning pattern is zigzag, as shown below. Figure 1 As indicated by the middle arrow. When the lidar system is working, the digital acquisition card records the azimuth and elevation angles of the 2D turntable in real time and uses them for subsequent point cloud calculations. This angular information helps the system determine the position and angle of the laser beam illumination, thereby generating the corresponding point cloud data.

[0060] Wherein, the azimuth angle α is the angle between the projection of the laser beam and the y-axis, with clockwise rotation being positive; the elevation angle β is the angle between the laser beam and its projection. The laser beam projection is a projection onto a plane determined by the x-axis and y-axis.

[0061] After calculating the distance R between the lidar and the target, the three-dimensional coordinates of the laser point cloud in the Cartesian coordinate system can be calculated by combining the pitch and azimuth angles of the turntable at the moment of laser emission, and the three-dimensional point cloud solution can be completed based on the calculation results.

[0062] The origin of the hyperspectral lidar system is denoted as (X0, Y0, Z0), and the coordinates of any scanning point are (X, Y, Z). The spatial coordinates of the scanning point in the Cartesian coordinate system are calculated as follows (2):

[0063]

[0064] The above calculations can generate a point cloud with spatial coordinates, where each point also contains its corresponding backscattering intensity.

[0065] 3) Calculation of reflectivity based on a standard diffuse reflector

[0066] When scanning a target using a hyperspectral lidar, the reflectivity of the target is calculated by the ratio of the backscattering intensity of the target to that of a standard diffuse reflector at the same distance, as shown in the following formula (3):

[0067]

[0068] Among them, f APD_ref (λ) is the backscattering intensity of a standard diffuse reflector, f APD_obj (λ) is the backscattering intensity of the target being detected, ρ ref (λ) is the reflectivity of the standard reflector, ρ obj (λ) is the reflectivity of the target being detected.

[0069] 4) Calculation of laser incident angle

[0070] Let the emission center of the hyperspectral lidar system be O(X0,Y0,Z0), and the scanning point be S(X,Y,Z). Then the incident laser vector is OS = (X-X0,Y-Y0,Z-Z0). First, the best-fit plane around the small region of the scanning point is calculated using the K-Nearest Neighbor (KNN) algorithm. A neighborhood set fitting plane is then established, and the normal vector of the fitting plane towards the lidar direction is denoted as the normal vector n of the scanning point. Figure 2 As shown. The laser incident angle θ at the scanning point is calculated according to formula (4), and the range of the incident angle is θ∈[0,90°].

[0071]

[0072] (2) Establish a dataset that meets the requirements for model training in terms of both quantity and quality.

[0073] Define root-type ground object classes and annotate point cloud data with these classes through visual interpretation and manual delineation. This results in a large and high-quality set of training, validation, and testing data to meet the data requirements of convolutional neural networks, thereby enhancing the model's generalization ability and classification performance.

[0074] (3) Establishment of a point cloud classification model that combines spatial and spectral features

[0075] 1) Definition of convolution kernel and convolution operation

[0076] In CNN models for raster images, the convolutional kernel consists of weight parameters from the grid. However, for 3D point cloud data from LiDAR, the data points are unordered, making it difficult to apply convolutional operations from raster images to point cloud convolution. This invention proposes a convolutional network for point clouds to extract spatial features from neighboring points.

[0077] A point cloud generated by a hyperspectral lidar point is a collection of 3D points. It contains a total of M points in the point cloud, specifically described by the following formula (5):

[0078]

[0079] In the formula: It is the m-th point in the point cloud.

[0080] Point P m The attributes describe its spatial coordinates and spectral reflectance information, expressed as the following formula (6):

[0081] f(P m ) = [X m ,Y m Z m ,ρ m (λ1),ρ m (λ2),…,ρ m (λ n (6)

[0082] In the formula, X m ,Y m Z m They represent point P respectively m The three-dimensional coordinates, ρ m (λ n Point P represents m At wavelength λ n The spectral reflectance at time H. P is defined based on H neighboring points. m 3D sensory field. Sensory field Defined as follows (7):

[0083]

[0084] In the formula: Represents "any", ||P m -P h ||Representative point P m and point P h The distance; Represented by distance ||P m -P h ||, point P m The H nearest neighbors are given. The nearest neighbors are described as {P}. h {h=1,2,…,H}. Based on the H nearest neighbors of point P. m The attribute is represented as These properties will be calculated and updated during the convolution operation of the point cloud.

[0085] In order to perform convolution operations in 3D point clouds, the present invention defines the 3D convolution kernel as follows (8):

[0086] K S ={k0,k1,…,k S-1} (8)

[0087] In the formula: S represents the number of weights in the convolution kernel. The center coordinates of the convolution kernel are k0 = (0,0,0). For each kernel point k in the convolution kernel, the dimension of the learned weights is consistent with the dimension of the attributes of each point. For example... Figure 3 As shown, the weight of each kernel point in the convolution kernel is defined as ω(k S ).

[0088] Based on H neighborhood points and convolution kernel K S The weight vector, the convolution operation in 3D point cloud is defined by the following formula (9):

[0089]

[0090] This invention defines functions Let P represent h -P m and K s The cosine similarity is shown in Equation (10). The higher the similarity, the stronger the influence of the convolution kernel on neighboring points. When When the maximum value is obtained, we get s when the maximum value is obtained, denoted as s′.

[0091]

[0092] In the formula, the operation<A,B> This represents the inner product operation of vectors A and B. The convolution operation is defined as follows (11):

[0093]

[0094] 2) Extracting spatial features from point clouds using convolution and pooling operations.

[0095] Collection of 3D point clouds The input to the point cloud convolutional network consists of M points. The convolutional layers perform the convolution operation as shown in Equation 11. Pooling is also an important part of the network, improving learning efficiency. Here, max pooling is used to obtain the receptive field of each point, with a downsampling rate of r.

[0096] The spatial feature extraction branch of the design includes an input layer, convolutional layers, max-pooling layers, and fully connected layers. The point cloud convolutional network is defined as PointConv-CNN, as follows: Figure 4 As shown.

[0097] The network parameters are configured as follows. First, in order to fuse spatial information from different bands and suppress noise, this invention uses Principal Component Analysis (PCA) to reduce the spectral dimension of the hyperspectral lidar data to a low-dimensional subspace. For each point, this invention selects the top few components whose cumulative contribution reaches a certain threshold. Figure 4 Taking the first three components as an example, the steps for extracting spatial features are explained. Next, the *a* neighboring points of each point are imported into the network. Figure 4 (Taking a = 512 as an example). Point P m The attribute is: f(P) m ) = [X m ,Y m Z m ,ρ m (λ1),ρ m (λ2),ρ m (λ3)]. Figure 4 The network in this example consists of four convolutional layers, with the number of kernels increasing from low to high: 16, 32, 64, and 128. Correspondingly, the model has four max-pooling layers with a fixed sampling rate. The weights S of the convolutional kernels are set according to the specific point cloud classification scenario, initially using a smaller value, and then selecting a suitable S value through cross-validation. The choice of the H value will significantly affect the receptive field. Typically, a smaller H value is initially chosen, and then a suitable H value is selected through cross-validation.

[0098] 3) Spectral feature extraction based on convolutional neural networks

[0099] The reflectance data of a single point in the original hyperspectral lidar data is a one-dimensional array. Convolutional Neural Networks (CNNs) extract spectral features through a series of convolution and pooling operations in the spectral dimension. The input layer of the model is I... cnn =[ρ(λ1),ρ(λ2),…,ρ(λ) n )],(n=1,2,…,N), where n is the wavelength index, N is the wavelength number, and ρ(λ n ) is the nth reflectance.

[0100] CNN consists of a series of data processing layers that extract representative features from the data through nonlinear transformations. The transformation operation is composed of a series of data processing layers, mainly including convolutional layers and pooling layers, forming a deep architecture. The convolutional layers mainly implement convolution operations through different convolutional kernels, as shown in the following formula (12):

[0101]

[0102] Where: matrix It is the i-th feature of the previous layer (the (l-1)-th layer). Let f be the j-th feature of the current layer (layer l), and M be the number of features. These features are fine-tuned through backpropagation. f is a non-linear function, and * represents the convolution operation. The convolution operation calculates the similarity between the convolution kernel and the relevant features. The higher the similarity, the larger the output value of the convolution operation.

[0103] Pooling layers reduce the spectral dimensionality and extract more representative features through downsampling. The dimensionality of the output features decreases in the pooling layers, while the number of input and output features remains unchanged. Max pooling and average pooling are two commonly used pooling operations. The fully connected layer maps the learned features to sample labels. Finally, the probability of the extracted features belonging to different class labels is calculated.

[0104] The spectral feature extraction CNN branch designed in this invention includes an input layer, a convolutional layer, a pooling layer, a ReLU layer after each convolutional layer, and a full connection layer. Forward propagation and back propagation are two processes in the training process. In this study, training data is input into the CNN model, forward propagation uses the current parameters to calculate the classification performance of the input, and the training parameters are fine-tuned through back propagation.

[0105] This invention will use the example of inputting raw hyperspectral lidar data containing 75 wavelengths of spectral reflectance data into a CNN model for illustration. Figure 5This is a network architecture for extracting spectral features. The input layer has a data dimension of 1×75. The first convolutional layer (C1) mainly uses 10 1×5 convolutional kernels to perform convolution operations, generating a 1×7 1×10 dimensional vector. Each element in the convolutional kernel is a learned weight. After the convolution operation, initial features can be extracted. After the convolutional layer, a pooling operation is performed, using max pooling with a stride of 2. Pooling is used to reduce the data dimensionality and extract features through downsampling. After the pooling layer, the data dimension is reduced by half. The third layer (C3) and the fifth layer (C5) convolve the features obtained in the previous layer with 1×3 convolutional kernels, producing vectors of 1×3 4×20 and 1×15×40, respectively. Finally, the sixth layer (P6) is added to the fully connected layer, and the output vector is the spectral features extracted from the original hyperspectral lidar data. The above parameters of the convolutional kernels are empirical values ​​obtained based on relevant research and are parameters with high classification accuracy obtained through multiple experiments.

[0106] 4) Point cloud classification model combining spatial and spectral features

[0107] To fully utilize the spectral and spatial features of the original point cloud, this invention designs a hyperspectral lidar point cloud classification model, "Spectral-PointConv-CNN," which combines spatial and spectral features. The proposed model framework is as follows: Figure 6 As shown, the network consists of two branches: spectral feature extraction and spatial feature extraction. The network creates relevant features for the input and feeds these features into fully connected layers to create joint features, thereby predicting the final label for each point in the point cloud.

[0108] In this invention, the original hyperspectral lidar data is input into the model. The spectral feature extraction branch focuses on distinguishing the basic attributes of the original laser point spectral information and the internal correlations between spectra. The spatial feature extraction branch focuses on the geometric features of the point cloud. The two fully connected layers of the two CNN branches are concatenated to generate a new fully connected layer, representing the fused features. The operation process is shown in formula (13), where F 光谱 It is a deep-level spectral eigenvector, F 空间 It is a spatial feature vector. F represents the fused feature vector.

[0109] F = [F 光谱 ,F 空间 (13)

[0110] (4) Training of hyperspectral lidar point cloud data classification model

[0111] The measured hyperspectral lidar point cloud data is divided into training set, validation set and test set. With appropriate learning rate and batch size set, a training model with ideal classification accuracy is obtained based on the point cloud classification model with joint spatial features and spectral features established above.

[0112] (5) Classification, prediction and accuracy evaluation of hyperspectral lidar point cloud data

[0113] A point cloud classification model based on a trained joint convolutional neural network and spectral features is used to classify and predict the test set of hyperspectral lidar point cloud data. The classification accuracy of each target is evaluated using the confusion matrix and Kappa coefficient.

[0114] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1.A method for point cloud classification combining spatial features and spectral features, characterized in that, Specifically comprising the following steps: Step one, define the convolution kernel and convolution operation of point cloud convolution; Step two, perform point cloud convolution operation and pooling operation based on the spatial feature convolutional neural network branch to extract spatial features; Step three, perform spectral dimension convolution operation and pooling operation based on the spectral feature convolutional neural network branch to extract spectral features; Step four, establish a point cloud classification model combining spatial features and spectral features; In step one, the point cloud generated by one hyperspectral lidar point is a set of 3D points , which contains a total of points in the point cloud, which is specifically described as formula (5): (5) In the formula: It is the first in point cloud One point; Point Attributes Describes its spatial coordinates and spectral reflectance information, expressed as follows Equation (6): (6) wherein respectively represent the three-dimensional coordinates of a point respectively represent the three-dimensional coordinates of a point respectively represent the three-dimensional coordinates of a point respectively represent the three-dimensional coordinates of a point the spectral reflectance at a wavelength of Based on a 3D receptive field defined by one neighboring point, the receptive field is defined as the following equation (7): (7) wherein: represents "any", represents a point and a point distance; denotes the k-nearest neighbors of a point based on distance k-nearest neighbors are described as based on the attributes of the k-nearest neighbors of a point ,​​​ In order to perform convolution operation in 3D point cloud, 3D convolution kernel is defined as formula (8) as follows: (8) wherein: represents the number of weights in the convolution kernel, and the center coordinate of the convolution kernel is , for each kernel point in the convolution kernel , the dimension of the weight in the deep learning process is consistent with the dimension of the attribute of each point, and the weight of each kernel point in the convolution kernel is defined as , based on neighbor points and the weight vector of the convolution kernel , the convolution operation in the 3D point cloud is defined as formula (9) as follows: (9) Definition function Indicates And The cosine similarity of formula (10), the higher the similarity, the stronger the influence of the convolution kernel on the neighborhood points, when Maximum value, get the maximum value of , recorded as ; (10) wherein , operation denotes the inner product operation of vectors A and B, and the convolution operation is defined as the following equation (11): (11) 。 2.The method of claim 1, wherein, In step two, A set of 3D point clouds is the input of the point cloud convolutional network, the convolutional layer performs convolution operation of formula (11), the pooling operation adopts the maximum pooling method to obtain the receptive field of each point, and the down-sampling rate is The spatial feature convolutional neural network branch includes an input layer, a convolutional layer, a maximum pooling layer and a fully connected layer. 3.The method of claim 1, wherein, In step three, The input layer of the spectral feature convolutional neural network branch is wherein is the wavelength number, is the number of wavelengths, is the reflectance of the th wavelength. The convolution layer of the spectral feature convolutional neural network branch realizes convolution operation through different convolution kernels, as formula (12) as follows: (12) Where: matrix It is the previous layer, that is, the first The first layer One characteristic, It is the current layer, i.e., the first layer. The j-th feature of the layer, where M is the number of features; and It is the current layer, i.e., the first layer. The convolution kernel parameters of the layer, It is a non-linear function, and It is a convolution operation; The spectral feature convolutional neural network branch contains an input layer, a convolution layer, a pooling layer, a ReLU layer after each convolution layer and a fully connected layer, and has two training processes of forward propagation and back propagation. 4.The method of claim 1, wherein, In step four, The point cloud classification model network is composed of a spectral feature convolutional neural network branch and a spatial feature convolutional neural network branch, full connection layers of the two branch networks are spliced to generate a new full connection layer representing fusion features, and the operation process is as shown in formula (13), wherein, is a deep-level spectral feature vector, is a spatial feature vector, and F represents a fusion feature vector, (13) 。

Images