Photon Extraction Method for Spaceborne Lidar

Through the rasterization and neural network training of point cloud data, the problem of difficulty in extracting signal photons in ICESat-2 satellite-borne lidar is solved, and automated noise point classification is realized, improving the signal-to-noise ratio and processing of data generalization ability.

CN118549908BActive Publication Date: 2024-11-26SHANDONG UNIV OF SCI & TECH
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202410672677.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2024-02-02
Filing Date
2024-05-28
Publication Date
2024-11-26
Estimated Expiration
2044-05-28

AI Technical Summary

Technical Problem

Photon counting of ICESat-2 satellite-borne lidar is difficult to distinguish signal photons from noise photons in the background of strong noise, making it difficult to extract signal photons from background noise, affecting the signal-to-noise ratio of data applications.

Method used

By processing global positioned point cloud data, manually labeling and establishing point cloud rasterized image datasets, building neural networks for supervision and training, performing point cloud classification and information recovery, and using rasterized and neural network automatic learning to achieve noise points and signals Classification of points.

Benefits of technology

It realizes automated point cloud data processing under various terrain conditions, has good generalization capabilities, can effectively distinguish noise points and signal points, and improves the signal-to-noise ratio of the data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004863123140000041
    Figure BDA0004863123140000041
  • Figure BDA0004863123140000081
    Figure BDA0004863123140000081
  • Figure BDA0004863123140000104
    Figure BDA0004863123140000104
Patent Text Reader

Abstract

The present invention discloses a photon extraction method for a satellite-borne laser radar, belongs to the field of laser radar measurement technology, and is used for photon extraction of a satellite-borne laser radar, including processing global positioning point cloud data, manually labeling the point cloud data, obtaining a data source, establishing a point cloud rasterized image data set according to the data source, constructing a neural network for supervised training, performing point cloud classification, performing point cloud information recovery, and performing accuracy verification between the point cloud after information recovery and the real point cloud label data. The present invention rasterizes the point cloud data into images, performs supervised classification in the presence of prior information, can be applied to a variety of terrain conditions, and the entire processing process is automated with good generalization capability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention discloses a photon extraction method for a satellite-borne laser radar, and belongs to the technical field of laser radar measurement. Background Art

[0002] ICESat-2 is a new generation of ice, cloud and land elevation satellite launched by NASA in 2018. The characteristics of ICESat-2's low energy consumption, high detection sensitivity and high repetition frequency have greatly improved the sampling density along the track. However, since the laser pulses emitted and detected by the ICESat-2 photon counting lidar are weak signals, it is difficult to distinguish the target object. The surface return pulses, atmospheric scattering, solar radiation and the instrument's own noise cause a large amount of background noise to be mixed in its echo data. In the strong noise background, the low signal-to-noise ratio makes it difficult to distinguish signal photons from noise photons. How to extract signal photons from background noise has become a key and basic task in the application of ICESat-2 data, and the effective realization of photon point cloud classification has become the key to subsequent applications. Summary of the invention

[0003] The purpose of the present invention is to provide a photon extraction method for a satellite-borne laser radar to solve the problem in the prior art that it is difficult to extract signal photons from background noise.

[0004] The photon extraction method of the satellite-borne laser radar includes processing global positioning point cloud data, manually labeling the point cloud data, obtaining the data source, establishing a point cloud rasterized image data set based on the data source, building a neural network for supervised training, performing point cloud classification, performing point cloud information recovery, and verifying the accuracy of the point cloud after information recovery with the real point cloud label data.

[0005] Manually labeling point cloud data includes: label 1 represents noise points, and label 2 represents signal points.

[0006] The point cloud rasterized image dataset is established based on the data source, including using the global positioning point cloud data ATL03 as the experimental data for denoising, setting the grid resolution according to the distance information and elevation information along the track, and taking the photon point cloud rasterized image as the input data. The rasterization includes:

[0007] S1. Set the grid resolution, the point cloud data footprint spacing is 0.7m, the grid resolution along the track direction is 0.5m, and the grid resolution in the elevation direction is 0.05m. Based on the set grid resolution, the clipping range in the distance direction along the track is 256m, and the clipping range in the elevation direction is 25.6m. The point cloud data is clipped in the along-track direction to generate images.

[0008] Rasterization includes:

[0009] S2. Create a grid and sample according to the grid resolution:

[0010] y_range=y_min:y_resolution:y_max;

[0011] x_range=x_min:x_resolution:x_max;

[0012] Where y_range and x_range are one-dimensional matrices generated by elevation and distance along the track according to the grid resolution, y_min is the initial value of the elevation of the clipping window, y_max is the end value of the elevation of the clipping window, x_min is the initial value of the distance along the track of the clipping window, x_max is the end value of the distance along the track of the clipping window, x_resolution is the resolution of the distance along the track, and y_resolution is the elevation resolution;

[0013] Create two two-dimensional matrices x_grid and y_grid. x_grid and y_grid are obtained according to x_range and y_range respectively. Each row in x_grid corresponds to the distance value along the grid track, and each column in y_grid corresponds to the value of the elevation information along the grid.

[0014] Rasterization includes:

[0015] S3. Build a KD tree index, determine whether the grid point is near each data point, and select the point cloud data within the cropping range;

[0016] The KD tree matrix kd_tree_x of the along-track distance is generated according to the row transpose of x_grid, and the KD tree matrix kd_tree_y of the elevation is generated according to y_grid. The dimensions of the two KD tree matrices are both 512×1, containing grid coordinate information. A k-nearest neighbor search is performed to match each grid with the along-track distance information b(j, 1) and elevation information b(j, 2) that actually correspond to each data point in the cropping window, find the adjacent data points of the grid, and obtain the corresponding grid index numbers id_x and id_y, where j represents the jth data point and b is the matrix storing the along-track distance information and elevation information.

[0017] Rasterization includes:

[0018] S4. Generate a data set in batches according to the classification labels, set the pixel values ​​of all grids to 0, initialize them to black, traverse the index numbers, and then add corresponding colors according to the label values ​​corresponding to the point cloud grids to generate the data set required for neural network training;

[0019] S4.1. Generate the original image, read the point cloud data, and according to the label value, the pixel value of the signal point position is 1, which is white, and the pixel value of the non-signal point position is 0, which is black. Generate a black-and-white binary image after the point cloud is rasterized in the direction of the distance along the track. The image size dimension is [512, 512, 1], and save the black-and-white binary image in png format;

[0020] S4.2. Generate a mask image. According to the existing label values, label value 1 corresponds to the noise point. Color is added according to the label value. The grid corresponding to the noise point is green, and the RGB value is [0, 255, 0]. Label 2 is a valid signal point, and the corresponding grid position is red, and the RGB value is [255, 0, 0].

[0021] S4.3. Save the color image in png format with image size dimension [512, 512, 3];

[0022] S4.4. In the distance direction along the track, a sliding window is used for cropping to batch generate binary original raster images of [512, 512, 1] and color mask raster images of [512, 512, 3] to construct the data set required for network model training.

[0023] Building a neural network for supervised training includes:

[0024] T1. Divide the data set into a training set and a validation set, with the ratio of 8:2;

[0025] T2. Set the main structure of the neural network model, including three layers;

[0026] T2.1. The first layer is the encoder, which is used to capture the input image features. It consists of convolution blocks and attention mechanism modules. Each convolution block consists of a convolution layer and a pooling layer.

[0027] First, 32 3*3 convolution kernels are used to perform the same convolution twice on the input feature map to adjust the input feature map to 32 channels. The image size is 32*512*512, and the saved output is recorded as A1. The pooling kernel size is 2*2 for maximum pooling to reduce the image size. The output size is 32*256*256, and the saved output is recorded as B1. B1 is saved after the output of the attention mechanism module and recorded as C1.

[0028] Use 64 3*3 convolution kernels to perform the same convolution twice on the input feature map C1 to adjust the input feature map to 64 channels. The image size is 64*256*256, and the saved output is recorded as A2. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 64*128*128, and the saved output is recorded as B2. B2 is saved after the output of the attention mechanism module and recorded as C2;

[0029] Use 128 3*3 convolution kernels twice to perform the same convolution on the feature map C2 to adjust the input feature map to 128 channels. The image size is 128*128*128, and the saved output is recorded as A3. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 128*64*64, and the saved output is recorded as B3. B3 is saved after the output of the attention mechanism module and recorded as C3;

[0030] Use 256 3*3 convolution kernels twice to perform the same convolution on the feature map C3 to adjust the input feature map to 256 channels. The image size is 256*64*64, and the output is saved as A4. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 256*32*32, and the output is saved as B4. B4 is output to C4 through the attention mechanism module;

[0031] Use 512 3*3 convolution kernels to perform two same convolutions on the feature map C4 to adjust the input feature map to 512 channels. The image size is 512*32*32, and the saved image is recorded as A5.

[0032] Building a neural network for supervised training includes:

[0033] T2.2. The second layer is the decoder, which upsamples the feature layer and stacks it for feature fusion. The final feature layer has the same height and width as the input image.

[0034] Perform transposed convolution upsampling on feature map A5, with a transposed convolution kernel size of 2*2, a step size of 2, and a padding mode of "same". A skip connection is constructed between the upsampled output feature map and the A4 feature map. Two same convolutions are performed using 256 3*3 convolution kernels. The output image size is 256*64*64, and the output image is saved and recorded as A6.

[0035] Perform transposed convolution upsampling on feature map A6. The transposed convolution kernel size is 2*2, the step size is 2, and the padding mode is "same". The upsampled output feature map is skip-connected with the A3 feature map. 128 3*3 convolution kernels are used to perform two same convolutions. The output image size is 128*128*128. The output image is saved and recorded as A7.

[0036] Perform transposed convolution upsampling on feature map A7, with a transposed convolution kernel size of 2*2, a step size of 2, and a padding mode of "same". A skip connection is constructed between the upsampled output feature map and the A2 feature map. 64 3*3 convolution kernels are used to perform two same convolutions. The output image size is 64*256*256. The output image is saved and recorded as A8.

[0037] The feature map A8 is upsampled by transposed convolution. The transposed convolution kernel size is 2*2, the step size is 2, and the padding method is "same". The upsampled output feature map is skipped with the A1 feature map. 32 3*3 convolution kernels are used to perform two same convolutions. The output image size is 32*512*512. A residual connection is constructed with the output feature map for joint output. The joint output is saved and recorded as A9.

[0038] Building a neural network for supervised training includes:

[0039] T2.3. The third layer is the prediction classification layer. A9 is convolved with a convolution kernel size of 1*1, the number of channels is changed, and the activation function is connected to convert it into a classification result. Each pixel is classified into three categories: noise points, valid signal points, and background. The corresponding category label values ​​are 1, 2, and 3 respectively.

[0040] Point cloud information recovery includes:

[0041] For a color point cloud raster image, the initial along-track distance corresponding to the generated color point cloud raster image is x1, the final along-track distance is x2, the initial elevation is y1, and the final elevation is y2. It is known that the grid resolution in the along-track distance direction is 0.5m, the grid resolution in the elevation direction is 0.05m, the grid index is (id_x, id_y), and the along-track distance X′ and elevation information Y′ restored at the grid center position are as follows:

[0042] X′=x1+x_resoution×(id_x-1 / 2);

[0043] Y′=y1+y_resoution×(id_y-1 / 2);

[0044] The real distance along the track is restored from the color point cloud raster image, and the elevation information set is {X′, Y′}. The real distance along the track corresponding to the color point cloud raster image, and the elevation information set is {X0, Y0}. The two information sets are looped through to calculate the Euclidean distance, and the two data sets are matched using the Euclidean distance. When the Euclidean distance D < < ε, that is,

[0045]

[0046] Complete point cloud information recovery, where ε is the minimum value;

[0047] A track of ICESat-2 point cloud data generates an original raster image set according to the sliding window cropping order. The original raster image generates a corresponding mask raster image set after network model training. When the sliding window cropping step l′ is equal to the window size l, the raster image set is directly reversely restored and spliced ​​according to the cropping order to generate a global mask raster image; when the sliding window cropping step l′ is less than the window size l, each channel of the raster image set is seamlessly spliced ​​using linear edge fusion to generate a global mask raster image, as shown below:

[0048] I(x,y)=α(x,y)*I1(x,y)+(1-α(x,y))*I2(x,y)

[0049] I(x,y) is the pixel value of the fused image. I1(x,y) and I2(x,y) are the pixel values ​​of the two images in the overlapping area. α(x,y) is the weight, ranging from 0 to 1;

[0050] According to the point cloud recovery method of a color point cloud raster image, the point cloud information of the global mask raster image is restored to realize the classification of noise points and signal points of one track of ICESat-2 point cloud data, and the classification of multi-track ICESat-2 point cloud data can be realized in batches.

[0051] The unlabeled point cloud data is cropped through sliding windows, point cloud rasterization, neural network automatic learning, and point cloud information recovery and matching to obtain point cloud label values ​​and complete the classification of noise points and signal points.

[0052] Compared with the prior art, the present invention has the following beneficial effects: the present invention rasterizes point cloud data into images, performs supervised classification based on prior information, is applicable to a variety of terrain conditions, and the entire processing process is automated with good generalization capabilities. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] The photon extraction method of the satellite-borne laser radar includes processing global positioning point cloud data, manually labeling the point cloud data, obtaining the data source, establishing a point cloud rasterized image data set based on the data source, building a neural network for supervised training, performing point cloud classification, performing point cloud information recovery, and verifying the accuracy of the point cloud after information recovery with the real point cloud label data.

[0055] Manually labeling point cloud data includes: label 1 represents noise points, and label 2 represents signal points.

[0056] The point cloud rasterized image dataset is established based on the data source, including using the global positioning point cloud data ATL03 as the experimental data for denoising, setting the grid resolution according to the distance information and elevation information along the track, and taking the photon point cloud rasterized image as the input data. The rasterization includes:

[0057] S1. Set the grid resolution, the point cloud data footprint spacing is 0.7m, the grid resolution along the track direction is 0.5m, and the grid resolution in the elevation direction is 0.05m. Based on the set grid resolution, the clipping range in the distance direction along the track is 256m, and the clipping range in the elevation direction is 25.6m. The point cloud data is clipped in the along-track direction to generate images.

[0058] Rasterization includes:

[0059] S2. Create a grid and sample according to the grid resolution:

[0060] y_range=y_min:y_resolution:y_max;

[0061] x_range=x_min:x_resolution:x_max;

[0062] Where y_range and x_range are one-dimensional matrices generated by elevation and distance along the track according to the grid resolution, y_min is the initial value of the elevation of the clipping window, y_max is the end value of the elevation of the clipping window, x_min is the initial value of the distance along the track of the clipping window, x_max is the end value of the distance along the track of the clipping window, x_resolution is the resolution of the distance along the track, and y_resolution is the elevation resolution;

[0063] Create two two-dimensional matrices x_grid and y_grid. x_grid and y_grid are obtained according to x_range and y_range respectively. Each row in x_grid corresponds to the distance value along the grid track, and each column in y_grid corresponds to the value of the elevation information along the grid.

[0064] Rasterization includes:

[0065] S3. Build a KD tree index, determine whether the grid point is near each data point, and select the point cloud data within the cropping range;

[0066] The KD tree matrix kd_tree_x of the along-track distance is generated according to the row transpose of x_grid, and the KD tree matrix kd_tree_y of the elevation is generated according to y_grid. The dimensions of the two KD tree matrices are both 512×1, containing grid coordinate information. A k-nearest neighbor search is performed to match each grid with the along-track distance information b(j, 1) and elevation information b(j, 2) that actually correspond to each data point in the cropping window, find the adjacent data points of the grid, and obtain the corresponding grid index numbers id_x and id_y, where j represents the jth data point and b is the matrix storing the along-track distance information and elevation information.

[0067] Rasterization includes:

[0068] S4. Generate a data set in batches according to the classification labels, set the pixel values ​​of all grids to 0, initialize them to black, traverse the index numbers, and then add corresponding colors according to the label values ​​corresponding to the point cloud grids to generate the data set required for neural network training;

[0069] S4.1. Generate the original image, read the point cloud data, and according to the label value, the pixel value of the signal point position is 1, which is white, and the pixel value of the non-signal point position is 0, which is black. Generate a black-and-white binary image after the point cloud is rasterized in the direction of the distance along the track. The image size dimension is [512, 512, 1], and save the black-and-white binary image in png format;

[0070] S4.2. Generate a mask image. According to the existing label values, label value 1 corresponds to the noise point. Color is added according to the label value. The grid corresponding to the noise point is green, and the RGB value is [0, 255, 0]. Label 2 is a valid signal point, and the corresponding grid position is red, and the RGB value is [255, 0, 0].

[0071] S4.3. Save the color image in png format with image size dimension [512, 512, 3];

[0072] S4.4. In the distance direction along the track, a sliding window is used for cropping to batch generate binary original raster images of [512, 512, 1] and color mask raster images of [512, 512, 3] to construct the data set required for network model training.

[0073] Building a neural network for supervised training includes:

[0074] T1. Divide the data set into a training set and a validation set, with the ratio of 8:2;

[0075] T2. Set the main structure of the neural network model, including three layers;

[0076] T2.1. The first layer is the encoder, which is used to capture the input image features. It consists of convolution blocks and attention mechanism modules. Each convolution block consists of a convolution layer and a pooling layer.

[0077] First, 32 3*3 convolution kernels are used to perform the same convolution twice on the input feature map to adjust the input feature map to 32 channels. The image size is 32*512*512, and the saved output is recorded as A1. The pooling kernel size is 2*2 for maximum pooling to reduce the image size. The output size is 32*256*256, and the saved output is recorded as B1. B1 is saved after the output of the attention mechanism module and recorded as C1.

[0078] Use 64 3*3 convolution kernels to perform the same convolution twice on the input feature map C1 to adjust the input feature map to 64 channels. The image size is 64*256*256, and the saved output is recorded as A2. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 64*128*128, and the saved output is recorded as B2. B2 is saved after the output of the attention mechanism module and recorded as C2;

[0079] Use 128 3*3 convolution kernels twice to perform the same convolution on the feature map C2 to adjust the input feature map to 128 channels. The image size is 128*128*128, and the saved output is recorded as A3. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 128*64*64, and the saved output is recorded as B3. B3 is saved after the output of the attention mechanism module and recorded as C3;

[0080] Use 256 3*3 convolution kernels twice to perform the same convolution on the feature map C3 to adjust the input feature map to 256 channels. The image size is 256*64*64, and the output is saved as A4. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 256*32*32, and the output is saved as B4. B4 is output to C4 through the attention mechanism module;

[0081] Use 512 3*3 convolution kernels to perform two same convolutions on the feature map C4 to adjust the input feature map to 512 channels. The image size is 512*32*32, and the saved image is recorded as A5.

[0082] Building a neural network for supervised training includes:

[0083] T2.2. The second layer is the decoder, which upsamples the feature layer and stacks it for feature fusion. The final feature layer has the same height and width as the input image.

[0084] Perform transposed convolution upsampling on feature map A5, with a transposed convolution kernel size of 2*2, a step size of 2, and a padding mode of "same". A skip connection is constructed between the upsampled output feature map and the A4 feature map. Two same convolutions are performed using 256 3*3 convolution kernels. The output image size is 256*64*64, and the output image is saved and recorded as A6.

[0085] Perform transposed convolution upsampling on feature map A6. The transposed convolution kernel size is 2*2, the step size is 2, and the padding mode is "same". The upsampled output feature map is skip-connected with the A3 feature map. 128 3*3 convolution kernels are used to perform two same convolutions. The output image size is 128*128*128. The output image is saved and recorded as A7.

[0086] Perform transposed convolution upsampling on feature map A7, with a transposed convolution kernel size of 2*2, a step size of 2, and a padding mode of "same". A skip connection is constructed between the upsampled output feature map and the A2 feature map. 64 3*3 convolution kernels are used to perform two same convolutions. The output image size is 64*256*256. The output image is saved and recorded as A8.

[0087] The feature map A8 is upsampled by transposed convolution. The transposed convolution kernel size is 2*2, the step size is 2, and the padding method is "same". The upsampled output feature map is skipped with the A1 feature map. 32 3*3 convolution kernels are used to perform two same convolutions. The output image size is 32*512*512. A residual connection is constructed with the output feature map for joint output. The joint output is saved and recorded as A9.

[0088] Building a neural network for supervised training includes:

[0089] T2.3. The third layer is the prediction classification layer. A9 is convolved with a convolution kernel size of 1*1, the number of channels is changed, and the activation function is connected to convert it into a classification result. Each pixel is classified into three categories: noise points, valid signal points, and background. The corresponding category label values ​​are 1, 2, and 3 respectively.

[0090] Point cloud information recovery includes:

[0091] For a color point cloud raster image, the initial along-track distance corresponding to the generated color point cloud raster image is x1, the final along-track distance is x2, the initial elevation is y1, and the final elevation is y2. It is known that the grid resolution in the along-track distance direction is 0.5m, the grid resolution in the elevation direction is 0.05m, the grid index is (id_x, id_y), and the along-track distance X′ and elevation information Y′ restored at the grid center position are as follows:

[0092] X′=x1+x_resoution×(id_x-1 / 2);

[0093] Y′=y1+y_resoution×(id_y-1 / 2);

[0094] The real distance along the track is restored by the color point cloud raster image, and the elevation information set is {X′, Y′}. The real distance along the track corresponding to the color point cloud raster image and the elevation information set is {X0, Y0}. The two information sets are looped through to calculate the Euclidean distance, and the two data sets are matched using the Euclidean distance. When the Euclidean distance D < < ε, that is:

[0095]

[0096] Complete point cloud information recovery, where ε is the minimum value;

[0097] A track of ICESat-2 point cloud data generates an original raster image set according to the sliding window cropping order. The original raster image generates a corresponding mask raster image set after network model training. When the sliding window cropping step l′ is equal to the window size l, the raster image set is directly reversely restored and spliced ​​according to the cropping order to generate a global mask raster image; when the sliding window cropping step l′ is less than the window size l, each channel of the raster image set is seamlessly spliced ​​using linear edge fusion to generate a global mask raster image, as shown below:

[0098] I(x,y)=α(x,y)*I1(x,y)+(1-α(x,y))*I2(x,y)

[0099] I(x,y) is the pixel value of the fused image. I1(x,y) and I2(x,y) are the pixel values ​​of the two images in the overlapping area. α(x,y) is the weight, ranging from 0 to 1;

[0100] According to the point cloud recovery method of a color point cloud raster image, the point cloud information of the global mask raster image is restored to achieve the classification of noise points and signal points of one track of ICESat-2 point cloud data, and batch classification of multi-track ICESat-2 point cloud data is achieved. The point cloud data that has not been manually annotated is cut through sliding windows, point cloud rasterization, neural network automatic learning, point cloud information recovery and matching, and the point cloud label value is obtained to complete the classification of noise points and signal points.

[0101] In order to further improve the generalization ability of the model, the present invention embeds an attention mechanism module in the corresponding layer of the network model. The attention mechanism module consists of two submodules: a channel attention module and a spatial attention module. They are used to weight the channel dimension and the spatial dimension of the feature map respectively. The feature map is successively input into the channel attention module and the spatial attention module to perform joint modeling of the spatial and channel attention mechanisms to dynamically adjust the feature responses of different positions and channels to enhance the network's attention to important features, thereby improving the representation ability of the model.

[0102] The channel attention module focuses on the correlation between different channels, and extracts the most important feature channels by weighting the channel dimension of the feature map. The channel attention module (CAM) introduces a global average pooling operation to pool the feature responses of each channel, and changes the input feature map F from C*H*W (C is the number of channels, H is the feature map height, and W is the feature map width) to C*1*1. Then, the weights between channels are learned by sharing a multi-layer perceptron to obtain the importance weight of each channel to enhance the response of important channels. Subsequently, a Sigmoid activation function is used to multiply the obtained weights with the original feature map F, changing it back to the size of C*H*W to obtain the feature map F′. As shown in the following formula, where M c ∈R C *1*1 It is a one-dimensional convolution of the channel attention module, and R is a real space set.

[0103]

[0104] The spatial attention module (SAM) focuses on the correlation between different spatial positions, and extracts the most important spatial positions by weighting the spatial dimensions of the feature map. The channel attention module outputs the feature map F′, obtains two 1*H*W feature maps through maximum pooling and average pooling, and then concatenates the two feature maps, converts them into 1-channel feature maps through 7*7 convolution, and obtains the attention weight of each position through the Sigmoid activation function. Finally, the obtained weight is multiplied with the original feature map to enhance the response of important positions, and the output result is converted back to a feature map F″ of size C*H*W. As shown in the following formula, M s ∈R 1*H*W It is a two-dimensional convolution of the spatial attention module, and R is a real space set.

[0105]

[0106] This network model constructs jump connections between different convolutional layers in the decoder stage, allowing the network to transfer information across multiple layers at different levels, so that the network model can better utilize various information of input features, prevent network degradation and retain important information, and improve the network's representation ability. In the A9 feature map output process, residual connections are constructed, direct connections across layers are introduced, and some input information is allowed to bypass some layers in the network, so that the network can learn the identity mapping more easily, can train deep networks more effectively, and solve the gradient disappearance and gradient explosion problems in the deep network training process, as well as accelerate network training and improve model performance.

[0107] The residual connection in the present invention adds the input c of the attention mechanism module to the output H(c) processed by the attention mechanism module to form the output h of the residual block. The mathematical expression is as follows:

[0108] h=H(c)+c.

[0109] The present invention uses ATL03 data as experimental data, and ATL03 is global positioning photon data. ICESat-2 generates 21 along-orbit geophysical products (ATL00-ATL21) divided into 4 levels (0-3 levels). The ATL03 data used in this study is a level 2 product, which records the precise latitude, longitude and elevation information of photons. However, since the photon point cloud is a two-dimensional discrete point, each original photon point carries two characteristic information, distance along the track and elevation, and convolution operations cannot be performed. The export format of the truly annotated point cloud data is csv format, and the elevation information and distance along the track information of the csv format point cloud data are read and rasterized. In order to make each point of the ICESat-2 point cloud data fall within a grid as much as possible, the grid resolution is set to be large enough so that the number of pixels is large enough. The rationality of the grid resolution setting depends on the characteristics of the ICESat-2 point cloud data. According to the set grid resolution, the clipping range along the track distance direction is 256m, and the elevation clipping range is 25.6m. The point cloud data is clipped in the along-track direction to generate an image. The image size meets the input image size requirements.

[0110] ICESat-2 point cloud data contains different terrain conditions, such as plains, mountains, hills, etc. Different terrain features have different elevation ranges. In order to be applicable to a variety of terrains, the present invention creates a histogram in the elevation direction. The creation of the histogram clarifies the concentrated distribution of the point cloud, the initial elevation size and the final elevation size. However, the elevation clipping range is still maintained at 25.6m.

[0111] For point cloud data on flat ground, the elevation range is 0 to 25.6m. The spacing of the y_range matrix is ​​0.05m, and the spacing of the x_range matrix is ​​0.5m. For point cloud data in mountainous areas, the terrain is undulating, and the initial elevation value y1 and the final elevation value y2 are determined according to the elevation histogram. The y_range matrix corresponding to the mountain terrain clipping window is y1, y1+0.05, y1+0.1...y2. Whether it is the original image or the mask image, the pixel values ​​of all grids are set to 0 before processing, initialized to black, and the pixel values ​​are updated according to the label values ​​corresponding to the point cloud grid by traversing the index number, adding different colors, and generating the data set required for neural network training.

[0112] The convolutional layer and fully connected layer operations of this network model are linear multiplication of the weight matrix and its corresponding input features, which expresses a linear mapping. is the input of the previous layer, is the output of this layer, is the weight parameter.

[0113]

[0114] In order to improve the nonlinear modeling ability and convergence speed of the network, the extracted feature map is added with nonlinear factors through the activation function to obtain a new feature map. The activation function uses an unsaturated nonlinear function. This network model uses the ReLU function. The ReLU function keeps the gradient from decaying when the input z is a positive number, and it will be suppressed when it is a negative number. The ReLU function only activates a small number of neurons with positive inputs, effectively alleviating the gradient vanishing problem of the network model during training, and the operation is simple and efficient.

[0115] The ReLU function input is z, and the output is g(z). The expression is as follows:

[0116] g(z)=max(0,z);

[0117] The main goal of the backpropagation algorithm is to calculate the gradient of each weight in the model in order to update the model parameters so that the model can better fit the training data. Specifically, the backpropagation algorithm of this network model needs to first calculate the gradient of the loss function for the output image, then calculate the gradient of each convolutional layer, and update the weight of the convolutional layer according to the gradient descent algorithm.

[0118] In the process of updating parameters, the Adam optimization algorithm is used to search for the optimal solution in the parameter space to minimize the loss function until the preset maximum number of iterations max_epoch is reached to obtain the optimal model. DiceLoss is often used for classification problems. The present invention also uses this loss function, which is expressed as:

[0119]

[0120] DiceLoss is derived from the Dice coefficient, which is a metric function used to evaluate the similarity of two samples. The larger the value, the more similar the two samples are. X represents the pixel label of the real classification image, and Y represents the pixel category of the classification image predicted by the model. |X∩Y| is approximated as the dot product between the pixels of the predicted image and the pixels of the real label image, and the dot product results are added. |X| and |Y| are respectively approximated as the addition of the pixels in their respective corresponding images. It can alleviate the negative impact of the imbalance of foreground and background (area) in the sample. The imbalance of foreground and background means that most areas in the image do not contain the target, and only a small part of the area contains the target. DiceLoss training focuses more on the mining of the foreground area, but there will be a loss saturation problem. Using DiceLoss alone often does not achieve good results.

[0121] The network model used in the present invention combines the DiceLoss loss function and the NLLLoss (negative log-likelihood loss) and adds the two together. The NLLLoss loss function ignores the black background category (ignore_index=3) to achieve weighted processing, further reducing the impact of the background being misjudged as other types and improving the classification accuracy.

[0122] The expression of NLLLoss involves the calculation of logarithms and probabilities. The output of the model is a probability distribution processed by the softmax function, and the target is an index representing the true category. The expression of NLLLoss is:

[0123]

[0124] Where: N is the number of samples, is the true category c in the model's predicted probability distribution for the i-th sample i probability.

[0125] The best trained model is imported into the network, and the divided test set is used as input for testing. According to the trained classification categories, the corresponding color raster images are generated: label 1 represents noise points, which is green, label 2 represents signal points, which is red, and label 3 represents black background.

[0126] The point cloud information restored by a grid corresponds to the real elevation and distance along the track information, and then obtains the label value generated by network training. Then the elevation and distance along the track information corresponding to all grids of the entire point cloud grid image can be restored.

[0127] The performance indicators of precision, recall and F1 value are used for evaluation and analysis.

[0128] ICESat-2 data is greatly affected by background noise, especially solar background noise, and the noise is discretely distributed throughout the study area. The accuracy is evaluated by defining the precision, recall and F1 value evaluation indicators. Let precision be the percentage of the number of correctly judged data to the number of identified data; recall is the percentage of the number of correctly judged data to the actual total number of data. F1 value is a comprehensive evaluation indicator that is designed to balance recall and precision, assuming that both are equally important.

[0129]

[0130] Where TP is the number of photons correctly classified as signal points, FP is the number of photons that are actually noise but are mistakenly classified as signals, and FN is the number of photons that are actually signal points but are mistakenly classified as noise.

[0131] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting photons from a space-borne laser radar, characterized in that: It includes processing global positioning point cloud data, manually labeling point cloud data, obtaining data sources, establishing point cloud rasterized image datasets based on data sources, building neural networks for supervised training, performing point cloud classification, performing point cloud information recovery, and verifying the accuracy of the point cloud after information recovery with the real point cloud label data; According to the data source, the point cloud rasterization image dataset is established, including using the global positioning point cloud data ATL03 as the experimental data for denoising, setting the grid resolution according to the distance information and elevation information along the track, and taking the photon point cloud rasterization image as the input data. The rasterization includes: S1. Set the grid resolution, the point cloud data footprint spacing is 0.7m, set the grid resolution in the along-track direction to 0.5m, set the grid resolution in the elevation direction to 0.05m, according to the set grid resolution, the along-track distance direction clipping range is 256m, the elevation clipping range is 25.6m, and the point cloud data is clipped in the along-track direction to generate images; Rasterization includes: S2. Create a grid and sample according to the grid resolution: y_range=y_min:y_resolution:y_max; x_range=x_min:x_resolution:x_max; Where y_range and x_range are one-dimensional matrices generated by elevation and distance along the track according to the grid resolution, y_min is the initial value of the elevation of the clipping window, y_max is the end value of the elevation of the clipping window, x_min is the initial value of the distance along the track of the clipping window, x_max is the end value of the distance along the track of the clipping window, x_resolution is the resolution of the distance along the track, and y_resolution is the elevation resolution; Create two two-dimensional matrices x_grid and y_grid. x_grid and y_grid are obtained according to x_range and y_range respectively. Each row in x_grid corresponds to the distance value along the grid track, and each column in y_grid corresponds to the value of the elevation information along the grid. Rasterization includes: S3. Build a KD tree index, determine whether the grid point is near each data point, and select the point cloud data within the cropping range; Generate the KD tree matrix kd_tree_x of the along-track distance according to the row transposition of x_grid, and generate the KD tree matrix kd_tree_y of the elevation according to y_grid. The dimensions of the two KD tree matrices are both 512×1, containing grid coordinate information. Perform k nearest neighbor search, match each grid with the along-track distance information b(j, 1) and elevation information b(j, 2) that actually correspond to each data point in the cropping window, find the adjacent data points of the grid, and obtain the corresponding grid index numbers id_x and id_y, where j represents the jth data point and b is the matrix storing the along-track distance information and elevation information. Rasterization includes: S4. Generate a data set in batches according to the classification labels, set the pixel values ​​of all grids to 0, initialize them to black, traverse the index numbers, and then add corresponding colors according to the label values ​​corresponding to the point cloud grids to generate the data set required for neural network training; S4.

1. Generate the original image, read the point cloud data, and according to the label value, the pixel value of the signal point position is 1, which is white, and the pixel value of the non-signal point position is 0, which is black. Generate a black-and-white binary image after the point cloud is rasterized in the direction of the distance along the track. The image size dimension is [512, 512, 1], and save the black-and-white binary image in png format; S4.

2. Generate a mask image. According to the existing label values, label value 1 corresponds to the noise point. Color is added according to the label value. The grid corresponding to the noise point is green, and the RGB value is [0, 255, 0]. Label 2 is a valid signal point, and the corresponding grid position is red, and the RGB value is [255, 0, 0]. S4.

3. Save the color image in png format with image size dimension [512, 512, 3]; S4.

4. In the distance direction along the track, use the sliding window to perform cropping, batch generate the binary original raster images of [512, 512, 1] and batch generate the color mask raster images of [512, 512, 3], and construct the data set required for network model training; Building a neural network for supervised training includes: T1. Divide the data set into a training set and a validation set, with the ratio of 8:2; T2. Set the main structure of the neural network model, including three layers; T2.

1. The first layer is the encoder, which is used to capture the input image features. It consists of convolution blocks and attention mechanism modules. Each convolution block consists of a convolution layer and a pooling layer. First, 32 3*3 convolution kernels are used to perform the same convolution twice on the input feature map to adjust the input feature map to 32 channels. The image size is 32*512*512, and the saved output is recorded as A1. The pooling kernel size is 2*2 for maximum pooling to reduce the image size. The output size is 32*256*256, and the saved output is recorded as B1. B1 is saved after the output of the attention mechanism module and recorded as C1. Use 64 3*3 convolution kernels to perform the same convolution twice on the input feature map C1 to adjust the input feature map to 64 channels. The image size is 64*256*256, and the saved output is recorded as A2. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 64*128*128, and the saved output is recorded as B2. B2 is saved after the output of the attention mechanism module and recorded as C2; Use 128 3*3 convolution kernels twice to perform the same convolution on the feature map C2 to adjust the input feature map to 128 channels. The image size is 128*128*128, and the saved output is recorded as A3. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 128*64*64, and the saved output is recorded as B3. B3 is saved after the output of the attention mechanism module and recorded as C3; Use 256 3*3 convolution kernels twice to perform the same convolution on the feature map C3 to adjust the input feature map to 256 channels. The image size is 256*64*64, and the output is saved as A4. Use a pooling kernel size of 2*2 for maximum pooling to reduce the image size. The output size is 256*32*32, and the output is saved as B4. B4 is output to C4 through the attention mechanism module; Use 512 3*3 convolution kernels to perform two same convolutions on feature map C4 to adjust the input feature map to 512 channels. The image size is 512*32*32, and the saved image is recorded as A5; Building a neural network for supervised training includes: T2.

2. The second layer is the decoder, which upsamples the feature layer and stacks it for feature fusion. The final feature layer has the same height and width as the input image. Perform transposed convolution upsampling on feature map A5, with a transposed convolution kernel size of 2*2, a step size of 2, and a padding mode of "same". A skip connection is constructed between the upsampled output feature map and the A4 feature map. Two same convolutions are performed using 256 3*3 convolution kernels. The output image size is 256*64*64, and the output image is saved and recorded as A6. Perform transposed convolution upsampling on feature map A6. The transposed convolution kernel size is 2*2, the step size is 2, and the padding mode is "same". The upsampled output feature map is skip-connected with the A3 feature map. 128 3*3 convolution kernels are used to perform two same convolutions. The output image size is 128*128*128. The output image is saved and recorded as A7. Perform transposed convolution upsampling on feature map A7, with a transposed convolution kernel size of 2*2, a step size of 2, and a padding mode of "same". A skip connection is constructed between the upsampled output feature map and the A2 feature map. 64 3*3 convolution kernels are used to perform two same convolutions. The output image size is 64*256*256. The output image is saved and recorded as A8. Perform transposed convolution upsampling on feature map A8, with a transposed convolution kernel size of 2*2, a stride of 2, and a padding mode of "same". A skip connection is constructed between the upsampled output feature map and the A1 feature map. 32 3*3 convolution kernels are used to perform two same convolutions. The output image size is 32*512*512. A residual connection is constructed with the output feature map for joint output, and the joint output is saved and recorded as A9. Building a neural network for supervised training includes: T2.

3. The third layer is the prediction classification layer. A9 is convolved with a convolution kernel size of 1*1, the number of channels is changed, and the activation function is connected to convert it into a classification result. Each pixel is classified into three categories: noise points, valid signal points, and background. The corresponding category label values ​​are 1, 2, and 3 respectively.

2. The photon extraction method of the space-borne laser radar according to claim 1, characterized in that: Manually labeling point cloud data includes: label 1 represents noise points, and label 2 represents signal points.

3. The photon extraction method of the space-borne laser radar according to claim 2, characterized in that: Point cloud information recovery includes: For a color point cloud raster image, the initial along-track distance corresponding to the generated color point cloud raster image is x1, the final along-track distance is x2, the initial elevation is y1, and the final elevation is y2. It is known that the grid resolution in the along-track distance direction is 0.5m, the grid resolution in the elevation direction is 0.05m, the grid index is (id_x, id_y), and the along-track distance X restored by the grid center position is ′ , elevation information Y ′ as follows: X ′ =x1+x_resloution×(id_x-1 / 2); Y ′ =y1+y_resloution×(id_y-1 / 2); The real distance along the track is restored from the color point cloud raster image, and the elevation information set is {X ′ , Y ′ }, the actual distance along the track corresponding to the color point cloud grid image, the elevation information set is {X0, Y0}, the two information sets are looped through to calculate the Euclidean distance, and the two data sets are matched using the Euclidean distance. When the Euclidean distance D < < ε, that is Complete point cloud information recovery, where ε is the minimum value; The unlabeled point cloud data is cropped through sliding windows, point cloud rasterization, neural network automatic learning, and point cloud information recovery and matching to obtain point cloud label values ​​and complete the classification of noise points and signal points.

Citation Information

Patent Citations

  • Full-waveform laser radar data de-noising method based on wavelet packet analysis and neural network

    CN105510900A

  • Photon counting laser radar data denoising method

    CN113281716A

  • Satellite-borne photon counting laser radar data denoising and filtering method and device

    CN115390047A

  • LiDAR point cloud filtering method integrated with attention mechanism multi-scale CNN

    CN115512226A

  • Three-dimensional laser radar point cloud semantic segmentation method and device based on deep learning

    CN116229057A