Laser radar data processing method and system

Abstract

The invention relates to the technical field of computer vision, and particularly discloses a laser radar data processing method and system, and the method employs bilateral filtering and Gaussian filtering to provide a more accurate and stable basis for multi-peak extraction and feature extraction. By adopting multi-peak extraction, the robustness of the signal is effectively improved, and noise interference is reduced; using 3D CNN to extract local space features, and capturing detail and local structure information; a Transform network is adopted to extract global features, and a long-distance dependency relationship is captured; the peak values extracted by the neural network are classified and analyzed by adopting multi-peak analysis, and the data quality is further optimized; the features output by the neural network and the features of multi-peak analysis are fused, so that the complementation of local and global features is realized, and the robustness and accuracy of feature extraction are improved; median filtering processing is carried out on the fusion features, isolated noise points are further removed, and the data quality is improved; and reconstructing the feature data after median filtering to generate optimized histogram data consistent with the dimension of the original histogram data.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a laser radar data processing method and system. Background Art

[0002] With the rapid development of technologies such as autonomous driving and robot navigation, solid-state LiDAR, as an important sensor in these fields, has been widely used due to its advantages of no mechanical parts, fast response speed, compactness and low cost. Solid-state LiDAR generates accurate three-dimensional point cloud data by emitting laser pulses and receiving their reflected signals. However, during the data acquisition process, solid-state LiDAR is susceptible to noise and ambient light interference, resulting in a decrease in data quality, which in turn affects subsequent processing and application effects. Therefore, how to effectively filter and optimize solid-state LiDAR data has become a research focus.

[0003] Traditional LiDAR data processing methods mainly rely on Gaussian filtering, median filtering and other techniques to reduce noise and optimize data. Although these methods can smooth data and reduce noise to a certain extent, they have obvious disadvantages in complex environments. First, these methods are sensitive to noise. When processing high-noise data, useful features are often weakened, especially at edges and details, resulting in the loss of important information. Secondly, traditional filtering methods are prone to excessive smoothing of data, blurring sharp features and edges, making it difficult to retain fine structures, and affecting subsequent target detection and classification accuracy. In addition, traditional methods are sensitive to parameter selection. Different filtering parameters need to be adjusted for different environments and application scenarios, which increases the complexity of practical applications. Finally, in complex scenarios, traditional filtering methods have high computational complexity and are difficult to meet the requirements of real-time processing, which is a significant limitation for applications that require fast response, such as autonomous driving and robot navigation. Summary of the invention

[0004] The present invention provides a laser radar data processing method and system, which solves the technical problem of how to effectively remove noise in solid-state laser radar data in a complex environment while retaining useful features, thereby improving the quality and reliability of solid-state laser radar data to meet the needs of fields such as autonomous driving and robot navigation.

[0005] In order to solve the above technical problems, the present invention provides a laser radar data processing method, comprising the steps of:

[0006] S1, using Gaussian filtering and bilateral filtering to perform noise reduction on the original histogram data of the laser radar;

[0007] S2. By using multi-peak extraction technology, the histogram data after Gaussian filtering and bilateral filtering are divided into multiple adjacent regions, and the local maximum value and its corresponding bin number are extracted in each region to obtain two corresponding sets of feature group data;

[0008] S3, the two sets of feature group data are dimensionally upgraded and respectively input into the 3D convolutional neural network, i.e., 3D CNN and Transformer network, to extract local features and global features;

[0009] S4. Perform multi-peak analysis on the feature vectors extracted by 3D CNN and Transformer networks through multi-peak analysis technology, extract four types of features including peak classification probability, flight time value, distance, and probability density function, and obtain the corresponding multi-peak analysis feature vectors;

[0010] S5, fusing the feature vectors extracted by the 3D CNN and Transformer networks with the multi-peak analysis feature vectors to obtain fused features;

[0011] S6, performing median filtering on the fused features;

[0012] S7. Reconstruct the histogram data of the filtered fusion features to generate optimized histogram data with the same dimension as the original histogram data.

[0013] Furthermore, the step S5 specifically includes the steps of:

[0014] S51, performing feature cascading on the feature vectors extracted by the 3D CNN and Transformer networks to obtain cascade features;

[0015] S52, performing weighted summation on the two groups of multi-peak analysis feature vectors to obtain multi-peak analysis fusion features;

[0016] S53, cascading the cascade features and the multi-peak analysis fusion features to obtain the final fusion features.

[0017] Furthermore, the 3D CNN includes an extended dimension layer, a first convolution layer, a first three-dimensional maximum pooling layer, a second convolution layer, a second three-dimensional maximum pooling layer, a first flattening layer and a first fully connected layer, which are arranged in sequence from input to output; the extended dimension layer is used to increase the dimension of the feature group data corresponding to the Gaussian filter to obtain a three-dimensional array adapted to the 3D CNN input; the first convolution layer is used to extract local spatial features; the first three-dimensional maximum pooling layer is used to perform maximum pooling to reduce the data dimension; the second convolution layer is used to further extract local spatial features; the second three-dimensional maximum pooling layer is used to perform maximum pooling to further reduce the data dimension; the first flattening layer is used to flatten the three-dimensional features into a one-dimensional feature vector using a flattening operation; the first fully connected layer is used to convert the flattened feature vector into an output vector.

[0018] Furthermore, the Transformer network includes a sequentially connected encoder module, a decoder module, a second flattening layer, and a second fully connected layer. The encoder module includes a plurality of sequentially connected encoders, and the decoder module includes a plurality of sequentially connected decoders, the output of the previous decoder is the input of the next decoder, and the output of the last encoder is input to each decoder.

[0019] Furthermore, each encoder from input to output includes:

[0020] The encoder multi-head attention module is used to take the input data as the query matrix Q, the key matrix K and the value matrix V, and perform weighted summation according to these weight matrices to obtain a new representation;

[0021] The encoder first residual connection and layer normalization module is used to add the output of the encoder multi-head attention module to the original input, followed by layer normalization;

[0022] Encoder feed-forward neural network for nonlinear mapping via two linear transformations and a ReLU activation function;

[0023] The encoder second residual connection and layer normalization module is used to add the output of the encoder feed-forward neural network to its input, followed by layer normalization.

[0024] Furthermore, each decoder from input to output includes:

[0025] The masked multi-head self-attention module is the same as the encoder multi-head attention module, but with the addition of a mask operation;

[0026] The decoder's first residual connection and layer normalization module is used to add the output of the masked multi-head attention module to the original input, followed by layer normalization;

[0027] Decoder multi-head attention module: used to generate new representations through the multi-head attention mechanism using the output of the last encoder as the key and value and the input of the decoder as the query;

[0028] The decoder second residual connection and layer normalization module is used to add the output of the decoder multi-head attention module to the original input and then perform layer normalization;

[0029] The decoder feedforward neural network is used to perform further nonlinear transformation on the features at each position;

[0030] The decoder third residual connection and layer normalization module is used to add the output of the decoder feedforward neural network to its input, followed by layer normalization.

[0031] Furthermore, in step S1, the formula for performing a convolution operation on the three-dimensional data data (x, y, z) using Gaussian filtering is:

[0032]

[0033] Wherein, k is the radius of the Gaussian filter window, G(i,j,l) represents the Gaussian filter function, (i,j,l) represents the coordinates within the Gaussian filter window, and G_data(x,y,z) represents the three-dimensional data obtained after Gaussian filtering of the three-dimensional data data(x,y,z);

[0034] The Gaussian filter function G(i,j,l) is expressed as follows:

[0035]

[0036] Among them, σ represents the standard deviation σ set in Gaussian filtering, and e is the natural base;

[0037] The three-dimensional data data(x,y,z) is processed using bilateral filtering to obtain smoothed data B_data(x,y,z):

[0038]

[0039] G r (data(x,y,z),data(x+o,y+p,z+q))

[0040] Where (o, p, q) represents the coordinates in the spatial domain Gaussian filter window, s is the radius of the spatial domain Gaussian filter window; W p is a weight normalization term, which is used to ensure that the pixel value after filtering is within a reasonable range. p Specifically:

[0041]

[0042] G s (o, p, q) represents the spatial domain Gaussian filter function, expressed as:

[0043]

[0044] Among them, σ s is the standard deviation of the Gaussian filter function in the spatial domain;

[0045] G r (data(x,y,z),data(x+o,y+p,z+q)) represents the grayscale domain Gaussian filter function, which is expressed as:

[0046]

[0047] Among them, σ r is the standard deviation of the gray-domain Gaussian filter function;

[0048] The original histogram data is converted into an array of dimension size (H, W, D) and recorded as three-dimensional data data(x, y, z), where x, y, z are the x-axis, y-axis and z-axis coordinates of any point in the original histogram data, x, y, z ≥ 0, and H, W, D are the x-axis range, y-axis range and z-axis range corresponding to the original histogram data.

[0049] Furthermore, the step S2 comprises the steps of:

[0050] S21, divide the Gaussian filtered data G_data(x, y, z) and the bilateral filtered data B_data(x, y, z) into N adjacent regions, each with a width of W r The number of regions N is determined according to the size of the histogram data: R m is the measurement range of the histogram data, then the i-th region G_region of the Gaussian filtered data G_data(x,y,z) i and the ith region B_region of the bilaterally filtered data B_data(x,y,z) i It can be expressed as:

[0051] G_region i ={G_data(x, y, z)|i·W r ≤x<(i+1)·W r}

[0052] B_region i = {B_data(x,y,z)|i·W r ≤x<(i+1)·Wr}

[0053] S22. Extract the local maximum value in each region and its corresponding bin number bn i , forming a feature group Recorded as The extraction formula is as follows:

[0054]

[0055] bn i = argmax(region i )

[0056] region i Refers to G_region i or B_region i.

[0057] Furthermore, the step S4 specifically includes the steps of:

[0058] S41. Use a fully connected feedforward neural network to calculate the output activation value FNN(M) of the output of the 3D CNN and Transformer networks:

[0059] FNN(M)=ReLU(MW1+b1)W2+b2

[0060] Where W1 and W2 are weight matrices of the fully connected layer, b1 and b2 are bias terms, M refers to the output of the 3D CNN or Transformer network, FNN(M) is the output of the fully connected feedforward neural network, and ReLU() is the ReLU activation function;

[0061] S42. Use the softmax function to convert the activation value FNN(M) into the corresponding classification probability Class(M):

[0062] Class(M)=softmax(FNN(M)W c +b c )

[0063] Among them, W c is the weight matrix of the classification layer; b c is the bias term of the classification layer;

[0064] S43, calculate the corresponding flight time value T according to the classification probability Class (M) TOF :

[0065] T TOF =Class(M)W t +b t

[0066] Among them, W t is the weight matrix of the TOF recovery layer; b t is the bias term of the TOF recovery layer;

[0067] S44. Calculate the actual distance D using the speed of light c and the time of flight TOF (M) TOF :

[0068]

[0069] S45, using the generated flight time T TOF Calculate the probability density function F PDF :

[0070]

[0071] Among them, rB and r LB is the decay constant at different time periods, r L is the photon arrival rate of the laser pulse, T p is the pulse width, t represents the current moment;

[0072] S46. The four types of features, namely, classification probability, flight time, distance and probability density function, corresponding to the calculated output features of the 3D CNN and Transformer networks respectively constitute the corresponding multi-peak analysis feature vectors output_3d_mpe and output_t_mpe.

[0073] The present invention also provides a laser radar data processing system, the key of which is that it includes a data preprocessing unit, a multi-peak extraction unit, a neural network feature extraction unit, a multi-peak analysis unit, a feature fusion unit, a median filtering unit and a data reconstruction unit, which are respectively used to execute steps S1 to S7 in the above-mentioned laser radar data processing method.

[0074] The laser radar data processing method and system provided by the present invention adopt bilateral filtering and Gaussian filtering to provide a more accurate and stable basis for multi-peak extraction and feature extraction; adopt multi-peak extraction to effectively improve the robustness of the signal and reduce noise interference; adopt 3D CNN to extract local spatial features and capture details and local structural information in histogram data; adopt Transformer network to extract global features and capture long-distance dependencies in data; adopt multi-peak analysis to classify and analyze the peaks extracted by the neural network to further optimize the data quality; fuse the features output by the neural network and the features of multi-peak analysis to achieve the complementarity of local and global features and improve the robustness and accuracy of feature extraction; use median filtering to process the results after feature fusion to further remove isolated noise points and improve data quality; reconstruct the feature data after median filtering to generate optimized histogram data consistent with the dimension of the original histogram data, so that it can be directly applied to subsequent analysis and processing. The method and system of the present invention can effectively remove noise in the original histogram data in a complex environment, retain useful features, and significantly improve the quality and reliability of solid-state laser radar histogram data, and are suitable for fields such as autonomous driving and robot navigation that require high-precision data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 is a flowchart of a laser radar data processing method provided by an embodiment of the present invention;

[0076] Figure 2 is a structural diagram of a 3D CNN provided by an embodiment of the present invention;

[0077] Figure 3is a structural diagram of a Transformer network provided by an embodiment of the present invention;

[0078] Figure 4 It is a detailed structural diagram of the Transformer network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0079] The following specifically illustrates the implementation mode of the present invention in conjunction with the accompanying drawings. The embodiments are provided for illustrative purposes only and cannot be understood as limiting the present invention. The accompanying drawings are provided for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0080] With the development of deep learning technology, methods using neural networks for data processing and optimization have gradually gained attention. 3D convolutional neural networks (3D CNNs) excel in local feature extraction, while Transformer networks have significant advantages in global feature capture. In addition, multi-peak extraction (MPE) and multi-peak analysis (MPA) technologies have shown significant advantages in solid-state lidar data processing. Multi-peak extraction technology improves the robustness of the signal by extracting local maxima from the histogram, while multi-peak analysis technology classifies and analyzes the extracted peaks through neural networks to further optimize data quality.

[0081] At present, this embodiment effectively combines multi-peak extraction and deep learning technology to efficiently filter and optimize solid-state lidar histogram data to overcome the shortcomings of traditional methods, effectively remove noise, retain useful features, and improve the real-time and robustness of data processing, improve data quality and reliability, and meet the needs of fields such as autonomous driving and robot navigation for high-precision lidar data.

[0082] Specifically, the embodiment of the present invention first provides a laser radar data processing method, such as Figure 1 As shown in the flowchart, the steps include:

[0083] S1. Data preprocessing: Use Gaussian filtering and bilateral filtering to perform noise reduction on the original histogram data of the lidar;

[0084] S2, multi-peak extraction: through the multi-peak extraction technology, the histogram data after Gaussian filtering and bilateral filtering are divided into multiple adjacent regions, and the local maximum value and its corresponding bin number are extracted in each region to obtain the corresponding two sets of feature group data;

[0085] S3, neural network feature extraction: the two sets of feature group data are dimensionally upgraded and input into the 3D convolutional neural network (3D CNN) and the Transformer network respectively. The 3D convolutional neural network (3D CNN) extracts local features, and the Transformer network extracts global features.

[0086] S4. Multi-peak analysis: Through multi-peak analysis technology, the feature vectors extracted by 3D CNN and Transformer network are analyzed multi-peak, and four types of features, namely peak classification probability, flight time (TOF) value, distance, and probability density function (PDF), are extracted to obtain the corresponding multi-peak analysis feature vectors.

[0087] S5, feature fusion: the feature vectors extracted by 3D CNN and Transformer network are fused with the feature vectors of multi-peak analysis to obtain fused features;

[0088] S6, median filtering: median filtering is performed on the fused features;

[0089] S7, data reconstruction: reconstruct the histogram data of the filtered fusion features to generate optimized histogram data with the same dimension as the original histogram data.

[0090] Step S1 uses bilateral filtering to combine information in the spatial domain and the grayscale domain, which can retain edge details while smoothing the data. The filtered data has better edge retention characteristics, further reduces noise interference, and improves data quality. The data smoothed by Gaussian filtering and the data processed by bilateral filtering are used as input data for the subsequent multi-peak extraction step, providing a more accurate and stable basis for multi-peak extraction and feature extraction. Step S2 uses multi-peak extraction to effectively improve the robustness of the signal, reduce noise interference, and ensure that the extracted peaks can reflect the real object information. Step S3 performs neural network feature extraction. 3D CNN extracts local spatial features through multiple convolutional layers and pooling layers, which is suitable for capturing details and local structural information in histogram data. 3D CNN is good at extracting local features and can effectively identify detail changes in histogram data; Transformer network performs well in extracting global features and can capture long-distance dependencies in data; Step S4 performs multi-peak analysis to classify and analyze the peaks extracted by the neural network to further optimize data quality; Step S5 performs feature fusion. The combination of the two realizes the complementarity of local and global features and improves the robustness and accuracy of feature extraction; Step S6 uses median filtering to process the results after feature fusion to further remove isolated noise points and improve data quality; Step S7 reconstructs the feature data after median filtering to generate optimized histogram data with the same dimension as the original histogram data, so that it can be directly applied to subsequent analysis and processing.

[0091] The following is a detailed description of each step.

[0092] (1) Step S1: Data preprocessing

[0093] Step S1 includes Gaussian filtering and bilateral filtering of the original three-dimensional histogram data. Before Gaussian filtering and bilateral filtering, the original histogram data H(x, y, z) of the laser radar is read first, and then converted into an array of dimension size (H, W, D) recorded as data(x, y, z), where x, y, z are the x-axis, y-axis and z-axis coordinates of any point in the original histogram data, x, y, z ≥ 0, and H, W, D are the x-axis range, y-axis range and z-axis range corresponding to the original histogram data. As a preferred example, the dimension of data(x, y, z) in this example is (150, 360, 288).

[0094] The Gaussian filtering process is as follows:

[0095] Use Gaussian filtering to perform convolution operation on three-dimensional data (x, y, z):

[0096]

[0097] Among them, k is the radius of the Gaussian filter window, G(i,j,l) represents the Gaussian filter function, (i,j,l) represents the coordinates within the Gaussian filter window, and G_data(x,y,z) represents the three-dimensional data obtained after Gaussian filtering of the three-dimensional data data(x,y,z).

[0098] The Gaussian filter function G(i,j,l) is expressed as follows:

[0099]

[0100] Among them, σ represents the standard deviation σ set in the Gaussian filter, and e is the natural base.

[0101] As a preferred example, in this example, σ is set to 1.2, and k is set to 3σ (ie, 3.6).

[0102] The bilateral filtering process is as follows:

[0103] The three-dimensional data data(x,y,z) is processed using bilateral filtering to obtain smoothed data B_data(x,y,z):

[0104]

[0105] G r (data(x,y,z),data(x+o,y+p,z+q))

[0106] (o, p, q) represents the coordinates in the spatial domain Gaussian filter window, s is the radius of the spatial domain Gaussian filter window, G s (o, p, q) represents the spatial domain Gaussian filter function, expressed as:

[0107]

[0108] Among them, σ s is the standard deviation of the Gaussian filter function in the spatial domain;

[0109] G r (data(x,y,z),data(x+o,y+p,z+q)) represents the grayscale domain Gaussian filter function, which is expressed as:

[0110]

[0111] Among them, σ r is the standard deviation of the gray-domain Gaussian filter function.

[0112] W p is a weight normalization term, which is used to ensure that the pixel value after filtering is within a reasonable range. p Specifically:

[0113]

[0114] As a preferred example, σ s Set to 3.0, σ r is set to 0.5, and s is set to 3σ s (i.e. 9.0).

[0115] Bilateral filtering combines information in the spatial domain and the grayscale domain, and can preserve edge details while smoothing the data. The filtered data B_data(x,y,z) has better edge preservation characteristics, further reduces noise interference, and improves data quality.

[0116] The data G_data(x,y,z) after Gaussian filtering and smoothing and the data B_data(x,y,z) after bilateral filtering will be used as input data for the subsequent multi-peak extraction step, providing a more accurate and stable basis for multi-peak extraction and feature extraction.

[0117] (2) Step S2: Multi-peak extraction

[0118] Step S2 is specifically as follows:

[0119] S2. Through the Multi-Peak Extraction (MPE) technology, the data G_data(x, y, z) after Gaussian filtering and the data B_data(x, y, z) after bilateral filtering are processed by multi-peak extraction.

[0120] The process of multi-peak extraction is: divide G_data(x,y,z) and B_data(x,y,z) into multiple adjacent regions, and extract the local maximum value in each region and its corresponding bin number bn i , forming a feature group Where i∈[1,N], N is the number of regions.

[0121] More specifically, step S2 includes the steps of:

[0122] S21, partition: divide the Gaussian filtered data G_data(x, y, z) and the bilateral filtered data B_data(x, y, z) into N adjacent regions, each with a width of W r The number of regions N is determined according to the size of the histogram data: R m is the measurement range of the histogram data, then the i-th region G_region of the Gaussian filtered data G_data(x,y,z) i and the ith region B_region of the bilaterally filtered data B_data(x,y,z) i It can be expressed as:

[0123] G_region i ={G_data(x,y,z)|i·W r ≤x<(i+1)·W r}

[0124] B_region i = {B_data(x, y, z)|i·W r ≤x<(i+1)·Wr}

[0125] S22, local maximum and bin number extraction: In each region, extract the local maximum and its corresponding bin number bn i , forming a feature group Recorded as The extraction formula is as follows:

[0126]

[0127] bn i = argmax(regioni )

[0128] region i Refers to G_region i or B_region i Finally, the feature group corresponding to G_data(x,y,z) is recorded as The feature group corresponding to B_data(x,y,z) is recorded as

[0129] Step S2 uses multi-peak extraction to effectively improve the robustness of the signal, reduce noise interference, and ensure that the extracted peaks can reflect the real object information.

[0130] (3) Step S3: Neural Network Feature Extraction

[0131] Step S3 specifically includes using a 3D convolutional neural network (3D CNN) to perform feature group data Perform feature extraction and use Transformer network to Perform feature extraction.

[0132] The structure of 3D CNN is as follows Figure 2 As shown, it includes an extended dimension layer, a first convolution layer, a first 3D maximum pooling layer, a second convolution layer, a second 3D maximum pooling layer, a first flattening layer and a first fully connected layer arranged in sequence from input to output. The extended dimension layer is used to The dimensionality is increased to obtain a three-dimensional array input_3d(x,y,z) adapted for 3D CNN input. The first convolutional layer is used to extract local spatial features using 32 3×3×3 convolution kernels. The first three-dimensional max pooling layer is used to use a 2×2×2 max pooling layer for max pooling to reduce the data dimension and retain the main features. The second convolutional layer is used to further extract local spatial features using 32 3×3×3 convolution kernels. The second three-dimensional max pooling layer is used to further reduce the data dimension using a 2×2×2 max pooling layer and retain the main features. The first flattening layer is used to flatten the three-dimensional features into a one-dimensional feature vector using a flattening operation. The first fully connected layer (using the activation function ReLU) is used to convert the flattened feature vector into a 512-dimensional output vector.

[0133] Among them, the extended dimension layer groups the feature data Input the extended dimension layer to increase the dimension and obtain the three-dimensional array input_3d(x,y,z) suitable for 3D CNN input:

[0134]

[0135] The original feature group size is (N, 2), and after dimensionality increase, it forms a three-dimensional array input_3d(x, y, z) of size (N, H, W, D).

[0136] like Figure 3 As shown in the figure, the Transformer network includes a sequentially connected encoder module, a decoder module, a second flattening layer, and a second fully connected layer. The encoder module includes multiple sequentially connected encoders, and the decoder module includes multiple sequentially connected decoders. The output of the previous decoder is the input of the next decoder, and the output of the last encoder is input to each decoder.

[0137] Feature group data after multi-peak extraction It is already a feature vector of fixed size and can be directly input into the Transformer network, denoted as input_t.

[0138] The detailed structure of the Transformer network is as follows Figure 4 As shown, each encoder from input to output includes:

[0139] 1. Encoder multi-head attention module: captures long-distance dependencies in the data, takes the input data input_t as the query matrix Q, key matrix K and value matrix V, and performs weighted summation according to these weight matrices to obtain a new representation;

[0140] 2. Encoder first residual connection and layer normalization module: add the output of the encoder multi-head attention module to the original input, and then perform layer normalization to keep the model's gradient stable;

[0141] 3. Encoder feedforward neural network (FNN): nonlinear mapping through two linear transformations and a ReLU activation function;

[0142] 4. Encoder second residual connection and layer normalization module: The output of the encoder feedforward neural network is added to its input, followed by layer normalization.

[0143] Each decoder from input to output includes:

[0144] 1. Masked multi-head self-attention module: In the process of generating the output sequence, only the current time step and the previous time step are considered to avoid future information leakage. It is similar to the multi-head attention module of the encoder, but with the addition of masking operation;

[0145] 2. Decoder first residual connection and layer normalization module: add the output of the masked multi-head attention module to the original input, and then perform layer normalization to keep the model's gradient stable;

[0146] 3. Decoder multi-head attention module: Use the output of the last encoder as the key and value, and the input of the decoder as the query, and generate new representations through the multi-head attention mechanism; combine the output of the last encoder with the current input of the decoder to capture the dependency between input and output;

[0147] 4. Decoder second residual connection and layer normalization module: add the output of the decoder multi-head attention module to the original input, and then perform layer normalization to keep the model's gradient stable;

[0148] 5. Decoder feedforward neural network: further nonlinear transformation of the features at each position.

[0149] 6. Decoder third residual connection and layer normalization module: The output of the decoder feedforward neural network is added to its input, followed by layer normalization.

[0150] As a preferred example, six encoders and six decoders are provided.

[0151] Among them, the specific operations of residual connection and layer normalization in encoder and decoder are:

[0152] Residual connection: retains the information of input X and adds the output of the multi-head attention mechanism to it. The formula is:

[0153] X res =X+MultiHead(Q,K,V)

[0154] Among them, X res is the output of the residual connection, and MultiHead(Q,K,V) is the multi-head attention output of the previous module.

[0155] Layer normalization: Normalize the results of the residual connection to ensure that the model remains stable during training. The formula is:

[0156] X′=LayerNorm(X res )

[0157]

[0158] Where LayerNorm() represents the layer normalization operation, X′ represents the output of layer normalization, and μ is X res The mean of Yes X res The variance of ∈ is a small constant to prevent the denominator from being zero, set to 10 -6 ; γ and β are learnable parameters used to rescale and offset the normalized output.

[0159] Finally, the output of the Transformer network is converted into a one-dimensional feature vector through the second flattening layer, and then the second fully connected layer is used to convert the flattened feature vector into a 512-dimensional output vector.

[0160] The Transformer network uses multiple encoders, multiple decoders, flattening layers, and extracts global features through multi-head self-attention mechanism, feedforward neural network, residual connection, and layer normalization.

[0161] Step S3 performs neural network feature extraction. 3D CNN extracts local spatial features through multiple convolutional layers and pooling layers, which is suitable for capturing details and local structural information in histogram data. 3D CNN is good at extracting local features and can effectively identify detail changes in histogram data. Transformer network performs well in extracting global features and can capture long-distance dependencies in data.

[0162] (4) Step S4: Multi-peak analysis

[0163] Step S4 specifically includes the following steps:

[0164] S41. Use a fully connected feedforward neural network (FNN) to calculate the output activation value FNN(M) of the output of the 3D CNN and Transformer networks:

[0165] FNN(M)=ReLU(MW1+b1)W2+b2

[0166] Among them, W1 and W2 are the weight matrices of the fully connected layer, b1 and b2 are bias terms, M refers to the output of the 3D CNN or Transformer network, FNN(M) is the output of the fully connected feedforward neural network (FNN), and ReLU() is the ReLU activation function.

[0167] S42. Use the softmax function to convert the activation value FNN(M) into the corresponding classification probability Class(M):

[0168] Class(M)=softmax(FNN(M)W c +b c )

[0169] Among them, W c is the weight matrix of the classification layer; b c is the bias term of the classification layer.

[0170] S43, calculate the corresponding time of flight (TOF) value T according to the classification probability Class (M) TOF :

[0171] T TOF=Class(M)W t +b t

[0172] Among them, W t is the weight matrix of the TOF recovery layer; b t is the bias term of the TOF recovery layer.

[0173] S44. Calculate the actual distance D using the speed of light c and the time of flight TOF (M) TOF :

[0174]

[0175] S45, using the generated flight time T TOF Calculate the probability density function F PDF :

[0176]

[0177] Among them, r B and r LB is the decay constant at different time periods, r L is the photon arrival rate of the laser pulse, T p is the pulse width, and t represents the current moment.

[0178] S46. The four types of features, namely, classification probability, flight time, distance and probability density function, corresponding to the calculated output features of the 3D CNN and Transformer networks respectively constitute the corresponding multi-peak analysis feature vectors output_3d_mpe and output_t_mpe.

[0179] Step S4 performs multi-peak analysis to classify and analyze the peak values ​​extracted by the neural network to further optimize the data quality.

[0180] (5) Step S5: Feature Fusion

[0181] Step S5 specifically includes the following steps:

[0182] S51. Perform feature concatenation on the feature vectors output_3d and output_t extracted by the 3D CNN and Transformer networks to obtain the concatenated feature fused_feature. The dimensions of output_3d and output_t are both (N, 512), and the dimension of fused_feature is (N, 1024).

[0183] S52. Perform weighted summation on the multi-peak analysis feature vectors output_3d_mpe and output_t_mpe to obtain the multi-peak analysis fusion feature feature_mpe:

[0184] feature_mpe=ρ·output_3d_pe+(1-ρ)·output_t_mpe

[0185] Among them, ρ is the weight coefficient, which is 0.4; the dimensions of output_3d_mpe and output_t_mpe are (N, 4), so the dimension of feature_mpe obtained is (N, 4).

[0186] S53. The cascade feature fused_feature and the multi-peak analysis fusion feature feature_mpe are cascaded again. The dimensions of fused_feature and feature_mpe are (N, 1024) and (N, 4) respectively. The result after cascading is recorded as feature, and the dimension is (N, 1028).

[0187] Step S5 performs feature fusion, and the combination of the two realizes the complementarity of local and global features, thereby improving the robustness and accuracy of feature extraction.

[0188] (6) Step S6: Median filtering

[0189] Median filtering is used in the result after feature fusion to further remove isolated noise points and improve data quality. Median filtering is also applicable when processing high-dimensional features, which can effectively smooth feature data and retain important structural information.

[0190] For the feature vector feature, a median filter is applied to smooth the data and remove noise points. The filter window size is v=3, and the median filter operation is as follows:

[0191] median_filtered ij = median(feature i,j-v:j+v )

[0192] Among them, median() means taking the median of the data in the window; i is the index of the feature vector; j represents the jth dimension of the feature vector.

[0193] The feature vector after median filtering is median_filtered, and its dimension is still (N, 1028). This feature vector is used as the final optimized feature vector for subsequent applications and analysis.

[0194] (6) Step S7: Reconstructing histogram data

[0195] The original dimension of the histogram data is (150, 360, 288), and the reconstruction process is as follows:

[0196] S71. First, map each value in the feature vector median_filtered to the corresponding coordinate of the histogram.

[0197] S72, then use the interpolate() method to assign the eigenvalues ​​to the corresponding bins:

[0198] histogramvalue(x,y,z)=interpolate(median_filtered,x,y,z)

[0199] The final output is the reconstructed histogram data histogramvalue(x,y,z), whose dimension is (150,360,288). This histogram data can be directly used for subsequent analysis and processing.

[0200] In order to apply the above method, an embodiment of the present invention also includes a lidar data processing system, which includes a data preprocessing unit, a multi-peak extraction unit, a neural network feature extraction unit, a multi-peak analysis unit, a feature fusion unit, a median filtering unit and a data reconstruction unit, which are respectively used to execute steps S1 to S7 in the above method.

[0201] Through the above steps, the method and system of the present invention can effectively remove noise from the original histogram data in a complex environment, retain useful features, and significantly improve the quality and reliability of solid-state lidar histogram data, and are suitable for fields such as autonomous driving and robot navigation that require high-precision data processing.

[0202] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A laser radar data processing method, characterized in that: Includes steps: S1, using Gaussian filtering and bilateral filtering to perform noise reduction on the original histogram data of the laser radar; S2. By using multi-peak extraction technology, the histogram data after Gaussian filtering and bilateral filtering are divided into multiple adjacent regions, and the local maximum value and its corresponding bin number are extracted in each region to obtain two corresponding sets of feature group data; S3, the two sets of feature group data are dimensionally upgraded and respectively input into the 3D convolutional neural network, i.e., 3D CNN and Transformer network, to extract local features and global features; S4. Perform multi-peak analysis on the feature vectors extracted by 3D CNN and Transformer networks through multi-peak analysis technology, extract four types of features including peak classification probability, flight time value, distance, and probability density function, and obtain the corresponding multi-peak analysis feature vectors; S5, fusing the feature vectors extracted by the 3D CNN and Transformer networks with the multi-peak analysis feature vectors to obtain fused features; S6, performing median filtering on the fused features; S7. Reconstruct the histogram data of the filtered fusion features to generate optimized histogram data with the same dimension as the original histogram data.

2. The laser radar data processing method according to claim 1, characterized in that: The step S5 specifically comprises the following steps: S51, performing feature cascading on the feature vectors extracted by the 3D CNN and Transformer networks to obtain cascade features; S52, performing weighted summation on the two groups of multi-peak analysis feature vectors to obtain multi-peak analysis fusion features; S53, cascading the cascade features and the multi-peak analysis fusion features to obtain the final fusion features.

3. The laser radar data processing method according to claim 1, characterized in that: 3D CNN includes an extended dimension layer, a first convolution layer, a first three-dimensional maximum pooling layer, a second convolution layer, a second three-dimensional maximum pooling layer, a first flattening layer and a first fully connected layer, which are arranged in sequence from input to output; the extended dimension layer is used to increase the dimension of the feature group data corresponding to the Gaussian filter to obtain a three-dimensional array adapted to the 3D CNN input; the first convolution layer is used to extract local spatial features; the first three-dimensional maximum pooling layer is used to perform maximum pooling to reduce the data dimension; the second convolution layer is used to further extract local spatial features; the second three-dimensional maximum pooling layer is used to perform maximum pooling to further reduce the data dimension; the first flattening layer is used to flatten the three-dimensional features into a one-dimensional feature vector using a flattening operation; the first fully connected layer is used to convert the flattened feature vector into an output vector.

4. The laser radar data processing method according to claim 1, characterized in that: The Transformer network includes a sequentially connected encoder module, a decoder module, a second flattening layer, and a second fully connected layer; the encoder module includes multiple sequentially connected encoders, and the decoder module includes multiple sequentially connected decoders, the output of the previous decoder is the input of the next decoder, and the output of the last encoder is input into each decoder.

5. The laser radar data processing method according to claim 4, characterized in that: Each encoder from input to output includes: The encoder multi-head attention module is used to take the input data as the query matrix Q, the key matrix K and the value matrix V, and perform weighted summation according to these weight matrices to obtain a new representation; The encoder first residual connection and layer normalization module is used to add the output of the encoder multi-head attention module to the original input, followed by layer normalization; Encoder feed-forward neural network for nonlinear mapping via two linear transformations and a ReLU activation function; The encoder second residual connection and layer normalization module is used to add the output of the encoder feed-forward neural network to its input, followed by layer normalization.

6. The laser radar data processing method according to claim 5, characterized in that: Each decoder from input to output includes: The masked multi-head self-attention module is the same as the encoder multi-head attention module, but with the addition of a mask operation; The decoder's first residual connection and layer normalization module is used to add the output of the masked multi-head attention module to the original input, followed by layer normalization; Decoder multi-head attention module: used to generate new representations through the multi-head attention mechanism using the output of the last encoder as the key and value and the input of the decoder as the query; The decoder second residual connection and layer normalization module is used to add the output of the decoder multi-head attention module to the original input and then perform layer normalization; The decoder feedforward neural network is used to perform further nonlinear transformation on the features at each position; The decoder third residual connection and layer normalization module is used to add the output of the decoder feedforward neural network to its input, followed by layer normalization.

7. The laser radar data processing method according to claim 1, characterized in that: In step S1, the formula for performing a convolution operation on the three-dimensional data data (x, y, z) using Gaussian filtering is: Wherein, k is the radius of the Gaussian filter window, G(i,j,l) represents the Gaussian filter function, (i,j,l) represents the coordinates within the Gaussian filter window, and G_data(x,y,z) represents the three-dimensional data obtained after Gaussian filtering of the three-dimensional data data(x,y,z); The Gaussian filter function G(i,j,l) is expressed as follows: Among them, σ represents the standard deviation σ set in Gaussian filtering, and e is the natural base; The three-dimensional data data(x,y,z) is processed using bilateral filtering to obtain smoothed data B_data(x,y,z): Among them, (o, p, q) represents the coordinates in the spatial domain Gaussian filter window, s is the radius of the spatial domain Gaussian filter window; Wp is the weight normalization term, which is used to ensure that the pixel value after filtering is within a reasonable range. Specifically, Wp is: Gs(o,p,q) represents the spatial domain Gaussian filter function, which is expressed as: Among them, σs is the standard deviation of the Gaussian filter function in the spatial domain; Gr(data(x,y,z),data(x+o,y+p,z+q)) represents the grayscale domain Gaussian filter function, which is expressed as: Among them, σr is the standard deviation of the grayscale domain Gaussian filter function; The original histogram data is converted into an array of dimension size (H, W, D) and recorded as three-dimensional data data(x, y, z), where x, y, z are the x-axis, y-axis and z-axis coordinates of any point in the original histogram data, x, y, z ≥ 0, and H, W, D are the x-axis range, y-axis range and z-axis range corresponding to the original histogram data.

8. The laser radar data processing method according to claim 1, characterized in that: The step S2 comprises the steps of: S21, divide the Gaussian filtered data G_data(x, y, z) and the bilateral filtered data B_data(x, y, z) into N adjacent regions, each with a width of Wr, and the number of regions N is determined according to the size of the histogram data: Rm is the measurement range of the histogram data, then the i-th region G_regioni of the Gaussian filtered data G_data(x, y, z) and the i-th region B_regioni of the bilateral filtered data B_data(x, y, z) can be expressed as: G_regioni={G_data(x,y,z)∣i·Wr≤x<(i+1)·Wr} B_regioni={B_data(x,y,z)∣i·Wr≤x<(i+1)·Wr} S22. Extract the local maximum value in each region and its corresponding bin number bni, forming a feature group Recorded as The extraction formula is as follows: bni = argmax(regioni) regioni refers to G_regioni or B_regioni.

9. The laser radar data processing method according to claim 1, characterized in that: The step S4 specifically comprises the following steps: S41. Use a fully connected feedforward neural network to calculate the output activation value FNN(M) of the output of the 3D CNN and Transformer networks: FNN(M)=ReLU(MW1+b1)W2+b2 Where W1 and W2 are weight matrices of the fully connected layer, b1 and b2 are bias terms, M refers to the output of the 3D CNN or Transformer network, FNN(M) is the output of the fully connected feedforward neural network, and ReLU() is the ReLU activation function; S42. Use the softmax function to convert the activation value FNN(M) into the corresponding classification probability Class(M): Class(M)=softmax(FNN(M)Wc+bc) Among them, Wc is the weight matrix of the classification layer; bc is the bias term of the classification layer; S43, calculate the corresponding flight time value T according to the classification probability Class (M) TOF : T TOF =Class(M)Wt+bt Where Wt is the weight matrix of the TOF recovery layer; bt is the bias term of the TOF recovery layer; S44. Calculate the actual distance D using the speed of light c and the time of flight TOF (M) TOF : S45, using the generated flight time T TOF Calculate the probability density function F PDF : Among them, r B and r LB is the decay constant at different time periods, r L is the photon arrival rate of the laser pulse, Tp is the pulse width, and t represents the current time; S46. The four types of features, namely, classification probability, flight time, distance and probability density function, corresponding to the calculated output features of the 3D CNN and Transformer networks respectively constitute the corresponding multi-peak analysis feature vectors output_3d_mpe and output_t_mpe.

10. A laser radar data processing system, characterized in that: It includes a data preprocessing unit, a multi-peak extraction unit, a neural network feature extraction unit, a multi-peak analysis unit, a feature fusion unit, a median filtering unit and a data reconstruction unit, which are respectively used to execute steps S1 to S7 in the laser radar data processing method described in any one of claims 1 to 9.

Images