Laser radar full-waveform intensity correction method based on spatial-temporal characteristic decoupling

Abstract

The invention discloses a laser radar full-waveform intensity correction method based on spatial-temporal feature decoupling, and the method comprises the following steps: (1) obtaining laser radar data, and synchronously carrying out the preprocessing of original full-waveform data; (2) constructing a waveform distortion model, and decomposing the full waveform data into a target component and a system noise component; (3) dividing the target component into a plurality of time sequences, and extracting time features and space features; (4) constructing a dynamic convolutional strength compensation network, and accurately predicting a distance compensation coefficient and a material compensation coefficient based on the spatial-temporal characteristics obtained in the step (3); and (5) enabling the compensation coefficient obtained in the step (4) to act on the laser intensity of the original target component to obtain corrected full-waveform intensity data. According to the method, the target reflection component and the system noise component are separated by establishing the waveform distortion model, and the distance compensation coefficient and the material compensation coefficient are accurately predicted by using the dynamic convolution intensity compensation network, so that high-precision correction of the laser radar full-waveform intensity is realized.

Description

Technical Field

[0001] This invention belongs to the field of LiDAR data processing technology, specifically relating to a LiDAR full waveform intensity correction method based on spatiotemporal feature decoupling. Background Technology

[0002] As an advanced active remote sensing technology, lidar can accurately obtain information such as the distance and intensity of a target by emitting a laser beam and receiving the reflected echo. It is widely used in urban planning, land resource surveys, geological disaster monitoring, autonomous driving and other fields.

[0003] Compared to traditional lidar, full-waveform lidar represents a significant technological breakthrough. It can completely record the echo waveform generated by the interaction between the laser pulse and the target, containing not only the target's distance information but also rich characteristic information such as the target's material, surface roughness, and geometry. Its application value in various industries is increasingly prominent. However, existing lidar full-waveform intensity correction methods mostly address only a single factor or a few factors, failing to solve the waveform superposition effect caused by pulse broadening (especially for high-reflectivity objects at close range), and also failing to fully consider the coupling relationships between multiple factors, lacking in-depth mining and effective utilization of the spatiotemporal characteristics of the data. Therefore, there is an urgent need for a lidar full-waveform intensity correction method that can comprehensively and effectively handle multiple interference factors and fully utilize spatiotemporal characteristics. Summary of the Invention

[0004] The purpose of this invention is to propose a method for full waveform intensity correction of lidar based on spatiotemporal feature decoupling. By establishing a waveform distortion model to separate the target reflection component from the system noise component, and using a dynamic convolution intensity compensation network to accurately predict the distance compensation coefficient and material compensation coefficient, a high-precision correction of the full waveform intensity of lidar is achieved, thereby improving data accuracy and reliability.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for full-waveform intensity correction of lidar based on spatiotemporal feature decoupling, the method is as follows:

[0007] Step (1): Acquire lidar data and simultaneously preprocess the raw full waveform data;

[0008] Step (2): Construct a waveform distortion model to decompose the full waveform data into target components and system noise components;

[0009] Step (3): Divide the target component into multiple time series and extract temporal and spatial features;

[0010] Step (4): Construct a dynamic convolutional intensity compensation network to accurately predict the distance compensation coefficient and material compensation coefficient based on the spatiotemporal features obtained in step (3);

[0011] Step (5): Apply the compensation coefficient obtained in step (4) to the laser intensity of the original target component to obtain the corrected full waveform intensity data.

[0012] Furthermore, the preprocessing method in step (1) is as follows: Step (11) the target area is collected using a laser acquisition device to obtain data, including the original laser radar data, timestamp, attitude information and other data; Step (12) the original full waveform data is denoised by Gaussian filtering; Step (13) the denoised data is normalized to facilitate subsequent data analysis and model processing.

[0013] Furthermore, the specific method for constructing the waveform distortion model in step (2) and decomposing the full waveform data into target components and system noise components is as follows:

[0014] Step (21) Based on the physical principles of laser radar signal transmission and reflection, establish a mathematical model of waveform distortion;

[0015] Step (22) Based on massive amounts of known waveform data, the K-SVD algorithm is used for iterative training to obtain a dictionary that can accurately characterize the target reflection signal and system noise features;

[0016] Step (23) uses a sparse coding method and combines it with the dictionary learned in step (22) to perform sparse calculation on the full waveform data obtained in step (21) to obtain the target reflection component.

[0017] Furthermore, the specific method for dividing the target component into multiple time series and extracting temporal and spatial features in step (3) is as follows:

[0018] Step (31) Based on the scanning frequency of the lidar, the target reflection signal obtained in step (22) is divided in the time dimension to obtain multi-time series data;

[0019] Step (32) Based on the scanning spatial range of the lidar, the target reflection signal obtained in step (22) is divided spatially to obtain multi-spatial sequence data;

[0020] Step (33) uses Fourier transform to obtain time features from the multi-time series data obtained in step (31);

[0021] Step (34) uses an edge detection algorithm to obtain spatial features from the multi-spatial sequence data obtained in step (32).

[0022] Furthermore, in step (4), a dynamic convolutional intensity compensation network is constructed. The specific method for accurately predicting the distance compensation coefficient and material compensation coefficient based on the spatiotemporal features obtained in step (3) is as follows:

[0023] Step (41) Construct a Dynamic Convolutional Intensity Compensation Network (DyIn-Net) with two branches;

[0024] Step (42) For the distance compensation coefficient branch in the dual-branch system, add a distance feature extraction layer to enhance the network's ability to learn distance-related features;

[0025] Step (43) For the material compensation system branch in the dual-branch system, a multi-scale feature fusion module is introduced to enhance the network's ability to identify and compensate for complex materials.

[0026] Step (44) splits the data obtained in step (3) into training set, validation set and test set, and inputs the spatiotemporal features of the training data into the network training in step (41) to update the network parameters, so as to accurately output the distance compensation coefficient and material compensation coefficient.

[0027] Furthermore, in step (5), the compensation coefficient obtained in step (4) is applied to the laser intensity of the original target component to obtain the corrected full waveform intensity data. The specific method is as follows:

[0028] Step (51) combines the material compensation coefficient and distance compensation coefficient from step (4) to compensate for the original intensity, and establishes a mapping relationship between the combined compensation coefficient and the target reflection signal;

[0029] Step (52) adopts a joint compensation method, performing distance compensation and material compensation simultaneously for each sampling point of the target reflection signal to obtain the corrected full waveform intensity;

[0030] Step (53) uses multiple evaluation indicators to conduct quality assessment and verification of the full waveform intensity after correction, and obtains the corrected data that meets the requirements.

[0031] The above technical solution can achieve the following beneficial effects:

[0032] This invention establishes a waveform distortion model and employs a sparse coding method to effectively separate the target reflection component from the system noise component, providing an accurate data basis for subsequent intensity correction.

[0033] The dynamic convolutional intensity compensation network proposed in this invention adopts a dual-branch structure to predict the distance compensation coefficient and the material compensation coefficient respectively. It can accurately distinguish the influence of distance factors and material factors on intensity, and achieve high-precision correction of the intensity of the entire LiDAR waveform. Compared with traditional methods, the correction accuracy is significantly improved.

[0034] The processing method based on spatiotemporal feature decoupling proposed in this invention decouples the changing features of time and space dimensions, enabling the correction method to better adapt to complex and ever-changing real-world application scenarios and improving the method's versatility and reliability. Attached Figure Description

[0035] Figure 1 A schematic diagram of a full-waveform intensity correction method for lidar based on spatiotemporal feature decoupling.

[0036] Figure 2 Schematic diagram of dynamic convolution intensity compensation network. Detailed Implementation

[0037] The invention will be further described below with reference to the accompanying drawings:

[0038] like Figure 1-2 As shown, a method and system for full-waveform intensity correction of lidar based on spatiotemporal feature decoupling includes the following steps:

[0039] Step 1: Acquire LiDAR data and simultaneously preprocess the raw full waveform data. The specific method for this step is as follows:

[0040] Step 1.1: Based on the autonomous driving data collection vehicle, acquire LiDAR data of the urban road environment, including LiDAR full waveform data at a collection frequency of 10Hz, GPS global position of the vehicle at 1Hz, inertial navigation data at 100Hz, and timestamps.

[0041] Step 1.2: Based on the characteristics of data noise in urban road environments, this embodiment selects a Gaussian kernel with a standard deviation of 1.8. Gaussian noise is removed through convolution operations to obtain a clearer full waveform signal.

[0042] Step 1.3: Normalize the entire waveform signal and map it to the interval [0, 1].

[0043] Step 2: Construct a waveform distortion model, decomposing the entire waveform data into target components and system noise components; the specific method for this step is as follows:

[0044] Step 2.1: Based on the physical principles of laser radar signal transmission and reflection, establish a waveform distortion mathematical model, as shown in formula (1);

[0045] S(t)=T(t)·A(t)+N(t) (1)

[0046] Where S(t) is the original waveform signal acquired by the lidar receiver at time t, T(t) is the target reflection component at time t, A(t) is the atmospheric attenuation component experienced by the laser during propagation, and N(t) is the noise of the lidar system itself.

[0047] Step 2.2: Based on 100,000 sets of massive labeled waveform data, and unifying the data into sequences of length 512, initialize the number of atoms in the K-SVD dictionary to K=256, and randomly generate the initial dictionary D0;

[0048] Step 2.3: Using the training data, iteratively update the dictionary D and sparse coefficients X based on the orthogonal matching pursuit (OMP) algorithm to obtain the final dictionary D that can characterize the target reflection intensity and system noise characteristics;

[0049] Step 2.4, further split dictionary D into target dictionary D T and noise dictionary D N Therefore, the distortion mathematical model of formula (1) can be rewritten as formula (2) as follows:

[0050] S(t)=D T α T (t)+D N α N (t) (2)

[0051] Where, α T α N The sparse coefficients of the target reflected signal and the system noise are respectively used to iteratively calculate α based on the alternating direction multiplier method (ADMM). T α N And obtain the value that minimizes the reconstruction error.

[0052] Step 2.5; Using the obtained... and dictionary D T The target reflection component can be calculated.

[0053] Step 3: Divide the target component into multiple time series and extract temporal and spatial features. The specific method for this step is as follows:

[0054] Step 3.1, based on the scanning frequency f of the lidar, divide the time series into segments... A second is divided into a basic time unit. In this embodiment, f = 10Hz, so every 0.1 seconds is divided into a time unit.

[0055] Step 3.2: Based on the scanning space range of the lidar, the horizontal direction is divided into 1° increments, and the vertical direction is divided by selecting an appropriate scanning angle. In this embodiment, the vertical direction is divided into 0.2° increments. In the distance direction, the vertical direction is divided from near to far according to a certain distance. In this embodiment, the vertical direction is divided into 5m increments.

[0056] Step 3.3: The multi-time series data obtained in Step 3.1 is transformed from the time domain to the frequency domain using Discrete Fourier Transform (DFT). The frequency domain data is then sorted in descending order based on the amplitude values ​​|y|, and the frequency values ​​f corresponding to the K frequency components with the largest amplitude values ​​(K=5 in this embodiment) are selected. i Amplitude value |y i |and phase value As its characteristic in the time dimension

[0057] Step 3.4: The multi-spatial sequence data obtained in Step 3.2 is processed using an edge detection algorithm (the commonly used Canny operator in this embodiment) to obtain the contour shape, and then the shape descriptor is calculated, such as geometric features F including perimeter, area, aspect ratio, and roundness. g Furthermore, the surface features F, such as surface normal vectors and curvature, within the spatial region are calculated. s These together constitute the features in the spatial dimension.

[0058] Step 4: Construct a dynamic convolutional intensity compensation network to accurately predict the distance compensation coefficient and material compensation coefficient based on the spatiotemporal features obtained in Step 3. This step is detailed below:

[0059] Step 4.1: Construct a dual-branch Dynamic Convolutional Intensity Compensation Network (DyIn-Net), as follows: Figure 2 As shown.

[0060] Step 4.2: In the feature processing module, different networks are used to process temporal and spatial features respectively. The temporal feature processing module uses a three-layer one-dimensional convolutional network structure with kernel sizes of 7, 5, and 3. The acquired feature dimension in this embodiment is [batch]. size ,64],batch size=64; Spatial features are processed using the commonly used PointNet++ architecture, which includes SA (Set Abstraction) and FP (Feature Propagation) layers to extract multi-level spatial features. In this embodiment, the extracted dimension is [batch_size, 128]; and an attention mechanism is used to perform feature fusion processing on temporal and spatial features to generate a spatiotemporal joint feature representation with a feature dimension of [batch_size, 192].

[0061] Step 4.3 involves incorporating the fused spatiotemporal features from Step 4.2 into a dynamic convolutional network for adaptive processing. A deformable convolution structure is employed, and an additional offset prediction network adaptively adjusts the sampling positions of the convolutional kernels to obtain the enhanced spatiotemporal feature representation. The output dimension is [batch_size, 512]. The offset prediction network consists of two fully connected layers, taking the spatiotemporal joint features as input and outputting the offset parameters of the convolutional kernels. The dynamic convolutional layer adjusts the sampling grid of the standard convolutional kernels based on the predicted offsets, enhancing the network's adaptability to targets of different shapes.

[0062] Step 4.4: Based on the spatiotemporal features obtained in 4.3, a dual-branch compensation prediction module is connected. The distance compensation coefficient branch contains four fully connected layers with 256, 128, 64, and 1 neurons respectively, used to predict the distance compensation coefficient. Its output is the compensation factor for laser intensity at different distances.

[0063] Step 4.5: For the material compensation system branch in the dual-branch architecture, a hybrid architecture is adopted. The first three layers are shared with the distance compensation branch, followed by a material classifier and a material coefficient regressor. The target material type and the corresponding compensation coefficient are jointly predicted. The output includes different material types such as metal, concrete, vegetation, etc. (for semantic understanding), and different compensation factors are given for different materials.

[0064] Step 4.6: For the material compensation system branch in the dual-branch architecture, a multi-scale feature fusion module is introduced to enhance the network's ability to identify and compensate for complex materials.

[0065] Step 4.7: Split the data obtained in step (3) into a training set, a validation set, and a test set, and use them as the data for steps 4.1 to 4.7.

[0066] In step 4.6, iterative training is performed on the network to obtain accurate distance compensation coefficient δ(d) and material compensation coefficient β(m). Here, δ(d) is a piecewise function related to distance d, as shown in formula (3), and β(m) is a classification mapping table related to material type m, as shown in formula (4).

[0067]

[0068] In the formula above, d, d1, d2, p, β1, β2, β3, β4, and β5 are all parameters that the network needs to predict.

[0069] Step 5: Apply the compensation coefficient obtained in Step 4 to the laser intensity of the original target component to obtain the corrected full waveform intensity data. The specific method for this step is as follows:

[0070] Step 5.1: Based on the distance compensation coefficient and material compensation coefficient obtained in Step 4, calculate the compensation value for each sampling point in the target reflection signal;

[0071] Step 5.2: Using a joint compensation method, distance compensation and material compensation are performed simultaneously for each sampling point of the target reflected signal to obtain the corrected full waveform intensity.

[0072] Step 5.3: Based on the training data, calculate the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) between the corrected data and the reference data to evaluate the correction effect. A higher PSNR value (over 35dB in this embodiment) is considered a good result, and a closer SSIM to 1 indicates a better correction effect and higher reliability of the network output parameters. Otherwise, the training and validation data need to be adjusted for further training to obtain a better correction coefficient.

[0073] The above descriptions are all preferred embodiments of the present invention. For those skilled in the art, any modifications to the present invention in various equivalent forms without departing from the principle of the present invention shall fall within the protection scope of the appended claims.

Claims

1. A method for full-waveform intensity correction of lidar based on spatiotemporal feature decoupling, characterized in that: The method is as follows: Step (1): Acquire lidar data and simultaneously preprocess the raw full waveform data; Step (2): Construct a waveform distortion model to decompose the full waveform data into target components and system noise components; Step (3): Divide the target component into multiple time series and extract temporal and spatial features; Step (4): Construct a dynamic convolutional intensity compensation network to accurately predict the distance compensation coefficient and material compensation coefficient based on the spatiotemporal features obtained in step (3); Step (5): Apply the compensation coefficient obtained in step (4) to the laser intensity of the original target component to obtain the corrected full waveform intensity data.

2. The method for full-waveform intensity correction of lidar based on spatiotemporal feature decoupling according to claim 1, characterized in that: The preprocessing method in step (1) is as follows: Step (11) the target area is collected using a laser acquisition device to obtain data, including the original laser radar data, timestamp, attitude information and other data; Step (12) the original full waveform data is denoised by Gaussian filtering; Step (13) the denoised data is normalized to facilitate subsequent data analysis and model processing.

3. The method for full-waveform intensity correction of lidar based on spatiotemporal feature decoupling according to claim 1, characterized in that: The specific method for constructing the waveform distortion model in step (2) and decomposing the full waveform data into target components and system noise components is as follows: Step (21) Based on the physical principles of laser radar signal transmission and reflection, establish a mathematical model of waveform distortion; Step (22) Based on massive amounts of known waveform data, the K-SVD algorithm is used for iterative training to obtain a dictionary that can accurately characterize the target reflection signal and system noise features; Step (23) uses a sparse coding method and combines it with the dictionary learned in step (22) to perform sparse calculation on the full waveform data obtained in step (21) to obtain the target reflection component.

4. The method for full-waveform intensity correction of lidar based on spatiotemporal feature decoupling according to claim 3, characterized in that: The specific method for dividing the target component into multiple time series and extracting temporal and spatial features in step (3) is as follows: Step (31) Based on the scanning frequency of the lidar, the target reflection signal obtained in step (22) is divided in the time dimension to obtain multi-time series data; Step (32) Based on the scanning spatial range of the lidar, the target reflection signal obtained in step (22) is divided spatially to obtain multi-spatial sequence data; Step (33) uses Fourier transform to obtain time features from the multi-time series data obtained in step (31); Step (34) uses an edge detection algorithm to obtain spatial features from the multi-spatial sequence data obtained in step (32).

5. The method for full-waveform intensity correction of lidar based on spatiotemporal feature decoupling according to claim 1, characterized in that: In step (4), a dynamic convolutional intensity compensation network is constructed. The specific method for accurately predicting the distance compensation coefficient and material compensation coefficient based on the spatiotemporal features obtained in step (3) is as follows: Step (41) Construct a Dynamic Convolutional Intensity Compensation Network (DyIn-Net) with two branches; Step (42) For the distance compensation coefficient branch in the dual-branch system, add a distance feature extraction layer to enhance the network's ability to learn distance-related features; Step (43) For the material compensation system branch in the dual-branch system, a multi-scale feature fusion module is introduced to enhance the network's ability to identify and compensate for complex materials; Step (44) splits the data obtained in step (3) into training set, validation set and test set, and inputs the spatiotemporal features of the training data into the network training of step (41) to update the network parameters, so as to accurately output the distance compensation coefficient and material compensation coefficient.

6. The method for full-waveform intensity correction of lidar based on spatiotemporal feature decoupling according to claim 1, characterized in that: The specific method for applying the compensation coefficient obtained in step (4) to the laser intensity of the original target component in step (5) to obtain the corrected full waveform intensity data is as follows: Step (51) combines the material compensation coefficient and distance compensation coefficient from step (4) to compensate for the original intensity, and establishes a mapping relationship between the combined compensation coefficient and the target reflection signal; Step (52) adopts a joint compensation method, performing distance compensation and material compensation simultaneously for each sampling point of the target reflection signal to obtain the corrected full waveform intensity; Step (53) uses multiple evaluation indicators to conduct quality assessment and verification of the full waveform intensity after correction, and obtains the corrected data that meets the requirements.

Images