Laser radar point cloud generation method based on equidistant projection converter

Abstract

The invention discloses a laser radar point cloud generation method based on an isometric projection converter, and the method comprises the steps: carrying out the feature extraction of targets of different scales through an isometric projection converter denoising network; through a designed control signal injector, the time step is better fused with the features extracted from the denoising network of the isometric projection converter; and finally, modulating high and low frequency signals, highlighting high frequency signals and suppressing low frequency signals in the generated equidistant projection drawing through a frequency domain modulator to obtain a finally generated laser radar point cloud. According to the invention, high-performance feature extraction of irregularly changing targets in equidistant projection can be realized, the generation process is controlled by a more powerful fusion mechanism, and finally a high-quality laser radar point cloud generation effect is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving, and in particular, is a method for generating a laser radar point cloud based on an isometric projection transformer. Background Art

[0002] LiDAR point cloud generation aims to generate extremely large amounts of diverse and highly realistic LiDAR point cloud data, which is of great significance in fields such as autonomous driving. Due to the high price of LiDAR point cloud data acquisition equipment, complex road conditions, and huge labor costs, LiDAR point cloud data in the current autonomous driving field is still in a very scarce state. LiDAR point cloud generation algorithms can provide a large amount of training and test data for existing models, and provide more complete and accurate environmental perception data for autonomous driving systems, thereby achieving safer and more reliable driving decisions. In recent years, thanks to the development of diffusion-denoising models, LiDAR point cloud data generation has made some progress. However, since the Unet used cannot capture long-range dependencies and the fusion of control signals is very low, LiDAR point cloud generation is still challenging when facing sparse data with exaggerated deformations.

[0003] There are usually two ways to generate LiDAR point clouds. The first one uses more traditional generative models, such as autoencoders and adversarial generative models. Representative methods include LG3D, DGLiDAR, and SnowflakeNet. Their training process is very unstable, and the generated LiDAR point clouds are very rough and have only fixed sizes, which greatly limits the diversity of the generated results. The second method uses a diffusion-denoising model. Representative methods include LiDARGen and R2DM. They both use Unet as a denoising network and can only extract features in a fixed frame. For LiDAR point clouds, they cannot capture the critical long-range dependencies and have large performance limitations. Summary of the invention

[0004] The purpose of the present invention is to provide a method for generating a lidar point cloud based on an isometric projection transformer to solve the problems existing in the existing methods. The method uses an isometric projection transformer to capture the long-range dependencies in the isometric projection image, efficiently realizes the learning of the lidar point cloud data, and efficiently integrates the control signal into the denoising process through a control signal embedder (CEI) to obtain a high-quality lidar point cloud generation result. Then, a frequency domain modulator is used to overcome the smooth transition characteristics of the isometric projection image and enhance the high-frequency part to highlight the target in the lidar point cloud.

[0005] The technical solution to achieve the purpose of the present invention is: a method for generating a laser radar point cloud based on an isometric projection transformer, comprising the following steps:

[0006] Step 1: Collect the lidar point cloud of the outdoor autonomous driving scene and project it into an isometric projection image of h×w×c. According to the length, width and number of channels of the isometric projection image, generate the corresponding Fourier position feature, and splice the Fourier position feature with the isometric projection image according to the channel dimension;

[0007] Step 2: Send the isometric projection image obtained after data processing to the isometric projection transformer denoising network to obtain the isometric image projection features and the reverse isometric image projection features; the denoising network includes the isometric projection attention mechanism (Equirectangular Attention, EA) and the reverse isometric projection attention mechanism (Reverse Equirectangular Attention, REA), which fully considers the physical characteristics of the isometric projection image and captures the isometric projection image features in a global modeling manner;

[0008] Step 3: Send the time-step control signal to the control-signal embedding injector (CEI), and fuse the control signal with the denoising features extracted in step 2 through the global attention mechanism;

[0009] Step 4: construct a frequency domain modulator (FM) to enhance the low-frequency part of the over-smoothed equidistant projection image to obtain the final equidistant projection image;

[0010] Step 5: Use the two-norm loss function to supervise the isometric projection image obtained in step 4, perform an end-to-end network learning process, and finally obtain a lidar point cloud generation model.

[0011] Furthermore, in step 1, it is necessary to generate Fourier position features and perform fusion. First, the collected laser radar point cloud x 0 The Fourier coordinate features are integrated in the convolution operation to map the coordinates from the three-dimensional space to the two-dimensional space at a high frequency to obtain the initial coordinate feature p. The steps are as follows:

[0012]

[0013] Among them, c represents the coordinates in the initial equidistant projection image, * represents the convolution operation, and arrange(Δ) represents taking out integers from 0 to Δ; then, Fourier mapping is used to obtain the Fourier position feature f, and the steps are as follows:

[0014] f=[sin(p),cos(p),sin(2p),cos(2p),…,sin(2 m-1 p),cos(2 m-1 p)] T #(2)

[0015] Among them, sin represents the sine function, cos represents the cosine function, and m represents the frequency adjustment parameter. After obtaining the Fourier feature f, it is combined with the input equidistant projection feature x 0 Splicing is done according to the channel dimension to enhance the high-frequency response in the feature extraction process and overcome the transition smoothness problem. The steps are as follows:

[0016]

[0017] in, Represents splicing by channel dimension.

[0018] Furthermore, in step 2, the isometric projection image obtained in step 1 is used to extract the isometric projection image features by the isometric projection attention mechanism. And use the inverse equidistant projection attention mechanism to upsample and obtain the inverse equidistant projection feature The isometric projection attention mechanism and the reverse isometric projection attention mechanism described in step 2 are both composed of the self-attention mechanism and the cross-attention mechanism. Represents x i and The eigenvalues ​​in are real numbers, h×w×c represents the extracted feature resolution, and c is the feature dimension.

[0019] Furthermore, in the isometric projection transformer denoising network of step 2:

[0020] (1) Isometric projection attention mechanism for global downsampling: This part accepts isometric projection image features x 0 As input, we output the equidistant projection image features x={x 1 ,x 2 ,x 3 ,x 4}; The specific implementation process is: using the stacking of isometric projection attention mechanism to process the input isometric projection image, the steps are:

[0021]

[0022] Among them, EA stands for isometric projection attention mechanism, Selfattention stands for self-attention mechanism, and Unflod stands for stacking operation;

[0023] (2) Inverse isometric projection attention mechanism for global upsampling: This part accepts the downsampled features x of the isometric projection image i As input, 4 inverse isometric projection image features are output, and the steps are:

[0024]

[0025] Among them, REA stands for the inverse equidistant projection attention mechanism, CEI stands for the control signal embedder, Flod stands for the unfolding operation, and ts stands for the time step sequence feature, that is, the time step control signal.

[0026] Furthermore, in the control signal embedder described in step 3, the main sequence features of the inverse equidistant projection image features obtained in the previous step are The main sequence features d and time step sequence features ts separated from the original image are first concatenated for self-attention operation, and then the new sequence is split in the order of concatenation to form a new query feature sequence Qd, key sequence Kts and value sequence Vts; the final inverse isometric projection image features are obtained by performing cross-attention operations, matrix multiplication and addition operations on the feature sequence. The steps are as follows:

[0027]

[0028] Among them, Crossattention represents the cross attention mechanism. Represents matrix multiplication.

[0029] Furthermore, the step 4 constructs a frequency domain modulator, and the specific implementation process is as follows:

[0030]

[0031] Among them, FM stands for frequency domain modulator, IDWT stands for discrete wavelet transform, DWT is the inverse discrete wavelet transform, and ⊙ represents the Hadamard product.

[0032] Furthermore, in the end-to-end network learning process, the loss function used is:

[0033]

[0034] Among them, Net represents the isometric projection transformer denoising network, and t represents the current denoising step; the loss function value is calculated, and the network parameters in the isometric projection denoising transformer are updated using the gradient descent method until the model converges to obtain a trained lidar point cloud generation model.

[0035] Compared with the prior art, the present invention has the following significant advantages:

[0036] (1) The present invention designs an isometric projection transformer denoising network, which includes a designed isometric projection attention mechanism and an inverse isometric projection attention mechanism. By capturing long-range dependencies, the sparse isometric projection image features can be better modeled. At the same time, objects with drastic deformation in the isometric projection image can be effectively modeled. Compared with previous methods, the present invention achieves the capture of long-range dependencies through this transformer, which is beneficial to improving the generation effect of lidar point cloud.

[0037] (2) The present invention constructs a control signal embedder to more efficiently transfer the time step features to the isometric projection image features. Compared with the previous method, the present invention realizes global information exchange and obtains more accurate isometric projection image features of the current time step. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is the overall framework diagram of the method of the present invention.

[0039] Figure 2 (a) is the overall structure diagram of the isometric projection denoising network, (b) is the structure diagram of the isometric projection attention mechanism, and (c) is the structure diagram of the inverse isometric projection attention mechanism.

[0040] Figure 3 is a structural diagram of the control signal injector.

[0041] Figure 4 It is the structural diagram of the frequency domain modulator.

[0042] Figure 5 This is a comparison chart of the generated results with other methods. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0044] The present invention provides a method for generating laser radar point cloud based on an isometric projection transformer. The training process is an end-to-end training process, and the specific steps are as follows:

[0045] Step 1: When the model accepts the original point cloud as input, the present invention first converts it into a Cartesian coordinate system

[0046] Projection to a spatial coordinate system Where θ is the inclination angle, φ is the azimuth angle, and d is the depth, and is changed as follows: d = log(d+1) / log(d max +1)∈[0,1] H×w .

[0047] Step 2: Generate Fourier coordinate features. First, the coordinates mapped from three-dimensional space to two-dimensional space need to be high-frequencyized through high-frequency convolution operations to obtain the initial coordinate features p. The steps are as follows:

[0048]

[0049] Wherein, c represents the coordinates in the initial equidistant projection image, * represents the convolution operation, and arrange(Δ) represents taking out integers from 0 to Δ. After that, the present invention uses Fourier mapping to obtain the Fourier position feature f, and the steps are as follows:

[0050] f=[sin(p),cos(p),sin(2p),cos(2p),…,sin(2 m-1 p),cos(2 m-1 p)] T #(2)

[0051] Among them, sin represents the sine function, cos represents the cosine function, and m represents the frequency adjustment parameter. After obtaining the Fourier feature f, it is combined with the input equidistant projection feature x 0 Splicing is done according to the channel dimension to enhance the high-frequency response in the feature extraction process and overcome the transition smoothness problem. The steps are as follows:

[0052] x 0 :=x 0 ⊕f#(3)

[0053] Among them, ⊕ represents splicing according to the channel dimension.

[0054] Step 3: Construct an isometric projection transformer, which consists of two parts: isometric projection attention mechanism and inverse isometric projection attention mechanism, which perform global downsampling and global upsampling respectively.

[0055] (1) Isometric projection attention mechanism for global downsampling: This part receives the isometric projection image x with added noise and Fourier position features 0 As input, we output the equidistant projection image features x={x 1 ,x 2 ,x 3 ,x 4}. The specific implementation process is:

[0056]

[0057] in, EA stands for Equidistant Projection Attention Mechanism, Selfattention stands for Self-Attention Mechanism, and Unflod stands for Stacking Operation.

[0058] (2) Inverse isometric projection attention mechanism for global upsampling: This part accepts the downsampled features x of the isometric projection image i As input, 4 inverse isometric projection image features are output, and the steps are:

[0059]

[0060] Among them, REA stands for the inverse equidistant projection attention mechanism, CEI stands for the control signal embedder, Flod stands for the unfolding operation, and ts stands for the time step sequence feature, that is, the time step control signal.

[0061] Step 4: Construct a control signal injector. The main sequence features of the inverse isometric projection image features obtained in the previous step The main sequence feature d and the time step sequence feature ts separated from the image are first concatenated for self-attention operation, and then the new sequence obtained is split in the order of concatenation to form a new query feature sequence Qd, key sequence Kts and value sequence Vts. These new sequences are used for cross-attention operation. In order to maintain the guiding effect of the time step sequence feature ts, the present invention performs additional matrix multiplication on it. In order to consolidate the importance of the main sequence feature d, it is added to the feature sequence after multiplication to obtain the final inverse isometric projection image feature, and the steps are as follows:

[0062]

[0063] Among them, Crossattention represents the cross attention mechanism. Represents matrix multiplication.

[0064] Step 5: Build the frequency domain modulator. The input to this part is The present invention utilizes discrete wavelet transform and inverse discrete wavelet transform, and designs a frequency gating function. The goal is to Decoding to multi-angle high-frequency wavelet bands facilitates frequency domain modulation, guiding the model to adaptively adjust the optimal frequency of each frequency band, greatly suppressing the over-smoothing phenomenon, and finally outputting The specific implementation process is:

[0065]

[0066] Among them, FM stands for frequency domain modulator, IDWT stands for discrete wavelet transform, DWT is the inverse discrete wavelet transform, and ⊙ represents the Hadamard product.

[0067] Step 6: Optimize the entire denoising network end-to-end. The loss function used in the present invention is:

[0068]

[0069] Among them, Net represents the isometric projection transformer denoising network, and t represents the current denoising step. Calculate the loss function value, and use the gradient descent method to update the network parameters in the isometric projection denoising transformer until the model converges to obtain a trained lidar point cloud generation model.

[0070] The present invention is further described below in conjunction with the accompanying drawings and embodiments of the specification. The described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0071] The present invention is described in detail below with reference to the accompanying drawings. The overall framework of the method is as follows: Figure 1 As shown in FIG. 1 , a method for generating laser radar point cloud based on an isometric projection transformer is shown in FIG. 1 , and the specific steps of its training are as follows:

[0072] Step 1: When the model accepts the original point cloud as input, the present invention first converts it into a Cartesian coordinate system Projection to a spatial coordinate system φ=atan2(y,x), where θ is the inclination angle, φ is the azimuth angle, and d is the depth, and is changed as follows: d=log(d+1) / log(d max +1)∈[0,1] H×w .

[0073] Step 2: If Figure 2 The left side of (a) shows the Fourier coordinate feature generation. First, the coordinates mapped from the three-dimensional space to the two-dimensional space need to be high-frequencyized through a high-frequency convolution operation to obtain the initial coordinate feature p. The steps are as follows:

[0074]

[0075] Wherein, c represents the coordinates in the initial equidistant projection image, * represents the convolution operation, and arrange(Δ) represents taking out integers from 0 to Δ. After that, the present invention uses Fourier mapping to obtain the Fourier position feature f, and the steps are as follows:

[0076] f=[sin(p),cos(p),sin(2p),cos(2p),…,sin(2 m-1 p),cos(2 m-1 p)] T #(2)

[0077] Among them, sin represents the sine function, cos represents the cosine function, and m represents the frequency adjustment parameter. After obtaining the Fourier feature f, it is combined with the input equidistant projection feature x 0Splicing is done according to the channel dimension to enhance the high-frequency response in the feature extraction process and overcome the transition smoothness problem. The steps are as follows:

[0078]

[0079] Among them, ⊕ represents splicing according to the channel dimension.

[0080] Step 3: If Figure 2 As shown in part (a), an isometric projection transformer is constructed, which consists of two parts: an isometric projection attention mechanism and an inverse isometric projection attention mechanism, which perform global downsampling and global upsampling respectively.

[0081] (1) The isometric projection attention mechanism is as follows Figure 2 As shown in part (b), global downsampling is performed through the isometric projection attention mechanism: This part receives the isometric projection image x with added noise and Fourier position features. 0 As input, we output the equidistant projection image features x={x 1 ,x 2 ,x 3 ,x 4}. The specific implementation process is:

[0082]

[0083] in, EA stands for Equidistant Projection Attention Mechanism, Selfattention stands for Self-Attention Mechanism, and Unflod stands for Stacking Operation.

[0084] (2) The inverse isometric projection attention mechanism is as follows Figure 2 As shown in part (c) of the paper, global upsampling is performed through the inverse isometric projection attention mechanism: this part accepts the downsampled features x of the isometric projection image i As input, 4 inverse isometric projection image features are output, and the steps are:

[0085]

[0086] Among them, REA stands for the inverse equidistant projection attention mechanism, CEI stands for the control signal embedder, Flod stands for the unfolding operation, and ts stands for the time step sequence feature, that is, the time step control signal.

[0087] Step 4: Figure 3 The structure of the control signal injector is shown. The main sequence features of the inverse isometric projection image features obtained in the previous step The main sequence feature d and the time step sequence feature ts separated from the image are first concatenated for self-attention operation, and then the new sequence is split in the order of concatenation to form a new query feature sequence Qd, key sequence Kts and value sequence Vts. These new sequences are used for cross attention operation. In order to maintain the guiding effect of the time step sequence feature ts, the present invention performs additional matrix multiplication on it. In order to consolidate the importance of the main sequence feature d, we add it to the feature sequence after multiplication to obtain the final inverse isometric projection image feature, and the steps are as follows:

[0088]

[0089] Among them, Crossattention represents the cross attention mechanism. Represents matrix multiplication.

[0090] Step 5: The structure of the frequency domain modulator is as follows Figure 4 As shown, the input for this part is The present invention utilizes discrete wavelet transform and inverse discrete wavelet transform, and designs a frequency gating function. The goal is to Decoding to multi-angle high-frequency wavelet bands facilitates frequency domain modulation, guiding the model to adaptively adjust the optimal frequency of each frequency band, greatly suppressing the over-smoothing phenomenon, and finally outputting The specific implementation process is:

[0091]

[0092] Among them, FM stands for frequency domain modulator, IDWT stands for discrete wavelet transform, DWT is the inverse discrete wavelet transform, and ⊙ represents the Hadamard product.

[0093] Step 6: Optimize the entire denoising network end-to-end. The loss function used in the present invention is:

[0094]

[0095] Among them, Net represents the isometric projection transformer denoising network, and t represents the current denoising step. Calculate the loss function value, and use the gradient descent method to update the network parameters in the isometric projection denoising transformer until the model converges to obtain a trained lidar point cloud generation model.

[0096] In order to further verify the feasibility and effectiveness of the method of the present invention, the technical effect of the present invention is further described in combination with experiments. The hardware platform used in the experiment of the present invention is: Intel (R) Xeon (R) Gold 6230 CPU @ 2.10GHz × 32, memory is 160G, 4 RTX 3090 GPUs with 24G video memory, and the software platform used in the experiment of the present invention is: Ubuntu18.04.6LTS operating system and Python3.8.17, Pytorch2.1.0 deep learning framework. The KITTI360 data set is used to analyze the laser radar point cloud generation effect obtained by this method. Here, the four indicators of Frechet range distance (FRD), Frechet point cloud distance (FPD), Jensen-Shannon divergence (JSD), and minimum matching distance (MMD) are used to evaluate the performance of laser radar point cloud generation.

[0097] The method of the present invention is used to train on the KITTI360 dataset. During generation, randomly sampled Gaussian noise is used as input to feed the model to obtain diversified lidar point cloud generation results. In order to test the performance of the method of the present invention, the proposed lidar point cloud generation method based on isometric projection transformer is compared with several existing methods, including: LiDARGAN, a method based on generative adversarial network proposed by Caccia, L. et al. in their paper “Deep generative modeling of lidar data. In: 2019 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 5034–5040. 2019.”, LiDARVAE, a method based on variational autoencoder proposed by Caccia, L. et al. in their paper “Deep generative modeling of lidar data. IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 5034–5040. 2019.”, and Sauer, A. et al. in their paper “Projected gans converge faster. Advances in Neural Information Processing Systems (NeurIPS). pp: 17480–17492 2021." The method based on generative adversarial network ProjectedGAN, the method based on diffusion-denoising model LiDARGen proposed by Zyrianov, V. et al. in their published paper "Learning to generate realistic lidar point clouds. European Conference on Computer Vision (ECCV). pp. 17–35. 2022.", and R2DM proposed by Nakashima, K. et al. in their published paper "Lidar data synthesis with denoising diffusion probabilistic models. arXiv preprint arXiv: 2309.09256 (2023)". The quantitative results are listed in Table 1. The intersection-union ratio of the above evaluation indicators is used as the evaluation criterion, and the values ​​of all evaluation indicators are as small as possible.

[0098] Table 1 Quantitative test results on KITTI360 using the method of the present invention and the prior art

[0099] method FPD FRD MMD JSD LiDARGAN - 3003.8 30.60 - LiDARVAE - 2261.5 10.00 16.10 ProjectedGAN - 2117.2 3.47 8.50 LiDAR Gen 90.29 579.39 7.39 7.38 R2DM 6.24 149.66 1.91 3.05 Method of the present invention 6.01 166.29 0.96 2.63

[0100] From the results in Table 1, it can be seen that the method of the present invention achieves the best results in three of the four indicators, among which FRD achieves the second best result, indicating that the lidar point cloud data generated by the present invention has the best authenticity and diversity. Figure 5 As shown, the laser radar point cloud data generated by the method of the present invention is more detailed than the existing method, and the target is generated more clearly.

[0101] The above embodiments should be regarded as illustrative and non-restrictive, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the invention and within the spirit and principles of the present invention should be covered by the protection scope of the present invention.

Claims

1. A method for generating laser radar point cloud based on isometric projection transformer, characterized in that: The following steps are included: Step 1: Collect the lidar point cloud of the outdoor autonomous driving scene and project it into an isometric projection image of h×w×c. According to the length, width and number of channels of the isometric projection image, generate the corresponding Fourier position feature, and splice the Fourier position feature with the isometric projection image according to the channel dimension; Step 2: The isometric projection image obtained after data processing is sent to the isometric projection transformer denoising network to obtain the isometric image projection features and the inverse isometric image projection features; the denoising network includes the isometric projection attention mechanism EA and the inverse isometric projection attention mechanism REA, which fully considers the physical characteristics of the isometric projection image and captures the isometric projection image features in a global modeling manner; Step 3: Send the time step control signal to the control signal embedder CEI, and fuse the control signal and the denoising features extracted in step 2 through the global attention mechanism; Step 4, construct a frequency domain modulator FM to enhance the low-frequency part of the over-smoothed equidistant projection image to obtain the final equidistant projection image; Step 5: Use the two-norm loss function to supervise the isometric projection image obtained in step 4, perform an end-to-end network learning process, and finally obtain a lidar point cloud generation model.

2. The method for generating laser radar point cloud based on isometric projection transformer according to claim 1, characterized in that: In step 1, Fourier position features need to be generated and fused. First, the collected laser radar point cloud x 0 The Fourier coordinate features are integrated in the convolution operation to map the coordinates from the three-dimensional space to the two-dimensional space at a high frequency to obtain the initial coordinate feature p. The steps are as follows: Among them, c represents the coordinates in the initial equidistant projection image, * represents the convolution operation, and arrange(Δ) represents taking out integers from 0 to Δ; then, Fourier mapping is used to obtain the Fourier position feature f, and the steps are as follows: f=[sin(p),cos(p),sin(2p),cos(2p),…,sin(2 m-1 p),cos(2 m-1 p)] T #(2) Where sin represents the sine function, cos represents the cosine function, and m represents the frequency adjustment parameter. After obtaining the Fourier feature f, it is compared with the input equidistant projection feature x 0 Splicing is done according to the channel dimension to enhance the high-frequency response in the feature extraction process and overcome the transition smoothness problem. The steps are as follows: in, Represents splicing by channel dimension.

3. The method for generating laser radar point cloud based on isometric projection transformer according to claim 1, characterized in that: In step 2, the isometric projection image obtained in step 1 is used to extract the isometric projection image features by the isometric projection attention mechanism. And use the inverse equidistant projection attention mechanism to upsample and obtain the inverse equidistant projection feature The isometric projection attention mechanism and the reverse isometric projection attention mechanism described in step 2 are both composed of the self-attention mechanism and the cross-attention mechanism. Represents x i and The eigenvalues ​​in are real numbers, h×w×c represents the extracted feature resolution, and c is the feature dimension.

4. The method for generating laser radar point cloud based on isometric projection transformer according to claim 1 or 3, characterized in that: In the isometric projection transformer denoising network described in step 2: (1) Isometric projection attention mechanism for global downsampling: This part accepts isometric projection image features x 0 As input, we output the equidistant projection image features x={x 1 ,x 2 ,x 3 ,x 4 }; The specific implementation process is: using the stacking of isometric projection attention mechanism to process the input isometric projection image, the steps are: Among them, EA stands for isometric projection attention mechanism, Selfattention stands for self-attention mechanism, and Unflod stands for stacking operation; (2) Inverse isometric projection attention mechanism for global upsampling: This part accepts the downsampled features x of the isometric projection image i As input, 4 inverse isometric projection image features are output, and the steps are: Among them, REA stands for the inverse equidistant projection attention mechanism, CEI stands for the control signal embedder, Flod stands for the unfolding operation, and ts stands for the time step sequence feature, that is, the time step control signal.

5. The method for generating laser radar point cloud based on isometric projection transformer according to claim 1, characterized in that: In the control signal embedder described in step 3, the main sequence features of the inverse equidistant projection image features obtained in the previous step are The main sequence features d and time step sequence features ts separated from the original image are first concatenated for self-attention operation, and then the new sequence is split in the order of concatenation to form a new query feature sequence Qd, key sequence Kts and value sequence Vts; the final inverse isometric projection image features are obtained by performing cross-attention operations, matrix multiplication and addition operations on the feature sequence. The steps are as follows: Among them, Crossattention represents the cross attention mechanism. Represents matrix multiplication.

6. The method for generating laser radar point cloud based on isometric projection transformer according to claim 1, characterized in that: The step 4 constructs a frequency domain modulator, and the specific implementation process is: Among them, FM stands for frequency domain modulator, IDWT stands for discrete wavelet transform, DWT is the inverse discrete wavelet transform, and ⊙ represents the Hadamard product.

7. The method for generating laser radar point cloud based on isometric projection transformer according to claim 1, characterized in that: The loss function used in the end-to-end network learning process is: Among them, Net represents the isometric projection transformer denoising network, and t represents the current denoising step; the loss function value is calculated, and the network parameters in the isometric projection denoising transformer are updated using the gradient descent method until the model converges to obtain a trained lidar point cloud generation model.

Images