Laser radar three-dimensional target detection method based on Coarse Zhengji network

Abstract

The invention provides a laser radar three-dimensional target detection method based on a coarse-to-mental network, and the method comprises the steps: firstly constructing a coarse positioning network based on a Transform network structure, and enabling the coarse positioning network to be used for positioning an interval to which an effective echo pulse signal belongs in one-dimensional time; and a fine positioning network based on a coding-decoding structure is further designed, and pulse peak positioning with higher time resolution is carried out on a high signal-to-noise ratio signal interval extracted by the coarse positioning network. The coarse positioning network of the method can eliminate noise signal segments irrelevant to effective echo pulses to the maximum extent, so that the influence of noise on the fine positioning network is reduced, and the signal-to-noise ratio of echo data and the three-dimensional target detection precision of the system are improved.

Description

Technical Field

[0001] This invention belongs to the field of target recognition and detection, specifically relating to a three-dimensional target detection method for lidar based on a coarse-to-fine neural network. Background Technology

[0002] In a LiDAR imaging system, a laser emits laser pulses of a specific wavelength to illuminate a target area. The echo signal reflected from the target area is then received by an array photodetector, which converts the optical signal into an electrical signal. The system obtains the time of flight of the laser pulse by analyzing the time difference between the echo pulse's electrical signal and the emitted pulse, and finally calculates the target distance based on the speed of light. LiDAR can simultaneously detect the reflectivity and three-dimensional structural information of a target area. Compared to visible light imaging and microwave radar detection technologies, it has advantages such as high accuracy in three-dimensional target detection, fast dynamic response speed, and strong anti-interference capability, making it widely applicable in remote sensing measurement, concealed target identification, and precision guidance in military battlefields.

[0003] LiDAR systems analyze echo signal data to obtain the time difference between laser pulse emission and reception, thereby measuring the distance information of the target scene to complete the three-dimensional detection of the target. However, in situations involving long-range target detection and weak echo signals from small targets, the signal-to-noise ratio of the reflected echo signal is extremely low due to environmental background noise and system hardware noise. Furthermore, the signal waveform deviates from the ideal distribution, making it difficult for the detection system to determine the arrival time of the echo pulse, posing a significant challenge to subsequent accurate three-dimensional target detection.

[0004] To achieve accurate 3D detection of distant, weak targets using lidar, this invention proposes a lidar 3D target detection method based on a coarse-to-fine neural network. This method first constructs a coarse localization network based on a Transformer network structure to locate the effective echo pulse signal interval in one-dimensional time. Then, a fine localization network based on an encoder-decoder structure is designed to perform higher time-resolution pulse peak localization on the high signal-to-noise ratio (SNR) signal interval extracted by the coarse localization network. The coarse localization network of this method can maximally eliminate noise signal segments unrelated to the effective echo pulse, thereby reducing the impact of noise on the fine localization network and improving the SNR of the echo data and the system's 3D target detection accuracy. Summary of the Invention

[0005] To achieve accurate 3D detection of distant, weak targets using lidar and reduce the interference of invalid noise on the feature fitting process of neural networks, this invention proposes a lidar 3D target detection method based on a coarse-to-fine neural network. This method first constructs a coarse localization network based on a Transformer network structure. Utilizing the global feature extraction capability of the Transformer network, it mines non-local spatiotemporal features in the time arrival data matrix, thereby accurately locating the effective echo pulse signal within its one-dimensional time domain. Subsequently, by truncating the located echo pulse interval, an effective time arrival data matrix is ​​constructed, eliminating the interference of invalid noise in the time channel and improving the signal-to-noise ratio (SNR) of the echo pulses. Further, a fine localization network based on an encoder-decoder structure is designed to perform higher time-resolution pulse peak localization on the high SNR effective time arrival data matrix extracted by the coarse localization network. In the coarse-to-fine echo pulse peak arrival time localization process of this method, the coarse localization network can eliminate noise signal segments unrelated to the effective echo pulses to the greatest extent, thereby reducing the impact of noise on the feature fitting process of the fine localization network and improving the echo pulse peak localization accuracy of the fine localization network. Finally, using the laser pulse time-of-flight formula, the prediction output of the coarse positioning network is transformed into low-range-resolution 3D scene information, and the prediction output of the fine positioning network is transformed into high-range-resolution 3D scene information. The two resolutions of 3D scene information are merged to restore the high-precision 3D information of the detected scene, thereby improving the 3D target detection accuracy of the lidar system.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A three-dimensional target detection method for lidar based on coarse-to-fine neural networks includes the following steps:

[0008] (1) Coarse localization network determines the time interval of the echo pulse: The coarse localization neural network consists of a local feature embedding module and a Transformer-based feature embedding module. The local feature embedding module is used to obtain local detailed features of the echo data, while the Transformer-based feature embedding module is used to extract non-local spatiotemporal features. The input of the coarse localization neural network is the target reflected echo pulse signal containing a large amount of noise. It also outputs the localization probability tensor of the region containing the valid signal. In this output tensor, elements This represents the probability that the echo pulse in the (i,j)th pixel channel falls within the interval.

[0009] (2) Constructing effective echo pulse arrival matrix data: based on the classification results of the coarse localization network CILN. The truncation operation takes the data matrix with noise as it arrives. The effective echo pulse signal segment is extracted, and then the effective time arrival data matrix is ​​constructed. Specifically, for the (i,j)th pixel, if the localization interval classification result predicted by the coarse localization network is k, then (k-1)T c To kT c The signal data in the time interval will be from P r Extract from the (i,j)th pixel to construct the effective time arrival data matrix. The corresponding pixels. Therefore, the effective time arrives at the data matrix. The dimensions are (H, W, T) c Moreover, relative to the original time arrival data matrix P r Theoretically, the effective time to arrive at the data matrix The time dimension contains a higher proportion of echo pulse data.

[0010] (3) The fine localization network locates the peak arrival time of the echo pulse: The fine localization network consists of an encoder for deep feature extraction and a decoder for parsing high-dimensional embedded features. The input to the fine localization network is the effective echo pulse data with a higher signal-to-noise ratio extracted by the coarse localization network. Its output is a peak localization tensor with higher temporal resolution. In this output tensor, elements This represents the probability that the echo pulse peak is located at the time point at pixel position (i,j).

[0011] (4) 3D scene data reconstruction: based on the predicted output values ​​of the coarse localization network The predicted output value of the fine localization network The reconstruction of 3D detection data can be achieved using the following formula. Here, (i, j) represents the pixel index of the sensor array; c represents the speed of light; Δt represents the time interval of the discrete time-domain signal; Argmar(*) represents the operation of taking the maximum index value; and K represents the length of the time-domain signal truncated by the truncation operation. This formula shows that the prediction result of the fine localization network is essentially a higher-resolution detail supplement to the output result of the coarse localization network. The distance resolution of the 3D data of the coarse localization network is 2cΔtK, while the distance resolution of the fine localization network is 2cΔt, the latter having a 3D spatial resolution several times higher than the former.

[0012]

[0013] The advantages of this invention compared to the prior art are:

[0014] 1. The method of the present invention eliminates signal segments containing invalid noise through a coarse positioning network, improves the signal-to-noise ratio of the echo data, reduces the interference of noise on the depth feature extraction of the fine positioning network, ensures the accuracy of its prediction of the arrival time of the echo signal, and thus improves the three-dimensional target detection accuracy of the lidar system on low signal-to-noise ratio target reflection echo data.

[0015] 2. A Transformer network structure is introduced, and this invention constructs a coarse localization network based on Transformer. By leveraging the self-attention mechanism of Transformer, the global receptive field in the spatiotemporal domain of the lidar echo signal is obtained, thereby more effectively focusing on the non-local spatial and temporal features in the echo signal data and achieving accurate localization of the time interval where the echo pulse is located.

[0016] 3. A fine-localization network based on an encoder-decoder structure was constructed. This invention utilizes the high-dimensional feature embedding capability of the encoder-decoder network structure to achieve depth feature extraction and peak localization processing of the lidar echo time-domain signal. In the coarse-to-fine network structure configuration, the Transformer-based coarse-localization network eliminates a large proportion of invalid noise information in the time domain, making the feature fitting process of the fine-localization network more defined by the effective echo pulses. This allows it to more accurately locate the peak time of the echo pulses, thereby achieving accurate three-dimensional target detection.

[0017] 4. This invention introduces a positional encoding mechanism, which can effectively encode and embed the feature distribution change information in the time dimension caused by the coarse localization network's truncation operation into the high-dimensional feature space of the fine localization network. This ensures that its three-dimensional convolutional kernel can accurately extract spatiotemporal correlation features, further improving the feature extraction capability and peak localization accuracy of the fine localization network.

[0018] 5. This invention constructs a temporal attention mechanism to guide the fine localization network to pay more attention to the feature information of effective echo pulses, avoid interference from truncated invalid noise in its feature extraction process, and improve the robustness of its feature extraction. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is a diagram of the coarse localization network structure according to an embodiment of the present invention;

[0021] Figure 2 This is a diagram of the fine localization network structure according to an embodiment of the present invention;

[0022] Figure 3This is a diagram of the coarse-to-fine three-dimensional target detection network structure of a lidar system according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] (1) Design of coarse localization network based on Transformer

[0025] The specific architecture of the coarse localization network is shown in the attached figure. Figure 1 As shown, it mainly consists of a local feature embedding module and a Transformer-based feature embedding module. The local feature embedding module aims to extract local detailed features of the captured time arrival data matrix, while the Transformer-based feature embedding module is responsible for extracting non-local spatiotemporal features.

[0026] As attached Figure 1 As shown, the local feature embedding module includes a 3D convolutional layer, a max pooling layer that downsamples the temporal channels by a factor of 2, and N C There are 10 basic feature embedding modules. Among these basic modules, the last N... C Each module downsamples the time channels by a factor of 2. Simultaneously, 3D convolutional kernels are used in the basic feature embedding module to extract local features from the time-arrival data matrix. This module processes the input tensor. Output high-dimensional embedded local features Where C e This indicates the dimensionality of the embedded feature.

[0027] The Transformer-based feature embedding module processes the high-dimensional embedded local features output by the local feature embedding module. Output effective interval location probability tensor As attached Figure 1 As shown, it processes the input high-dimensional local features F through a 2D spatial downsampling layer. e This yielded local features with a spatial resolution reduced by a factor of 8. To reduce computational complexity, the second stage of this module then processes the local features F through four Transformer layers. e0 To extract its non-local spatiotemporal features, the Transformer layer based on the self-attention mechanism first processes F... inObtain the normalized attention matrix In this process, the Q, key K, and value V matrices are queried through F respectively. in The linear transformation is obtained as shown in the following formula:

[0028]

[0029] In the formula, W q W k , and W v All are learnable linear transformation layers; N represents the total number of features across all time and spatial locations; C e This represents the feature embedding dimension. Subsequently, the query matrix Q and the transpose key matrix K are used... T The attention matrix can be obtained by performing matrix multiplication. in, Let this be the attention matrix. To ensure the numerical stability of the computation, the attention matrix needs to be optimized. Normalization is then performed. This normalization process follows the formula below:

[0030]

[0031] In the formula, α i，j This represents the normalized weights in the attention matrix. Subsequently, the Transformer layer uses the normalized attention matrix A and the value matrix V to perform feature aggregation, obtaining self-attention features. Among them, F sa The calculation process for each row vector is as follows: Representing F sa The i-th row vector; the coefficients α of the normalized attention weight matrix A. i，k This represents the importance of the feature vector at position k to the feature vector at position i; The value feature represents the k-th position; the value matrix V contains feature vectors for all temporal and spatial locations, therefore the self-attention mechanism has a global receptive field capable of extracting non-local spatial and temporal correlation features. Finally, the self-attention feature F sa After processing by the forward propagation layer, and combined with the input feature F in The residual connections are summed to generate the output features of the Transformer layer. This process follows the formula: F out =LBR(F sa )+F in In the formula, LBR(*) represents the forward propagation layer. The four Transformer layers sequentially output four different semantic levels of features F. e1 F e2 F e3 and Fe4 And all dimensions are With F e1 To F e4 In this transition, the proportion of local information gradually decreases, while the proportion of non-local spatiotemporal features increases. Features at four different semantic levels are processed by a multi-level feature fusion and upsampling module to generate an effective interval localization probability tensor. In this tensor, elements This indicates that among the echo signals of (i, j) pixel channels, the effective echo pulses are located in the interval [(k-1)T]. C ,kT C The probability of ].

[0032] (2) Design of loss function for coarse localization network

[0033] The truth classification label t of the coarse localization network C It can be obtained through the following formula:

[0034]

[0035] Where R represents the true depth of the target region; Δt represents the temporal resolution; c represents the speed of light; N c This parameter represents the number of downsampling layers in the time dimension during feature extraction in the local feature embedding module. It determines the number of classification categories K in the coarse-range localization network. During the training of the coarse-range localization network, the cross-entropy loss function is used to measure the distribution difference between the model's predicted results and the ground truth labels. Specifically, the ground truth label t... C Transformed into one-hot encoding strategy Subsequently, the coarse positioning network predicts the effective range positioning probability. and truth labels The cross-entropy loss between them is calculated using the following formula:

[0036]

[0037] To ensure spatial smoothness of the classification results, the following total variation loss is introduced:

[0038]

[0039] In the calculation of total variation loss, since the argmax function is not differentiable, the softargmax function is used to generate C. i,j The coarse localization network approximates the classification result with the highest probability. Ultimately, the optimization objective of the coarse localization network is a weighted sum of cross-entropy and total variation loss, which follows this formula:

[0040]

[0041] As attached Figure 3 As shown, based on the classification results of the coarse localization network The truncation operation takes the data matrix with noise as it arrives. Extract the effective echo pulse signal segments and then construct the effective time arrival data matrix. For the (i,j)th pixel, if the coarse localization network predicts the localization interval classification result as k, then (k-1)T c To kT c The signal data in the time interval will be from P r Extract from the (i,j)th pixel to construct the effective time arrival data matrix. The corresponding pixels. Therefore, the effective time arrives at the data matrix. The dimensions are (H, W, T) c Moreover, relative to the original time arrival data matrix P r Theoretically, the effective time to arrive at the data matrix The time dimension contains a higher proportion of echo pulse data.

[0042] (3) Design of fine localization network based on encoder-decoder structure

[0043] This invention constructs a precise positioning network to locate the peak position of echo pulses in the effective time arrival data matrix, thereby determining the pulse's flight time and completing three-dimensional detection. (See attached diagram) Figure 2 As shown, the network mainly consists of an encoder responsible for deep feature extraction and a decoder responsible for parsing high-dimensional embedded features, with the input being a data matrix that arrives at the specified time. Output peak localization probability tensor In this output tensor, elements This represents the probability that the peak of the echo pulse is located at time point k at pixel position (i, j).

[0044] As attached Figure 2 As shown, the encoder primarily employs a lightweight residual connection structure for deep feature extraction. The encoder's structural design is similar to the local feature embedding module of the coarse localization network, which embeds the input tensor... Encoding to high-dimensional embedding features The decoder is constructed by alternating stacks of 3D deconvolutional layers and ReLU activation layers, which parses and upsamples the high-dimensional embedding features F from the encoder output. e This leads to the output of the predicted peak localization probability tensor. In this tensor, elements This represents the probability that the peak of the echo pulse is located at time point k at pixel position (i, j).

[0045] Furthermore, this invention introduces a position encoding module and a temporal attention module between the convolutional layers of the fine localization network, enabling it to output more accurate echo signal arrival times. The former effectively transmits the position information predicted by the coarse localization network to the fine localization network, ensuring that it can identify echo pulse position changes caused by truncation operations; while the latter guides the fine localization network to prioritize effective echo pulse information, further improving its feature extraction and peak localization accuracy.

[0046] Extracting a noisy time-arrival data matrix into a valid one alters the feature distribution of the original signal along the time dimension; that is, the signal, originally widely distributed along the time dimension, is compressed. If the fine localization network fails to correctly perceive the impact of the truncation operation on the feature distribution along the time dimension, the spatiotemporal correlation features extracted by its 3D convolutional kernels will be interfered with. Therefore, this invention constructs a position encoding module, enabling the fine localization network to correctly perceive the changes in feature distribution along the time dimension of the time-arrival data matrix caused by the truncation operation. The input to the position encoding module is the classification result output by the coarse localization network. in, This represents the classification probability output by the coarse localization network, and its size is... And F CR The classification result is of size (). The position encoding module contains a 3D convolutional layer with kernel size (3,3,1) and stride (1,1,1) to encode the classification result F. CR It is effectively embedded into the high-dimensional feature space of the fine localization network. Therefore, the position encoding module outputs a position encoding feature F of size (C, H, W, 1). PE , where C represents the dimension of the high-dimensional feature space. (See attached image) Figure 2 As shown, the location coding feature F PE The truncation operation directly adds and fuses the input features from each convolutional layer in the encoder and decoder to effectively transmit positional change information caused by the truncation operation. This is because the truncation operation is based on the interval localization result F output by the coarse localization network. CR And the execution of this positioning result F CR This reflects the changes in feature distribution caused by the truncation operation over the time channel. Based on this, the location encoding module receives the location result F. CR As input, this effectively transmits the information on the changes in feature distribution in the temporal dimension caused by the truncation operation to the high-dimensional feature space of the fine localization network, ensuring that its three-dimensional convolutional kernel can accurately extract spatiotemporal related features.

[0047] This invention also constructs a temporal attention module to guide the fine-localization network to focus more on the feature information of effective echo pulses. (See attached diagram) Figure 2 As shown, the module first places an F with dimensions (c, h, w, t) inThe tensor is reshaped into a tensor of size (c,t,h,w), and then processed through three 2D convolutional layers to obtain tensor F4 of size (c,t,h,w). Subsequently, by performing a sigmoid normalization operation on F4 in the time dimension, the temporal attention weight tensor F is obtained. score And resize it to (c,h,w,t). Through F score and F in The element-wise multiplication operation ultimately yields the output feature F of the temporal attention module. out =F score ·F in The temporal attention module outputs features F. out The size is (c,h,w,t), and the input tensor F in The dimensions are completely consistent. This module uses 2D convolution kernels with a kernel size of (1,1) to treat the temporal channel of the tensor as the feature channel of a two-dimensional convolution operation, thereby achieving deep feature extraction from 1D temporal signals. Its acquired attention weights F... score Each element is a quantitative indicator of attention to different time channels learned by the neural network, based on the distribution characteristics of 1D time-domain signals.

[0048] (4) Design of loss function for fine localization network

[0049] The truth classification label t of the fine localization network F It can be obtained through the following formula:

[0050]

[0051] in, R represents the probability of the effective interval predicted by the coarse localization network; Δt represents the ground truth depth of the target region; c represents the speed of light; N represents the distance between the predicted intervals. c This represents the number of downsampling layers in the time dimension during feature extraction in the local feature embedding module. During the training of the fine localization network, the cross-entropy loss function is used to measure the distributional difference between the model's predicted results and the ground truth labels. The ground truth value t F Re-encoded using a one-hot encoding strategy Peak localization probability of output and truth labels The cross-entropy loss between them is calculated using the following formula:

[0052]

[0053] To ensure the spatial smoothness of the classification results of the precise maximum localization network, the following total variation loss is introduced:

[0054]

[0055] Ultimately, the optimization objective of the fine-localization network is a weighted sum of cross-entropy and total variation loss, which follows the formula:

[0056]

[0057] (5) Design of a coarse-to-fine lidar three-dimensional target detection network

[0058] In lidar echo signal data, the effective echo pulses occupy a small portion of the time domain, with the remainder containing only invalid noise information. Existing lidar 3D target detection neural networks process signals according to the following formula.

[0059]

[0060] In the formula, f(⊙|θ) is the neural network model, θ represents the parameters to be optimized, and P r The signal is a noisy lidar echo. The target reflection echo signal is denoised by the neural network, where c represents the speed of light and Argmax(*) is the peak localization operator. However, the above processing method extracts features in the entire time domain of the target reflection echo signal, resulting in the network learning features being mainly defined by noise signals, thus reducing the accuracy of the system's 3D target detection.

[0061] To address the above issues, a three-dimensional target detection method for lidar based on a coarse-to-fine cascaded neural network was constructed by concatenating a coarse-to-fine localization neural network. The processing of the target area echo data received by the lidar follows the formula below.

[0062]

[0063] First, the coarse localization network Coarse(⊙|θ) C According to the input signal P r The output variable is used to predict the time interval of the effective echo pulse. This represents the probability that the valid echo pulse of the (i, j)th pixel channel is located in the kth time interval. Subsequently, the truncation operation Crop(*) is performed based on the localization probability output by the coarse localization network. From the original signal P r The effective echo pulse time interval is extracted to construct effective echo pulse data with a smaller noise ratio and a higher signal-to-noise ratio. Furthermore, the fine-localization network Fine(⊙|θ) F For valid echo pulse data Perform peak localization processing with higher temporal resolution to output the predicted echo pulse arrival time. Complete noise signal P r To no noise signal This conversion enables denoising of lidar echo data. Through the prediction results... By executing the Argmax(*) maximum index positioning operation, the lidar system can accurately determine the arrival time of the echo pulse, thereby achieving accurate three-dimensional target detection.

[0064] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional target detection method for lidar based on coarse-to-fine neural networks, characterized in that, Includes the following steps: (1) Design of a coarse localization network based on Transformer; (2) Design of loss function for coarse localization network; (3) Design of a fine localization network based on an encoder-decoder structure; (4) Design of loss function for fine localization network; (5) Design of a coarse-to-fine lidar three-dimensional target detection network.

2. The method for three-dimensional target detection using lidar based on a coarse-to-fine neural network according to claim 1, characterized in that, The specific architecture of the coarse localization network mainly consists of a local feature embedding module and a Transformer-based feature embedding module. The local feature embedding module aims to extract local detailed features of the capture time arrival data matrix, while the Transformer-based feature embedding module is responsible for extracting non-local spatiotemporal features.

3. The method for three-dimensional target detection of lidar based on a coarse-to-fine neural network according to claim 2, characterized in that, The local feature embedding module includes a 3D convolutional layer, a max pooling layer that downsamples the temporal channels by a factor of 2, and N C There are 10 basic feature embedding modules; among these basic modules, the last N... C -1 modules all perform a 2x downsampling of the time channel. Meanwhile, the 3D convolution kernel is used in the basic feature embedding module to extract local features from the time arrival data matrix. This module processes input tensors. Output high-dimensional embedded local features Where C e This indicates the dimensionality of the embedded feature.

4. The method for three-dimensional target detection using lidar based on a coarse-to-fine neural network according to claim 2, characterized in that, Based on the feature embedding module of the Transformer, the high-dimensional embedded local features output by the local feature embedding module are processed. Output effective interval location probability tensor It processes the input high-dimensional local features F through a 2D spatial downsampling layer. e This yielded local features with a spatial resolution reduced by a factor of 8. The second stage of this module then processes the local feature F through four Transformer layers. e0 To extract its non-local spatiotemporal features; the Transformer layer based on the self-attention mechanism first processes F in Obtain the normalized attention matrix In this process, the Q, key K, and value V matrices are queried through F respectively. in Obtained by linear transformation.

5. The method for three-dimensional target detection of lidar based on a coarse-to-fine neural network according to claim 4, characterized in that, The specific formula for the linear transformation is as follows: In the formula, W q W k , and W v All are learnable linear transformation layers; N represents the total number of features across all time and spatial locations; C e Indicates the feature embedding dimension; Subsequently, by querying matrix Q and transpose key matrix K T The attention matrix can be obtained by performing matrix multiplication. in, This is the attention matrix.

6. The method for three-dimensional target detection of lidar based on a coarse-to-fine neural network according to claim 5, characterized in that, To ensure the numerical stability of the computation, the attention matrix needs to be adjusted. Normalization is performed; this normalization process follows the formula below: In the formula, α i,j This represents the normalized weights in the attention matrix.

7. The method for three-dimensional target detection using lidar based on a coarse-to-fine neural network according to claim 6, characterized in that, Subsequently, the Transformer layer uses the normalized attention matrix A and the value matrix V to perform feature aggregation processing to obtain self-attention features. Among them, F sa The calculation process for each row vector is as follows: Representing F sa The i-th row vector; the coefficients α of the normalized attention weight matrix A. i,k This represents the importance of the feature vector at position k to the feature vector at position i; The value feature represents the k-th position; the value matrix V contains feature vectors of all time and spatial positions, so the self-attention mechanism has a global receptive field that can extract non-local spatial and temporal correlation features.

8. The method for three-dimensional target detection of lidar based on a coarse-to-fine neural network according to claim 7, characterized in that, Finally, the self-attention feature F sa After processing by the forward propagation layer, and combined with the input feature F in The residual connections are summed to generate the output features of the Transformer layer. The process follows the formula: F out =LBR(F sa )+F in In the formula, LBR(*) represents the forward propagation layer; the four Transformer layers sequentially output four different semantic levels of features F. e1 F e2 F e3 and F e4 And all dimensions are With F e1 To F e4 During the transition, the proportion of local information gradually decreases, while the proportion of non-local spatiotemporal features increases; features at four different semantic levels are processed by a multi-level feature fusion and upsampling module to generate an effective interval localization probability tensor. In this tensor, elements This indicates that among the echo signals of (i, j) pixel channels, the effective echo pulses are located in the interval [(k-1)T]. C kT C The probability of ].

9. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to any one of claims 1-8, characterized in that, The truth classification label t of the coarse localization network C It can be obtained through the following formula: Where R represents the true depth of the target region; Δt represents the temporal resolution; c represents the speed of light; N c This parameter represents the number of time-dimension downsampling layers during the feature extraction process of the local feature embedding module. This parameter determines the number of classification categories K in the coarse-area localization network.

10. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to claim 9, characterized in that, During the training of the coarse localization network, the cross-entropy loss function is used to measure the distributional difference between the model's predicted results and the ground truth labels; specifically, the ground truth label t C Transformed into one-hot encoding strategy Subsequently, the coarse positioning network predicts the effective range positioning probability. and truth labels The cross-entropy loss between them is calculated using the following formula:

11. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to claim 10, characterized in that, To ensure spatial smoothness of the classification results, the following total variation loss is introduced: In the calculation of total variation loss, since the argmax function is not differentiable, the softargmax function is used to generate C. i，j The coarse localization network approximates the classification result with the highest probability. Ultimately, the optimization objective of the coarse localization network is a weighted sum of cross-entropy and total variation loss, which follows this formula:

12. The method for three-dimensional target detection of lidar based on a coarse-to-fine neural network according to claim 11, characterized in that, Based on the classification results of the coarse localization network The truncation operation takes the data matrix with noise as it arrives. Extract the effective echo pulse signal segments and then construct the effective time arrival data matrix. For the (i, j)th pixel, if the coarse localization network predicts the localization interval classification result as k, then (k-1)T c To kT c The signal data in the time interval will be from P r Extract from the (i, j)th pixel to construct the effective time arrival data matrix. The corresponding pixel; therefore, the effective time arrives at the data matrix. The dimensions are (H, W, T) c Moreover, relative to the original time arrival data matrix P r Theoretically, the effective time to reach the data matrix The time dimension contains a higher proportion of echo pulse data.

13. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to any one of claims 1-12, characterized in that, A fine positioning network is constructed to locate the peak position of the echo pulse in the effective time arrival data matrix, so as to determine the flight time of the pulse and thus complete the three-dimensional detection.

14. The method for three-dimensional target detection of lidar based on a coarse-to-fine neural network according to claim 13, characterized in that, The network mainly consists of an encoder responsible for deep feature extraction and a decoder responsible for parsing high-dimensional embedded features, with the input being a data matrix arriving at the effective time. Output peak localization probability tensor In this output tensor, elements This represents the probability that the peak of the echo pulse is located at time point k at pixel position (i, j).

15. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to claim 14, characterized in that, The encoder primarily employs a lightweight residual connection structure for deep feature extraction; its structural design is similar to the local feature embedding module of the coarse localization network, and it uses the input tensor... Encoding to high-dimensional embedding features The decoder is constructed by alternating stacks of 3D deconvolutional layers and ReLU activation layers, parsing and upsampling the high-dimensional embedding features F output by the encoder. e This leads to the output of the predicted peak localization probability tensor. In this tensor, elements This represents the probability that the peak of the echo pulse is located at time point k at pixel position (i, j).

16. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to claim 15, characterized in that, A position encoding module and a temporal attention module are introduced between the convolutional layers of the fine localization network to enable it to output more accurate echo signal arrival times. The position encoding module effectively transmits the position information predicted by the coarse localization network to the fine localization network, ensuring that it can identify echo pulse position changes caused by truncation operations. The temporal attention module guides the fine localization network to prioritize effective echo pulse information, further improving its feature extraction and peak localization accuracy.

17. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to claim 16, characterized in that, The input to the location encoding module is the classification result output by the coarse localization network: in, This represents the classification probability output by the coarse localization network, and its size is... And F CR The classification result is of size (H, W); the position encoding module includes a 3D convolutional layer with kernel size (3,3,1) and stride (1,1,1) to encode the classification result F. CR Effectively embedded into the high-dimensional feature space of the fine localization network; the position encoding module outputs a position encoding feature F of size (C, H, W, 1). PE Where C represents the dimension of the high-dimensional feature space; the position encoding feature F PE The truncation operation directly adds and fuses the input features of each convolutional layer in the encoder and decoder to effectively transmit the positional change information caused by the truncation operation; since the truncation operation is based on the interval localization result F output by the coarse localization network... CR And the execution of this positioning result F CR This reflects the changes in feature distribution caused by the truncation operation over the time channel; based on this, the location encoding module receives the positioning result F. CR As input, this effectively transmits the feature distribution change information in the time dimension caused by the truncation operation to the high-dimensional feature space of the fine localization network, ensuring that its three-dimensional convolutional kernel can accurately extract spatiotemporal correlation features.

18. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to claim 17, characterized in that, A temporal attention module was also constructed to guide the fine-localization network to focus more on the feature information of effective echo pulses; the temporal attention module first applies an F-type vector of size (c,h,w,t) to the localization network. in The tensor is reshaped into a tensor of size (c,t,h,w), and then processed through three 2D convolutional layers to obtain a tensor F4 of size (c,t,h,w). Subsequently, by performing a sigmoid normalization operation on F4 in the time dimension, the temporal attention weight tensor F is obtained. score and resize it to (c, h, w, t); via F score and F in The element-wise multiplication operation ultimately yields the output feature F of the temporal attention module. out =F score ·F in The output feature F of the time attention module out The dimensions are (c, h, w, t), and the input tensor F in The dimensions are completely consistent; this module uses 2D convolution kernels with a kernel size of (1,1) to treat the temporal channel of the tensor as the feature channel of a two-dimensional convolution operation, thereby realizing the extraction of deep features from 1D temporal signals; its acquired attention weight F score Each element is a quantitative indicator of attention to different time channels learned by the neural network, based on the distribution characteristics of 1D time-domain signals.

19. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to any one of claims 1-17, characterized in that, The truth classification label t of the fine localization network F It can be obtained through the following formula: in, R represents the probability of the effective interval predicted by the coarse localization network; Δt represents the ground truth depth of the target region; c represents the speed of light; N represents the distance between the predicted intervals. c This indicates the number of time-dimension downsampling layers during the feature extraction process in the local feature embedding module; During the training of the precise localization network, the cross-entropy loss function is used to measure the distribution difference between the model's predicted results and the ground truth labels; the ground truth value t F Re-encoded using a one-hot encoding strategy Output peak localization probability and truth labels The cross-entropy loss between them is calculated using the following formula: To ensure the spatial smoothness of the classification results of the precise maximum localization network, the following total variation loss is introduced: Ultimately, the optimization objective of the fine-localization network is a weighted sum of cross-entropy and total variation loss, which follows the formula:

20. A three-dimensional target detection method for lidar based on a coarse-to-fine neural network according to any one of claims 1-18, characterized in that, A three-dimensional target detection method for lidar based on a coarse-to-fine cascaded neural network was constructed by cascading a coarse-to-fine localization neural network. The processing of the target area echo data received by the lidar follows the formula below. First, the coarse localization network Coarse(⊙|θ) C According to the input signal P r The output variable is used to predict the time interval of the effective echo pulse. This represents the probability that the valid echo pulse of the (i, j)th pixel channel is located in the kth time interval; subsequently, the truncation operation Crop(*) is performed based on the localization probability output by the coarse localization network. From the original signal P r The effective echo pulse time interval is extracted to construct effective echo pulse data with a smaller noise ratio and a higher signal-to-noise ratio. Furthermore, the fine-localization network Fine(⊙|θ) F For valid echo pulse data Perform peak localization processing with higher temporal resolution to output the predicted echo pulse arrival time. Complete noise signal P r To no noise signal The conversion is used to achieve noise reduction processing of lidar echo data; through the prediction results... By executing the Argmax(*) maximum index positioning operation, the lidar system can accurately determine the arrival time of the echo pulse, thereby achieving accurate three-dimensional target detection.

Images