A laser radar target detection method, system, terminal and storage medium

Abstract

The present invention discloses a laser radar target detection method, system, terminal and storage medium, the method comprising: acquiring target point cloud data, judging whether the total number of points of the target point cloud data exceeds the maximum point limit; when the total number of points of the target point cloud data exceeds the maximum point limit, weighted sampling is performed on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data; the sampled point cloud data is volume columnarized to obtain volume columnarized point cloud data, the volume columnarized point cloud data is processed according to an improved feature encoding module, and a target tensor is output; a pseudo image is generated according to the target tensor, features of the pseudo image are extracted according to a two-dimensional convolutional neural network, and the extracted features are detected and regressed using a detection head to obtain a detection result. The present invention improves the accuracy and robustness in object recognition and segmentation tasks.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a laser radar target detection method, system, terminal and storage medium. Background Art

[0002] At present, most of the 3D object detection algorithms widely used in the field of autonomous driving have the characteristics of high detection accuracy, fast speed, and easy deployment on mobile terminal computing devices. For example, the PointPillars algorithm is a 3D object detection algorithm widely used in the field of autonomous driving. Its core idea is to convert the 3D point cloud data generated by the lidar into a pseudo image, and then use a 2D convolutional neural network for feature extraction.

[0003] However, these algorithms project three-dimensional point cloud information onto a two-dimensional plane, which will lose some of the detailed features of the object, resulting in low recognition accuracy for small targets (such as pedestrians and bicycles). Therefore, they have certain limitations in practical applications.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a laser radar target detection method, system, terminal and computer-readable storage medium, aiming to solve the problem in the prior art that the current three-dimensional target detection algorithm loses some of the object's detailed features when projecting three-dimensional point cloud information onto a two-dimensional plane, resulting in low recognition accuracy for small targets (such as pedestrians, bicycles, etc.), and therefore has certain limitations in practical applications.

[0006] To achieve the above object, the present invention provides a laser radar target detection method, the laser radar target detection method comprising the following steps:

[0007] Obtain target point cloud data, and determine whether the total number of points of the target point cloud data exceeds a maximum point limit;

[0008] When the total number of points of the target point cloud data exceeds the maximum point limit, weighted sampling is performed on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data;

[0009] The sampled point cloud data is volume-columnized to obtain volume-columnized point cloud data, and the volume-columnized point cloud data is processed according to the improved feature encoding module to output a target tensor;

[0010] A pseudo image is generated according to the target tensor, features of the pseudo image are extracted according to a two-dimensional convolutional neural network, and a detection head is used to detect and regress the extracted features to obtain a detection result.

[0011] Optionally, performing weighted sampling on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data specifically includes:

[0012] According to the spatial coordinates of each point in the target point cloud data, the depth of each point is calculated, and each depth is normalized to obtain a normalized depth value of each point;

[0013] Calculating a distance threshold according to the depth of each point, and calculating a depth score for each point according to the distance threshold and the normalized depth value of each point;

[0014] Based on the K-nearest neighbor algorithm, the local density of each point is calculated, and the density score of each point is calculated according to the local density of each point;

[0015] According to the density score and depth score of each point, the depth weight and density weight of each point are calculated;

[0016] According to the depth weight and density weight of each point, the depth score and density score of each point are fused to obtain the fusion score of each point, and the fusion score of each point is normalized to obtain the sampling probability of each point;

[0017] According to the sampling probability of each point, the maximum number of sampling points is limited to constitute the sampling point cloud data from the target point cloud data.

[0018] Optionally, calculating a distance threshold according to the depth of each point, and calculating a depth score of each point according to the distance threshold and a normalized depth value of each point, specifically includes:

[0019] According to the depths of all points, a maximum depth value and a minimum depth value are obtained, and the distance threshold is calculated according to the maximum depth value and the minimum depth value;

[0020] Determine whether the depth of each point is less than the distance threshold. If it is, multiply the normalized depth value of the point by the factor to obtain the depth score of the point.

[0021] When the depth of a point is not less than the distance threshold, the normalized depth value of the point is used as the corresponding depth score.

[0022] Optionally, the calculating the local density of each point based on the K-nearest neighbor algorithm and calculating the density score of each point according to the local density of each point specifically includes:

[0023] Based on the K-nearest neighbor algorithm, the average distance between each point and the preset points closest to it is calculated to obtain the local density of each point;

[0024] Get the local density maximum value and the local density minimum value among all local densities;

[0025] Normalization is performed according to the local density of each point, the local density maximum value and the local density minimum value to obtain a density score for each point.

[0026] Optionally, the processing of the volume-columnized point cloud data according to the improved feature encoding module to output a target tensor specifically includes:

[0027] The feature encoding module centers each point according to the body column point cloud data to obtain the coordinates of the center point and the coordinates of the body column centroid;

[0028] Calculating the covariance matrix of each point cloud in the body column according to the coordinates of the centroid of the body column and the coordinates of each central point, and performing eigenvalue decomposition on each of the covariance matrices to obtain an eigenvector of each covariance matrix;

[0029] According to the eigenvector of each covariance matrix, a normal vector corresponding to each point is obtained, and a three-dimensional tensor is generated according to the normal vector of each point and the volume columnized point cloud data;

[0030] The three-dimensional tensor is input into a point cloud feature enhancement module based on a self-attention mechanism for linear transformation to obtain a target tensor and output it.

[0031] Optionally, the step of inputting the three-dimensional tensor into a point cloud feature enhancement module based on a self-attention mechanism for linear transformation to obtain and output a target tensor specifically includes:

[0032] The point cloud feature enhancement module based on the self-attention mechanism performs three linear changes on the three-dimensional tensor, and performs residual connection on the results of the second linear change and the third linear change to obtain linear features, wherein normalization is performed after each linear change, and nonlinearity is introduced according to the activation function to obtain the result of the linear change, the result of the first linear change is used as the input of the second linear change after the self-attention mechanism, and the result of the second linear change is used as the input of the third linear change;

[0033] The maximum value of the linear features is obtained, and the maximum value of the linear features is used as the target tensor and output.

[0034] Optionally, the step of acquiring the target point cloud data and determining whether the total number of points of the target point cloud data exceeds a maximum point limit further includes:

[0035] When the total number of points of the target point cloud data does not exceed the maximum point limit, the current sampling point cloud data is volume-columnized to obtain the volume-columnized point cloud data at this time, and the corresponding volume-columnized point cloud data is processed according to the improved feature encoding module to output the corresponding target tensor;

[0036] The current target tensor is feature extracted according to the two-dimensional convolutional neural network, and the extracted features are detected and regressed using the detection head to obtain the corresponding detection results.

[0037] In addition, to achieve the above object, the present invention also provides a laser radar target detection system, wherein the laser radar target detection system comprises:

[0038] A judgment module is used to obtain target point cloud data and judge whether the total number of points of the target point cloud data exceeds the maximum point limit;

[0039] A sampling module, used for performing weighted sampling on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data when the total number of points of the target point cloud data exceeds the maximum point limit;

[0040] An output module, used for volume-columnizing the sampled point cloud data to obtain volume-columnized point cloud data, processing the volume-columnized point cloud data according to the improved feature encoding module, and outputting a target tensor;

[0041] The result generation module is used to generate a pseudo image according to the target tensor, extract features of the pseudo image according to a two-dimensional convolutional neural network, and use a detection head to detect and regress the extracted features to obtain a detection result.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a laser radar target detection program stored in the memory and executable on the processor, and when the laser radar target detection program is executed by the processor, the steps of the laser radar target detection method as described above are implemented.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a laser radar target detection program, and when the laser radar target detection program is executed by a processor, the steps of the laser radar target detection method as described above are implemented.

[0044] In the present invention, the target point cloud data is obtained, and it is determined whether the total number of points of the target point cloud data exceeds the maximum point limit; when the total number of points of the target point cloud data exceeds the maximum point limit, the target point cloud data is weighted sampled by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data; the sampled point cloud data is columnarized to obtain columnarized point cloud data, and the columnarized point cloud data is processed according to the improved feature encoding module to output a target tensor; a pseudo image is generated according to the target tensor, and features of the pseudo image are extracted according to a two-dimensional convolutional neural network, and the extracted features are detected and regressed using a detection head to obtain a detection result. The present invention solves the problem of sparse area information being easily ignored in current sampling methods by combining the depth information and density distribution of the point cloud through a point cloud sampling method based on density and depth dynamic weight fusion, thereby significantly improving the representativeness and processing efficiency of point cloud data; at the same time, the present invention improves the feature encoding module, thereby improving the accuracy and robustness of the overall algorithm in object recognition and segmentation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a preferred embodiment of the laser radar target detection method of the present invention;

[0046] Figure 2 is a schematic diagram of an improved feature encoding module in the laser radar target detection method of the present invention;

[0047] Figure 3 It is a structural schematic diagram of a point cloud feature enhancement module based on a self-attention mechanism in a laser radar target detection method of the present invention;

[0048] Figure 4 It is a schematic diagram of an implementation process of the laser radar target detection method of the present invention;

[0049] Figure 5 is a structural diagram of a preferred embodiment of the laser radar target detection system of the present invention;

[0050] Figure 6 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] Among the three-dimensional target detection algorithms currently widely used in the field of autonomous driving, most of them are characterized by high detection accuracy, fast speed, and easy deployment on mobile terminal computing devices. For example, the PointPillars algorithm is a three-dimensional target detection algorithm widely used in the field of autonomous driving. Its core idea is to convert the three-dimensional point cloud data generated by the lidar into a pseudo image, and then use a two-dimensional convolutional neural network for feature extraction. Specifically, the point cloud in the three-dimensional space is first columnarized, that is, divided into three-dimensional columnar spaces (Pillar) with equal spacing, and the average height, number of point clouds and other features in each column are counted to generate a two-dimensional pseudo image (Pseudo Image). Subsequently, a two-dimensional convolutional neural network is used to extract the features of the pseudo image and perform target detection and classification. The advantage of this feature extraction method is that it uses two-dimensional convolution instead of three-dimensional convolution, which greatly improves the detection speed, while maintaining relatively good accuracy, and can meet the real-time performance requirements of autonomous driving scenarios. However, these algorithms project the three-dimensional point cloud information onto a two-dimensional plane, which will lose some of the detailed features of the object, resulting in low recognition accuracy for small targets (such as pedestrians and bicycles), so there are certain limitations in practical applications.

[0053] In response to one or more of the above problems, the present invention obtains target point cloud data and determines whether the total number of points of the target point cloud data exceeds the maximum point number limit; when the total number of points of the target point cloud data exceeds the maximum point number limit, weighted sampling is performed on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data; the sampled point cloud data is columnarized to obtain columnarized point cloud data, and the columnarized point cloud data is processed according to an improved feature encoding module to output a target tensor; a pseudo image is generated according to the target tensor, features of the pseudo image are extracted according to a two-dimensional convolutional neural network, and the extracted features are detected and regressed using a detection head to obtain a detection result.

[0054] The laser radar target detection method described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the laser radar target detection method includes the following steps:

[0055] Step S10: Acquire target point cloud data, and determine whether the total number of points of the target point cloud data exceeds a maximum point limit.

[0056] It should be noted that, in the present invention, it is necessary to detect the laser radar target, specifically to obtain the corresponding target point cloud data, so as to realize the detection of the laser radar target through the target point cloud data.

[0057] Specifically, in the present invention, A frame of target point cloud data composed of points , expressed as , where each point is represented by , , , represents the coordinates of a point, represents the reflectivity of a point, Indicates the serial number of a point in the point cloud. p 1. p 2. p 3. p 4. p N Respectively represent the 1st, 2nd, 3rd, 4th and 5th points in the target point cloud data. N For the target point cloud data, determine whether its total number of points exceeds the maximum point limit. Exceeded the maximum point limit When , a point cloud sampling method based on dynamic weight fusion of density and distance will be used for sampling.

[0058] Step S20: When the total number of points of the target point cloud data exceeds the maximum point limit, weighted sampling is performed on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data.

[0059] Specifically, when the total number of points of the target point cloud data exceeds the maximum point limit, in order to solve the problem of sparse area information being easily ignored, thereby significantly improving the representativeness and processing efficiency of the point cloud data, the present invention performs weighted sampling on the target point cloud data through a point cloud sampling method based on density and depth dynamic weight fusion, thereby obtaining sampled point cloud data.

[0060] Furthermore, the point cloud sampling method based on density and depth dynamic weight fusion is used to perform weighted sampling on the target point cloud data to obtain the sampled point cloud data, which specifically includes:

[0061] Calculating the depth of each point according to the spatial coordinates of each point in the target point cloud data, and normalizing each depth to obtain a normalized depth value of each point;

[0062] Calculating a distance threshold according to the depth of each point, and calculating a depth score for each point according to the distance threshold and the normalized depth value of each point;

[0063] Based on the K-nearest neighbor algorithm, the local density of each point is calculated, and the density score of each point is calculated according to the local density of each point;

[0064] According to the density score and depth score of each point, the depth weight and density weight of each point are calculated;

[0065] According to the depth weight and density weight of each point, the depth score and density score of each point are fused to obtain the fusion score of each point, and the fusion score of each point is normalized to obtain the sampling probability of each point;

[0066] According to the sampling probability of each point, the maximum number of sampling points is limited to constitute the sampling point cloud data from the target point cloud data.

[0067] Specifically, in the present invention, the depth of each point is first calculated according to the spatial coordinates of each point in the target point cloud data. , that is, the Euclidean distance from the laser radar to each point, and its calculation process is expressed as:

[0068] ;

[0069] in, Represents the spatial coordinates of a point.

[0070] After that, the depth value Normalization is performed, and the greater the corresponding depth, the greater the sampling probability. The normalization formula is:

[0071] ;

[0072] in, Indicate point The normalized depth value of and They are the maximum depth and the minimum depth respectively. The maximum depth and the minimum depth can be obtained after calculating the depth of each point.

[0073] Further, the calculating a distance threshold according to the depth of each point, and calculating a depth score of each point according to the distance threshold and the normalized depth value of each point, specifically includes:

[0074] According to the depths of all points, a maximum depth value and a minimum depth value are obtained, and the distance threshold is calculated according to the maximum depth value and the minimum depth value;

[0075] Determine whether the depth of each point is less than the distance threshold. If it is, multiply the normalized depth value of the point by the factor to obtain the depth score of the point.

[0076] When the depth of a point is not less than the distance threshold, the normalized depth value of the point is used as the corresponding depth score.

[0077] After calculating the normalized depth value of each point, the depth score of each point is calculated accordingly. The distance threshold is first calculated by the maximum depth value and the minimum depth value. , which is calculated as:

[0078] ;

[0079] After that, for each point, determine whether its depth is less than the distance threshold. If its depth satisfy , then multiply its depth score by a factor of 1.5. After adjustment, we get the depth score of each point for:

[0080] ;

[0081] Furthermore, the method of calculating the local density of each point based on the K-nearest neighbor algorithm and calculating the density score of each point according to the local density of each point specifically includes:

[0082] Based on the K-nearest neighbor algorithm, the average distance between each point and the preset points closest to it is calculated to obtain the local density of each point;

[0083] Get the local density maximum value and the local density minimum value among all local densities;

[0084] Normalization is performed according to the local density of each point, the local density maximum value and the local density minimum value to obtain a density score for each point.

[0085] Specifically, after calculating the depth score of each point, the density score of each point is calculated accordingly. Specifically, the K-Nearest Neighbors Algorithm (KNN) is used to estimate the local density of each point, that is, to calculate the average distance between each point and its nearest 10 points. The calculation formula is as follows:

[0086] ;

[0087] in, Indicates The local density of the point, Indicate point With point After obtaining the local density of each point, the local maximum density and local minimum density are obtained accordingly.

[0088] According to the local density, local density maximum and local density minimum of each point, normalization can be performed to obtain the corresponding density score, which makes the sampling probability of points in sparse areas higher, and the normalization process is expressed as:

[0089] ;

[0090] in, For point The density score of and are the local density maximum and local density minimum, respectively.

[0091] After that, the depth weight and density weight of each point are calculated according to the density score and depth score of each point. The specific process is as follows:

[0092] ;

[0093] ;

[0094] in, is the depth weight, is the density weight, is the variance of the depth score, is the variance of the density score.

[0095] Use adaptive weights to fuse the depth score and density score to get the fusion score for each point for:

[0096] ;

[0097] Scoring the Fusion Normalize and get the sampling probability of each point for:

[0098] ;

[0099] Sampling is performed according to the sampling probability, that is, from the target point cloud data According to the probability distribution , randomly sample without replacement points, sampling process S It can be expressed as:

[0100] ;

[0101] in, s 1. s 2 and s k represents the first, second and kth points sampled, s i Indicates a certain point sampled. The specific sampling rules are:

[0102] ;

[0103] in, Indicates A point is a point The probability of Indicates jThe sampling probability of a point is j, and j represents the serial number of a point in the point cloud.

[0104] After sampling, the sampling point cloud data is obtained, which is expressed as:

[0105] ;

[0106] choice Represents the sampled point cloud data P new Corresponding to A collection of labels.

[0107] Step S30, cylindricalize the sampled point cloud data to obtain cylindrical point cloud data, process the cylindrical point cloud data according to the improved feature encoding module, and output a target tensor.

[0108] It should be noted that after sampling the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain the sampled point cloud data, in order to improve the accuracy and robustness of the overall algorithm in object recognition and segmentation tasks, the present invention improves the feature encoding module, proposes to integrate the normal vector feature into the feature encoding module, and designs a point cloud feature enhancement module (Self-Attention-Based Point Feature Enhancement Module, SAPFE) based on the self-attention mechanism.

[0109] Specifically, the sampling point cloud data is first columnarized. In three-dimensional space, For each body column, the improved feature encoding module expands the dimension of each point in it to obtain the corresponding representation of each point after the dimension expansion. ,in, represents the offset from the point to the centroid of the body column, represents the offset from the point to the center of the body column, The normal vector of the point The components of the normal vector It is obtained by calculating the covariance matrix of the point cloud within each body column and then performing eigenvalue decomposition on it.

[0110] Furthermore, the improved feature encoding module is used to process the volume-columnized point cloud data and output a target tensor, specifically including:

[0111] The feature encoding module centers each point according to the body column point cloud data to obtain the coordinates of the center point and the coordinates of the body column centroid;

[0112] Calculating the covariance matrix of each point cloud in the body column according to the coordinates of the centroid of the body column and the coordinates of each central point, and performing eigenvalue decomposition on each of the covariance matrices to obtain an eigenvector of each covariance matrix;

[0113] According to the eigenvector of each covariance matrix, a normal vector corresponding to each point is obtained, and a three-dimensional tensor is generated according to the normal vector of each point and the volume columnized point cloud data;

[0114] The three-dimensional tensor is input into a point cloud feature enhancement module based on a self-attention mechanism for linear transformation to obtain a target tensor and output it.

[0115] Specifically, Figure 2 As shown in FIG. 1 , it is an improved feature encoding module. In this module, each point is centered to obtain the coordinates of the centered point and the coordinates of the centroid of the body column, thereby calculating the corresponding normal vector. The offset of each point to the centroid of the body column is calculated from the coordinates of the centroid of the body column to obtain the geometric center of the body column. According to the geometric center of the body column, the offset from the point to the center of the body column can be obtained. Then, by combining the original features of the point cloud, the normal vector, the offset from the point to the centroid of the body column and the offset from the point to the center of the body column, the representation of each point after dimensional expansion can be obtained, which is then input into the SAPFE module.

[0116] Among them, centralization means subtracting the coordinates of the center of mass of the body column from the coordinates of each point, which can be expressed as:

[0117] ;

[0118] ;

[0119] in, It is The three-dimensional coordinate vector of the point, that is ; is the three-dimensional coordinate vector of the centroid of the body column, that is, the coordinates of the centroid of the body column ; is the number of points in the body column; is the coordinate of the central point, that is, the coordinate of the central point ;

[0120] The covariance matrix of the point cloud in each body column is calculated by the coordinates of the centroid of the body column and each centralized point, and the covariance matrix It is a matrix used to describe the scatter of point clouds in different directions and measure the joint changes of the three-dimensional coordinates of the point clouds in different directions. Specifically, the covariance matrix The formula is:

[0121] ;

[0122] After that, by performing eigenvalue decomposition on the covariance matrix, the scatter of the point cloud in different directions can be obtained. The relationship between eigenvalues ​​and eigenvectors is as follows:

[0123] ;

[0124] in, It is the eigenvalue of the covariance matrix, indicating the variance of the point cloud in this direction; It is the eigenvector of the covariance matrix, indicating the distribution direction of the point cloud in this direction.

[0125] Select the normal vector through the eigenvector ,in , is the eigenvector corresponding to the minimum eigenvalue of the covariance matrix, then It can be expressed as .

[0126] After all features are calculated, we get a dimension of ( ) three-dimensional tensor (Tensor) ,in, Indicates the number of body columns, Represents the number of point clouds contained in each body column. In order to enhance the point cloud features, the point cloud feature enhancement module based on the self-attention mechanism is used to process the tensor.

[0127] Furthermore, the three-dimensional tensor is input into a point cloud feature enhancement module based on a self-attention mechanism for linear transformation to obtain a target tensor and output it, specifically including:

[0128] The point cloud feature enhancement module based on the self-attention mechanism performs three linear changes on the three-dimensional tensor, and performs residual connection on the results of the second linear change and the third linear change to obtain linear features, wherein normalization is performed after each linear change, and nonlinearity is introduced according to the activation function to obtain the result of the linear change, the result of the first linear change is used as the input of the second linear change after the self-attention mechanism, and the result of the second linear change is used as the input of the third linear change;

[0129] The maximum value of the linear features is obtained, and the maximum value of the linear features is used as the target tensor and output.

[0130] Specifically, Figure 3 Each body column corresponds to a batch, and linear changes are performed in linear layer 1, linear layer 2, and linear layer 3 in turn. The first linear change is performed first, and the first linear feature is calculated. ,in and are the linearly changing weights and bias terms respectively. Normalization is performed to adjust the data distribution, and then the ReLU activation function is applied to introduce nonlinearity into the network to obtain the result of the first linear change. ; Afterwards, the present invention introduces the self-attention mechanism to calculate the attention weight , and then apply the attention weights to , and get the weighted features ,in Represents element-by-element multiplication. For the weighted features Perform a second linear change and calculate the second linear characteristic , normalize and apply ReLU activation function to get the result of the second linear change . Perform the third linear change to obtain the third linear feature ,right Normalize and apply the ReLU activation function to get the result of the third linear change In order to introduce residual connection, the present invention will and Add together to get the linear feature , and then the maximum value of the linear feature is extracted through the maximum pooling operation , and use it as the output of the network, that is, the target tensor. At this time, the output is a dimension of ( ), where Indicates the number of body columns.

[0131] Step S40: Generate a pseudo image according to the target tensor, extract features of the pseudo image according to a two-dimensional convolutional neural network, and use a detection head to detect and regress the extracted features to obtain a detection result.

[0132] Specifically, after obtaining the target tensor, in order to be able to apply a 2D convolutional neural network, that is, a two-dimensional convolutional neural network, on the point cloud data, the body column features are projected back to the original position according to the coordinates of the center point of the body column in the target tensor to obtain the corresponding pseudo image. Then, the two-dimensional convolutional neural network is used to extract features from the pseudo image, and the corresponding detection head is used to detect and regress the extracted features to obtain the detection results.

[0133] In addition, the step of acquiring the target point cloud data and determining whether the total number of points of the target point cloud data exceeds the maximum point limit further includes:

[0134] When the total number of points of the target point cloud data does not exceed the maximum point limit, the current sampling point cloud data is volume-columnized to obtain the volume-columnized point cloud data at this time, and the corresponding volume-columnized point cloud data is processed according to the improved feature encoding module to output the corresponding target tensor;

[0135] The current target tensor is feature extracted according to the two-dimensional convolutional neural network, and the extracted features are detected and regressed using the detection head to obtain the corresponding detection results.

[0136] In the present invention, when the total number of points of the target point cloud data does not exceed the maximum point limit, no sampling is performed, a pseudo image is generated through the current target point cloud data, and a corresponding detection result is obtained.

[0137] Furthermore, if Figure 4 As shown, an implementation process of the present invention is that after acquiring the point cloud, the target point cloud data is sampled by a point cloud sampling method based on density and depth dynamic weight fusion, and then the sampling result is columnarized and input into the improved feature encoding module, and a pseudo image is generated according to the corresponding output target tensor, and then a 2D backbone network, i.e., a two-dimensional convolutional neural network, and a detection head are used to output the detection result.

[0138] In the present invention, the effectiveness of the present invention is also tested, based on the OpenPCDet (a 3D object detection code library) framework, using the KITTI dataset for testing. The test dataset contains 7481 training samples and 7518 test samples, where the training samples are further divided into 3712 training sets and 3769 validation sets, and the corresponding maximum point limit is set to 16384. During the test, only the lidar point cloud data of the training set is used for training and validation for three categories: cars, pedestrians, and bicycles. The evaluation of object detection is based on the size, truncation, and occlusion of the object, and is classified into three difficulty levels: easy, medium, and difficult. AP (Average Precision) is used to evaluate the detection results of each category, and the average AP value of three categories of medium difficulty, namely mAP (Mean Average Precision), is used as the comprehensive performance indicator of the model. The corresponding methods used are F-PointNets (FrustumPointNets), AVOD-FPN (an Aggregate View Object Detection network (FeaturePyramid)), SECOND (Sparsely Embedded Convolutional Detection), PointRCNN, VoxelNet, PointPillars and the corresponding method of the present invention.

[0139] Training and evaluation were done in PyTorch (a framework for machine learning and deep learning), with four 4090 GPUs and an Intel Xeon Gold 6226R CPU. The network was trained for 90 batches, with the mini-batch size set to 8, the optimizer selected as Adam, and the initial learning rate set to 0.003.

[0140] Tables 1 and 2 show the evaluation results of the present invention in 3D target detection tasks and BEV (Bird's Eye View) detection tasks, respectively, and compare them with existing target detection algorithms. The results show that the corresponding method of the present invention significantly improves the performance, and the mAP value is increased from 60.58% to 63.76%. In particular, in terms of pedestrian 3D target detection, the detection performance of each difficulty level is improved by 4.01%, 4.48% and 4.37% respectively.

[0141] Table 1. Results of testing 3D detection benchmarks:

[0142]

[0143] Table 2. Results of testing the BEV test benchmark:

[0144]

[0145] The present invention obtains target point cloud data, and determines whether the total number of points of the target point cloud data exceeds the maximum point limit; when the total number of points of the target point cloud data exceeds the maximum point limit, the target point cloud data is weighted sampled by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data; the sampled point cloud data is columnarized to obtain columnarized point cloud data, and the columnarized point cloud data is processed according to the improved feature encoding module to output a target tensor; a pseudo image is generated according to the target tensor, and features of the pseudo image are extracted according to a two-dimensional convolutional neural network, and the extracted features are detected and regressed using a detection head to obtain a detection result. The present invention solves the problem of sparse area information being easily ignored in current sampling methods by combining the depth information and density distribution of the point cloud through a point cloud sampling method based on density and depth dynamic weight fusion, thereby significantly improving the representativeness and processing efficiency of point cloud data; at the same time, the present invention improves the feature encoding module, thereby improving the accuracy and robustness of the overall algorithm in object recognition and segmentation tasks.

[0146] Furthermore, if Figure 5 As shown, based on the above laser radar target detection method, the present invention also provides a laser radar target detection system, wherein the laser radar target detection system includes:

[0147] A judgment module 51 is used to obtain target point cloud data and judge whether the total number of points of the target point cloud data exceeds the maximum point limit;

[0148] A sampling module 52 is used for performing weighted sampling on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data when the total number of points of the target point cloud data exceeds the maximum point limit;

[0149] An output module 53, used to perform volume columnization on the sampled point cloud data to obtain volume columnized point cloud data, process the volume columnized point cloud data according to the improved feature encoding module, and output a target tensor;

[0150] The result generation module 54 is used to generate a pseudo image according to the target tensor, extract features of the pseudo image according to the two-dimensional convolutional neural network, and use a detection head to detect and regress the extracted features to obtain a detection result.

[0151] Furthermore, if Figure 6 As shown, based on the above-mentioned laser radar target detection method and system, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 6 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0152] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a laser radar target detection program 40 is stored on the memory 20, and the laser radar target detection program 40 can be executed by the processor 10, thereby realizing the laser radar target detection method in the present invention.

[0153] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the laser radar target detection method.

[0154] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface.

[0155] In one embodiment, when the processor 10 executes the laser radar target detection program 40 in the memory 20, the steps of the above laser radar target detection method are implemented.

[0156] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a laser radar target detection program, and when the laser radar target detection program is executed by a processor, the steps of the laser radar target detection method as described above are implemented.

[0157] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or terminal including the element.

[0158] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0159] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A laser radar target detection method, characterized in that: The laser radar target detection method comprises: Obtain target point cloud data, and determine whether the total number of points of the target point cloud data exceeds a maximum point limit; When the total number of points of the target point cloud data exceeds the maximum point limit, weighted sampling is performed on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data; The sampled point cloud data is volume-columnized to obtain volume-columnized point cloud data, and the volume-columnized point cloud data is processed according to the improved feature encoding module to output a target tensor; Generate a pseudo image according to the target tensor, extract features of the pseudo image according to a two-dimensional convolutional neural network, and use a detection head to detect and regress the extracted features to obtain a detection result; The method of performing weighted sampling on the target point cloud data by using a point cloud sampling method based on dynamic weight fusion of density and depth to obtain sampled point cloud data specifically includes: According to the spatial coordinates of each point in the target point cloud data, the depth of each point is calculated, and each depth is normalized to obtain a normalized depth value of each point; Calculating a distance threshold according to the depth of each point, and calculating a depth score for each point according to the distance threshold and the normalized depth value of each point; Based on the K-nearest neighbor algorithm, the local density of each point is calculated, and the density score of each point is calculated according to the local density of each point; According to the density score and depth score of each point, the depth weight and density weight of each point are calculated; According to the depth weight and density weight of each point, the depth score and density score of each point are fused to obtain the fusion score of each point, and the fusion score of each point is normalized to obtain the sampling probability of each point; According to the sampling probability of each point, the maximum number of sampling points is limited to constitute the sampling point cloud data from the target point cloud data.

2. The laser radar target detection method according to claim 1, characterized in that: The step of calculating a distance threshold according to the depth of each point, and calculating a depth score of each point according to the distance threshold and the normalized depth value of each point, specifically includes: According to the depths of all points, a maximum depth value and a minimum depth value are obtained, and the distance threshold is calculated according to the maximum depth value and the minimum depth value; Determine whether the depth of each point is less than the distance threshold. If it is, multiply the normalized depth value of the point by the factor to obtain the depth score of the point. When the depth of a point is not less than the distance threshold, the normalized depth value of the point is used as the corresponding depth score.

3. The laser radar target detection method according to claim 1, characterized in that: The method of calculating the local density of each point based on the K-nearest neighbor algorithm and calculating the density score of each point according to the local density of each point specifically includes: Based on the K-nearest neighbor algorithm, the average distance between each point and the preset points closest to it is calculated to obtain the local density of each point; Get the local density maximum value and the local density minimum value among all local densities; Normalization is performed according to the local density of each point, the local density maximum value and the local density minimum value to obtain a density score for each point.

4. The laser radar target detection method according to claim 1, characterized in that: The processing of the volume-columnized point cloud data according to the improved feature encoding module and outputting a target tensor specifically includes: The feature encoding module centers each point according to the body column point cloud data to obtain the coordinates of the center point and the coordinates of the body column centroid; Calculating the covariance matrix of each point cloud in the body column according to the coordinates of the centroid of the body column and the coordinates of each central point, and performing eigenvalue decomposition on each of the covariance matrices to obtain an eigenvector of each covariance matrix; According to the eigenvector of each covariance matrix, a normal vector corresponding to each point is obtained, and a three-dimensional tensor is generated according to the normal vector of each point and the volume columnized point cloud data; The three-dimensional tensor is input into a point cloud feature enhancement module based on a self-attention mechanism for linear transformation to obtain a target tensor and output it.

5. The laser radar target detection method according to claim 4, characterized in that: The three-dimensional tensor is input into a point cloud feature enhancement module based on a self-attention mechanism for linear transformation to obtain a target tensor and output it, specifically including: The point cloud feature enhancement module based on the self-attention mechanism performs three linear changes on the three-dimensional tensor, and performs residual connection on the results of the second linear change and the third linear change to obtain linear features, wherein normalization is performed after each linear change, and nonlinearity is introduced according to the activation function to obtain the result of the linear change, the result of the first linear change is used as the input of the second linear change after the self-attention mechanism, and the result of the second linear change is used as the input of the third linear change; The maximum value of the linear features is obtained, and the maximum value of the linear features is used as the target tensor and output.

6. The laser radar target detection method according to claim 1, characterized in that: The step of acquiring target point cloud data and determining whether the total number of points of the target point cloud data exceeds a maximum point limit further includes: When the total number of points of the target point cloud data does not exceed the maximum point limit, the current sampling point cloud data is volume-columnized to obtain the volume-columnized point cloud data at this time, and the corresponding volume-columnized point cloud data is processed according to the improved feature encoding module to output the corresponding target tensor; The current target tensor is feature extracted according to the two-dimensional convolutional neural network, and the extracted features are detected and regressed using the detection head to obtain the corresponding detection results.

7. A laser radar target detection system, characterized in that: The laser radar target detection system comprises: A judgment module is used to obtain target point cloud data and judge whether the total number of points of the target point cloud data exceeds the maximum point limit; A sampling module, used for performing weighted sampling on the target point cloud data by a point cloud sampling method based on density and depth dynamic weight fusion to obtain sampled point cloud data when the total number of points of the target point cloud data exceeds the maximum point limit; An output module, used for volume-columnizing the sampled point cloud data to obtain volume-columnized point cloud data, processing the volume-columnized point cloud data according to the improved feature encoding module, and outputting a target tensor; A result generation module, used for generating a pseudo image according to the target tensor, extracting features of the pseudo image according to a two-dimensional convolutional neural network, and using a detection head to detect and regress the extracted features to obtain a detection result; The method of performing weighted sampling on the target point cloud data by using a point cloud sampling method based on dynamic weight fusion of density and depth to obtain sampled point cloud data specifically includes: According to the spatial coordinates of each point in the target point cloud data, the depth of each point is calculated, and each depth is normalized to obtain a normalized depth value of each point; Calculating a distance threshold according to the depth of each point, and calculating a depth score for each point according to the distance threshold and the normalized depth value of each point; Based on the K-nearest neighbor algorithm, the local density of each point is calculated, and the density score of each point is calculated according to the local density of each point; According to the density score and depth score of each point, the depth weight and density weight of each point are calculated; According to the depth weight and density weight of each point, the depth score and density score of each point are fused to obtain the fusion score of each point, and the fusion score of each point is normalized to obtain the sampling probability of each point; According to the sampling probability of each point, the maximum number of sampling points is limited to constitute the sampling point cloud data from the target point cloud data.

8. A terminal, characterized in that: The terminal includes: a memory, a processor, and a laser radar target detection program stored in the memory and executable on the processor. When the laser radar target detection program is executed by the processor, the steps of the laser radar target detection method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a laser radar target detection program, and when the laser radar target detection program is executed by a processor, the steps of the laser radar target detection method according to any one of claims 1 to 6 are implemented.

Images