A point cloud object detection method based on attention and sampling learning

Abstract

The present invention relates to a point cloud object detection method based on attention and sampling learning, comprising the following steps: 1) collecting point cloud data of the target to be detected; 2) extracting point cloud features from the point cloud data using a point cloud extraction network; 3) filtering target index point features based on the point cloud features using the k-nearest value downsampling method; 4) aggregating the point cloud features into candidate target features based on the target index point features using an adaptive point cloud feature aggregation network; and 5) generating the category, position, and size information of the target to be detected using a multilayer perceptron based on the candidate target features. Compared with existing technologies, this invention has the advantages of high reliability and accuracy.

Description

Technical Field

[0001] The present invention relates to the fields of intelligent vehicles and computer vision, and in particular to a point cloud target detection method based on attention and sampling learning. Background Art

[0002] With the development of society and the improvement of people's living standards, the number of cars in my country has gradually increased. Consequently, road traffic safety has become a major issue. The development of smart cars is expected to further improve car safety and reduce the loss of life and property caused by traffic accidents.

[0003] Smart cars require environmental sensors and corresponding object detection methods to accurately detect objects in their surroundings, including vehicles, pedestrians, non-motorized vehicles, and static obstacles. Currently, commonly used environmental perception sensors include cameras, lidar, and millimeter-wave radar. Image data collected by cameras lacks depth information, making it difficult to accurately locate objects in space and is easily affected by ambient lighting. While millimeter-wave radar can capture 3D position information, the 3D information is sparse, making it prone to missed detections. Lidar can densely sample the 3D position of the surrounding surface, generating accurate 3D point cloud data. Point cloud object detection, powered by lidar, can accurately determine the size and spatial position of objects. Currently, point cloud object detection technology approaches can be categorized into three types: voxel-based (first discretizing the point cloud into regular voxels, then applying a neural network to detect objects), projection-based (first projecting the point cloud onto one or more planar views, then applying a neural network to detect objects), and direct point cloud detection (unchanging the point cloud representation and directly applying a neural network to detect objects). When processing point clouds, information loss in the point cloud processing process (voxelization, projection) should be avoided as much as possible.

[0004] Compared to the other two approaches, the direct point cloud detection approach suffers from less information loss during point cloud preprocessing because it doesn't involve voxelization or projection. However, even with this approach, the sampling phase of the layer-by-layer point cloud feature extraction process suffers from point cloud information loss, which urgently needs improvement. Furthermore, in traffic scenarios, targets are easily obscured by other targets or obstacles, creating a bottleneck for intelligent driving technology. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a point cloud target detection method based on attention and sampling learning, which has high accuracy and strong reliability.

[0006] Improves the detection performance of obscured vehicles and the reliability of the intelligent car's environmental perception system.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A point cloud object detection method based on attention and sampling learning includes the following steps:

[0009] 1) Collect point cloud data of the target to be detected;

[0010] 2) Extracting point cloud features through a point cloud extraction network based on point cloud data;

[0011] 3) Based on the point cloud features, the target index point features are filtered out through the k-nearest value downsampling method;

[0012] 4) Based on the target index point features, the point cloud features are aggregated into candidate target features through an adaptive point cloud feature aggregation network;

[0013] 5) Based on the characteristics of the candidate targets, the category, position and size information of the target to be detected is generated through a multi-layer perceptron.

[0014] Furthermore, the point cloud extraction network includes a plurality of point cloud attention networks and a plurality of layers of feature propagation networks, the plurality of point cloud attention networks are connected in sequence, and the plurality of layers of feature propagation networks are jump-connected to the plurality of point cloud attention networks;

[0015] The point cloud attention network includes a first self-attention model, a sampling learning network, two point networks and a first feedforward network;

[0016] The point cloud target detection method proposed in the present invention extracts point cloud features through a point cloud extraction network and aggregates the point cloud features into candidate target features, thereby enhancing the point cloud target detection method's ability to detect occluded targets. In the process of extracting point cloud features, sampling learning is used to reduce the information loss of the point cloud in the sampling.

[0017] Furthermore, the process of extracting point cloud features by each point cloud attention network includes the following steps:

[0018] 201) Inputting the input features into the sampling learning network to generate a sub-point cloud;

[0019] 202) Inputting the input features and the generated sub-point cloud into one of the point networks to obtain a neighborhood feature vector F1, and passing the coordinates corresponding to F1 through a multi-layer perceptron to generate a point cloud position code;

[0020] 203) After adding F1 to the point cloud position code generated in step 202), the result is projected into a point cloud query sequence Q through a linear projection layer;

[0021] 204) extracting a subset F′1 of F1 by a farthest point sampling algorithm, inputting F′1 and F1 into another point network to generate a neighborhood feature vector set, adding the neighborhood feature vector set to the point cloud position code generated in step 202), and then projecting through a linear projection layer to generate a point cloud feature sequence, including a point cloud key feature sequence K and a point cloud value feature sequence V;

[0022] 205) Input Q, K and V into the first self-attention model to generate new point cloud features;

[0023] Among them, for the sampling learning network of the first point cloud attention network, the input features are point cloud data, and for the sampling learning networks of the remaining point cloud attention networks, the input features are the point cloud features output by the previous point cloud attention network;

[0024] The point cloud extraction network is based on a point cloud attention network that combines a point network, a first self-attention model, and a sampling learning network. First, a sub-point cloud in the point cloud is sampled through the sampling learning network. Then, the input features and the neighborhood point cloud features of the sub-point cloud, i.e., local features, are extracted through the point network. Finally, the first self-attention model is used to implement feature exchange among all the local features of the point cloud to extract the global features of the point cloud. Multiple layers of point cloud attention networks are superimposed to extract point cloud features containing local feature information and global feature information layer by layer. Subsequently, the extracted point cloud features are propagated to more points through several layers of feature propagation layers, i.e., linear interpolation.

[0025] The point cloud extraction network can extract both local features of the point cloud and global features of the point cloud, thereby reducing information loss of the point cloud during sampling.

[0026] Furthermore, the step 201) includes:

[0027] 211) Mapping the input features to a high-dimensional space through a multi-layer perceptron to generate a high-dimensional point cloud feature vector matrix;

[0028] 212) Through the maximum pooling operation, the maximum value of each feature channel in the high-dimensional point cloud feature vector matrix is ​​retained to obtain the global feature;

[0029] 213) Splicing the global features onto each high-dimensional point cloud feature vector to obtain spliced ​​point cloud features;

[0030] 214) Inputting the spliced ​​point cloud features into a multi-layer perceptron to obtain a sampling matrix;

[0031] 215) Perform matrix multiplication on the sampling matrix and the input features to obtain the sub-point cloud.

[0032] Furthermore, the process of generating a neighborhood feature vector set by the midpoint network in step 204) includes:

[0033] 221) Taking the elements of F′1 as the sphere center, search for the elements of F1 in the spherical neighborhood with a radius of r, and use the elements of F1 in the neighborhood as the neighborhood feature set of the sphere center;

[0034] 222) Input the domain feature set of each element of F′1 into the multi-layer perceptron to obtain a high-dimensional point cloud feature vector;

[0035] 223) Perform a maximum pooling operation on the high-dimensional point cloud feature vector of the domain feature set of each element of F′1, and generate a neighborhood feature vector for each element neighborhood of F′1, and each neighborhood feature vector constitutes a neighborhood feature vector set.

[0036] Furthermore, the step 205) includes:

[0037] 231) Divide Q, K and V into n groups along the feature dimension, and add them to the corresponding point cloud position coding combination to obtain the point cloud feature sequence Q′ with point cloud position information i , K′ i and V′ i , i=1,2,…,n:

[0038] 232) Calculate the attention score matrix A i , the calculation formula is:

[0039]

[0040] Where, d is the number of feature channels;

[0041] 233) Calculate the intermediate vector sequence group F i , the calculation formula is:

[0042] F i =A i V′ i

[0043] F i Splicing along the feature channel dimension to obtain the intermediate feature vector sequence F2;

[0044] 234) adding F2 to the neighborhood feature vector F1 generated in step 202) and performing layer normalization to generate feature F3;

[0045] 235) Input F3 into the first feedforward network, then add the features generated by the first feedforward network to F3, and perform layer normalization to generate new point cloud features.

[0046] Furthermore, the adaptive point cloud feature aggregation network includes several decoding layers, each decoding layer includes a second self-attention model, a mutual attention model and a second feedforward network;

[0047] Aggregate point cloud features into target features layer by layer through several decoding layers;

[0048] Furthermore, the step 4) includes:

[0049] 401) The first decoding layer uses the target index feature generated in step 3) as input, and the remaining decoding layers use the first target feature output by the previous decoding layer as input to generate new first target features and second target features, and input the new first target features into the multi-layer perceptron to generate target position information, and generate a target position code based on the target position information. The new second target feature is obtained by adding the target position code generated by the previous decoding layer and the first target feature;

[0050] The input first target feature first passes through the second self-attention model to realize information exchange between the first target features, and then the connection between the first target feature and the point cloud feature is extracted through the mutual attention model to realize adaptive feature aggregation.

[0051] 402) Input the second target feature vector into three linear projection layers to generate a target query feature sequence Point cloud key feature sequence Point cloud value feature sequence

[0052] 403) Replace Q, K, and V in step 231) respectively, and execute steps 231) to 233) to generate an intermediate feature vector sequence

[0053] 404) The intermediate feature vector sequence After adding the target features of the decoding layer input, the features are generated after layer normalization.

[0054] 405) Features After adding it to the target position code and linearly projecting it, we get the target query feature sequence

[0055] 406) After adding the point cloud features extracted in step 2) to the point cloud position code, two linear projection layers are passed to obtain the point cloud key feature sequences. and point cloud value feature sequence

[0056] 407) Replace Q, K, and V in step 231) respectively, and execute steps 231) to 233) to generate an intermediate feature vector sequence

[0057] 408) The intermediate feature vector sequence and features After addition, the features are generated through layer normalization.

[0058] 409) Input the second feedforward network, and then combine the features generated by the second feedforward network with Add and perform layer normalization to obtain the first target feature vector output by the current decoding layer;

[0059] 410) Determine whether the current decoding layer is the last decoding layer. If so, use the first target feature vector output by the current decoding layer as a candidate target feature and the step ends. Otherwise, execute step 401).

[0060] Furthermore, point cloud feature propagation is achieved between the several layers of feature propagation networks by using a feature propagation strategy based on distance interpolation and a skip connection method;

[0061] Assuming that the feature propagation network at the end of the point cloud extraction network is the first-layer feature propagation network, and the l-1-layer feature propagation network is the layer to be propagated, the specific process of the point cloud feature propagation includes the following steps:

[0062] 241) Find the k nearest neighbors of the midpoint in the l-1th layer from the lth layer feature propagation network;

[0063] 242) Calculate the distance d(x,x) from the nearest neighbor to the point to be propagated i ′), x and x i ′ are the coordinates of the point to be propagated and the nearest neighbor point respectively;

[0064] 243) The point cloud features output by the l-th layer feature propagation network are propagated to the l-1-th layer feature propagation network. The propagation formula is:

[0065]

[0066] ω i =1 / d(x,x i ′) p

[0067] Among them, F is the point cloud feature propagated by the l-1 layer feature propagation network, f i is the nearest adjacent point feature, p is the distance index;

[0068] 244) The point cloud features obtained by propagation of the l-1 layer feature propagation network are concatenated with the point cloud features extracted by the point cloud attention network that is jump-connected to the l-1 layer feature propagation network, and after a 1*1 convolution, the point cloud features output by the l-1 layer feature propagation network are obtained.

[0069] Furthermore, the step 3) includes:

[0070] A multi-layer perceptron is used to generate a confidence score for each point cloud feature extracted in step 2) that represents the degree of proximity between the point cloud feature and the center of the target to be detected, and the feature with the highest confidence score is selected in turn as the target index point feature.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] (1) The point cloud extraction network of the present invention takes the point cloud attention network that combines the point network, the first self-attention model and the sampling learning network as the core structure. First, the sub-point cloud in the point cloud is sampled by the sampling learning network, and then the input features and the neighborhood point cloud features of the sub-point cloud, i.e., local features, are extracted by the point network. Finally, the first self-attention model is used to realize feature exchange in all the local features of the point cloud to extract the global features of the point cloud. The multi-layer point cloud attention network is superimposed to extract the point cloud features containing local feature information and global feature information layer by layer. Subsequently, after several layers of feature propagation layers, i.e., linear interpolation, the extracted point cloud features are propagated to more points. The point cloud extraction network can extract both local features of the point cloud and global features of the point cloud, thereby reducing the information loss of the point cloud in sampling and achieving high detection accuracy.

[0073] (2) The present invention uses the k-nearest value downsampling method to screen out target index point features based on point cloud features, aggregates point cloud features into candidate target features through an adaptive point cloud feature aggregation network based on the target index point features, and generates category, position and size information of the target to be detected through a multi-layer perceptron based on the candidate target features. The adaptive point cloud feature aggregation network includes several layers of decoding layers, each decoding layer includes a second self-attention model, a mutual attention model and a second feedforward network. The input first target feature first passes through the second self-attention model to realize information exchange between the first target features, and then the connection between the first target feature and the point cloud feature is extracted through the mutual attention model to realize adaptive feature aggregation. The point cloud features are aggregated into the target features layer by layer through several layers of decoding layers, thereby enhancing the network's ability to detect occluded targets, and the detection accuracy is high and the reliability is strong. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a detection process framework diagram of the present invention;

[0075] Figure 2This is a schematic diagram of the structure of the point cloud extraction network;

[0076] Figure 3 Schematic diagram of the decoding layer structure. DETAILED DESCRIPTION

[0077] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0078] A point cloud object detection method based on attention and sampling learning, such as Figure 1 , including the following steps:

[0079] 1) Collect point cloud data of the target to be detected;

[0080] 2) Extracting point cloud features through a point cloud extraction network based on point cloud data;

[0081] 3) Based on the point cloud features, the target index point features are filtered out through the k-nearest value downsampling method;

[0082] 4) Based on the target index point features, the point cloud features are aggregated into candidate target features through an adaptive point cloud feature aggregation network;

[0083] 5) Based on the characteristics of the candidate targets, the category, position and size information of the target to be detected is generated through a multi-layer perceptron.

[0084] The point cloud target detection method proposed in this embodiment is based on a point cloud target detection device. The point cloud target detection device includes a controller and a laser radar installed on a vehicle. The laser radar collects point cloud data around the vehicle, and the controller receives the point cloud data collected by the laser radar through a data line, thereby enabling the smart car to accurately detect targets such as vehicles, pedestrians, and static obstacles in the surrounding environment.

[0085] like Figure 3 ,The point cloud extraction network includes four point cloud attention networks and two layers of feature propagation networks.,Several point cloud attention networks are connected in sequence, and several layers of feature propagation networks are jump-connected with several point cloud attention networks;

[0086] The point cloud attention network includes the first self-attention model, the sampling learning network, two point networks (PointNet) and the first feedforward network;

[0087] The point cloud target detection method proposed in this embodiment extracts point cloud features through a point cloud extraction network and aggregates the point cloud features into candidate target features, thereby enhancing the point cloud target detection method's ability to detect occluded targets. During the point cloud feature extraction process, sampling learning is used to reduce the information loss of the point cloud in the sampling.

[0088] The process of extracting point cloud features from each point cloud attention network includes the following steps:

[0089] 201) Inputting the input features into the sampling learning network to generate a sub-point cloud;

[0090] 202) Inputting the input features and the generated sub-point cloud into one of the point networks to obtain a neighborhood feature vector F1, and passing the coordinates corresponding to F1 through a multi-layer perceptron to generate a point cloud position code;

[0091] 203) After adding F1 to the point cloud position code generated in step 202), the result is projected into a point cloud query sequence Q through a linear projection layer;

[0092] 204) extracting a subset F′1 of F1 by a farthest point sampling algorithm, inputting F′1 and F1 into another point network to generate a neighborhood feature vector set, adding the neighborhood feature vector set to the point cloud position code generated in step 202), and then projecting through a linear projection layer to generate a point cloud feature sequence, including a point cloud key feature sequence K and a point cloud value feature sequence V;

[0093] 205) Input Q, K and V into the first self-attention model to generate new point cloud features;

[0094] Among them, for the sampling learning network of the first point cloud attention network, the input features are point cloud data, and for the sampling learning networks of the remaining point cloud attention networks, the input features are the point cloud features output by the previous point cloud attention network;

[0095] The point cloud extraction network uses a point cloud attention network that combines a point network, a first self-attention model, and a sampling learning network as its core structure. First, a sub-point cloud in the point cloud is sampled through the sampling learning network. Then, the input features and the neighborhood point cloud features of the sub-point cloud, i.e., local features, are extracted through the point network. Finally, the first self-attention model is used to implement feature exchange among all the local features of the point cloud to extract the global features of the point cloud. Multiple layers of point cloud attention networks are superimposed to extract point cloud features containing local and global feature information layer by layer. Subsequently, the extracted point cloud features are propagated to more points through several layers of feature propagation layers, i.e., linear interpolation.

[0096] The point cloud extraction network can extract both local and global features of the point cloud, reducing the information loss of the point cloud during sampling.

[0097] Step 201) includes:

[0098] 211) Map the N*3 or N*(3+C) dimensional input features to a high-dimensional space through a multi-layer perceptron to generate an N*C1 high-dimensional point cloud feature vector matrix;

[0099] 212) Through the maximum pooling operation, the maximum value of each feature channel in the N*C1 high-dimensional point cloud feature vector matrix is ​​retained to obtain a 1*C1 dimensional global feature;

[0100] 213) Splicing the global features onto each high-dimensional point cloud feature vector to obtain N*(2*C1)-dimensional spliced ​​point cloud features;

[0101] 214) Input the spliced ​​point cloud features into the multi-layer perceptron to obtain an N*N1 sampling matrix;

[0102] 215) Perform matrix multiplication on the sampling matrix and the input features to obtain the sub-point cloud.

[0103] Step 204) The process of generating a neighborhood feature vector set from the midpoint network includes:

[0104] 221) Taking the elements of F′1 as the sphere center, search for the elements of F1 in the spherical neighborhood with a radius of r, and use the elements of F1 in the neighborhood as the neighborhood feature set of the sphere center;

[0105] 222) Input the domain feature set of each element of F′1 into the multi-layer perceptron to obtain a high-dimensional point cloud feature vector;

[0106] 223) Perform a maximum pooling operation on the high-dimensional point cloud feature vector of the domain feature set of each element of F′1, and generate a neighborhood feature vector for each element neighborhood of F′1, and each neighborhood feature vector constitutes a neighborhood feature vector set.

[0107] Step 205) includes:

[0108] 231) Divide Q, K and V into n groups along the feature dimension, and add them to the corresponding point cloud position coding combination to obtain the point cloud feature sequence Q′ with point cloud position information i , K′ i and V′ i , i=1,2,…,n:

[0109] 232) Calculate the attention score matrix A i , the calculation formula is:

[0110]

[0111] Where, d is the number of feature channels;

[0112] 233) Calculate the intermediate vector sequence group F i , the calculation formula is:

[0113] F i =A i V′ i

[0114] F i Splicing along the feature channel dimension to obtain the intermediate feature vector sequence F2;

[0115] 234) adding F2 to the neighborhood feature vector F1 generated in step 202) and performing layer normalization to generate feature F3;

[0116] 235) Input F3 into the first feedforward network, then add the features generated by the first feedforward network to F3, and perform layer normalization to generate new point cloud features.

[0117] The adaptive point cloud feature aggregation network includes several layers of decoding layers, such as Figure 3 ,Each decoding layer includes a second self-attention model, a mutual attention model, and a second feedforward network;

[0118] Aggregate point cloud features into target features layer by layer through several decoding layers;

[0119] Step 3) includes:

[0120] Generate a confidence score for each point cloud feature extracted in step 2) through the trained multi-layer perceptron, which represents the degree of proximity between the point cloud feature and the center of the target to be detected, and select the feature with the highest confidence score as the target index point feature in turn;

[0121] The training process of the multilayer perceptron trained in step 3) includes:

[0122] The point label is set to true if it is inside the target ground truth bounding box and is one of the k points closest to the target center, otherwise it is false, and the focal loss function is used for supervision.

[0123] Step 4) includes:

[0124] 401) The first decoding layer uses the target index feature generated in step 3) as input, and the remaining decoding layers use the first target feature output by the previous decoding layer as input to generate new first target features and second target features, and input the new first target features into the multi-layer perceptron to generate target position information, and generate a target position code based on the target position information. The new second target feature is obtained by adding the target position code generated by the previous decoding layer and the first target feature;

[0125] The input first target feature first passes through the second self-attention model to realize information exchange between the first target features, and then the connection between the first target feature and the point cloud feature is extracted through the mutual attention model to realize adaptive feature aggregation.

[0126] 402) Input the second target feature vector into three linear projection layers to generate a target query feature sequence Point cloud key feature sequence Point cloud value feature sequence

[0127] 403) Replace Q, K, and V in step 231) respectively, and execute steps 231) to 233) to generate an intermediate feature vector sequence

[0128] 404) The intermediate feature vector sequence After adding the target features of the decoding layer input, the features are generated after layer normalization.

[0129] 405) Features After adding it to the target position code and linearly projecting it, we get the target query feature sequence

[0130] 406) After adding the point cloud features extracted in step 2) to the point cloud position code, two linear projection layers are passed to obtain the point cloud key feature sequences. and point cloud value feature sequence

[0131] 407) Replace Q, K, and V in step 231) respectively, and execute steps 231) to 233) to generate an intermediate feature vector sequence

[0132] 408) The intermediate feature vector sequence and features After addition, the features are generated through layer normalization.

[0133] 409) Input the second feedforward network, and then combine the features generated by the second feedforward network with Add and perform layer normalization to obtain the first target feature vector output by the current decoding layer;

[0134] 410) Determine whether the current decoding layer is the last decoding layer. If so, use the first target feature vector output by the current decoding layer as a candidate target feature and the step ends. Otherwise, execute step 401).

[0135] Point cloud feature propagation is achieved between several layers of feature propagation networks through feature propagation strategies based on distance interpolation and skip connections;

[0136] Assume that the feature propagation network at the end of the point cloud extraction network is the first-layer feature propagation network, and the l-1-layer feature propagation network is the layer to be propagated. The specific process of point cloud feature propagation includes the following steps:

[0137] 241) Find the k nearest neighbors of the midpoint in the l-1th layer from the lth layer feature propagation network;

[0138] 242) Calculate the distance d(x,x) from the nearest neighbor to the point to be propagated i ′), x and x i ′ are the coordinates of the point to be propagated and the nearest neighbor point respectively;

[0139] 243) The point cloud features output by the l-th layer feature propagation network are propagated to the l-1-th layer feature propagation network. The propagation formula is:

[0140]

[0141] ω i =1 / d(x,x i ′) p

[0142] Among them, F is the point cloud feature propagated by the l-1 layer feature propagation network, f i is the nearest adjacent point feature, p is the distance index;

[0143] 244) The point cloud features obtained by propagation of the l-1 layer feature propagation network are concatenated with the point cloud features extracted by the point cloud attention network that is jump-connected to the l-1 layer feature propagation network, and after a 1*1 convolution, the point cloud features output by the l-1 layer feature propagation network are obtained.

[0144] This embodiment proposes a point cloud target detection method based on attention and sampling learning, which takes enhancing the detection ability of the point cloud target detection algorithm for occluded targets and reducing the point cloud information loss in the sampling stage during the point cloud feature extraction process as the entry point. By extracting point cloud features and aggregating point cloud global features, the network's ability to detect occluded targets is enhanced. In the point cloud sampling stage, the sampling learning method is used to reduce the information loss of the point cloud during sampling. The point cloud target detection method proposed in this embodiment can achieve an average detection accuracy of 79.9% for occluded vehicle targets, which is 8.7% higher than the performance of the unimproved point cloud target detection network.

[0145] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A point cloud object detection method based on attention and sampling learning, characterized in that: The following steps are involved: 1) Collect point cloud data of the target to be detected; 2) extracting point cloud features from the point cloud data using a point cloud extraction network, wherein the point cloud extraction network includes a plurality of point cloud attention networks and a plurality of layers of feature propagation networks, wherein the plurality of point cloud attention networks are sequentially connected, and the plurality of layers of feature propagation networks are jump-connected to the plurality of point cloud attention networks; The point cloud attention network includes a first self-attention model, a sampling learning network, two point networks, and a first feedforward network. The process of extracting point cloud features from each point cloud attention network includes the following steps: 201) Input the input features into the sampling learning network to generate a sub-point cloud; 202) Input the input features and the generated sub-point cloud into one of the point networks to obtain the neighborhood feature vector , the The corresponding coordinates are passed through a multi-layer perceptron to generate point cloud position encoding; 203) will After adding the point cloud position code generated in step 202), it is projected into a point cloud query sequence through a linear projection layer. ; 204) Extraction through the farthest point sampling algorithm Subset of ,Will and Input another point network to generate a neighborhood feature vector set, add the neighborhood feature vector set to the point cloud position code generated in step 202), and then project it through a linear projection layer to generate a point cloud feature sequence, including a point cloud key feature sequence and point cloud value feature sequence ; 205) will 、 and Input the first self-attention model to generate new point cloud features. The specific steps include: 231) 、 and Divided along the feature dimension Groups are added to the corresponding point cloud position coding combination to obtain the point cloud feature sequence with point cloud position information 、 and , : 232) Calculate the attention score matrix , the calculation formula is: in, is the number of feature channels; 233) Calculate the intermediate vector sequence group , the calculation formula is: Will Splicing along the feature channel dimension to obtain the intermediate feature vector sequence ; 234) The neighborhood feature vector generated in step 202) Add and normalize the layers to generate features ; 235) will Input the first feedforward network, and then combine the features generated by the first feedforward network with Add and perform layer normalization to generate new point cloud features; Among them, for the sampling learning network of the first point cloud attention network, the input features are point cloud data, and for the sampling learning networks of the remaining point cloud attention networks, the input features are the point cloud features output by the previous point cloud attention network; 3) Based on the point cloud features, the target index point features are filtered out through the k-nearest value downsampling method; 4) Based on the target index point features, the point cloud features are aggregated into candidate target features through an adaptive point cloud feature aggregation network. The adaptive point cloud feature aggregation network includes several decoding layers, each of which includes a second self-attention model, a mutual attention model, and a second feedforward network. The specific steps of aggregating point cloud features into candidate target features include: 401) The first decoding layer uses the target index feature generated in step 3) as input, and the remaining decoding layers use the first target feature output by the previous decoding layer as input to generate new first target features and second target features. The new first target features are input into the multi-layer perceptron to generate target position information, and a target position code is generated based on the target position information. The new second target feature is obtained by adding the target position code generated by the previous decoding layer and the first target feature. 402) Input the second target feature vector into three linear projection layers to generate a target query feature sequence , point cloud key feature sequence , point cloud value feature sequence ; 403) 、 、 Respectively replace step 231) 、 、 , and execute steps 231)~233) to generate an intermediate feature vector sequence ; 404) The intermediate feature vector sequence After adding the target features of the decoding layer input, the features are generated after layer normalization. ; 405) Features After adding it to the target position code and linearly projecting it, we get the target query feature sequence ; 406) After adding the point cloud features extracted in step 2) to the point cloud position code, two linear projection layers are passed to obtain the point cloud key feature sequence. and point cloud value feature sequence ; 407) 、 、 Respectively replace step 231) 、 、 , and execute steps 231)~233) to generate an intermediate feature vector sequence ; 408) The intermediate feature vector sequence and features After addition, the features are generated through layer normalization. ; 409) will Input the second feedforward network, and then combine the features generated by the second feedforward network with Add and perform layer normalization to obtain the first target feature vector output by the current decoding layer; 410) Determine whether the current decoding layer is the last decoding layer. If so, use the first target feature vector output by the current decoding layer as a candidate target feature and the step ends. Otherwise, execute step 401); 5) Based on the candidate target features, the multi-layer perceptron is used to generate the category, location, and size information of the target to be detected.

2. A point cloud target detection method based on attention and sampling learning according to claim 1, characterized in that: The step 201) includes: 211) Mapping input features to high-dimensional space through a multi-layer perceptron to generate a high-dimensional point cloud feature vector matrix; 212) Through the maximum pooling operation, the maximum value of each feature channel in the high-dimensional point cloud feature vector matrix is ​​retained to obtain the global feature; 213) Splicing the global features onto each high-dimensional point cloud feature vector to obtain spliced ​​point cloud features; 214) Input the spliced ​​point cloud features into the multi-layer perceptron to obtain a sampling matrix; 215) Perform matrix multiplication on the sampling matrix and the input features to obtain the sub-point cloud.

3. The point cloud target detection method based on attention and sampling learning according to claim 1, characterized in that: The process of generating a neighborhood feature vector set by the midpoint network in step 204) includes: 221) The element is the center of the sphere, and the search radius is In the spherical neighborhood of elements, and the neighborhood The elements of are taken as the neighborhood feature set of the sphere center; 222) The domain feature set of each element is input into the multi-layer perceptron to obtain the high-dimensional point cloud feature vector; 223) Yes The high-dimensional point cloud feature vector of the domain feature set of each element is subjected to the maximum pooling operation, Each element neighborhood of generates a neighborhood feature vector, and each neighborhood feature vector constitutes a neighborhood feature vector set.

4. The point cloud target detection method based on attention and sampling learning according to claim 1, characterized in that: The feature propagation strategy based on distance interpolation and skip connection is used to realize point cloud feature propagation between the several layers of feature propagation networks. Assume that the feature propagation network at the end of the point cloud extraction network is the first layer feature propagation network, The -1 layer feature propagation network is the layer to be propagated. The specific process of point cloud feature propagation includes the following steps: 241) From Find the first midpoint of the layer nearest neighbor points; 242) Calculate the distance from the nearest neighbor to the point to be propagated , and are the coordinates of the point to be propagated and the nearest neighbor point respectively; 243) The point cloud features output by the layer feature propagation network are propagated to the Layer feature propagation network, the propagation formula is: in, For the Point cloud features propagated by layer feature propagation network, is the nearest adjacent feature, is the distance index; 244) The point cloud features obtained by the layer feature propagation network are connected with the first The point cloud features extracted by the point cloud attention network of the jump connection of the layer feature propagation network are concatenated and passed through a 1*1 convolution to obtain the first Point cloud features output by the layer feature propagation network.

5. The point cloud target detection method based on attention and sampling learning according to claim 1, characterized in that: The step 3) includes: A multi-layer perceptron is used to generate a confidence score for each point cloud feature extracted in step 2) that represents the degree of proximity between the point cloud feature and the center of the target to be detected, and the feature with the highest confidence score is selected in turn as the target index point feature.

Images