A detection and recognition method and device based on three-dimensional information enhancement of laser point cloud

Abstract

The present invention relates to a detection and recognition method and device based on three-dimensional information enhancement of laser point cloud, the method comprising: voxelizing the original point cloud data to obtain a number of voxels; extracting the point cloud voxel features of each non-empty voxel to obtain a number of feature maps; inputting the several feature maps into a region proposal network to generate a first-stage candidate box; inputting the first-stage candidate box into a point cloud space shape completion network to obtain a target point set; extracting point cloud structure information from the target point set to obtain global structure information; sampling non-empty voxel features near each key point from the original point cloud data and combining them as key point features; extracting key point features to obtain grid point features; fusing grid point features and global structure information to obtain enhanced features; performing confidence prediction and candidate box refinement on the enhanced features to obtain confidence and bounding box parameters. The method enhances the feature representation of point cloud data and improves the accuracy of laser point cloud target detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and target detection and recognition, and in particular relates to a detection and recognition method based on three-dimensional information enhancement of laser point cloud. Background Art

[0002] Since imaging LiDAR has the advantages of high angular resolution, strong anti-interference ability, and high detection accuracy, it can obtain three-dimensional (3D) point cloud data that reflects the three-dimensional geometric shape, angle, and distance information of the target scene. Therefore, LiDAR-based 3D point cloud target detection technology has a wide range of application value in the field of unmanned driving.

[0003] Most existing 3D point cloud target detection methods use two processing methods, voxel-based or raw point cloud-based, to learn features from sparse and spatially irregular point cloud data. By voxelizing point cloud data into a regular grid, 2D convolution can be used to effectively detect 3D objects, but this quantization inevitably leads to information loss, which reduces positioning accuracy. On the contrary, the method of directly learning features from the raw point cloud and completing the prediction can retain accurate point cloud location information, but the computational cost is very high and there is a problem of limited receptive field.

[0004] Existing research proposes a voxelized point cloud representation method to obtain the voxelized features of the point cloud, and then converts the point cloud into a high-dimensional volume expression through 3D convolution, and finally outputs the detection result through the region proposal network. However, voxelization leads to errors in the point cloud position information and inflexible receptive field. Another study proposed to use the original point cloud information generated by the lidar directly as input and designed an end-to-end point cloud data processing network. However, the point cloud data processing network extracts the feature description of each independent point and the feature description of the global point cloud, and does not consider local features and structural constraints, which affects its performance in processing point cloud data in complex scenes.

[0005] It can be seen that the existing methods have the problems of difficulty in extracting point cloud data features and low accuracy in point cloud target detection. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a detection and recognition method and device based on three-dimensional information enhancement of laser point cloud. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] The embodiment of the present invention provides a detection and recognition method based on three-dimensional information enhancement of laser point cloud, comprising the steps of:

[0008] voxelize the original point cloud data to obtain a number of voxels;

[0009] Extracting point cloud voxel features of each non-empty voxel in the plurality of voxels using a sparse convolutional network to obtain a plurality of feature maps;

[0010] Inputting the several feature maps into a region proposal network to generate a first-stage candidate box;

[0011] Input the candidate box of the first stage into the point cloud space shape completion network to complete the point cloud space shape and obtain the target point set;

[0012] Extracting point cloud structure information from the target point set using a multi-scale grouping strategy to obtain global structure information;

[0013] Sampling the non-empty voxel features near each key point from the original point cloud data and combining them as key point features;

[0014] A feature extraction module combining a graph neural network and an attention mechanism is used to extract the key point features to obtain grid point features;

[0015] Fusing the grid point features with the global structure information to obtain enhanced features;

[0016] Confidence prediction and candidate box refinement are performed on the enhanced features to obtain confidence and bounding box parameters.

[0017] In one embodiment of the present invention, the original point cloud data is voxelized to obtain a plurality of voxels, including:

[0018] The original point cloud data is divided into three-dimensional grids according to a fixed resolution to obtain the plurality of voxels.

[0019] In one embodiment of the present invention, a sparse convolutional network is used to extract point cloud voxel features of each non-empty voxel in the plurality of voxels to obtain a plurality of feature maps, including:

[0020] Constructing the sparse convolutional network;

[0021] The average coordinates of the midpoints of each non-empty voxel and the reflectivity in the original point cloud data are input into the sparse convolutional network as initial features, and the plurality of feature maps are output.

[0022] In one embodiment of the present invention, the first-stage candidate box is input into a point cloud space shape completion network to complete the point cloud space shape, and a target point set is obtained, including:

[0023] The point set inside the candidate box of the first stage is processed by using the ROI-aware pooling module to obtain a first matrix;

[0024] Inputting the first matrix into a multi-layer perceptron to obtain a first intermediate feature, and inputting the first intermediate feature into a maximum pooling layer to obtain a second intermediate feature;

[0025] combining the second intermediate feature with the first intermediate feature to obtain a third intermediate feature;

[0026] Inputting the third intermediate feature into the multi-layer perceptron and the maximum pooling layer in sequence to obtain the global feature;

[0027] The global features are superimposed through a fully connected layer to generate the target point set.

[0028] In one embodiment of the present invention, a multi-scale grouping strategy is used to extract point cloud structure information from the target point set to obtain global structure information, including:

[0029] Selecting a number of points from the target point set to form a target set using a farthest point sampling algorithm;

[0030] A multi-layer perceptron and a maximum pooling layer are used to extract the local context of each point in the target set to obtain a second matrix;

[0031] extracting structural information from the second matrix using the multi-scale grouping strategy to obtain a target tensor;

[0032] The target tensor is passed through a fully connected layer to generate the global structure information.

[0033] In one embodiment of the present invention, sampling the non-empty voxel features near each key point from the original point cloud data and combining them as key point features includes:

[0034] Using the farthest point sampling algorithm to extract a number of key points from the original point cloud data;

[0035] Calculate the non-empty voxel feature set within the target radius with the key point as the center in the kth sparse convolution module of the sparse convolution network for each key point:

[0036]

[0037] in, Represents the relative coordinates of the key points, d i represents the key point, r k represents the target radius, represents the three-dimensional coordinates of the key points in the Kth sparse convolution module, Represents the features output in the Kth sparse convolution module;

[0038] The output feature of each key point in the k-th sparse convolution module is generated using the non-empty voxel feature set:

[0039]

[0040] Among them, M(·) represents the random sampling operation, and G(·) represents the operation performed by the multi-layer perceptron;

[0041] The output features of each key point in each sparse convolution module are connected to obtain the key point features.

[0042] In one embodiment of the present invention, the grid point features are:

[0043]

[0044] where Ω(r) represents all points within a fixed radius r of a grid point, W(·) represents the space used to map graph edges to a scalar or vector weight space, and σ * represents a learnable gate function, represents the linear projection of the position difference between two nodes, K i =Linear(f i ), K i Indicates key mapping, V i Represents the characteristics of node i.

[0045] In one embodiment of the present invention, the grid point features and the global structure information are fused to obtain enhanced features, including:

[0046] The grid point features and the global structure information are fused using a perspective channel attention module to obtain enhanced features.

[0047] In one embodiment of the present invention, confidence prediction and candidate box refinement are performed on the enhanced features to obtain confidence and bounding box parameters, including:

[0048] The enhanced features are sequentially input into two-layer multi-layer perceptrons to extract feature vectors, and the feature vectors are input into the first branch network for confidence prediction and the second branch network for candidate box refinement to obtain the confidence and the bounding box parameters.

[0049] Another embodiment of the present invention provides a detection and recognition device based on three-dimensional information enhancement of laser point cloud, comprising:

[0050] The point cloud voxelization module is used to voxelize the original point cloud data to obtain a number of voxels;

[0051] A feature map extraction module, used for extracting point cloud voxel features of each non-empty voxel in the plurality of voxels using a sparse convolutional network to obtain a plurality of feature maps;

[0052] A candidate box generation module, used to input the plurality of feature maps into a region proposal network to generate a first-stage candidate box;

[0053] A point cloud space shape completion module is used to input the candidate box of the first stage into the point cloud space shape completion network to complete the point cloud space shape and obtain a target point set;

[0054] A global structure information extraction module is used to extract point cloud structure information from the target point set using a multi-scale grouping strategy to obtain global structure information;

[0055] A key point feature sampling module, used to sample the non-empty voxel features near each key point from the original point cloud data and combine them as key point features;

[0056] A grid point feature extraction module is used to extract the key point features by using a feature extraction module combining a graph neural network and an attention mechanism to obtain grid point features;

[0057] An enhanced feature fusion module, used to fuse the grid point features with the global structure information to obtain enhanced features;

[0058] The confidence and bounding box parameter calculation module is used to perform confidence prediction and candidate box refinement on the enhanced feature to obtain confidence and bounding box parameters.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] The detection and recognition method of the present invention completes the spatial shape of the point cloud, so that the extracted structural information contains more semantic representations, enhances the feature representation of the point cloud data, has stronger characterization capabilities than the feature extraction method used in the existing methods, overcomes the problem of difficulty in extracting features of point cloud data in the existing methods, and improves the accuracy of laser point cloud target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic flow chart of a detection and recognition method for three-dimensional information enhancement based on laser point cloud provided by an embodiment of the present invention;

[0062] Figure 2 A schematic flow chart of another detection and recognition method for three-dimensional information enhancement based on laser point cloud provided by an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of a process of a detection and recognition method based on three-dimensional information enhancement of laser point cloud provided by an embodiment of the present invention;

[0064] Figure 4A schematic diagram of a flow chart of a feature extraction method combining a graph neural network and an attention mechanism provided in an embodiment of the present invention;

[0065] Figure 5 A schematic structural diagram of a detection and recognition device for three-dimensional information enhancement based on laser point cloud provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0067] Embodiment 1

[0068] See also Figure 1 , Figure 2 and Figure 3 , Figure 1 A schematic diagram of a flow chart of a detection and recognition method for three-dimensional information enhancement based on laser point cloud provided by an embodiment of the present invention, Figure 2 A schematic flow chart of another detection and recognition method based on three-dimensional information enhancement of laser point cloud provided by an embodiment of the present invention, Figure 3 A process diagram of a detection and recognition method based on three-dimensional information enhancement of laser point cloud provided by an embodiment of the present invention. The detection and recognition method comprises the following steps:

[0069] S1. voxelize the original point cloud data to obtain a number of voxels.

[0070] Specifically, the original point cloud data is divided into a three-dimensional grid according to a fixed resolution to obtain the plurality of voxels. The steps are as follows: assuming that the range of the point cloud in the 3D space is the length L, width W, and height H along the X, Y, and Z axes, and then correspondingly defining the length, width, and height of each voxel as vL, vW, and vH, where v is less than 1, thereby forming a plurality of voxels in the point cloud area.

[0071] S2. Using a sparse convolutional network to extract point cloud voxel features of each non-empty voxel in the plurality of voxels, to obtain a plurality of feature maps.

[0072] S21. Build a sparse convolutional network.

[0073] Specifically, the sparse convolutional network is composed of several sparse convolutional modules connected in sequence, each module is composed of several stages connected in sequence, and each stage includes several sub-manifold convolutional layers and a normal sparse convolutional layer. In this embodiment, the sparse convolutional network is composed of three identical sparse convolutional modules connected in sequence, each module is composed of two stages connected in sequence, and each stage contains 2 sub-manifold convolutional layers and a normal sparse convolutional layer.

[0074] S22. Input the average coordinates of the midpoints of each non-empty voxel and the reflectivity in the original point cloud data as initial features into the sparse convolutional network, and output the plurality of feature maps.

[0075] Specifically, the average coordinates (x, y, z) of each non-empty voxel midpoint and the reflectivity R in the original point cloud data are input as initial features into the sparse convolutional network. The sparse convolutional network can perform downsampling on the z-axis, and the obtained network output voxel feature vector is used as the extracted feature map.

[0076] S3. Input the several feature maps into the region proposal network to generate first-stage candidate boxes.

[0077] Specifically, a region proposal network is established, and several extracted feature maps are input into the region proposal network to obtain the first-stage candidate boxes.

[0078] S4. Input the candidate box of the first stage into the point cloud space shape completion network to complete the point cloud space shape and obtain the target point set.

[0079] Specifically, a point cloud spatial shape completion network is constructed, which includes a ROI-aware pooling module, a multi-layer perceptron, a maximum pooling layer, a feature combination module, a multi-layer perceptron, a maximum pooling layer and a fully connected layer connected in sequence.

[0080] S41. Use the ROI-aware pooling module to process the point set inside the candidate box in the first stage to obtain a first matrix.

[0081] Specifically, suppose a first-stage candidate box is selected and its internal point set is represented as {P i |i=1,...,N}, where Pi is a vector of points with coordinates (x, y, z), and N is the number of points. Then, the point set Pi is processed by the ROI-aware pooling module to obtain an n×3 first matrix M, where r p is the pooling layer size.

[0082] S42. Input the first matrix into a multi-layer perceptron to obtain a first intermediate feature, and input the first intermediate feature into a maximum pooling layer to obtain a second intermediate feature.

[0083] Specifically, the n×3 first matrix M is input into the multi-layer perceptron to output the first intermediate feature v'; then the first intermediate feature v' is input into the maximum pooling layer to obtain the 256-dimensional second intermediate feature v.

[0084] S43: Combine the second intermediate feature with the first intermediate feature to obtain a third intermediate feature.

[0085] Specifically, a vector concatenation operation is performed on the 256-dimensional second intermediate feature v and the first intermediate feature v' to obtain a 512-dimensional third intermediate feature w'.

[0086] S44, inputting the third intermediate feature into a multi-layer perceptron and a maximum pooling layer in sequence to obtain a global feature.

[0087] Specifically, the obtained 512-dimensional third intermediate feature w' is input into the multi-layer perceptron again, and the output feature of the multi-layer perceptron is input into the maximum pooling layer, and finally a 1024-dimensional local feature w is obtained.

[0088] S45, superimposing the global features through a fully connected layer to generate the target point set.

[0089] Specifically, the 1024-dimensional global feature w is superimposed through a fully connected layer (FCN) to generate a target point set This is a 1024×3 matrix representing the complete spatial shape.

[0090] S5. Use a multi-scale grouping strategy to extract point cloud structure information from the target point set to obtain global structure information.

[0091] S51, using the farthest point sampling algorithm to select a number of points from the target point set to form a target set.

[0092] Specifically, the farthest point sampling algorithm (FPS) is used to select the target point set Select m points from the target set S, where {S i |i=1,...,m}, take the T nearest neighboring points of each point in the target set S and group them to obtain an m×T×3 tensor.

[0093] S52. Using a multi-layer perceptron and a maximum pooling layer, extract the local context of each point in the target set to obtain a second matrix.

[0094] Specifically, the target set S is input into a multi-layer perceptron, and the output of the multi-layer perceptron is input into a maximum pooling layer. The maximum pooling layer outputs the local context of each point in the target set S to obtain a second matrix N of size m×C1, where C1 represents the number of channels.

[0095] S53: extracting structural information from the second matrix using the multi-scale grouping strategy to obtain a target tensor.

[0096] Specifically, a multi-scale grouping (MSG) strategy is used to extract structural information from the second matrix N, and a target tensor of size m×(C1+C1) is output.

[0097] S54. Generate the global structure information by passing the target tensor through a fully connected layer.

[0098] Specifically, the target tensor of m×(C1+C1) is input into the fully connected layer to generate global structural information in, Represents a feature set with C1 channels.

[0099] S6. Sample the non-empty voxel features near each key point from the original point cloud data and combine them as key point features.

[0100] S61. Use the farthest point sampling algorithm to extract a number of key points from the original point cloud data.

[0101] Specifically, we first use the Farthest Point Sampling (FPS) algorithm to sample n key points d from the original point cloud data. i For example, for the KITTI dataset n=2048 and for the Waymo dataset n=4096.

[0102] Then sample the non-empty voxel features near each key point and combine them as the key point feature F m , the specific steps are as follows:

[0103] S62, calculating a non-empty voxel feature set within a target radius range with the key point as the center in the kth sparse convolution module of the sparse convolution network for each key point.

[0104] Specifically, first define is the feature output from the kth sparse convolutional module of the sparse convolutional network in step S2, N k is the number of non-empty voxels in the kth sparse convolution module, then for each key point d i , it can be defined that in the kth sparse convolution module, it is centered on itself and r k In the circular area with the target radius, the feature set of non-empty voxels is:

[0105]

[0106] in, Represents the relative coordinates of the key points, d i represents the key point, r k represents the target radius, represents the three-dimensional coordinates of the key points in the Kth sparse convolution module, Represents the features output in the Kth sparse convolution module;.

[0107] S63: Generate output features of each key point in the kth sparse convolution module using the non-empty voxel feature set.

[0108] Specifically, the key point d is generated using the non-empty voxel feature set i The output feature f in the kth sparse convolution module i (pvk) :

[0109]

[0110] Among them, M(·) represents the random sampling operation, and G(·) represents the operation performed by the multi-layer perceptron.

[0111] S64, connecting the output features of each key point in each sparse convolution module to obtain the key point features.

[0112] Specifically, for each key point d i , and get the output feature f i (pvk) Finally, the features of each sparse convolution module are connected to obtain the total feature, that is, the key point feature F m .

[0113] S7. Use a feature extraction module that combines a graph neural network and an attention mechanism to extract the key point features to obtain grid point features.

[0114] Specifically, we construct a feature extraction module that combines graph neural network and attention mechanism, and transform the key point feature F m Input into the feature extraction module to extract features and obtain grid point features.

[0115] See also Figure 4 , Figure 4 A flow chart of a feature extraction method combining a graph neural network and an attention mechanism provided in an embodiment of the present invention. The steps of the feature extraction method are as follows:

[0116] S71. The graph neural network regards each point in the point cloud as a node in the graph structure, and generates edges between each point and its neighboring points; the feature F of node i in the graph structure i m Represented as V i =MLP(F i m ), the edge can be expressed as the linear projection of the position difference between two nodes, and its expression formula is: The key point feature F m Input to the feature extraction module, extract features from adjacent nodes through weighted combination operation The calculation is as follows:

[0117]

[0118] Where W(·) is used to map graph edges into a scalar or vector weight space, ⊙ represents the scalar-vector product between the learned weights and graph nodes, represents the linear projection of the position difference between two nodes, p i represents the position of the i-th node, p j represents the position of the jth node, V i =MLP(F i m ) represents the feature of node i, F i m Represents key point features.

[0119] S72. In the concept of attention mechanism, can be considered as the number of pixels from the grid point p gird To point p i The request mapping, V i can be considered as the function of feature f i Get the value map, key map K i It can be expressed as K i =Linear(f i ), the feature extraction process can be expressed as the following formula: The extracted features are:

[0120]

[0121] Where W(·) represents the softmax function.

[0122] S73. Combining equations (3) and (4), we get the grid point features:

[0123]

[0124] where Ω(r) represents all points within a fixed radius r of a grid point, W(·) represents the space used to map graph edges to a scalar or vector weight space, and σ * Represents a learning gate function, which is implemented by linear projection and sigmoid function. represents the linear projection of the position difference between two nodes, K i =Linear(f i ), K i Indicates key mapping, V i Represents the characteristics of node i.

[0125] Formula (5) combines the principles of graph neural network and attention mechanism, and is a flexible and effective feature extraction method. It can adaptively extract features from geometric information Q through a learnable gate function.pos , semantic information K and combination Q pos Learn more meaningful point cloud features from K, thereby obtaining grid point features F g .

[0126] S8. Fusing the grid point features and the global structure information to obtain enhanced features.

[0127] Specifically, Figure 3 As shown in the figure, we construct a perspective channel attention module and transform the grid point feature F g and global structural information F s Input to the perspective channel attention module and output the enhanced feature F e Furthermore, in the perspective channel attention module, the grid point feature F g and global structural information F s The total features obtained by the connection are then input into the two-way maximum pooling layer, the linear layer and the Rule function for processing respectively. After that, the array elements in the two-way processing results are multiplied in turn, and the multiplication results are processed by the sigimoid function. Finally, the connected features and the processing results of the sigimoid function are multiplied by matrix multiplication to obtain the enhanced features F e .

[0128] S9. Perform confidence prediction and candidate box refinement on the enhanced features to obtain confidence and bounding box parameters.

[0129] Specifically, the enhanced features are sequentially input into two layers of multi-layer perceptrons to extract feature vectors, and the feature vectors are input into the first branch network for confidence prediction, and the feature vectors are input into the second branch network for candidate box refinement to obtain the confidence and the bounding box parameters, thereby obtaining the detection and recognition results. Furthermore, the bounding box parameters can determine the target range of the detection and recognition, and the confidence is used to judge the accuracy of the bounding box parameters.

[0130] First, confidence prediction is performed. Specifically, the confidence prediction is calculated by calculating the intersection over union (IoU) between the 3D prediction candidate region and its corresponding true value box. For the kth 3D prediction candidate region, its confidence value y k is normalized to be between [0,1] as follows:

[0131] y k =min(1,max(0,2IOU k -0.5)) (6)

[0132] Among them, IOU k is the intersection-over-union ratio between the kth 3D prediction candidate region and its corresponding ground-truth box.

[0133] Then, the candidate box is refined. Specifically, the candidate box is refined by encoding each point of the 3D prediction candidate box.

[0134] Furthermore, after obtaining the confidence and bounding box parameters, The loss function is optimized:

[0135]

[0136] Where x = ti-ti * , ti is the predicted coordinate vector, and ti* is the true value box coordinate vector.

[0137] By optimizing the loss function, the recognition rate of the target results detected by the algorithm and the actual real target can be measured.

[0138] Furthermore, this embodiment verifies the recognition rate of target detection through simulation.

[0139] 1. Simulation conditions

[0140] The computer system used in this embodiment is the "Rongtian" MultiGPU SCS4880 series supercomputing server. The CPU processor is the Intel Xeon E5-2630V4 processor. It integrates 8 NVIDIA RTX 2080Ti 11GB graphics cards, and the hard disk consists of 1 system disk and 2 storage disks. The operating system uses Ubuntu 16.0.4, and the operating environment is NET.Framework 4.5. And install the support of related libraries such as CUDA 9.0, Python 3.5, and CUDNN 0.7.

[0141] 2. Simulation content

[0142] According to the above detection and recognition method, the target detection and recognition rate is calculated. Through calculation, the target detection and recognition rate is 78.9%. It can be seen that the detection and recognition method based on three-dimensional information enhancement of laser point cloud adopted in this embodiment fully exploits the information contained in the point cloud data, can achieve higher detection and recognition accuracy, and is advanced.

[0143] In summary, the detection and recognition method of this embodiment completes the spatial shape of the point cloud, so the extracted structural information contains more semantic representations, enhances the feature representation of the point cloud data, and has stronger characterization capabilities than the feature extraction method used in the existing methods. It overcomes the problem that the existing methods are difficult to extract features from point cloud data, improves the accuracy of laser point cloud target detection, and can be used in unmanned driving fields such as obstacle recognition and path planning.

[0144] Embodiment 2

[0145] Based on Example 1, see Figure 5 , Figure 5 The present invention provides a schematic diagram of a detection and recognition device based on three-dimensional information enhancement of laser point cloud. The detection and recognition device includes: a point cloud voxelization module, a feature map extraction module, a candidate box generation module, a point cloud space shape completion module, a global structure information extraction module, a key point feature sampling module, a grid point feature extraction module, an enhanced feature fusion module, and a confidence and bounding box parameter calculation module, wherein:

[0146] The point cloud voxelization module is used to voxelize the original point cloud data to obtain a number of voxels. The feature map extraction module is connected to the point cloud voxelization module, and is used to extract the point cloud voxel features of each non-empty voxel in the said number of voxels using a sparse convolutional network to obtain a number of feature maps. The candidate box generation module is connected to the feature map extraction module, and is used to input the said several feature maps into the region proposal network to generate the first stage candidate box. The point cloud space shape completion module is connected to the candidate box generation module, and is used to input the first stage candidate box into the point cloud space shape completion network to complete the point cloud space shape to obtain the target point set. The global structure information extraction module is connected to the point cloud space shape completion module, and is used to extract point cloud structure information from the said target point set using a multi-scale grouping strategy to obtain global structure information. The key point feature sampling module is connected to the point cloud voxelization module and the feature map extraction module, and is used to sample the said non-empty voxel features near each key point from the said original point cloud data and combine them as key point features. The grid point feature extraction module is connected to the key point feature sampling module, and is used to extract the key point features using the feature extraction module combining the graph neural network and the attention mechanism to obtain the grid point features. The enhanced feature fusion module is connected to the global structure information extraction module and the grid point feature extraction module, and is used to fuse the grid point features with the global structure information to obtain enhanced features. The confidence and bounding box parameter calculation module is connected to the enhanced feature fusion module, and is used to perform confidence prediction and candidate box refinement on the enhanced features to obtain confidence and bounding box parameters.

[0147] For the specific implementation methods of the above modules, please refer to Example 1, which will not be repeated in this example.

[0148] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A detection and recognition method based on three-dimensional information enhancement of laser point cloud, characterized in that: Includes steps: voxelize the original point cloud data to obtain a number of voxels; Extracting point cloud voxel features of each non-empty voxel in the plurality of voxels using a sparse convolutional network to obtain a plurality of feature maps; Inputting the several feature maps into a region proposal network to generate a first-stage candidate box; Inputting the first-stage candidate box into the point cloud space shape completion network to complete the point cloud space shape, and obtaining the target point set; including: using the ROI-aware pooling module to process the point set inside the first-stage candidate box to obtain a first matrix; inputting the first matrix into a multi-layer perceptron to obtain a first intermediate feature, and inputting the first intermediate feature into a maximum pooling layer to obtain a second intermediate feature; combining the second intermediate feature with the first intermediate feature to obtain a third intermediate feature; inputting the third intermediate feature into a multi-layer perceptron and a maximum pooling layer in sequence to obtain a global feature; and superimposing the global feature through a fully connected layer to generate the target point set; Extracting point cloud structure information from the target point set using a multi-scale grouping strategy to obtain global structure information; including: selecting a number of points from the target point set to form a target set using a farthest point sampling algorithm; extracting the local context of each point in the target set using a multi-layer perceptron and a maximum pooling layer to obtain a second matrix; extracting structure information from the second matrix using the multi-scale grouping strategy to obtain a target tensor; and generating the global structure information from the target tensor through a fully connected layer; Sampling the non-empty voxel features near each key point from the original point cloud data and combining them as key point features; comprising: taking a number of key points from the original point cloud data using a farthest point sampling algorithm; Calculate the non-empty voxel feature set within the target radius with the key point as the center in the kth sparse convolution module of the sparse convolution network for each key point: in, Represents the relative coordinates of the key points, d i represents the key point, r k represents the target radius, represents the three-dimensional coordinates of the key points in the Kth sparse convolution module, Represents the features output in the Kth sparse convolution module; The output feature of each key point in the k-th sparse convolution module is generated using the non-empty voxel feature set: Among them, M(·) represents the random sampling operation, and G(·) represents the operation performed by the multi-layer perceptron; Connecting the output features of each key point in each sparse convolution module to obtain the key point features; The key point features are extracted using a feature extraction module that combines a graph neural network and an attention mechanism to obtain grid point features; the grid point features are: Where Ω(r) represents all points within a fixed radius r of a grid point, W(·) represents the space used to map graph edges to a scalar or vector weight space, and σ * represents a learnable gate function, represents the linear projection of the position difference between two nodes, K i =Linear(f i ), K i Indicates key mapping, V i Represents the characteristics of node i; Fusing the grid point features with the global structure information to obtain enhanced features; Confidence prediction and candidate box refinement are performed on the enhanced features to obtain confidence and bounding box parameters.

2. The detection and recognition method based on three-dimensional information enhancement of laser point cloud according to claim 1 is characterized in that: The original point cloud data is voxelized to obtain several voxels, including: The original point cloud data is divided into three-dimensional grids according to a fixed resolution to obtain the plurality of voxels.

3. The detection and recognition method based on three-dimensional information enhancement of laser point cloud according to claim 1 is characterized in that: The point cloud voxel features of each non-empty voxel in the plurality of voxels are extracted using a sparse convolutional network to obtain a plurality of feature maps, including: Constructing the sparse convolutional network; The average coordinates of the midpoints of each non-empty voxel and the reflectivity in the original point cloud data are input into the sparse convolutional network as initial features, and the plurality of feature maps are output.

4. The detection and recognition method based on three-dimensional information enhancement of laser point cloud according to claim 1, characterized in that: The grid point features and the global structure information are fused to obtain enhanced features, including: The grid point features and the global structure information are fused using a perspective channel attention module to obtain enhanced features.

5. The detection and recognition method based on three-dimensional information enhancement of laser point cloud according to claim 1, characterized in that: Confidence prediction and candidate box refinement are performed on the enhanced features to obtain confidence and bounding box parameters, including: The enhanced features are sequentially input into two-layer multi-layer perceptrons to extract feature vectors, and the feature vectors are input into the first branch network for confidence prediction and the second branch network for candidate box refinement to obtain the confidence and the bounding box parameters.

6. A detection and recognition device based on three-dimensional information enhancement of laser point cloud, characterized in that: include: The point cloud voxelization module is used to voxelize the original point cloud data to obtain a number of voxels; A feature map extraction module, used for extracting point cloud voxel features of each non-empty voxel in the plurality of voxels using a sparse convolutional network to obtain a plurality of feature maps; A candidate box generation module, used to input the plurality of feature maps into a region proposal network to generate a first-stage candidate box; A point cloud space shape completion module, used for inputting the first stage candidate box into the point cloud space shape completion network to complete the point cloud space shape and obtain a target point set; including: using the ROI perception pooling module to process the point set inside the first stage candidate box to obtain a first matrix; inputting the first matrix into a multi-layer perceptron to obtain a first intermediate feature, and inputting the first intermediate feature into a maximum pooling layer to obtain a second intermediate feature; combining the second intermediate feature with the first intermediate feature to obtain a third intermediate feature; inputting the third intermediate feature into a multi-layer perceptron and a maximum pooling layer in sequence to obtain a global feature; superimposing the global feature through a fully connected layer to generate the target point set; A global structure information extraction module is used to extract point cloud structure information from the target point set using a multi-scale grouping strategy to obtain global structure information; including: using a farthest point sampling algorithm to select a number of points from the target point set to form a target set; using a multi-layer perceptron and a maximum pooling layer to extract the local context of each point in the target set to obtain a second matrix; using the multi-scale grouping strategy to extract structure information from the second matrix to obtain a target tensor; passing the target tensor through a fully connected layer to generate the global structure information; A key point feature sampling module is used to sample the non-empty voxel features near each key point from the original point cloud data and combine them as key point features; including: using the farthest point sampling algorithm to take a number of key points from the original point cloud data; Calculate the non-empty voxel feature set within the target radius with the key point as the center in the kth sparse convolution module of the sparse convolution network for each key point: in, Represents the relative coordinates of the key points, d i represents the key point, r k represents the target radius, represents the three-dimensional coordinates of the key points in the Kth sparse convolution module, Represents the features output in the Kth sparse convolution module; The output feature of each key point in the k-th sparse convolution module is generated using the non-empty voxel feature set: Among them, M(W) represents the random sampling operation, and G(W) represents the operation performed by the multi-layer perceptron; Connecting the output features of each key point in each sparse convolution module to obtain the key point features; A grid point feature extraction module is used to extract the key point features using a feature extraction module combining a graph neural network and an attention mechanism to obtain grid point features; the grid point features are: where Ω(r) represents all points within a fixed radius r of a grid point, W(·) represents the space used to map graph edges to a scalar or vector weight space, and σ * represents a learnable gate function, represents the linear projection of the position difference between two nodes, K i =Linear(f i ), K i Indicates key mapping, V i Represents the characteristics of node i; An enhanced feature fusion module, used to fuse the grid point features with the global structure information to obtain enhanced features; The confidence and bounding box parameter calculation module is used to perform confidence prediction and candidate box refinement on the enhanced feature to obtain confidence and bounding box parameters.

Images