Point Cloud Scene Recognition Method, Device and Storage Medium Based on Dynamic Sparse Feature Fusion

Abstract

The present invention discloses a point cloud scene recognition method based on dynamic sparse feature fusion. This point cloud scene recognition method uses multiple three-dimensional sub-convolutions to extract detailed features from the downsampled point cloud features. Since only one dimension of the convolution kernel in this three-dimensional sub-convolution is not 1, the number of parameters is reduced during the process of extracting detailed features. At the same time, the number of three-dimensional sub-convolutions can be flexibly configured based on the current scene information, providing computing efficiency while accurately extracting useful features in the current scene. During the upsampling process, weight distribution is performed on the feature points and feature channels of the upsampled point cloud features and the point cloud encoding features of the corresponding size to optimize the local features of the point cloud, and thus the point cloud frame of the current scene can be obtained more accurately.

Description

Technical Field

[0001] The present invention belongs to the field of point cloud recognition, and particularly relates to a point cloud scene recognition method, device, and storage medium based on dynamic sparse feature fusion. Background Art

[0002] With the development of robot technology, the application fields of robots have also begun to expand from traditional factory manufacturing fields to on-site and service fields such as homes, hospitals, elderly care centers, military, tourism, transportation, exploration, and disaster relief and rescue. The characteristics of such environments are often uncertain or changing, and how robots perceive and understand the environment has become a challenge. The fundamental and core issue of robot environmental perception and understanding is scene recognition.

[0003] Scene recognition is performed through a point cloud recognition method. The principle of the point cloud recognition method is to extract several feature points with significant geometric features from the point cloud data obtained by a three-dimensional measurement device to represent the whole target, statistically analyze the neighbor information of the feature points, and compare it with the features of known objects in the model library, so as to complete the matching recognition of the target. Currently, the mainly used point cloud recognition methods mainly achieve this by comparing the global feature similarity of the current point cloud and the historical point cloud. Thanks to the rapid development of deep learning, currently, neural networks are mainly used to extract the global features of the point cloud.

[0004] The literature Pointnetvlad: Deep point cloud based retrieval for large-scale place recognition[J] / / Pointnetvlad: Deep point cloud based retrieval for large-scale place recognition.2018:4470-4479 discloses using a neural network to extract global features from 3D point clouds, but the influence of neighboring points is not considered in the feature extraction. To solve this problem, it is mainly solved through two ideas: K-nearest neighbor and three-dimensional sparse convolution.

[0005] The literature "Dagc: Employing dual attention and graph convolution for pointcloud based place recognition"[J] / / ICMR'20: Proceedings of the 2020 International Conference on Multimedia Retrieval and the literature "Lpd-net: 3d pointcloud learning for large-scale place recognition and environment analysis"[J] / / 2019 IEEE / CVF International Conference on Computer Vision(ICCV) disclose that the K-nearest neighbor-based method finds the nearest neighbors of local points through clustering, and then uses a neural network to extract the features of these nearest neighbors, playing a role in strengthening local features.

[0006] The literature "SVT-Net: Super Light-Weight Sparse Voxel Transformer for LargeScale Place Recognition"[J] / / AAAI-22 Technical Tracks 1 discloses that the three-dimensional sparse convolution-based method first voxelizes the point cloud and regularly discretizes it into a three-dimensional space of a fixed size, so that three-dimensional convolution can be applied in the three-dimensional space, similar to the two-dimensional convolution operation on images. However, the point cloud scene recognition methods disclosed in the above literature still have great room for improvement in reducing the number of computing parameters, providing computing efficiency, and optimizing the fusion mechanism to improve the accuracy of point cloud recognition. Summary of the Invention

[0007] The present invention provides a point cloud scene recognition method based on dynamic sparse feature fusion, and using this method can efficiently and accurately recognize the point cloud frame of the current scene.

[0008] The present invention provides a point cloud scene recognition method based on dynamic sparse feature fusion, including:

[0009] Preprocess the original point cloud obtained from the current scene to obtain a preprocessed point cloud;

[0010] Construct a downsampling network, where the downsampling network includes cascaded downsampling modules at multiple levels. Each downsampling module includes a downsampling layer and an asymmetric convolutional layer. The preprocessed point cloud is input into the downsampling layer to obtain downsampled point cloud features, and the detailed features of the downsampled point cloud features are extracted through the asymmetric convolutional layer to obtain point cloud encoding features. The asymmetric convolutional layer includes multiple three-dimensional sub-convolutions, and only one dimension in the convolution kernel of the three-dimensional sub-convolution is not 1;

[0011] Construct an upsampling network, where the upsampling network includes cascaded upsampling modules at multiple levels. The point cloud decoding features with the largest size obtained through the cascaded upsampling modules at multiple levels are used as point cloud local features, and the feature points in the point cloud local features are aggregated to obtain the current point cloud global features;

[0012] Among them, each upsampling module includes an upsampling layer and a fusion layer. Through the fusion layer, weight distribution is performed on the feature points and channels of the upsampled point cloud features output by the upsampling layer and the point cloud encoding features of the corresponding size, and then the weighted upsampled point cloud features and the point cloud encoding features of the corresponding size are added together to obtain the point cloud decoding features of the corresponding size;

[0013] Compare the current point cloud global features with the global features in the historical database, and use the point cloud frame corresponding to the global feature with the closest Euclidean distance as the recognized point cloud of the current scene.

[0014] Further, the specific steps of the weight distribution on the feature points and channels of the upsampled point cloud features output by the upsampling layer and the point cloud encoding features of the corresponding size provided in the embodiment of the present invention are as follows:

[0015] The fusion layer includes a dynamic sparse feature fusion unit, which is composed of alternately stacked channel attention mechanisms and feature point attention mechanisms. Among them, through the channel attention mechanism, weights are calculated for each channel of the upsampled point cloud features and the point cloud encoding features of the corresponding size, and the obtained weights of each channel are multiplied by the corresponding channels to obtain the channel-weighted upsampled point cloud features and the point cloud encoding features of the corresponding size;

[0016] Through the feature point attention mechanism, weights are calculated for each feature point of the upsampled point cloud features and the point cloud encoding features of the corresponding size, and the obtained weights of each feature point are multiplied by the corresponding feature points to obtain the feature point-weighted upsampled point cloud features and the point cloud encoding features of the corresponding size.

[0017] Further, the fusion layer provided in the embodiment of the present invention further includes a combining unit, and the weighted upsampled point cloud features and the point cloud encoding features of the corresponding size are added through the combining unit.

[0018] Further, the specific steps for extracting the detailed features of the downsampled point cloud features through the asymmetric convolution layer provided in the embodiments of the present invention are as follows:

[0019] The number of three-dimensional sub-convolutions in the asymmetric convolution layer and the dimensional size of the convolution kernels in each three-dimensional sub-convolution are preset based on the current scene. The detailed features of the downsampled point cloud features are extracted through the set multiple three-dimensional sub-convolutions to obtain the point cloud encoded features.

[0020] Further, in the embodiments of the present invention, the smallest-size point cloud encoded features are extracted through the last-level downsampling module. In the last-level downsampling module, the asymmetric convolution layer is a dilated asymmetric convolution layer, and the convolution kernel is a dilated convolution kernel, which increases the receptive field of the smallest-size point cloud encoded features through this dilated convolution kernel.

[0021] Further, the downsampling network provided in the embodiments of the present invention further includes a three-dimensional convolution layer. The preprocessed point cloud is input into the three-dimensional convolution layer to obtain the preprocessed point cloud features, and the preprocessed point cloud features are input into the multi-level cascaded downsampling module for multi-level feature extraction to obtain multi-size point cloud encoded features.

[0022] Further, the preprocessing of the original point cloud obtained from the current scene provided in the embodiments of the present invention includes:

[0023] Filter the original point cloud according to the set height and radius range thresholds, voxelize the filtered original point cloud according to the set spatial resolution, and uniformly downsample the voxelized original point cloud to obtain the preprocessed point cloud.

[0024] The present invention also provides a point cloud scene recognition device based on dynamic sparse feature fusion, including:

[0025] A preprocessing module, configured to obtain the original point cloud from the current scene, and downsample the voxelized original point cloud to obtain the preprocessed point cloud;

[0026] A point cloud feature encoding module, configured to construct a downsampling network. The downsampling network includes multi-level cascaded downsampling modules. The downsampling module includes a downsampling layer and an asymmetric convolution layer. The preprocessed point cloud is input into the downsampling layer to obtain the downsampled point cloud features, and the detailed features of the downsampled point cloud features are extracted through the asymmetric convolution layer to obtain the point cloud encoded features. The asymmetric convolution layer includes multiple three-dimensional sub-convolutions, and only one dimension in the convolution kernel of the three-dimensional sub-convolution is not 1;

[0027] A point cloud feature decoding module, configured to construct an upsampling network. The upsampling network includes multi-level cascaded upsampling modules, and the largest-size point cloud decoded features obtained through the multi-level cascaded upsampling modules are used as the point cloud local features;

[0028] Among them, each upsampling module includes an upsampling layer and a fusion layer. The fusion layer respectively assigns weights to the feature points and feature channels of the upsampled point cloud features output by the upsampling layer and the point cloud encoded features of the corresponding size, and then adds the weighted upsampled point cloud features and the point cloud encoded features of the corresponding size to obtain the point cloud decoded features of the corresponding size;

[0029] An aggregation module is used to aggregate the feature points in the point cloud local features to obtain the current point cloud global features;

[0030] A retrieval module is used to compare the current point cloud global features with the global features in the historical database, and use the point cloud frame corresponding to the global feature with the closest Euclidean distance as the recognized point cloud of the current scene.

[0031] The present invention also provides a point cloud scene recognition device based on dynamic sparse feature fusion, including a memory and one or more processors. An executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the above-mentioned point cloud scene recognition method based on dynamic sparse feature fusion.

[0032] The present invention also provides a computer-readable storage medium, characterized in that a program is stored thereon, and when the program is executed by a processor, it implements the above-mentioned point cloud scene recognition method based on dynamic sparse feature fusion.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] In the downsampling process of the present invention, multiple three-dimensional sub-convolutions are used to extract detailed features from the downsampled point cloud features. Since only one dimension of the convolution kernel in the three-dimensional sub-convolution is not 1, the number of parameters is reduced during the process of extracting detailed features, and at the same time, the number of three-dimensional sub-convolutions can be flexibly configured based on the current scene information, providing operation efficiency while accurately extracting useful features in the current scene; in the upsampling process of the present invention, weights are assigned to the feature points and feature channels of the upsampled point cloud features and the point cloud encoded features of the corresponding size to optimize the point cloud local features, and then the point cloud frame of the current scene can be obtained more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of a point cloud scene recognition method based on dynamic sparse feature fusion provided by a specific embodiment of the present invention;

[0036] Figure 2 It is a flow block diagram of a point cloud scene recognition method based on dynamic sparse feature fusion provided by a specific embodiment of the present invention;

[0037] Figure 3The structural block diagram of a point cloud scene recognition device based on dynamic sparse feature fusion provided by a specific embodiment of the present invention. Detailed implementation manners

[0038] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. It should be noted that, without conflict, the features in the following embodiments and implementation manners can be combined with each other.

[0039] Unless otherwise specified, the terms (including scientific and technical terms) used herein have the ordinary meaning understood by those skilled in the art to which this technology belongs. In addition, it can be understood that the terms defined in the commonly used dictionary should be understood as having a meaning consistent with the context of the relevant field, and should not be understood as idealized or overly formal meanings.

[0040] In order to achieve the goals of reducing the number of parameters, improving the operation efficiency, and optimizing the local point cloud features to obtain a point cloud frame that is more matched to the current scene, the present invention provides a point cloud scene recognition method based on dynamic sparse feature fusion, as Figure 1 、 Figure 2 shown, including:

[0041] S1: Preprocess the original point cloud to obtain a preprocessed point cloud. The specific steps are as follows: obtain the original point cloud from the current scene, filter the original point cloud according to the set height and radius range thresholds, voxelize the filtered original point cloud according to the set spatial resolution, and uniformly downsample the voxelized original point cloud to obtain the preprocessed point cloud.

[0042] In a specific embodiment, the specific steps of uniformly downsampling the voxelized point cloud are as follows: select some point clouds in the voxelized point cloud as sampling points, construct a point cloud region group based on each sampling point, and replace the point cloud region group where the sampling point is located with the sampling point.

[0043] S2: Downsample the preprocessed point cloud to obtain multi-scale point cloud encoded features. The specific steps are as follows: construct a downsampling network, which includes multiple levels of cascaded downsampling modules. Each level of downsampling module includes a downsampling layer (downsample conv) and an asymmetric convolutional layer (DFC conv). Input the preprocessed point cloud into the downsampling layer to obtain downsampled point cloud features, and input the downsampled point cloud features into the asymmetric convolutional layer for detailed feature extraction to obtain point cloud encoded features. Through the layer-by-layer feature extraction of the multi-level cascaded downsampling modules, multi-scale point cloud encoded features are obtained. The asymmetric convolutional layer includes multiple three-dimensional sub-convolutions, and only one dimension in the convolution kernel of the three-dimensional sub-convolution is not 1. In the present invention, by setting that only one dimension in the convolution kernel of the three-dimensional sub-convolution is not 1 and the other dimensions are all 1, it is equivalent to decomposing the extraction of detailed features through three-dimensional convolution in the prior art into the extraction of detailed features through multiple one-dimensional sub-convolutions, thereby being able to reduce the number of parameters.

[0044] In a specific embodiment, based on the current scene, the number of three-dimensional sub-convolutions in the asymmetric convolutional layer and the dimension size of the convolution kernel in each three-dimensional sub-convolution are preset in advance. The downsampled point cloud features are subjected to detailed feature extraction through the set multiple three-dimensional sub-convolutions to obtain point cloud encoded features. Since only one dimension in the convolution kernel of the three-dimensional sub-convolution provided by the present invention is not 1, the three-dimensional sub-convolution provided by the present invention can be regarded as a one-dimensional sub-convolution. Therefore, multiple one-dimensional sub-convolutions with different dimensions can be flexibly configured based on the scene information features of the current scene, so as to be able to extract the detailed features of the downsampled point cloud features in a targeted manner to reflect the current scene information.

[0045] In a specific embodiment, the last-level downsampling module in the multi-level cascaded downsampling module is used to extract the smallest-size point cloud encoded features. The asymmetric convolutional layer in the last-level downsampling module is a dilated asymmetric convolutional layer (DFCconv(dilated d)), and the convolution kernel in the dilated asymmetric convolutional layer is a dilated convolution kernel. The receptive field of the smallest-size point cloud encoded features is increased through this dilated convolution kernel.

[0046] In a specific embodiment, the downsampling network further includes an ordinary three-dimensional convolutional layer (typical conv). Input the preprocessed point cloud into the three-dimensional convolutional layer to obtain preprocessed point cloud features, and input the preprocessed point cloud features into the multi-level cascaded downsampling module for multi-level feature extraction to obtain multi-scale point cloud encoded features, that is, multiple sizes of point element encoded features are output through multiple cascaded downsampling modules. Among them, the last-level downsampling module outputs the smallest-size point element encoded features.

[0047] S3: Construct an upsampling network, perform multi-level cascaded upsampling through the upsampling network to obtain point cloud local features, and aggregate the point cloud local features to obtain the current point cloud global feature:

[0048] Among them, each level of upsampling module includes an upsampling layer and a fusion layer. The fusion layer respectively assigns weights to the feature points and channels of the upsampled point cloud features and the point cloud coding features of the corresponding size output by the upsampling layer, and then adds the weighted upsampled point cloud features and the point cloud coding features of the corresponding size to obtain the point cloud decoding features of the corresponding size. In the process of upsampling at each level, the present invention assigns weights to the feature points and channels of the upsampled point cloud features and the point cloud coding features of the corresponding size to obtain more important feature points and channels, shielding unnecessary feature points and channels so that the feature points and channels match the information features of the current scene, thereby optimizing the local features of the point cloud, and further accurately obtaining a point cloud frame that matches the current scene.

[0049] In a specific embodiment, the feature points in the local feature LF of the point cloud are aggregated to obtain the current point cloud global feature descriptor:

[0050]

[0051] Among them, f is the feature point of the local feature of the point cloud, |LF| represents the number of feature points in LF; p f are the learning parameters.

[0052] In a specific embodiment, the fusion layer provided by the present invention includes a dynamic sparse feature fusion (DSFF) unit, which is composed of an alternating stack of a channel attention mechanism (channel-mixer) and a feature point attention mechanism (point-mixer), wherein the channel attention mechanism is used to calculate the weight of each channel of the upsampled point cloud features and the point cloud coding features of the corresponding size, and each channel weight obtained is multiplied by the corresponding channel to obtain the channel-weighted upsampled point cloud features and the point cloud coding features of the corresponding size.

[0053] The point attention mechanism is used to calculate the weight of each feature point of the upsampled point cloud features and the point cloud coding features of the corresponding size, and the weight of each feature point is multiplied by the corresponding feature point to obtain the weighted upsampled point cloud features and the point cloud coding features of the corresponding size.

[0054] In a specific embodiment, the fusion layer further includes a combining unit, which adds the weighted upsampled point cloud features and the point cloud coding features of the corresponding size.

[0055] S4: Compare the Euclidean distance between the global features of the current point cloud and the global features in the historical database, and use the point cloud frame corresponding to the global feature with the closest Euclidean distance as the recognition point cloud of the current scene.

[0056] Embodiment

[0057] In a specific embodiment of the present invention, a data preprocessing module data_process is constructed. A frame of raw point cloud is obtained from the current scene, and the raw point cloud is preprocessed by the data preprocessing module, that is, points within a fixed spatial range are selected from the raw point cloud. The fixed spatial range is between -20 cm and 200 cm in height and within a radius range of 30 m. Then, the selected point cloud is voxelized according to the set spatial resolution, and the voxelized point cloud is uniformly downsampled to control the number of points in the voxelized point cloud to 4096.

[0058] In a specific embodiment of the present invention, a downsampling network is constructed. The structure of the downsampling network is shown in Table 1. The constructed downsampling network includes a three-dimensional convolutional layer and four downsampling modules. Each layer of the downsampling module includes a downsampling layer and an asymmetric convolutional layer:

[0059] Among them, the convolutional kernel size of the three-dimensional convolutional layer is 5*5*5, and the stride is 1; the convolutional kernel size of the downsampling layer is 3*3*3, and the stride is 2; the asymmetric convolutional layer type network layer contains 8 sub-layers, and each sub-layer is a three-dimensional convolutional layer with a stride of 1. The convolutional kernel sizes are 3*1*1, 1*3*1, 1*1*3, 1*1*3, 3*1*1, 1*3*1, 1*1*3, 1*1*3 respectively; DFC conv(dilated 2) means that on the basis of the asymmetric convolutional layer, the convolutional kernel is changed to a dilated convolutional kernel with a dilation of 2.

[0060] Table 1 Structure settings of the downsampling network

[0061]

[0062] In a specific embodiment of the present invention, the downsampled point cloud encoding feature X9 output by the 9th layer in Table 1 is upsampled to obtain the upsampled point cloud feature XU9. The size of the upsampled point cloud feature XU9 is the same as that of the downsampled point cloud encoding X7 output by the 7th layer in Table 1. The upsampled point cloud feature XU9 and the downsampled point cloud encoding X7 are respectively input into the dynamic sparse feature fusion unit. The feature points and channels of the upsampled point cloud feature XU9 and the downsampled point cloud encoding X7 are weighted by the dynamic sparse feature fusion unit to obtain XU9' and X7' respectively. XU9' and X7' are added to obtain F5. F5 is upsampled to obtain FU5. The size of FU5 is the same as that of the downsampled point cloud encoding X5 output by the 5th layer in Table 1. FU5 and X5 are respectively input into the dynamic sparse feature fusion unit to obtain FU5' and X5' respectively. FU5' and X5' are added to obtain the local point cloud feature LF. The feature points in the obtained local point cloud feature LF are aggregated to obtain the current point cloud global feature descriptor as:

[0063]

[0064] Among them, f is the feature point of the local feature of the point cloud, and |LF| represents the number of feature points in LF; p f is a learned parameter, and p is preset f = 2.7, and the size of the obtained global feature descriptor of the point cloud is 1*256.

[0065] In the specific embodiment of the present invention, the dynamic sparse feature fusion unit provided is composed of two network layers, namely, a channel attention mechanism and a feature point attention mechanism, which are alternately stacked. The specific configuration is shown in Table 2. The channel attention mechanism and the feature point attention mechanism are essentially multi-layer perceptrons. The fully connected networks built in the channel attention mechanism and the feature point attention mechanism calculate a weight for each channel and each feature point respectively, and then multiply the calculated weight by the feature of the corresponding channel or feature point to obtain the upsampled point cloud feature after weighting and the point cloud encoding feature of the corresponding size.

[0066] Table 2 Network structure configuration of the dynamic sparse feature fusion (DSFF) module

[0067] Hierarchy Type Number of fully-connected layers Input size Hidden layer size 1 channel-mixer 1 256 256 2 point-mixer 1 256 256 3 channel-mixer 1 256 256 4 point-mixer 1 256 256

[0068] In a specific embodiment, a retrieval module SEA is constructed. The global feature descriptor of the current point cloud is input, and it is compared with the global features extracted in the historical database, and the point cloud frame cloud_nearest with the closest Euclidean distance between the global feature and the descriptor is returned as the recognition result of the current scene.

[0069] Based on the same inventive concept, the present invention also provides a point cloud scene recognition device based on dynamic sparse feature fusion, as Figure 3 shown, including:

[0070] A preprocessing module, which is used to obtain the original point cloud from the current scene, filter the original point cloud according to the set height and radius range thresholds, voxelize the filtered original point cloud according to the set spatial resolution, and uniformly downsample the voxelized original point cloud to obtain the preprocessed point cloud.

[0071] The point cloud feature encoding module ENC is used to construct a downsampling network. The downsampling network includes multiple levels of cascaded downsampling modules. Each level of downsampling module includes a downsampling layer (downsample conv) and an asymmetric convolutional layer (DFC conv). The preprocessed point cloud is input into the downsampling layer to obtain downsampled point cloud features. The downsampled point cloud features are input into the asymmetric convolutional layer for detailed feature extraction to obtain point cloud encoding features. Through the hierarchical feature extraction of the multiple levels of cascaded downsampling modules, multi-scale point cloud encoding features are obtained. The asymmetric convolutional layer includes multiple three-dimensional sub-convolutions, and only one dimension in the convolutional kernel of the three-dimensional sub-convolution is not 1. In the present invention, by setting that only one dimension in the convolutional kernel of the three-dimensional sub-convolution is not 1 and the other dimensions are all 1, it is equivalent to decomposing the extraction of detailed features through three-dimensional convolution in the prior art into multiple one-dimensional convolutions for extracting detailed features, thereby being able to reduce the number of parameters.

[0072] Optionally, based on the current scene, the number of three-dimensional sub-convolutions in the asymmetric convolutional layer and the dimension size of the convolutional kernel in each three-dimensional sub-convolution are preset. The downsampled point cloud features are subjected to detailed feature extraction through the set multiple three-dimensional sub-convolutions to obtain point cloud encoding features.

[0073] Optionally, the last level of downsampling module in the point cloud feature encoding module is used to extract the smallest-scale point cloud encoding features. The asymmetric convolutional layer used in the last level of downsampling module is a dilated asymmetric convolutional layer. The convolutional kernel in the dilated asymmetric convolutional layer is a dilated convolutional kernel, and the receptive field of the smallest-scale point cloud encoding features is increased through this dilated convolutional kernel.

[0074] Optionally, the downsampling network in the point cloud feature encoding module further includes a three-dimensional convolutional layer. The preprocessed point cloud is input into the three-dimensional convolutional layer to obtain preprocessed point cloud features. The preprocessed point cloud features are input into the multiple levels of cascaded downsampling modules for multi-level feature extraction to obtain multi-scale point cloud encoding features.

[0075] The point cloud feature decoding module DEC is used to construct an upsampling network, perform multi-level linked upsampling to obtain local point cloud features, and aggregate the local point cloud features to obtain the current global point cloud feature: Among them, each level of upsampling module includes an upsampling layer and a fusion layer. The fusion layer respectively assigns weights to the feature points and channels of the upsampled point cloud feature output by the upsampling layer and the point cloud encoding feature of the corresponding size, and then adds the weighted upsampled point cloud feature and the point cloud encoding feature of the corresponding size to obtain the point cloud decoding feature of the corresponding size. In the upsampling process of each level of the present invention, weights are assigned to the feature points and channels of the upsampled point cloud feature and the point cloud encoding feature of the corresponding size to obtain relatively important feature points and channels, shield unnecessary feature points and channels, so that the feature points and channels match the information features of the current scene, thereby optimizing the local point cloud features and further accurately obtaining a point cloud frame that matches the current scene.

[0076] Optionally, the fusion layer in the point cloud feature decoding module includes a dynamic sparse feature fusion unit, which is composed of alternately stacked channel attention mechanisms and feature point attention mechanisms. Among them, the channel attention mechanism calculates weights for each channel of the upsampled point cloud feature and the point cloud encoding feature of the corresponding size, and multiplies each obtained channel weight by the corresponding channel to obtain the upsampled point cloud feature and the point cloud encoding feature of the corresponding size after channel weighting; the point attention mechanism calculates weights for each feature point of the upsampled point cloud feature and the point cloud encoding feature of the corresponding size, and multiplies the weight of each obtained feature point by the corresponding feature point to obtain the upsampled point cloud feature and the point cloud encoding feature of the corresponding size after feature point weighting.

[0077] Optionally, the fusion layer in the point cloud feature decoding module further includes a combining unit, which adds the weighted upsampled point cloud feature and the point cloud encoding feature of the corresponding size through the combining unit.

[0078] The aggregation module is used to aggregate the feature points in the local point cloud feature to obtain the current global point cloud feature.

[0079] The retrieval module is used to compare the current global point cloud feature with the global features in the historical database, and use the point cloud frame corresponding to the global feature with the closest Euclidean distance as the recognized point cloud of the current scene.

[0080] Since the principle of the device in the embodiment of the present invention for solving problems is similar to the point cloud scene recognition method in the above embodiment of the present invention, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be elaborated here.

[0081] Based on the same inventive concept, the present invention further provides a point cloud scene recognition device based on dynamic sparse feature fusion, which is characterized by including a memory and one or more processors. An executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the point cloud scene recognition method in any one of the above embodiments.

[0082] Based on the same inventive concept, the present invention further provides a computer-readable storage medium, which is characterized in that a program is stored thereon. When the program is executed by a processor, it implements the point cloud scene recognition method in any one of the above embodiments.

[0083] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A point cloud scene recognition method based on dynamic sparse feature fusion, characterized in that Including: Preprocessing the original point cloud obtained from the current scene to obtain a preprocessed point cloud; Constructing a downsampling network, the downsampling network includes a multi-level cascaded downsampling module, the downsampling module includes a downsampling layer and an asymmetric convolutional layer, inputting the preprocessed point cloud into the downsampling layer to obtain downsampled point cloud features, and extracting the detailed features of the downsampled point cloud features through the asymmetric convolutional layer to obtain point cloud encoded features. The asymmetric convolutional layer includes multiple three-dimensional sub-convolutions, and only one dimension in the convolutional kernel of the three-dimensional sub-convolution is not 1; Constructing an upsampling network, the upsampling network includes a multi-level cascaded upsampling module, using the largest-sized point cloud decoded feature obtained through the multi-level cascaded upsampling module as the point cloud local feature, and aggregating the feature points in the point cloud local feature to obtain the current point cloud global feature; Wherein, each level of the upsampling module includes an upsampling layer and a fusion layer. Through the fusion layer, weight distribution is respectively performed on the feature points and channels of the upsampled point cloud feature output by the upsampling layer and the point cloud encoded feature of the corresponding size, and then the weighted upsampled point cloud feature and the point cloud encoded feature of the corresponding size are added together to obtain the point cloud decoded feature of the corresponding size; Comparing the current point cloud global feature with the global features in the historical database, and using the point cloud frame corresponding to the global feature with the closest Euclidean distance as the recognized point cloud of the current scene.

2. The method for point cloud scene recognition based on dynamic sparse feature fusion according to claim 1, wherein The weight distribution of the feature points and channels of the upsampled point cloud feature output by the upsampling layer and the point cloud encoded feature of the corresponding size through the fusion layer respectively includes: The fusion layer includes a dynamic sparse feature fusion unit, and the dynamic sparse feature fusion unit is composed of alternately stacked channel attention mechanism and feature point attention mechanism. Among them, the weight of each channel of the upsampled point cloud feature and the point cloud encoded feature of the corresponding size is calculated through the channel attention mechanism, and the obtained weight of each channel is multiplied by the corresponding channel to obtain the channel-weighted upsampled point cloud feature and the point cloud encoded feature of the corresponding size; The weight of each feature point of the upsampled point cloud feature and the point cloud encoded feature of the corresponding size is calculated through the feature point attention mechanism, and the obtained weight of each feature point is multiplied by the corresponding feature point to obtain the feature point-weighted upsampled point cloud feature and the point cloud encoded feature of the corresponding size.

3. The method for point cloud scene recognition based on dynamic sparse feature fusion according to claim 2, wherein The fusion layer further includes a combining unit, and the weighted upsampled point cloud feature and the point cloud encoded feature of the corresponding size are added through the combining unit.

4. The method for point cloud scene recognition based on dynamic sparse feature fusion according to claim 1, wherein The extraction of the detailed features of the downsampled point cloud features through the asymmetric convolutional layer includes: The number of three-dimensional sub-convolutions in the asymmetric convolutional layer and the dimension size of the convolutional kernel in each three-dimensional sub-convolution are preset based on the current scene, and the detailed features of the downsampled point cloud features are extracted through the set multiple three-dimensional sub-convolutions to obtain point cloud encoded features.

5. The method for point cloud scene recognition based on dynamic sparse feature fusion according to claim 1, wherein Extracting the smallest-sized point cloud encoded feature through the last-level downsampling module. The asymmetric convolutional layer in the last-level downsampling module is a dilated asymmetric convolutional layer, and the convolutional kernel is a dilated convolutional kernel, and the receptive field of the smallest-sized point cloud encoded feature is increased through this dilated convolutional kernel.

6. The method for point cloud scene recognition based on dynamic sparse feature fusion according to claim 1, wherein The downsampling network further includes a three-dimensional convolutional layer. The preprocessed point cloud is input into the three-dimensional convolutional layer to obtain preprocessed point cloud features, and the preprocessed point cloud features are input into a multi-level cascaded downsampling module for multi-level feature extraction to obtain multi-scale point cloud encoded features.

7. The method for point cloud scene recognition based on dynamic sparse feature fusion according to claim 1, wherein The preprocessing of the original point cloud obtained from the current scene includes: Filtering the original point cloud according to the set height and radius range thresholds, voxelizing the filtered original point cloud according to the set spatial resolution, and uniformly downsampling the voxelized original point cloud to obtain the preprocessed point cloud.

8. A point cloud scene recognition device based on dynamic sparse feature fusion, characterized in that, It includes: A preprocessing module for obtaining the original point cloud from the current scene and downsampling the voxelized original point cloud to obtain the preprocessed point cloud; A point cloud feature encoding module for constructing a downsampling network. The downsampling network includes a multi-level cascaded downsampling module. The downsampling module includes a downsampling layer and an asymmetric convolutional layer. The preprocessed point cloud is input into the downsampling layer to obtain downsampled point cloud features, and the detailed features of the downsampled point cloud features are extracted through the asymmetric convolutional layer to obtain point cloud encoded features. The asymmetric convolutional layer includes multiple three-dimensional sub-convolutions, and only one dimension in the convolution kernel of the three-dimensional sub-convolution is not 1; A point cloud feature decoding module for constructing an upsampling network. The upsampling network includes a multi-level cascaded upsampling module, and the point cloud decoding feature with the largest size obtained through the multi-level cascaded upsampling module is used as the point cloud local feature; Among them, each upsampling module includes an upsampling layer and a fusion layer. The fusion layer respectively assigns weights to the feature points and feature channels of the upsampled point cloud features output by the upsampling layer and the point cloud encoded features of the corresponding size, and then adds the weighted upsampled point cloud features and the point cloud encoded features of the corresponding size to obtain the point cloud decoding features of the corresponding size; An aggregation module for aggregating the feature points in the point cloud local features to obtain the current point cloud global feature; A retrieval module for comparing the current point cloud global feature with the global features in the historical database, and taking the point cloud frame corresponding to the global feature with the closest Euclidean distance as the recognized point cloud of the current scene.

9. A point cloud scene recognition device based on dynamic sparse feature fusion, characterized in that, It includes a memory and one or more processors. Executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the point cloud scene recognition method based on dynamic sparse feature fusion according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A program is stored thereon. When the program is executed by a processor, it implements the point cloud scene recognition method based on dynamic sparse feature fusion according to any one of claims 1-7.

Images