Perception method for decoupling graph convolution point cloud perception model based on lightweight geometric information

Abstract

The application provides a perception method based on a light-weight geometric information decoupling graph convolution point cloud perception model, and belongs to the technical fields of point cloud perception and deep learning. The method comprises the following steps: obtaining candidate region retrieval by discretizing the original point cloud data space coordinates through 2D voxel down-sampling based on the light-weight geometric information decoupling graph convolution point cloud perception model; obtaining adjacent edge index by using a scale factor-based expansion KNN algorithm based on the candidate region. The encoder performs multiple graph convolution and down-sampling processes based on the adjacent edge index and the point cloud subset, and the decoder performs multiple up-sampling and graph convolution processes based on the point cloud subset and the adjacent edge index. The application can complete point cloud classification by extracting multi-layer graph convolution features through the encoder. For point cloud segmentation and target identification tasks, the decoder gradually reconstructs spatial details in the up-sampling stage, effectively compensating for the loss of geometric information caused by down-sampling in the encoding process.

Description

Technical Field

[0001] This application belongs to the field of point cloud perception and deep learning technology, specifically involving a perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model. Background Technology

[0002] With the rapid development of fields such as autonomous driving, robot navigation, and augmented reality, the importance of 3D point cloud perception technology is becoming increasingly prominent. As a typical type of unstructured data, point clouds have characteristics such as disorder, sparsity, and irregularity, making it difficult to directly transfer traditional convolutional neural networks designed for regular images. They generally suffer from problems such as low computational efficiency and insufficient ability to model local geometric structures.

[0003] Currently, point cloud processing methods are mainly divided into the following categories: (1) methods based on multi-view rendering, which rely on two-dimensional convolutional networks but are sensitive to viewpoint changes; (2) methods based on voxel meshing, which are susceptible to quantization errors and whose computational overhead increases cubically with resolution; (3) methods based on direct point cloud processing, such as the PointNet series of networks, which can preserve the original geometric information but still have limitations in local feature interaction and multi-level context modeling. In recent years, graph convolutional neural networks have shown superior performance in point cloud classification, segmentation, and object detection tasks because they can directly model the topological relationships between points in point clouds.

[0004] However, existing graph convolution methods still face the following key challenges: First, the high coupling between geometric structure information and semantic features in point clouds leads to poor robustness of the network to geometric transformations and makes it difficult to explicitly utilize geometric priors; Second, the local neighborhood feature aggregation method is singular, making it difficult to effectively distinguish key geometric structures such as edges and corners in complex scenes; Third, the computational complexity of global K-nearest neighbor search required to construct the graph structure is high, making it difficult to meet the needs of real-time processing of large-scale point clouds; Fourth, during feature encoding, downsampling operations are prone to causing the loss of spatial details, while existing upsampling methods mostly rely on interpolation or deconvolution, making it difficult to accurately recover geometric details. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this application provides a perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model. The technical problem to be solved by this application is achieved through the following technical solution: A perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model includes: S100: Acquire raw point cloud data, corresponding task requirements, and a constructed lightweight geometric information decoupling graph convolutional point cloud perception model; wherein, the task requirements include classification tasks, segmentation tasks, and recognition tasks. S200: The original point cloud data spatial coordinates are discretized into voxel indices using 2D voxel downsampling, and the XY plane components are mapped into one-dimensional sequence indices to transform the floating-point distance comparison of spatial groupings into candidate region retrieval based on integer indices. Based on the candidate regions, the adjacent edge index is obtained using the scale factor-based extended KNN algorithm. The output features are obtained by graph convolution and downsampling based on the point cloud subsets and adjacent edge indexes to perform classification, segmentation or recognition tasks.

[0006] Beneficial effects: This application provides a perception method based on a lightweight geometric information decoupled graph convolutional point cloud perception model. The original point cloud data is input into this model to obtain the processed result. In this model, the processing module downsamples the original point cloud data to obtain a subset of the point cloud, and uses a scale-factor-based extended KNN (K-Nearest Neighbor) algorithm to obtain the neighbor index. The encoder performs multiple graph convolutions and downsampling processes based on the neighbor index and the point cloud subset, while the decoder performs multiple upsampling and graph convolution processes based on the point cloud subset and the neighbor index. For point cloud classification tasks, this application can complete point cloud classification by extracting multi-layer graph convolutional features using the encoder; for point cloud segmentation and target recognition tasks, the decoder gradually reconstructs spatial details during the upsampling stage, effectively compensating for the geometric information loss caused by downsampling during the encoding process. This application avoids the original layer-by-layer high-dimensional feature space K-Nearest Neighbor calculation, effectively reducing the computational overhead and complexity of graph structure updates while maintaining high-precision point cloud perception capabilities.

[0007] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model provided in this application. Figure 2 This is a detailed architecture diagram of the lightweight geometric information decoupling graph convolutional point cloud perception model provided by the present invention; Figure 3 This is a schematic diagram of the specific structure of the fusion module and prediction layer provided by the present invention; Figure 4 This is a diagram of the point cloud perception model architecture for classification tasks provided by the present invention. Figure 5 This is a diagram of the point cloud perception model architecture for segmentation and target recognition tasks provided by the present invention. Detailed Implementation

[0009] The present application will be described in further detail below with reference to specific embodiments, but the implementation of the present application is not limited thereto.

[0010] like Figure 1 As shown, this application provides a perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model, including: S100: Acquire raw point cloud data, corresponding task requirements, and a constructed lightweight geometric information decoupling graph convolutional point cloud perception model; wherein, the task requirements include classification tasks, segmentation tasks, and recognition tasks. S200: The original point cloud data spatial coordinates are discretized into voxel indices using 2D voxel downsampling, and the XY plane components are mapped into one-dimensional sequence indices to transform the floating-point distance comparison of spatial groupings into candidate region retrieval based on integer indices. Based on the candidate regions, the adjacent edge index is obtained using the scale factor-based extended KNN algorithm. The output features are obtained by graph convolution and downsampling based on the point cloud subsets and adjacent edge indexes to perform classification, segmentation or recognition tasks.

[0011] refer to Figure 2 and Figure 3 As shown, the lightweight geometric information decoupled graph convolutional point cloud perception model of this application includes a processing module, an encoder, a decoder, a fusion module selector, a fusion module, and a prediction layer. The processing module uses downsampling and dilated K-nearest neighbor algorithms to process the input data. The encoder includes four alternately connected coded graph convolutional modules and three downsampling layers. The four coded graph convolutional modules consist of four stacked coded graph convolutional layers, with a downsampling layer between each two coded graph convolutional modules. The decoder includes three decoded graph convolutional modules, three decoded convolutional layers, and three... The upsampling layer; the three decoding graph convolutional modules include three stacked decoding graph convolutional layers, with one upsampling layer followed by one decoding convolutional layer, and one decoding convolutional layer followed by one decoding graph convolutional module; the outputs of the four encoding graph convolutional modules and the three decoding graph convolutional modules are all connected to the input of the fusion module selector, the output of the encoder is connected to the input of the decoder, and the upsampling layer and the corresponding downsampling layer are connected by skip connections; both the encoding graph convolutional layer and the decoding graph convolutional layer include a max pooling layer, a convolutional layer, and a... connected sequentially. (Linear rectified function) activation layer and normalization layer; except for the first encoding graph convolutional layer, all other encoding graph convolutional layers in the encoder use residual connections, and all decoding graph convolutional layers use residual connections; When the task requirement is a classification task, the encoding graph convolutional layer and all downsampling layers are in the working state, and the decoder is in the non-working state. When the task requirement is a segmentation task or a recognition task, both the encoder and the decoder are in working state. The input of the fusion module selector comes from the output of the last coded graph convolutional layer in each coded graph convolutional module and the output of the last decoded graph convolutional layer in each decoded graph convolutional module.

[0012] As a specific implementation of this application, S200 includes: S210: The original point cloud data is downsampled using 2D (two-dimensional) voxels to obtain a subset of the point cloud. The neighbor index is obtained using the scale factor-based dilated K-nearest neighbor algorithm. Based on the neighbor index and the subset of the point cloud, multiple graph convolutions and downsampling processes are performed to obtain the first output feature of the multi-layer graph convolution. S220: When the task requirement is a classification task, the first output feature is deeply fused to obtain the first fused feature. Classification is performed based on the first fused feature to obtain the classification result. When the task requirement is a segmentation or recognition task, the first output feature is upsampled and graph convolution is performed multiple times to obtain the second output feature. Some output features are selected from the first output feature and the second output feature and deeply fused to obtain the second fused feature. Segmentation or recognition is then performed based on the second fused feature to obtain the execution result of the segmentation or recognition task.

[0013] refer to Figure 4 As shown, the processing module downsamples the original point cloud data using 2D voxel downsampling to obtain multiple point cloud subsets that retain key geometric features; it then uses a scale factor-based dilated K-nearest neighbor algorithm to obtain neighboring edge indices on these point cloud subsets; based on these neighboring edge indices and the point cloud subsets, the encoder performs multiple graph convolutions and downsampling processes on the original point cloud data and the point cloud subsets to obtain the first output feature of the multi-layer graph convolution; the fusion module selector outputs the first output feature to the fusion module, which performs deep fusion of all the first output features to obtain the first fused feature; and the prediction layer performs classification based on the first fused feature to obtain the classification result for the classification task.

[0014] refer to Figure 5As shown, when the task requirement is segmentation or recognition, the original point cloud data is input into a lightweight geometric information decoupled graph convolutional point cloud perception model. The processing module downsamples the original point cloud data using 2D voxel downsampling to obtain multiple point cloud subsets that retain key geometric features. The scale factor-based dilated K-nearest neighbor algorithm is used to obtain neighboring edge indices on these point cloud subsets. The encoder, based on the neighboring edge indices and point cloud subsets, performs multiple graph convolutions and downsampling processes on the original point cloud data and the point cloud subsets to obtain a first output feature of multi-layer graph convolution. The decoder performs multiple upsampling and graph convolution operations on the first output feature to obtain a second output feature of multi-layer graph convolution. The fusion module selector selects a portion of the output features from the first and second output features and inputs them into the fusion module. The fusion module performs deep fusion on the selected output features to obtain a second fused feature. Segmentation or recognition is then performed based on the second fused feature to obtain the execution result of the segmentation or recognition task.

[0015] In one specific embodiment of this application, the downsampling process performed on the original point cloud data in S210 or S220 by 2D voxel downsampling to obtain multiple point cloud subsets that retain key geometric features includes: a1. Extract the three-dimensional spatial coordinates of the original point cloud data, construct a 2D voxel mesh system, and discretize the extracted three-dimensional spatial coordinates into voxel indices; This step extracts the three-dimensional spatial coordinates of the original point cloud data, constructs a 2D voxel mesh system based on the three-dimensional spatial coordinates, and then discretizes the original coordinates corresponding to the original point cloud data into voxel indices. b1, the XY plane components of the voxel index are mapped to a one-dimensional index sequence through linearization processing, so as to obtain multiple linearized groups by spatial grouping; The mapping relationship is as follows:

[0016] in, , These represent the number of voxel grids in the X and Y directions, respectively. 、 is the integer index of the voxel in the XY plane.

[0017] c1, calculate the mean coordinates of each grouping point in the original coordinate space as the centroid; In this step, the mean coordinates of all points in each linearized group are calculated within the original coordinate space, and this mean is used as the centroid; wherein, the original coordinate space is the coordinate space of the original point cloud data; The centroid calculation relation is as follows:

[0018] where is the set of point indices in the linearized grouping and is the XY coordinate of the i point, and denotes belonging to the two-dimensional real vector space.

[0019] d1. Calculate the distance from each point in each linearized grouping to the centroid, and select the point with the minimum distance as the representative point of the corresponding voxel grid in the 2D voxel grid system; the representative point calculation relation is:

[0020] where is the Euclidean norm.

[0021] e1. Gather all the representative points, and by comparing with the preset target number of points, obtain multiple point cloud subsets with a fixed number of points by means of random sampling or random complementing; In this step, the representative points of all voxel grids are gathered and compared with the preset target number of points. If the number of representative points is greater than or equal to the target number of points, randomly select from the representative points the same number of representative points as the target number of points; if less, retain all the representative points and randomly complement from the remaining unselected points until finally obtaining multiple point cloud subsets with a fixed number of points equal to the target number of points.

[0022] Specifically, if the number of representative points M1 is greater than or equal to the target M2, randomly select M2 from the representative points; if M1 < M2, retain all the representative points and randomly complement to M2 from the remaining unselected points (allowing repeated sampling if the candidates are insufficient), ensuring that the returned length is fixed at M2.

[0023] In a specific implementation manner of this application, obtaining the adjacent edge indices on the point cloud subsets in S210 and S220 by using the dilation K-nearest neighbor algorithm based on the scale factor both includes: a2. Adopt a double-block strategy to divide the original point cloud data and each point cloud subset into multiple query blocks and multiple reference blocks; b2. For the original point cloud data, calculate the first distance matrix between each corresponding query block and each reference block, and for each point cloud subset, calculate the second distance matrix between each corresponding query block and each reference block; c2, based on the first distance matrix and the second distance matrix, select points from the corresponding reference block whose distance is less than or equal to the radius threshold as candidate neighbor points for each point in each corresponding query block; This application uses a scale factor as a radius threshold to filter the distances of neighboring points, fuses the current optimal distance candidate set, and updates the top-k nearest neighbors. The scale factor is determined based on specific tests on the dataset. Specifically, when the inflation coefficient is greater than 1, the algorithm performs inflation sampling, that is, sampling from the top-k (first k) nearest neighbors in ascending order of distance step size to k, to enhance neighborhood diversity. The inflation rate is designed differently for different point cloud perception tasks. Specifically, the inflation rate is designed to be 1 for classification, while for tasks such as segmentation and object recognition, every 4 layers are grouped together, with the same inflation rate within each group, and the inflation rates between groups are 1, 2, 3, and 4 respectively.

[0024] d2, merge all candidate neighbor points, and then obtain multiple final neighbor points that meet the quantity requirements by selection or filling. If the number of candidate nearest neighbors is insufficient, appropriate processing is performed based on the self-filling flag to obtain multiple final neighbor points that meet the quantity requirements.

[0025] e2, the multiple final neighbor points are mapped to the original point cloud data to form a first neighbor edge index, and the point cloud subset is mapped to the multiple final neighbor points to form a second neighbor edge index; f2, the first neighboring edge index and the second neighboring edge index are combined to form a neighboring edge index.

[0026] In one specific embodiment of this application, in steps S210 and S220, the original point cloud data and the subset of point clouds are subjected to multiple graph convolution and downsampling processes to obtain the first output feature of the multi-layer graph convolution, including: a3, the first layer of the encoding graph convolutional layer takes each point as the center point, uses the neighbor edge index to extract the feature vector of the neighbor point, calculates the difference between the feature vector of the center point and the neighbor point, and then performs max pooling aggregation, convolution, activation and normalization to obtain the output feature of the first layer of the encoding graph convolution; In the first coding graph convolutional module of this application, the first coding graph convolutional layer takes the original point cloud data and each point cloud subset as center points, and then uses the corresponding neighbor index to extract the feature vectors of the corresponding neighbor points in the first layer; the difference between the feature vectors of the center point and each neighbor point is aggregated by max pooling in the neighbor dimension, and then convolution, activation and normalization are performed to obtain the first layer of coding normalized features, which are used as the output features of the first layer of coding graph convolution. This application utilizes an adjacent edge index to extract features of the center point. Features of neighboring points Calculate the difference between eigenvectors and through function pairs Calculate the maximum value across K dimensions to obtain The maximum pooling calculation formula is:

[0027] In one specific embodiment, the original point cloud data contains 1024 points, the number of neighbors (i.e., dimension K) is fixed at 16, the convolutional input channels are 3, and the output channels are 64. During the max pooling aggregation process, a local neighborhood graph is constructed by calculating the difference in feature vectors of the 16 neighboring points, and the local neighborhood graph is aggregated by max pooling, which requires simultaneous calculation of all input channels.

[0028] b3, the remaining coding graph convolutional layers use the neighbor index to extract the neighbor point feature vectors from the output features of the upper coding graph convolution, perform max pooling aggregation, convolution, activation, and normalization processing, and then add them to the output features of the upper layer through residual connection to obtain the output features of this layer; In the first coding graph convolutional module of this application, the second to fourth coding graph convolutional layers use the corresponding neighbor index to extract the first-layer feature vectors of neighboring points from the output features of the previous coding graph convolutional layer; the difference between the feature vectors of the center point and each neighboring point is aggregated by max pooling in the neighbor dimension, and then convolution, activation, and normalization are performed to obtain the normalized features of this layer; the normalized features of this layer are added to the output features of the previous coding graph convolutional layer through residual connection to obtain the output features of the current coding graph convolutional layer. c3, the downsampling layer downsamples the output features of the previous encoding map convolutional module and outputs them to the next encoding map convolutional module; The first downsampling layer of this application downsamples the output features of the first coding map convolutional module to obtain the first downsampled output features, which are then input into the second coding map convolutional module. d3, using the convolutional output features of each layer of the encoding graph as the first output feature; In this application, each of the second to fourth coding graph convolutional modules uses the corresponding neighbor index to extract the first-layer feature vector of neighboring points from the output features of the previous coding graph convolution. The difference between the feature vectors of the center point and each neighboring point is aggregated by max pooling along the neighbor dimension, and then processed by convolution, activation, and normalization to obtain the normalized feature of this layer. The normalized feature of this layer is added to the output feature of the previous coding graph convolution through residual connection to obtain the output feature of this layer. The second to third downsampling layers downsample the output features of the previous coding graph convolutional module to obtain the downsampled output features of this layer, which are then input into the next coding graph convolutional module. The output features of each coding graph convolution are used as the first output features.

[0029] When the task requirement is classification, the first output features of the 16 layers extracted by the encoder (with point dimensions of 1024, 512, 256, and 128 respectively) are padded to a 1024-dimensional feature set and then concatenated before being fed into the fusion module. The fusion module consists of a single 1×1 convolutional layer. When the number of concatenation layers is reduced to 4, the computational cost of the network model is compressed to its maximum. The output of the fusion module is processed by max pooling and average pooling in the 1024-dimensional feature set to obtain two global features. These are then concatenated along the feature channel dimension to form a comprehensive global feature representation. The relationship between max pooling and average pooling is as follows:

[0030] in, This represents the global feature of max pooling. Indicates fusion characteristics, b For batch size, N For points, c The number of feature channels, This represents the global features of average pooling.

[0031] Global features are input into the prediction layer for classification prediction, resulting in a total of 40 categories.

[0032] The encoder in this application consists of 16 layers of coding graph convolutional layers. Every 4 layers of coding graph convolutional layers undergo a downsampling operation, which reduces the number of points N=1024 and the multi-level dimension to half of the original value, thereby reducing the computational load of the network.

[0033] In one specific embodiment of this application, S220 involves performing multiple upsampling and graph convolution operations on the first output feature to obtain a second output feature with multiple graph convolutions, including: a4, the upsampling layer upsamples the first output feature; In this application, the first upsampling layer performs upsampling processing on the first output feature to obtain the first upsampling output feature; This application generates upsampled features through a progressive upsampling operation. The principle is to copy the shallow features output by the early-stage encoding map convolutional module in the encoder, and then replace the feature values ​​of each voxel grid in the deep features output by the later-stage encoding map convolutional module with the points located within the same voxel grid in the shallow features. The downsampling algorithm directly obtains the correspondence between downsampled points and other points within the same voxel grid.

[0034] b4, the decoding convolutional layer convolves the output features of the upsampling layer; the first decoding convolutional layer of this application convolves the output features of the first upsampling layer to obtain the first convolutional output features; c4, the decoding graph convolutional layer uses the neighbor index to extract the feature vector of neighbor points from the input features, and then processes it through max pooling aggregation, convolution, activation, and normalization. Finally, it is added to the input features through residual connection to obtain the output features of the decoding graph convolutional layer of this layer. The input of the first decoding graph convolutional layer is the output of the decoding convolutional layer, and the rest are the output features of the previous layer's decoding graph convolutional layer. In this application, each decoding graph convolutional layer uses the corresponding neighbor index to extract the first-layer feature vector of neighbor points from the input features; the difference between the feature vectors of the center point and each neighbor point is aggregated by max pooling in the neighbor dimension, and then processed by convolution, activation, and normalization to obtain the decoding normalized feature of this layer; the decoding normalized feature of this layer is added to the output feature of the first layer convolution through residual connection to obtain the decoding graph convolution output feature of this layer; wherein, the input feature of the first layer decoding graph convolutional layer in each decoding graph convolutional module is the output feature of the previous layer decoding graph convolution, and the input of the second to third layers decoding graph convolutional layers is the output feature of the previous layer decoding graph convolution; d4 uses the convolutional output features of each layer of the decoded graph as the second output feature.

[0035] The decoder in this application uses stacked 3 layers of decoding graph convolutional layers for enhanced iteration, such as... Figure 5 As shown, upsampling and skip connections are repeated 3 times, thus restoring the number of points from 128 to 1024. The upsampling operation will be performed 3 times, upsampling 128 points of the 15th coding graph convolutional layer to 256, upsampling 256 points of the 11th coding graph convolutional layer to 512, and upsampling 512 points of the 7th coding graph convolutional layer to 1024.

[0036] The upsampled points are concatenated with the features of the original corresponding points, increasing the output channel dimension from 64 to 128. Then, a dimensionality reduction operation is performed to reduce the high dimension of the output channel back to 64.

[0037] In one specific embodiment of this application, in step S220, selecting a portion of the output features from the first output features and the second output features, and then performing deep fusion to obtain the second fused feature includes: The fusion module selector selects the first output feature of the last coded graph convolutional layer in each coded graph convolutional module and the second output feature of the last decoded graph convolutional layer in each decoded graph convolutional module as partial output features; the fusion module performs deep fusion on the selected partial output features to obtain the second fused feature.

[0038] The fusion module of this application consists of 7 layers, of which 4 layers correspond to the output features of the 4th, 7th, 11th and 15th layers of the encoding graph convolutional layer, respectively, and the other 3 layers are the features of the 3rd, 7th and 11th layers of the U-net (U-shaped network) decoding graph convolutional layer. These features are filled back to the 1024-point dimension by upsampling copy replacement method or zero-padding method before being spliced ​​and sent to the fusion layer for fusion.

[0039] This application fills the shallow features output by the convolutional module of the early decoding map in a specific encoder with the enhanced local features of a specific layer in the U-net decoder. After restoring the original N points in the number of points dimension, the features are concatenated and then input into the fusion module to fuse the shallow features with the deep features output by the encoder. The fused second feature is then input into the max pooling module, pooled in the number of points dimension, and then sent to the prediction layer for segmentation, target recognition, and other predictions.

[0040] The lightweight geometric information decoupled graph convolutional point cloud perception network model proposed in this application significantly reduces the computational cost of the point cloud perception network model through a 2D voxel downsampling algorithm, a multi-scale neighborhood construction strategy based on geometric structure, and an encoder-decoder architecture, while maintaining high-precision point cloud perception capabilities.

[0041] The following uses the Modelnet40 dataset, S3DIS dataset, and PartNet v0 dataset, which are representative datasets in the field of point cloud perception, to verify the effectiveness of the method in this application.

[0042] Specifically, the accuracy of the model in this application was verified based on the Modelnet40 dataset. Please refer to Table 1, which shows the performance comparison results of the method provided in this application embodiment on the classification task compared with the ResGCN (Residual Graph Convolutional Network) model [Li G, Muller M, Thabet A, et al. Deepgcns: Can gcns go as deep as cnns[C] / / Proceedings of the IEEE / CVF international conference on computer vision.2019: 9267-9276]. The experimental results show that compared with the ResGCN-16 network, the LGDGCN (Local and Global Decoupled Graph Convolutional Network) of this application, such as the cat16 or cat4 model, has a significant reduction in computational cost and its performance is more balanced across classes.

[0043] Table 1. Performance comparison results of the method provided in this application embodiment on classification tasks compared with other methods.

[0044] Specifically, the accuracy of the model in this application for each category was verified based on the S3DIS dataset. Please refer to Table 2, which shows the performance of the method provided in this application on indoor scene segmentation tasks, compared with existing methods Pointnet (a point set deep learning network for 3D classification and segmentation) model [CR Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 1(2):4, 2017. 2, 3, 5, 8] and Pointnet++ model [CRQi,L.Yi, H.Su, and L.J.Guibas.Pointnet++: Deep hierarchical feature learning on point sets in a metricspace. In Advances in Neural Information Processing Systems, pages 5099–5108,2017. 2, [8] and the performance comparison results of the ResGCN model. The experimental results show that the U_LGDGCN model of this application has a significant reduction in computational cost and better segmentation accuracy in 8 categories.

[0045] Table 2. Performance comparison results of the method provided in this application embodiment compared with other methods on indoor scene segmentation tasks.

[0046] Specifically, the accuracy of the model in this application for each category was verified based on the PartNet v0 dataset. Please refer to Table 3, which shows the performance comparison results of the method provided by the embodiments of this application on the part segmentation task compared with other methods. The experimental results show that the model of this application has a slightly improved overall accuracy while significantly reducing the amount of computation, and performs better in more than half of the specific categories.

[0047] Table 3. Performance comparison results of the method provided in this application embodiment compared with other methods in component segmentation tasks.

[0048] This application provides a lightweight geometric information decoupled graph convolutional point cloud perception network model, which uses a two-dimensional voxel downsampling algorithm to compress point cloud data in the XY plane to reduce computational overhead; it utilizes a scale factor-based expanded K-nearest neighbor algorithm to quickly construct a multi-scale neighborhood graph structure; and through the U-Net decoding architecture, it fuses multi-level features through skip connections and combines a progressive upsampling mechanism to recover spatial details, thereby achieving lightweight model while ensuring perception accuracy.

[0049] It is worth noting that the terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0050] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of this application and should not be construed as limiting the specific implementation of this application to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of this application, and all such modifications or substitutions should be considered within the scope of protection of this application.

Claims

1. A perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model, characterized in that, include: S100: Acquire raw point cloud data, corresponding task requirements, and a constructed lightweight geometric information decoupling graph convolutional point cloud perception model; wherein, the task requirements include classification tasks, segmentation tasks, and recognition tasks. S200: The original point cloud data spatial coordinates are discretized into voxel indices using 2D voxel downsampling, and the XY plane components are mapped into one-dimensional sequence indices to transform the floating-point distance comparison of spatial groupings into candidate region retrieval based on integer indices. Based on the candidate regions, the adjacent edge index is obtained using the scale factor-based extended KNN algorithm. The output features are obtained by graph convolution and downsampling based on the point cloud subsets and adjacent edge indexes to perform classification, segmentation or recognition tasks.

2. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 1, characterized in that, S200 includes: S210: Obtain a subset of point cloud data by 2D voxel downsampling of the original point cloud data, and obtain the neighbor index using the scale factor-based dilated KNN algorithm; perform multiple graph convolution and downsampling processes based on the neighbor index and the point cloud subset to obtain the first output feature of the multi-layer graph convolution. S220: When the task requirement is a classification task, the first output feature is deeply fused to obtain the first fused feature. Classification is performed based on the first fused feature to obtain the classification result. When the task requirement is a segmentation or recognition task, the first output feature is upsampled and graph convolution is performed multiple times to obtain the second output feature. Some output features are selected from the first output feature and the second output feature and deeply fused to obtain the second fused feature. Segmentation or recognition is then performed based on the second fused feature to obtain the execution result of the segmentation or recognition task.

3. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 2, characterized in that, The lightweight geometric information decoupled graph convolutional point cloud perception model includes a processing module, an encoder, a decoder, a fusion module selector, a fusion module, and a prediction layer. The processing module uses downsampling and dilated KNN algorithms to process the input. The encoder consists of four coded graph convolutional modules and three downsampling layers, with each coded graph convolutional module containing four coded graph convolutional layers. The decoder consists of three upsampling layers, three decoding convolutional layers, and three decoding graph convolutional modules, with each decoding graph convolutional module containing three decoding graph convolutional layers. The outputs of each coded graph convolutional module and decoding graph convolutional module are connected to the fusion module selector, the encoder output is connected to the decoder, and the upsampling layer is skipped to the corresponding downsampling layer. Both the coded graph convolutional layer and the decoding graph convolutional layer include a max pooling layer, a convolutional layer, a ReLU activation layer, and a normalization layer connected in sequence.

4. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 3, characterized in that, S210 includes: When the task requirement is a classification task, the processing module obtains a subset of the point cloud by downsampling the original point cloud data using 2D voxels, and obtains the neighbor index using the scale factor-based dilated KNN algorithm; the encoder performs multiple graph convolutions and downsampling processes based on the neighbor index and the point cloud subset to obtain the first output feature of the multi-layer graph convolution; the fusion module selector outputs the first output feature to the fusion module, and the fusion module deeply fuses all the first output features to obtain the first fused feature; the prediction layer performs classification based on the first fused feature to obtain the classification result of the classification task.

5. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 3, characterized in that, S220 includes: When the task requirement is segmentation or recognition, the processing module obtains a subset of the point cloud by downsampling the original point cloud data using 2D voxels, and obtains the neighbor index using the scale factor-based dilated KNN algorithm; the encoder performs multiple graph convolutions and downsampling processes based on the neighbor index and the point cloud subset to obtain the first output feature; the decoder performs multiple upsampling and graph convolution operations on the first output feature to obtain the second output feature; the fusion module selector selects some output features from the first and second output features and inputs them into the fusion module; the fusion module deeply fuses them to obtain the second fused feature, and the prediction layer performs segmentation or recognition based on the second fused feature to obtain the execution result of the segmentation or recognition task.

6. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 2, characterized in that, In S210 or S220, the original point cloud data is downsampled using 2D voxel downsampling to obtain multiple point cloud subsets that retain key geometric features, including: a1. Extract the three-dimensional spatial coordinates of the original point cloud data, construct a 2D voxel mesh system, and discretize the extracted three-dimensional spatial coordinates into voxel indices; b1, through linearization, maps the XY plane components of the voxel index to a one-dimensional index sequence, and performs spatial grouping; c1, calculate the mean coordinates of each grouping point in the original coordinate space as the centroid; d1, calculate the distance from each point in the group to the centroid, and select the point with the smallest distance as the representative point of the corresponding voxel grid; e1 gathers all representative points and, by comparing them with the preset target number of points, obtains multiple point cloud subsets with a fixed number of points through random sampling or random completion.

7. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 2, characterized in that, The methods described in S210 and S220 for obtaining neighbor indexes on the point cloud subset using the scale factor-based dilated KNN algorithm both include: a2, adopts a dual-blocking strategy to divide the original point cloud data and each point cloud subset into multiple query blocks and multiple reference blocks; b2, for the original point cloud data, calculate the first distance matrix between each query block and each reference block, and for each point cloud subset, calculate the second distance matrix between each query block and each reference block; c2, based on the first distance matrix and the second distance matrix, select points from the corresponding reference block whose distance is less than or equal to the radius threshold as candidate neighbor points for each point in each corresponding query block; d2, merge all candidate neighbor points, and then obtain multiple final neighbor points that meet the quantity requirements by selection or filling. e2, the multiple final neighbor points are mapped to the original point cloud data to form a first neighbor edge index, and the point cloud subset is mapped to the multiple final neighbor points to form a second neighbor edge index; f2, the first neighboring edge index and the second neighboring edge index are combined to form a neighboring edge index.

8. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 3, characterized in that, The process of performing multiple graph convolutions and downsampling on the original point cloud data and the point cloud subset as described in S210 and S220 to obtain the first output feature of the multi-layer graph convolution includes: a3, the first layer of the encoding graph convolutional layer takes each point as the center point, uses the neighbor edge index to extract the feature vector of the neighbor point, calculates the difference between the feature vector of the center point and the neighbor point, and then performs max pooling aggregation, convolution, activation and normalization to obtain the output feature of the first layer of the encoding graph convolution; b3, the remaining coding graph convolutional layers use the neighbor index to extract the neighbor point feature vectors from the output features of the upper coding graph convolution, perform max pooling aggregation, convolution, activation, and normalization processing, and then add them to the output features of the upper layer through residual connection to obtain the output features of this layer; c3, the downsampling layer downsamples the output features of the previous encoding map convolutional module and outputs them to the next encoding map convolutional module; d3 uses the convolutional output features of each layer of the encoding graph as the first output feature.

9. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 3, characterized in that, In S220, the second output feature obtained by performing multiple upsampling and graph convolution operations on the first output feature includes: a4, the upsampling layer upsamples the first output feature; b4, the decoding convolutional layer convolves the output features of the upsampling layer; c4, the decoding graph convolutional layer uses the neighbor index to extract the feature vector of neighbor points from the input features, and then processes it through max pooling aggregation, convolution, activation, and normalization. Finally, it is added to the input features through residual connection to obtain the output features of the decoding graph convolutional layer of this layer. The input of the first decoding graph convolutional layer is the output of the decoding convolutional layer, and the rest are the output features of the previous layer's decoding graph convolutional layer. d4 uses the convolutional output features of each layer of the decoded graph as the second output feature.

10. The perception method based on a lightweight geometric information decoupling graph convolutional point cloud perception model according to claim 3, characterized in that, In S220, a subset of output features are selected from the first and second output features, and then deep fusion is performed to obtain the second fused feature, which includes: The fusion module selector selects the first output feature of the last coded graph convolutional layer in each coded graph convolutional module and the second output feature of the last decoded graph convolutional layer in each decoded graph convolutional module as partial output features; the fusion module performs deep fusion on the selected partial output features to obtain the second fused feature.

Images