Indoor point cloud semantic segmentation method for enhancing geometric structure characteristics of object

Abstract

The invention discloses an indoor point cloud semantic segmentation method for enhancing geometric structure characteristics of an object, and belongs to the technical field of computer vision. Aiming at the problems that a point cloud semantic segmentation method does not fully consider local geometric structure features of a point cloud and is insufficient in sensitivity to local features, a point vector convolution module PVConv based on point vector feature decomposition is provided, a point vector convolution network PVCNN is formed by combining three layers of PVConv, the local features of the point cloud are extracted, and the point cloud semantic segmentation method based on the three layers of PVConv is obtained. Each layer of convolution in the point vector convolution network determines a local spherical neighborhood according to a given radius, and a center point and neighborhood features are extracted in the neighborhood; secondly, a window slice attention module learns point cloud long-distance features through a slice window attention mechanism WSA, self-attention calculation is carried out on three planes of a cubic window obtained through point cloud space division, the slice window attention mechanism expands the receptive field of attention while improving the network calculation efficiency, and the self-attention calculation efficiency is improved; and the extraction capability of local and global features of the network is balanced.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to an indoor point cloud semantic segmentation method for enhancing the geometric structure features of objects. Background Art

[0002] Indoor point cloud semantic segmentation is a hot topic in computer vision research and is crucial for understanding indoor scenes. Due to the irregular and disordered nature of point cloud data, and the high complexity, diverse object types, and large number of small objects in indoor scenes, manual point cloud segmentation is inefficient and inaccurate. Therefore, efficient indoor point cloud semantic segmentation algorithms can significantly improve segmentation performance, saving both human and material resources, and providing fundamental support for interior design, indoor control, and indoor safety. Therefore, research on indoor point cloud segmentation algorithms is of great significance.

[0003] In recent years, with the increasing maturity of depth sensing technology and 3D acquisition equipment, as well as the advancement of computer technology, researchers at home and abroad have proposed numerous deep learning-based point cloud semantic segmentation methods, primarily including multi-view, voxel, and point-based methods. Multi-view methods project point cloud data into two dimensions and process the data using a 2D convolutional network. In 2019, Milioto et al. proposed RangeNet++, which converts the input point cloud into a range image representation and performs a fully convolutional network segmentation on the range image. They also proposed a post-processing algorithm to address the discretization errors caused by the projection transformation and the ambiguity of the convolutional neural network output. View-based methods are prone to losing key information when processing large-scale point clouds and suffer from low computational efficiency and high resource consumption. Voxel-based methods utilize voxel grids to regularize disordered and unstructured point clouds. In 2018, Wang et al. proposed the Adaptive O-CNN, a convolutional neural network that voxelizes point cloud data using a patch-guided adaptive octree shape representation. The voxel-based method cannot fully retain the original spatial information of the point cloud when segmenting point cloud data, and the voxelization method has the problem of high time and hardware overhead. The point-based method is a method that directly processes the points in the point cloud data as input. In 2021, Zhao et al. proposed PointTransformer, introduced a self-attention mechanism in the neighborhood of the point, and used a multi-layer perceptron to encode the relative position of the center point and its neighboring points to learn features. In 2022, Su et al. proposed DLA-Net, which uses dual local attention blocks to learn the local feature representation of each point and its neighborhood, effectively capturing the geometric information of the local area, and combining multi-scale feature fusion to improve segmentation accuracy.

[0004] Existing research has achieved many results and progress, but there are still challenges that require further study and resolution. Current point cloud segmentation methods pay insufficient attention to local geometric structure information, resulting in errors in the classification and segmentation of small indoor objects. Therefore, this field still needs the introduction of more ideas to achieve comprehensive and in-depth local feature extraction from point cloud data. Summary of the Invention

[0005] In order to address the problems that point cloud semantic segmentation methods do not fully consider the local geometric structure features of point clouds and are insufficiently sensitive to local features, the present invention provides an indoor point cloud semantic segmentation method that enhances the geometric structure features of objects.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for indoor point cloud semantic segmentation for enhancing object geometric structure features, the method comprising the following steps:

[0008] Step 1: Organize and clean the point cloud data to reduce data noise and remove redundant points;

[0009] Step 2: Build an indoor point cloud semantic segmentation network model that enhances the geometric structure features of objects;

[0010] The network model adopts an encoder-decoder structure, which consists of a point embedding layer, an LGE-Block module, a downsampling module, and an upsampling module;

[0011] A single layer of PVConv is used as the point embedding layer of the network model to perform preliminary geometric feature extraction on the input point cloud data. The LGE-Block module is connected after the point embedding layer to ensure that the local features are fully transmitted to the subsequent modules.

[0012] The LGE-Block module consists of a layer normalization (LN), an LGE-Former module, and a feed-forward neural network (FNN). The LGE-Former module, consisting of a point vector convolutional neural network (PVCNN) module and a window slice attention module, performs feature extraction and attention calculation on point cloud data.

[0013] The specific operation of the point vector convolution module (PVCNN) is: points dimensional point cloud, with each point As the center, The local neighborhood is constructed with a radius of spherical shape. , the neighborhood contains all Neighboring points ; Extract the center point feature and neighborhood geometric structure feature of the neighborhood, and aggregate the center point feature and neighborhood feature;

[0014] The center point feature extraction uses a size of The weight matrix performs a linear transformation on the center point features, which is specifically defined as follows:

[0015]

[0016] Where, is the center point feature, is the weight matrix for extracting center point features, Indicates the level of the current PVConv in the point vector convolution module;

[0017] The neighborhood feature extraction is based on the center point Construct a three-dimensional coordinate system for the origin, and use 6 unit orthogonal bases to divide the three-dimensional space into 8 spatial quadrants in positive and negative directions, so that any position and direction vector Expressed using three orthogonal bases of the corresponding quadrants in the eight quadrants, the direction vector Project onto three corresponding orthogonal bases and calculate the direction vector The angle between the three directions and the orthogonal basis in each direction is calculated using the three direction weight matrices, and the features of the three directions are aggregated to obtain the direction vector Edge features , the specific definition is as follows:

[0018]

[0019] Where, It is used to represent the direction vector The set of three orthogonal bases, the characteristic of each direction uses the coefficient Perform aggregation, Represents the weights in three directions, Indicates the Point on layer characteristics; represents the direction vector; Indicates the center point; represents the neighborhood points;

[0020] The center point feature and neighborhood feature aggregation adopt the maximum pooling operation, using the distance function Opposite edge features Perform weighted aggregation, which is specifically defined as follows:

[0021]

[0022] Where, is the radius selected by the current PVConv, is the center point and neighboring points The distance between them; aggregate the overall features, and the specific definition is as follows:

[0023]

[0024] Where, Represents the neighborhood geometric structure characteristics, represents the distance function, Represents edge features; Indicates Centered Is the point set of radius; the points obtained The basic features and their neighborhood geometric structure features are input to the next layer PVConv in the point vector convolution module, or input to subsequent modules for further processing.

[0025] The specific operation of the LGE-Block module is as follows: the input features are first normalized through the normalization layer; then, the local geometric structure features of the data are extracted through the point vector convolution module in the LGE-Former module, and the window slice attention module learns the long-range features of the point cloud through the slice window attention mechanism WSA; then the output features and the original features are added through the residual connection to perform preliminary feature fusion; finally, nonlinear transformation is performed through the feedforward network to further extract high-order features.

[0026] The specific operation of the window slice attention module is: divide the PVConv output point cloud into The cube window is decomposed into three orthogonal two-dimensional planes, denoted as , generate non-overlapping two-dimensional windows on each plane, perform self-attention operation in each two-dimensional window; splice the attention calculation results along the feature channel dimension, and perform linear transformation on the splicing results to obtain the final attention output;

[0027] The attention calculation process is as follows: let the two-dimensional window size be , the point set contained in the window is calculated using express, Indicates the order of the current window, which contains points, and the number of attention heads is , the order of the heads is recorded as ; First, perform the query, key, and value calculations, as defined below:

[0028]

[0029] Where, Represents the query, key and value of the window, and uses a linear function to transform them linearly, mapping the original channel number to ; Perform matrix multiplication on the query matrix and the key transposed matrix to obtain the similarity matrix, which is denoted as matrix At the same time, the relative position encoding features shared by the three two-dimensional planes are combined with the point cloud spatial position information, and the similarity matrix is ​​quantified using the Softmax function. The attention weight of each point relative to the query point is obtained by normalizing the position encoding information. The specific definition is as follows:

[0030]

[0031] Where, is the feature dimension of the midpoint set in the window, is the number of attention heads, and are two learnable positional encodings, Indicates the Query on a window; Indicates the The key on the window; multiply the attention weight and its corresponding value to get the attention output of each head, which is specifically defined as follows:

[0032]

[0033] Where, represents the number of attention heads, represents the order of the current attention head, Indicates the corresponding value; Plane and Execution on the plane The same calculation is performed on the same plane; the calculation results of the three attentions are combined to obtain the output result, which is specifically defined as follows:

[0034]

[0035] Where, Plane The calculation results of the three two-dimensional window attention, Represents the concatenation operation of feature channels.

[0036] The specific operation of the downsampling module is: First, according to the point cloud coordinates Use the farthest sampling FPS to select the sampling point; then, use The nearest neighbor algorithm KNN queries the neighboring point set of the sampling point in the original point set, obtains the group index, and at the same time, the point cloud features Input to the normalization and linear combination layer to normalize the data; finally, according to the group index, use the maximum pooling to fuse the features of the projected data and output Input to the subsequent network layer, the number of point clouds after downsampling is reduced from Reduce to .

[0037] The specific operation of the upsampling module is: first, the decoder features After normalization and linear combination layer projection; then, at the current point cloud coordinates And the point cloud coordinates of the previous stage Between the two layers, the weighted average interpolation method of PointNet++ is used for interpolation to obtain the coordinate information; finally, the projected features, the interpolated coordinate results and the encoder features of the previous stage are added to obtain the features of the next layer decoder. .

[0038] Step 3: Input the point cloud data obtained in step 1 into the network model obtained in step 2 to perform point cloud semantic segmentation with enhanced local geometric structure features;

[0039] Taking point cloud data as input, LGE-Net is designed as the backbone network, and an encoder-decoder structure is adopted. The point vector convolution module is used as the point embedding layer of the backbone network to perform preliminary geometric feature extraction on the input point cloud data. In order to simplify the model structure and reduce the complexity of the model, and at the same time avoid the input data directly passing through the downsampling layer to cause local detail loss, LGE-Block is connected after the point embedding layer to ensure that the local features are fully passed to the subsequent modules. The point cloud data undergoes multi-layer downsampling and LGE-Block alternating processing for feature extraction. The decoder gradually restores the feature dimension, fuses point cloud features at different levels, and uses the multi-layer perceptron MLP to predict the semantic category of the point cloud data and output the segmentation result.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] The method of the present invention is suitable for point cloud segmentation in indoor scenes. By adding a point vector convolution module to the encoder to extract the local geometric structure features of the point cloud, adding a window slice attention module to expand the receptive field and extract the global features of the point cloud, while reducing the network calculation amount, the segmentation efficiency of the point cloud is improved and the overall segmentation accuracy of the network is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the indoor point cloud semantic segmentation method for enhancing the geometric structure features of objects according to the present invention;

[0043] Figure 2 is a schematic diagram of the LGE-Block module of the present invention;

[0044] Figure 3 Schematic diagram of the point vector convolution module of the present invention;

[0045] Figure 4 is a schematic diagram of the window slice attention module of the present invention;

[0046] Figure 5 is a schematic diagram of a downsampling module of the present invention;

[0047] Figure 6 is a schematic diagram of an upsampling module of the present invention;

[0048] Figure 7 This is the visualization result of indoor point cloud segmentation using different algorithms. DETAILED DESCRIPTION

[0049] To gain a deeper understanding of the present invention, we will provide a comprehensive and detailed description thereof. However, the present invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a comprehensive understanding of the present disclosure.

[0050] A method for indoor point cloud semantic segmentation that enhances object geometric structure features includes the following steps:

[0051] Step 1: Obtain indoor point cloud data and perform denoising, point cloud downsampling, and redundant data elimination operations;

[0052] Specifically, S3DIS and ScanNet200 are used as datasets for the indoor point cloud semantic segmentation task of enhancing the geometric structure features of objects in the present invention. The S3DIS dataset contains 271 independent rooms, covering various functional spaces such as offices, conference rooms, corridors, and toilets. The dataset contains 273 million scan points and 13 annotation categories. Each point contains 6 data points, X, Y, Z, R, G, and B, representing the spatial coordinates and color of the point; the ScanNet200 dataset contains a total of 1523 collected scene data, providing 200 categories of semantic segmentation and instance-level annotation, and supporting experimental testing in complex scenes. Among them, 1201 scenes are used for training and 312 scenes are used for testing.

[0053] Step 2: Construct an indoor point cloud semantic segmentation network model (LGE-Net) that enhances the geometric structure features of objects. Figure 1 As shown in Figure 2, the network mainly adopts an encoder-decoder structure, which is mainly composed of a point embedding layer, an LGE-Block module, a downsampling module, and an upsampling module.

[0054] 1) Point Embedding Layer

[0055] A single layer of PVConv is used as the point embedding layer of the network model to perform preliminary geometric feature extraction on the input point cloud data. To simplify the model structure and reduce complexity, while also preventing the loss of local details caused by the input data passing directly through the downsampling layer, an LGE-Block is added after the point embedding layer to ensure that local features are fully passed to subsequent modules.

[0056] 2) LGE-Block module

[0057] The LGE-Block module consists of a layer normalization (LN), an LGE-Former module, and a feed forward neural network (FNN). Figure 2 As shown in the figure, the input features are first normalized by a normalization layer. Subsequently, the point vector convolution module in the LGE-Former module extracts the local geometric structure features of the data. The window slice attention module learns the long-range features of the point cloud through the slice window attention mechanism (WSA). The output features are then added to the original features through a residual connection for preliminary feature fusion. Finally, a nonlinear transformation is performed through a feedforward network to further extract high-order features.

[0058] The point vector convolution module is as follows Figure 3 As shown, the input has points dimensional point cloud, with each point As the center, The local neighborhood is constructed with a radius and a spherical shape. , the neighborhood contains all Neighboring points ; Extract the center point features and neighborhood geometric structure features for the neighborhood, and aggregate the center point features and neighborhood features.

[0059] The center point feature extraction uses a size of The weight matrix performs a linear transformation on the center point features, which is specifically defined as follows:

[0060]

[0061] Where, is the center point feature, is the weight matrix for extracting center point features, Indicates the level of the current PVConv in the point vector convolution module;

[0062] The neighborhood geometric structure feature extraction is based on the center point Construct a three-dimensional coordinate system for the origin, and use 6 unit orthogonal bases to divide the three-dimensional space into 8 spatial quadrants in the positive and negative directions, so that the vector of any position and direction The direction vector can be expressed using the three orthogonal bases of the corresponding quadrants in the 8 quadrants. Project onto the three corresponding orthogonal bases and calculate The angle between the three directions and the basis in each direction is calculated using the three direction weight matrices, and the features of the three directions are aggregated to obtain the vector Edge features , the specific definition is as follows:

[0063]

[0064] Where, It is used to represent the direction vector The set of three orthogonal bases, the characteristic of each direction uses the coefficient Perform aggregation, Represents the weights in three directions, Indicates the Point on layer characteristics; represents the direction vector; Indicates the center point; represents the neighborhood points;

[0065] The center point feature and neighborhood feature aggregation adopt the maximum pooling operation, using the distance function Opposite edge features Perform weighted aggregation, which is specifically defined as follows:

[0066]

[0067] Where, is the radius selected by the current PVConv, is the center point and neighboring points The function has the characteristic of monotonically decreasing with distance. The closer the neighborhood points are to the center point, the greater the weight of their edge features in aggregation.

[0068] Aggregate the overall features, and the specific definition is as follows:

[0069]

[0070] Where, Represents the neighborhood geometric structure characteristics, represents the distance function, Represents edge features; Indicates Centered Is the point set of radius; the points obtained The basic features and their neighborhood geometric structure features are input to the next layer PVConv in the point vector convolution module, or input to subsequent modules for further processing.

[0071] The window slice attention module is as follows Figure 4 As shown, the PVConv output point cloud is divided into The cube window is decomposed into three orthogonal two-dimensional planes, denoted as , generate non-overlapping two-dimensional windows on each plane, perform self-attention operation in each two-dimensional window; splice the attention calculation results along the feature channel dimension, and perform linear transformation on the splicing results to obtain the final attention output;

[0072] Specifically, the attention calculation process is as follows: let the two-dimensional window size be , the point set contained in the window is calculated using express, Indicates the order of the current window, which contains points, and the number of attention heads is , the order of the heads is recorded as ; First, perform the query, key, and value calculations, as defined below:

[0073]

[0074] Where, Represents the query, key and value of the window, and uses a linear function to transform them linearly, mapping the original channel number to ; Perform matrix multiplication on the query matrix and the key transposed matrix to obtain the similarity matrix, which is denoted as matrix At the same time, the relative position encoding features shared by the three two-dimensional planes are combined with the point cloud spatial position information, and the similarity matrix is ​​quantified using the Softmax function. The attention weight of each point relative to the query point is obtained by normalizing the position encoding information. The specific definition is as follows:

[0075]

[0076] Where, is the feature dimension of the midpoint set in the window, is the number of attention heads, and are two learnable positional encodings, Indicates the Query on a window; Indicates the The key on the window; multiply the attention weight and its corresponding value to get the attention output of each head, which is specifically defined as follows:

[0077]

[0078] Where, represents the number of attention heads, represents the order of the current attention head, Indicates the corresponding value; Plane and Execution on the plane The same calculation is performed on the same plane; the calculation results of the three attentions are combined to obtain the output result, which is specifically defined as follows:

[0079]

[0080] Where, Plane The calculation results of the three two-dimensional window attention, Represents the concatenation operation of feature channels.

[0081] 3) Downsampling module

[0082] The downsampling module is as follows Figure 5 As shown, in the designated area First, according to the point cloud coordinates Use the farthest sampling FPS to select the sampling point; then, use The nearest neighbor algorithm KNN queries the neighboring point set of the sampling point in the original point set, obtains the group index, and at the same time, the point cloud features Input to the normalization and linear combination layer to normalize the data; finally, according to the group index, use the maximum pooling to fuse the features of the projected data and output Input to the subsequent network layer, the number of point clouds after downsampling is reduced from Reduce to .

[0083] 4) Upsampling module

[0084] The upsampling module is as follows Figure 6 As shown, first, the decoder features After normalization and linear combination layer projection; then, at the current point cloud coordinates And the point cloud coordinates of the previous stage Between the two layers, the weighted average interpolation method of PointNet++ is used for interpolation to obtain the coordinate information; finally, the projected features, the interpolated coordinate results and the encoder features of the previous stage are added to obtain the features of the next layer decoder. .

[0085] Step 3: Input the point cloud data obtained in step 1 into the network model obtained in step 2 to perform point cloud semantic segmentation with enhanced local geometric structure features;

[0086] Taking point cloud data as input, LGE-Net is designed as the backbone network, adopting an encoder-decoder structure. The point vector convolution module is used as the point embedding layer of the backbone network to perform preliminary geometric feature extraction on the input point cloud data. LGE-Block is connected after the point embedding layer to ensure that the local features are fully passed to subsequent modules. The point cloud data undergoes multi-layer downsampling and LGE-Block alternating processing for feature extraction. The decoder gradually restores the feature dimension, fuses point cloud features at different levels, and uses a multi-layer perceptron (MLP) to predict the semantic category of the point cloud data and output the segmentation result.

[0087] Specifically, the point cloud data was preprocessed. 6-fold cross-validation was used for the S3DIS dataset, with Area5 as an independent test set. The ScanNet200 dataset used 1201 scenes for training and 312 scenes for testing. Redundant points in the scenes were eliminated, and the original number of point clouds was downsampled to reduce the size of the point cloud. The Adam optimizer with an initial learning rate of 5e-4, weight decay of 0.02, a batch size of 8, and 500 training epochs was used. After every 15 training epochs, the learning rate was reduced by 50%. The results were obtained through multiple tests, as shown in Table 1. Figure 7 .

[0088] Table 1 Comparison of results of different methods on public datasets (unit: %)

[0089]

[0090] Table 1 lists the evaluation index comparison of the method of the present invention on some advanced methods of the public datasets S3DIS and ScanNet200. As can be seen from Table 1, the performance of the method has been improved year by year, and the LGE-Net method has achieved an average segmentation accuracy of 91.5%, an average segmentation accuracy of 78.6% and an average intersection-over-union ratio of 74.5% on the S3DIS dataset; and an average intersection-over-union ratio of 75.5% on the ScanNet200 dataset, which is in a leading position. From the comparison results, it can be seen that the method of the present invention can effectively improve the OA and mAcc indicators by further extracting local geometric structure features on the LGE-Net backbone network for indoor point cloud segmentation, indicating that local feature extraction for indoor point cloud segmentation is very important. The window slice attention module of the LGE-Net method fully extracts global features and plays a key role in segmentation.

[0091] Figure 6 The visualization results of our method on the S3DIS dataset are presented. Three representative scenes from the test set in region 5 are shown. The Stratified-Transformer model is used as a representative comparison model, and the LGE-Net visualization results are compared with it. The locations where our model performs well are circled with black dashed circles. The visualization results show that compared to the ground-truth scene segmentation results, the Stratified-Transformer performs poorly on small objects and columns, primarily on objects such as clutter, chairs, and sofas. In scene (a), the column and some clutter are mistakenly identified as a table, sofa, and wall. This problem persists in scene (b), where the chair handle is mistakenly identified as a tabletop, resulting in poor segmentation between the clutter and the table. In scene (c), three images of clutter are mistakenly identified as walls, and some points on the chair are mistakenly identified as clutter. LGE-Net achieves better segmentation results at similar or identical locations, with fewer misclassified points, a stronger segmentation integrity, and a closer-to-true boundary between the clutter and the table. The results show that the model proposed in this paper has excellent performance in local feature processing and small object edge recognition.

[0092] It should be noted that embodiments of the present invention may be provided as methods, systems, and / or computer program products. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0093] Various aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0094] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0095] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0096] It should be noted that those skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be based on the appended claims.

[0097] Any matters not described in detail in this specification are prior art known to those skilled in the art. Although the above description of the present invention is based on specific embodiments to facilitate understanding of the present invention by those skilled in the art, it should be understood that the present invention is not limited to the scope of the specific embodiments. As long as various modifications are within the spirit and scope of the present invention as defined and determined by the appended claims, such modifications will be obvious to those skilled in the art, and all inventions and creations utilizing the concepts of the present invention are protected.

Claims

1. A method for semantic segmentation of indoor point clouds by enhancing geometric structural features of objects, characterized by: The method comprises the following steps: Step 1: Organize and clean the point cloud data to reduce data noise and remove redundant points; Step 2: Build an indoor point cloud semantic segmentation network model that enhances the geometric structure features of objects; Step 3: Input the point cloud data obtained in step 1 into the network model obtained in step 2 to perform point cloud semantic segmentation with enhanced local geometric structure features.

2. The method for indoor point cloud semantic segmentation with enhanced object geometric structure features according to claim 1, characterized in that: The network model adopts an encoder-decoder structure, which consists of a point embedding layer, an LGE-Block module, a downsampling module, and an upsampling module; A single layer of PVConv is used as the point embedding layer of the network model to perform preliminary geometric feature extraction on the input point cloud data; The LGE-Block module is connected after the point embedding layer to ensure that the local features are fully transmitted to the subsequent modules; The LGE-Block module consists of layer normalization, LGE-Former module and feedforward neural network; The LGE-Former module is composed of a point vector convolution module composed of three layers of PVConv and a window slice attention module to perform feature extraction and attention calculation on point cloud data.

3. The method for indoor point cloud semantic segmentation with enhanced object geometric structure features according to claim 2, characterized in that: The specific operation of the point vector convolution module is: points dimensional point cloud, with each point As the center, The local neighborhood is constructed with a radius of spherical shape. , the neighborhood contains all Neighboring points ; Extract the center point feature and the neighborhood geometric structure feature for the neighborhood, and aggregate the center point feature and the neighborhood feature; The center point feature extraction uses a size of The weight matrix performs a linear transformation on the center point features, which is specifically defined as follows: , where is the center point feature, is the weight matrix for extracting center point features, Indicates the level of the current PVConv in the point vector convolution module; The neighborhood feature extraction is based on the center point Construct a three-dimensional coordinate system for the origin, and use 6 unit orthogonal bases to divide the three-dimensional space into 8 spatial quadrants in positive and negative directions, so that any position and direction vector Expressed using three orthogonal bases of the corresponding quadrants in the eight quadrants, the direction vector Project onto three corresponding orthogonal bases and calculate the direction vector The angle between the three directions and the orthogonal basis in each direction is calculated using the three direction weight matrices, and the features of the three directions are aggregated to obtain the direction vector Edge features , the specific definition is as follows: , where It is used to represent the direction vector The set of three orthogonal bases, the characteristic of each direction uses the coefficient Perform aggregation, Represents the weights in three directions, Indicates the Point on layer characteristics; represents the direction vector; Indicates the center point; represents the neighborhood points; The center point feature and neighborhood feature aggregation adopt the maximum pooling operation, using the distance function Opposite edge features Perform weighted aggregation, which is specifically defined as follows: , where is the radius selected by the current PVConv, is the center point and neighborhood points The distance between them; aggregate the overall features, and the specific definition is as follows: , where Represents the neighborhood geometric structure characteristics, represents the distance function, Represents edge features; Indicates Centered Is the point set of radius; the points obtained The basic features and their neighborhood geometric structure features are input to the next layer PVConv in the point vector convolution module, or input to subsequent modules for further processing.

4. The method for indoor point cloud semantic segmentation with enhanced object geometric structure features according to claim 3, characterized in that: The specific operation of the LGE-Block module is as follows: the input features are first normalized through the normalization layer; then, the local geometric structure features of the data are extracted through the point vector convolution module in the LGE-Former module, and the window slice attention module learns the long-range features of the point cloud through the slice window attention mechanism WSA; then the output features and the original features are added through the residual connection to perform preliminary feature fusion; finally, nonlinear transformation is performed through the feedforward network to further extract high-order features.

5. The method for indoor point cloud semantic segmentation with enhanced object geometric structure features according to claim 4, characterized in that: The specific operation of the window slice attention module is: divide the PVConv output point cloud into The cube window is decomposed into three orthogonal two-dimensional planes, denoted as , generate non-overlapping two-dimensional windows on each plane and perform self-attention operations in each two-dimensional window; Concatenate the attention calculation results along the feature channel dimension and perform a linear transformation on the concatenated results to obtain the final attention output; The attention calculation process is as follows: let the two-dimensional window size be , the point set contained in the window is calculated using express, Indicates the order of the current window, which contains points, and the number of attention heads is , the order of the heads is recorded as ; First, perform the query, key, and value calculations, as defined below: , where Represents the query, key and value of the window, and uses a linear function to transform them linearly, mapping the original channel number to ; Perform matrix multiplication on the query matrix and the key transpose matrix to obtain the similarity matrix, which is denoted as matrix At the same time, the relative position encoding features shared by the three two-dimensional planes are combined with the point cloud spatial position information, and the similarity matrix is ​​quantified using the Softmax function. The attention weight of each point relative to the query point is obtained by normalizing the position encoding information. The specific definition is as follows: , where is the feature dimension of the midpoint set in the window, is the number of attention heads, and are two learnable positional encodings, Indicates the Query on a window; Indicates the The key on the window; multiply the attention weight and its corresponding value to get the attention output of each head, which is specifically defined as follows: , where represents the number of attention heads, represents the order of the current attention head, Indicates the corresponding value; Plane and Execution on the plane The same calculation is performed on the same plane; the calculation results of the three attentions are combined to obtain the output result, which is specifically defined as follows: , where Plane The calculation results of the three two-dimensional window attention, Represents the concatenation operation of feature channels.

6. The method for indoor point cloud semantic segmentation with enhanced object geometric structure features according to claim 5, characterized in that: The specific operation of the downsampling module is: First, according to the point cloud coordinates Use the farthest sampling FPS to select the sampling point; then, use The nearest neighbor algorithm KNN queries the neighboring point set of the sampling point in the original point set, obtains the group index, and at the same time, the point cloud features Input to the normalization and linear combination layer to normalize the data; finally, according to the group index, use the maximum pooling to fuse the features of the projected data and output Input to the subsequent network layer, the number of point clouds after downsampling is reduced from Reduce to .

7. According to the method for indoor point cloud semantic segmentation with enhanced object geometric structure features according to claim 6, the specific operation of the upsampling module is as follows: first, the decoder feature After normalization and linear combination layer projection; then, at the current point cloud coordinates And the point cloud coordinates of the previous stage Between the two layers, the weighted average interpolation method of PointNet++ is used for interpolation to obtain the coordinate information; finally, the projected features, the interpolated coordinate results and the encoder features of the previous stage are added to obtain the features of the next layer decoder. .

8. The method for indoor point cloud semantic segmentation for enhancing object geometric structure features according to claim 7, characterized in that: The local geometric structure enhanced point cloud semantic segmentation in step 3 extracts local features of the point cloud by modeling local structural features, and performs detailed segmentation of the point cloud. The specific segmentation process includes: Taking point cloud data as input, LGE-Net is designed as the backbone network, adopting an encoder-decoder structure. The point vector convolution module is used as the point embedding layer of the backbone network to perform preliminary geometric feature extraction on the input point cloud data. LGE-Block is connected after the point embedding layer to ensure that the local features are fully passed to subsequent modules. The point cloud data undergoes multi-layer downsampling and LGE-Block alternating processing for feature extraction. The decoder gradually restores the feature dimension, fuses point cloud features at different levels, and uses a multi-layer perceptron (MLP) to predict the semantic category of the point cloud data and output the segmentation result.

Images