An indoor point cloud semantic segmentation method for enhancing geometric features of an object

Abstract

The application discloses an indoor point cloud semantic segmentation method for enhancing geometric structure features of objects and belongs to the technical field of computer vision. In view of the problem that the point cloud semantic segmentation method does not fully consider the local geometric structure features of the point cloud and is not sensitive enough to local features, a point vector convolution module PVConv based on point vector feature decomposition is proposed, three layers of PVConv are combined, a point vector convolution network PVCNN is formed, local features of the point cloud are extracted, and a local spherical neighborhood is determined according to a given radius in each convolution layer of the point vector convolution network. The center point and the neighborhood features are extracted in the neighborhood. Secondly, a window slice attention module learns long-distance features of the point cloud through a slice window attention mechanism WSA, performs self-attention calculation on three planes of a cubic window obtained by dividing the space of the point cloud, expands the receptive field of attention while improving the calculation efficiency of the network, and balances the extraction capabilities of local and global features of the network.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to an indoor point cloud semantic segmentation method that enhances the geometric structural features of objects. Background Technology

[0002] Indoor point cloud semantic segmentation is a research hotspot in computer vision and is crucial for understanding indoor scenes. Due to the irregularity and disorder of point cloud data, and the high complexity, diverse object types, and large number of small objects in indoor scenes, manual point cloud segmentation is inefficient and inaccurate. Therefore, efficient indoor point cloud semantic segmentation algorithms can significantly improve segmentation levels, save human and material resources, and play a fundamental role in ensuring indoor design, indoor control, and indoor safety. Thus, research on indoor point cloud segmentation algorithms is of great significance.

[0003] In recent years, with the increasing maturity of depth sensing technology and 3D acquisition equipment, as well as the development of computer technology, researchers at home and abroad have proposed numerous point cloud semantic segmentation methods based on deep learning, mainly including multi-view methods, voxel-based methods, and point-based methods. Multi-view methods project point cloud data into two dimensions and use a two-dimensional convolutional network to process the data. In 2019, Milioto et al. proposed RangeNet++, which converts the input point cloud into a distance image representation, performs segmentation on the distance image using a fully convolutional network, and proposes a post-processing algorithm to solve the discretization error caused by the projection transformation and the problem of blurred output results from the convolutional neural network. View-based methods are prone to losing key information when processing large-scale point clouds and suffer from low computational efficiency and high resource consumption. Voxel-based methods utilize voxel grids to regularize disordered and unstructured point clouds. In 2018, Wang et al. proposed Adaptive O-CNN, which uses a patch-guided adaptive octree shape representation method to voxelize point cloud data. Voxel-based methods, when segmenting point cloud data, cannot completely retain the original spatial information of the point cloud, and voxelization methods suffer from high time and hardware overhead. Point-based methods directly process the points in the point cloud data as input. In 2021, Zhao et al. proposed PointTransformer, which introduces a self-attention mechanism in the neighborhood of a point and uses a multilayer perceptron to encode the relative positions of the center point and its neighboring points to learn features. In 2022, Su et al. proposed DLA-Net, which uses dual local attention blocks to learn the local feature representation of each point and its neighborhood, effectively capturing the geometric information of local regions, and combining multi-scale feature fusion to improve segmentation accuracy.

[0004] Existing research has yielded many results and progress, but there are still problems that require further research and solutions: current point cloud segmentation methods do not pay enough attention to local geometric structure information, leading to errors in the classification and segmentation of small objects indoors. Therefore, this field still needs to introduce more ideas to achieve comprehensive and in-depth extraction of local features from point cloud data. Summary of the Invention

[0005] To address the problem that point cloud semantic segmentation methods do not fully consider the local geometric structure features of point clouds and lack sensitivity to local features, this invention provides an indoor point cloud semantic segmentation method that enhances the geometric structure features of objects.

[0006] To achieve the above objectives, the present invention employs the following technical solutions:

[0007] An indoor point cloud semantic segmentation method for enhancing the geometric structural features of objects, the method comprising the following steps:

[0008] Step 1: Clean and process the point cloud data to reduce noise and remove redundant points;

[0009] Step 2: Construct an indoor point cloud semantic segmentation network model to enhance the geometric features of objects;

[0010] The network model adopts an encoder-decoder structure, consisting of a point embedding layer, an LGE-Block module, a downsampling module, and an upsampling module.

[0011] A single-layer PVConv is used as the point embedding layer of the network model to perform preliminary geometric feature extraction on the input point cloud data; an LGE-Block module is connected after the point embedding layer to ensure that local features are completely transmitted to subsequent modules.

[0012] The LGE-Block module consists of Layer Normalization (LN), the LGE-Former module, and a Feed Forward Neural Network (FNN). The LGE-Former module consists of a Point Vector Convolutional (PVCNN) module and a window slicing attention module, which perform feature extraction and attention calculation on the point cloud data.

[0013] The specific operation of the point vector convolution module (PVCNN) is as follows: For the input having... points 3D point cloud, with each point Centered on, Construct a local neighborhood with radius and spherical shape. The neighborhood contains all that satisfy Neighboring points ; Extract center point features and neighborhood geometric structure features for the neighborhood, and aggregate the center point features and neighborhood features;

[0014] The center point feature extraction uses a size of The weight matrix performs a linear transformation on the features of the center point, as defined in the following equation:

[0015]

[0016] In the formula, It is a feature of the center point. It is a weight matrix for extracting features from the center point. This indicates the current layer of PVConv within the point vector convolution module;

[0017] The neighborhood feature extraction is based on the center point. A three-dimensional coordinate system is constructed with the origin as the reference point. Using six orthonormal bases, the three-dimensional space is divided into eight spatial quadrants with positive and negative directions, such that any position and direction vector... Representing the direction vector using three orthogonal bases corresponding to the eight quadrants. Project the vectors onto the three corresponding orthogonal bases and calculate the direction vectors. The angle between the orthogonal basis in each direction is calculated using three directional weight matrices for each direction. The features from the three directions are then aggregated to obtain the direction vector. edge features The specific definition is shown in the following formula:

[0018]

[0019] In the formula, It is used to represent direction vectors The set of three orthogonal bases, the characteristic of each direction using coefficients Perform aggregation. Indicates the weights in the three directions. Indicates the first adjacent points on the layer Features; Represents the direction vector; Indicates the center point; Represents neighboring points;

[0020] The aggregation of the center point features and neighborhood features employs max pooling and uses a distance function. opposite edge features Weighted aggregation is performed, as defined in the following formula:

[0021]

[0022] In the formula, It is the size of the radius selected by PVConv. It is the center point and neighboring points The distance between them; the aggregation of overall features is specifically defined as follows:

[0023]

[0024] In the formula, Represents the geometric structure features of the neighborhood. Represents the distance function. Represent edge features; Indicates Center The set of points with radius; the obtained points The basic features and their neighborhood geometric features are input into the next layer PVConv in the point vector convolution module, or into subsequent modules for further processing.

[0025] The LGE-Block module operates as follows: the input features are first normalized through a normalization layer; then, the point vector convolution module in the LGE-Former module extracts the local geometric structure features of the data, and the window slicing attention module learns the long-range features of the point cloud through the slicing window attention mechanism; then, the output features and the original features are added through residual connections to perform preliminary feature fusion; finally, a feedforward network is used for nonlinear transformation to further extract higher-order features.

[0026] The specific operation of the window slicing attention module is as follows: The PVConv output point cloud is divided into segments of size [size missing]. Within a cube window; the cube window is decomposed into three orthogonal two-dimensional planes, denoted as... Non-overlapping two-dimensional windows are generated in each plane, and self-attention operation is performed in each two-dimensional window; the attention calculation results are spliced ​​along the feature channel dimension, and the spliced ​​results are linearly transformed to obtain the final attention output;

[0027] The attention calculation process is as follows: Let the size of the two-dimensional window be... The point set contained in the window uses express, Indicates the current window order, which contains... There are [number] points, and the number of attention heads is [number]. The order of the heads is recorded as First, perform the query, key, and value calculations, as specifically defined in the following formula:

[0028]

[0029] In the formula, The query, key, and value of the window are represented by a linear function, which performs a linear transformation on them, mapping the original number of channels to... Multiply the query matrix and the transpose of the key matrix to obtain a similarity matrix, denoted as matrix A. Simultaneously, by combining the relative position encoding features shared by the three two-dimensional planes and incorporating point cloud spatial position information, the Softmax function is used to optimize the similarity matrix. The attention weight of each point relative to the query point is calculated by normalizing the location encoding information, as shown in the following formula:

[0030]

[0031] In the formula, The feature dimension of the point set in the window. For the number of attention heads, and These are two learnable positional codes. Indicates the first Query on a single window; Indicates the first The attention is calculated by multiplying the attention weights by their corresponding values ​​to obtain the attention output for each head, as shown in the following formula:

[0032]

[0033] In the formula, Indicates the number of attention heads. Indicates the current order of attention. Indicates the corresponding value; in plane and Execution on the plane and The calculations are performed on the same plane; the results of the three attention calculations are combined to obtain the output result, as specifically defined in the following formula:

[0034]

[0035] In the formula, They are planes The calculation results of attention for three two-dimensional windows. This indicates the splicing operation of feature channels.

[0036] The specific operation of the downsampling module is as follows: in a specified area First, based on the point cloud coordinates... Select sampling points using the furthest point sampling FPS; then, use... The KNN algorithm queries the set of neighboring points of the sampled points in the original point set to obtain a grouping index, and simultaneously incorporates point cloud features. The data is input to a normalization and linear combination layer for normalization; finally, based on the grouped index, max pooling is used to fuse the features of the projected data, and the output is... The input is fed into subsequent network layers, and after downsampling, the number of point clouds is reduced from... Reduce to .

[0037] The specific operation of the upsampling module is as follows: First, decoder features... Projection is performed after normalization and linear combination layers; then, at the current point cloud coordinates... Point cloud coordinates from the previous stage In between, PointNet++'s weighted average interpolation method is used to perform interpolation operations to obtain coordinate information; finally, the projected features, the interpolated coordinates, and the features from the previous encoder stage are added to obtain the features for the next layer decoder. .

[0038] Step 3: Input the point cloud data obtained in Step 1 into the network model obtained in Step 2 to perform point cloud semantic segmentation with local geometric structure feature enhancement;

[0039] Using point cloud data as input, an LGE-Net backbone network is designed with an encoder-decoder structure. Point vector convolution modules serve as point embedding layers in the backbone network to perform preliminary geometric feature extraction on the input point cloud data. To simplify the model structure and reduce model complexity, and to avoid the loss of local details caused by the input data directly passing through the downsampling layer, an LGE-Block is added after the point embedding layer to ensure that local features are completely transmitted to subsequent modules. The point cloud data undergoes feature extraction through alternating processing of multiple downsampling layers and LGE-Block. The decoder gradually restores the feature dimensions, fuses point cloud features at different levels, and uses a multilayer perceptron (MLP) to predict the semantic category of the point cloud data, outputting the segmentation result.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] The method of this invention is applicable to point cloud segmentation in indoor scenes. By adding a point vector convolution module to the encoder to extract local geometric features of the point cloud, and adding a window slicing attention module to expand the receptive field and extract global features of the point cloud, the method reduces the computational load of the network, thereby improving the segmentation efficiency of point cloud segmentation and the overall segmentation accuracy of the network. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the indoor point cloud semantic segmentation method for enhancing the geometric structural features of objects according to the present invention;

[0043] Figure 2 This is a schematic diagram of the LGE-Block module of the present invention;

[0044] Figure 3 This is a schematic diagram of the point vector convolution module of the present invention;

[0045] Figure 4 This is a schematic diagram of the window slicing attention module of the present invention;

[0046] Figure 5 This is a schematic diagram of the downsampling module of the present invention;

[0047] Figure 6 This is a schematic diagram of the upsampling module of the present invention;

[0048] Figure 7 These are visualization results of indoor point cloud segmentation using different algorithms. Detailed Implementation

[0049] To gain a deeper understanding of this invention, we will provide a comprehensive and detailed description. However, this invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a full understanding of the disclosure of this invention.

[0050] An indoor point cloud semantic segmentation method for enhancing the geometric structural features of objects includes the following steps:

[0051] Step 1: Acquire indoor point cloud data and perform noise reduction, point cloud downsampling, and redundant data elimination operations on it;

[0052] Specifically, S3DIS and ScanNet200 were used as the datasets for the indoor point cloud semantic segmentation task of enhancing object geometric features in this invention. The S3DIS dataset contains 271 individual rooms, covering diverse functional spaces such as offices, meeting rooms, corridors, and restrooms. The dataset contains 273 million scan points and 13 labeled categories. Each point contains six data points (X, Y, Z, R, G, B) representing the point's spatial coordinates and color. The ScanNet200 dataset contains 1523 collected scene data points, providing 200 classes of semantic segmentation and instance-level annotations, supporting experimental testing in complex scenarios. Of these, 1201 scenes were used for training, and 312 scenes were used for testing.

[0053] Step 2: Construct an indoor point cloud semantic segmentation network model (LGE-Net) to enhance the geometric features of objects. The LGE-Net is as follows: Figure 1 As shown, the network mainly adopts an encoder-decoder structure, consisting primarily of a point embedding layer, an LGE-Block module, a downsampling module, and an upsampling module.

[0054] 1) Point embedding layer

[0055] A single-layer PVConv is used as the point embedding layer in the network model to perform preliminary geometric feature extraction on the input point cloud data. To simplify the model structure and reduce model complexity, and to avoid the loss of local details caused by the input data directly passing through the downsampling layer, an LGE-Block is connected after the point embedding layer to ensure that local features are completely transmitted to subsequent modules.

[0056] 2) LGE-Block module

[0057] The LGE-Block module consists of Layer Normalization (LN), an LGE-Former module, and a Feed Forward Neural Network (FNN). The LGE-Block module is as follows: Figure 2 As shown, the input features are first normalized through a normalization layer; then, the point vector convolution module in the LGE-Former module extracts the local geometric structure features of the data, and the window slicing attention module learns the long-range features of the point cloud through the slicing window attention mechanism; then, the output features and the original features are added through residual connections to perform preliminary feature fusion; finally, a feedforward network is used for nonlinear transformation to further extract higher-order features.

[0058] The point vector convolution module is as follows: Figure 3 As shown, for the input having points 3D point cloud, with each point Centered on, Construct a local neighborhood with radius and spherical shape. The neighborhood contains all that satisfy Neighboring points The center point feature and neighborhood geometric structure feature are extracted for the neighborhood, and the center point feature and neighborhood feature are aggregated.

[0059] The center point feature extraction uses a size of The weight matrix performs a linear transformation on the features of the center point, as defined in the following equation:

[0060]

[0061] In the formula, It is a feature of the center point. It is a weight matrix for extracting features from the center point. This indicates the current layer of PVConv within the point vector convolution module;

[0062] The neighborhood feature extraction is based on the center point. Construct a three-dimensional coordinate system with the origin, and use six orthonormal bases to divide the three-dimensional space into eight spatial quadrants with positive and negative directions. This allows vectors of any position and direction to... All can be represented using three orthogonal bases corresponding to the eight quadrants. The direction vectors... Project onto three corresponding orthogonal bases and calculate The angle between the vector and the basis vector in each direction is calculated using three directional weight matrices. The features from the three directions are then aggregated to obtain a vector. edge features The specific definition is shown in the following formula:

[0063]

[0064] In the formula, It is used to represent direction vectors The set of three orthogonal bases, the characteristic of each direction using coefficients Perform aggregation. Indicates the weights in the three directions. Indicates the first adjacent points on the layer Features; Represents the direction vector; Indicates the center point; Represents neighboring points;

[0065] The aggregation of the center point features and neighborhood features employs max pooling and uses a distance function. opposite edge features Weighted aggregation is performed, as defined in the following formula:

[0066]

[0067] In the formula, It is the size of the radius selected by PVConv. It is the center point and neighboring points The distance between them; the function has the characteristic of monotonically decreasing with distance, and the closer the neighboring points are to the center point, the greater the weight of their edge features during aggregation.

[0068] The overall features are aggregated, and the specific definition is shown in the following formula:

[0069]

[0070] In the formula, Represents the geometric structure features of the neighborhood. Represents the distance function. Represent edge features; Indicates Center The set of points with radius; the obtained points The basic features and their neighborhood geometric features are input into the next layer PVConv in the point vector convolution module, or into subsequent modules for further processing.

[0071] The window slicing attention module, such as Figure 4 As shown, the PVConv output point cloud is divided into areas of size [size missing]. Within a cube window; the cube window is decomposed into three orthogonal two-dimensional planes, denoted as... Non-overlapping two-dimensional windows are generated in each plane, and self-attention operation is performed in each two-dimensional window; the attention calculation results are spliced ​​along the feature channel dimension, and the spliced ​​results are linearly transformed to obtain the final attention output;

[0072] Specifically, the attention calculation process is as follows: Let the size of the two-dimensional window be... The point set contained in the window uses express, Indicates the current window order, which contains... There are [number] points, and the number of attention heads is [number]. The order of the heads is recorded as First, perform the query, key, and value calculations, as specifically defined in the following formula:

[0073]

[0074] In the formula, The query, key, and value of the window are represented by a linear function, which performs a linear transformation on them, mapping the original number of channels to... Multiply the query matrix and the transpose of the key matrix to obtain a similarity matrix, denoted as matrix A. Simultaneously, by combining the relative position encoding features shared by the three two-dimensional planes and incorporating point cloud spatial position information, the Softmax function is used to optimize the similarity matrix. The attention weight of each point relative to the query point is calculated by normalizing the location encoding information, as shown in the following formula:

[0075]

[0076] In the formula, The feature dimension of the point set in the window. For the number of attention heads, and These are two learnable positional codes. Indicates the first Query on a single window; Indicates the first The attention is calculated by multiplying the attention weights by their corresponding values ​​to obtain the attention output for each head, as shown in the following formula:

[0077]

[0078] In the formula, Indicates the number of attention heads. Indicates the current order of attention. Indicates the corresponding value; in plane and Execution on the plane and The calculations are performed on the same plane; the results of the three attention calculations are combined to obtain the output result, as specifically defined in the following formula:

[0079]

[0080] In the formula, They are planes The calculation results of attention for three two-dimensional windows. This indicates the splicing operation of feature channels.

[0081] 3) Downsampling module

[0082] The downsampling module is as follows Figure 5 As shown, in the specified area First, based on the point cloud coordinates... Select sampling points using the furthest point sampling FPS; then, use... The KNN algorithm queries the set of neighboring points of the sampled points in the original point set to obtain a grouping index, and simultaneously incorporates point cloud features. The data is input to a normalization and linear combination layer for normalization; finally, based on the grouped index, max pooling is used to fuse the features of the projected data, and the output is... The input is fed into subsequent network layers, and after downsampling, the number of point clouds is reduced from... Reduce to .

[0083] 4) Upsampling module

[0084] The upsampling module is as follows Figure 6 As shown, firstly, decoder features Projection is performed after normalization and linear combination layers; then, at the current point cloud coordinates... Point cloud coordinates from the previous stage In between, PointNet++'s weighted average interpolation method is used to perform interpolation operations to obtain coordinate information; finally, the projected features, the interpolated coordinates, and the features from the previous encoder stage are added to obtain the features for the next layer decoder. .

[0085] Step 3: Input the point cloud data obtained in Step 1 into the network model obtained in Step 2 to perform point cloud semantic segmentation with local geometric structure feature enhancement;

[0086] Using point cloud data as input, an LGE-Net backbone network is designed, employing an encoder-decoder structure. A point vector convolution module serves as the point embedding layer of the backbone network to perform preliminary geometric feature extraction on the input point cloud data. An LGE-Block is then connected after the point embedding layer to ensure that local features are completely transmitted to subsequent modules. The point cloud data undergoes feature extraction through alternating multi-layer downsampling and LGE-Block processing. The decoder gradually restores the feature dimensions, fuses point cloud features from different levels, and uses a multilayer perceptron (MLP) to predict the semantic category of the point cloud data, outputting the segmentation result.

[0087] Specifically, the point cloud data was preprocessed. The S3DIS dataset used 6-fold cross-validation, with Area5 as the independent test set. The ScanNet200 dataset used 1201 scenes for training and 312 scenes for testing. Redundant points in the scenes were eliminated, and the original point cloud size was downsampled to reduce the number of points. An Adam optimizer with an initial learning rate of 5e-4, a weight decay of 0.02, a batch size of 8, and 500 training epochs were used. The learning rate was reduced by 50% every 15 epochs. Results were obtained through multiple tests; see Table 1 for details. Figure 7 .

[0088] Table 1. Comparison of results for different methods on public datasets (unit: %)

[0089]

[0090] Table 1 lists the evaluation metrics of the method of this invention on some state-of-the-art methods on the public datasets S3DIS and ScanNet200. As shown in Table 1, the performance of the method has improved year by year. The LGE-Net method achieves an average segmentation accuracy of 91.5%, an average segmentation precision of 78.6%, and an average intersection-over-union (IoU) of 74.5% on the S3DIS dataset; and an IoU of 75.5% on the ScanNet200 dataset, placing it among the top performers. The comparison results show that the method of this invention, by further extracting local geometric features on the LGE-Net backbone for indoor point cloud segmentation, can effectively improve the OA and mAcc metrics, indicating that local feature extraction is crucial for indoor point cloud segmentation. The window slicing attention module of the LGE-Net method fully extracts global features and plays a key role in segmentation.

[0091] Figure 6 The visualization results of the proposed method on the S3DIS dataset are presented. Three representative scenes from the test set in region 5 are shown. The Stratified-Transformer model is selected as the representative comparison model, and its visualization results are compared with LGE-Net. The areas where the proposed model has an advantage in segmentation performance are highlighted with black dashed circles. The visualization results show that compared to the scene segmentation results with the real labels, the Stratified-Transformer performs poorly in segmenting small objects and columnar objects, mainly focusing on clutter, chairs, and sofas. In scene (a), columns and some clutter are incorrectly identified as tables, sofas, and walls. This problem persists in scene (b), where chair handles are identified as tabletops, and the segmentation of the boundaries between clutter and tables is poor. In scene (c), three portrait-like clutter objects are identified as walls, and some points on the chair are identified as clutter. LGE-Net achieves better segmentation results in similar or identical locations, with fewer segmentation errors, stronger overall segmentation, and the segmentation boundaries between clutter and tables are closer to the true shapes. The results show that the proposed model performs well in local feature processing and small object edge recognition.

[0092] It should be noted that embodiments of the present invention can be provided as methods, systems, and / or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0094] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0095] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0096] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

[0097] Contents not described in detail in this specification are prior art known to those skilled in the art. Although illustrative specific embodiments of the invention have been described above to facilitate understanding by those skilled in the art, it should be understood that the invention is not limited to the scope of the specific embodiments. Various modifications are readily apparent to those skilled in the art as long as they fall within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of this invention are protected.

Claims

1. An indoor point cloud semantic segmentation method for enhancing the geometric structural features of objects, characterized in that: The method includes the following steps: Step 1: Clean and process the point cloud data to reduce noise and remove redundant points; Step 2: Construct an indoor point cloud semantic segmentation network model to enhance the geometric features of objects; Step 3: Input the point cloud data obtained in Step 1 into the network model obtained in Step 2 to perform point cloud semantic segmentation with local geometric structure feature enhancement; The network model adopts an encoder-decoder structure, consisting of a point embedding layer, an LGE-Block module, a downsampling module, and an upsampling module. A single-layer PVConv is used as the point embedding layer of the network model to perform preliminary geometric feature extraction on the input point cloud data; an LGE-Block module is connected after the point embedding layer to ensure that local features are completely transmitted to subsequent modules. The LGE-Block module consists of layer normalization, an LGE-Former module, and a feedforward neural network. The LGE-Former module consists of a point vector convolution module composed of three layers of PVConv and a window slicing attention module to perform feature extraction and attention calculation on point cloud data. The specific operation of the point vector convolution module is as follows: For the input having... points 3D point cloud, with each point Centered on, Construct a local neighborhood with radius and spherical shape. The neighborhood contains all that satisfy Neighboring points ; Extract center point features and neighborhood geometric structure features for the neighborhood, and aggregate the center point features and neighborhood features; The center point feature extraction uses a size of The weight matrix performs a linear transformation on the features of the center point, as defined in the following equation: In the formula, It is a feature of the center point. It is a weight matrix for extracting features from the center point. This indicates the current layer of PVConv within the point vector convolution module; The neighborhood feature extraction is based on the center point. A three-dimensional coordinate system is constructed with the origin as the reference point. Using six orthonormal bases, the three-dimensional space is divided into eight spatial quadrants with positive and negative directions, such that any position and direction vector... Representing the direction vector using three orthogonal bases corresponding to the eight quadrants. Project the vectors onto the three corresponding orthogonal bases and calculate the direction vectors. The angle between the orthogonal basis in each direction is calculated using three directional weight matrices for each direction. The features from the three directions are then aggregated to obtain the direction vector. edge features The specific definition is shown in the following formula: In the formula, It is used to represent direction vectors The set of three orthogonal bases, the characteristic of each direction using coefficients Perform aggregation. Indicates the weights in the three directions. Indicates the first adjacent points on the layer Features; Represents the direction vector; Indicates the center point; Represents neighboring points; The aggregation of the center point features and neighborhood features employs max pooling and uses a distance function. opposite edge features Weighted aggregation is performed, as defined in the following formula: In the formula, It is the size of the radius selected by PVConv. It is the center point and neighboring points The distance between them; the aggregation of overall features is specifically defined as follows: In the formula, Represents the geometric structure features of the neighborhood. Represents the distance function. Represent edge features; Indicates Center The set of points with radius; the obtained points The basic features and their neighborhood geometric features are input into the next layer PVConv in the point vector convolution module, or into subsequent modules for further processing.

2. The indoor point cloud semantic segmentation method for enhancing the geometric structural features of objects according to claim 1, characterized in that: The LGE-Block module operates as follows: the input features are first normalized through a normalization layer; then, the point vector convolution module in the LGE-Former module extracts the local geometric structure features of the data, and the window slicing attention module learns the long-range features of the point cloud through the slicing window attention mechanism; then, the output features and the original features are added through residual connections to perform preliminary feature fusion; finally, a feedforward network is used for nonlinear transformation to further extract higher-order features.

3. The indoor point cloud semantic segmentation method for enhancing the geometric structural features of objects according to claim 2, characterized in that: The specific operation of the window slicing attention module is as follows: The PVConv output point cloud is divided into segments of size [size missing]. Within a cube window; the cube window is decomposed into three orthogonal two-dimensional planes, denoted as... Generate non-overlapping two-dimensional windows in each plane, and perform self-attention operations within each two-dimensional window; The attention calculation results are concatenated along the feature channel dimension, and a linear transformation is performed on the concatenated results to obtain the final attention output. The attention calculation process is as follows: Let the size of the two-dimensional window be... The point set contained in the window uses express, Indicates the current window order, which contains... There are [number] points, and the number of attention heads is [number]. The order of the heads is recorded as First, perform the query, key, and value calculations, as specifically defined in the following formula: In the formula, The query, key, and value of the window are represented by a linear function, which performs a linear transformation on them, mapping the original number of channels to... Multiply the query matrix and the transpose of the key matrix to obtain a similarity matrix, denoted as matrix A. Simultaneously, by combining the relative position encoding features shared by the three two-dimensional planes and incorporating point cloud spatial position information, the Softmax function is used to optimize the similarity matrix. The attention weight of each point relative to the query point is calculated by normalizing the location encoding information, as shown in the following formula: In the formula, The feature dimension of the point set in the window. For the number of attention heads, and These are two learnable positional codes. Indicates the first Query on a single window; Indicates the first The attention is calculated by multiplying the attention weights by their corresponding values ​​to obtain the attention output for each head, as shown in the following formula: In the formula, Indicates the number of attention heads. Indicates the current order of attention. Indicates the corresponding value; in plane and Execution on the plane and The calculations are performed on the same plane; the results of the three attention calculations are combined to obtain the output result, as specifically defined in the following formula: In the formula, planes The calculation results of attention for three two-dimensional windows. This indicates the splicing operation of feature channels.

4. The indoor point cloud semantic segmentation method for enhancing the geometric structural features of objects according to claim 3, characterized in that: The specific operation of the downsampling module is as follows: in a specified area First, based on the point cloud coordinates... Select sampling points using the furthest point sampling FPS; then, use... The KNN algorithm queries the set of neighboring points of the sampled points in the original point set to obtain a grouping index, and simultaneously incorporates point cloud features. The data is input to a normalization and linear combination layer for normalization; finally, based on the grouped index, max pooling is used to fuse the features of the projected data, and the output is... The input is fed into subsequent network layers, and after downsampling, the number of point clouds is reduced from... Reduce to .

5. The indoor point cloud semantic segmentation method for enhancing object geometric structure features according to claim 4, wherein the specific operation of the upsampling module is as follows: First, decoder features... Projection is performed after normalization and linear combination layers; then, at the current point cloud coordinates... Point cloud coordinates from the previous stage In between, PointNet++'s weighted average interpolation method is used to perform interpolation operations to obtain coordinate information; finally, the projected features, the interpolated coordinates, and the features from the previous encoder stage are added to obtain the features for the next layer decoder. .

6. The indoor point cloud semantic segmentation method for enhancing the geometric structural features of objects according to claim 5, characterized in that: The point cloud semantic segmentation with enhanced local geometry in step 3 involves modeling local structural features, extracting local features of the point cloud, and performing detailed segmentation of the point cloud. The specific segmentation process includes: Using point cloud data as input, an LGE-Net backbone network is designed, employing an encoder-decoder structure. A point vector convolution module serves as the point embedding layer of the backbone network to perform preliminary geometric feature extraction on the input point cloud data. An LGE-Block is then connected after the point embedding layer to ensure that local features are completely transmitted to subsequent modules. The point cloud data undergoes feature extraction through alternating multi-layer downsampling and LGE-Block processing. The decoder gradually restores the feature dimensions, fuses point cloud features from different levels, and uses a multilayer perceptron (MLP) to predict the semantic category of the point cloud data, outputting the segmentation result.

Images