Indoor point cloud semantic segmentation method based on self-adaption and multi-scale feature fusion

Abstract

The invention discloses an indoor point cloud semantic segmentation method based on self-adaption and multi-scale feature fusion. The indoor point cloud semantic segmentation method comprises the steps of indoor point cloud preprocessing, construction of an indoor point cloud semantic segmentation network, training of the indoor point cloud semantic segmentation network and testing of the indoor point cloud semantic segmentation network. A self-adaptive local neighborhood search module is adopted to construct a graph structure of indoor point cloud, and the problem that local details are lost due to the fixed neighborhood radius in an existing method is solved; a context perception attention convolution module and a graph space attention convolution module are adopted to obtain key context semantic features and complex space geometric features between the points respectively; indoor point cloud spatial pyramid pooling is introduced into a multi-scale feature fusion module based on double attention, multi-level features obtained by different receptive fields are combined, and the ability of the whole network to extract and express features of different scales is enhanced. Comparison experiments show that the method can accurately identify semantic categories of a few samples and easily-confused building structures in a complex indoor environment.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular relates to a semantic segmentation method for indoor point clouds. Background Art

[0002] Indoor 3D point cloud data is a digital representation of indoor space obtained by high-precision scanning technology (such as laser radar), and each point has attributes such as spatial coordinates, color, and reflectivity. In the field of computer vision, complex semantic segmentation of indoor point clouds is to classify each point in the point cloud into its corresponding semantic category, such as wall, floor, furniture, etc.

[0003] At present, the main point cloud semantic segmentation methods can be roughly divided into three categories: point processing-based, Transformer-based, and graph convolutional neural network-based methods. Point processing-based methods can operate directly on point clouds without building complex data structures. However, it is difficult to capture long-distance spatial geometric dependencies in point clouds, thereby ignoring global context information. The emerging Transformer-based method uses the self-attention mechanism to capture the dependencies between long-distance point clouds and effectively process three-dimensional point clouds with complex structures. However, this method has high computational costs, is not easy to process large-scale scene point clouds, and lacks sufficient understanding of the local structure of point clouds. The graph convolutional neural network-based method constructs the point cloud as a graph structure to better capture local neighborhood information and global spatial structure features. This method can more comprehensively process point cloud data with complex geometric and topological structures, and has therefore become the first choice for complex semantic segmentation methods for indoor point clouds.

[0004] Existing segmentation methods based on graph convolution can fully consider the topological relationship between points, but because they usually use a fixed neighborhood radius method to construct the graph structure, the captured local features will lose detailed information. The point cloud density of indoor spaces also varies greatly due to spatial layout and object distribution. Therefore, existing methods may lose some contextual information when processing high-density areas, and may not be able to fully capture sufficient local features when processing low-density areas. In addition, the effective fusion of features of different scales is also crucial to improving the semantic segmentation accuracy of point clouds in complex and diverse indoor environments. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide an indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion with strong adaptability and good segmentation performance.

[0006] The technical solution steps adopted to solve the above technical problems are as follows:

[0007] (1) Indoor point cloud preprocessing

[0008] Select the indoor point cloud dataset and perform preprocessing: divide the 272 rooms in the S3DIS point cloud dataset into sampling blocks containing 4096 points with an area of ​​1m×1m and a spacing of 0.5m; and divide them into training set and test set with a ratio of 5:1.

[0009] (2) Constructing an indoor point cloud semantic segmentation network

[0010] The indoor point cloud semantic segmentation network is composed of an adaptive local neighborhood search module, a multi-scale feature fusion module, and a shared multi-layer perceptron MLP connected in series.

[0011] The multi-scale feature fusion module is composed of a feature extraction branch 1, a feature extraction branch 2, a feature extraction branch 3, and an indoor point cloud spatial pyramid pooling layer connected in series.

[0012] (3) Training indoor point cloud semantic segmentation network

[0013] 1) Constructing weighted cross entropy loss function

[0014] The weighted cross entropy loss function is constructed according to formula (1):

[0015]

[0016] Among them, C is the number of categories, C is a finite positive integer, n i is the i-th category, f i is the relative frequency of category i in the data set, y i is the true probability of the ith sample, is the predicted probability of the ith sample.

[0017] 2) Training indoor point cloud semantic segmentation network

[0018] The training set is input into the indoor point cloud semantic segmentation network for training. The operating system is Ubuntu 22.04, the GPU is Nvidia RTX4090D, the CPU is Intel Xeon Platinum 8474C, and the training parameters are: the initial learning rate is 0.001, the momentum decay is 0.5, the batch size is 32, the number of training rounds is 200, and the training is carried out until the weighted cross entropy loss function converges.

[0019] (4) Test the indoor point cloud semantic segmentation network

[0020] The test set is input into the trained indoor point cloud semantic segmentation network for testing, and the indoor point cloud semantic segmentation results are output and visualized.

[0021] In step (2) of the present invention, in constructing an indoor point cloud semantic segmentation network, the feature extraction branch 1 is composed of a context-aware attention convolution module 1 and a graph space attention convolution module 1 in parallel; the feature extraction branch 2 is composed of a context-aware attention convolution module 2 and a graph space attention convolution module 2 in parallel; the feature extraction branch 3 is composed of a context-aware attention convolution module 3 and a graph space attention convolution module 3 in parallel.

[0022] The construction method of the adaptive local neighborhood search module of the present invention is:

[0023] Determine the adaptive local neighborhood range N of each point according to formula (2): i :

[0024]

[0025] Among them, n is the number of neighbor points, which is a finite positive integer, and d(p ij ) is p i With p j The Euclidean distance between i ) and D(p j ) are local densities, p i is the center point of the local neighborhood, p j For p i neighbor points, KNN is the number selection operation of the k nearest neighbor points in the neighborhood, and the value range of k is 15 to 45.

[0026] The adaptive local neighborhood range N of each point i The cycle is repeated n times to construct an adaptive local neighborhood search module.

[0027] The construction method of the context-aware attention convolution module 1 of the present invention is as follows:

[0028] According to formula (3), the feature F extracted by the context-aware attention convolution module 1 is determined i %:

[0029]

[0030] Among them, α x is the attention score, F i is the local feature of the adaptive local neighborhood search module, m is the total number of neighbor points, M h is auxiliary information used to enhance the model's judgment of key features, h is a general term for other feature attributes, w is the number of nearest neighbor points, w ranges from 16 to 36, R is the number of nearest neighbors, which is determined by the adaptive local neighborhood search module, and r x is the xth neighborhood, s is the position coordinate of the point cloud, p iis the center point of the neighborhood, is the feature of the kth neighbor point in the xth neighborhood, is the feature of the neighbor point in the xth neighborhood.

[0031] Construct a context-aware attention convolution module 1; the construction method of context-aware attention convolution module 2 and context-aware attention convolution module 3 is the same as the construction method of context-aware attention convolution module 1.

[0032] The construction method of the graph space attention convolution module 1 of the present invention is as follows:

[0033] According to formula (4), we can get the feature Z extracted by graph space attention convolution module 1:

[0034] Z=σ(θ((H×W T )⊙((H×W T ) T )))×H(4)

[0035] H=X×W

[0036] Among them, X represents the n×9-dimensional features in the adaptive local neighborhood search module, W represents the 9×m-dimensional weights, θ is the LeakyReLU function for obtaining similarity features, σ is the softmax function of the normalization operation, and ⊙ is the element-by-element multiplication.

[0037] The graph space attention convolution module 1 is constructed. The construction method of the graph space attention convolution module 2 and the graph space attention convolution module 3 is the same as the construction method of the graph space attention convolution module 1.

[0038] The method for constructing the indoor point cloud spatial pyramid pooling layer of the present invention is as follows:

[0039] According to formula (5), the feature F of the indoor point cloud after spatial pyramid pooling layer aggregation is obtained:

[0040]

[0041] Among them, Con represents multi-scale feature concatenation, L represents overlapping pooling operation, F represents feature map, s i is the position coordinate of the i-th indoor point, and the value of i is a finite positive integer.

[0042] Construct a spatial pyramid pooling layer for indoor point cloud.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention adaptively adjusts the local receptive field of each point through the density change of the point cloud, which can alleviate the problem of local detail loss caused by the fixed neighborhood radius in other existing methods; the two modules of context-aware attention convolution and graph space attention convolution are used to obtain the key contextual semantic features and complex spatial geometric features between points respectively, so as to improve the network's ability to utilize spatial information of three-dimensional point clouds; the indoor point cloud spatial pyramid pooling layer combines the multi-level features obtained from different receptive fields, and enhances the ability of the entire network to extract and express features of different scales. Comparative experiments show that the present invention can achieve more accurate recognition and segmentation in complex indoor environments, especially the semantic categories of a few samples and easily confusing architectural structures, providing a new technical reference for indoor three-dimensional scene analysis and reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of embodiment 1 of the present invention.

[0046] Figure 2 It is a structural diagram of the indoor point cloud semantic segmentation network.

[0047] Figure 3 yes Figure 2 Schematic diagram of the structure of the multi-scale feature fusion module.

[0048] Figure 4 This is the original indoor point cloud and semantic segmentation result diagram of Example 1 of the present invention. DETAILED DESCRIPTION

[0049] The present invention is further described below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0050] Example 1

[0051] exist Figure 1 In the embodiment, the indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion is composed of the following steps:

[0052] (1) Indoor point cloud preprocessing

[0053] Select the indoor point cloud dataset and perform preprocessing: divide the 272 rooms in the S3DIS point cloud dataset into sampling blocks containing 4096 points with an area of ​​1m×1m and a spacing of 0.5m; and divide them into training set and test set with a ratio of 5:1.

[0054] (2) Constructing an indoor point cloud semantic segmentation network

[0055] Figure 2 A schematic diagram of the structure of the indoor point cloud semantic segmentation network in this embodiment is given. Figure 2In the embodiment, the indoor point cloud semantic segmentation network is composed of an adaptive local neighborhood search module, a multi-scale feature fusion module, and a shared multi-layer perceptron MLP connected in series.

[0056] The construction method of the adaptive local neighborhood search module of this embodiment is:

[0057] Determine the adaptive local neighborhood range N of each point according to formula (1): i :

[0058]

[0059] Among them, n is the number of neighbor points, which is a finite positive integer, and d(p ij ) is p i With p j The Euclidean distance between i ) and D(p j ) are local densities, p i is the center point of the local neighborhood, p j For p i neighbor points, KNN is a number selection operation of the k nearest neighbor points in the neighborhood, the value range of k is 15 to 45, and the value of k in this embodiment is 30.

[0060] The adaptive local neighborhood range N of each point i The cycle is repeated n times to construct an adaptive local neighborhood search module.

[0061] Figure 3 Given Figure 2 Schematic diagram of the structure of the multi-scale feature fusion module. Figure 3 In the embodiment, the multi-scale feature fusion module is composed of feature extraction branch 1, feature extraction branch 2, feature extraction branch 3, and indoor point cloud spatial pyramid pooling layer connected in series in sequence.

[0062] The feature extraction branch 1 of this embodiment is composed of a context-aware attention convolution module 1 and a graph space attention convolution module 1 in parallel. The feature extraction branch 2 is composed of a context-aware attention convolution module 2 and a graph space attention convolution module 2 in parallel. The feature extraction branch 3 is composed of a context-aware attention convolution module 3 and a graph space attention convolution module 3 in parallel.

[0063] The construction method of the context-aware attention convolution module 1 of this embodiment is as follows:

[0064] According to formula (2), the features extracted by the context-aware attention convolution module 1 are determined

[0065]

[0066] Among them, α x is the attention score, F i is the local feature of the adaptive local neighborhood search module, m is the total number of neighbor points, M h is auxiliary information used to enhance the model's judgment of key features, h is a general term for other feature attributes, w is the number of nearest neighbor points, w ranges from 16 to 36, and in this embodiment, w is 32, R is the number of nearest neighbors, which is determined specifically according to the adaptive local neighborhood search module, and r x is the xth neighborhood, s is the position coordinate of the point cloud, p i is the center point of the neighborhood, is the feature of the kth neighbor point in the xth neighborhood, is the feature of the neighbor point in the xth neighborhood.

[0067] The context-aware attention convolution module 1 is constructed. The construction method of the context-aware attention convolution module 2 and the context-aware attention convolution module 3 is the same as the construction method of the context-aware attention convolution module 1.

[0068] The construction method of the graph space attention convolution module 1 of this embodiment is as follows:

[0069] According to formula (3), we can get the feature Z extracted by graph space attention convolution module 1:

[0070] Z=σ(θ((H×W T )⊙((H×W T ) T )))×H(3)

[0071] H=X×W

[0072] Among them, X represents the n×9-dimensional features in the adaptive local neighborhood search module, W represents the 9×m-dimensional weights, θ is the LeakyReLU function for obtaining similarity features, σ is the softmax function of the normalization operation, and ⊙ is the element-by-element multiplication.

[0073] The graph space attention convolution module 1 is constructed. The construction method of the graph space attention convolution module 2 and the graph space attention convolution module 3 is the same as the construction method of the graph space attention convolution module 1.

[0074] Since the present invention adopts two modules, context-aware attention convolution and graph space attention convolution, to respectively obtain the key contextual semantic features and complex spatial geometric features between points, the network's ability to utilize the spatial information of three-dimensional point clouds is improved.

[0075] The method for constructing the indoor point cloud spatial pyramid pooling layer in this embodiment is as follows:

[0076] According to formula (4), the feature of indoor point cloud after spatial pyramid pooling layer aggregation is obtained:

[0077]

[0078] Among them, Con represents multi-scale feature concatenation, L represents overlapping pooling operation, F represents feature map, s i is the position coordinate of the i-th indoor point, and the value of i is a finite positive integer.

[0079] Construct a spatial pyramid pooling layer for indoor point cloud.

[0080] Since the present invention adopts the indoor point cloud spatial pyramid pooling layer to combine the multi-level features obtained from different receptive fields, the ability of the entire network to extract and express features of different scales is enhanced.

[0081] (3) Training indoor point cloud semantic segmentation network

[0082] 1) Constructing weighted cross entropy loss function

[0083] The weighted cross entropy loss function is constructed as follows:

[0084]

[0085] Where C is the number of categories, and the value of C is a finite positive integer. In this example, the value of C is 13, n i is the i-th category, f i is the relative frequency of category i in the data set, y i is the true probability of the ith sample, is the predicted probability of the ith sample.

[0086] 2) Training indoor point cloud semantic segmentation network

[0087] The training set is input into the indoor point cloud semantic segmentation network for training. The operating system is Ubuntu 22.04, the GPU is Nvidia RTX 4090D, the CPU is Intel Xeon Platinum 8474C, and the training parameters are: the initial learning rate is 0.001, the momentum decay is 0.5, the batch size is 32, the number of training rounds is 200, and the training is carried out until the weighted cross entropy loss function converges.

[0088] (4) Test the indoor point cloud semantic segmentation network

[0089] The test set is input into the trained indoor point cloud semantic segmentation network for testing, and the indoor point cloud semantic segmentation results are output and visualized.

[0090] Complete the indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion.

[0091] Example 2

[0092] The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion in this embodiment consists of the following steps:

[0093] (1) Indoor point cloud preprocessing

[0094] This step is the same as in Example 1.

[0095] (2) Constructing an indoor point cloud semantic segmentation network

[0096] The structure of the indoor point cloud semantic segmentation network is the same as that in Example 1.

[0097] The construction method of the adaptive local neighborhood search module of this embodiment is:

[0098] Determine the adaptive local neighborhood range N of each point according to formula (1): i :

[0099] The expression of formula (1) is the same as that of Example 1.

[0100] In formula (1), KNN is a number selection operation of the k nearest neighbor points in a neighborhood, and the value range of k is 15 to 45. In this embodiment, the value of k is 15. The meanings and value ranges of other parameters are the same as those in Embodiment 1.

[0101] The construction method of the context-aware attention convolution module 1 of this embodiment is as follows:

[0102] According to formula (2), the features extracted by the context-aware attention convolution module 1 are determined

[0103] The expression of formula (2) is the same as that of Example 1.

[0104] In formula (2), m is the total number of neighbor points, w is the number of nearest neighbor points, and the value range of w is 16 to 36. In this embodiment, the value of w is 16. The meanings and value ranges of other parameters are the same as those in Example 1.

[0105] The construction method of the graph space attention convolution module 1 of this embodiment is as follows:

[0106] According to formula (3), we can get the feature Z extracted by graph space attention convolution module 1:

[0107] The expression of formula (3) is the same as that of Example 1.

[0108] In formula (3), the meaning and value range of each parameter are the same as those in Example 1.

[0109] The method for constructing the indoor point cloud spatial pyramid pooling layer in this embodiment is as follows:

[0110] According to formula (4), the feature F of the indoor point cloud after spatial pyramid pooling layer aggregation is obtained:

[0111] The expression of formula (4) is the same as that of Example 1.

[0112] In formula (4), the meaning and value range of each parameter are the same as those in Example 1.

[0113] The other steps are the same as those in Example 1. The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion is completed.

[0114] Example 3

[0115] The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion in this embodiment consists of the following steps:

[0116] (1) Indoor point cloud preprocessing

[0117] This step is the same as in Example 1.

[0118] (2) Constructing an indoor point cloud semantic segmentation network

[0119] The structure of the indoor point cloud semantic segmentation network is the same as that in Example 1.

[0120] The construction method of the adaptive local neighborhood search module of this embodiment is:

[0121] Determine the adaptive local neighborhood range N of each point according to formula (1): i :

[0122] The expression of formula (1) is the same as that of Example 1.

[0123] In formula (1), KNN is a number selection operation of the k nearest neighbor points in a neighborhood, and the value range of k is 15 to 45. The value of k in this embodiment is 45. The meanings and value ranges of other parameters are the same as those in embodiment 1.

[0124] The construction method of the context-aware attention convolution module 1 of this embodiment is as follows:

[0125] According to formula (2), the feature F extracted by the context-aware attention convolution module 1 is determined i %:

[0126] The expression of formula (2) is the same as that of Example 1.

[0127] In formula (2), m is the total number of neighbor points, w is the number of nearest neighbor points, the value range of w is 16 to 36, the value of w in this embodiment is 36, and the meanings and value ranges of other parameters are the same as those in Example 1.

[0128] The construction method of the graph space attention convolution module 1 of this embodiment is as follows:

[0129] According to formula (3), we can get the feature Z extracted by graph space attention convolution module 1:

[0130] The expression of formula (3) is the same as that of Example 1.

[0131] In formula (3), the meaning and value range of each parameter are the same as those in Example 1.

[0132] The method for constructing the indoor point cloud spatial pyramid pooling layer in this embodiment is as follows:

[0133] According to formula (4), the feature F of the indoor point cloud after spatial pyramid pooling layer aggregation is obtained:

[0134] The expression of formula (4) is the same as that of Example 1.

[0135] In formula (4), the meaning and value range of each parameter are the same as those in Example 1.

[0136] The other steps are the same as those in Example 1. The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion is completed.

[0137] In order to demonstrate the beneficial effects of the present invention, an experiment was conducted using the method of Example 1 of the present invention, and the experimental results are as follows:

[0138] (1) Computation and simulation experiments

[0139] The method of Example 1 of the present invention was used to carry out an experiment, and the experimental results are shown in Figure 4 ,exist Figure 4 In the figure, (1) represents the original indoor point cloud image, and (2) represents the semantic segmentation result image. The semantic segmentation result image uses different colors to represent different semantic categories of the indoor point cloud. Figure 4 Medium light green is for the ceiling, dark green is for the bookcase, light blue is for the walls, dark blue is for the floor, red is for the chairs, purple is for the table, blue-purple is for the windows, and black is for the sundries.

[0140] (2) Computational simulation and comparative experiments

[0141] The indoor point cloud semantic segmentation method based on adaptive local neighborhood search and multi-scale feature fusion of Example 1 of the present invention was compared with other indoor point cloud semantic segmentation methods LGGCM (hereinafter referred to as Comparison 1), SegGCN (hereinafter referred to as Comparison 2), IAF-Net (hereinafter referred to as Comparison 3), NeiEA-Net (hereinafter referred to as Comparison 4), and FA-ResNet (hereinafter referred to as Comparison 5) for comparison experiments. The mean intersection over union (mIoU), overall segmentation accuracy (OA) and intersection over union (IoU) of 13 semantic categories were used as semantic segmentation performance indicators. The semantic categories include ceiling (hereinafter referred to as ceil.), floor (hereinafter referred to as floor), wall (hereinafter referred to as wall), beam (hereinafter referred to as beam), column (hereinafter referred to as col.), window (hereinafter referred to as wind.), door (hereinafter referred to as door), table (hereinafter referred to as tab.), chair (hereinafter referred to as chair), sofa (hereinafter referred to as sofa), bookcase (hereinafter referred to as book.), board (hereinafter referred to as board), and sundries (hereinafter referred to as clut.).

[0142] Determine the intersection over union (IoU) as follows:

[0143]

[0144] Among them, i is the true value, j is the predicted value, k is the number of semantic categories, and p ii To predict i as i, p ij To predict i as j, p ki To predict k as i.

[0145] Determine the mean intersection over union (mIoU) as follows:

[0146]

[0147] Among them, i is the true value, j is the predicted value, k is the number of semantic categories, and p ii To predict i as i, p ij To predict i as j, p ji To predict j as i.

[0148] The overall segmentation accuracy OA is determined as follows:

[0149]

[0150] Among them, i is the true value, j is the predicted value, k is the number of semantic categories, and p ii To predict i as i, p ij To predict i as j.

[0151] The experimental comparison results on region 5 of the S3DIS dataset are shown in Table 1.

[0152] Table 1 Semantic segmentation results of the proposed method and other comparative methods

[0153] Experimental methods QUR OA ceil. floor wall beam col. wind. door tab. chair sofa book. board clut. Comparison 1 63.3 88.8 94.8 98.3 81.5 0.0 35.9 63.3 43.5 80.2 88.4 68.8 55.7 64.6 47.8 Comparison 2 63.6 88.2 93.7 98.6 80.6 0.0 28.5 42.6 74.5 80.9 88.7 69.0 71.3 44.4 54.3 Contrast 3 64.6 88.4 91.4 98.6 81.8 0.0 34.9 62.0 54.7 79.7 86.9 49.9 72.4 74.8 52.1 Contrast 4 66.1 88.5 92.9 97.4 83.3 0.0 34.9 61.8 55.3 78.8 86.7 77.1 69.5 67.9 54.2 Contrast 5 68.1 88.9 94.0 97.9 82.6 0.0 39.1 64.1 65.2 78.5 87.5 77.6 69.8 73.3 56.0 This method 68.7 88.8 95.3 98.6 82.7 0.0 39.4 65.8 68.3 78.6 87.9 71.3 70.5 75.2 59.7

[0154] As shown in Table 1, the mIoU of the proposed method in the task of indoor point cloud semantic segmentation is 68.7%, and the highest segmentation accuracy is achieved on 6 categories (ceiling, floor, pillar, window, wooden board and debris). The present invention adaptively constructs the graph structure, effectively extracts the local detail features, context features and global spatial relations of the indoor point cloud, and improves the overall performance of semantic segmentation, especially the segmentation on a few categories (pillars and windows).

[0155] In summary, the experimental results compared with other segmentation methods show that the present invention effectively alleviates the problem of local detail loss caused by traditional graph construction, integrates the multi-scale features of indoor point clouds, has the advantages of strong adaptability and good segmentation performance, and is conducive to scene understanding and reconstruction of indoor three-dimensional space.

Claims

1. A method for indoor point cloud semantic segmentation based on adaptive and multi-scale feature fusion, characterized in that It consists of the following steps: (1) Indoor point cloud preprocessing Select the indoor point cloud dataset and perform preprocessing: divide the 272 rooms in the S3DIS point cloud dataset into sampling blocks containing 4096 points with an area of ​​1m×1m and a spacing of 0.5m; divide them into training set and test set with a ratio of 5:1; (2) Constructing an indoor point cloud semantic segmentation network The indoor point cloud semantic segmentation network is composed of an adaptive local neighborhood search module, a multi-scale feature fusion module, and a shared multi-layer perceptron MLP in series. The multi-scale feature fusion module is composed of feature extraction branch 1, feature extraction branch 2, feature extraction branch 3, and indoor point cloud space pyramid pooling layer connected in series in sequence; (3) Training indoor point cloud semantic segmentation network 1) Constructing weighted cross entropy loss function The weighted cross entropy loss function is constructed according to formula (1): Among them, C is the number of categories, C is a finite positive integer, n i is the i-th category, f i is the relative frequency of category i in the data set, y i is the true probability of the ith sample, is the predicted probability of the i-th sample; 2) Training indoor point cloud semantic segmentation network The training set is input into the indoor point cloud semantic segmentation network for training. The operating system is Ubuntu 22.04, the GPU is Nvidia RTX 4090D, the CPU is Intel Xeon Platinum 8474C, and the training parameters are: the initial learning rate is 0.001, the momentum decay is 0.5, the batch size is 32, the number of training rounds is 200, and the training is carried out until the weighted cross entropy loss function converges; (4) Testing the indoor point cloud semantic segmentation network The test set is input into the trained indoor point cloud semantic segmentation network for testing, and the indoor point cloud semantic segmentation results are output and visualized.

2. The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion according to claim 1 is characterized in that In step (2) of constructing the indoor point cloud semantic segmentation network, the feature extraction branch 1 is composed of a context-aware attention convolution module 1 and a graph space attention convolution module 1 in parallel; the feature extraction branch 2 is composed of a context-aware attention convolution module 2 and a graph space attention convolution module 2 in parallel; the feature extraction branch 3 is composed of a context-aware attention convolution module 3 and a graph space attention convolution module 3 in parallel.

3. The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion according to claim 1, characterized in that: The construction method of the adaptive local neighborhood search module is: Determine the adaptive local neighborhood range N of each point according to formula (2): i : D(p i )=|KNN(p i )| D(p j )=|KNN(p j )| Among them, n is the number of neighbor points, which is a finite positive integer, and d(p ij ) is p i With p j The Euclidean distance between i ) and D(p j ) are local densities, p i is the center point of the local neighborhood, p j For p i Neighbor points, KNN is the number selection operation of the nearest k neighbor points in the neighborhood, and the value range of k is 15 to 45; The adaptive local neighborhood range N of each point i The cycle is repeated n times to construct an adaptive local neighborhood search module.

4. The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion according to claim 1 is characterized in that The construction method of the context-aware attention convolution module 1 is as follows: According to formula (3), the features extracted by the context-aware attention convolution module 1 are determined Among them, α x is the attention score, F i is the local feature of the adaptive local neighborhood search module, m is the total number of neighbor points, M h is auxiliary information used to enhance the model's judgment of key features, h is a general term for other feature attributes, w is the number of nearest neighbor points, w ranges from 16 to 36, R is the number of nearest neighbors, which is determined by the adaptive local neighborhood search module, and r x is the xth neighborhood, s is the position coordinate of the point cloud, p i is the center point of the neighborhood, is the feature of the kth neighbor point in the xth neighborhood, is the feature of the neighbor point in the xth neighborhood; Construct a context-aware attention convolution module 1; the construction method of context-aware attention convolution module 2 and context-aware attention convolution module 3 is the same as the construction method of context-aware attention convolution module 1.

5. The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion according to claim 1, characterized in that The construction method of the graph space attention convolution module 1 is as follows: According to formula (4), we can get the feature Z extracted by graph space attention convolution module 1: Z=σ(θ((H×W T )⊙((H×W T ) T )))×H (4) H=X×W Among them, X represents the n×9-dimensional features in the adaptive local neighborhood search module, W represents the 9×m-dimensional weight, θ is the LeakyReLU function for obtaining similarity features, σ is the softmax function of the normalization operation, and ⊙ is the element-by-element multiplication; The graph space attention convolution module 1 is constructed. The construction method of the graph space attention convolution module 2 and the graph space attention convolution module 3 is the same as the construction method of the graph space attention convolution module 1.

6. The indoor point cloud semantic segmentation method based on adaptive and multi-scale feature fusion according to claim 1, characterized in that The method for constructing the indoor point cloud spatial pyramid pooling layer is as follows: According to formula (5), the feature of indoor point cloud after spatial pyramid pooling layer aggregation is obtained: Among them, Con represents multi-scale feature concatenation, L represents overlapping pooling operation, F represents feature map, s i is the position coordinate of the i-th indoor point, and the value of i is a finite positive integer; Construct a spatial pyramid pooling layer for indoor point cloud.

Images