A fine-grained scene parsing method and system for indoor three-dimensional point clouds

Abstract

The application discloses a fine-grained scene analysis method and system for indoor three-dimensional point clouds, and belongs to the technical field of intelligent processing of three-dimensional point clouds. The method firstly acquires indoor scene point clouds and performs initial feature embedding on three-dimensional coordinates and color attributes; in the encoding stage, a boundary perception local feature enhancement module is introduced, a local boundary response is generated through feature differences, relative coordinate differences and geometric distances of a center point and neighborhood points, and a gated fusion mechanism is adopted to enhance features of a category boundary area; in the decoding stage, a hierarchical semantic guided refinement module is constructed, deep semantic information is gradually returned to middle and low layer features, local cross-layer semantic aggregation and gated residual fusion are adopted to improve fine-grained component recovery capability; and during training, a point-by-point semantic segmentation loss and a boundary constraint loss are combined for joint optimization. The application can improve boundary blur, category adhesion and small-scale component missing problems in indoor point cloud component identification, and improve the accuracy and structural integrity of semantic segmentation.

Description

Technical Field

[0001] This invention belongs to the field of intelligent processing technology of 3D point clouds, and particularly relates to a fine-grained scene analysis method and system for indoor 3D point clouds, which can be used in application scenarios such as automatic identification of indoor components, robot navigation, digital twins, intelligent space perception and indoor 3D modeling. Background Technology

[0002] With the development of technologies such as LiDAR, depth cameras, service robots, augmented reality, and smart homes, indoor 3D point clouds have become an important data form for expressing spatial structure and component information. Indoor point cloud semantic segmentation can assign semantic category labels to each point in a scene, enabling automatic identification of components such as walls, floors, ceilings, doors, windows, and furniture. It is a key technology in applications such as robot navigation, digital twins, and intelligent spatial perception.

[0003] While existing point cloud semantic segmentation methods can achieve indoor scene recognition to a certain extent, they still have shortcomings. Due to the characteristics of indoor point clouds, such as disorder, uneven density, frequent occlusion, and significant differences in class scale, conventional deep learning models are prone to problems such as blurred boundaries, class overlap, and local omissions when processing small-scale components such as doors, windows, columns, panels, and furniture edges. At the same time, existing networks rely heavily on ordinary neighborhood aggregation during feature extraction, lacking explicit modeling of class boundary regions and local geometric abrupt changes; in the decoding stage, the fusion of deep semantic information and shallow spatial details is insufficient, making it difficult to effectively recover complex boundaries and fine-grained component structures.

[0004] Furthermore, existing methods typically rely primarily on point-by-point classification loss for training, lacking specific constraints on boundary regions, resulting in insufficient ability to discriminate boundaries between adjacent components. Therefore, enhancing the feature representation of boundary regions, achieving effective interaction between deep semantics and shallow spatial details, and improving the segmentation integrity of small-scale components are pressing issues that current indoor point cloud semantic segmentation technologies need to address. Summary of the Invention

[0005] To address the problems of blurred boundaries, missed classification of small-scale components, category overlap, and insufficient fusion of shallow and deep features in existing technologies for indoor point cloud component recognition, this invention proposes a fine-grained scene parsing method and system for indoor 3D point clouds. This method improves the accuracy, boundary clarity, and structural integrity of indoor point cloud component recognition by introducing a boundary-aware local feature enhancement module in the encoding stage, constructing a hierarchical semantic-guided refinement module in the decoding stage, and jointly optimizing the module with boundary constraint loss.

[0006] The above objectives are achieved through the following technical solutions:

[0007] This invention provides a fine-grained scene analysis method and system for indoor 3D point clouds, the method comprising the following steps:

[0008] S1. The input point cloud undergoes initial mapping through a feature embedding layer, converting the original coordinates and attribute information into a high-dimensional point feature representation;

[0009] S2. PointNeXt is used as the basic backbone network, and hierarchical features from shallow to deep are obtained through multi-stage sampling, neighborhood query, and local aggregation. and corresponding coordinate set L represents the total number of levels in the encoding stage. A boundary-aware local feature enhancement module is introduced in the encoding stage. It generates local boundary responses by using the feature differences, relative coordinate differences and geometric distances between the center point and neighboring points, and uses a gating fusion mechanism to enhance the features of the category boundary region.

[0010] S3. In the decoding stage, a hierarchical semantic guidance and refinement module is introduced. Starting from deep high semantic features, it is backed up to the mid-to-low-level feature representations along the decoding path. The fine-grained spatial structure is restored through cross-layer semantic aggregation and gated residual fusion.

[0011] S4. During the training phase, the semantic segmentation main loss and boundary constraint loss are jointly optimized. The boundary labels are automatically generated by point-by-point semantic annotation, eliminating the need for additional manual boundary annotation. At the same time, a joint optimization objective consisting of semantic segmentation loss and boundary constraint loss is used to supervise the network from two levels: global semantic discrimination and local boundary constraint.

[0012] S5. The semantic category of each point is directly output during the inference phase.

[0013] Furthermore, the specific method for step S1 is as follows:

[0014] Given a given original indoor point cloud First, the point cloud is preprocessed by coordinate normalization, block sampling, or voxel sampling to adapt it to the network input scale. Then, the feature embedding layer maps the 3D spatial coordinates and initial attribute features to initial point features. ;in Indicates the first The three-dimensional spatial coordinates of the points Indicates the first Initial attribute features of each point Enter the number of points.

[0015] Furthermore, the specific method for step S2 is as follows:

[0016] In the shallow and mid-level feature stages of the PointNeXt encoder, a boundary-aware local feature enhancement module is set up. Local boundary descriptors are constructed using feature differences, relative coordinate differences, and geometric distances between the center point and its neighbors. The boundary response is then adaptively injected into the original features through a gated fusion mechanism. Specifically:

[0017] Let the first The first in the layer The features and coordinates of each point are denoted as follows: and , , No. The first in the layer The local neighborhood set of a point is denoted as . First, construct the local boundary descriptor. : in, This indicates a feature concatenation operation. Denotes the Euclidean norm. and indicates They represent the first The first in the layer Neighboring points of each point Features and coordinates;

[0018] After obtaining the local boundary descriptor Then, a lightweight mapping function is used to perform a nonlinear transformation on the neighborhood difference information, and max pooling is performed along the neighborhood dimension to obtain the point-level boundary response. : in, This represents the boundary mapping function composed of a multilayer sensing mechanism. This indicates max pooling along the neighborhood dimension, used to highlight the most significant boundary response patterns;

[0019] Subsequently, the boundary response features are gated and fused with the original hierarchical features to obtain the boundary-enhanced feature representation: in, and These are learnable mapping parameters; This represents the Sigmoid activation function; This represents element-wise multiplication; Indicates the first The first of the levels The boundary gating weights of each point are used to adaptively control the degree to which the boundary response features enhance the original point features; Indicates the first The first of the levels The output features of each point after boundary awareness enhancement are the enhanced features after gating fusion.

[0020] Furthermore, the specific method for step S3 is as follows:

[0021] S3. In the decoding stage, a hierarchical semantic guidance and refinement module is introduced. Starting from deep high semantic features, it is backed up to the mid-to-low-level feature representations along the decoding path. The fine-grained spatial structure is restored through cross-layer semantic aggregation and gated residual fusion.

[0022] A hierarchical semantic guidance and refinement module is set up in the decoding stage. This module starts from the deep semantic features at the end of the encoder and backtracks to the middle and shallow features level by level along the decoding stage. Through cross-layer semantic aggregation, high-level semantics guide the low-level spatial details, thereby continuously narrowing the semantic gap between different levels of features while restoring spatial resolution. This enhances the model's ability to recover small-scale objects, local boundaries, and fine-grained components. Specifically:

[0023] Let the first The encoded features of the layer after boundary enhancement are ,in These represent different decoding levels from shallow to deep, while the deepest encoded feature at the end of the encoder is denoted as... For the deepest features at the end of the encoder Since it lacks deeper semantic references, an initial semantic projection function is first used. Map it to a deep initial semantic representation : in, This represents the deepest encoded feature at the end of the encoder; This represents the deepest encoded feature acting at the end of the encoder. The initial semantic projection function;

[0024] Subsequently, for shallower levels, the decoder first passes through the feature propagation unit. The current layer boundary enhancement features are fused with the deeper backpropagation semantic features to obtain the basic decoding features at the current scale: in, Indicates an upsampling operation. Indicates the first The feature representation of a layer after being recovered by the base decoder, i.e., the base decoded features at the current scale. This represents deeper semantic refinement features in the backhaul; the current-scale basic decoding features. It also contains local geometric details at the current scale, boundary enhancement information, and upsampled semantic information from deeper layers, serving as the main input for subsequent hierarchical semantic guidance refinement;

[0025] After obtaining the basic decoding features at the current scale, the hierarchical semantic guidance refinement module then... As the main input, it is used to refine the hierarchical semantic features through deeper feedback. As a reference input, the refinement function is guided by hierarchical semantics. Get the output of the current layer: in, Indicates the first Layer-level semantic refinement features;

[0026] Hierarchical semantic guidance refinement function It consists of two parts: local cross-layer semantic aggregation and gated residual fusion. First, it decodes the basic features at the current scale. Perform bottleneck mapping to obtain the semantic representation of the current layer. : in, Indicates by The bottleneck mapping function consists of convolutional layers, normalization layers, and nonlinear activation functions;

[0027] Subsequently, the semantic representation of the current layer will be... As query features, more refined semantic features will be returned at a deeper level. As supporting features, queries, key-value mappings, and other functions are performed separately: in, This represents the query mapping parameters, used to represent the semantics of the current layer. Mapped to query features ; This represents the key mapping parameters, used to refine features to a deeper level of semantic detail. Mapping to key features ; The value mapping parameters are used to refine features to a deeper level of semantic detail. Mapping to value features , , and These represent the current layer query features, deeper layer key features, and value features, respectively.

[0028] Next, a neighborhood-based local attention mechanism is introduced into cross-layer semantic aggregation, for coordinates... The The first in the layer Each point, in a deeper point set Select its local neighborhood And combining relative position coding to model cross-layer geometric relationships, the first A point and its neighboring points The relative positions between them are represented as follows: in, Indicates the first Layer The point and the first Layer Neighborhood Points The relative positions of the layers between them Indicates the first Neighborhood points in the layer 3D coordinates;

[0029] Cross-layer attention weights are based on query, key features, and relative position encoding. Represented as: in, and These represent the relative position encoding functions applied to the query branch and the relative position encoding functions applied to the key branch, respectively. For feature dimension, Indicates the first Layer Query characteristics of each point Indicates the first Layer Neighborhood Points The key feature is that the superscript T indicates the transpose operation;

[0030] Based on cross-layer attention weights This yields the current layer context features guided by deep semantics. : in, Indicating the first under the guidance of deep semantics Layer The current layer context features of each point Indicates the first Layer The point is the first Layer Neighborhood Points Cross-layer attention weights, Indicates by the first Hierarchical semantic refinement features Value mapping parameters The mapping obtained, the first Layer Neighborhood Points Value characteristics;

[0031] To simplify the representation, by and The results were obtained through querying, key-value mapping, cross-layer attention weight calculation, and weighted summation. The process is written as follows: in, This represents a cross-layer semantic aggregation operation based on local neighborhood relationships. This indicates the first [item] obtained under the guidance of deep semantics. Layer-to-layer contextual features;

[0032] In obtaining the first Layer-to-layer context features Subsequently, to avoid deep semantic information overly covering shallow geometric details, a gated residual fusion mechanism is further adopted to adaptively balance the contributions between the deep semantic context and the current layer's basic decoding features. First, the current layer's basic decoding features are mapped to a channel space consistent with the context features: in, For learnable linear mapping parameters, For the first The current layer's details are preserved.

[0033] Subsequently, gating weights are generated based on cross-layer context features and current layer residual information. : in, Indicates the output mapping function, Represents the gated mapping function. Use the Sigmoid activation function;

[0034] Finally, the first The refined output of the layer is represented as follows: In summary, the hierarchical semantic guidance refinement process can be summarized as follows: .

[0035] Furthermore, the specific method for step S4 is as follows:

[0036] During the training phase, semantic segmentation main loss and boundary constraint loss are jointly optimized, and the overall loss function is... The definition is as follows: in, This is the main loss for semantic segmentation, used to constrain the consistency between the network output and the pointwise semantic labels; Boundary constraint loss is used to enhance the model's ability to identify boundary regions; The boundary loss weight coefficient is used to adjust the relative contribution between the main task supervision and the auxiliary task supervision.

[0037] Semantic segmentation main loss Using pointwise cross-entropy loss, let the input point cloud have a total of There are points, and the number of categories is . , No. The true category label for each point is: The model predicts that it belongs to the first The probability of class is Then the semantic segmentation main loss Defined as: in, For indicator functions, Represent the natural logarithm function;

[0038] Semantic segmentation main loss The loss function supervises the network from the perspective of overall category discrimination, enabling the model to learn the distribution patterns of different semantic categories in the feature space;

[0039] To construct boundary supervision signals, boundary labels are generated online based on real semantic labels. Specifically, for points... If its local neighborhood If a point exists that has a semantic label different from the center point, then that point is defined as a boundary point; otherwise, it is defined as a non-boundary point, and the boundary label... Represented as: in, Point Located in the category boundary area, This indicates that it is a non-boundary point; the boundary label is automatically generated by point-by-point semantic annotation, without the need for additional manual boundary annotation. Invalid labeled points are ignored in the process of boundary label construction and loss calculation.

[0040] Boundary prediction branch output Boundary prediction values ​​for each point Its boundary probability for: Based on boundary truth values and predicted probability The boundary constraint loss is expressed in the form of binary cross-entropy: .

[0041] This invention also provides a fine-grained scene parsing system for indoor 3D point clouds. This system is used to run the aforementioned method and includes a feature embedding module, a PointNeXt encoder, a hierarchical semantic refinement decoder, and a joint optimization module for semantic segmentation main loss and boundary constraint loss. The feature embedding module maps the input point cloud into high-dimensional point features. The PointNeXt encoder extracts hierarchical features of different resolutions through multi-stage downsampling and local neighborhood aggregation. The hierarchical semantic refinement decoder starts with deep, high-semantic features and progressively backtracks along the decoding path to mid-to-low-level feature representations, recovering the fine-grained spatial structure through cross-layer semantic aggregation and gated residual fusion. The joint optimization module for semantic segmentation main loss and boundary constraint loss automatically generates boundary labels from point-by-point semantic annotations and directly outputs the semantic category of each point.

[0042] The advantages of this invention compared to the prior art are:

[0043] 1. By introducing a boundary-aware local feature enhancement module during the encoding stage, this invention explicitly models the feature differences, spatial location differences, and geometric distances between the center point and its neighbors. This enhances the network's discriminative ability in class boundary regions and areas of local geometric abrupt changes, reducing the risk of over-averaging of features on both sides of the boundary during neighborhood aggregation. Through a gated fusion mechanism that adaptively injects boundary responses, this invention can enhance detailed representations in boundary regions while maintaining semantic feature stability in homogeneous regions, avoiding noise amplification or feature perturbation caused by simple global enhancement.

[0044] 2. Through the hierarchical semantic refinement module, this invention transmits deep high-semantic information back to mid-to-low-level spatial detail features step by step, improving the problem of insufficient interaction between deep semantics and shallow geometric details in conventional decoding structures, thereby enhancing the segmentation integrity of small-scale components, low-frequency categories and complex boundary regions.

[0045] 3. By automatically generating boundary labels and introducing boundary constraint loss, this invention enables the model training process to pay more attention to the class boundary region without increasing the additional manual annotation cost, thereby improving problems such as discontinuous boundary contours, blurred local structures, and class adhesion. Attached Figure Description

[0046] Figure 1 This is the overall flowchart of the present invention;

[0047] Figure 2 This is a system architecture diagram of the present invention;

[0048] Figure 3 This is a diagram of the point cloud input and hierarchical feature encoding architecture of the present invention;

[0049] Figure 4 This is a diagram of the boundary-aware local feature enhancement architecture of the present invention;

[0050] Figure 5 This is a diagram of the deep semantic initialization architecture of the present invention;

[0051] Figure 6 This is a schematic diagram of the hierarchical semantic refinement structure of the present invention;

[0052] Figure 7 This is an example of the ScanNet v2 dataset mentioned in the embodiments of the present invention;

[0053] Figure 8 This refers to the distribution of the percentage of points in each semantic category of ScanNet v2 mentioned in this embodiment of the invention;

[0054] Figure 9 Visualization of the ScanNet v2 semantic segmentation results mentioned in the embodiments of the present invention;

[0055] Figure 10 Visualization of the ScanNet v2 semantic segmentation results mentioned in the embodiments of the present invention (detail 1). Figure 10 In the figure, (a) shows the actual classification results with manual annotation, (b) shows the semantic segmentation results of the method of the present invention, (c) shows the semantic segmentation results of the KPConv method, (d) shows the semantic segmentation results of the PT method, (e) shows the semantic segmentation results of the PointNet++ method, and (f) shows the semantic segmentation results of the PointCNN method; the black rectangle is used to indicate the segmentation differences of different methods in local detail regions.

[0056] Figure 11 Visualization of the ScanNet v2 semantic segmentation results mentioned in the embodiments of the present invention (detail 2). Figure 11 In the figure, (a) shows the actual classification results with manual annotation, (b) shows the semantic segmentation results of the method of the present invention, (c) shows the semantic segmentation results of the KPConv method, (d) shows the semantic segmentation results of the PT method, (e) shows the semantic segmentation results of the PointNet++ method, and (f) shows the semantic segmentation results of the PointCNN method; the black rectangle is used to indicate the segmentation differences of different methods in local detail regions. Detailed Implementation

[0057] like Figure 1-2 As shown, this invention first provides a fine-grained scene parsing method for indoor 3D point clouds, which includes the following steps:

[0058] S1. The input point cloud undergoes initial mapping through a feature embedding layer, converting the original coordinates and attribute information into a high-dimensional point feature representation.

[0059] Given a given original indoor point cloud ,in Indicates the first The three-dimensional spatial coordinates of the points This indicates its initial attribute characteristics. To input the number of points, the point cloud is first preprocessed, including coordinate normalization, block sampling, or voxel sampling, to adapt it to the network input size. Then, the feature embedding layer maps the 3D spatial coordinates and initial attribute features to initial point features. .

[0060] S2. For example Figure 3 As shown, PointNeXt is used as the basic backbone network, and hierarchical features from shallow to deep are obtained through multi-stage sampling, neighborhood query, and local aggregation. and corresponding coordinate set L represents the total number of levels in the encoding stage; shallow features mainly preserve local geometric edges, spatial contours, and neighborhood details, while mid-to-deep features form stronger semantic expressions and a larger receptive field of context. This hierarchical feature structure provides the foundation for subsequent boundary enhancement and semantic refinement.

[0061] In indoor point cloud scenarios, there are often numerous adjacent, nested, or connected relationships between different components, such as the connection areas between doors / windows and walls, furniture edges, component corners, and beam-column intersections. These areas are accompanied by obvious local geometric abrupt changes and semantic transitions, making them the most prone locations for misclassification and boundary adhesion in point cloud semantic segmentation. Therefore, this invention sets up boundary-aware local feature enhancement modules in the shallow and mid-level feature stages of the PointNeXt encoder. These modules utilize the feature differences, relative coordinate differences, and geometric distances between the center point and neighboring points to construct local boundary descriptors, and adaptively inject the boundary responses into the original features through a gating fusion mechanism.

[0062] like Figure 4 As shown, let the first... The first in the layer The features and coordinates of each point are denoted as follows: and , Its local neighborhood set is denoted as To explicitly model the geometric and feature differences between the center point and its neighbors, a local boundary descriptor is first constructed. : in, This indicates a feature concatenation operation. Denotes the Euclidean norm. and indicates They represent the first The first in the layer Neighboring points of each point Features and coordinates. This descriptor It also includes information on center point features, neighborhood feature differences, relative coordinate differences, and geometric distance, which can better reflect the discontinuous features caused by component edges, category boundaries, and local structural abrupt changes.

[0063] After obtaining the local boundary descriptor, a lightweight mapping function is used to perform a nonlinear transformation on the neighborhood difference information, and max pooling is performed along the neighborhood dimension to obtain the point-level boundary response. : in, This represents the boundary mapping function composed of a multilayer sensing mechanism. This indicates max pooling along the neighborhood dimension, used to highlight the most significant boundary response patterns. Compared to average aggregation, it is more effective at preserving discriminative information such as sharp local changes, component outlines, and category transition regions.

[0064] Subsequently, the boundary response features are gated and fused with the original hierarchical features to obtain the boundary-enhanced feature representation: in, and For learnable mapping parameters, This represents the Sigmoid activation function. This represents element-wise multiplication. Indicates the first The first of the levels The boundary gating weights of each point are used to adaptively control the degree to which the boundary response features enhance the original point features. Indicates the first The first of the levels The output features of each point after boundary-aware enhancement are the enhanced features after gated fusion. This gating mechanism can adaptively adjust the injection ratio of boundary information according to the boundary response strength of the current point: when the point is located at the class boundary or in a region of geometric abrupt change, the model tends to enhance the boundary branch features; when the point is in a homogeneous region, the boundary enhancement effect is suppressed, thereby maintaining the stability of the original semantic features.

[0065] This invention preferably introduces boundary-aware local feature enhancement modules only in the shallow and mid-level feature stages, rather than repeating these modules in the deepest layer of the encoder. This is because shallow and mid-level features are closer to the original spatial structure, containing richer geometric edges, local contours, and neighborhood details; while deep features, after multiple downsampling and semantic aggregation, have significantly abstracted local boundary information, making further boundary enhancement difficult to achieve ideal results. Therefore, this invention explicitly injects boundary information into the first half of the backbone network, providing a clearer and more stable local structural foundation for subsequent decoding refinement.

[0066] S3. In the decoding stage, a hierarchical semantic guidance and refinement module is introduced. Starting from deep high semantic features, it is backed up to the mid-to-low-level feature representations along the decoding path. The fine-grained spatial structure is restored through cross-layer semantic aggregation and gated residual fusion.

[0067] Traditional decoding stages primarily rely on feature propagation and skip connections to achieve high-resolution reconstruction, but deep semantic features have limited ability to guide shallow spatial details. Especially for small-scale components and low-frequency categories, shallow features preserve spatial details but lack semantic discriminative power, while deep features offer strong semantic expression but low spatial resolution. To address this, this invention incorporates a hierarchical semantic guidance and refinement module in the decoding stage. This module starts with the deep semantic features at the encoder's end and propagates them back to mid-level and shallow features step-by-step along the decoding stage. Through cross-layer semantic aggregation, it guides lower-level spatial details with higher-level semantics, thereby restoring spatial resolution while continuously narrowing the semantic gap between different levels of features, enhancing the model's ability to recover small-scale objects, local boundaries, and fine-grained components.

[0068] Let the first The encoded features of the layer after boundary enhancement are ,in These represent different decoding levels from shallow to deep, while the deepest encoded feature is still denoted as... .like Figure 5 As shown, for the deepest feature at the end of the encoder Since it lacks deeper semantic references, this invention first employs an initial semantic projection function. Map it to a deep initial semantic representation : in, This represents the deepest encoded feature at the end of the encoder; This represents the initial semantic projection function applied to the encoded features of the 4th layer.

[0069] Subsequently, for shallower levels, the decoder first passes through the feature propagation unit. The current layer boundary enhancement features are fused with the deeper backpropagation semantic features to obtain the basic decoding features at the current scale: in, Indicates an upsampling operation. Indicates the first The feature representation of a layer after being recovered by the base decoder, i.e., the base decoded features at the current scale. This represents more refined semantic features that are returned at a deeper level. This is the current-scale basic decoding feature. It also contains local geometric details at the current scale, boundary enhancement information, and upsampled semantic information from deeper layers, serving as the main input for subsequent hierarchical semantic guidance and refinement.

[0070] like Figure 6 As shown, after obtaining the basic decoding features at the current scale, the hierarchical semantic guidance refinement module uses... As the main input, semantic refinement features are fed back at a deeper level. As a reference input, the refinement function is guided by hierarchical semantics. Get the output of the current layer: in, Indicates the first Layer-level semantic refinement features. This formula shows that the output of the current layer not only inherits the spatial details and boundary structure information recovered by the basic decoder, but is also subject to the backtracking constraints of deeper semantic features, thereby realizing the stepwise transfer of deep semantics to shallow spatial representation.

[0071] Specifically, hierarchical semantic guidance refinement function It mainly consists of two parts: local cross-layer semantic aggregation and gated residual fusion. First, it decodes the basic features at the current scale. Perform bottleneck mapping to obtain the semantic representation of the current layer. : in, Indicates by The bottleneck mapping function consists of convolutional layers, normalization layers, and nonlinear activation functions.

[0072] Then, the current layer features As query features, more refined semantic features will be returned at a deeper level. As supporting features, queries, key-value mappings, and other functions are performed separately: in, This represents the query mapping parameters, used to represent the semantics of the current layer. Mapped to query features ; This represents the key mapping parameters, used to refine features to a deeper level of semantic detail. Mapping to key features ; The value mapping parameters are used to refine features to a deeper level of semantic detail. Mapping to value features . , , and These represent the current layer query features, deeper layer key features, and value features, respectively.

[0073] To preserve the local spatial structure of the point cloud, this invention introduces a neighborhood-based local attention mechanism in cross-layer semantic aggregation. For coordinates... The The first in the layer Each point, in a deeper point set Select its local neighborhood And combine relative position coding to model cross-layer geometric relationships. A point and its neighboring points The relative positions between them are represented as follows: in, Indicates the first Layer The point and the first Layer Neighborhood Points The relative positions of the layers between them Indicates the first Neighborhood points in the layer 3D coordinates;

[0074] Cross-layer attention weights are based on query, key features, and relative position encoding. Represented as: in, and These represent the relative position encoding functions applied to the query branch and the relative position encoding functions applied to the key branch, respectively. For feature dimension, Indicates the first Layer Query characteristics of each point Indicates the first Layer Neighborhood Points The key features are represented by the superscript T, which indicates the transpose operation. By introducing cross-layer relative position encoding, the model can preserve the spatial relationship between the current layer point and the deep neighboring points during semantic backpropagation, thus preventing high-level semantic information from deviating from the local geometric structure during cross-scale propagation.

[0075] Based on the aforementioned attention weights, the current layer context features guided by deep semantics can be obtained. : in, Indicating the first under the guidance of deep semantics Layer The current layer context features of each point Indicates the first Layer The point is the first Layer Neighborhood Points Cross-layer attention weights, Indicates by the first Hierarchical semantic refinement features Value mapping parameters The mapping obtained, the first Layer Neighborhood Points The value characteristics.

[0076] To simplify the representation, by and The results were obtained through querying, key-value mapping, cross-layer attention weight calculation, and weighted summation. The process can be written as: in, This represents a cross-layer semantic aggregation operation based on local neighborhood relationships. This indicates the first [item] obtained under the guidance of deep semantics. Layer-to-layer contextual features. This process enables high-level semantics to selectively act on current-scale features in a locally relevant manner, rather than through indiscriminate information fusion via simple concatenation or direct addition.

[0077] In obtaining the first Layer-to-layer context features Subsequently, to avoid deep semantic information overly covering shallow geometric details, this invention further employs a gated residual fusion mechanism to adaptively balance the contributions between the deep semantic context and the current layer's basic decoding features. First, the current layer's basic decoding features are mapped to a channel space consistent with the context features: in, For learnable linear mapping parameters, For the first The current layer retains the details of the layer.

[0078] Subsequently, gating weights are generated based on cross-layer context features and current layer residual information. : in, Indicates the output mapping function, Represents the gated mapping function. This is the Sigmoid activation function. Finally, the... The refined output of the layer is represented as follows: in, This represents element-wise multiplication. The gated residual structure adaptively adjusts the ratio between deep semantic enhancement information and current-layer spatial detail information based on different points and their local context. When the deep semantic context is more helpful for classifying the current point, the model tends to enhance the semantic information. The model retains more details when the spatial details of the current layer are more critical for boundary and local structure recovery. This allows for the collection of detailed information, thereby enhancing semantic consistency while preventing the shallow geometric structure from being overly covered.

[0079] In summary, the hierarchical semantic guidance refinement process can be summarized as follows: As can be seen from the above process, the hierarchical semantic guidance and refinement module does not simply splice and fuse features from different levels. Instead, based on the basic decoded features, it achieves step-by-step guidance of deep semantics to shallow spatial details through local cross-layer semantic aggregation and gated residual fusion. This module can alleviate the problem of insufficient semantic transmission in conventional decoding structures, enabling shallow features to obtain stronger category consistency constraints while maintaining boundary details and spatial structure, thereby improving the segmentation integrity of small-scale targets, low-frequency categories, and complex boundary regions.

[0080] S4. During the training phase, a joint optimization of semantic segmentation main loss and boundary constraint loss is adopted. Boundary labels are automatically generated by point-by-point semantic annotation, eliminating the need for additional manual boundary annotation. To simultaneously ensure point-level semantic classification accuracy and the ability to discriminate category boundary regions, this invention employs a joint optimization objective composed of semantic segmentation loss and boundary constraint loss. The semantic segmentation loss is used to supervise the network's semantic category prediction for each point, while the boundary constraint loss guides the model to focus on the boundary regions between different semantic categories, thereby alleviating common problems in indoor point cloud scenarios such as boundary ambiguity, category sticking, and local misclassification. This is because when relying solely on semantic segmentation loss, the model is easily dominated by large-area categories, with relatively insufficient attention paid to category boundaries and the edges of small-scale components. To enhance the responsiveness of the final segmentation features to semantic boundaries, this invention sets a boundary discrimination branch after the final decoded feature D^1 and constrains it through explicit boundary supervision.

[0081] The overall loss function is defined as follows: in, This is the main loss for semantic segmentation, used to constrain the consistency between the network output and the pointwise semantic labels; Boundary constraint loss is used to enhance the model's ability to identify boundary regions; These are the boundary loss weights, used to adjust the relative contributions between the main task supervision and the auxiliary task supervision. The semantic segmentation main loss uses pointwise cross-entropy loss. Let the input point cloud have a total of... There are points, and the number of categories is . , No. The true category label for each point is: The model predicts that it belongs to the first The probability of class is Then the semantic segmentation loss is defined as: in, For indicator functions, This represents the natural logarithm function.

[0082] Semantic segmentation main loss The loss function supervises the network from the perspective of overall category discrimination, enabling the model to learn the distribution patterns of different semantic categories in the feature space.

[0083] To construct boundary supervision signals, this invention generates boundary labels online based on real semantic labels. For points... If its local neighborhood If a point exists that has a semantic label different from the center point, then that point is defined as a boundary point; otherwise, it is defined as a non-boundary point. Boundary label It can be represented as: in, Point Located in the category boundary area, This indicates that it is a non-boundary point. The boundary label is automatically generated by point-by-point semantic annotation, without the need for additional manual boundary annotation. Invalid labeled points are ignored in both boundary label construction and loss calculation to avoid invalid supervision interfering with model training.

[0084] Boundary prediction branch output Boundary prediction values ​​for each point Its boundary probability for: Based on boundary truth values and predicted probability The boundary constraint loss is expressed in the form of binary cross-entropy: .

[0085] In practical training, a binary cross-entropy loss with sigmoid can be used to numerically stabilize the boundary branches. This loss explicitly distinguishes between boundary points and non-boundary points, guiding the network to enhance its response to class boundary regions during training, thereby improving problems such as discontinuous boundary contours, blurred local structures, and missed classification of small-scale components.

[0086] Ultimately, this invention supervises the network from two levels—global semantic discrimination and local boundary constraints—by jointly optimizing the semantic segmentation main loss and boundary constraint loss. This allows the model to maintain overall classification performance while paying more attention to fine-grained structures and complex boundary regions in indoor point clouds.

[0087] S5. The semantic category of each point is directly output during the inference phase.

[0088] like Figure 2 As shown, this invention also provides a fine-grained scene parsing system for indoor 3D point clouds, including a feature embedding module, a PointNeXt encoder, a hierarchical semantic refinement decoder, and a joint optimization module for semantic segmentation main loss and boundary constraint loss. The feature embedding module maps the input point cloud into high-dimensional point features. The PointNeXt encoder extracts hierarchical features of different resolutions through multi-stage downsampling and local neighborhood aggregation. The hierarchical semantic refinement decoder starts with deep high-semantic features and progressively backtracks along the decoding path to mid-to-low-level feature representations, recovering the fine-grained spatial structure through cross-layer semantic aggregation and gated residual fusion. The joint optimization module for semantic segmentation main loss and boundary constraint loss automatically generates boundary labels from point-by-point semantic annotations and directly outputs the semantic category of each point.

[0089] Experimental verification:

[0090] This invention uses the ScanNet v2 dataset for experimental verification. Data preprocessing steps include point cloud transformation of the 3D mesh of the indoor scene, extraction of spatial coordinates and color features of points, normalization of coordinate and color information, and data augmentation through random rotation, random scaling, coordinate perturbation, and color perturbation to improve the model's generalization ability in complex indoor scenes. ScanNet v2, released by Dai et al. in 2017, is a large-scale real-world indoor scene RGB-D dataset primarily used for tasks such as 3D point cloud semantic segmentation, instance segmentation, 3D object detection, and indoor scene understanding. ScanNet v2 further refines and improves the semantic annotation results based on the original dataset and is one of the commonly used benchmark datasets in the field of indoor point cloud semantic segmentation. The ScanNet v2 dataset was acquired using a handheld RGB-D camera and contains approximately 2.5 million RGB-D images and 1513 labeled indoor scene scans, covering 707 different indoor spaces. The scenes in the dataset mainly include real-world indoor environments such as residences, offices, conference rooms, classrooms, and restrooms. Each scene provides RGB-D images, camera intrinsic and extrinsic parameters, camera pose, 3D surface reconstruction models, texture information, instance annotations, and semantic annotations. Following the official dataset partitioning, 1201 scenes are used for model training, and 312 scenes are used for model validation. The ScanNet v2 benchmark platform also provides 100 test scenes without publicly available real labels to evaluate the final performance of the model. Data samples from the ScanNet v2 dataset are shown below. Figure 7 As shown.

[0091] Specifically, the ScanNet v2 dataset is organized around independent scenes as the basic unit. Each scene is numbered in the format "sceneXXXX_YY", where "XXXX" represents the indoor space number and "YY" represents different scan sequences corresponding to the same space. In the semantic segmentation task, this invention uses 20 semantic categories officially defined by ScanNet v2 for model training and performance evaluation, specifically including walls, floors, cabinets, beds, chairs, sofas, tables, doors, windows, bookshelves, photos, counters, desks, curtains, refrigerators, shower curtains, toilets, sinks, bathtubs, and others.

[0092] Table 1 shows the overall test accuracy of PointCNN, PointNet++, KPConv, PT, and the method of this invention on the ScanNet v2 dataset. The method of this invention achieves the best results in all three metrics: OA, mIoU, and mAcc, reaching 89.16%, 70.83%, and 78.80%, respectively. Compared to PointCNN, the method of this invention improves OA, mIoU, and mAcc by 12.04, 26.06, and 24.74 percentage points, respectively; compared to PointNet++, by 8.93, 16.49, and 16.52 percentage points, respectively; compared to KPConv, by 4.07, 8.12, and 4.46 percentage points, respectively; and compared to PT, by 1.66, 3.38, and 1.63 percentage points, respectively. The stable improvement of the method of this invention in these two metrics indicates that it not only improves the overall point-level classification accuracy but also effectively enhances the balanced recognition ability for low-frequency categories, small-scale targets, and complex boundary regions.

[0093] Table 1 Comparison of mIoU, mAcc, and allAcc results: PontCNN 77.12 44.77 54.06 PointNet++ 80.23 54.34 62.28 KPConv 85.09 62.71 74.34 PT 87.5 67.45 77.17 Ours 89.16 70.83 78.80

[0094] Table 2 shows the IoU and Acc test results of PointCNN, PointNet++, KPConv, PT, and our invention on various semantic categories in the ScanNet v2 dataset. Bold text indicates the best value within the same category, and underlined text indicates the second-best value. As shown in the table, regarding the IoU metric, our invention achieves the best results in 16 categories (bed, chair, sofa, table, door, bookshelf, photo, counter, desk, curtain, refrigerator, shower curtain, toilet, sink, bathtub, and other furniture) and the second-best results in 4 categories (wall, floor, cabinet, and window). Regarding the Acc metric, our invention achieves the best results in 10 categories (chair, sofa, table, door, photo, desk, refrigerator, shower curtain, toilet, and sink) and the second-best results in 9 categories (wall, cabinet, bed, window, bookshelf, counter, curtain, bathtub, and other furniture). Notably, our invention achieves the second-best result in the Acc metric for the bookshelf category, tied with KPConv. This demonstrates that the present invention not only maintains stable recognition capabilities for large-area indoor structural categories such as walls and floors, but also exhibits significant advantages in categories with complex boundaries, small scale, or low frequency, such as doors, windows, counters, photos, refrigerators, shower curtains, and sinks. Particularly in more difficult-to-recognize categories like photos, refrigerators, and shower curtains, the IoU (Intersection over Union) of the present invention reaches 36.71%, 65.05%, and 59.60%, respectively, showing a significant improvement compared to other comparative methods. This indicates that the present invention can effectively address issues such as occlusion, missing point clouds, sensor noise, and uneven distribution of object categories present in real-world ScanNet v2 scanning scenarios. The reason why the present invention achieves superior semantic segmentation performance compared to PointCNN, PointNet++, KPConv, and PT is mainly due to its simultaneous enhancement of local boundary discrimination capabilities and hierarchical semantic refinement capabilities. On the one hand, the boundary-aware local feature enhancement module, by jointly modeling the feature differences, relative coordinate differences, and geometric distances between the center point and its neighbors, enables the model to more accurately perceive the local geometric changes of furniture outlines, door and window edges, and the boundaries of adjacent objects, thereby reducing boundary adhesion and erroneous predictions between different categories. On the other hand, the hierarchical semantic guidance refinement module progressively passes deep semantic information to mid- and low-level features, providing clear category semantic guidance for shallow features with high spatial resolution, and improving the segmentation integrity of small-scale objects, low-frequency categories, and incomplete scanned regions. Therefore, this invention achieves optimal or near-optimal results on most categories of the ScanNet v2 dataset, demonstrating a more balanced, accurate, and robust semantic segmentation capability in complex real-world indoor scenes.

[0095] Table 2 Comparison of IoU and Acc results for various categories in Scannet v2:

[0096] like Figure 8 As shown, taking categories with a point percentage of less than 1% as examples, photos account for only 0.52% of the points, and the IoU and Acc of the method of this invention reach 36.71% and 45.92% respectively, an improvement of 8.10 and 11.82 percentage points compared to PT; counters account for 0.51% of the points, and the IoU of the method of this invention reaches 68.74%, an improvement of 13.31 and 10.91 percentage points compared to KPConv and PT respectively; refrigerators account for 0.55% of the points, and the IoU and Acc of the method of this invention... The IoU and Acc rates for the present invention reached 65.05% and 70.24% respectively, representing improvements of 7.55 and 6.28 percentage points compared to PT. For the shower curtain, which accounted for only 0.33% of the points, the IoU and Acc rates of the present invention reached 59.60% and 64.22% respectively, improvements of 8.74 and 6.36 percentage points compared to KPConv. For the toilet, which accounted for only 0.33% of the points, the IoU and Acc rates of the present invention reached 93.51% and 96.54% respectively, improvements of 2.56 and 2.19 percentage points compared to PT. For the sink, which accounted for 0.27% of the points, the IoU of the present invention reached 64.04%, an improvement of 8.36 percentage points compared to PT. These improvements indicate that the present invention can better preserve target boundaries and local structures in complex adjacency regions, resulting in more complete semantic regions for small-scale or local targets in the segmentation results.

[0097] Figure 9This paper presents multi-dimensional information from three randomly selected room scenes (scene0011_00, scene0203_00, and scene0256_00) in the ScanNet v2 standard test set, including RGB point cloud images, manually annotated ground truth class labels, and a visual comparison of the semantic information recognition results of four methods. As shown in the figures, in scene0011_00, compared to other methods, the method of this invention provides more complete segmentation results for the areas of tables, chairs, doors, and cabinets (black rectangles), effectively reducing class adhesion between furniture and walls / floors. However, the segmentation performance for chairs far from the desk and tables in the distance (red rectangles) is poor. In scene0203_00, the sofa, tabletop, and curtains (blue rectangles) have complex shapes and are adjacent to each other. Other methods are prone to local fragmentation or class confusion, while the method of this invention can better maintain the continuity of the target contours, making the boundaries between the sofa and surrounding areas, and between the curtains and walls, clearer. In scene0256_00, the segmentation results of the method of this invention in the adjacent areas of the desktop, window, and wall (pink rectangle) are closer to the ground truth, with relatively fewer local misclassifications, demonstrating better class consistency. Overall, the method of this invention exhibits good segmentation integrity and boundary preservation capabilities in all three scenes, effectively alleviating common problems in complex indoor scenes such as boundary ambiguity, class adhesion, and missed classification of small-scale targets. This is mainly due to the explicit modeling of local geometric abrupt changes and class boundary regions by the boundary awareness enhancement module, and the effective fusion of deep semantic information and shallow spatial details by the hierarchical semantic guidance refinement module. Figure 10 , Figure 11 In the two detailed scenarios, the black-framed areas indicate that the segmentation effect is better than the other four models.

Claims

1. A fine-grained scene parsing method for indoor 3D point clouds, characterized in that, The method includes the following steps: S1. The input point cloud undergoes initial mapping through a feature embedding layer, converting the original coordinates and attribute information into a high-dimensional point feature representation; S2. PointNeXt is used as the basic backbone network, and hierarchical features from shallow to deep are obtained through multi-stage sampling, neighborhood query, and local aggregation. and corresponding coordinate set L represents the total number of levels in the encoding stage. A boundary-aware local feature enhancement module is introduced in the encoding stage. It generates local boundary responses by using the feature differences, relative coordinate differences and geometric distances between the center point and neighboring points, and uses a gating fusion mechanism to enhance the features of the category boundary region. S3. In the decoding stage, a hierarchical semantic guidance and refinement module is introduced. Starting from deep high semantic features, it is backed up to the mid-to-low-level feature representations along the decoding path. The fine-grained spatial structure is restored through cross-layer semantic aggregation and gated residual fusion. S4. During the training phase, the semantic segmentation main loss and boundary constraint loss are jointly optimized. The boundary labels are automatically generated by point-by-point semantic annotation, eliminating the need for additional manual boundary annotation. At the same time, a joint optimization objective consisting of semantic segmentation loss and boundary constraint loss is used to supervise the network from two levels: global semantic discrimination and local boundary constraint. S5. The semantic category of each point is directly output during the inference phase.

2. The fine-grained scene parsing method for indoor 3D point clouds according to claim 1, characterized in that, The specific method for step S1 is as follows: Given a given original indoor point cloud First, the point cloud is preprocessed by coordinate normalization, block sampling, or voxel sampling to adapt it to the network input scale. Then, the feature embedding layer maps the 3D spatial coordinates and initial attribute features to initial point features. ;in Indicates the first The three-dimensional spatial coordinates of the points Indicates the first Initial attribute features of each point Enter the number of points.

3. The fine-grained scene parsing method for indoor 3D point clouds according to claim 1, characterized in that, The specific method for step S2 is as follows: In the shallow and mid-level feature stages of the PointNeXt encoder, a boundary-aware local feature enhancement module is set up. Local boundary descriptors are constructed using feature differences, relative coordinate differences, and geometric distances between the center point and its neighbors. The boundary response is then adaptively injected into the original features through a gated fusion mechanism. Specifically: Let the first The first in the layer The features and coordinates of each point are denoted as follows: and , , No. The first in the layer The local neighborhood set of a point is denoted as . First, construct the local boundary descriptor. : in, This indicates a feature concatenation operation. Describing the Euclidean norm, and indicates They represent the first The first in the layer Neighboring points of each point Features and coordinates; After obtaining the local boundary descriptor Then, a lightweight mapping function is used to perform a nonlinear transformation on the neighborhood difference information, and max pooling is performed along the neighborhood dimension to obtain the point-level boundary response. : in, This represents the boundary mapping function composed of a multilayer sensing mechanism. This indicates max pooling along the neighborhood dimension, used to highlight the most significant boundary response patterns; Subsequently, the boundary response features are gated and fused with the original hierarchical features to obtain the boundary-enhanced feature representation: in, and These are learnable mapping parameters; This represents the Sigmoid activation function; This represents element-wise multiplication; Indicates the first The first of the levels The boundary gating weights of each point are used to adaptively control the degree to which the boundary response features enhance the original point features; Indicates the first The first of the levels The output features of each point after boundary awareness enhancement are the enhanced features after gating fusion.

4. The fine-grained scene parsing method for indoor 3D point clouds according to claim 1, characterized in that, The specific method for step S3 is as follows: S3. In the decoding stage, a hierarchical semantic guidance and refinement module is introduced. Starting from deep high semantic features, it is backed up to the mid-to-low-level feature representations along the decoding path. The fine-grained spatial structure is restored through cross-layer semantic aggregation and gated residual fusion. A hierarchical semantic guidance and refinement module is set up in the decoding stage. This module starts from the deep semantic features at the end of the encoder and backtracks to the middle and shallow features level by level along the decoding stage. Through cross-layer semantic aggregation, high-level semantics guide the low-level spatial details, thereby continuously narrowing the semantic gap between different levels of features while restoring spatial resolution. This enhances the model's ability to recover small-scale objects, local boundaries, and fine-grained components. Specifically: Let the first The encoded features of the layer after boundary enhancement are ,in These represent different decoding levels from shallow to deep, while the deepest encoded feature at the end of the encoder is denoted as... For the deepest features at the end of the encoder Since it lacks deeper semantic references, an initial semantic projection function is first used. Map it to a deep initial semantic representation : in, This represents the deepest encoded feature at the end of the encoder; This represents the deepest encoded feature acting at the end of the encoder. The initial semantic projection function; Subsequently, for shallower levels, the decoder first passes through the feature propagation unit. The current layer boundary enhancement features are fused with the deeper backpropagation semantic features to obtain the basic decoding features at the current scale: in, Indicates an upsampling operation. Indicates the first The feature representation of a layer after being recovered by the base decoder, i.e., the base decoded features at the current scale. This represents deeper semantic refinement features in the backhaul; the current-scale basic decoding features. It also contains local geometric details at the current scale, boundary enhancement information, and upsampled semantic information from deeper layers, serving as the main input for subsequent hierarchical semantic guidance refinement; After obtaining the basic decoding features at the current scale, the hierarchical semantic guidance refinement module uses... As the main input, it is used to refine the hierarchical semantic features through deeper feedback. As a reference input, the refinement function is guided by hierarchical semantics. Get the output of the current layer: in, Indicates the first Layer-level semantic refinement features; Hierarchical semantic guidance refinement function It consists of two parts: local cross-layer semantic aggregation and gated residual fusion. First, it decodes the basic features at the current scale. Perform bottleneck mapping to obtain the semantic representation of the current layer. : in, Indicates by The bottleneck mapping function consists of convolutional layers, normalization layers, and nonlinear activation functions; Subsequently, the semantic representation of the current layer will be... As query features, more refined semantic features will be returned at a deeper level. As supporting features, queries, key-value mappings, and other functions are performed separately: in, This represents the query mapping parameters, used to represent the semantics of the current layer. Mapped to query features ; This represents the key mapping parameters, used to refine features to a deeper level of semantic detail. Mapping to key features ; The value mapping parameters are used to refine features to a deeper level of semantic detail. Mapping to value features , , and These represent the current layer query features, deeper layer key features, and value features, respectively. Next, a neighborhood-based local attention mechanism is introduced into cross-layer semantic aggregation, for coordinates... The The first in the layer Each point, in a deeper point set Select its local neighborhood And combining relative position coding to model cross-layer geometric relationships, the first A point and its neighboring points The relative positions between them are represented as follows: in, Indicates the first Layer The point and the first Layer Neighborhood Points The relative positions of the layers between them Indicates the first Neighborhood points in the layer 3D coordinates; Cross-layer attention weights are based on query, key features, and relative position encoding. Represented as: in, and These represent the relative position encoding functions applied to the query branch and the relative position encoding functions applied to the key branch, respectively. For feature dimension, Indicates the first Layer Query characteristics of each point Indicates the first Layer Neighborhood Points The key feature is that the superscript T indicates the transpose operation; Based on cross-layer attention weights This yields the current layer context features guided by deep semantics. : in, Indicating the first under the guidance of deep semantics Layer The current layer context features of each point Indicates the first Layer The point is the first Layer Neighborhood Points Cross-layer attention weights, Indicates by the first Hierarchical semantic refinement features Value mapping parameters The mapping obtained, the first Layer Neighborhood Points Value characteristics; To simplify the representation, by and The results were obtained through querying, key-value mapping, cross-layer attention weight calculation, and weighted summation. The process is written as follows: in, This represents a cross-layer semantic aggregation operation based on local neighborhood relationships. This indicates the first [item] obtained under the guidance of deep semantics. Layer-to-layer contextual features; In obtaining the first Layer-to-layer context features Subsequently, to avoid deep semantic information overly covering shallow geometric details, a gated residual fusion mechanism is further adopted to adaptively balance the contributions between the deep semantic context and the current layer's basic decoding features. First, the current layer's basic decoding features are mapped to a channel space consistent with the context features: in, For learnable linear mapping parameters, For the first The current layer's details are preserved. Subsequently, gating weights are generated based on cross-layer context features and current layer residual information. : in, Indicates the output mapping function, Represents the gated mapping function. Use the Sigmoid activation function; Finally, the first The refined output of the layer is represented as: In summary, the hierarchical semantic guidance refinement process can be summarized as follows: 。 5. The fine-grained scene parsing method for indoor 3D point clouds according to claim 1, characterized in that, The specific method for step S4 is as follows: During the training phase, semantic segmentation main loss and boundary constraint loss are jointly optimized, and the overall loss function is... The definition is as follows: in, This is the main loss for semantic segmentation, used to constrain the consistency between the network output and the pointwise semantic labels; Boundary constraint loss is used to enhance the model's ability to identify boundary regions; The boundary loss weight coefficient is used to adjust the relative contribution between the main task supervision and the auxiliary task supervision. Semantic segmentation main loss Using pointwise cross-entropy loss, let the input point cloud have a total of There are points, and the number of categories is . , No. The true category label for each point is: The model predicts that it belongs to the first The probability of class is Then the semantic segmentation main loss Defined as: in, For indicator functions, Represent the natural logarithm function; Semantic segmentation main loss The loss function supervises the network from the perspective of overall category discrimination, enabling the model to learn the distribution patterns of different semantic categories in the feature space; To construct boundary supervision signals, boundary labels are generated online based on real semantic labels. Specifically, for points... If its local neighborhood If a point exists that has a semantic label different from the center point, then that point is defined as a boundary point; otherwise, it is defined as a non-boundary point, and the boundary label... Represented as: in, Point Located in the category boundary area, This indicates that it is a non-boundary point; the boundary label is automatically generated by point-by-point semantic annotation, without the need for additional manual boundary annotation. Invalid labeled points are ignored in the process of boundary label construction and loss calculation. Boundary prediction branch output Boundary prediction values ​​for each point Its boundary probability for: Based on boundary truth values and predicted probability The boundary constraint loss is expressed in the form of binary cross-entropy: 。 6. A fine-grained scene parsing system for indoor 3D point clouds, characterized in that, The system is used to run the method described in any one of claims 1-5, and the system includes a feature embedding module, a PointNeXt encoder, a hierarchical semantic refinement decoder, and a joint optimization module for semantic segmentation main loss and boundary constraint loss; The feature embedding module is used to map the input point cloud into high-dimensional point features; the PointNeXt encoder is used to extract hierarchical features of different resolutions through multi-stage downsampling and local neighborhood aggregation. The hierarchical semantic refinement decoder is used to start with deep high-semantic features and backtrack to mid-to-low-level feature representations along the decoding path. It restores the fine-grained spatial structure through cross-layer semantic aggregation and gated residual fusion. The semantic segmentation main loss and boundary constraint loss joint optimization module is used to automatically generate boundary labels from point-by-point semantic annotations and directly output the semantic category of each point.

Images