A sample automatic repairing method and system for indoor complex scenes
Patent Information
- Application Number
- CN202610871665.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-16
- Publication Date
- 2026-10-23
AI Technical Summary
[0004]而在采用上述点云修复算法时，通常需要海量的标注样本进行训练，但室内场景点云数据的精细化标注极其耗时耗力（如管道节点、法兰盘等结构的标注需专业知识和人工干预），导致数据获取成本高，进而在点云修复阶段，对于全局几何结构复杂的交叉节点，传统算法难以从有限数据的局部可见点云推断背面结构和不同场景的结构语义，导致修复结果失真和缺乏结构合理性
本发明通过通过融合BIM语义信息与多模态扫描数据，并基于上述数据，利用全局几何推理对密度差异大于阈值的室内点云进行重建来得到初步完整点云，以及通过初步完整点云得到的语义类别定义管道连接约束，从而进行点云的修复，既可在无需大量标注的情况下即可建立先验结构约束，能够在缺乏完整标注样本的条件下，基于室内分区后的独立房间作为局部修复条件，实现了从有限可见点云中推断复杂交叉节点的背面几何与结构语义，避免了因标注不足导致的结构失真和修复结果缺乏合理性的问题，从而在降低数据获取成本的同时，显著提升了修复点云的几何完整性与结构合理性。
Smart Images
Figure CN122966475A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of point cloud restoration technology, and in particular to an automatic sample restoration method and system for complex indoor scenes. Background Technology
[0002] In the process of 3D reconstruction of various indoor scenes, large pipeline systems in environments such as computer rooms and factories pose a great challenge to point cloud data acquisition and processing due to their dense layout, complex topology and multiple occlusions.
[0003] Existing technologies include point cloud inpainting algorithms based on deep learning methods for processing 3D reconstructions, such as PointNet++ and 3D-GAN. These solutions attempt to address the issues of missing and incomplete point cloud models in various indoor scenes through automated learning. Their core idea is to utilize large-scale labeled datasets for supervised training to learn the global geometric features and distribution patterns of point clouds, thereby achieving local inpainting. For example, PointNet++ processes point cloud data through hierarchical feature extraction, while 3D-GAN generates missing point clouds using generative adversarial networks. Under ideal conditions (i.e., when data is sufficient), these methods can achieve high inpainting accuracy, such as a chamfer distance (CD error) as low as 1.38 × 10⁻³ and a normal consistency of 92.5%.
[0004] When using the aforementioned point cloud restoration algorithm, a massive amount of labeled samples are usually required for training. However, the fine-grained labeling of point cloud data in indoor scenes is extremely time-consuming and labor-intensive (the labeling of structures such as pipe nodes and flanges requires professional knowledge and manual intervention), resulting in high data acquisition costs. Consequently, in the point cloud restoration stage, for intersection nodes with complex global geometric structures, traditional algorithms struggle to infer the back structure and structural semantics of different scenes from the locally visible point cloud of limited data, leading to distorted restoration results and a lack of structural rationality. Summary of the Invention
[0005] The purpose of this invention is to address the shortcomings of the prior art by providing an automatic sample repair method and system for complex indoor scenarios, thereby solving the problems in the prior art.
[0006] This invention specifically provides the following technical solution: an automatic sample repair method for complex indoor scenes, comprising the following steps: Collect point cloud data of various indoor scenes, and obtain BIM semantic information, multi-view scan point cloud and structured light scan data related to various indoor scenes; Based on point cloud data, BIM semantic information, multi-view scanned point cloud and structured light scan data are integrated to construct a global geometric constraint map, and global geometric reasoning is performed on the occluded area based on the global geometric constraint map. For indoor point clouds with density differences exceeding a threshold after global geometric inference, spatial encoding is performed through curvature-sensing sampling. Multi-scale feature alignment is then performed on the spatially encoded indoor point clouds, and bounding box uniformity is used to adaptively reconstruct the aligned multi-scale features to obtain a preliminary complete point cloud. Semantic segmentation and attention feature propagation are performed on the preliminary complete point cloud to generate semantic category information of the point cloud. Based on the semantic category, pipeline connection constraints in the point cloud are defined and integrated into the point cloud repair process. Each space after indoor partitioning is used as a local repair condition to perform local repair of the preliminary complete point cloud. Global and local geometric features are extracted from the point cloud after local restoration, and dynamic feature fusion is performed on the extracted global and local geometric features to output the final restored point cloud.
[0007] Preferably, the process of fusing BIM semantic information, multi-view scan point cloud, and structured light scan data to construct a global geometric constraint map, and then performing global geometric reasoning on the occluded area based on the global geometric constraint map, specifically involves: Extract the parametric features of pipelines from the BIM model and construct a global geometric constraint graph (GCG) to provide non-global geometric parametric rules and topological constraints for generating point clouds. A reinforcement learning agent is introduced to optimize the point cloud registration process and obtain global point cloud data with optimal registration accuracy. The key occluded areas in the global point cloud data are scanned to obtain local point clouds, and the heterogeneous data are unified into a common feature space through a fusion network. The heterogeneous data includes parameterized rules and topological constraints of the non-global geometry of the point cloud, as well as global point cloud data and local point clouds with optimal registration accuracy.
[0008] Preferably, an improved POCO network is used to process indoor point clouds with density differences greater than a threshold to obtain a preliminary complete point cloud. The improved POCO network is based on the POCO network, replacing the farthest point sampling of the POCO network with curvature-aware sampling. Skip connections, attention gating and latent spatial alignment units are added to the POCO network encoder-decoder structure, and the scale adjustment strategy of the POCO network is improved to bounding box uniformity.
[0009] Preferably, the step of spatial encoding through curvature-sensing sampling and multi-scale feature alignment of the spatially encoded indoor point cloud specifically involves: Curvature-aware sampling is used to spatially encode indoor point clouds with density differences greater than a threshold, resulting in multi-scale features after spatial encoding. The specific expression for the sampling probability function during spatial encoding is as follows: P sampling =λ ·k +(1- l )· d ; in k The curvature value, d The reciprocal of the point density, l As a balance factor, P sampling This is a probability value; Based on an encoder-decoder structure that incorporates skip connections, attention gating, and latent spatial alignment units, deformable convolution processing is applied to spatially encoded multi-scale features to obtain aligned multi-scale features.
[0010] Preferably, the step of performing semantic segmentation and attention feature propagation on the preliminary complete point cloud to generate semantic category information of the point cloud specifically involves: A KPConv network is used to perform semantic segmentation on the point cloud to obtain semantic category information for pipes, flanges, and valves. Attention feature propagation processing is then applied to this semantic category information to obtain enhanced semantic category information for the segmentation results. The specific expression for attention feature propagation processing is as follows: AFP F =SoftMax(( W q F ) T ( W k F ))·( W v F ); in, W q , W k , W v The projection matrix is used, and AFP is the result of attention feature propagation processing, which performs neighborhood feature aggregation. F This is a dynamic coordinate characteristic function.
[0011] Preferably, the step of defining pipe connection constraints in the point cloud based on semantic categories, integrating pipe connection constraints into the point cloud repair process, and using each space after indoor partitioning as a local repair condition to perform preliminary local repair of the complete point cloud, specifically: Pipeline connection constraints are defined based on semantic category information. Specifically, the pipeline connection constraints are as follows: when connecting straight pipes, the axes are kept collinear; the flange is in contact with the pipe end face; and the pipe diameter parameters at both ends of the valve are consistent. The pipe connection constraints are incorporated into the repair process using an energy function, specifically expressed as follows: E semantic =Σ f( r i )· d (violate) ri )); in f ( r i ) represents the rule weight. r i For the first i Semantic rules, d To violate the indicator function, `violate` is a Boolean condition. E semantic It is an energy function; During the restoration process, the preliminary complete point cloud is projected onto a two-dimensional plane, and spatial partitioning is performed using morphological operations. Each space is restored independently, ultimately yielding the local restoration result of the preliminary complete point cloud.
[0012] Preferably, the step of extracting global and local geometric features from the locally repaired point cloud, and then dynamically fusing the extracted global and local features to output the final repaired point cloud, specifically involves: A dual-path generative adversarial network is used to extract global and local geometric features from the point cloud after local restoration. The generator architecture of the dual-path generative adversarial network includes a structural branch and a detail branch. The structural branch uses a multi-layer MLP model to process global geometric features, while the detail branch uses a graph convolutional network to extract local features. When performing dynamic feature fusion of extracted global and local geometric features in a dual-path generative adversarial network, a dynamic feature fusion module (DFF) is used for feature fusion. The specific expression is as follows: F out = α · F global +(1- α )· F local ; α = s (MLP([ F global ; F local ])); MLP stands for Multilayer Perceptron Model; α The dynamic weighting coefficients are between 0 and 1. s It is the Sigmoid activation function. F globa These are global geometric features extracted from branches of the generator structure. F localThese are local features extracted from the generator's detail branch. F out This refers to the fusion features output by the DFF module.
[0013] This invention provides an automatic sample repair system for complex indoor scenarios, comprising: The data acquisition module is used to collect point cloud data of various indoor scenes and obtain BIM semantic information, multi-view scan point cloud and structured light scan data related to various indoor scenes. The global geometry reasoning module is used to construct a global geometric constraint map based on point cloud data, by integrating BIM semantic information, multi-view scanned point cloud and structured light scan data, and to perform global geometric reasoning on occluded areas based on the global geometric constraint map. The initial point cloud acquisition module is used to spatially encode indoor point clouds with density differences greater than a threshold after global geometric inference by using curvature-sensing sampling. The spatially encoded indoor point clouds are then aligned with multi-scale features, and bounding box uniformity is used to adaptively reconstruct the aligned multi-scale features to obtain a preliminary complete point cloud. The local repair module is used to perform semantic segmentation and attention feature propagation on the preliminary complete point cloud, generate semantic category information of the point cloud, define pipeline connection constraints in the point cloud based on semantic categories, integrate pipeline connection constraints into the point cloud repair process, and use each space after indoor partitioning as a local repair condition to perform local repair of the preliminary complete point cloud. The output module is used to extract global and local geometric features from the point cloud after local repair, and to perform dynamic feature fusion on the extracted global and local geometric features to output the final repaired point cloud.
[0014] The present invention provides a computer device, including a memory and a processor. The memory stores a program, and when the program is executed by the processor, the processor performs the steps of the above-described automatic sample repair method for complex indoor scenarios.
[0015] The present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described automatic sample repair method for complex indoor scenarios.
[0016] Compared with the prior art, the present invention has the following significant advantages: This invention repairs point clouds by fusing BIM semantic information with multimodal scanning data and using global geometric reasoning to reconstruct indoor point clouds with density differences exceeding a threshold. It obtains a preliminary complete point cloud and defines pipeline connection constraints based on semantic categories derived from the preliminary complete point cloud. This allows for the establishment of prior structural constraints without extensive annotation. Furthermore, it enables the inference of the back geometry and structural semantics of complex intersection nodes from a limited visible point cloud, even in the absence of complete labeled samples, using independent rooms after indoor partitioning as local repair conditions. This avoids structural distortion and lack of reasonableness in repair results caused by insufficient annotation, thus significantly improving the geometric integrity and structural rationality of the repaired point cloud while reducing data acquisition costs. Attached Figure Description
[0017] Figure 1 This is a flowchart provided in an embodiment of the present invention; Figure 2 This is a diagram of the encoder-decoder structure provided in an embodiment of the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0019] This invention introduces a meta-learning feature adaptation module, significantly reducing sample requirements; it employs multi-source data fusion and semantic guidance mechanisms to enhance global geometric reasoning and semantic rationality. While the shortcomings of existing technologies are not absolute, they make efficient and high-precision repair difficult in various indoor scenarios, and this invention addresses these pain points. In environments with significant occlusion and complex pipe systems, existing technologies mainly face the following technical deficiencies:
[0020] The reliance on large amounts of data and the high cost of annotation: Current mainstream deep learning methods (such as repair algorithms based on PointNet++ and 3D-GAN) heavily rely on large-scale, high-quality labeled data for training. For example, traditional supervised learning typically requires more than 500 labeled samples to achieve high accuracy (CD error of approximately 1.38 × 10⁻³, normal consistency of 92.5%), while detailed annotation of indoor scene point clouds (such as pipe nodes, flanges, and other structures) requires professional knowledge and manual intervention, which is extremely costly and difficult to obtain. Although transfer learning can reduce the sample requirement (to approximately 100 samples), its performance drops significantly (CD error increases to 2.90 × 10⁻³, normal consistency drops to 87.3%). The root cause is that existing models lack cross-domain generalization ability and cannot quickly adapt to new scenarios from a small number of samples.
[0021] (1) Insufficient global geometric reasoning ability leads to repair distortion: Existing methods mostly rely on a single data source (such as single-view laser scanning point cloud), making it difficult to effectively reason about the global geometric structure of occluded areas (such as the back of pipes or intersection nodes). Traditional rule-based interpolation or pure learning methods (such as triangular mesh hole repair algorithms) are prone to structural discontinuities or distortions when the point cloud density is uneven or occlusion is severe. The fundamental reason is the lack of multi-source data fusion mechanisms and prior knowledge guidance, which makes it impossible to fully utilize the parametric constraints in the BIM model (such as pipe diameter and connection rules) to supplement missing information.
[0022] (2) Lack of semantic understanding leads to unreasonable repair results: Most existing solutions ignore scene semantic information (such as pipe type and connection rules) and only focus on global geometric feature matching, which may result in repair results that violate engineering specifications (such as pipe diameter mismatch or incorrect connection angle). For example, although repair methods based on GAN or convolutional neural networks can generate complete point clouds, they cannot guarantee structural rationality. This is because the model does not incorporate domain knowledge (such as BIM rule base) and lacks a semantic-global geometry joint optimization mechanism.
[0023] (3) Low processing efficiency and poor real-time performance: Traditional methods (such as triangular mesh-based repair algorithms) are computationally inefficient when the amount of point cloud data is large (it requires the construction of dense meshes, which takes more than 48 hours), while the batch processing mode based on deep learning is difficult to meet the needs of real-time interaction (such as online updates of BIM models). The main reason is that the algorithm does not optimize computational redundancy and lacks dynamic optimization strategies (such as reinforcement learning-guided real-time generation).
[0024] This invention aims to address the aforementioned technical deficiencies and provide an automatic point cloud model repair algorithm for various indoor scenarios with small sample sizes. Specific objectives include: (1) achieving high-precision repair under small sample conditions, significantly reducing the need for labeled data (achieving a CD error of 1.95×10⁻³ and a normal consistency of 91.2%); (2) enhancing global geometric reasoning capabilities through multi-source data fusion and semantic guidance, ensuring that the repair results conform to both global geometric continuity and engineering rationality; and (3) introducing a real-time optimization mechanism to improve processing efficiency, supporting millisecond-level repair responses, and meeting the real-time application needs of smart cities, BIM management, and other fields. The core of this invention lies in constructing a multi-stage collaborative point cloud repair framework, achieving high-precision reconstruction of occluded areas through the organic combination of multiple innovative technical modules. This framework particularly optimizes learning efficiency under small sample conditions, achieving excellent performance under limited real data conditions.
[0025] Reference Figure 1 This invention provides an automatic sample repair method for complex indoor scenarios, comprising: Step S1: Collect point cloud data of various indoor scenes, especially complex scenes, and obtain BIM semantic information, multi-view scan point cloud and structured light scan data related to various indoor scenes.
[0026] To address the scarcity of labeled data for indoor scenes, this solution employs a meta-learning strategy to construct a feature transfer mechanism. This module comprises a two-layer optimized structure: a basic task learning layer and a scene adaptation layer.
[0027] The foundational task learning layer is pre-trained on large synthetic datasets such as ShapeNet, enabling the model to master general global geometric feature representation capabilities. An improved POCO network is used as the basic architecture, whose multi-scale feature extraction capabilities can effectively capture the global geometric characteristics of structures such as pipes and joints. The pre-training stage focuses on learning key global geometric features such as curvature changes and connection topology, laying the foundation for subsequent transfer learning.
[0028] Scene Adaptation Layer: A Model-Independent Meta-Learning (MAML) algorithm is introduced to achieve rapid scene adaptation. By constructing a Support Set and a Query Set, the pre-trained model is fine-tuned on a small number of real-world scene samples (e.g., 10-15 labeled pipeline nodes). A comparison between this invention and traditional algorithms is shown in Table 1. The key innovation lies in the design of a globally geometry-sensitive loss function:
[0029] L meta = α · L chamfer + β ·L normal + c · L curvature in, L chamfer To ensure overall shape consistency, L normal Constrained surface continuity, L curvature Preserving curvature characteristics. This combination of loss functions significantly improves the model's generalization ability under sparse sample conditions.
[0030] Table 1 Performance Comparison of Small Sample Training Strategies traditional supervision learning 500+ 1.38 92.5 48+ basic transfer learning 100 2.90 87.3 24 The method of the invention 15 1.95 91.2 5.5 Step S2: Based on point cloud data, integrate BIM semantic information, multi-view scanned point cloud and structured light scan data to construct a global geometric constraint map, and perform global geometric reasoning on the occluded area based on the global geometric constraint map.
[0031] To overcome the limitations of a single scanning perspective, a multi-source data fusion mechanism was constructed to enhance global geometric reasoning capabilities through the complementarity of heterogeneous data. BIM Semantic Guidance: Extract parametric features (pipe diameter, connection type, routing rules) from the BIM model to construct a Global Geometric Constraint Graph (GCG). GCG nodes represent pipe components, and edges represent connection relationships and spatial constraints. During the repair process, the GCG provides topological constraints for the generated point cloud, ensuring that the repair results conform to engineering specifications. For example, for tee joints, the GCG constrains the diameter ratio and connection angle of each branch pipe to avoid generating unreasonable structures.
[0032] Multi-view scanning fusion: An improved ICP-RL algorithm is used to register multi-site cloud data. A reinforcement learning agent (RL-Agent) is innovatively introduced to optimize the point cloud registration process, and the registration accuracy is improved by guiding the application through a reward function.
[0033] Reward = 1 / (1+CD ( P transformed , P target )); This strategy significantly improves the registration success rate in areas with low overlap (by approximately 37% compared to traditional ICP).
[0034] Structured light-assisted enhancement: For critical occluded areas, a low-cost structured light scanner is deployed to acquire local high-precision point clouds. Heterogeneous data are unified into a common feature space through a Feature Gating Fusion Network.
[0035] F fused = s ( W ·[ F lidar ; F structured,light ])⊙( V ·[ F lidar ; F structured,light ]); in s For the sigmoid function, W , V The parameters are learnable, and ⊙ represents element-wise multiplication. This network can adaptively weight feature contributions from different data sources.
[0036] Step S3: For indoor point clouds with density differences greater than a threshold after global geometric inference, spatial encoding is performed through curvature-aware sampling. Multi-scale feature alignment is then performed on the spatially encoded indoor point clouds, and bounding box uniformity is used to adaptively reconstruct the aligned multi-scale features to obtain a preliminary complete point cloud.
[0037] For indoor point clouds with large density differences, this solution adopts an improved POCO (Point Completion Network) architecture to achieve high-fidelity reconstruction: Spatial coding optimization: The traditional farthest point sampling (FPS) is improved to curvature-aware sampling (CAS). The sampling probability function is defined as:
[0038] P sampling = l · k + (1- l ) · d ; Where κ is the curvature value (calculated through eigenvalue decomposition): k = l min / ( l 1+ l 2+ l 3)), d The reciprocal of the point density, l As a balance factor, P samplingThis represents a probability value. This strategy retains more points in feature-rich regions, significantly improving the quality of detail reconstruction at pipe connections.
[0039] Multi-scale feature fusion: such as Figure 2 As shown, skip connections and attention gating are added to the encoder-decoder structure. The key innovation is the introduction of the Latent Space Alignment Unit (LSAU), which aligns multi-scale features through deformable convolutions to solve the feature misalignment problem caused by density differences.
[0040] Adaptive Reconstruction Strategy: The scaling strategy of the original POCO model is improved by adopting bounding box uniformity. By unifying the bounding box scale between training and testing data, the hole problem caused by uneven density is avoided. Experiments show that this improvement increases the reconstruction completeness of the back side of the pipe by approximately 42%.
[0041] Step S4: Perform semantic segmentation and attention feature propagation on the preliminary complete point cloud to generate semantic category information of the point cloud. Define pipeline connection constraints in the point cloud based on semantic categories, integrate pipeline connection constraints into the point cloud repair process, and use each space after indoor partitioning as a local repair condition to perform local repair of the preliminary complete point cloud.
[0042] Structural rationality repair is achieved through joint semantic-global geometric optimization: Semantic segmentation guidance: The KPConv network is used for semantic segmentation of point clouds (categories such as pipes, flanges, and valves). The segmentation results are enhanced through Attention Feature Propagation (AFP) mechanism.
[0043] AFP F = SoftMax (( W q F ) T ( W k F )) · ( W v F ) in W q , W k , W v The projection matrix is used, and AFP is the result of attention feature propagation processing. Effective neighborhood feature aggregation is performed to improve the recognition accuracy of small parts. F This is a dynamic coordinate characteristic function.
[0044] Semantic rule base construction: Define pipeline connection constraints based on semantic category information. Specifically, the pipeline connection constraints are: keep the axes collinear when connecting straight pipes; fit the flange with the pipe end face; and ensure that the pipe diameter parameters at both ends of the valve are consistent.
[0045] These rules are incorporated into the repair process in the form of energy functions, that is, the pipe connection constraints are incorporated into the repair process in the form of energy functions, specifically as follows: E semantic = Σ f ( r i )· d (violate) r i )); in f ( r i ) represents the rule weight. r i For the first i Semantic rules, d To violate the indicator function, `violate` is a Boolean condition. E semantic It is the energy function.
[0046] Room-level semantic map: In this spatial partitioning example, room partitioning is specifically implemented to generate a room semantic map to guide local repair. This is achieved by projecting point clouds onto a two-dimensional plane and combining morphological operations (opening and closing operations) to generate precise room partitions. Each room is repaired independently to ensure local structural consistency.
[0047] Step S5: Extract global and local geometric features from the locally repaired point cloud, and perform dynamic feature fusion on the extracted global and local geometric features to enhance high-frequency features, and output the final repaired point cloud.
[0048] To achieve high-frequency detail enhancement in point cloud restoration, this solution designs a dual-path generative adversarial network: Generator architecture: It adopts a dual-branch structure: structural branch: 3-layer MLP to process global geometric features; detail branch: graph convolutional network (GCN) to extract local subtle features.
[0049] The dual-branch outputs are integrated through the Dynamic Feature Fusion (DFF) module: F out = α · F global + (1- α ) ·F local ; α = s (MLP ([ F global ; F local ])); MLP stands for Multilayer Perceptron Model; α is a dynamic weighting coefficient between 0 and 1, used to balance the contribution ratio of global features and local features in the final output. Its value is dynamically calculated by the next formula. s Using the Sigmoid activation function, the output of the MLP is mapped to the (0,1) interval to obtain effective weights. α ; F globa The global geometric features are extracted from the generator structure branch (3-layer MLP). F local These are subtle local features extracted from the generator detail branch (Graph Convolutional Network GCN). F out This refers to the fusion features output by the DFF module.
[0050] Discriminator Innovation: A multi-scale discriminator (MSD) is constructed, comprising three sub-discriminators: a point-level discriminator (detecting the authenticity of local point distribution), a block-level discriminator (verifying local surface continuity), and a global discriminator (evaluating the overall structural rationality). Through multi-scale collaborative supervision, the generation quality of details such as flange threads and welds is effectively improved.
[0051] Reinforcement learning optimization: A reinforcement learning agent (RL-Agent) is introduced to search for the optimal latent vector. The agent is guided by a reward function to determine the generation direction, specifically:
[0052] Reward = l 1·(1-CD)+ l 2· Semantic Acc - l 3· Time Cost ; in, Reward For the reward function value, l 1. l 2. l 3 represents three hyperparameters, and CD is the chamfer distance. Semantic Acc For semantic accuracy, Time CostThis reduces time costs. The mechanism shortens repair time to milliseconds, meeting real-time requirements.
[0053] This invention addresses the three core challenges of point cloud repair in various indoor scenarios (big data dependence, insufficient global geometric reasoning, and semantic gaps). By integrating innovative technologies such as meta-learning, multi-source data fusion, and semantic guidance, it achieves the following technical effects: (1) High-precision repair and strong generalization ability: Small sample size and high accuracy: A CD error of 1.95×10⁻³ and normal consistency of 91.2% can be achieved with only a small number of labeled samples, significantly outperforming traditional supervised learning (requires more than 500+ samples, CD error 1.38×10⁻³) and basic transfer learning (requires more than 100+ samples, CD error 2.90×10⁻³). Cross-scene adaptability: Through the meta-learning feature adaptation module, the model can maintain stability in unknown scenarios (such as data centers with different pipeline layouts), with the correction error fluctuation range controlled within ±0.2×10⁻³.
[0054] (2) Complex structure repair and detail restoration: Global geometric rationality: The multi-source data fusion module (BIM + point cloud + structured light) improves the repair accuracy of pipe connections and enhances the completeness of the back-side reconstruction of complex structures such as flanges and valves. Detail fidelity: The generative adversarial optimization module enhances high-frequency details (such as threads and welds) through a multi-scale discriminator (point / block / global), improving detail reproduction compared to traditional methods.
[0055] (3) High-efficiency processing and real-time performance: Millisecond-level repair: Reinforcement learning agents (RL-Agents) optimize latent vector search, achieving a repair time of ≤50ms (RTX 3090) for single-room scenarios (2 million points), meeting real-time interaction requirements (such as online BIM updates). Resource efficiency: Training time is only 5.5 hours (traditional methods require 48+ hours), reducing computational resource consumption by 60%.
[0056] (4) Semantic rationality guarantee: Rule-based constraint repair: Semantic rule base (such as pipe diameter consistency, flange fit) ensures that the repair results conform to engineering specifications, improving semantic accuracy by 28%. Local consistency: Room-level semantic graphs ensure the structural coherence of zone repairs and avoid global topology errors.
[0057] Comparison with the closest existing technology and its beneficial effects: The closest existing technology is the point cloud repair method based on deep learning (such as PointNet++ and 3D-GAN), which learns global features through large-scale labeled data. However, it has the following limitations and the corresponding improvements of this invention are shown in Table 2.
[0058] Table 2 Comparison between the prior art and the present invention This invention specifically discloses the following: a meta-learning adaptation strategy: a two-layer optimization framework (basic task pre-training + scene fine-tuning) and a global geometry-sensitive loss function ( L curvature The core content is to ensure generalization ability under small sample sizes. Multi-source fusion network: Feature gating fusion formula ( F fused = s ( W ·[ F lidar ; F structured,light ])⊙( V ·[ F lidar ; F structured,light This feature can adaptively weight the feature contributions from different data sources, avoiding heterogeneous data conflicts. Semantic rule base: pipeline connection rules (such as collinearity constraints) and energy functions (…). E semantic This is an internal enterprise knowledge base and requires authorized access.
[0059] The present invention provides the following specific economic benefits: Cost reduction: Labeling costs are reduced by 97% (from 500 samples to 15 samples), and manpower input is reduced by 80%. Efficiency improvement: The BIM model update cycle is shortened from 2 weeks to 8 hours, and project delivery speed is increased by 5 times. Application expansion: It can cover high-value scenarios such as cultural relic restoration and smart cities, with an expected increase in benefits of 2 million yuan per project per year.
[0060] This invention proposes an automatic sample repair system for complex indoor scenarios, comprising: The data acquisition module is used to collect point cloud data of various indoor scenes and obtain BIM semantic information, multi-view scanned point clouds, and structured light scan data related to these scenes. The global geometric inference module is used to construct a global geometric constraint map based on the point cloud data, fusing BIM semantic information, multi-view scanned point clouds, and structured light scan data, and to perform global geometric inference on occluded areas based on this map. The initial point cloud acquisition module is used to spatially encode indoor point clouds with density differences exceeding a threshold after global geometric inference through curvature-sensing sampling, perform multi-scale feature alignment on the spatially encoded indoor point clouds, and utilize package... Box unification adaptively reconstructs aligned multi-scale features to obtain a preliminary complete point cloud. The local repair module performs semantic segmentation and attention feature propagation on the preliminary complete point cloud to generate semantic category information of the point cloud. Based on the semantic category, it defines pipeline connection constraints in the point cloud and integrates these constraints into the point cloud repair process. Each space after indoor partitioning is used as a local repair condition to perform local repair of the preliminary complete point cloud. The output module extracts global and local geometric features from the locally repaired point cloud and performs dynamic feature fusion on the extracted global and local features to output the final repaired point cloud.
[0061] The present invention also provides a computer device, including a memory and a processor. The memory stores a program, and when the program is executed by the processor, the processor performs the steps of an automatic sample repair method for complex indoor scenarios.
[0062] According to the disclosed embodiments, the computer device can communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth communication, etc.) or with any device that enables the computing device to communicate with one or more other computing devices (e.g., router, demodulator, etc.).
[0063] The present invention also provides a storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps of an automatic sample repair method for complex indoor scenarios.
[0064] According to the disclosed embodiments, the storage medium can be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, the storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0065] The above description, in conjunction with specific preferred embodiments, provides a more detailed explanation of the present invention. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered to fall within the scope of protection of the present invention.
Claims
1. An automatic sample repair method for complex indoor scenarios, characterized in that, Including the following steps: Collect point cloud data of various indoor scenes, and obtain BIM semantic information, multi-view scan point cloud and structured light scan data related to various indoor scenes; Based on point cloud data, BIM semantic information, multi-view scanned point cloud and structured light scan data are integrated to construct a global geometric constraint map, and global geometric reasoning is performed on the occluded area based on the global geometric constraint map. For indoor point clouds with density differences exceeding a threshold after global geometric inference, spatial encoding is performed through curvature-sensing sampling. Multi-scale feature alignment is then performed on the spatially encoded indoor point clouds, and bounding box uniformity is used to adaptively reconstruct the aligned multi-scale features to obtain a preliminary complete point cloud. Semantic segmentation and attention feature propagation are performed on the preliminary complete point cloud to generate semantic category information of the point cloud. Based on the semantic category, pipeline connection constraints in the point cloud are defined and integrated into the point cloud repair process. Each space after indoor partitioning is used as a local repair condition to perform local repair of the preliminary complete point cloud. Global and local geometric features are extracted from the point cloud after local restoration, and dynamic feature fusion is performed on the extracted global and local geometric features to output the final restored point cloud.
2. The automatic sample repair method for complex indoor scenarios as described in claim 1, characterized in that, The process involves fusing BIM semantic information, multi-view scan point clouds, and structured light scan data to construct a global geometric constraint map. Based on this map, global geometric reasoning is then performed on occluded areas. Specifically: Extract the parametric features of pipelines from the BIM model and construct a global geometric constraint graph (GCG) to provide non-global geometric parametric rules and topological constraints for generating point clouds. A reinforcement learning agent is introduced to optimize the point cloud registration process and obtain global point cloud data with optimal registration accuracy. The key occluded areas in the global point cloud data are scanned to obtain local point clouds, and the heterogeneous data are unified into a common feature space through a fusion network. The heterogeneous data includes parameterized rules and topological constraints of the non-global geometry of the point cloud, as well as global point cloud data and local point clouds with optimal registration accuracy.
3. The automatic sample repair method for complex indoor scenarios as described in claim 1, characterized in that, An improved POCO network is used to process indoor point clouds with density differences greater than a threshold to obtain a preliminary complete point cloud. The improved POCO network is based on the POCO network, replacing the farthest point sampling of the POCO network with curvature-aware sampling. Skip connections, attention gating and latent spatial alignment units are added to the POCO network encoder-decoder structure. The scale adjustment strategy of the POCO network is improved to bounding box uniformity.
4. The automatic sample repair method for complex indoor scenarios as described in claim 3, characterized in that, The process of spatial encoding via curvature-sensing sampling, followed by multi-scale feature alignment of the spatially encoded indoor point cloud, specifically involves: Curvature-sensing sampling was used to spatially encode indoor point clouds with density differences greater than a threshold, resulting in multi-scale features after spatial encoding. The specific expression for the sampling probability function during spatial encoding is as follows: P sampling =λ ·κ +(1- λ ) d ; in κ The curvature value, d The reciprocal of the point density, λ As a balance factor, P sampling This is a probability value; Based on an encoder-decoder structure that incorporates skip connections, attention gating, and latent spatial alignment units, deformable convolution processing is applied to spatially encoded multi-scale features to obtain aligned multi-scale features.
5. The automatic sample repair method for complex indoor scenes as described in claim 1, wherein the semantic segmentation and attention feature propagation processing of the preliminary complete point cloud to generate semantic category information of the point cloud specifically comprises: A KPConv network is used to perform semantic segmentation on the point cloud to obtain semantic category information for pipes, flanges, and valves. Attention feature propagation processing is then applied to this semantic category information to obtain enhanced semantic category information for the segmentation results. The specific expression for attention feature propagation processing is as follows: AFP( F )=SoftMax(( W q F ) T ( W k F ))·( W v F ); in, W q , W k , W v The projection matrix is used, and AFP is the result of attention feature propagation processing, which performs neighborhood feature aggregation. F This is a dynamic coordinate characteristic function.
6. The automatic sample repair method for complex indoor scenes as described in claim 5, wherein the step of defining pipeline connection constraints in the point cloud based on semantic categories, integrating the pipeline connection constraints into the point cloud repair process, and using each space after indoor partitioning as a local repair condition to perform preliminary local repair of the complete point cloud, specifically includes: Pipeline connection constraints are defined based on semantic category information. Specifically, the pipeline connection constraints are as follows: when connecting straight pipes, the axes are kept collinear; the flange is in contact with the pipe end face; and the pipe diameter parameters at both ends of the valve are consistent. The pipe connection constraints are incorporated into the repair process using an energy function, specifically expressed as follows: E semantic =Σ φ ( r i )· δ (violate( r i )); in φ ( r i ) represents the rule weight. r i For the first i Semantic rules, δ To violate the indicator function, `violate` is a Boolean condition. E semantic It is an energy function; During the restoration process, the preliminary complete point cloud is projected onto a two-dimensional plane, and spatial partitioning is performed using morphological operations. Each space is restored independently, ultimately yielding the local restoration result of the preliminary complete point cloud.
7. The automatic sample repair method for complex indoor scenes as described in claim 1, wherein the step of extracting global and local geometric features from the locally repaired point cloud, and dynamically fusing the extracted global and local geometric features to output the final repaired point cloud, specifically comprises: A dual-path generative adversarial network is used to extract global and local geometric features from the point cloud after local restoration. The generator architecture of the dual-path generative adversarial network includes a structural branch and a detail branch. The structural branch uses a multi-layer MLP model to process global geometric features, while the detail branch uses a graph convolutional network to extract local features. When performing dynamic feature fusion of extracted global and local geometric features in a dual-path generative adversarial network, a dynamic feature fusion module (DFF) is used for feature fusion. The specific expression is as follows: F out = α · F global +(1- α )· F local ； α = σ (MLP([ F global ; F local ])); in, MLP stands for Multilayer Perceptron Model; α The dynamic weighting coefficients are between 0 and 1. σ It is the Sigmoid activation function. F globa These are global geometric features extracted from branches of the generator structure. F local These are local features extracted from the generator's detail branch. F out This refers to the fusion features output by the DFF module.
8. An automatic sample repair system for complex indoor scenarios, characterized in that, include: The data acquisition module is used to collect point cloud data of various indoor scenes and obtain BIM semantic information, multi-view scan point cloud and structured light scan data related to various indoor scenes. The global geometry reasoning module is used to construct a global geometric constraint map based on point cloud data, by integrating BIM semantic information, multi-view scanned point cloud and structured light scan data, and to perform global geometric reasoning on occluded areas based on the global geometric constraint map. The initial point cloud acquisition module is used to spatially encode indoor point clouds with density differences greater than a threshold after global geometric inference by using curvature-sensing sampling. The spatially encoded indoor point clouds are then aligned with multi-scale features, and bounding box uniformity is used to adaptively reconstruct the aligned multi-scale features to obtain a preliminary complete point cloud. The local repair module is used to perform semantic segmentation and attention feature propagation on the preliminary complete point cloud, generate semantic category information of the point cloud, define pipeline connection constraints in the point cloud based on semantic categories, integrate pipeline connection constraints into the point cloud repair process, and use each space after indoor partitioning as a local repair condition to perform local repair of the preliminary complete point cloud. The output module is used to extract global and local geometric features from the point cloud after local repair, and to perform dynamic feature fusion on the extracted global and local geometric features to output the final repaired point cloud.
9. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores a program that, when executed by the processor, causes the processor to perform the steps of the automatic sample repair method for complex indoor scenarios as described in any one of claims 1 to 7.
10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the automatic sample repair method for complex indoor scenarios as described in any one of claims 1 to 7.