3D modeling repair method based on point cloud deep learning incomplete model

Abstract

The application discloses a 3D modeling repair method based on a point cloud deep learning incomplete model, and relates to the technical field of 3D modeling repair.The specific steps of the method are as follows: firstly, pre-processing and normalization are performed on cross-source heterogeneous incomplete point clouds collected by multiple sensors to unify the input format;then, Bayesian confidence probability modeling and topology propagation are performed to optimize the confidence;then, the high-uncertainty area is actively calibrated;finally, the calibrated confidence is used as the weight to fuse features and complete the incomplete point cloud by using a cross-source Bayesian weighted fusion repair algorithm to generate a complete 3D model.The application uniformly formats the point cloud data of multiple sensors by pre-processing and normalization, accurately calculates and optimizes the confidence, and reduces the noise error;then, the calibrated confidence is used as the weighted fusion basis to accurately fuse geometric features and supplement missing data, effectively restores the complete form of the incomplete model, improves the accuracy and integrity, and adapts to various 3D modeling repair requirements.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling and restoration technology, specifically to a method for 3D modeling and restoration of incomplete models based on point cloud deep learning. Background Technology

[0002] In numerous fields such as industrial manufacturing, cultural relic preservation, and reverse engineering, 3D modeling and restoration of incomplete models is a core technical requirement. In various scenarios, due to factors such as equipment acquisition limitations, damage to the object itself, and external environmental interference, the acquired point cloud data often suffers from incompleteness and heterogeneity, making it difficult to directly use it for constructing a complete 3D model. To solve this problem, the industry has gradually adopted a multi-sensor collaborative acquisition approach to obtain cross-source point cloud data. By integrating the acquisition advantages of different sensors, the limitations of a single sensor in data acquisition can be compensated for. With the development of deep learning technology, point cloud deep learning methods have been widely applied in the field of 3D modeling and restoration, becoming an important technical means to process incomplete point cloud data and achieve accurate model restoration. Therefore, the fusion and completion of cross-source point clouds has become a research focus of point cloud deep learning in the direction of 3D modeling and restoration.

[0003] Traditional 3D modeling and restoration methods for incomplete models lack targeted preprocessing and normalization design when dealing with cross-source heterogeneous incomplete point cloud data. This makes it difficult to effectively unify data collected by different sensors, resulting in poor compatibility of cross-source point clouds. At the same time, traditional methods do not accurately quantify the reliability of point cloud data. In the feature fusion process, they often adopt an equalization fusion method, which cannot distinguish the true structural representation ability of each point in the point cloud data. This makes them susceptible to interference factors such as noise and outliers, making it difficult for the fused features to accurately reflect the true structure of the model. In addition, traditional restoration methods lack effective calibration means for high-uncertainty regions with low confidence. The feature supplementation of missing regions during the completion process is not accurate enough. Ultimately, this leads to insufficient accuracy and completeness of 3D modeling and restoration, making it difficult to meet the actual needs of high-precision modeling and restoration. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for 3D modeling and repairing incomplete models based on point cloud deep learning. Addressing the challenge of repairing incomplete point cloud data from different sources and heterogeneous structures, this invention designs a complete technical process from preprocessing to final completion. First, point clouds collected from multiple sensors undergo normalization preprocessing. Then, Bayesian confidence probability modeling and topological propagation quantization are used to optimize the reliability of each point in the point cloud. Secondary confidence correction is achieved through active calibration in high-uncertainty regions. Finally, cross-source feature weighted fusion is implemented using the calibrated confidence as weight, and complementary feature injection is combined to complete the 3D completion of the incomplete point cloud. This method effectively solves the problems of poor compatibility, low data reliability, and insufficient feature fusion in cross-source point clouds, significantly improving the accuracy and completeness of 3D modeling and repair, and providing an efficient solution for the digital repair of incomplete models in various fields.

[0005] To solve the above-mentioned technical problems, this invention provides the following technical solution: a 3D modeling and repair method for incomplete models based on point cloud deep learning, the method comprising: S1, Cross-source point cloud preprocessing normalization: Used to preprocess multi-channel cross-source heterogeneous residual point cloud data collected by multiple types of sensors. Through coordinate system unification, resolution and point number normalization, multi-scale geometric feature extraction and feature dimension unification, the point cloud data of each channel has a unified input format. S2, Bayesian confidence probability modeling: For each point cloud after normalization preprocessing, the Bayesian probability confidence modeling algorithm is used. Combining the local geometric features of the points with cross-source topological association, the probability distribution parameters of each point belonging to the real structure are calculated, and the initial Bayesian confidence result containing the point-level confidence probability distribution is output. S3, Bayesian confidence topology propagation: Based on the initial Bayesian confidence result, a Bayesian graph neural network adapted to the spatial topology of the point cloud is constructed. Through the confidence propagation algorithm in the graph neural network, the confidence information is iteratively updated and propagated between adjacent points, and the topology-optimized Bayesian confidence result is output. S4, Active calibration of high uncertainty regions: Based on the Bayes confidence results after topology optimization, identify high uncertainty regions in the point cloud with confidence scores below a set threshold, trigger an active perception optimization strategy for the identified high uncertainty regions, and perform secondary correction and calibration of the confidence scores of the high uncertainty regions. S5, Cross-source feature fusion point cloud completion: Using the calibrated Bayesian confidence as the basis for weighted fusion, the cross-source Bayesian weighted fusion repair algorithm is applied. The geometric features of different point clouds are weighted and fused with complementary features using the calibrated Bayesian confidence as the weight. Based on the fused global features, the incomplete point cloud is completed in three dimensions to generate a complete 3D model.

[0006] Furthermore, in the cross-source point cloud preprocessing normalization step, various types of sensors, including ground-based lidar, handheld structured light scanners, industrial CT scanners, and UAV multi-view reconstruction equipment, are used. During the input preprocessing, invalid point filtering and outlier removal operations are performed sequentially on the multi-channel cross-source heterogeneous residual point cloud data. Specifically, invalid point filtering removes point data whose coordinate values ​​exceed a preset three-dimensional spatial range. Outlier removal uses a statistical filtering method to calculate the average distance from each point to its k nearest neighbors and removes point data whose average distance exceeds a preset distance threshold. Then, coordinate system unification, resolution and point number normalization, multi-scale geometric feature extraction, and feature dimension unification are performed sequentially.

[0007] Furthermore, in the coordinate system operation, a coarse registration operation is performed first, followed by a fine registration operation. The coarse registration operation specifically involves extracting local geometric features of each point cloud, calculating the similarity matrix between features of different point clouds, selecting the top 10% of feature pairs based on the similarity matrix, solving the initial spatial transformation matrix using the selected feature pairs, and completing the preliminary spatial alignment of each point cloud based on the initial spatial transformation matrix. The fine registration operation specifically involves calculating the normal vector of each point in the point cloud after coarse registration, using the Euclidean distance from the point to the corresponding tangent plane as a constraint, solving the optimal spatial transformation matrix through multiple rounds of iterative calculation, and completing the coordinate system one of each point cloud based on the optimal spatial transformation matrix.

[0008] Furthermore, the mathematical expression for the Yeats probability confidence modeling algorithm is:

[0009] in, For the midpoint of the point cloud The initial Bayesian confidence level, The balance coefficient is a value ranging from 0.3 to 0.7. For point The confidence level mean term, For point The confidence level variance term, For point The cross-source topological association term; the input dimensions of the confidence mean term and the confidence variance term include points. Local density, point Spatial deviation of points corresponding to other waypoint clouds, points The local curvature stability, the inherent noise level of the corresponding sensor, and the point The probability of occlusion at the scanning viewpoint, and the cross-source topological association terms through points The topological connectivity with neighboring points is calculated; in the Bayesian probability confidence modeling process, the probability distribution parameters of a point belonging to the true structure are calculated by combining the local geometric features of each point with the cross-source topological association. The local geometric features include the point's local density, local curvature stability, and normal vector distribution characteristics. Local density is obtained by statistically analyzing the ratio of the number of points within a certain neighborhood of the point to the volume of the neighborhood space. Local curvature stability is obtained by calculating the variance of the curvature difference between the point and its neighboring points. Normal vector distribution characteristics are obtained by statistically analyzing the consistency of the normal vector direction of all points within the point's neighborhood. The cross-source topological association... The connection is obtained by comparing the spatial coordinate deviation and topological connectivity of the point with the corresponding points in other waypoint clouds. First, the corresponding matching point of the point in other waypoint clouds is determined. Then, the spatial distance and normal vector deviation between the point and the matching point are calculated. At the same time, the similarity of the neighborhood topology of the point and the matching point are combined. By integrating the above local geometric features and various parameters of cross-source topological association, feature mapping and fusion are performed through a multi-layer fully connected network. Finally, the probability distribution parameters of each point belonging to the real structure are calculated. The probability distribution parameters include the mean parameter that represents the reliability of the point and the variance parameter that represents the stability of the confidence of the point.

[0010] Furthermore, the Bayesian graph neural network adapted to the spatial topology of point clouds constructs an undirected graph structure by treating each point in the point cloud as an independent graph node and the k-nearest neighbor topological relationships of the point cloud as graph edges, where k ranges from 8 to 32. The Bayesian graph neural network is configured with 3 to 6 Bayesian graph convolutional layers, each layer containing a graph attention calculation unit, a Bayesian probability update unit, and a neighborhood information aggregation unit. The graph attention calculation unit calculates the propagation weights between adjacent nodes, the Bayesian probability update unit iteratively updates the node probability parameters based on the initial Bayesian confidence results, and the neighborhood information aggregation unit aggregates and transmits the confidence information of adjacent nodes. The output of the Bayesian graph neural network is configured with a dimension mapping layer, whose output dimension is consistent with the dimension of the initial Bayesian confidence results.

[0011] Furthermore, the mathematical expression for the confidence propagation algorithm is:

[0012] in, The point after the kth iteration Bayesian confidence level For point The set of neighboring points, adjacent points Relative to point The propagation weight, The adjacent points after the (k-1)th iteration Bayesian confidence level The adaptive propagation coefficient has a value ranging from 0.1 to 0.5. For point The confidence correction increment; the iteration termination condition of the confidence propagation algorithm is that the change in Bayesian confidence of all points is less than 0.001 after two consecutive iterations, or the number of iterations reaches the preset upper limit of 50. The propagation weight is obtained through the output of the graph attention calculation unit of the Bayesian graph neural network, and the confidence correction increment is obtained through the points. The initial Bayesian confidence score is calculated as the difference between the initial Bayesian confidence score and the Bayesian confidence score after the previous iteration. During the confidence score propagation process, relying on the network structure of the graph neural network, the initial Bayesian confidence score result is used as the propagation starting data. The neighboring points determined in the point cloud spatial topology are used as the propagation path. The confidence score information is iteratively updated and propagated round by round. During the propagation process, the association weight between each point and all its neighboring points is calculated first. The association weight is determined based on the spatial distance between neighboring points, the similarity of local geometric features, and the cross-source topological association degree. The closer the distance, the higher the feature similarity, and the stronger the topological association degree, the greater the association weight. Based on the calculated association weight, the confidence score information of each neighboring point is weighted and aggregated to obtain the confidence score update increment of that point. The update increment is fused with the current confidence score value of that point to complete the iterative update of the confidence score of that point. In each iteration, the confidence score of all points is synchronously updated and propagated until the preset iteration termination condition is met, and the iterative update and propagation operation is stopped. At this time, the confidence score result of all points output is the Bayesian confidence score result after topology optimization.

[0013] Furthermore, the set threshold is a fixed value predetermined through sensor calibration experiments, with a value range of 0.1 to 0.3. The high uncertainty region is a continuous region composed of all points whose confidence variance exceeds the set threshold. The active perception optimization strategy includes backtracking the original acquisition data of the corresponding sensor and initiating a manual annotation request. Backtracking the original acquisition data of the corresponding sensor specifically involves reading the original point cloud frame data corresponding to the high uncertainty region from the storage medium of the corresponding sensor, comparing the coordinate differences between the original point cloud frame data and the preprocessed point cloud data, and performing secondary correction and calibration of the confidence of the high uncertainty region based on the comparison results. Initiating a manual annotation request specifically involves generating an annotation task containing the coordinate range of the high uncertainty region when the confidence variance still exceeds the set threshold after backtracking the original acquisition data, pushing it to the annotation platform, and performing secondary correction and calibration of the confidence of the high uncertainty region based on the annotation results returned by the annotation platform.

[0014] Furthermore, the mathematical expression of the cross-source Bayesian weighted fusion repair algorithm is:

[0015] in, For the fused global features, For cross-source point clouds, The value range is from 2 to 8. For the s-th path, the midpoint of the cloud The calibrated Bayesian confidence score For the s-th path, the midpoint of the cloud Geometric features, The characteristic balance coefficient is a value ranging from 0.2 to 0.6. For point The corresponding cross-source complementary features; during the weighted fusion process, the geometric features of each point in each path point cloud are multiplied element-wise with the corresponding calibrated Bayesian confidence, and then the multiplication results of all path point clouds are added element-wise to obtain the initial fused features. The cross-source complementary features are then concatenated with the initial fused features to obtain the fused global features.

[0016] Furthermore, the complementary feature injection operation specifically includes: first, determining the overlapping area between different path point clouds through spatial coordinate comparison. The overlapping area is a three-dimensional spatial range in which point data exists in two or more path point clouds. Then, calculating the geometric differences between corresponding points in different path point clouds within the overlapping area. The calculation dimensions of geometric differences include coordinate deviation, normal vector deviation, and curvature deviation. Based on the geometric differences, identifying the structurally complementary area between cross-source point clouds. The structurally complementary area is a three-dimensional spatial range in which data is missing in one path point cloud and complete data exists in at least one path point cloud. Extracting the geometric features of the complete point cloud within the structurally complementary area as cross-source complementary features. Injecting the cross-source complementary features into the corresponding spatial position of the fused global features.

[0017] Beneficial effects Compared with existing technologies, this method for repairing incomplete 3D models based on point cloud deep learning has the following advantages: I. This invention achieves format unification of data collected by multiple sensors through cross-source point cloud preprocessing and normalization. Combining Bayesian confidence probability modeling and topology propagation, it accurately calculates and optimizes the confidence parameters of each point in the point cloud, making the real structural representation of the point cloud data more reliable. Relying on Bayesian graph neural networks to complete the iterative propagation of confidence, it can fully explore the spatial topological correlation information of the point cloud, making the confidence distribution fit the actual spatial structure of the point cloud. At the same time, through the active calibration strategy of high uncertainty region, it performs secondary correction on low confidence region, effectively reducing the influence of noise and error in point cloud data, and significantly improving the quality and effectiveness of point cloud data. This lays an accurate and reliable point cloud data foundation for subsequent incomplete model repair, and solves the problems of poor compatibility and insufficient data reliability of cross-source heterogeneous point cloud data.

[0018] II. This invention uses calibrated Bayesian confidence as the weighted fusion basis and employs a cross-source Bayesian weighted fusion repair algorithm to achieve accurate fusion of geometric features of point clouds from different sources. Combined with complementary feature injection, it accurately identifies and extracts structural complementary region features of cross-source point clouds, effectively supplementing missing data. This method allows the point cloud completion process to fully combine the advantageous data of each source point cloud, enabling the fused global features to comprehensively and accurately reflect the true three-dimensional structure of the model. Based on this, the completed 3D incomplete point cloud can effectively restore the complete form of the incomplete model, improve the accuracy and completeness of 3D modeling repair, and solve the technical problems of insufficient feature fusion and poor missing region completion effect in traditional repair methods. It is suitable for the 3D modeling repair needs of various incomplete models.

[0019] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0021] Figure 1 This is a flowchart of a 3D modeling and repair method for incomplete models based on point cloud deep learning; Figure 2 This is a schematic diagram illustrating the data transmission between each step of the 3D modeling and repair method for incomplete models based on point cloud deep learning. Detailed Implementation

[0022] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0023] Example 1: 3D modeling and repair of damaged industrial precision parts.

[0024] This embodiment focuses on 3D modeling and repair of missing aero-engine blades. The blade suffered localized damage due to manufacturing impact, requiring restoration of its complete structure to support subsequent inspection and reprocessing. Three types of sensors—ground-based LiDAR, industrial CT, and a handheld structured light scanner—were used to collect cross-source heterogeneous defect cloud data from three dimensions: the blade's outer contour, internal structure, and the damaged area. Based on the method of this invention, a complete 3D model of the blade was constructed, ensuring the repaired model accurately matches the original design specifications of the blade. Specific implementation steps are as follows: Figure 1 As shown: S1, Cross-source point cloud preprocessing and normalization: Preprocessing and normalization operations are performed on the three defect cloud data collected by ground-based lidar, industrial CT, and handheld structured light scanner. First, invalid point filtering is performed to remove point data whose coordinate values ​​exceed the preset three-dimensional spatial range of the blade. Removing invalid data reduces subsequent computational redundancy. Then, statistical filtering is used to remove outliers. The average distance from each point to its k nearest neighbors is calculated, and point data whose average distance exceeds the preset distance threshold is removed to eliminate abnormal point interference caused by equipment errors during the scanning process.

[0025] After noise removal, coordinate system unification is performed. First, coarse registration is done, extracting local geometric features from the three point clouds, calculating the similarity matrix between features, selecting the top 10% of feature pairs with the highest matching degree, and solving for the initial spatial transformation matrix. This completes preliminary spatial alignment, laying the foundation for subsequent precise registration. Then, fine registration is performed, calculating the normal vector of each point in the coarsely registered point cloud. Using the Euclidean distance from the point to the corresponding tangent plane as a constraint, the optimal spatial transformation matrix is ​​solved through multiple iterations, achieving coordinate system unification for the three point clouds and ensuring that point cloud data from different sensors are in the same spatial coordinate system. Subsequently, resolution and point count normalization, multi-scale geometric feature extraction, and feature dimension unification are performed sequentially to ensure a unified input format for the three point cloud data, guaranteeing the consistency and effectiveness of subsequent algorithm processing and preparing data for subsequent confidence modeling and topology propagation.

[0026] S2, Bayesian confidence probability modeling: For the three-way point cloud after normalization preprocessing, a Bayesian probability confidence modeling algorithm is used for modeling calculations. The algorithm combines the local geometric features of each point with cross-source topological associations to calculate the probability distribution parameters of the point belonging to the true blade structure. Local geometric features include local density, local curvature stability, and normal vector distribution characteristics. Local density is obtained by calculating the ratio of the number of points within a certain neighborhood of a statistical point to the volume of the neighborhood space, reflecting the local density of the point cloud. Local curvature stability is obtained by calculating the variance of the curvature difference between a point and its neighbors, reflecting the stability of the blade surface geometry. Normal vector distribution characteristics are obtained by the consistency of the normal vector direction of all points within the neighborhood of a statistical point, characterizing the regularity of the local geometric morphology. The mathematical expression of the Bayesian probability confidence modeling algorithm is:

[0027] in, For the midpoint of the point cloud The initial Bayesian confidence level, The balance coefficient is a value ranging from 0.3 to 0.7. For point The confidence level mean term, For point The confidence level variance term, For point The cross-source topology association term is obtained by comparing the spatial coordinate deviation and topological connection relationship between the point and the corresponding position points of the other two point clouds. First, the corresponding matching point of the point in the other point clouds is determined, and then the spatial distance and normal vector deviation between the point and the matching point are calculated. At the same time, the similarity of the neighborhood topology structure of the two can reflect the feature matching degree of different source point clouds at the same position.

[0028] By combining the above parameters, feature mapping and fusion are performed through a multi-layer fully connected network to fully integrate various features and improve the accuracy of parameter calculation. Finally, the probability distribution parameter of each point belonging to the true structure of the blade is calculated. This parameter includes the mean parameter representing the reliability of the point and the variance parameter representing the stability of the confidence of the point. The output includes the initial Bayesian confidence result containing the point-level confidence probability distribution, providing accurate initial confidence data for subsequent topology propagation and giving subsequent confidence optimization a clear starting basis.

[0029] S3, Bayesian confidence topology propagation: Based on the initial Bayesian confidence results, a Bayesian graph neural network adapted to the spatial topology of the point cloud is constructed. Each point in the point cloud is treated as an independent graph node, and the k-nearest neighbor topological relationship of the point cloud is used as the graph edge. The value of k is selected as 16 to construct an undirected graph structure, so that the network structure accurately fits the spatial distribution characteristics of the point cloud. This Bayesian graph neural network is set with 4 Bayesian graph convolutional layers. Each convolutional layer has a built-in graph attention calculation unit, Bayesian probability update unit, and neighborhood information aggregation unit. A dimension mapping layer is set at the output end, and its output dimension is consistent with the dimension of the initial Bayesian confidence results, ensuring the adaptability of the network output data to subsequent processing.

[0030] Based on this network, a confidence propagation algorithm is used to iteratively update and propagate confidence information between neighboring points. The mathematical expression of the confidence propagation algorithm is:

[0031] in, The point after the kth iteration Bayesian confidence level For point The set of neighboring points, adjacent points Relative to point The propagation weight, The adjacent points after the (k-1)th iteration Bayesian confidence level The adaptive propagation coefficient has a value ranging from 0.1 to 0.5. For point The confidence correction increment uses the initial Bayesian confidence result as the starting data for propagation and the neighboring points determined in the point cloud spatial topology as the propagation path. The association weight between each point and its neighboring points is calculated in each round. The association weight is determined based on the spatial distance between neighboring points, the similarity of local geometric features, and the cross-source topological association. The closer the distance, the higher the feature similarity, and the stronger the topological association, the greater the association weight, so that the confidence propagation is more in line with the spatial geometric features of the point cloud. Then, the confidence information of neighboring points is weighted and aggregated based on the association weight to obtain the confidence update increment of the point. The update increment is fused with the current confidence value of the point to complete the iterative update. All points carry out each round of update and propagation synchronously, so that the confidence information can be reasonably transmitted and optimized in the point cloud space.

[0032] The iteration stops when the change in Bayesian confidence score for all points is less than 0.001 after two consecutive iterations. The topology-optimized Bayesian confidence score is then output, making the confidence score data of the point cloud more closely match the real structural features of the blade and improving the spatial consistency and accuracy of the confidence score.

[0033] S4, Active calibration in high uncertainty regions: Based on the Bayesian confidence results after topology optimization, high uncertainty regions were identified and calibrated. In this implementation, the threshold was set to 0.2, which was predetermined by the sensor calibration experiment. A continuous region consisting of all points with confidence variance values ​​exceeding 0.2 was identified as a high uncertainty region. The edge of the blade defect and the internal micropore region of the industrial CT scan were identified as high uncertainty regions, and the low reliability regions in the point cloud were accurately located.

[0034] To optimize the active perception strategy for this region, the initial steps involved reviewing the original data acquired by ground-based LiDAR, industrial CT, and handheld structured light scanners. Original point cloud frames corresponding to high-uncertainty areas were retrieved from the storage media of each device. The coordinate differences between the original point cloud frames and the preprocessed point cloud data were compared. Based on the comparison results, the confidence level of this region underwent an initial secondary correction and calibration to restore the effective information in the original data and improve the accuracy of the confidence level. After the initial calibration, the confidence level variance of the micropore region inside the blade still exceeded 0.2. Therefore, a labeling task containing the coordinate range of this region was generated and pushed to a professional labeling platform. Industrial component modeling and labeling personnel completed the point cloud feature labeling for this region. Based on the labeling results returned by the platform, the confidence level of the micropore region inside the blade underwent a secondary correction and calibration. Professional manual labeling supplemented the data with precise feature information, making the confidence level data of high-uncertainty areas more closely match the actual structure. This completed the confidence level calibration work for all high-uncertainty areas, significantly improving the reliability and accuracy of the overall point cloud confidence level.

[0035] S5, Cross-source Feature Fusion Point Cloud Completion: Using the calibrated Bayesian confidence score as the basis for weighted fusion, a cross-source Bayesian weighted fusion repair algorithm is applied to process the three-way point clouds, completing the 3D incomplete point cloud. This allows the completed point cloud to fully integrate the advantageous features of different source data. The mathematical expression of the cross-source Bayesian weighted fusion repair algorithm is as follows:

[0036] in, For the fused global features, For cross-source point clouds, The value range is from 2 to 8. For the s-th path, the midpoint of the cloud The calibrated Bayesian confidence score For the s-th path, the midpoint of the cloud Geometric features, The characteristic balance coefficient is a value ranging from 0.2 to 0.6. For point The corresponding cross-source complementary features.

[0037] First, the geometric features of each point in each source point cloud are multiplied element-wise with the corresponding calibrated Bayesian confidence score, allowing high-confidence features to have appropriate weights in the fusion process. Then, the multiplication results of the three source point clouds are added element-wise to obtain the initial fusion feature, achieving preliminary fusion of features from different source point clouds. Simultaneously, complementary feature injection is performed. First, the overlapping area between the three source point clouds is determined by spatial coordinate comparison. Then, geometric differences such as coordinate deviation, normal vector deviation, and curvature deviation of corresponding points in different source point clouds within the overlapping area are calculated. Based on these geometric differences, structurally complementary regions between cross-source point clouds are identified. In the case of a missing part of the point cloud acquired by a handheld structured light scanner in the blade, some data is missing, while the outer contour point cloud acquired by a ground-based lidar has complete data in this area. This area is the structurally complementary region. The complementary position of the point cloud data is accurately located, and the complete geometric features of the ground-based lidar point cloud in this area are extracted as cross-source complementary features. These features are injected into the corresponding spatial position of the initial fusion feature to fill in the feature data in the missing area. Finally, the fusion is completed by combining the feature balancing coefficient, resulting in the fused global feature, ensuring both completeness and accuracy. Finally, based on the fused global features, the defect cloud of the aero-engine blade is three-dimensionally completed to generate a complete and accurate 3D model of the blade, restoring the original structure and details of the blade, meeting the modeling and repair needs of industrial precision parts, and providing accurate three-dimensional data support for the subsequent inspection and reprocessing of the blade.

[0038] In summary, this embodiment applies a 3D modeling and repair method for incomplete aero-engine blades based on point cloud deep learning to the repair of incomplete components. Multi-dimensional cross-source point clouds are collected using three types of sensors. Data standardization is achieved through preprocessing and normalization. An initial confidence level is obtained using a Bayesian probabilistic confidence modeling algorithm. Topology optimization is then performed using a Bayesian graph neural network and a confidence propagation algorithm. Active calibration is applied to high-uncertainty regions to improve data reliability. Finally, based on the calibrated confidence level, a cross-source Bayesian weighted fusion repair algorithm is used to complete feature fusion and point cloud completion. The resulting complete 3D model accurately reconstructs the blade structure, meeting the 3D data requirements for the inspection and reprocessing of industrial precision parts. This verifies the accuracy and practicality of the method in the repair of industrial precision parts.

[0039] Example 2: 3D modeling and restoration of incomplete ancient building stone carvings.

[0040] This embodiment focuses on the 3D modeling and restoration of damaged sections of a Ming and Qing dynasty stone archway. Due to weathering and impacts, the archway suffers from multiple decorative defects and structural damage. To achieve digital restoration and replication of the ancient building while preserving its original appearance and structural features, three types of sensors were used to collect heterogeneous point cloud data of the damaged sections: a ground-based LiDAR, a UAV multi-view reconstruction device, and a handheld structured light scanner. The ground-based LiDAR collected the overall structural point cloud of the archway, the UAV multi-view reconstruction device collected the point cloud of decorative elements at the top and upper parts of the archway, and the handheld structured light scanner collected the detailed point cloud of the damaged areas. Using the method of this invention, a 3D restoration model of the damaged archway was completed, allowing the restored 3D model to accurately restore the historical appearance and structural details of the archway. Specific implementation steps are as follows: Figure 2 As shown: S1, Cross-source point cloud preprocessing and normalization: Cross-source point cloud preprocessing and normalization operations are performed on the three-channel defect point cloud data collected by ground-based lidar, UAV multi-view reconstruction equipment, and handheld structured light scanner. First, invalid points are filtered out, and point data with coordinate values ​​exceeding the preset three-dimensional spatial range of the archway are removed to eliminate interference from ambient noise. Then, statistical filtering is used to remove outliers. The average distance from each point to its k nearest neighbors is calculated, and point data with average distances exceeding the preset distance threshold are removed. Outliers caused by equipment vibration and environmental factors during the scanning process are removed, thus improving the purity of the point cloud data.

[0041] After noise removal, coordinate system unification is implemented. First, coarse registration is performed to extract local geometric features of the three point clouds, calculate the similarity matrix between features of different point clouds, and select the top 10% of feature pairs with the highest matching degree. The initial spatial transformation matrix is ​​solved using these feature pairs to complete the initial spatial alignment of the three point clouds, laying the foundation for subsequent precise registration. Then, fine registration is performed to calculate the normal vector of each point in the point cloud after coarse registration. Using the Euclidean distance from the point to the corresponding tangent plane as a constraint, the optimal spatial transformation matrix is ​​solved through multiple rounds of iterative calculation to achieve coordinate system unification of the three point clouds, so that the archway point cloud data collected by different sensors are unified into the same spatial coordinate system.

[0042] The following steps are performed sequentially: resolution and point count normalization, multi-scale geometric feature extraction, and feature dimension unification. This ensures that the three point cloud data sources form a unified input format, eliminates format differences between data collected from different sensors, guarantees the smoothness and accuracy of subsequent algorithm processing, and provides standardized point cloud data for subsequent confidence modeling and other steps.

[0043] S2, Bayesian confidence probability modeling: For the three-way archway point cloud after normalization preprocessing, the Bayesian probabilistic confidence modeling algorithm is used to calculate the point-level confidence. The calculation combines the local geometric features of each point with cross-source topological associations, allowing the confidence calculation to fully integrate the local features and cross-source association features of the points, thus improving the accuracy of the calculation results. The mathematical expression of the Bayesian probabilistic confidence modeling algorithm is as follows:

[0044] in, For the midpoint of the point cloud The initial Bayesian confidence level, The balance coefficient is a value ranging from 0.3 to 0.7. For point The confidence level mean term, For point The confidence level variance term, For point The cross-source topological association term is defined as follows: Local geometric features include local density, local curvature stability, and normal vector distribution characteristics. Local density is obtained by statistically analyzing the ratio of the number of points within a certain neighborhood of a point to the volume of the neighborhood space, reflecting the density of point distribution in a local area of ​​the archway point cloud. Local curvature stability is obtained by calculating the variance of the curvature difference between a point and its neighboring points, reflecting the geometric stability of the surface patterns and structure of the archway stone carvings. Normal vector distribution characteristics are obtained by statistically analyzing the consistency of the normal vector direction of all points within the neighborhood of a point, characterizing the regularity of the local geometric shape of the archway. Cross-source topological association is determined by comparing the spatial coordinate deviation and topological connection relationship between the point and the corresponding points in the other two point clouds. First, the corresponding matching point of the point in the other point clouds is found, then the spatial distance and normal vector deviation between the point and the matching point are calculated. At the same time, the similarity of the topological structure of their respective neighborhoods is combined to reflect the degree of feature matching of different source point clouds at the same location on the archway.

[0045] The aforementioned local geometric features and parameters associated with cross-source topology are input into a multilayer fully connected network for feature mapping and fusion. This allows for deep integration of various features to fully extract effective information. Ultimately, the probability distribution parameters for each point belonging to the true structure of the archway are calculated. These parameters include the mean parameter representing the reliability of the point and the variance parameter representing the stability of the point's confidence. The output includes the initial Bayesian confidence result containing the point-level confidence probability distribution, providing accurate initial data for subsequent Bayesian confidence topology propagation and giving the optimization process of topology propagation a clear and reliable starting basis.

[0046] S3, Bayesian confidence topology propagation: Based on the initial Bayesian confidence results, a Bayesian graph neural network adapted to the spatial topology of the archway point cloud is constructed. Each point in the point cloud is treated as an independent graph node, and the k-nearest neighbor topological relationship of the point cloud is used as the graph edge. In this implementation, the value of k is selected as 24 to construct an undirected graph structure, so that the network structure is highly consistent with the spatial topological features of the archway point cloud. The Bayesian graph neural network is set with 5 Bayesian graph convolutional layers. Each Bayesian graph convolutional layer has a built-in graph attention calculation unit, Bayesian probability update unit, and neighborhood information aggregation unit. A dimension mapping layer is set at the network output end, and its output dimension is consistent with the dimension of the initial Bayesian confidence results, ensuring that the network output data can be directly connected to the subsequent processing flow.

[0047] Based on this network, a confidence propagation algorithm is executed to iteratively update and propagate confidence information between neighboring points. The initial Bayesian confidence result is used as the starting data for propagation, and neighboring points in the point cloud spatial topology are used as the propagation path. Confidence updates are performed round by round. The mathematical expression of the confidence propagation algorithm is:

[0048] in, The point after the kth iteration Bayesian confidence level For point The set of neighboring points, adjacent points Relative to point The propagation weight, The adjacent points after the (k-1)th iteration Bayesian confidence level The adaptive propagation coefficient has a value ranging from 0.1 to 0.5. For point The confidence correction increment is as follows: First, the association weight between each point and all its neighboring points is calculated. The association weight is determined based on the spatial distance between neighboring points, the similarity of local geometric features, and the cross-source topological association. The closer the distance, the higher the feature similarity, and the stronger the topological association, the greater the corresponding association weight. This makes the weight allocation of confidence propagation more in line with the actual spatial features of the archway point cloud. Then, the confidence information of neighboring points is weighted and aggregated based on the association weight to obtain the confidence update increment of that point. The update increment is then fused with the current confidence value of that point to complete the iterative update of the confidence of that point, so that the confidence of each point can absorb the effective information of neighboring points.

[0049] All points are updated and propagated synchronously in each round of iteration to ensure the spatial synchronicity of confidence optimization. When the iteration reaches the preset limit of 50 iterations in this implementation, the iteration operation is stopped and the Bayesian confidence result after topology optimization is output, so that the confidence data of the archway point cloud is more consistent with its real spatial structure and decorative features, and the spatial consistency and overall reliability of the confidence data are improved.

[0050] S4, Active calibration in high uncertainty regions: Based on the Bayesian confidence results after topology optimization, high uncertainty regions of the archway point cloud were identified and calibrated. In this implementation, the threshold was set to 0.25 through sensor calibration experiments. Continuous regions composed of all points with confidence variance values ​​exceeding 0.25 were identified as high uncertainty regions. The archway's decorative defects, the weathered stone layer, and the high-angle areas scanned by the UAV were identified as high uncertainty regions. Low reliability regions in the archway point cloud were accurately located, providing direction for subsequent accurate calibration.

[0051] To optimize the active perception strategy for this region, we first trace back the original data collected by the three types of sensors, read the original point cloud frame data corresponding to the high uncertainty region from the storage medium of each device, compare the coordinate differences between the original point cloud frame data and the preprocessed point cloud data, and perform the first secondary correction and calibration of the confidence level of the high uncertainty region based on the comparison results, restore the effective feature information in the original scan data, and initially improve the accuracy of the confidence level data in this region. After the initial calibration, the confidence variance of the missing fine ornamentation on the archway still exceeded 0.25. Therefore, a labeling task containing the coordinate range of the missing ornamentation area was generated and pushed to a professional labeling platform for digital modeling of ancient buildings. The labeling personnel for ancient building protection completed the labeling of the point cloud features and ornamentation structure of the area. With the help of the professionals' experience in ancient building research and modeling, accurate feature information was supplemented. Based on the labeling results returned by the labeling platform, the confidence of the missing ornamentation area was corrected and calibrated a second time, so that the confidence data of the area closely matched the original ornamentation and structural features of the archway. The confidence calibration of all high uncertainty areas was completed, and the accuracy and reliability of the overall point cloud confidence data of the archway were comprehensively improved.

[0052] S5, Cross-source Feature Fusion Point Cloud Completion: Using the calibrated Bayesian confidence score as the weighted fusion criterion, a cross-source Bayesian weighted fusion repair algorithm is applied to perform cross-source feature fusion and point cloud completion on the three-way archway point clouds. This allows the completion process to fully combine the advantages of different source point clouds, improving the completeness and accuracy of the model. The mathematical expression of the cross-source Bayesian weighted fusion repair algorithm is as follows:

[0053] in, For the fused global features, For cross-source point clouds, The value range is from 2 to 8. For the s-th path, the midpoint of the cloud The calibrated Bayesian confidence score For the s-th path, the midpoint of the cloud Geometric features, The characteristic balance coefficient is a value ranging from 0.2 to 0.6. For point The corresponding cross-source complementary features.

[0054] First, a weighted fusion operation is performed. For each point in each path of the point cloud, the geometric features are multiplied element-wise with the corresponding calibrated Bayesian confidence score. This allows the high-confidence geometric features to play a dominant role in the fusion, while the influence of low-confidence features is reasonably reduced. Then, the multiplication results of the three path point clouds are added element-wise to obtain the initial fusion features of the archway point cloud, thus achieving the basic fusion of features from different source archway point clouds.

[0055] Subsequently, complementary feature injection was performed. First, the overlapping area between the three source point clouds was determined by spatial coordinate comparison, i.e., the three-dimensional spatial range where point data exists in two or more source point clouds. Then, geometric differences such as coordinate deviation, normal vector deviation, and curvature deviation of corresponding points in different source point clouds within the overlapping area were calculated. Based on geometric differences, structurally complementary areas between cross-source point clouds were identified. At the high point of the archway where the ornamentation was missing, the point cloud from the UAV multi-view reconstruction equipment had missing data, while the point cloud from the ground-based lidar had complete structural data in this area. At the local fine ornamentation missing area, the point cloud from the handheld structured light scanner had missing data, while the feature data after the ancient building was annotated could supplement the complete features of this area. The above areas are all structurally complementary areas, accurately locating the missing and missing data positions of the archway point cloud. The complete geometric features within the structurally complementary areas were extracted as cross-source complementary features and injected into the corresponding spatial positions of the initial fusion features to fill the feature data in the missing areas of the archway point cloud, making the fusion features more complete. The fusion was completed by combining the feature balance coefficient, resulting in the fused global features. The global features not only combined the advantages of different source point clouds but also filled the feature gaps in the missing areas.

[0056] Finally, based on the fused global features, the defect cloud of the ancient stone archway was 3D-completed, restoring the complete patterns and structure of the archway, replicating its historical appearance and details, and generating an accurate 3D model for the archway's defect restoration. This meets the needs of digital restoration and replication of ancient buildings, providing accurate 3D data support for the protection, digital archiving, and physical replication of ancient architectural relics. In summary, this embodiment applies the 3D modeling and restoration method to the digital restoration of stone archways in Ming and Qing dynasty architecture. It utilizes three types of sensors to collect cross-source point clouds from different parts of the archway. After preprocessing and normalization to eliminate data discrepancies, Bayesian probability confidence modeling and topology propagation are used to calculate and optimize the point cloud confidence. For high-uncertainty areas, calibration is achieved through backtracking of original data and professional manual annotation. Finally, a cross-source Bayesian weighted fusion restoration algorithm is used to achieve feature fusion and complementary injection, ultimately completing the point cloud and generating an accurate 3D model that fully restores the archway's patterns and structure. This provides reliable 3D data for the digital archiving, cultural relic protection, and physical replication of ancient buildings, demonstrating the method's adaptability and effectiveness in ancient building restoration scenarios.

[0057] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for 3D modeling and repairing incomplete models based on point cloud deep learning, characterized in that, The method includes: S1, Cross-source point cloud preprocessing normalization: Used to preprocess multi-channel cross-source heterogeneous residual point cloud data collected by multiple types of sensors. Through coordinate system unification, resolution and point number normalization, multi-scale geometric feature extraction and feature dimension unification, the point cloud data of each channel has a unified input format. S2, Bayesian confidence probability modeling: For each point cloud after normalization preprocessing, the Bayesian probability confidence modeling algorithm is used. Combining the local geometric features of the points with cross-source topological association, the probability distribution parameters of each point belonging to the real structure are calculated, and the initial Bayesian confidence result containing the point-level confidence probability distribution is output. S3, Bayesian confidence topology propagation: Based on the initial Bayesian confidence result, a Bayesian graph neural network adapted to the spatial topology of the point cloud is constructed. Through the confidence propagation algorithm in the graph neural network, the confidence information is iteratively updated and propagated between adjacent points, and the topology-optimized Bayesian confidence result is output. S4, Active calibration of high uncertainty regions: Based on the Bayes confidence results after topology optimization, identify high uncertainty regions in the point cloud with confidence scores below a set threshold, trigger an active perception optimization strategy for the identified high uncertainty regions, and perform secondary correction and calibration of the confidence scores of the high uncertainty regions. S5, Cross-source feature fusion point cloud completion: Using the calibrated Bayesian confidence as the basis for weighted fusion, the cross-source Bayesian weighted fusion repair algorithm is applied. The geometric features of different point clouds are weighted and fused with complementary features using the calibrated Bayesian confidence as the weight. Based on the fused global features, the incomplete point cloud is completed in three dimensions to generate a complete 3D model.

2. The method for 3D modeling and repairing incomplete models based on point cloud deep learning according to claim 1, characterized in that, In step S1, the cross-source point cloud preprocessing normalization step involves multiple types of sensors, including ground-based lidar, handheld structured light scanners, industrial CT scanners, and UAV multi-view reconstruction equipment. During the input preprocessing, invalid point filtering and outlier removal operations are performed sequentially on the multi-source heterogeneous residual point cloud data. Specifically, invalid point filtering removes point data whose coordinate values ​​exceed a preset three-dimensional spatial range. Outlier removal uses a statistical filtering method to calculate the average distance from each point to its k nearest neighbors and removes point data whose average distance exceeds a preset distance threshold. Then, coordinate system unification, resolution and point number normalization, multi-scale geometric feature extraction, and feature dimension unification are performed sequentially.

3. The method for 3D modeling and repairing incomplete models based on point cloud deep learning according to claim 2, characterized in that, In step S1, the coordinate system operation first performs a coarse registration operation, followed by a fine registration operation. The coarse registration operation specifically involves extracting local geometric features of each point cloud, calculating the similarity matrix between features of different point clouds, selecting the top 10% of feature pairs based on the similarity matrix, solving the initial spatial transformation matrix using the selected feature pairs, and completing the initial spatial alignment of each point cloud based on the initial spatial transformation matrix. The fine registration operation specifically involves calculating the normal vector of each point in the point cloud after coarse registration, using the Euclidean distance from the point to the corresponding tangent plane as a constraint, solving the optimal spatial transformation matrix through multiple rounds of iterative calculation, and completing the coordinate system one of each point cloud based on the optimal spatial transformation matrix.

4. The method for 3D modeling and repairing incomplete models based on point cloud deep learning according to claim 1, characterized in that, In step S2, the mathematical expression of the Yeats probability confidence modeling algorithm is: in, For the midpoint of the point cloud The initial Bayesian confidence level, The balance coefficient is a value ranging from 0.3 to 0.

7. For point The confidence level mean term, For point The confidence level variance term, For point The cross-source topological association term; the input dimensions of the confidence mean term and the confidence variance term include points. Local density, point Spatial deviation of points corresponding to other waypoint clouds, points The local curvature stability, the inherent noise level of the corresponding sensor, and the point The probability of occlusion at the scanning viewpoint, and the cross-source topological association terms through points The topological connectivity with adjacent points is calculated.

5. The method for 3D modeling and repairing incomplete models based on point cloud deep learning according to claim 1, characterized in that, In step S3, the Bayesian graph neural network adapted to the spatial topology of the point cloud constructs an undirected graph structure by treating each point in the point cloud as an independent graph node and the k-nearest neighbor topological relationship of the point cloud as graph edges, where the value of k ranges from 8 to 32. The Bayesian graph neural network is configured with 3 to 6 Bayesian graph convolutional layers, each layer containing a graph attention calculation unit, a Bayesian probability update unit, and a neighborhood information aggregation unit. The graph attention calculation unit is used to calculate the propagation weights between adjacent nodes, the Bayesian probability update unit is used to iteratively update the node probability parameters based on the initial Bayesian confidence result, and the neighborhood information aggregation unit is used to aggregate and transmit the confidence information of adjacent nodes. The output end of the Bayesian graph neural network is configured with a dimension mapping layer, the output dimension of which is consistent with the dimension of the initial Bayesian confidence result.

6. The method for 3D modeling and repairing incomplete models based on point cloud deep learning according to claim 1, characterized in that, In step S3, the mathematical expression of the confidence propagation algorithm is: in, The point after the kth iteration Bayesian confidence level For point The set of neighboring points, adjacent points Relative to point The propagation weight, The adjacent points after the (k-1)th iteration Bayesian confidence level The adaptive propagation coefficient has a value ranging from 0.1 to 0.

5. For point The confidence correction increment; the iteration termination condition of the confidence propagation algorithm is that the change in Bayesian confidence of all points is less than 0.001 after two consecutive iterations, or the number of iterations reaches the preset upper limit of 50. The propagation weight is obtained through the output of the graph attention calculation unit of the Bayesian graph neural network, and the confidence correction increment is obtained through the points. The difference between the initial Bayesian confidence score and the Bayesian confidence score after the previous iteration is calculated.

7. The method for 3D modeling and repairing incomplete models based on point cloud deep learning according to claim 1, characterized in that, In step S4, the set threshold is a fixed value predetermined through sensor calibration experiments, and the value range of the set threshold is 0.1 to 0.

3. The high uncertainty region is a continuous region composed of all points whose confidence variance values ​​exceed the set threshold. The active perception optimization strategy includes backtracking the original acquisition data of the corresponding sensor and initiating a manual annotation request. Backtracking the original acquisition data of the corresponding sensor specifically involves reading the original point cloud frame data corresponding to the high uncertainty region from the storage medium of the corresponding sensor, comparing the coordinate differences between the original point cloud frame data and the preprocessed point cloud data, and completing the secondary correction and calibration of the confidence of the high uncertainty region based on the comparison results. Initiating a manual annotation request specifically involves generating an annotation task containing the coordinate range of high uncertainty regions when the confidence variance still exceeds a set threshold after backtracking the original collected data. This task is then pushed to the annotation platform, and a secondary correction and calibration of the confidence of the high uncertainty regions is performed based on the annotation results returned by the platform.

8. The method for 3D modeling and repairing incomplete models based on point cloud deep learning according to claim 1, characterized in that, In step S5, the mathematical expression of the cross-source Bayesian weighted fusion repair algorithm is: in, For the fused global features, For cross-source point clouds, The value range is from 2 to 8. For the s-th path, the midpoint of the cloud The calibrated Bayesian confidence score For the s-th path, the midpoint of the cloud Geometric features, The characteristic balance coefficient is a value ranging from 0.2 to 0.

6. For point The corresponding cross-source complementary features; during the weighted fusion process, the geometric features of each point in each path point cloud are multiplied element-wise with the corresponding calibrated Bayesian confidence, and then the multiplication results of all path point clouds are added element-wise to obtain the initial fused features. The cross-source complementary features are then concatenated with the initial fused features to obtain the fused global features.

9. The method for 3D modeling and repairing incomplete models based on point cloud deep learning according to claim 1, characterized in that, In step S5, the complementary feature injection operation specifically includes: first, determining the overlapping area between different path point clouds by comparing spatial coordinates. The overlapping area is the three-dimensional spatial range where point data exists in two or more path point clouds. Then, calculating the geometric differences between corresponding points of different path point clouds within the overlapping area. The calculation dimensions of geometric differences include coordinate deviation, normal vector deviation, and curvature deviation. Based on the geometric differences, identifying the structurally complementary area between cross-source point clouds. The structurally complementary area is the three-dimensional spatial range where data is missing in one path point cloud and complete data exists in at least one path point cloud. Extracting the geometric features of the complete point cloud within the structurally complementary area as cross-source complementary features. Injecting the cross-source complementary features into the corresponding spatial position of the fused global features.

Images