A boundary-guided dual-branch enhanced point cloud hole completion measurement method and system

Abstract

The application discloses a kind of boundary guide double-branch enhanced point cloud pit hole completion measurement method and system, belong to pit hole three-dimensional point cloud processing technical field, method includes: extracting pit hole boundary point set and constructing boundary guide feature, form enhanced input point cloud by splicing boundary guide feature with point cloud coordinates, so that completion network perceives the spatial distribution of pit hole boundary area, transition area and pit bottom area in input stage;Local discriminant features and multi-scale geometric features are extracted by group convolution operation and multi-scale heterogeneous convolution operation respectively, and two types of features are adaptively weighted and fused using a gating fusion mechanism according to the boundary guide feature;Based on the fusion feature, a completed point cloud is generated, and a closed geometric body is constructed using the completed reference surface, the actual defect surface and the boundary side wall to calculate the pit volume.The application improves the continuity of pit hole boundary completion and the stability of volume measurement, and is suitable for workpiece surface defect detection and quantitative evaluation.

Description

Technical Field

[0001] This application relates to the field of 3D point cloud processing technology for potholes, specifically to a boundary-guided dual-branch enhanced point cloud pothole completion measurement method and system. Background Technology

[0002] With the development of 3D laser scanning, structured light measurement, and industrial vision inspection technologies, point cloud data has been widely used in scenarios such as surface inspection of mechanical parts, wear analysis, defect identification, and digital reconstruction. For workpiece surfaces with pits, dents, localized erosion, or material detachment, relying solely on the acquired raw point cloud for morphological analysis and volume calculation often results in significant errors due to the presence of missing areas, occluded areas, and sparse boundary regions.

[0003] In existing technologies, deep learning methods are commonly used to complete incomplete point cloud structures. Existing deep learning point cloud completion networks, such as PF-Net and its improved methods, have a certain application foundation in general incomplete point cloud reconstruction, but they still have the following shortcomings in scenarios with pit-like defects.

[0004] First, existing deep learning point cloud completion networks have limited ability to jointly express local fine-grained features and cross-scale geometric features. Especially when the edges of the pits change sharply, the bottom curvature is complex, and the missing span is irregular, problems such as the completed surface being too flat, the boundary shrinking, or the local shape being distorted may occur.

[0005] Second, existing point cloud completion training widely uses chamfer distance (CD) as the loss function. This type of loss function uses nearest neighbor pairs as the main constraint, which is insufficient in characterizing the overall distribution relationship of the point set and the global quality transfer relationship. It is easy to cause the predicted point cloud to be close to the real point cloud in terms of local distance, but still have deviations in terms of point distribution uniformity, boundary consistency and overall shape.

[0006] Third, in terms of calculating the volume of pits, existing solutions are mostly based on directly fitting the reference surface to the original defect point cloud or using simple interpolation methods to recover the missing surface. These methods are highly dependent on boundary quality and local geometric priors, making it difficult to obtain stable and high-precision volume results on complex surfaces.

[0007] In existing technologies for detecting pit-type defects, point cloud completion and volume calculation are usually treated as two relatively independent steps. The completion model mainly pursues the similarity of point cloud morphology, but does not jointly constrain the continuity of pit boundaries, the reliability of reference surfaces, and the stability of volume calculation. As a result, although the completion result is visually complete, it may still produce large errors when used as a reference surface for volume measurement. Summary of the Invention

[0008] To help solve the above-mentioned technical problems, this application provides a point cloud pit completion measurement method and system with boundary-guided dual-branch enhancement.

[0009] In a first aspect, this application provides a boundary-guided dual-branch enhanced point cloud pit completion measurement method, which includes: Obtain the original point cloud of the surface of the object under test and perform preprocessing; Extract the set of boundary points of the pits, and construct boundary guiding features for the points in the point cloud to be completed based on the set of boundary points of the pits. Then, stitch the boundary guiding features with the three-dimensional coordinate features of the points to form an enhanced input point cloud. The input point cloud is input into a pre-trained completion network, which outputs a completed point cloud region. The completion process of the completion network includes: Encode the enhanced input point cloud to obtain multi-level geometric features; The feature channels are grouped by group convolution operation to obtain the first branch features; By performing multi-scale heterogeneous convolution operations, features are extracted at at least three different neighborhood scales to obtain the second branch features. Through gated fusion operation, fusion weights are generated based on boundary guiding features in the enhanced input point cloud. Adaptive weighted fusion of the first branch features and the second branch features is then performed to obtain the fused features. Generate complete region point clouds based on fusion features; The point cloud of the repaired area is stitched together with the original complete area point cloud to obtain the complete surface point cloud after the pothole repair; the pothole volume is calculated based on the complete surface point cloud and the original pothole area point cloud to obtain the pothole volume measurement result.

[0010] The extraction of the pit boundary point set specifically includes: Normal vector and curvature estimation based on the local neighborhood of a point; The suspected pit area is initially segmented based on curvature abrupt change, normal vector change rate, and local depth deviation; wherein, the normal vector change rate is measured by the deviation between the point normal vector and the average normal vector of its neighborhood, and the local depth deviation is represented by the normal distance of the point relative to its neighborhood fitting reference surface. By constructing a comprehensive scoring function The points are judged, and those with scores greater than a preset threshold are identified as candidate pothole points; where k i For local curvature, As a measure of normal vector deviation, For local depth deviation, w1, w2 and w3 are weighting coefficients, and i is the index of the point in its respective point cloud; The set of boundary points of potholes can be extracted from the candidate potholes using region growing, boundary tracking or clustering methods.

[0011] The boundary guidance features include boundary markers, normalized distance from a point to the pit boundary, local curvature, and normal vector deviation; the enhanced input point cloud is represented as... , where p i Let b be the three-dimensional coordinates of the i-th point. i For boundary identification, d i Let n be the normalized distance from the point to the boundary of the pit. i For the normal vector deviation characteristic, h i For local depth deviation characteristics, N e To increase the total number of points in the input point cloud, k i For local curvature.

[0012] The gating fusion operation includes: The normalized distance d from each point in the enhanced input point cloud to the edge of the pit is calculated as follows: i Local curvature k i and normal vector deviation n i Gated weights are generated by mapping weights and using the Sigmoid activation function. ,in For the Sigmoid function, W g For the gated mapping weights, b g For bias terms; according to For the first branch feature F g With the second branch feature F h We perform weighted fusion to obtain the fusion feature F. f ,in This represents element-wise multiplication, where g represents the feature F of the first branch. g The value comes from the group convolution branch and serves as the group index for that branch; h is used to represent the feature F of that second branch. h It comes from the multi-scale heterogeneous convolution branch.

[0013] The at least three different neighborhood scales include small neighborhood, medium neighborhood, and large neighborhood; wherein, the small neighborhood uses a set of nearest neighbors with K=8 or a spherical neighborhood with radius r1, the medium neighborhood uses a set of nearest neighbors with K=16 or a spherical neighborhood with radius r2, and the large neighborhood uses a set of nearest neighbors with K=32 or a spherical neighborhood with radius r3, where r1 < r2 < r3; the second branch feature F h It can be obtained by feature concatenation or by weighted summation of learnable weights.

[0014] Based on the fusion feature F f Generating a complete point cloud of the region, specifically including: Through coarse-completion decoder D c Generate coarsely completed point cloud P coarse =D c (Ff ); By refining decoder D f The coarsely completed point cloud and fused features are refined to generate a finely completed point cloud P. fine =D f (P coarse ,F f ), which serves as the point cloud of the completed region.

[0015] The completion network is trained using a composite loss function for supervision. The composite loss function is: , , , L is the weighting coefficient. emd The EMD approximation loss based on the Sinkhorn algorithm is used to constrain the overall distribution consistency between the predicted point cloud and the real point cloud. The L geo The boundary geometric constraint loss is a weighted sum of the boundary position continuity term, normal continuity loss, curvature continuity loss, and smoothing regularization loss. The L volume The volume consistency loss is used to constrain the predicted pothole volume V calculated based on the completion results. pred Compared with the actual volume label V true Consistency between them.

[0016] The calculation of the EMD approximation loss based on the Sinkhorn algorithm includes: Let p1 be the i-th point in the predicted point cloud. i The j-th point in the real point cloud is q1. j Construct a cost matrix between the predicted point cloud and the actual point cloud. i and j are the indices of the point in its respective point cloud. An entropy regularization term is introduced into the optimal transmission problem, and the approximate optimal transmission matrix is ​​solved through Sinkhorn iteration. , The approximate loss of EMD is calculated using the following formula: Where n is the number of predicted point cloud points and m is the number of actual point cloud points. This represents the approximately optimal transmission weight.

[0017] The calculation of the pit volume based on the complete surface point cloud and the original pit region point cloud specifically includes: Using the completed region as a reference surface, the point cloud of the original pit region as the actual defect surface, and the set of pit boundary points to construct a sidewall closed region, forming a closed geometry; Calculate the volume of the closed geometry; where, when using the height difference integral method, the volume is: , h ref h is the reference surface height. def The height of the defect surface is denoted as M. When using the tetrahedral decomposition method, the volume is the absolute value of the sum of the oriented volumes of each triangular facet. M′ is the number of discrete meshes divided in the projection region along the u direction, and N′ is the number of discrete meshes divided in the projection region along the v direction.

[0018] In a second aspect, this application provides a boundary-guided bi-branch enhanced point cloud pit completion measurement system, wherein the system is used to perform the boundary-guided bi-branch enhanced point cloud pit completion measurement method as described in any one aspect, the system comprising: The point cloud preprocessing module is used to denoise, downsample, and normalize the raw point cloud to obtain a standardized point cloud. The pothole identification and feature enhancement module is used to extract the pothole boundary point set, and construct boundary guiding features for the point cloud to be completed based on the pothole boundary point set. The boundary guiding features are then concatenated with the three-dimensional coordinate features of the points to form an enhanced input point cloud. The boundary guiding features include boundary markers, normalized distance from the point to the pothole boundary, local curvature, normal vector deviation, and local depth deviation. A boundary-guided dual-branch feature-enhanced point cloud completion network is used to process the enhanced input point cloud and output a completed region point cloud; the point cloud completion network includes: The encoding module is used to encode the enhanced input point cloud to obtain multi-level geometric features; The dual-branch multi-scale feature enhancement module includes a group convolution branch and a multi-scale heterogeneous convolution branch, wherein the multi-scale heterogeneous convolution branch includes a small neighborhood branch, a medium neighborhood branch, and a large neighborhood branch. The gated fusion module is used to generate fusion weights based on the boundary guidance features in the enhanced input point cloud, and to adaptively weight and fuse the output features of the group convolutional branch and the multi-scale heterogeneous convolutional branch to obtain fused features. The decoding and generation module is used to output the point cloud of the completed region based on the fused features through a coarse-to-fine hierarchical generation method; The post-processing and stitching module is used to stitch the point cloud of the completed region with the point cloud of the original complete region, and to perform local smoothing processing on the transition zone of the pit boundary to obtain the complete surface point cloud after the pit is repaired. The volume measurement module is used to use the completed region as a reference surface, the point cloud of the original pit region as the actual defect surface, and the set of the pit boundary points to construct a sidewall closed region and form a closed geometry. The volume of the closed geometry is calculated by the height difference integral method or the tetrahedral decomposition method to obtain the pit volume measurement result. The results output and visualization module is used to output the completed point cloud, the pothole volume measurement results, and the visualization image.

[0019] In summary, compared with the prior art, this application has at least the following beneficial effects: First, this application constructs boundary guidance features such as boundary markers, boundary distances, local curvature, normal vector deviation, and local depth deviation, enabling the completion network to distinguish between the pit boundary area, transition area, and pit bottom area, thereby improving the continuity of pit edge completion.

[0020] Second, this application extracts local mutation features, transition surface features and overall curvature trend features by collaboratively using group convolutional branches and heterogeneous convolutional branches in small, medium and large neighborhoods, and improves the multi-scale expression capability under complex pit morphology by adaptively allocating feature weights through a gating fusion module.

[0021] Third, this application constructs a composite loss function consisting of Sinkhorn-EMD loss, boundary geometric constraint loss, and volume consistency loss, so that the completion result not only meets the point cloud geometric similarity requirements, but also meets the requirements of boundary position continuity, normal continuity, curvature continuity, and volume measurement stability.

[0022] Fourth, this application constructs a closed difference volume based on the complete reference surface, the actual defect surface, and the lateral closure constraint of the pit boundary. Compared with directly fitting the defect surface or local interpolation, this can improve the stability of complex pit volume measurement.

[0023] Fifth, this application constructs a complete technical process from point cloud acquisition, defect identification, model completion to volume measurement, which has good engineering practicality and system integration value. Attached Figure Description

[0024] Figure 1 This is the overall flowchart of the method for pit point cloud completion and volume measurement based on boundary-guided bi-branch feature enhancement in this application.

[0025] Figure 2 This is a schematic diagram of the process for identifying the pit area and extracting the point cloud to be completed in this application; Figure 3 This is a schematic diagram of the boundary-guided dual-branch feature enhancement point cloud completion network structure in this application; Figure 4 This is a schematic diagram of the structure of the group convolution branch, small neighborhood heterogeneous branch, medium neighborhood heterogeneous branch, large neighborhood heterogeneous branch and gated fusion module in this application; Figure 5 This is a flowchart illustrating the construction of the composite loss function and network training for pothole volume measurement in this application. Figure 6This is a schematic diagram illustrating the construction of the reference surface, the actual defect surface, and the closed difference body in this application; Figure 7 This is a visualization of the point cloud and pit volume results before and after completion in this application. Detailed Implementation

[0026] The present application will be further described below with reference to the accompanying drawings. The principles of the present application are very clear to those skilled in the art. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0027] Combination Figures 1 to 6 It is understood that the method for pit point cloud completion and volume measurement based on boundary-guided bi-branch feature enhancement in this application includes the following steps: S1. Obtain the point cloud of the surface of the object under test and perform preprocessing: The original three-dimensional point cloud data of the surface of the object to be tested is collected. The original three-dimensional point cloud data is processed by outlier removal, noise suppression, downsampling, coordinate unification and scale normalization to obtain standardized point cloud data for subsequent pit identification and point cloud completion.

[0028] Let the original point cloud obtained be: , .

[0029] Where N is the number of points in the original point cloud, p i Let i be the i-th three-dimensional point.

[0030] Furthermore, the preprocessing includes at least one of the following: S11. Use statistical filtering or radius filtering to remove outliers; S111, For each point p i Take its k nearest neighbor set N k (p i If ), then its neighborhood average distance is defined as: ; where d i This is the normalized distance from the point to the boundary of the pit.

[0031] S112. Calculate the mean distance of all points. with standard deviation : ; .

[0032] When satisfied When this happens, the corresponding point is identified as an outlier and removed. This is the outlier determination coefficient.

[0033] S12. Use voxel downsampling or farthest point sampling to reduce point cloud redundancy; S121. After outlier removal, voxel downsampling is performed on the point cloud. The space is divided into sections with side length l. v For a voxel grid, for the point set V in the m1-th voxel m1 Its representative point is defined as: ; Among them, V m1 This represents the number of points within the voxel.

[0034] S13. Translate and scale the point cloud to map it to a unified coordinate space; S131. Subsequently, the point cloud is centralized and normalized. The centroid of the point cloud is defined as: ; The translated point cloud is represented as follows: ; S132. Take the maximum distance from all translated points to the origin as the scale factor: ; The normalized point cloud is defined as follows: ; Thus, a standardized point cloud is obtained: ; S14. Establish a local coordinate system for the workpiece when necessary to reduce the impact of different acquisition postures on subsequent model inference.

[0035] S2. Identify the pit and defect area and extract the point cloud to be completed.

[0036] Based on the preprocessed point cloud data, local geometric analysis is performed on the surface of the object under test to determine the pit defect region, pit boundary and defect neighborhood point set, and the residual defect cloud subset is extracted from the overall point cloud as input for the completion network.

[0037] Further, step S2 includes: S21. Estimate the normal vector and curvature of the local neighborhood of the point cloud; S211, Construction of Local Covariance Matrix: For each point Take its k nearest neighbor set The neighborhood centroid is: ; Then its local covariance matrix is: ; For C i Perform eigenvalue decomposition: m2=0,1,2.

[0038] And order: ; Then the eigenvector corresponding to the smallest eigenvalue It can be used as the normal vector of that point: ; It should be noted that in this application, "i" and "j" are the indices of a point within its corresponding point cloud. When i and j are used as subscripts or superscripts for symbols other than points, it indicates that the other symbol corresponds to a point, as in the example above. Wait, at this time C i The 'i' has no meaning and is simply a label. Therefore, when 'i' and 'j' are used as subscripts or superscripts of symbols other than dots, this application does not emphasize their meaning.

[0039] S212, Local curvature estimation: point The curvature is defined as: ; When k i A larger value indicates a strong geometric change in the area where the point is located, and it is more likely to be located at the edge of a pit or in a defect area.

[0040] S22. Based on information such as curvature abrupt change, normal vector change rate, local depth deviation, or surface continuity interruption, perform initial segmentation of the suspected pit area; Furthermore, the local depth deviation can be represented by the normal distance of the test point relative to its neighboring fitting reference surface, or by the height difference between the test point and the locally fitted surface of the surrounding intact surface. The local depth deviation helps distinguish between normal surface undulations and pit-like depressions, improving the accuracy of initial pit segmentation.

[0041] S221, Setting Points The average value of the normal vector within the neighborhood is: ; The normal vector deviation metric is: ; when A larger value indicates a significant change in the orientation of the local surface.

[0042] S23. Use region growing, boundary tracking or clustering methods to extract the set of boundary points of the pits; S24. The point cloud input to be completed is formed by the point set of the pit boundary and the point set of the surrounding complete area, so that the network can obtain both the defect area information and the geometric prior information of the surrounding complete surface.

[0043] S241. Comprehensive scoring function for constructible pit points: ; where k i For local curvature, As a measure of normal vector deviation, For local depth deviation, w1, w2 and w3 are weighting coefficients, and i is the index of the point in its respective point cloud; When satisfied At that time, the point The point was identified as a candidate for a pit, among which, T S This is a preset threshold.

[0044] In some implementations, the pothole area can be obtained by manual marking; in other implementations, the pothole area can be obtained by an automatic detection algorithm, and this application does not limit this.

[0045] S3. Construct pit boundary guiding features and form an enhanced input point cloud: Based on the set of pit boundary points, a boundary guidance feature is constructed for each point in the point cloud to be completed. The boundary guidance feature includes at least one of the following: boundary identifier, normalized distance from the point to the pit boundary, local curvature, normal vector deviation, and local depth deviation. Preferably, the enhanced input point cloud simultaneously includes boundary identifier, boundary distance, local curvature, normal vector deviation, and local depth deviation to enhance the network's ability to distinguish between pit boundary regions, transition regions, and pit bottom regions.

[0046] For any point in the point cloud, if it belongs to the pit boundary point set, a boundary identifier value of 1 is assigned; otherwise, a boundary identifier value of 0 is assigned. The minimum Euclidean distance from the point to the pit boundary point set is calculated and normalized to obtain the boundary distance feature. The boundary guidance feature is then concatenated with the 3D coordinate feature of the point to form an enhanced input point cloud.

[0047] In this way, the completion network can distinguish between the edge area, transition area and bottom area of ​​the pit, so that the network can prioritize maintaining the continuity of the pit boundary when generating the completion point cloud, and reduce the problems of boundary shrinkage, edge collapse and overly flat bottom of the completion surface.

[0048] The augmented input point cloud can be represented as , where p i Let b be the three-dimensional coordinates of the i-th point. i For boundary identification, d i k is the normalized distance from the point to the boundary of the pit. i For local curvature, n iFor the normal vector deviation characteristic, h i For local depth deviation characteristics, N e To increase the total number of points in the input point cloud, k i For local curvature.

[0049] S4. Construct a boundary-guided dual-branch feature-enhanced point cloud completion network: A boundary-guided feature input module, group convolutional branch, multi-scale heterogeneous convolutional branch, and gated fusion module are introduced into the point cloud completion basic network to construct a boundary-guided dual-branch feature-enhanced point cloud completion network.

[0050] Let the input point cloud features be represented as: ; Where N1 represents the number of input points and C1 represents the number of feature channels.

[0051] S41, Encoding Module: The input point cloud input encoding module is enhanced. The encoding module jointly encodes the 3D coordinate features, boundary marker features, boundary distance features, curvature features, and normal vector deviation features to obtain multi-level geometric features for representing the pit boundary area, transition area, and pit bottom area.

[0052] S411. First, global features are extracted through the encoding module: ; Among them, P e To enhance the input point cloud, This represents the encoding mapping, where F is the multi-level geometric feature obtained after encoding.

[0053] During the feature enhancement stage, the input features are simultaneously fed into the group convolution branch and the multi-scale heterogeneous convolution branch.

[0054] S42, Dual-branch multi-scale feature enhancement module: The features output by the encoding module are simultaneously fed into the group convolution branch and the multi-scale heterogeneous convolution branch.

[0055] S421, Group Convolution Branch: Group convolution branches are used to divide the input channels into multiple groups and perform convolution operations on each group. This reduces the number of parameters and computational complexity while improving the ability to distinguish local geometric features and reducing feature redundancy. For local abrupt changes near the boundaries of pits, group convolution branches can enhance the structural response within the local channels, allowing boundary detail features to be preserved in the subsequent fusion process.

[0056] Divide the input features into Z groups according to channels: ; The convolution output of the g-th group (g > 0 and < or equal to Z) is: ; in, W represents the convolution operation. g For the g-th group of convolution kernels, b g For bias terms, This is the activation function.

[0057] The total output of the group convolution is: ; The purpose of group convolution is to reduce the number of parameters and redundant computation, and to enhance the ability to discriminate local geometric details.

[0058] S422, Multi-scale heterogeneous convolution branch: The multi-scale heterogeneous convolution branches include small neighborhood branches, medium neighborhood branches, and large neighborhood branches. Among them, the small neighborhood branch is used to extract local abrupt change features at the pit boundary, the medium neighborhood branch is used to extract the transition surface features between the pit edge and the pit bottom, and the large neighborhood branch is used to extract the overall curvature trend features of the intact surface around the defect.

[0059] The aforementioned neighborhood branches at different scales, combined with boundary guidance features such as boundary distance, local curvature, and normal vector deviation, enable the network to adopt geometric perception methods at different scales for different regions of the pit.

[0060] Let F be the output of the small neighborhood branch, the medium neighborhood branch, and the large neighborhood branch. s F m F l The output of the multi-scale heterogeneous convolution branch is: ; or ; in , , These are the learnable weights corresponding to the small neighborhood branch, medium neighborhood branch, and large neighborhood branch, respectively.

[0061] Small neighborhood branches use a set of nearest neighbors with K=8, medium neighborhood branches use a set of nearest neighbors with K=16, and large neighborhood branches use a set of nearest neighbors with K=32; or they can use spherical neighborhoods with radii r1, r2, and r3 respectively, where r1 is less than r2 and less than r3.

[0062] In a specific embodiment, the small neighborhood branch uses a set of nearest neighbors with K = 8 to centrally perceive the high-frequency geometric mutations at the pothole boundary; the medium neighborhood branch uses a set of nearest neighbors with K = 16 to effectively capture the changing trend of the transition surface between the pothole edge and the pothole bottom; the large neighborhood branch uses a set of nearest neighbors with K = 32 to obtain the macroscopic curvature distribution characteristics of the complete surface around the defect. In another embodiment, spherical neighborhoods with radii r1, r2, and r3 can also be used respectively, where r1 < r2 < r3. The features extracted by the above different-scale branches can be fused by feature stitching operations or weighted summation with learnable weights to obtain the output features of the multi-scale heterogeneous convolutional branch.

[0063] By setting the multi-scale heterogeneous convolutional branch with different neighborhood scales, the network can establish multi-scale geometric correlations between the pothole boundary, the pothole bottom transition area, and the complete surface around the defect, thereby improving the completion stability under complex pothole morphologies.

[0064] S43. Gated Feature Fusion Module: Input the local discriminative features output by the grouped convolutional branch and the multi-scale features output by the multi-scale heterogeneous convolutional branch into the gated fusion module. Different from simple feature stitching, the gated fusion module can adaptively adjust the feature contributions of each branch according to the pothole boundary distance and curvature changes, enabling different regions to obtain differentiated feature expressions. The gated fusion module generates fusion weights based on the pothole boundary distance feature, the local curvature feature, and the normal vector deviation feature.

[0065] Let the output of the grouped convolutional branch be F g , and the output of the multi-scale heterogeneous convolutional branch be F h , and the gated weight be G. Then the fused feature is represented as F f . Among them, the gated weight G is generated based on boundary guiding features such as boundary distance, local curvature, and normal vector deviation, and is used to adaptively adjust the feature contributions of the grouped convolutional branch and the multi-scale heterogeneous convolutional branch.

[0066] The gated weight can be expressed as: ; where represents the Sigmoid function, W g represents the gated mapping weight, b g represents the bias term, d i represents the normalized distance from the point to the pothole boundary, k i represents the local curvature, n i represents the normal vector deviation feature, g is used to indicate that the first branch feature F g comes from the grouped convolutional branch and serves as the group index of the grouped convolutional branch, and h is used to indicate that the second branch feature F hIt comes from the multi-scale heterogeneous convolution branch.

[0067] Specifically, the generation process of the gating weight G is as follows: the normalized distance d from each point in the enhanced input point cloud to the pit boundary is calculated. i Local curvature k i and the deviation of the normal vector n i As input, through learnable mapping weights W g and bias term b g A linear transformation is performed, followed by a Sigmoid activation function to map the output to the (0,1) interval, yielding gating weights used to control the contribution ratio of the two branches' features. These gating weights ensure that in the pit boundary region (i.e., d... i Smaller, k i In larger regions, the local discriminative features of the group convolutional branches receive greater weight; in the intact surface region at the bottom of the pit and around it (i.e., d... i Larger, k i In smaller regions, the overall geometric trend features of multi-scale heterogeneous convolution branches receive greater weight.

[0068] The fusion feature can then be expressed as: ; in, This indicates element-wise multiplication.

[0069] Through the above-mentioned gating fusion method, the network can adaptively adjust the weights of local discriminative features and multi-scale geometric features according to the geometric features of different regions of the pit, so that the boundary region, transition region and bottom region of the pit can obtain differentiated feature representations, thereby improving the continuity and stability of the completed reference surface.

[0070] S44, Decoding Generation Module: Based on the fusion features output by the gated fusion module, a coarse-to-fine hierarchical generation method is adopted to output the completed point cloud. Specifically, the decoding and generation module first generates a coarse completed point cloud to restore the overall reference surface trend of the pit area; then, based on the coarse completed point cloud and the fusion features, it performs a fine completed point cloud generation to restore the transition area of ​​the pit boundary and local surface details.

[0071] First, output the coarsely completed point cloud: ; Then, based on the coarsely completed point cloud and the fused features, a refined point cloud is generated: ; Among them, P fine To complete the point cloud, F f D represents the fusion feature output by the gated fusion module. c For coarse-completion decoders, Df To refine the decoder, P coarse To roughly complete the point cloud.

[0072] By using the above coarse-to-fine decoding generation method, we can take into account both the overall trend recovery of the reference surface of the pit area and the reconstruction of local details of the pit boundary, reduce the problems of boundary breakage, steps and excessive flatness of the pit bottom in the completed surface, and make the completed result more suitable as a reference surface for subsequent pit volume measurement.

[0073] S5. Construct a composite loss function for pothole volume measurement and train the network.

[0074] This application constructs a composite loss function for pothole volume measurement. The composite loss function includes EMD approximation loss based on the Sinkhorn algorithm, boundary geometric constraint loss, and volume consistency loss, which are used to simultaneously constrain the overall point cloud distribution, pothole boundary continuity, and volume measurement stability.

[0075] Let the predicted point cloud be: ; The true point cloud is: ; Where n is the total number of points in the predicted point cloud, m is the total number of points in the ground truth point cloud, and the i-th point in the predicted point cloud is p1. i The j-th point in the real point cloud is q1. j .

[0076] Further, step S5 includes: S51. Construct a cost matrix between the predicted point cloud and the real point cloud, wherein the cost matrix is ​​used to characterize the geometric distance relationship between any predicted point and any real point. S511. First, construct the cost matrix between the two point sets: ; The cost matrix is ​​then expressed as: ; Define the discrete probability distributions of the predicted point cloud and the true point cloud as follows: ; ; In the case of uniform distribution: ; Let the transfer matrix be: ; And satisfy edge constraints: , ; in, and These are all column vectors containing only 1s.

[0077] S52. Introduce an entropy regularization term into the optimal transmission problem to construct an optimal transmission model with entropy regularization; ; in, ; ; ; is the entropy regularity coefficient.

[0078] Further define the kernel matrix: ; The transfer matrix can then be represented as: ; in, , For iterative scaling of vectors, This means transforming the input vector into a diagonal matrix with the elements of that vector as the main diagonal elements.

[0079] S53. Solve the approximate optimal transfer matrix between the predicted point cloud and the real point cloud through Sinkhorn iteration; The Sinkhorn iterative update formula is: ; ; The division is performed element-wise. After iterative convergence, an approximately optimal transfer matrix is ​​obtained: ; S54. Calculate the EMD approximation loss based on the Sinkhorn algorithm: Based on the cost matrix constructed in step S51 and the approximate optimal transfer matrix obtained in step S53, calculate the approximate EMD loss value between the predicted point cloud and the real point cloud.

[0080] Let the cost matrix constructed in step S51 be: ; The approximate optimal transfer matrix obtained through Sinkhorn iteration in step S53 is: ; The approximate loss for EMD based on the Sinkhorn algorithm is defined as follows: ; Among them, C ij It can reflect the geometric distance cost between the i-th point in the predicted point cloud and the j-th point in the real point cloud. This represents the approximately optimal transmission weight for the transmission from the i-th point in the predicted point cloud to the j-th point in the real point cloud.

[0081] By using the above-mentioned EMD approximation loss, the matching relationship between the predicted point cloud and the real point cloud can be constrained from the overall distribution relationship, reducing the problems of uneven point cloud distribution, boundary collapse and local shape distortion caused by relying solely on nearest neighbor distance.

[0082] S55. Construct the boundary geometric constraint loss in the composite loss function.

[0083] To improve the continuity and stability of the completed point cloud in the pit boundary region, in addition to the EMD approximation loss, boundary position continuity, normal continuity, curvature continuity, and smoothing regularization terms are further constructed. Through these constraints, the completed point cloud simultaneously satisfies the requirements of position continuity, orientation continuity, curvature change continuity, and local topography smoothness within the pit boundary transition zone.

[0084] S551, Boundary Position Continuity Term: A boundary transition zone is constructed on both sides of the pit boundary, the boundary transition zone including an inner transition zone located inside the pit area and an outer transition zone located outside the intact surface area.

[0085] Let the set of points in the completed point cloud within the boundary transition zone be: ; The set of neighboring points of the original intact surface within the boundary transition zone is: ; The continuous term at the boundary position can then be expressed as: ; Where p2 represents the point within the transition zone of the completed point cloud boundary, and q2 represents the neighboring point within the transition zone of the original complete surface boundary.

[0086] S552, Normal continuity loss: To ensure better directional continuity between the completed reference surface and the original complete surface at the pit boundary, the normal vector deviation between the completed point and its neighboring original point is calculated.

[0087] Let the normal vector of the completed point p2 be n. p The normal vector of its neighboring original point q2 is n. q Then the normal continuity loss can be expressed as: ; Where, n p ·nq This represents the dot product of two normal vectors.

[0088] S553, loss of curvature continuity: To reduce the abrupt curvature change between the completed reference surface and the original intact surface at the pit boundary, the local curvature deviation between the completed point and its neighboring original point within the boundary transition zone is constrained. Let the local curvature of the completed point be k. p The local curvature of its neighboring original point is k. q Then the curvature continuity loss can be expressed as: ; Alternatively, it can be expressed in the form of squared error: ; By using curvature continuity loss, problems such as abrupt curvature changes, local bulges, or boundary collapses at the boundaries of potholes can be reduced in the completed surface.

[0089] S554, Local Smoothing Regularized Loss: To reduce local noise, spikes, and discontinuities in the completed point cloud, a local smoothing regularization term is constructed. Let N(P2) be the set of neighborhood points of the completed point p2, then the local smoothing regularization loss can be expressed as: ; Where r is a neighborhood point of point p2, w pr The neighborhood weights can be expressed as Gaussian weights. ; in, The parameter used to control the degree of neighborhood weight decay.

[0090] S555, Boundary geometric constraint loss: The boundary geometric constraint loss is composed of the boundary position continuity term, normal continuity loss, curvature continuity loss, and smoothing regularization loss. ; Among them, L pos For the continuous term at the boundary position, L normal For normal continuity loss, L curv For curvature continuity loss, L smooth For local smoothing regularization loss, arrive These are the weighting coefficients.

[0091] By using the aforementioned boundary geometric constraint loss, problems such as steps, cracks, boundary shrinkage, abrupt curvature changes, and local discontinuities can be reduced at the boundaries of the completed point cloud, thereby improving the continuity of the completed reference surface and its stability for volume measurement.

[0092] S56. Construct the volume consistency loss and train the network: To enable the point cloud completion results to directly serve the pothole volume measurement, a volume consistency loss is further constructed during the training process.

[0093] During the training phase, a pit-missing region is simulated based on complete surface point cloud samples to obtain incomplete input point clouds, real complete point clouds, and real pit volume labels. The real pit volume labels are calculated from the closed difference volume formed between the complete surface and the simulated pit surface.

[0094] The completed point cloud output by the network is used to construct a prediction reference surface, the simulated pit defect surface is used to construct the actual defect surface, and the predicted pit volume is calculated based on the closed difference volume formed between the prediction reference surface and the actual defect surface.

[0095] Let the predicted crater volume be: V pred ; The actual pothole volume label is: V true ; The loss of volume uniformity can then be expressed as: ; in, To prevent extremely small constants with a denominator of zero.

[0096] Alternatively, the volume uniformity loss can also be expressed as a squared error: ; Finally, the total loss function for network training is: ; Among them, L emd For the EMD approximation loss based on the Sinkhorn algorithm, L geo For boundary geometric constraint loss, L volume For loss of volume consistency, , , These are the weighting coefficients.

[0097] The boundary-guided dual-branch feature enhancement point cloud completion network is trained based on the total loss function until the network converges, thus obtaining the trained pit point cloud completion model.

[0098] Through the training method described above, the completion network can not only learn the overall geometric distribution relationship between the predicted point cloud and the real complete point cloud, but also maintain the geometric continuity of the pit boundary region and make the completion reference surface more suitable as the reference surface for subsequent pit volume measurement.

[0099] It should be noted that the volume consistency loss is used during the model training phase. During the actual detection phase, it is not necessary to provide a true pit volume label. The trained point cloud completion model only outputs the point cloud of the completed region based on the point cloud to be completed and its boundary guiding features, and then calculates the pit volume based on the closed difference volume formed between the completion reference surface and the actual defect surface.

[0100] S6. Perform pit point cloud completion: The point cloud to be completed and its corresponding boundary guiding features are input into the trained point cloud completion model, which outputs the point cloud of the completed region. The point cloud of the completed region is then stitched or registered with the original complete region point cloud to obtain the complete surface point cloud after the pothole repair.

[0101] Further, step S6 includes: S61. Input the residual defect cloud, the set of pit boundary points, the defect neighborhood point cloud, and the boundary guidance features.

[0102] S62. The predicted and completed point cloud is obtained through the encoding module, the dual-branch multi-scale feature enhancement module, the gating fusion module and the decoding generation module.

[0103] Let the original complete region point cloud be: P valid ; The completed point cloud region is as follows: P comp ; The fused complete point cloud is then defined as: ; S63. Perform boundary alignment, duplicate point removal, and local smoothing on the predicted and completed point cloud. A boundary transition zone is constructed on both sides of the pit boundary. The boundary transition zone includes an inner transition zone located inside the pit area and an outer transition zone located outside the intact surface area. The width of the inner and outer transition zones can be determined based on the maximum opening size of the pit, the average point spacing of the point cloud, or a preset distance threshold; preferably, the width of the transition zone is 3 to 10 times the average point spacing.

[0104] For the completed points within the boundary transition zone, calculate their positional deviation, normal vector angle deviation, and curvature deviation from the adjacent original complete surface points, and combine them with local smoothing regularization terms to optimize the boundary continuity of the completed point cloud.

[0105] ; Wherein, weight w ij Gaussian weighting can be used: ; This step improves the geometric continuity between the completed region and the original complete region, reducing the impact of splicing errors on subsequent reference surface reconstruction and volume measurement processes.

[0106] The local smoothing process employs a weighted averaging method to adjust the positions of points within the boundary transition zone. Specifically, for each completed point within the boundary transition zone, its neighborhood set is taken, and a weighted average of the neighborhood points is calculated using Gaussian weights to update the point's position. The degree of decay of the Gaussian weights is determined by a parameter... The value of the control can be set according to the average point spacing of the point cloud. Through the above local smoothing process, the slight misalignment and local undulation at the junction of the completed area and the original area can be effectively eliminated, improving the overall geometric continuity of the final complete surface point cloud.

[0107] S64. Embed the processed and completed point cloud into the corresponding position of the original point cloud to form a complete surface point cloud.

[0108] In some implementations, to improve the naturalness of the transition at the pit boundary, the transition zone between the completed area and the original area can be optimized to ensure that the splicing meets the geometric continuity requirements.

[0109] S7. Construct a pit reference surface based on the completion results and perform volume measurement: When calculating the volume of a pit, a unified projection region is first determined using the set of pit boundary points as constraints. Then, the complete reference surface and the actual defect surface are mapped onto the unified projection region to ensure that they have consistent integral boundaries. Finally, the volume of the pit is calculated based on the height difference between the reference surface and the actual defect surface or the closed mesh difference volume.

[0110] When the reference surface and the actual defect surface can be represented as height functions on a uniform projection area, the height difference integral method is preferred; when the pit morphology is complex or there are non-single-valued curved surfaces, the closed triangular mesh tetrahedral decomposition method or the voxel integral method is used.

[0111] Further, step S7 includes: S71. Based on the completed surface point cloud, reconstruct the reference surface within the pit area; Let the triangular mesh of the reference surface after completion be represented as: ;in, K represents the k-th triangular facet in the triangular mesh. ref The total number of triangular faces in the triangular mesh of the reference surface.

[0112] S72. Based on the original defect point cloud, reconstruct the actual defect surface within the pit area; Let the triangular mesh of the original defect surface be represented as: ;in, K represents the l-th triangular facet in the triangular mesh. def This represents the total number of triangular faces in the triangular mesh of the actual defect surface.

[0113] S73. Based on the set of pit boundary points, establish a closed sidewall region between the reference surface and the actual defect surface to form a closed geometry; Let the closed mesh of the sidewalls constructed based on the pit boundary be: M b ; S74. Mesh the closed geometry; The closed geometry mesh is then represented as: ; S75. Calculate the pit volume based on the representation of the reference surface and the actual defect surface: Based on the geometric representations of the completed reference surface and the actual defect surface, the corresponding method for calculating the pit volume is selected. When both the completed reference surface and the actual defect surface can be represented as height functions on a unified projection area, the height difference integral method is preferred for calculating the pit volume. When the pit morphology is complex, the surface contains non-single-value regions, or it is not convenient to establish a unified height function, the closed triangular mesh tetrahedral decomposition method or the voxel integral method is used to calculate the closed difference volume.

[0114] In a preferred embodiment, both the complete reference surface and the actual defect surface can be represented as a uniform projection area. If the height is a function of the height, then the volume of the pit can be expressed as: ; After discretization, it can be represented as: ; Among them, h ref To complete the reference surface height, h def The original actual surface height of the defect. and For discrete grid step size, Let M′ be the uniform projection region defined by the pit boundary, and N′ be the number of discrete grids divided in the projection region along the u direction.

[0115] In another implementation, when the pit morphology is complex or the reference surface and the actual defect surface are difficult to represent uniformly as a height function, a closed triangular mesh is constructed based on the completed reference surface, the actual defect surface, and the pit boundary sidewalls. The volume of the closed difference volume is then calculated using the tetrahedral decomposition method. For any triangular facet in the closed mesh, its tetrahedral oriented volume with the origin is: ; The volume of the pit is the absolute value of the sum of the directed volumes of the corresponding tetrahedrons for all the triangular facets: ; Where K is the total number of triangular faces in the closed mesh, p k1 p k2 p k3 These are the three vertices of the k-th triangular facet.

[0116] In the above manner, this application can select an appropriate volume calculation method according to the geometric complexity of the pit area: for pit areas that can be projected as a height field, the height difference integral method is used to improve the calculation efficiency; for complex pit morphology, the closed triangular mesh tetrahedral decomposition method or voxel integral method is used to improve the adaptability and stability of volume calculation.

[0117] It should be noted that when both the reference surface for completion and the actual defect surface are single-valued surfaces, a unified projection area can be established through projection. When a one-to-one mapping relationship is established, the height difference integral method can significantly reduce computational complexity and improve the efficiency of volume measurement. When pits have complex morphological features such as overhangs, concavities, or lateral erosion, resulting in multi-value regions on the reference surface or defect surface in the projection direction, the height function representation will fail. In this case, using the tetrahedral decomposition method to perform volume integration on the closed mesh can ensure the theoretical accuracy and engineering robustness of the volume calculation.

[0118] S8. Output the results and visualize them: Output the complete point cloud after pothole completion, pothole boundary information, pothole area reference surface, pothole volume calculation results, and corresponding 3D visualization image or inspection report.

[0119] Furthermore, the visualization includes at least one of the following: S81. Comparison of point cloud before and after completion; S82, Pothole boundaries are highlighted; S83, Overlay display of reference surface and defect surface; S84, Output of pit volume values ​​and statistical information.

[0120] Figure 7 This is a visualization of the point cloud and pit volume results before and after completion in this application. (Example:) Figure 7As shown, the left side displays the defect cloud before completion, where significant data gaps exist in the pit areas. The middle side displays the complete surface point cloud after completion, where the point cloud of the completed area is stitched and merged with the original complete area point cloud, completely restoring the pit areas. The right side shows the volume measurement results, where the reference surface formed by the point cloud of the completed area serves as the upper boundary, the actual defect surface formed by the original defect point cloud serves as the lower boundary, and the sidewalls enclosed by the pit boundary close the upper and lower surfaces, forming a closed geometry. The pit volume V is calculated through integration. This visualization method can intuitively show the morphological comparison before and after completion, the spatial relationship of volume measurement elements, and the quantitative evaluation results.

[0121] This application also proposes a boundary-guided bi-branch enhanced point cloud pit completion measurement system for performing the above-described boundary-guided bi-branch enhanced point cloud pit completion measurement method. The system includes: The point cloud preprocessing module is used to denoise, downsample, and normalize the raw point cloud to obtain a standardized point cloud. The pothole identification and feature enhancement module is used to extract the pothole boundary point set, and construct boundary guiding features for the point cloud to be completed based on the pothole boundary point set. The boundary guiding features are then concatenated with the three-dimensional coordinate features of the points to form an enhanced input point cloud. The boundary guiding features include boundary markers, normalized distance from the point to the pothole boundary, local curvature, normal vector deviation, and local depth deviation. A boundary-guided dual-branch feature-enhanced point cloud completion network is used to process the enhanced input point cloud and output a completed region point cloud; the point cloud completion network includes: The encoding module is used to encode the enhanced input point cloud to obtain multi-level geometric features; The dual-branch multi-scale feature enhancement module includes a group convolution branch and a multi-scale heterogeneous convolution branch, wherein the multi-scale heterogeneous convolution branch includes a small neighborhood branch, a medium neighborhood branch, and a large neighborhood branch. The gated fusion module is used to generate fusion weights based on the boundary guidance features in the enhanced input point cloud, and to adaptively weight and fuse the output features of the group convolutional branch and the multi-scale heterogeneous convolutional branch to obtain fused features. The decoding and generation module is used to output the point cloud of the completed region based on the fused features through a coarse-to-fine hierarchical generation method; The post-processing and stitching module is used to stitch the point cloud of the completed region with the point cloud of the original complete region, and to perform local smoothing processing on the transition zone of the pit boundary to obtain the complete surface point cloud after the pit is repaired. The volume measurement module is used to use the completed region as a reference surface, the point cloud of the original pit region as the actual defect surface, and the set of the pit boundary points to construct a sidewall closed region and form a closed geometry. The volume of the closed geometry is calculated by the height difference integral method or the tetrahedral decomposition method to obtain the pit volume measurement result. The results output and visualization module is used to output the completed point cloud, the pothole volume measurement results, and the visualization image.

[0122] The key innovations of this application and their mechanisms of action.

[0123] To make the innovativeness of this application clearer, the key technical points and corresponding mechanisms of action of this application are explained as follows.

[0124] 1. An enhanced input mechanism based on pit boundary guidance features.

[0125] This application does not directly input ordinary residual point clouds into the point cloud completion network, but rather, based on the identification of pit regions and the extraction of pit boundaries, further constructs pit boundary guiding features. The pit boundary guiding features include at least one of the following: boundary marker, normalized distance from a point to the pit boundary, local curvature, normal vector deviation, and local depth deviation.

[0126] By concatenating the aforementioned boundary guidance features with the 3D coordinate features of the point cloud to form an enhanced input point cloud, the completion network can obtain prior information such as the location of the pit boundary, the degree of boundary abrupt change, the pit transition region, and the pit bottom region during the feature extraction stage. Thus, the network can distinguish between the pit boundary region, the boundary transition region, and the pit bottom region, avoiding the indiscriminate completion of pit regions as ordinary missing regions.

[0127] This mechanism can improve the completion network's ability to perceive the contours and continuity of pit edges, and reduce the probability of retraction, breakage, steps and edge collapse at the pit boundaries of the completed surface, thus providing a reliable input basis for the subsequent construction of a stable pit reference surface.

[0128] 2. A dual-branch feature enhancement structure that combines group convolution, multi-scale heterogeneous convolution branches, and gated fusion.

[0129] This application sets up a group convolution branch and a multi-scale heterogeneous convolution branch in the point cloud completion network to form a dual-branch multi-scale feature enhancement structure. Among them, the group convolution branch is used to divide the input feature channels into multiple groups and perform convolution operations on them separately, so as to reduce channel redundancy and feature coupling complexity, while enhancing the local structure discrimination ability, so that the detailed features near the pit boundary are preserved in the subsequent completion process.

[0130] The multi-scale heterogeneous convolutional branches include small-neighborhood branches, medium-neighborhood branches, and large-neighborhood branches. The small-neighborhood branch extracts local abrupt changes at the pit boundary, the medium-neighborhood branch extracts the transition surface features between the pit edge and the pit bottom, and the large-neighborhood branch extracts the overall curvature trend features of the intact surface surrounding the defect. Through the synergistic effect of different neighborhood scales, the network can simultaneously acquire local boundary details, pit transition surface changes, and the overall geometric prior of the surrounding intact surface.

[0131] Furthermore, this application includes a gated fusion module for adaptively fusing the output features of group convolutional branches and multi-scale heterogeneous convolutional branches. The gated fusion module generates fusion weights based on boundary-guided features such as the normalized distance from the point to the pit boundary, local curvature, and normal vector deviation. This ensures that the pit boundary region retains more local abrupt changes and detailed features, the pit transition region retains more surface continuity features, and the pit bottom and surrounding intact surface region retain more overall geometric trend features.

[0132] Through the synergistic effect of the above-mentioned group convolutional branches, multi-scale heterogeneous convolutional branches, and gated fusion modules, this application can form differentiated feature representations for the boundary region, transition region, and bottom region of the pit, overcoming the problem that the single path feature extraction method is insufficient in representing the complex morphology of the pit, reducing the problems of breakage, steps, shrinkage, and excessive flatness at the pit boundary in the completed point cloud, and making the completed result more suitable as a reference surface for subsequent pit volume measurement.

[0133] 3. Composite loss function for volume measurement.

[0134] This application does not merely use a single chamfer distance or a single EMD loss to constrain the point cloud completion results, but instead constructs a composite loss function for pothole volume measurement. The composite loss function includes an EMD approximation loss based on the Sinkhorn algorithm, a boundary geometric constraint loss, and a volume consistency loss. The boundary geometric constraint loss is weighted by a boundary position continuity term, a normal continuity loss, a curvature continuity loss, and a smoothing regularization loss.

[0135] Among them, the EMD approximation loss based on the Sinkhorn algorithm is used to constrain the overall distribution consistency between the predicted point cloud and the real point cloud; the boundary position continuity term is used to constrain the position continuity between the completed point cloud and the original complete point cloud at the pit boundary; the normal continuity loss is used to constrain the directional continuity between the completed reference surface and the original surface; the curvature continuity loss is used to constrain the continuity of curvature change at the pit boundary; and the smoothing regularization loss is used to reduce local noise and discontinuous fluctuations in the completed point cloud.

[0136] Through the aforementioned composite loss function, the optimization objective of the completion network is no longer limited to the similarity of point cloud morphology, but is further constrained towards the task of pit volume measurement, so that the completion result can be used more stably as a reference surface before defect formation, thereby improving the reliability of subsequent pit volume measurement.

[0137] 4. A method for calculating the closed difference volume based on the complete reference surface and the actual defect surface.

[0138] This application couples the point cloud completion result with the pit volume measurement process. Instead of directly fitting or interpolating the original pit point cloud to calculate the volume, it uses the reference surface before the defect formation obtained by the completion network as the upper boundary, the original pit defect surface as the lower boundary, and constructs lateral closure constraints with the pit boundary point set to form a closed difference volume.

[0139] In the volume calculation process, firstly, a unified projection area or a unified closed boundary is determined based on the set of pit boundary points, so that the complete reference surface and the actual defect surface have the same calculation range; then, the volume of the pit defect is calculated based on the integral of the height difference between the reference surface and the actual defect surface, or based on the tetrahedral decomposition method of the closed triangular mesh.

[0140] Through the above method, this application transforms the point cloud completion result from "visual integrity restoration" to "volume measurement reference surface construction," enabling the completed point cloud to directly participate in the quantitative assessment of defect volume. Compared with the method of directly fitting a reference surface based on the original defect point cloud, this method can reduce the impact of boundary sparsity, occlusion, and pit bottom noise on the volume calculation results, and improve the stability and engineering applicability of volume measurement of complex pit defects.

[0141] The present application will be further described below with reference to specific embodiments, but the scope of protection of the present application is not limited to the following embodiments.

[0142] Example 1: Training of a pothole point cloud completion model.

[0143] First, several high-quality point cloud samples containing complete surfaces are collected. Missing regions at different locations, scales, and depths are then simulated and constructed on these complete point cloud samples to form the required incomplete / imperfect point cloud-real complete point cloud sample pairs for training. Next, the sample point clouds are denoised, downsampled, and normalized. The processed incomplete / imperfect point cloud, the set of pit boundary points, and boundary-guided features are then input into a boundary-guided dual-branch feature-enhanced point cloud completion network. During training, the network parameters are optimized using a composite loss function consisting of Sinkhorn-EMD loss, boundary geometric constraint loss, and volume consistency loss until the model converges, resulting in a trained model for pit completion.

[0144] Example 2: Point cloud completion of actual crater.

[0145] A 3D scan of the workpiece surface is performed to obtain the original point cloud. After filtering, denoising, and normalization, the pit region and its boundaries are extracted to form a point cloud to be completed. The point cloud to be completed, the pit boundary point set, boundary distance features, curvature features, and normal vector deviation features are input into a trained point cloud completion model, outputting the completed area point cloud. Subsequently, boundary alignment and local smoothing are performed between the completed point cloud and the original point cloud to obtain the complete surface point cloud.

[0146] Example 3: Measurement of pit volume.

[0147] After obtaining the complete surface point cloud, the reference surface formed by the completed region is used as the reference surface before the defect formation, and the actual defect surface formed by the point cloud of the original pit region is used as the actual surface under the defect state. A closed sidewall region is constructed based on the pit boundary to form a closed geometry. This closed geometry is then triangularly meshed, and its volume is calculated using tetrahedral decomposition volume integration; the result is the pit volume.

[0148] Example 4: Further explanation of the dual-branch feature enhancement structure and gating fusion mechanism.

[0149] In this embodiment, the boundary-guided dual-branch feature enhancement point cloud completion network includes a group convolutional branch, a multi-scale heterogeneous convolutional branch, and a gated fusion module. The group convolutional branch is used to reduce the coupling complexity between feature channels, reduce redundant computation, and enhance the local structure discrimination ability. The multi-scale heterogeneous convolutional branch includes a small neighborhood branch, a medium neighborhood branch, and a large neighborhood branch. The small neighborhood branch focuses on extracting local abrupt change features at the pit boundary, the medium neighborhood branch focuses on extracting the transition surface features between the pit edge and the pit bottom, and the large neighborhood branch focuses on extracting the overall curvature trend features of the intact surface around the defect. In one implementation, the small neighborhood branch uses a nearest neighbor set of K=8, the medium neighborhood branch uses a nearest neighbor set of K=16, and the large neighborhood branch uses a nearest neighbor set of K=32, or spherical neighborhoods with radii r1, r2, and r3, respectively, where r1 < r2 < r3. The gated fusion module adaptively adjusts the feature weights of each branch based on characteristics such as boundary distance, local curvature, and normal vector deviation. This ensures that the pit boundary region retains more local abrupt changes and detailed features, the transition region retains more surface continuity features, and the pit bottom and surrounding intact surface region retain more overall geometric trend features. Through this structure, the network can establish multi-scale geometric relationships between the pit boundary, transition region, pit bottom region, and surrounding intact surface, reducing problems such as boundary breaks, steps, shrinkage, and excessive flatness of the pit bottom in the completed point cloud. This makes the completed result more suitable as a reference surface for subsequent pit volume measurement.

[0150] Example 5: Further explanation of the composite loss function.

[0151] In this embodiment, the training process of the point cloud completion network is constrained by a composite loss function oriented towards pothole volume measurement. This composite loss function includes Sinkhorn-EMD loss, boundary geometric constraint loss, and volume consistency loss. The boundary geometric constraint loss is weighted by boundary position continuity, normal continuity loss, curvature continuity loss, and local smoothing regularization loss. This composite loss function allows the model training to simultaneously focus on the overall distribution consistency of the predicted point cloud, the continuity of pothole boundary transitions, and the consistency between the predicted volume and the true volume label.

[0152] Example 6: Simulated pit sample construction and volume label generation.

[0153] In this embodiment, to obtain residual point clouds, true complete point clouds, and true pit volume labels for network training, a complete surface point cloud without obvious defects is first acquired, and a local area is randomly selected on the complete point cloud as the simulated pit region. Then, simulated pits with different depths, opening sizes, boundary steepness, and bottom curvatures are generated using a Gaussian concavity function, an ellipsoidal concavity function, or a random perturbation concavity function. For example, a Gaussian concavity function can be used: ; A simulated pit is constructed, where A represents the pit depth, a and b represent the pit opening dimensions, and (x0, y0) represents the pit center position. By changing A, a, and b and superimposing random perturbations, various pit morphology samples can be generated. Finally, the original complete point cloud is used as the real complete point cloud, and the point cloud after generating the simulated pit is used as the incomplete input point cloud. The real pit volume label is calculated based on the closed difference volume formed between the complete surface and the simulated pit surface, and used for training the point cloud completion network and the volume consistency loss function.

[0154] The technical scope of this application is not limited to the contents of the above specification. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this application, and all such modifications and variations should fall within the scope of this application.

Claims

1. A point cloud pit completion measurement method with boundary-guided dual-branch enhancement, characterized in that, include: Obtain the original point cloud of the surface of the object under test and perform preprocessing; Extract the set of boundary points of the pits, and construct boundary guiding features for the points in the point cloud to be completed based on the set of boundary points of the pits. Then, stitch the boundary guiding features with the three-dimensional coordinate features of the points to form an enhanced input point cloud. The input point cloud is input into a pre-trained completion network, which outputs a completed point cloud region. The completion process of the completion network includes: Encode the enhanced input point cloud to obtain multi-level geometric features; The feature channels are grouped by group convolution operation to obtain the first branch features; By performing multi-scale heterogeneous convolution operations, features are extracted at at least three different neighborhood scales to obtain the second branch features. Through gated fusion operation, fusion weights are generated based on boundary guiding features in the enhanced input point cloud. Adaptive weighted fusion of the first branch features and the second branch features is then performed to obtain the fused features. Generate complete region point clouds based on fusion features; The point cloud of the repaired area is stitched together with the point cloud of the original complete area to obtain the complete surface point cloud after the pothole is repaired; the pothole volume is calculated based on the complete surface point cloud and the point cloud of the original pothole area to obtain the pothole volume measurement result. The boundary guidance features include boundary markers, normalized distance from a point to the pit boundary, local curvature, and normal vector deviation; the enhanced input point cloud is represented as... , where p i Let b be the three-dimensional coordinates of the i-th point. i For boundary identification, d i Let n be the normalized distance from the point to the boundary of the pit. i For the normal vector deviation characteristic, h i For local depth deviation characteristics, N e To increase the total number of points in the input point cloud, k i For local curvature; The gate fusion operation includes: The normalized distance d from each point in the enhanced input point cloud to the edge of the pit is calculated as follows: i Local curvature k i and normal vector deviation n i Gated weights are generated by mapping weights and using the Sigmoid activation function. ,in For the Sigmoid function, W g For the gated mapping weights, b g For bias terms; according to For the first branch feature F g With the second branch feature F h We perform weighted fusion to obtain the fusion feature F. f ,in This represents element-wise multiplication, and g is used to represent the feature F of the first branch. g The value comes from the group convolution branch and serves as the group index for that branch; h is used to represent the feature F of that second branch. h Derived from multi-scale heterogeneous convolution branches; The at least three different neighborhood scales include small neighborhood, medium neighborhood, and large neighborhood; wherein, the small neighborhood uses a set of nearest neighbors with K=8 or a spherical neighborhood with radius r1, the medium neighborhood uses a set of nearest neighbors with K=16 or a spherical neighborhood with radius r2, and the large neighborhood uses a set of nearest neighbors with K=32 or a spherical neighborhood with radius r3, where r1 < r2 < r3; the second branch feature F h It can be obtained through feature concatenation or weighted summation using learnable weights; Based on the fusion feature F f Generate a complete point cloud of the region, specifically including: Through coarse-completion decoder D c Generate coarsely completed point cloud P coarse =D c (F f ); By refining decoder D f The coarsely completed point cloud and fused features are refined to generate a finely completed point cloud P. fine =D f (P coarse ,F f ), as the point cloud of the completed region; The completion network is trained using a composite loss function for supervision. The composite loss function is: , , , L is the weighting coefficient. emd The EMD approximation loss based on the Sinkhorn algorithm is used to constrain the overall distribution consistency between the predicted point cloud and the real point cloud. The L geo The boundary geometric constraint loss is a weighted sum of the boundary position continuity term, normal continuity loss, curvature continuity loss, and smoothing regularization loss. The L volume The volume consistency loss is used to constrain the predicted pothole volume V calculated based on the completion results. pred Compared with the actual volume label V true Consistency between them; The calculation of the pit volume based on the complete surface point cloud and the original pit region point cloud specifically includes: Using the completed region as a reference surface, the point cloud of the original pit region as the actual defect surface, and the set of pit boundary points to construct a sidewall closed region, forming a closed geometry; Calculate the volume of the closed geometry; where, when using the height difference integral method, the volume is: ， h ref h is the reference surface height. def The height of the defect surface is denoted as M. When using the tetrahedral decomposition method, the volume is the absolute value of the sum of the oriented volumes of each triangular facet. M′ is the number of discrete meshes divided in the projection region along the u direction, and N′ is the number of discrete meshes divided in the projection region along the v direction.

2. The method according to claim 1, characterized in that, The extraction of the pit boundary point set specifically includes: Normal vector and curvature estimation based on the local neighborhood of a point; The suspected pit area is initially segmented based on curvature abrupt change, normal vector change rate, and local depth deviation; wherein, the normal vector change rate is measured by the deviation between the point normal vector and the average normal vector of its neighborhood, and the local depth deviation is represented by the normal distance of the point relative to its neighborhood fitting reference surface. By constructing a comprehensive scoring function The points are judged, and those with scores greater than a preset threshold are identified as candidate pothole points; where k i For local curvature, As a measure of normal vector deviation, For local depth deviation, w1, w2 and w3 are weighting coefficients, and i is the index of the point in its respective point cloud; The set of boundary points of potholes can be extracted from the candidate potholes using region growing, boundary tracking or clustering methods.

3. The method according to claim 1, characterized in that, Based on the fusion feature F f Generate a complete point cloud of the region, specifically including: Through coarse-completion decoder D c Generate coarsely completed point cloud P coarse =D c (F f ); By refining decoder D f The coarsely completed point cloud and fused features are refined to generate a finely completed point cloud P. fine =D f (P coarse ,F f ), which serves as the point cloud of the completed region.

4. The method according to claim 1, characterized in that, The calculation of the EMD approximation loss based on the Sinkhorn algorithm includes: Let p1 be the i-th point in the predicted point cloud. i The j-th point in the real point cloud is q1. j Construct a cost matrix between the predicted point cloud and the actual point cloud. i and j are the indices of the point in its respective point cloud. An entropy regularization term is introduced into the optimal transmission problem, and the approximate optimal transmission matrix is ​​solved through Sinkhorn iteration. , The approximate loss of EMD is calculated using the following formula: Where n is the number of predicted point cloud points and m is the number of actual point cloud points. This represents the approximately optimal transmission weight.

5. A boundary-guided dual-branch enhanced point cloud pit completion measurement system, characterized in that, The system is used to perform the boundary-guided bi-branch enhanced point cloud pit completion measurement method as described in any one of claims 1 to 4, the system comprising: The point cloud preprocessing module is used to denoise, downsample, and normalize the raw point cloud to obtain a standardized point cloud. The pothole identification and feature enhancement module is used to extract the pothole boundary point set, and construct boundary guiding features for the point cloud to be completed based on the pothole boundary point set. The boundary guiding features are then concatenated with the three-dimensional coordinate features of the points to form an enhanced input point cloud. The boundary guiding features include boundary markers, normalized distance from the point to the pothole boundary, local curvature, normal vector deviation, and local depth deviation. A boundary-guided dual-branch feature-enhanced point cloud completion network is used to process the enhanced input point cloud and output a completed region point cloud; the point cloud completion network includes: The encoding module is used to encode the enhanced input point cloud to obtain multi-level geometric features; The dual-branch multi-scale feature enhancement module includes a group convolution branch and a multi-scale heterogeneous convolution branch, wherein the multi-scale heterogeneous convolution branch includes a small neighborhood branch, a medium neighborhood branch, and a large neighborhood branch. The gated fusion module is used to generate fusion weights based on the boundary guidance features in the enhanced input point cloud, and to adaptively weight and fuse the output features of the group convolutional branch and the multi-scale heterogeneous convolutional branch to obtain fused features. The decoding and generation module is used to output the point cloud of the completed region based on the fused features through a coarse-to-fine hierarchical generation method; The post-processing and stitching module is used to stitch the point cloud of the completed region with the point cloud of the original complete region, and to perform local smoothing processing on the transition zone of the pit boundary to obtain the complete surface point cloud after the pit is repaired. The volume measurement module is used to use the completed region as a reference surface, the point cloud of the original pit region as the actual defect surface, and the set of the pit boundary points to construct a sidewall closed region and form a closed geometry. The volume of the closed geometry is calculated by the height difference integral method or the tetrahedral decomposition method to obtain the pit volume measurement result. The results output and visualization module is used to output the completed point cloud, the pothole volume measurement results, and the visualization image.

Images