Three-dimensional point cloud defect detection method and system based on double feature memory bank comparison

Abstract

The application relates to the field of three-dimensional vision and industrial detection data processing technology, and discloses a three-dimensional point cloud defect detection method and system based on double-feature memory bank comparison. Depth data of a to-be-detected object is collected, global point cloud is obtained through preprocessing and multi-view splicing, double-resolution data of the global point cloud is shunted, sparse point cloud is used for registration, full-quantity high-definition point cloud is isolated and reserved, real-scale reconstruction point cloud is obtained through structure feature perception registration, global pose graph optimization and topological restoration, the registered point cloud is rigidly aligned, local and global features are extracted by using a fixed-weight feature extraction operator, the two kinds of features are respectively matched with a pre-constructed local reference feature library and a global reference feature library, neighborhood feature distribution density is counted and abnormally scored according to a matching result, and a defect detection area is output. The application does not need online training, can adapt to machining tolerance and effectively suppress noise, and realizes stable and high-precision defect detection.

Description

Technical Field

[0001] This invention relates to the field of 3D vision and industrial inspection data processing technology, specifically to a 3D point cloud defect detection method and system based on dual feature memory comparison. Background Technology

[0002] The surface integrity and deformation state (dents, damage, scratches, etc.) of mechanical equipment and industrial parts are key indicators for assessing their operational safety and product quality. With the development of 3D vision technology, 3D point cloud geometric reconstruction has become a core monitoring method.

[0003] Existing 3D point cloud defect detection methods are mainly divided into three categories: The first category is the geometric threshold method, which judges anomalies by setting a fixed spatial distance threshold. This method is extremely sensitive to sensor ranging noise and changes in the natural curvature of the surface, and is prone to false alarms. The second category is the CAD template comparison method, which relies heavily on pre-imported standard CAD models as the detection benchmark. When faced with non-standard customized parts lacking drawings or actual parts with reasonable machining tolerances, the template comparison method is difficult to apply, and the micro-accumulated misalignment generated by multi-view scanning and stitching is easily confused with real surface defects. The third category is feature learning methods based on deep learning. These methods usually require the collection of a large amount of real defect data for learning and updating network parameters. However, in actual industrial sites, due to the high product yield, there is often a severe lack of real defect samples, resulting in poor generalization ability of conventional feature learning methods.

[0004] In summary, in complex industrial scenarios, existing 3D point cloud defect detection methods suffer from instability and high false alarm rates due to sensor dynamic noise, microscopic misalignment during multi-view stitching, and manufacturing tolerances. Achieving stable and high-precision defect detection without requiring training on specific objects has become a crucial problem that urgently needs to be solved. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide a three-dimensional point cloud defect detection method and system based on dual feature memory comparison, in order to solve the bottleneck problems of existing detection methods being unstable in complex industrial scenarios, having a high false alarm rate, and requiring a large number of defect samples for online training.

[0006] A 3D point cloud defect detection method based on dual-feature memory comparison includes the following steps:

[0007] Step 1: Collect depth data of the object under test, perform preprocessing and multi-view stitching to obtain the global point cloud of the object under test;

[0008] Step 2: Spatial registration of the global point cloud: The global point cloud is split into dual-resolution data streams, voxel downsampling is performed to obtain a sparse point cloud, and then registration is performed. The full high-resolution point cloud is physically isolated and preserved. Structural feature-aware registration is performed on the sparse point cloud to extract superpoints and construct a geometric structure descriptor. Local relative poses are solved by matching, and the absolute pose is obtained by optimization through the global pose graph. Based on the absolute pose, the full high-resolution point cloud is spatially superimposed and inversely scaled to restore the real-scale reconstructed point cloud.

[0009] Step 3: Perform rigid spatial alignment of the registered point cloud under the global coordinate system; extract local and global features of the point cloud under the test using feature extraction operators;

[0010] Step 4: Perform nearest neighbor matching between the local features and the pre-statically constructed local reference feature library, and perform nearest neighbor matching between the global features and the pre-statically constructed global reference feature library;

[0011] Step 5: Based on the nearest neighbor matching results, statistically analyze the distribution density of neighborhood features in the local reference feature library and / or the global reference feature library, and accordingly reweight the anomaly score to output the final defect detection area.

[0012] A 3D point cloud defect detection system based on dual-feature memory comparison includes:

[0013] The data acquisition and preprocessing module is used to acquire depth data of the object under test, perform preprocessing and multi-view stitching, and obtain the global point cloud of the object under test.

[0014] The spatial registration module is used to perform dual-resolution data splitting on the global point cloud, voxel downsampling to obtain sparse point clouds, and then perform registration calculations, while physically isolating and preserving the full high-definition point cloud; it performs structural feature-aware registration on the sparse point cloud, extracts superpoints and constructs geometric structure descriptors; it matches and solves the local relative pose, and optimizes it through the global pose graph to obtain the absolute pose; based on the absolute pose, it performs spatial overlay and inverse scaling on the full high-definition point cloud to restore the real-scale reconstructed point cloud;

[0015] The feature extraction module is used to rigidly align the registered point cloud to be tested in the global coordinate system; and to extract the local and global features of the point cloud to be tested using feature extraction operators.

[0016] The feature matching module is used to perform nearest neighbor matching between the local features and a pre-statically constructed local reference feature library, and to perform nearest neighbor matching between the global features and a pre-statically constructed global reference feature library.

[0017] The defect detection module is used to statistically analyze the distribution density of neighborhood features in the local reference feature library and / or the global reference feature library based on the nearest neighbor matching results, and then perform probability reweighting on the anomaly score accordingly to output the final defect detection area.

[0018] The beneficial effects of this invention are:

[0019] 1) No online training required, overcoming the industrial bottleneck of insufficient defect samples. This invention uses a pre-built static reference feature library, constructed only based on a small number of qualified standard entity samples or historical detection data. During the detection process, the weights of the feature extraction operators are fixed and not updated, completely eliminating the requirement for online training that iterates model parameters on a large number of real defect samples. This solves the problem of few defect samples in industrial settings and the difficulty in implementing online training.

[0020] 2) Not reliant on CAD templates, adaptable to non-standard customized parts inspection. This invention does not rely on standard CAD models as inspection benchmarks. The reference feature library is derived from physical samples rather than design drawings, which can effectively handle non-standard customized parts without standard drawings.

[0021] 3) Adaptive processing tolerance to reduce false alarm rate. This invention uses a probability reweighting mechanism to dynamically adjust the anomaly scoring weights based on the distribution density of neighborhood features in the reference feature library. In sparsely distributed feature regions (corresponding to regions where processing tolerances are permissible), the scoring weights are automatically reduced to suppress false positives, while in densely distributed feature regions, extremely high anomaly scoring weights are assigned to minor deviations, achieving automatic adaptation to processing tolerances. Furthermore, this invention combines local features (capture position offset) and global features (capture topology disruption / shape distortion) for joint decision-making, further suppressing false alarms caused by sensor noise and splicing misalignment.

[0022] 4) Suppressing noise interference and improving detection stability. The front end of this invention combines TSDF voxel fusion denoising and structural feature perception registration mechanism to effectively suppress sensor dynamic noise during the data acquisition stage. It also uses a dual-resolution data splitting strategy to physically isolate the registration solution from the high-definition point cloud, avoiding damage to the microscopic topology during the registration process and ensuring the accuracy of subsequent feature extraction and comparison. Attached Figure Description

[0023] Figure 1 The flowchart illustrates the overall process of the three-dimensional point cloud defect detection method provided in this embodiment of the invention.

[0024] Figure 2 This is a schematic diagram of dual-resolution data splitting and spatial registration provided in an embodiment of the present invention. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0026] This embodiment provides a method for detecting defects in three-dimensional point clouds based on dual-feature memory comparison, which specifically includes the following steps. Figure 1 The overall process of this method is shown.

[0027] Step 1: Deep data acquisition and preprocessing.

[0028] Depth data of the object under test is collected, preprocessed, and stitched together from multiple perspectives to obtain the global point cloud of the object. Details are as follows:

[0029] a. Image sequence acquisition and hybrid visual odometry:

[0030] Multi-view depth image sequences of the object under test are acquired through RGB-D video streams. Inter-frame motion estimation is performed using a hybrid visual odometry system, which combines photometric consistency constraints and geometric consistency constraints to achieve robust estimation of pose transformations between consecutive frames.

[0031] The RealSense industrial depth camera is used to perform a smooth scan around the small mechanical device to be inspected, acquiring a time-stamped, synchronized RGB-D image sequence. To avoid memory overflow and local drift caused by registering a long sequence all at once, the total length is... The video frames are arranged in a fixed frame rate window. Slice and divide into Each independent local time period.

[0032] In the Within a local time period, the first frame of that segment is selected as the origin of the local coordinate system. For two adjacent frames... and By combining photometric consistency (RGB pixel difference) and geometric consistency (depth point-to-surface distance), a hybrid objective function is constructed. By minimizing this objective function, the local relative pose between two adjacent frames is solved. This allows for the recursive acquisition of the camera's global pose relative to the first frame for each frame within that segment. .

[0033] b. Physical denoising and fusion based on TSDF voxel mesh:

[0034] Voxel fusion technology using TSDF (Truncated Symbol Distance Function) is employed to denoise and fuse multi-frame depth data.

[0035] After acquiring the local inter-frame pose, a truncated symbolic distance field voxel space (TSDF Volume) with a set voxel resolution (Voxel Size = 0.005m) is established. Depth maps of all frames within this time period are then generated. Using the corresponding camera global pose Project onto this voxel space. For any voxel point in the space... The weighted average method is used to update the truncated sign distance value to the nearest object surface. With weight The formula is as follows:

[0036] ;

[0037] ;

[0038] In the formula, and voxel points The truncated symbol distance value and cumulative weight after data fusion of the previous frame; For the current number Frame depth map projected onto voxel points The measured cutoff distance at that location; For the current number The measurement confidence weights of the frames are updated by iterative weighted averaging to obtain the latest truncation symbol distance value and weight for the voxel.

[0039] Limited by the depth camera hardware, single-frame depth maps have severe ranging noise and high-frequency spikes. The purpose of this step is to superimpose the depth data of several frames into the same three-dimensional space and calculate the average value, which physically "smooths out" the high-frequency scanning ranging noise on the surface of mechanical parts.

[0040] c. Fragment point cloud extraction and normal estimation:

[0041] Within the current time period After all frame images are fused, the Marching Cubes algorithm is used to extract continuous 3D point cloud coordinates from the zero isosurface of the TSDF voxel mesh. Then, based on the KD-Tree hybrid search strategy, the geometric normal vector of each point is calculated, outputting a high-precision local fragment point cloud that filters out high-frequency sensor ranging noise and faithfully preserves the true physical deformation characteristics of the part's surface. .

[0042] By traversing all segment sequences, a high-resolution point cloud collection of segments arranged in scan time sequence is finally obtained. .

[0043] Step 2: Dual-resolution data splitting and spatial registration.

[0044] The global point cloud is split into two resolutions, and voxel downsampling is performed to obtain a sparse point cloud. Then, registration is performed, and the full high-definition point cloud is physically isolated and preserved. The sparse point cloud is registered with structural features, superpoints are extracted and geometric structure descriptors are constructed. Local relative poses are solved by matching, and absolute poses are obtained by optimization through the global pose graph. Based on the absolute poses, the full high-definition point cloud is spatially superimposed and inversely scaled to restore the real-scale reconstructed point cloud.

[0045] Figure 2 The process of dual-resolution data splitting and spatial registration is illustrated. The specific process is as follows:

[0046] a. Dual-resolution data offloading for physical topology preservation:

[0047] This step is one of the key innovations of this invention. It involves introducing a dynamic scale mapping factor. The original 3D spatial coordinates are uniformly magnified, and the precision of floating-point coordinates is transferred to the integer domain, thus preserving sub-voxel-level microscopic geometric and topological information in subsequent voxel downsampling. After scaling, the point cloud is voxel downsampled to obtain a sparse point cloud for subsequent registration calculation; simultaneously, the full high-resolution point cloud that did not participate in downsampling is physically isolated and preserved without any processing until registration is complete. Through this dual-resolution data splitting strategy, the efficiency of registration calculation and the fidelity of point cloud topology are decoupled.

[0048] To meet the stringent requirement of maintaining the original topological continuity of the point cloud surface for subsequent microscopic feature extraction, it is essential to avoid downsampling operations during the registration process from disrupting the original surface geometry. Therefore, a dual-resolution data splitting mechanism is established. The lower bound of the standard receptive field of the registration operator is obtained. Diagonal dimension of the physical enclosure of the machine under test Dynamically calculate the scale mapping factor For the first Frame local fragment point cloud Multiply its spatial coordinates by the dynamic scale mapping factor Magnify to obtain full high-resolution magnified point cloud Subsequently, using the set voxel resolution parameters... right Voxel downsampling is performed, and the maximum number of points is limited to obtain a sparse point cloud for network inference and solution. .

[0049] The formula is as follows:

[0050] ;

[0051] ;

[0052] ;

[0053] In the formula, For dynamic scaling factor, The set margin factor; This is the point cloud of the local fragment in frame k output from step 1. This is a scaled-down, full-resolution view of the point cloud. This is a sparse point cloud generated through downsampling. The set voxel resolution parameters, The standard receptive field lower limit size for the registration operator; The diagonal dimension of the physical enclosure of the mechanical equipment under test; The voxel downsampling operation performs downsampling on the magnified full point cloud using a set voxel size, thereby obtaining a sparse point cloud for network inference and solution.

[0054] The generated sparse point cloud It is used only as input to feature extraction operators to solve the absolute transformation matrix of the pose map, and the absolute transformation matrix is ​​forced to act unidirectionally on the physical isolated and preserved full high-definition point cloud, thereby achieving physical restoration without destroying any micro-topology of the original point cloud.

[0055] b. Pose calculation based on structural feature perception and local transformation:

[0056] Structural feature-aware registration is performed on sparse point clouds. Neighboring sparse point clouds are input into a spatial structural feature extraction operator with rigid transformation invariance to extract the set of superpoints in the coarsest layer. and To make feature matching resistant to rigid body transformations, the geometric relationships between superpoints are explicitly encoded: the dual distance embedding between superpoints is calculated separately. Angular embedding between triplet superpoints The fused geometric structure descriptor is obtained by applying the projection matrix and max pooling operations. After obtaining the feature descriptors, for each set of matched superpoints corresponding to local dense point clusters, the local relative transformation matrix is ​​solved by minimizing the weighted squared error. .

[0057] The formula is as follows:

[0058] ;

[0059] ;

[0060] ;

[0061] ;

[0062] In the formula, For overpoint and Dual distance embedding between Hyperparameters for controlling distance sensitivity; To surpass the point , Together they constitute the third superpoint of the triple; For the angle embedding between triplet superpoints, Hyperparameters for controlling angle sensitivity; For the fused geometric structure descriptor, and This is the corresponding projection weight matrix; To obtain the local relative transformation matrix, To match the confidence weights of point pairs, and The coordinates of the matching points for a local dense cluster of points. Let be the spatial rigid body transformation matrix to be optimized. The index number of the matching point pair; It is the arithmetic square of the included angle.

[0063] This formula uses a closed-form solution of weighted singular value decomposition (SVD) for iterative fine-tuning, and directly outputs a high-precision initial relative pose.

[0064] c. Construction and optimization of global pose graph with constrained spatial continuity:

[0065] To ensure the consistency of the overall geometric framework and prevent spatial loop breaks or microscopic misalignments caused by multi-view splicing, a constraint check needs to be performed on each pair of estimated relative transformation matrices. First, the inlier rate of the matched point pairs is calculated. A maximum span sliding window constraint is applied, and an information weight matrix is ​​established only for logical frames that satisfy the connectivity condition. This constructs a global pose graph. Subsequently, a nonlinear global optimization algorithm (LM algorithm) is invoked, using the first frame of the sequence as a reference, to minimize the global residual in the global pose graph and amortize the local cumulative error.

[0066] The formula is as follows:

[0067] ;

[0068] ;

[0069] In the formula, To match the inlier rate of point pairs, Let C be the total number of matching point pairs, and C be the set of all matching point pairs in the local dense point cluster. This is a conditional indicator function, used when the spatial distance error between point pairs is less than the tolerance radius. Take 1 at the time; This represents the set of all high-precision absolute poses obtained after optimization of the global pose graph. For the edge The relative pose measurement residuals on the surface This is the information weight matrix for the corresponding connected frames. and This is the high-precision absolute pose obtained through optimization of the global pose graph. It is the set of all edges in the global pose graph.

[0070] d. Topological continuity basis reconstruction and physical coordinate output:

[0071] After establishing global geometric consistency constraints and obtaining all absolute poses, the sparse point cloud used only for registration feature calculation is discarded. Directly extract the full high-resolution magnified point cloud data that was physically isolated and preserved in the first step without any topological destruction. Compare it with the corresponding absolute pose matrix. Perform rigid body transformation and spatial superposition, and finally apply an inverse scaling factor to the stitched global massive point cloud. To restore the actual physical coordinate space size.

[0072] The formula is as follows:

[0073] ;

[0074] ;

[0075] In the formula, To perform rigid body transformation and complete spatial stacking of the globally magnified point cloud, For the first The absolute pose matrix of the segment point cloud after global optimization; Reconstruct the point cloud at the true scale of the output after applying an inverse scaling factor.

[0076] The process outputs a real-scale reconstructed point cloud. It maintains the original geometric continuity and microstructure consistency of the mechanical equipment surface, completely eliminates the interference of discrete faults and splicing distortions, and directly inputs the necessary geometric base data into step S3 and subsequent feature extraction and library matching stages.

[0077] Step 3: Rigid spatial alignment and dual feature extraction.

[0078] The registered point cloud to be tested is rigidly aligned in the global coordinate system; local and global features of the point cloud to be tested are extracted using feature extraction operators.

[0079] For the complete point cloud with global geometric consistency output in step S2, this invention abandons the traditional deterministic spatial threshold determination logic and introduces a reference feature bank (Memory Bank) comparison mechanism. Through feature mapping and probabilistic inference, it achieves keen detection and localization of deformation defects. The specific process is as follows: Figure 2 As shown.

[0080] a. Rigid spatial alignment between the point cloud to be measured and the reference coordinate system:

[0081] The reconstructed point cloud is rigidly spatially aligned with the point cloud in the reference coordinate system. A coarse alignment is performed using the RANSAC (Random Sample Consensus) algorithm to remove obvious outliers and isolated values. Subsequently, a fine registration is performed using the ICP (Iterative Closest Point) algorithm, outputting the aligned point cloud to be measured. This step aims to eliminate overall translation and rotation errors caused by different initial scanning viewpoints, ensuring that the physical coordinates of the point cloud to be measured are rigorously mapped to the same standard vector space of the reference feature library, thus generating the aligned point cloud to be measured.

[0082] The formula is as follows:

[0083] ;

[0084] ;

[0085] In the formula, Reconstruct the global point cloud to be tested (i.e., the real-scale point cloud finally output in step 2). For the global point cloud to be tested The first in Three-dimensional point coordinates; The coordinates of the corresponding point matched in the standard reference sample; and These are the three-dimensional rotation matrix and translation vector to be solved, respectively; The convergence weights are adaptively assigned based on the angle between the point and the normal vector. The global alignment pose matrix that minimizes the weighted squared error; To produce a spatially aligned point cloud for the final output, This represents the total number of points in the global point cloud to be tested. and These are the pose matrices aligned globally. The optimal three-dimensional rotation matrix and translation vector obtained from the decomposition.

[0086] b. Multi-scale dual feature extraction based on feature extraction operators:

[0087] Using pre-trained feature extraction operators, local and global features are extracted from each point in the point cloud to be tested. The local features are the point's three-dimensional spatial coordinates and the geometric statistics of its local neighborhood; the global features are extracted using a high-dimensional semantic encoder with a fixed receptive field to capture the structural topology and shape distortion information of the point cloud. It is important to note that the weights of the feature extraction operators are fixed during the feature extraction stage and are not updated online to ensure consistency in the feature space between the reference feature library and the features to be tested.

[0088] Spatial Alignment of Point Clouds The input feature extraction operator performs a dual-feature decoupling operation. This feature extraction operator can be implemented by a pre-trained model, performing only forward inference during online detection. First, for each point in the point cloud, its three-dimensional physical space absolute coordinates are extracted as local features. First, it is used to characterize microscopic localization attributes; second, high-dimensional semantic features of the point within its local receptive field neighborhood are extracted as global features. It is used to characterize the relationship between macroscopic geometry and spatial structure.

[0089] The formula is as follows:

[0090] ;

[0091] ;

[0092] In the formula, For the extracted local features, their spatial three-dimensional coordinates ( )constitute; For the extracted global features; The mapping function of the feature extraction operator. 3D point coordinates Local spatial neighborhood; The operator weight parameter matrix for this feature extraction operator is fixed in advance and not updated online.

[0093] Step 4: Nearest neighbor matching using dual reference feature libraries.

[0094] The local features are matched with the nearest neighbor of a pre-statically constructed local reference feature library, and the global features are matched with the nearest neighbor of a pre-statically constructed global reference feature library.

[0095] The reference feature library includes a local reference feature library. With global reference feature library It is important to emphasize that this memory is not just for extracting a few specific corners, but is built for the entire surface of the prototype. The algorithm divides the entire 3D prototype into countless point groups and extracts features from all locations. To prevent memory explosion caused by high-density point clouds in industrial applications, the algorithm uses coreset sampling during the offline construction phase. This technique effectively eliminates highly repetitive redundant features, thus perfectly preserving the skeletal feature set covering the entire entity while controlling the size of the memory. For any feature in the point cloud to be tested, the K-nearest neighbor search algorithm is used to independently search for the spatially nearest normal feature in the corresponding dimension's reference feature library, establishing the optimal matching mapping between the feature to be tested and the normal reference basis.

[0096] The formula is as follows:

[0097] ;

[0098] ;

[0099] In the formula, For the local reference feature library and the local features to be tested The optimal matching normal local feature with minimum Euclidean distance; For the global reference feature library and the global features to be tested The optimal match for normal global features is the one with the smallest distance; The L2 norm distance metric represents the feature space.

[0100] Step 5: Probability reweighted anomaly scoring and defect localization.

[0101] Based on the nearest neighbor matching results, the distribution density of neighborhood features in the local reference feature library and / or global reference feature library is statistically analyzed, and the anomaly score is probabilistically reweighted accordingly to output the final defect detection area.

[0102] This step is the core innovation of this invention. Unlike traditional methods that use fixed thresholds or uniform weighting for anomaly scoring, this invention introduces a probabilistic reweighting mechanism. For each local feature of a test point, the distance between it and the optimal matching feature is calculated and converted into a similarity metric using an exponential function, which serves as the numerator. The denominator is the average similarity of the optimal matching feature within its K nearest neighbors, thus measuring the local distribution density of the feature in the reference feature library. The denominator is adjusted by a penalty coefficient to obtain the probabilistically reweighted local anomaly score. This scoring mechanism ensures that: when the feature distribution in the region where the optimal matching feature is located is sparse (corresponding to regions with allowable processing tolerances), the denominator is small but suppressed by the penalty coefficient, and the overall score is lowered to suppress false positives; when the feature distribution in this region is dense (corresponding to geometrically defined non-tolerance regions), the denominator is close to the numerator, and the score remains highly sensitive to small deviations.

[0103] Traditional spatial domain detection relies on rigid deterministic absolute distances, while this step introduces an importance re-weighting mechanism, transforming the detection logic into dynamic probabilistic reasoning. "Point-level scoring" refers to the algorithm assigning coordinates to each individual 3D point on the part's surface when traversing the point cloud. Each feature is assigned an independent anomaly score. In an ideal, defect-free state, the test features completely overlap with the features in the memory bank, and this score theoretically approaches 0.

[0104] For areas with manufacturing tolerances (such as slight offsets in plug pins), the feature distribution of the normal prototype in the memory will be relatively "sparse" or "broad". The reweighting mechanism calculates the relative relationship between the test features and neighboring features, and in probabilistic inference, treats this slight offset as normal fluctuation, thereby dynamically reducing the anomaly score of this area and preventing false alarms. Conversely, for non-tolerance locations such as shells that are highly regular and do not allow tolerance deformation, the feature distribution of the standard prototype in the memory will be extremely "dense" and highly consistent. When the shell under test undergoes a real geometric change (dent or breakage), the extracted features will instantly move away from these dense clusters of normal features in multidimensional space. At this time, the reweighting mechanism will assign an extremely high anomaly confidence to this deviation. Instead of "tolerating" it, it will cause the anomaly score of this area to soar exponentially because it breaks the originally high consistency, making even minor defects on the shell impossible to hide.

[0105] Furthermore, in industrial quality inspection, if at least one point cloud anomaly score significantly exceeds the standard, the test object will be judged as abnormal. To prevent single-point random noise from the sensor from triggering system alarms, a cutoff threshold based on statistical confidence intervals is set to extract... The system identifies a set of high-resolution points indicating potential defects and applies a mean-shift density clustering algorithm to adaptively aggregate discrete high-resolution outliers into physically continuous defect blocks. Finally, it reconstructs the three-dimensional geometric boundaries and outputs the absolute size information.

[0106] The formula is as follows:

[0107] ;

[0108] ;

[0109] In the formula, This is the local anomaly score after reweighting based on the importance of the probability distribution. In this formula, the first part is the basic test feature distance, and the part in parentheses in the second part is the reweighting term: its numerator is the exponent of the test feature distance, and the denominator is... This represents the optimal matching normal local feature in the reference feature library. Recent a set of neighborhood features Distribution density, To traverse the neighborhood feature set Mathematical iteration variables in.

[0110] The penalty coefficient dynamically changes with the density or dispersion of the neighborhood feature distribution to achieve tolerance adaptation. The final comprehensive anomaly score is equal to the local anomaly score. Global anomaly scoring The average of these two factors is chosen for evaluation because local features are extremely sensitive to "positional offset," while global features are extremely sensitive to "topological disruption and shape distortion." Taking the average of the two factors ensures that the algorithm can capture both positional assembly deviations and identify physical damage and deformation, thus forming a comprehensive joint decision.

[0111] For the comprehensive anomaly score, a double cutoff threshold based on statistical confidence intervals is set to extract the set of high-scoring points suspected of defects. Then, a density clustering algorithm is applied to aggregate the discrete high-scoring anomaly points in space into physically continuous defect blocks. Finally, the three-dimensional geometric boundary is reconstructed and the absolute size information is output.

[0112] Comprehensive anomaly score Perform K-nearest neighbor spatial smoothing to eliminate high-frequency isolated noise from the sensor and calculate the smoothing score. :

[0113] ;

[0114] in, For distance points in space Recent A set of neighboring points; The nearest neighbor number parameter is set.

[0115] Set pre-filter score threshold Extracting the required information The points constitute a high-scoring candidate point set.

[0116] For the set of high-scoring candidate points, the Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm is applied within a set clustering search radius. and minimum cluster number This adaptively aggregates discrete outliers with mutually reachable densities in space into physically continuous candidate blocks. .

[0117] To further suppress false defect noise clusters, each candidate block is calculated. Regional average outlier score :

[0118] ;

[0119] in, Candidate blocks Total number of points included.

[0120] Set regional scoring thresholds ,when The candidate block was determined to be a real physical defect block. .

[0121] The 3D physical bounding box parameters extracted for the physical defect blocks are calculated as follows:

[0122] ;

[0123] in, For real physical defect blocks The three-dimensional point coordinates in the data. To output the absolute size boundary of the real physical defect block in three-dimensional space.

[0124] Another embodiment of the present invention provides a three-dimensional point cloud defect detection system based on dual feature memory comparison, comprising:

[0125] The data acquisition and preprocessing module is used to acquire depth data of the object under test, perform preprocessing and multi-view stitching, and obtain the global point cloud of the object under test.

[0126] The spatial registration module is used to perform dual-resolution data splitting on the global point cloud, voxel downsampling to obtain sparse point clouds, and then perform registration calculations, while physically isolating and preserving the full high-definition point cloud; it performs structural feature-aware registration on the sparse point cloud, extracts superpoints and constructs geometric structure descriptors; it matches and solves the local relative pose, and optimizes it through the global pose graph to obtain the absolute pose; based on the absolute pose, it performs spatial overlay and inverse scaling on the full high-definition point cloud to restore the real-scale reconstructed point cloud;

[0127] The feature extraction module is used to rigidly align the registered point cloud to be tested in the global coordinate system; and to extract the local and global features of the point cloud to be tested using feature extraction operators.

[0128] The feature matching module is used to perform nearest neighbor matching between the local features and a pre-statically constructed local reference feature library, and to perform nearest neighbor matching between the global features and a pre-statically constructed global reference feature library.

[0129] The defect detection module is used to statistically analyze the distribution density of neighborhood features in the local reference feature library and / or the global reference feature library based on the nearest neighbor matching results, and then perform probability reweighting on the anomaly score accordingly to output the final defect detection area.

[0130] In summary, the key points of the technical solution of this invention are as follows:

[0131] 1) Collect depth data of the object under test and perform preprocessing and multi-view stitching to obtain high-quality global point cloud.

[0132] 2) The registered point cloud to be tested is rigidly aligned in the global coordinate system, and the local and global features of the point cloud to be tested are extracted using feature extraction operators.

[0133] 3) The extracted features to be tested are matched with the nearest neighbor of the pre-statically constructed reference feature library, the anomaly score is calculated and the final defect detection area is output.

[0134] This invention completely eliminates the requirement for online training on defect sample data. It directly extracts features using feature extraction operators and compares them with a reference feature library derived directly from a small number of qualified standard entity samples or historical inspection data. This not only solves the bottleneck problem of limited defect samples in industrial settings that prevent online training, but also overcomes the strong dependence of traditional methods on standard CAD templates, significantly improving the applicability to non-standard customized parts. Furthermore, this invention combines denoising and registration mechanisms at the front end and adopts an anomaly scoring metric that fuses local and global features at the back end. This greatly suppresses false positives caused by blind spots in the scanning view and splicing misalignment, demonstrating strong anti-noise interference capabilities. Finally, the processing flow constructed by this invention supports cross-scene migration detection and can adapt to dynamic background error fluctuations caused by machining tolerances, effectively improving the accuracy of defect location and system stability under complex working conditions, and giving the system high-precision generalization capabilities.

Claims

1. A method for detecting defects in three-dimensional point clouds based on dual-feature memory comparison, characterized in that, Includes the following steps: Step 1: Collect depth data of the object under test, perform preprocessing and multi-view stitching to obtain the global point cloud of the object under test; Step 2: Spatial registration of global point cloud: The global point cloud is split into dual resolution data streams, voxel downsampling is performed to obtain sparse point cloud, and then registration calculation is performed. The full high-definition point cloud is physically isolated and preserved. Structural feature-aware registration is performed on sparse point clouds to extract superpoints and construct geometric structure descriptors; local relative poses are solved by matching and absolute poses are obtained by optimization through global pose graph; spatial overlay and inverse scaling are performed on the full high-definition point cloud based on the absolute poses to restore the real-scale reconstructed point cloud. Step 3: Perform rigid spatial alignment of the registered point cloud in the global coordinate system; The local and global features of the point cloud to be measured are extracted using feature extraction operators; Step 4: Perform nearest neighbor matching between the local features and the pre-statically constructed local reference feature library, and perform nearest neighbor matching between the global features and the pre-statically constructed global reference feature library; Step 5: Based on the nearest neighbor matching results, statistically analyze the distribution density of neighborhood features in the local reference feature library and / or the global reference feature library, and accordingly reweight the anomaly score to output the final defect detection area.

2. The three-dimensional point cloud defect detection method based on dual-feature memory comparison according to claim 1, characterized in that, The preprocessing described in step 1 includes: a) Inter-frame motion estimation using hybrid visual odometry: A synchronous RGB-D image sequence of the object under test is acquired, and inter-frame motion estimation is performed using hybrid visual odometry. The hybrid visual odometry combines photometric consistency and geometric consistency to construct a hybrid objective function. The local relative transformation matrix between adjacent frames is solved by minimizing the hybrid objective function. b) Denoising using truncated symbolic distance field voxel fusion: A truncated symbolic distance field voxel space is established. Depth maps from each frame are projected into this voxel space using their corresponding poses. The truncated symbolic distance values ​​of voxel points are updated using a weighted average with their respective weights. Sensor ranging noise is suppressed through temporal fusion of multi-frame depth data, and a fused point cloud is output. Specifically, the weighted average update involves: for any voxel point in the space… The weighted average method is used to update the truncated sign distance value to the nearest object surface. With weight As shown in the following formula: ； ； In the formula, and voxel points The truncated symbol distance value and cumulative weight after data fusion of the previous frame; For the current number Frame depth map projected onto voxel points The measured cutoff distance at that location; For the current number Frame measurement confidence weights; c) Extract the point cloud fragments and estimate their normals: The Marching Cubes algorithm is used to extract isosurfaces from the TSDF voxel field, generate local fragment point clouds, estimate their normals, and output a high-fidelity set of local fragment point clouds.

3. The three-dimensional point cloud defect detection method based on dual-feature memory comparison according to claim 1, characterized in that, Step 2, the dual-resolution data splitting and registration calculation includes: Calculate the dynamic scale mapping factor Multiply the spatial coordinates of the global point cloud by The image is magnified to obtain a full-resolution magnified point cloud. Voxel downsampling is then performed on this full-resolution magnified point cloud to obtain a sparse point cloud. The full-resolution magnified point cloud is physically isolated and preserved. The sparse point cloud is used to solve for the absolute transformation matrix of the pose graph. The absolute transformation matrix acts unidirectionally on the full-resolution magnified point cloud. The formula is as follows: ； ； ； In the formula, For dynamic scaling factor, The set margin factor; This is the point cloud of the local fragment in frame k output from step 1. This is a scaled-down, full-resolution view of the point cloud. This is a sparse point cloud generated through downsampling. The set voxel resolution parameters; The standard receptive field lower limit size for the registration operator; The diagonal dimension of the physical enclosure of the mechanical equipment under test; The voxel downsampling operation performs downsampling on the magnified full point cloud using a set voxel size, thereby obtaining a sparse point cloud for network inference and solution.

4. The three-dimensional point cloud defect detection method based on dual-feature memory comparison according to claim 3, characterized in that, In step 2, the specific steps of performing structural feature-aware registration on the sparse point cloud, extracting superpoints, and constructing a geometric structure descriptor are as follows: Input adjacent sparse point clouds into a spatial structure feature extraction operator with rigid transformation invariance to extract a set of superpoints; calculate the dual distance embedding between superpoints: ； in, For overpoint and Dual distance embedding between Hyperparameters for controlling distance sensitivity; Calculate the angular embedding between triplet superpoints : ； in, Hyperparameters for controlling angle sensitivity; To surpass the point , Together they constitute the third superpoint of the triple; The arithmetic square of the included angle value; The geometric descriptor is generated by combining the projection matrix and max pooling. : ； in, and This is the corresponding projection weight matrix; For the local dense point clusters corresponding to the matched superpoints, the local relative transformation matrix is ​​solved by minimizing the weighted squared error. : ； in, To match the confidence weights of point pairs, and The coordinates of the matching points for a local dense cluster of points; Let be the spatial rigid body transformation matrix to be optimized. The index number of the matching point pair.

5. The three-dimensional point cloud defect detection method based on dual-feature memory comparison according to claim 4, characterized in that, In step 2, the global pose graph optimization specifically involves: Calculate the inlier rate of matched point pairs : ； in, The total number of matching pairs. This is a conditional indicator function, used when the spatial distance error between point pairs is less than the tolerance radius. The value is 1; C is the set of all matching point pairs in the local dense point cluster; By applying a maximum span sliding window constraint, an information matrix is ​​established for logical frames that satisfy the connectivity condition, a global pose graph is constructed, and the global residual is minimized through a nonlinear global optimization algorithm. Local cumulative errors are amortized to solve for the absolute pose set. ； in, This represents the set of all high-precision absolute poses obtained after optimization of the global pose graph. For the edge The relative pose measurement residuals on the surface This is the information weight matrix for the corresponding connected frames. and This is the high-precision absolute pose obtained through global graph optimization; It is the set of all edges in the global pose graph.

6. The three-dimensional point cloud defect detection method based on dual-feature memory comparison according to claim 5, characterized in that, The spatial overlay and inverse scaling described in step 2 are specifically as follows: Extract full high-resolution magnified point cloud Compare it with the corresponding absolute pose matrix Perform rigid body transformation and spatial superposition: ； in, To perform rigid body transformation and complete spatial stacking of the globally magnified point cloud, For the first The absolute pose matrix of the point cloud after global optimization, where K is the total number of segments; Apply an inverse scaling factor to the stitched global massive point cloud. To restore the actual physical coordinate space size: ； in, Reconstruct the point cloud at the true scale of the output after applying an inverse scaling factor.

7. The three-dimensional point cloud defect detection method based on dual-feature memory comparison according to claim 6, characterized in that, Step 3 specifically includes: Step 3.1: Rigidly align the point cloud to be measured with the reference coordinate system; A coarse alignment is performed using a random sampling consensus algorithm to remove outliers and isolated values. Then, a fine registration is performed using an iterative nearest-point algorithm, outputting the aligned point cloud of the test object, as shown in the following formula: ； ； In the formula, The global point cloud to be tested is reconstructed at the real scale, which is the final output of step 2. For the global point cloud to be tested The first in Three-dimensional point coordinates; The coordinates of the corresponding point matched in the standard reference sample. For standard reference sample point cloud; and These are the three-dimensional rotation matrix and translation vector to be solved, respectively; The convergence weights are adaptively assigned based on the angle between the point and the normal vector. The global alignment pose matrix that minimizes the weighted squared error; The final output is a spatially aligned point cloud; This represents the total number of points in the global point cloud to be tested. and These are the pose matrices aligned globally. The optimal 3D rotation matrix and translation vector obtained from the decomposition; Step 3.2: Multi-scale dual-feature extraction based on feature extraction operators; Spatial Alignment of Point Clouds In the input feature extraction operator, a dual-feature decoupling operation is performed; firstly, for each point in the point cloud, its three-dimensional physical space absolute coordinates are extracted as local features. First, it is used to characterize microscopic localization attributes; second, high-dimensional semantic features of the point within its local receptive field neighborhood are extracted as global features. This is used to characterize the relationship between macroscopic geometric shape and spatial structure; the formula is as follows: ； ； In the formula, For the extracted local features, their spatial three-dimensional coordinates ( )constitute; For the extracted global features; The mapping function for the feature extraction operator. 3D point coordinates The local spatial neighborhood; The operator weight parameter matrix for this feature extraction operator is fixed in advance and not updated online.

8. The three-dimensional point cloud defect detection method based on dual-feature memory comparison according to claim 1, characterized in that, The construction method of the local reference feature library and the global reference feature library in step 4 is as follows: based on a small number of qualified standard entity samples or historical detection data, redundant features are removed by sampling the core set and then statically constructed. Specifically: For any feature in the point cloud to be tested, the K-nearest neighbor search algorithm is used to independently search for the nearest normal feature in the corresponding dimension of the reference feature library, and to establish the optimal matching mapping between the feature to be tested and the normal reference basis; the formula is as follows: ； ； In the formula, For the local reference feature library and the local features to be tested The optimal matching normal local feature with minimum Euclidean distance; For the global reference feature library and the global features to be tested The optimal match for normal global features is the one with the smallest distance; L2 norm distance metric in feature space; For local reference features, For local reference feature library; As a global reference feature, This serves as a global reference feature library.

9. The three-dimensional point cloud defect detection method based on dual-feature memory comparison according to claim 8, characterized in that, Step 5 specifically includes: Calculate the local anomaly score after reweighting based on the importance of the probability distribution. : ； The first part of the formula is the basic test feature distance, and the part in parentheses in the second part is the reweighting term; its numerator is the exponent of the test feature distance, and the denominator is... This represents the optimal matching normal local feature in the local reference feature library. Recent a set of neighborhood features Distribution density; To traverse the neighborhood feature set Mathematical iteration variables in; The comprehensive anomaly score is obtained by averaging the local anomaly scores and the global anomaly scores. : ； in, A global anomaly score; and These are the weights for local anomaly scores and global anomaly scores, respectively. A double cutoff threshold based on statistical confidence intervals is set to extract a set of high-scoring points suspected of defects. Then, a density clustering algorithm is applied to aggregate the discrete high-scoring anomalies in space into physically continuous defect blocks. Finally, the three-dimensional geometric boundary is reconstructed and the absolute size information is output. Comprehensive anomaly score Perform K-nearest neighbor spatial smoothing to eliminate high-frequency isolated noise from the sensor and calculate the smoothing score. : ； in, The coordinates of a three-dimensional point in space Recent A set of neighboring points; The nearest neighbor number parameter is set; Set pre-filter score threshold Extracting the required information The points constitute a set of high-scoring candidate points; For the set of high-scoring candidate points, a density-based noise-based spatial clustering algorithm is applied within a set clustering search radius. and minimum cluster number This adaptively aggregates discrete outliers with mutually reachable densities in space into physically continuous candidate blocks. ; To further suppress false defect noise clusters, each candidate block is calculated. Regional average outlier score : ； in, Candidate blocks Total number of points included; Set regional scoring thresholds ,when The candidate block was determined to be a real physical defect block. ; The 3D physical bounding box parameters extracted for the physical defect blocks are calculated as follows: ； in, For real physical defect blocks The three-dimensional point coordinates in the data. To output the absolute size boundary of the real physical defect block in three-dimensional space.

10. A three-dimensional point cloud defect detection system based on dual-feature memory comparison, characterized in that, include: The data acquisition and preprocessing module is used to acquire depth data of the object under test, perform preprocessing and multi-view stitching, and obtain the global point cloud of the object under test. The spatial registration module is used to perform dual-resolution data splitting on the global point cloud, perform voxel downsampling to obtain sparse point cloud, and then perform registration calculation, while the full high-definition point cloud is physically isolated and preserved. Structural feature-aware registration is performed on sparse point clouds to extract superpoints and construct geometric structure descriptors; local relative poses are solved by matching and absolute poses are obtained by optimization through global pose graph; spatial overlay and inverse scaling are performed on the full high-definition point cloud based on the absolute poses to restore the real-scale reconstructed point cloud. The feature extraction module is used to perform rigid spatial alignment of the registered point cloud under test in the global coordinate system; The local and global features of the point cloud to be measured are extracted using feature extraction operators; The feature matching module is used to perform nearest neighbor matching between the local features and a pre-statically constructed local reference feature library, and to perform nearest neighbor matching between the global features and a pre-statically constructed global reference feature library. The defect detection module is used to statistically analyze the distribution density of neighborhood features in the local reference feature library and / or the global reference feature library based on the nearest neighbor matching results, and then perform probability reweighting on the anomaly score accordingly to output the final defect detection area.

Images