Three-dimensional industrial assembly anomaly detection method and system based on non-equilibrium optimal transmission and local relationship residual

Abstract

The application discloses a three-dimensional industrial assembly anomaly detection method and system based on non-equilibrium optimal transmission and local relationship residual, and the method comprises the following steps: obtaining a reference point cloud and a to-be-detected point cloud of a three-dimensional assembly object, obtaining point-level local feature anomaly scores, point-level coverage missing scores and point-level relationship anomaly scores of local input features of the aligned reference point cloud and the to-be-detected point cloud respectively, fusing the point-level local feature anomaly scores, the point-level coverage missing scores and the point-level relationship anomaly scores corresponding to each test point in the to-be-detected point cloud, and outputting an object-level anomaly score as a detection result. The application unifies the reference point cloud template normalization, the local defect sensitive feature, the reference coverage missing estimation and the local relationship residual for the logical anomaly detection of the three-dimensional industrial assembly object point cloud, and realizes the point-level positioning and the object-level judgment of the assembly anomalies such as missing, missing parts, misassembly and pose deviation.

Description

Technical Field

[0001] This invention relates to the field of industrial 3D point cloud detection technology, specifically to a 3D industrial assembly anomaly detection method and system based on unbalanced optimal transmission and local relation residuals. Background Technology

[0002] Logical anomaly detection in 3D industrial assembly scenarios differs from visual defects such as scratches, stains, and dents on 2D surfaces. 3D assembly anomalies typically manifest as missing parts, extra parts, incorrect parts, component pose deviations, and inconsistencies in local assembly relationships. These 3D assembly anomalies do not necessarily correspond to new local material categories, nor do they necessarily produce significant texture changes on a single local surface. Rather, they are more often reflected in the lack of coverage, spatial offset, and relationship distortion of the structure under test relative to the normal reference structure.

[0003] In existing technologies, one type of method mainly relies on local geometric features of point clouds or point-to-point distances for anomaly detection. This type of method can detect defects with significant local shape changes, but when the anomaly involves missing parts, missing components, or abnormal assembly relationships, there may be no directly visible anomaly points in the test point cloud, making it easy to miss detections when extracting features only from the test point side. Another type of method relies on comparing the distance between the test point cloud and a standard template after global registration. For example, the non-contact inspection method for complex assembled workpieces based on laser 3D scanning disclosed in Chinese patent application CN117824502A suffers from unstable distance comparisons due to variations in sampling density or outliers interfering with the registration results.

[0004] The main reasons for the above problems are as follows: First, the normal point cloud and the test point cloud must be in a comparable coordinate system. Conventional PCA principal axis alignment is easily affected by axis sign flipping, axis swapping, local missing parts, and uneven sampling. Second, missing parts or abnormalities require judging from the reference template side whether the normal structure has been interpreted by the test observation, rather than just looking for abnormal points from the test point side. Third, misassembly and pose deviation are not only reflected in local distance changes, but also in changes in the relative relationships between parts. Existing 3D anomaly detection methods cannot solve the above problems at the same time.

[0005] Some 3D anomaly detection methods require training classifiers or segmentation networks for specific anomaly types. In real-world industrial assembly scenarios, anomaly samples are rare, complex in type, and costly to collect, making it difficult to construct a dataset covering all anomaly morphologies. Therefore, how to achieve point-level logical anomaly detection without relying on retraining with test samples, given only normal reference templates or a small number of normal samples, is also a problem that existing technologies need to solve. Summary of the Invention

[0006] The technical problem solved by this invention is: to address the difficulty of existing 3D point cloud detection methods in adapting to 3D industrial assembly anomalies, and to provide a 3D industrial assembly anomaly detection method and system based on unbalanced optimal transmission and local relation residuals.

[0007] This invention is achieved using the following technical solution:

[0008] This invention first discloses a three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transport and local relation residuals, comprising the following steps:

[0009] S100: Obtain the reference point cloud and the point cloud to be detected of the 3D assembly object, and align the reference point cloud and the point cloud to be detected to the same coordinate system.

[0010] S200: Extract the local input features of the aligned reference point cloud and the point cloud to be detected, respectively, and input them into the pre-trained defect-sensitive feature extractor. Output the corresponding defect-sensitive local features. Obtain the point-level local feature anomaly score based on the feature difference between the corresponding defect-sensitive local features of the reference point cloud and the point cloud to be detected.

[0011] S300. Construct the local distribution in the neighborhood of the reference center anchor point in the aligned reference point cloud. Soft match the reference quality with the local distribution corresponding to the point cloud to be detected by estimating the reference quality through unbalanced optimal transmission. The point-level coverage missing score of the point cloud to be detected is obtained from the total unmatched reference quality.

[0012] S400. Obtain the point-level relationship anomaly score based on the test-side anchor point relationship vector of the aligned point cloud to be detected and the local relationship residual between the reference point cloud and the corresponding reference-side relationship vector.

[0013] S500: The point-level local feature anomaly score, point-level coverage missing score, and point-level relationship anomaly score corresponding to each test point in the point cloud to be detected are fused together, and the object-level anomaly score is output as the detection result.

[0014] In the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals of the present invention, further, in step S100, aligning the reference point cloud and the point cloud to be detected to the same coordinate system includes the following sub-steps:

[0015] S101. Normalize the reference point cloud and the point cloud to be detected based on their own centroids, and transform them to their respective local coordinate systems.

[0016] S102. Select the outer anchor point and the middle anchor point based on the radial distance quantiles from the reference point cloud and the point cloud to be detected to their respective centroids in their respective local coordinate systems.

[0017] S103. Determine the first principal axis direction based on the direction voting tensor of the outer anchor point, project the middle anchor point onto the orthogonal subspace of the first principal axis to determine the second principal axis direction, and cross-product the first principal axis direction and the second principal axis direction to obtain the third principal axis direction.

[0018] S104. The reference point cloud and the point cloud to be detected are constructed with right-handed orthogonal bases based on their respective centroids and three principal axis directions, and mapped to their respective normalized coordinates. ICP is performed with the reference point cloud as the fixed end and the point cloud to be detected as the moving end to complete the coordinate system normalization, so that the reference point cloud and the point cloud to be detected are aligned in the same coordinate system.

[0019] In the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals of the present invention, the defect-sensitive feature extractor is further based on a lightweight geometric feature encoder, which extracts local patch features of the input point cloud sample through the encoder, including local input channels, convolutional layers, stable geometric branches, defect-sensitive branches, gated fusion units, pooling layers and projection heads;

[0020] The local input channel generates a seven-dimensional local patch feature from the preprocessed point cloud sample. The first three dimensions are the relative coordinates of the point cloud sample after centering and radius normalization, the middle three dimensions are the normal vector of the point cloud sample, and the last dimension is the normalized radius of the point cloud sample.

[0021] The convolutional layer maps the seven-dimensional local patch features into 64-dimensional point features through two 1×1 convolutions, which are then input into the stable geometric branch and the defect-sensitive branch respectively.

[0022] The stable geometric branch outputs stable geometric branch output features through two depth-separable convolutional blocks with 64 channels and a kernel size of 3.

[0023] The defect-sensitive branch adopts a three-scale parallel structure, and through... Convolutional fusion outputs defect-sensitive branch output features;

[0024] The gated fusion unit performs weighted fusion of the stable geometric branch output features and the defect-sensitive branch output features to output a fused feature.

[0025] The pooling layer concatenates the max pooling and average pooling results of the fused features, and obtains the patch descriptor of the local patch features through the projection head.

[0026] In the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transport and local relation residuals of the present invention, the pre-training of the defect-sensitive feature extractor is further optimized using a classification head based on the local patch features output by the lightweight geometric feature encoder and the corresponding patch feature labels. The specific training process is as follows:

[0027] Obtain pre-trained point cloud samples, and perform centering and radius normalization preprocessing based on the centroid of the point cloud samples themselves;

[0028] The preprocessed point cloud samples are input into a lightweight geometric feature encoder, which outputs patch descriptors of local patch features. The classification head then uses classification loss, center constraint, and margin constraint to jointly optimize the defect-sensitive feature extractor based on the patch descriptors of the local patch features and the corresponding patch feature labels.

[0029] The classification loss is the classification loss of the supervised classification head in distinguishing between normal local patch features and abnormal local patch features;

[0030] The center constraint is based on the center loss that makes local patch features of the same category move closer to the center of the corresponding patch feature label category.

[0031] The interval constraint is constrained by the distance difference between the local Patch feature and the normal and abnormal categories in the Patch feature label as the interval loss.

[0032] In the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals of the present invention, the calculation process of the point-level local feature anomaly score is further as follows:

[0033] S201. Sample local patch pairs corresponding to several positions in the reference point cloud and the point cloud to be detected, and input them into the pre-trained defect-sensitive feature extractor to obtain the defect-sensitive local features of the reference point cloud and the point cloud to be detected, respectively.

[0034] S202. Calculate the center-level local feature anomaly score between the defect-sensitive local features at the sampling locations of the reference point cloud and the point cloud to be detected. :

[0035] ;

[0036] and To obtain the defect-sensitive local features of the reference point cloud and the point cloud to be detected at sampling position j, Represents the L2 norm;

[0037] S203. The center-level local feature anomaly score is propagated to each test point in the point cloud to be detected by nearest neighbor interpolation to form a point-level local feature anomaly score.

[0038] In the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals of the present invention, step S300 further includes the following sub-steps:

[0039] S301. Sample a local reference patch at the reference center anchor point in the reference point cloud, and construct a test candidate patch in the point cloud to be detected centered on the same reference center anchor point.

[0040] S302. If there is a local reference patch that is empty, the coverage missing score of the corresponding reference center anchor point in the reference point cloud is set to 0.

[0041] If any of the test candidate patches is empty, the coverage missing score of the corresponding reference center anchor point in the point cloud to be detected is set to 1.

[0042] S303. Construct the squared distance matrix between the reference point and the test candidate point in the corresponding Patch from the non-empty local reference Patch and test candidate Patch.

[0043] The median of the squared distances from each reference point in the local reference patch to the corresponding reference center anchor point is used as the local scale. The squared distance matrix is ​​normalized to a geometric cost matrix. Based on the geometric cost matrix, a Gibbs kernel is constructed to represent the soft matching similarity between the reference points in the local reference patch and the test points in the test candidate patch.

[0044] Define the discrete quality of the reference point in the local reference patch and the test point in the test candidate patch in the corresponding patch. Iterate the unbalanced optimal transmission by updating the scaling vectors of the reference side of the local reference patch and the test side of the test candidate patch to obtain the soft matching matrix from the reference point to the test candidate point. Use the total unmatched reference quality as the anchor point level to cover the missing score.

[0045] S304. The anchor-level coverage missing scores from steps S302 and S303 are propagated to each test point in the point cloud to be detected through nearest neighbor interpolation to form point-level coverage missing scores.

[0046] In the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residual of the present invention, in step S301, a local reference patch of the reference point cloud and a test candidate patch of the point cloud to be detected are constructed using a K-nearest neighbor algorithm with a preset number of neighbors.

[0047] In the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals of the present invention, the point-level relation anomaly score in step S400 is further obtained through the following process:

[0048] S401. Sample several test center anchor points in the point cloud to be detected to construct a nearest neighbor graph, and find the nearest reference nearest neighbor center of the corresponding test center anchor point in the reference point cloud. Obtain the relationship vectors of the point cloud to be detected on the test side and the reference point cloud on the reference side in the nearest neighbor query respectively.

[0049] S402. Define the center-level relation residual of the test center anchor point as the local relation residual by using the residual of the side length ratio of the nearest neighbor side length between the test center anchor point and the reference nearest neighbor center and the direction residual of the relation vector.

[0050] S403. The center-level relationship residual is interpolated to the test points in the point cloud to be detected by means of the nearest neighbor centers to obtain the point-level relationship anomaly score.

[0051] In the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residual of the present invention, in step S500, the point-level local feature anomaly score, the point-level coverage missing score, and the point-level relation anomaly score are fused with equal weights to obtain the final point-level anomaly map of the point cloud to be detected, and the average of the highest 10% of the point-level scores is taken as the output object-level anomaly score of the point cloud to be detected.

[0052] This invention also discloses a three-dimensional industrial assembly anomaly detection system based on unbalanced optimal transport and local relation residuals, used to implement the steps of the above-mentioned method of this invention, including:

[0053] The point cloud alignment module acquires the reference point cloud and the point cloud to be detected of the 3D assembled object, and aligns the reference point cloud and the point cloud to be detected to the same coordinate system.

[0054] The point-level local feature anomaly extraction module embeds a defect-sensitive feature extractor, which extracts local input features of the aligned reference point cloud and the point cloud to be detected, respectively. The extracted features are input into the pre-trained defect-sensitive feature extractor, and the corresponding defect-sensitive local features are output. The point-level local feature anomaly score is obtained based on the feature difference between the corresponding defect-sensitive local features of the reference point cloud and the point cloud to be detected.

[0055] The point-level coverage loss estimation module constructs the local distribution in the neighborhood of the reference center anchor point in the aligned reference point cloud, and performs soft matching between the reference quality estimated by unbalanced optimal transmission and the local distribution corresponding to the point cloud to be detected. The point-level coverage loss score of the point cloud to be detected is obtained from the total unmatched reference quality.

[0056] The point-level relationship anomaly detection module obtains the point-level relationship anomaly score based on the local relationship residual between the test-side anchor point relationship vector of the aligned point cloud to be detected and the corresponding reference-side relationship vector of the reference point cloud.

[0057] The object-level anomaly fusion module fuses the point-level local feature anomaly scores, point-level coverage loss scores, and point-level relationship anomaly scores corresponding to each test point in the point cloud to be detected, and outputs the object-level anomaly score as the detection result.

[0058] The present invention, by adopting the above technical solution, has the following beneficial effects:

[0059] (1) This invention proposes a 3D point cloud detection method for logical anomalies in industrial assembly scenarios, addressing issues such as missing parts, multiple parts, incorrect parts, component pose deviations, and inconsistencies in local assembly relationships. Unlike traditional surface defect detection methods that primarily focus on local appearance anomalies such as scratches, dents, and stains, this invention focuses on determining whether the point cloud to be detected has structural coverage gaps, spatial offsets, and relationship changes relative to a normal reference point cloud template. Therefore, this invention is better suited for logical anomaly detection tasks in 3D assemblies, structural parts, and complex industrial components.

[0060] (2) This invention normalizes the coordinate systems of the reference point cloud and the point cloud to be detected before detection by a robust point cloud normalization coordinate system construction method, so that the reference point cloud and the point cloud to be detected enter a comparable unified space. This method reduces the problems of axis sign flipping, axis exchange, local missing and uneven sampling that are easy to occur in the principal axis alignment of conventional principal component analysis (PCA) through steps such as centroid centering, radial quantile anchor point selection, outer anchor point direction voting, middle anchor point orthogonal projection and small-range ICP refinement. As a result, the alignment stability between the reference point cloud template and the point cloud to be detected can be improved, providing a reliable basis for subsequent local feature comparison, coverage judgment and relation residual calculation.

[0061] (3) This invention uses a coverage loss judgment guided by a reference point cloud to determine whether a normal structure can be interpreted by the test point cloud, starting from the reference point cloud template side. For anomalies such as missing parts, missing components, or missing local structures, there may be no directly visible abnormal points in the point cloud to be tested, and it is easy to miss the anomaly if only looking for anomalies from the test side. This invention regards the local reference patch of the reference point cloud and the test candidate patch of the point cloud to be tested as two local distributions, and calculates whether the reference quality can be softly matched by the point cloud to be tested through unbalanced optimal transmission. The reference quality that fails to match is used as the basis for calculating the coverage loss score. This coverage loss judgment branch can effectively discover logical anomalies such as missing parts, missing components, and missing local structures that are difficult to reliably identify using traditional point-to-point distance methods.

[0062] (4) This invention not only focuses on the geometric differences of a single point or a single local region, but also further analyzes whether the relative spatial relationships between components and their neighbors remain consistent through a local relationship anomaly branch. By comparing the local relationship residuals of the corresponding neighborhoods in the point cloud to be detected and the reference point cloud template, problems such as component offset, misassembly, abnormal assembly direction, and changes in local structural relationships can be found. This local relationship anomaly branch makes up for the shortcomings of simple distance comparison and local feature comparison in characterizing changes in local structural relationships, making the detection results more consistent with the actual performance of logical anomalies in three-dimensional industrial assembly.

[0063] (5) This invention constructs a lightweight defect-sensitive feature extractor for extracting geometric features of local patches in the reference point cloud and the point cloud to be detected. A lightweight geometric feature encoder is used in combination with a stable geometric branch and a defect-sensitive branch. On the one hand, it retains stable geometric information such as local surface shape, edge changes and neighborhood arrangement. On the other hand, it enhances the ability to perceive abrupt changes in local structure and abnormal geometric patterns. After training, only the encoder part is retained for feature extraction, and the classification head is removed. This reduces the inference complexity while ensuring the ability to represent local defects, making it suitable for practical deployment in industrial inspection scenarios.

[0064] In summary, the three-dimensional industrial assembly anomaly detection method and system based on unbalanced optimal transmission and local relation residuals provided by this invention combines reference point cloud template normalization, local defect sensitive features, reference coverage missing estimation, and local relation residuals for logical anomaly detection of point clouds of three-dimensional industrial assembly objects, realizing point-level localization and object-level judgment of assembly anomalies such as missing parts, incorrect assembly, and pose deviation.

[0065] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Attached Figure Description

[0066] Figure 1 This is a flowchart of the three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals of the present invention.

[0067] Figure 2 This is a block diagram of the three-dimensional industrial assembly anomaly detection system based on unbalanced optimal transmission and local relation residuals of the present invention.

[0068] Figure 3 This is a schematic diagram of the defect-sensitive feature extractor (DS-LGFE) of the present invention. Detailed Implementation

[0069] Example

[0070] See Figure 1The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transport and local relation residuals provided by the present invention, as shown in the figure, specifically includes the following steps:

[0071] S100: Obtain the reference point cloud and the point cloud to be detected of the 3D assembly object, and align the reference point cloud and the point cloud to be detected to the same coordinate system.

[0072] Use 3D scanning to obtain a reference point cloud template library and a point cloud to be inspected for a 3D assembled product object. Normal reference point cloud template library It consists of reference point cloud samples of the same product category assembled normally. The goal of this invention is to provide a reference point cloud for each test point p in the point cloud to be tested without retraining on the point cloud samples to be tested. i Estimating point-level outlier scores And obtain the object-level anomaly score S of the three-dimensional assembled product to be inspected. obj .

[0073] Aligning the reference point cloud and the point cloud to be detected to the same coordinate system in this step includes the following sub-steps:

[0074] S101. Normalize the reference point cloud and the point cloud to be detected based on their own centroids, and transform them into their respective local coordinate systems.

[0075] Specifically, for any input point cloud in the reference point cloud and the point cloud to be detected... ,in, Let N be the i-th 3D point in the input point cloud, and N be the number of points in the input point cloud. First, calculate its centroid c, and then translate each point to a local coordinate system with the centroid as the origin:

[0076] ;

[0077] .

[0078] in, express Centralized points Indicates the point The radial distance to the center of mass.

[0079] S102. Select outer and middle anchor points based on the radial distance quantiles from the reference point cloud and the point cloud to be detected to their respective centroids in their local coordinate systems. The outer and middle anchor points are specifically represented as follows:

[0080] ;

[0081] .

[0082] in, This represents the outer set of anchor points, used to estimate the first principal axis direction of the input point cloud; This represents the set of intermediate anchor points used to estimate the second principal axis direction of the input point cloud. and These represent the lower and upper quantiles of the radial distance corresponding to the outer anchor point, respectively. and These represent the lower and upper quantiles of the radial distance corresponding to the middle anchor points, respectively. In practical applications, the outer anchor point set uses the 0.90 to 0.99 quantile range of the radial distance, and the middle anchor point set uses the 0.45 to 0.75 quantile range of the radial distance.

[0083] S103. Determine the first principal axis direction based on the direction voting tensor of the outer anchor point, project the middle anchor point onto the orthogonal subspace of the first principal axis to determine the second principal axis direction, and cross-multiply the first principal axis direction with the second principal axis direction to obtain the third principal axis direction.

[0084] For the outer and middle anchor points selected by radial distance quantiles, this invention further introduces local sampling density compensation weights and radial robust weights to reduce the impact of sampling unevenness, noise points, and local outliers on principal axis estimation. The anchor point weights are obtained based on the local KNN average distances of the outer and middle anchor points.

[0085] ;

[0086] ;

[0087] .

[0088] in, The local k-nearest neighbor average distance of the i-th anchor point, including the outer and middle anchor points, is used to reflect the sampling density near the anchor point. This represents the median of the local k-nearest neighbor average distances for all anchor points; express The median absolute deviation is used to measure the dispersion of local sampling density; It is a preset, extremely small positive number used to avoid the denominator being zero; This represents a Huber-type robust weight function, used to reduce the impact of outlier anchors; The local sampling density compensation weight for the i-th anchor point is represented by bi; the radial robust weight is used to reduce the impact of abnormal radial anchor points; and wi represents the final normalized anchor point weight, which is determined by... The result is obtained by multiplying bi and then normalizing.

[0089] The direction voting tensor of the outer anchor points is obtained by combining the anchor point weights, and the first principal axis direction is determined accordingly. The direction voting matrix constructed by the outer anchor points is as follows:

[0090] ;

[0091] .

[0092] in, For the outer anchor point set, Let i be the coordinates of the i-th outer anchor point after centralization. Let Euclidean norm be the outer anchor point. The anchor weights corresponding to the outer anchor points. This is a directional voting matrix constructed from the outer anchor points. It is a unit direction vector. This is the primary axis direction. By weighted voting on the unit directions of the outer anchor points, the main extension direction of the point cloud's outer structure is obtained, and this will make... The largest unit vector is taken as the first principal axis.

[0093] By combining the anchor weights of the middle-level anchors, the middle-level anchors are projected onto the orthogonal subspace of the first principal axis direction, and the second principal axis direction is determined accordingly. The third principal axis direction is determined by performing a vector cross product between the first and second principal axis directions.

[0094] The specific calculation process is as follows:

[0095] ;

[0096] ;

[0097] ;

[0098] .

[0099] in, This represents the centered coordinates of the i-th middle-layer anchor point after subtracting the centroid. Indicates the direction of the first principal axis; Indicates will Projected onto the first principal axis The vector obtained after orthogonal subspaces; express The projected length along the first principal axis; express The projection component along the first principal axis. This represents the robust covariance matrix of the middle-level anchor points after projection; This represents the robust weight of the i-th middle-layer anchor point; This represents the weighted average of the middle layer anchor points after projection; This represents the covariance contribution of the anchor point relative to the weighted mean after projection; ϵI represents the preset minimum positive number; I represents the identity matrix, and ϵI is used to improve the numerical stability of the covariance matrix. Representation matrix The eigenvector corresponding to the largest eigenvalue; Indicates the direction of the second principal axis, which is determined by... Remove on the first spindle The components in the direction are then normalized to obtain the result. Indicates the direction of the third principal axis. This represents the vector normalization operation. This represents the cross product of vectors.

[0100] S104. Perform ICP refinement on the reference point cloud and the point cloud to be detected to complete coordinate system normalization and align the reference point cloud and the point cloud to be detected in the same coordinate system.

[0101] First, right-handed orthogonal bases are constructed for the reference point cloud and the point cloud to be detected based on their respective centroids and three principal axes, and then mapped to their respective canonical coordinate systems. Then, ICP is performed with the reference point cloud as the fixed endpoint and the point cloud to be detected as the moving endpoint to obtain the residual rigid body transformation, which is then applied to the point cloud to be detected. Finally, both are unified to the canonical coordinate system of the reference point cloud. To avoid reversing the signs of the principal axes, the three axes must adopt the same deterministic sign rule.

[0102] S200: Extract the local input features of the aligned reference point cloud and the point cloud to be detected, respectively, and input them into the pre-trained defect-sensitive feature extractor. Output the corresponding defect-sensitive local features. Obtain the point-level local feature anomaly score based on the feature difference between the corresponding defect-sensitive local features of the reference point cloud and the point cloud to be detected.

[0103] The Defect Sensitive Feature Extractor (DS-LGFE) is based on a lightweight geometric feature encoder. It extracts local patch features from input point cloud samples through the encoder, and learns these local geometric features through training to identify defect features in the point cloud to be detected. It includes a local input channel, convolutional layers, a stable geometric branch, a defect-sensitive branch, a gated fusion unit, a pooling layer, and a projection head. Its structural principle is as follows: Figure 3As shown; the local input channel generates seven-dimensional local patch features from the preprocessed point cloud samples, where the first three dimensions are the relative coordinates of the point cloud samples after centering and radius normalization, the middle three dimensions are the normal vectors of the point cloud samples, and the last dimension is the normalized radius of the point cloud samples; the convolutional layer maps the seven-dimensional local patch features into 64-dimensional point features through two 1×1 convolutions, which are then input into the stable geometric branch and the defect-sensitive branch respectively; the stable geometric branch outputs stable geometric branch output features through two depthwise separable convolutional blocks with 64 channels and a kernel size of 3; the defect-sensitive branch adopts a three-scale parallel structure and through... The convolutional fusion outputs the defect-sensitive branch output features; the gated fusion unit performs weighted fusion of the stable geometric branch output features and the defect-sensitive branch output features to output fused features; the pooling layer concatenates the max pooling result and average pooling result of the fused features, and obtains the patch descriptor of the local patch features through the projection head.

[0104] During the pre-training process of the defect-sensitive feature extractor, the encoder is used to extract local patch features from the training point cloud samples. A classification head is used for supervised optimization based on the local patch features output by the lightweight geometric feature encoder and the corresponding patch feature labels, enabling the encoder to learn the differences between normal and abnormal structures. After training, the classification head is removed during the model export stage, retaining only the encoder's network weights and corresponding configuration parameters for subsequent feature extraction in point cloud anomaly detection. The specific training process is as follows:

[0105] A defect-sensitive feature extractor (DS-LGFE) is constructed to obtain pre-trained point cloud samples. Based on the centroid of the point cloud samples, centering and radius normalization preprocessing are performed.

[0106] Specifically, point cloud samples are selected from the Real3DAD dataset for training. First, the point cloud samples are centered based on their own centroid, and then normalized using the maximum radius as the scale factor, thereby reducing the differences in position and scale between different samples. Given a center... Query its One neighbor, and build :

[0107] ;

[0108] .

[0109] in, This represents the coordinates of the center point of the j-th patch in the point cloud sample. Point cloud samples are represented by The coordinates of the k-th nearest neighbor obtained from the center query. Indicates nearest neighbor points The corresponding kNN normal vector. Let represent the local normalized radius of the j-th patch in the point cloud sample, calculated from the distance from all nearest neighbors within that patch to the center point. The maximum Euclidean distance was calculated. The input feature of the k-th point in the j-th patch of the point cloud sample is the input feature of the entire patch. From K composition, It is a K×7 local feature matrix, where the first three dimensions are the centered and radius-normalized relative coordinates, the middle three dimensions are the normal vectors, and the last dimension is the normalized radius.

[0110] The corresponding Patch feature labels are given by the proportion of outliers in the neighborhood:

[0111] .

[0112] in, This represents the center point of the j-th local patch in the point cloud sample. Indicates the center point Let K be the set of nearest neighbors obtained from the center query, and let K represent the number of nearest neighbors in this patch. This represents the k-th point in the nearest neighbor set. This is a point-level anomaly label; it is set to 1 if the point is an anomaly, and 0 otherwise. In the formula... This indicates the proportion of outliers within this patch. This is the anomaly ratio threshold. When the proportion of outliers in the neighborhood is greater than or equal to... hour, This indicates that the patch is marked as an exception patch; otherwise... This indicates that the patch is marked as a normal patch.

[0113] In the default training configuration, =0.05, K=256. This means that when the proportion of outliers in a training patch reaches 5% or more, the patch is used as an outlier sample in the training; at the same time, balanced sampling is used to alleviate the problem of imbalance between the number of normal patches and outlier patches.

[0114] The Defect Sensitive Feature Extractor (DS-LGFE) has 7 input channels and an output feature dimension of [missing value]. First, two layers. Conv-BN-ReLU maps patch features to 64 channels:

[0115] .

[0116] in, This represents the input feature of the j-th patch in the point cloud sample, which consists of the 7-dimensional geometric features of the K nearest neighbors within that patch. and These represent 1×1 convolutions with 32 and 64 output channels, respectively. BN represents batch normalization, and σ represents a non-linear activation function. This formula maps the original 7-dimensional local geometric input to 64-dimensional point features H through two layers of 1×1 convolutions, providing the basic feature representation for subsequent stable geometric branches and defect-sensitive branches.

[0117] The 64-dimensional point features H are then input into two complementary branches: a stable geometry branch and a defect-sensitive branch. The stable geometry branch consists of two depthwise separable convolutional blocks with 64 channels and a kernel size of 3.

[0118] .

[0119] Here, H represents the 64-dimensional point features obtained from the first two 1×1 convolutions, and D3(.) represents a depthwise separable convolutional block with a kernel size of 3. This branch aggregates neighborhood information through two consecutive local convolutions to extract relatively stable local geometric structure features, such as local surface shape, edge variations, and neighborhood arrangement relationships. The resulting H... s This represents the characteristics of the stable geometric branch output.

[0120] The defect-sensitive branch employs a parallel structure with three scales: 1, 3, and 5, and... Convolutional fusion into 64 channels:

[0121] .

[0122] in, , and These represent feature extraction branches at three different scales. It is a 1×1 Conv-BN-ReLU, used to extract pointwise channel features; and Depthwise separable convolutions are applied after each 1×1 projection to capture local geometric changes at different ranges. `Concat[.]` represents channel concatenation, and `Fuse(.)` represents fusing the concatenated multi-scale features through 1×1 convolutions. This branch enhances the network's ability to perceive local defects, structural abrupt changes, and anomalous geometric patterns; the output feature of the defect-sensitive branch is denoted as... .

[0123] To adaptively fuse stable geometric features and defect-sensitive features, the gating module outputs feature H from the stable geometric branch. s and defect-sensitive branch output features Perform max pooling separately, then concatenate the features H output from the stable geometric branch. s and defect-sensitive branch output features The 128-dimensional descriptor is fed into the MLP, and the weights of the two branches are obtained through softmax:

[0124] ;

[0125] .

[0126] MaxPool(.) is used to extract the most salient global response from each branch, and Concat(.) concatenates the pooling results of two branches. The MLP has a 128→64→2 structure and outputs two weights, namely the weights of the stable geometric branches. and defect-sensitive branch weights Softmax is used to ensure that two weights have comparable relative sizes. This represents the features after weighted fusion.

[0127] Finally, the fusion features The max pooling and average pooling results are concatenated, and the patch descriptor is obtained through the projection head.

[0128] ;

[0129] .

[0130] in, `Proj(.)` represents the global description vector of the j-th patch, obtained by concatenating max-pooling and average-pooling features. Max-pooling focuses more on the most salient local responses, while average-pooling focuses more on the overall structural distribution. Combining max-pooling and average-pooling can simultaneously preserve local anomalous responses and overall geometric information. `Proj(.)` represents the projection head, whose structure is 128→256→d, containing ReLU and dropout in between. Describing the L2 norm, Used to avoid a denominator of zero. This represents the final normalized patch feature, i.e., the patch descriptor used for feature distance comparison, center constraint training, and anomaly score calculation in subsequent training constraints. This normalization operation keeps the feature length of different patches consistent, which facilitates subsequent distance comparison and center constraint training.

[0131] During testing, both the reference point cloud and the point cloud to be detected follow the same patch construction, encoding, dual-branch fusion, pooling, and projection process as in pre-training, and share the same set of frozen weights; only the classification header is removed. The output is a defect-sensitive patch descriptor, which does not necessarily mean that the patch is abnormal.

[0132] After the pre-trained point cloud samples are input into the lightweight geometric feature encoder, which outputs patch descriptors for local patch features, the classification head jointly optimizes the defect-sensitive feature extractor using classification loss, center constraint, and margin constraint based on the patch descriptors and corresponding patch feature labels. The classification loss is the loss used by the supervised classification head to distinguish between normal and abnormal local patch features; the center constraint uses a center loss that moves local patch features of the same category closer to the center of their corresponding patch feature label category; and the margin constraint uses the distance difference between the local patch feature and the normal and abnormal categories in the patch feature labels as the margin loss.

[0133] Specifically, let's set Normalized patch features for the training process of the Defect Sensitive Feature Extractor (DS-LGFE). For the patch labels during training, If the centers are two types that can be learned and normalized, then the training objective is:

[0134] .

[0135] in, This represents the classification loss, used to supervise the classification head in distinguishing between normal and abnormal patches. It can be calculated using binary classification Softmax cross-entropy, as shown below:

[0136] .

[0137] Where B is the batch size. Output the logit for the classification header of the i-th sample belonging to class c. , Let be the true label of the i-th sample.

[0138] This represents the center loss, used to constrain features of the same patch to move closer to the center of the corresponding category; This represents the margin loss, used to separate normal class centers from abnormal class centers and enhance the separability of the two types of features. , and These represent the weights of the three loss terms, used to control the influence of different training objectives on the total loss. Specifically, they are expressed as follows:

[0139] .

[0140] Where B represents the number of patches in a batch. This represents the center of the category corresponding to point cloud sample i. If... =0, then it corresponds to the normal class center c0; if =1, then the corresponding exception class center is c1. This represents the similarity between the two features. The smaller the loss, the closer the patch feature is to its own class center, thus making similar features more clustered.

[0141] The margin loss is composed of the distance difference between the sample and the correct or incorrect center and the distance between the two classes of centers:

[0142] .

[0143] in, This represents the patch features extracted by the encoder from point cloud samples. Go to the correct category center distance, This represents the patch features extracted by the encoder from point cloud samples. To another category center The distance. The first requirement is that the sample should be closer to its own class center and farther away from the misclass center. The second requirement is that the normal class center c0 and the abnormal class center c1 should maintain a distance of at least [missing information]. The distance between the two types of features is reduced, allowing the encoder to learn local geometric representations that are easier to distinguish between normal and abnormal structures. In this embodiment... Set it to 1.

[0144] This step utilizes the Defect Sensitive Feature Extractor (DS-LGFE) to extract features from corresponding local patches in the reference point cloud and the point cloud to be detected, and calculates the feature differences between the two to obtain point-level local feature anomaly scores. The specific process is as follows:

[0145] S201. Sample local patch pairs corresponding to several positions in the reference point cloud and the point cloud to be detected, and input them into the pre-trained defect-sensitive feature extractor to obtain the defect-sensitive local features of the reference point cloud and the point cloud to be detected, respectively.

[0146] Specifically, first sample several central anchor points in the point cloud P′ to be detected. And find the nearest reference center in the reference point cloud. This forms a set of corresponding local patch pairs. Subsequently, using the same patch construction method, local input features are extracted from the point cloud to be detected and the reference point cloud, respectively. These features are then input into a shared defect-sensitive feature extractor (DS-LGFE) to obtain the defect-sensitive patch descriptors corresponding to the point cloud to be detected and the reference point cloud. and The defect-sensitive feature extractor outputs defect-sensitive local features (defect-sensitive patch descriptors) for both normal and abnormal patches. Whether a patch is abnormal is determined by the difference score between the reference descriptor and the descriptor to be detected.

[0147] S202. Calculate the center-level local feature anomaly score between the defect-sensitive local features at the sampling locations of the reference point cloud and the point cloud to be detected. :

[0148] ;

[0149] and These are the defect-sensitive local features of the reference point cloud and the point cloud to be detected at sampling position j, respectively. This represents the L2 norm.

[0150] S203. The score of local feature anomalies at the central level is obtained through nearest neighbor interpolation. The anomaly score is propagated to each test point in the point cloud to be detected, forming a point-level local feature anomaly score. .

[0151] S300. Construct the local distribution within the neighborhood of the reference center anchor point in the aligned reference point cloud. Soft match the reference quality with the local distribution corresponding to the point cloud to be detected using unbalanced optimal transmission estimation. The point-level coverage missing score of the point cloud to be detected is obtained from the total unmatched reference quality.

[0152] In 3D industrial assembly, defects such as missing parts, missing components, and missing local structures may not be directly visible in the test point cloud. Therefore, this invention sets up a coverage gap branch starting from the reference point cloud template side to determine whether normal structures can be interpreted by the observations of the point cloud to be tested. Unlike the rule-based approach of explicitly calculating coverage and superimposing distance terms, this invention treats the reference structures and test candidate structures within the neighborhood of the reference center anchor point of each reference point cloud as two local distributions. It estimates whether the reference quality can be soft-matched by the test point cloud through unbalanced optimal transmission, and only uses the reference quality that fails to match to calculate the coverage gap score.

[0153] This step includes the following sub-steps:

[0154] S301. Sample multiple reference center anchor points in the reference point cloud, construct local reference patches using the K-Nearest Neighbor (kNN) algorithm with a preset number of neighbors K, and construct test candidate patches in the point cloud to be detected centered on the same reference center anchor point using the K-Nearest Neighbor (kNN) algorithm with a preset number of neighbors K; the definitions of local reference patches and test candidate patches are as follows:

[0155] ;

[0156] .

[0157] in, Represents the normalized reference point cloud. This represents the standardized point cloud to be detected. This represents the l-th reference center anchor point. Indicated by The local reference patch obtained by querying the reference point cloud with the center as the reference point. Indicates the reference center anchor point of the same sampling. The test candidate patch is obtained by querying the point cloud to be tested, with the center as the target. and These represent the number of nearest neighbor queries within the reference point cloud on the reference side and the number of neighbor queries within the point cloud to be detected on the test side, respectively. In the current implementation, both are set to 24. M and N represent the actual number of reference points and the number of test points obtained, respectively. This refers to the i-th reference point in the reference patch. This represents the j-th test candidate point in the test candidate patch.

[0158] S302. If the local reference patch is empty, the anchor point will not participate in the coverage loss score, and the coverage loss score of the corresponding reference center anchor point in the reference point cloud will be set to 0. If the test candidate patch is empty, the reference structure cannot be interpreted by the test observation, and the coverage loss score of the corresponding reference center anchor point in the point cloud to be detected will be set to 1.

[0159] S303. For non-empty local patches, construct the squared distance matrix between the reference point and the test candidate point in the corresponding patch from the non-empty local reference patch and test candidate patch:

[0160] .

[0161] in, Indicates reference point With test candidate points The squared Euclidean distance between them. This squared distance matrix is ​​used to measure the spatial difference between points in the reference local structure and candidate points in the point cloud to be detected. The larger the value, the higher the reference point. With test candidate points The more spatially difficult the matching, the better. Reference points and test candidate points refer to all points in both patches, not just anchor points. If the reference patch contains M points... The test candidate patch contains N points. Then construct an M×N squared distance matrix. .

[0162] Use the median of the squared distances from the local reference patch to the corresponding reference center anchor point as the local scale:

[0163] .

[0164] in, This represents the local scale of the l-th local reference patch. Indicates reference point To the reference anchor point The squared distance is calculated. After obtaining the local scale, the squared distance matrix is ​​normalized to the geometric cost matrix.

[0165] .

[0166] in, This represents the normalized geometric matching cost. To prevent extremely small constants with a denominator of zero, division by a local scale is used. Local patches of different sizes can be compared at a relatively uniform scale. When the test candidate patch is far removed from the reference local structure, and The values ​​are all relatively large, resulting in higher unmatched reference quality in subsequent unbalanced optimal transmissions. The cost matrix uses a scale determined by the local structure of the reference to avoid the median cross-set distance within the patch being canceled out when all test candidate points in the missing region are far apart.

[0167] Based on the geometric cost matrix, a Gibbs kernel is constructed to represent the soft matching similarity between reference points in the local reference patch and test points in the test candidate patch:

[0168] ;

[0169] .

[0170] in, It is the (i,j)th element of the Gibbs kernel matrix G, which represents the reference point in this invention. With test candidate points The soft matching similarity between them. The smaller the value, the closer the two are and the greater the soft matching similarity. The larger the value, the further apart the two elements are, and the smaller the similarity of soft matching. The larger the value, the stronger the match. The smaller the value, the weaker the match. The entropy smoothing intensity of the soft match is controlled by ρ, which controls the degree of unbalanced mass relaxation. In this embodiment, ε=0.05 and ρ=1. θ is the exponential parameter in the unbalanced Sinkhorn update.

[0171] Define the discrete quality of the reference point in the local reference patch and the test point in the test candidate patch within the corresponding patch. On the normalized cost matrix, let the discrete quality of the reference side of the local reference patch and the test side of the test candidate patch be denoted as quality vectors a and b, respectively:

[0172] ;

[0173] .

[0174] in, This represents the quality of the i-th reference point in the local reference patch. Let M represent the quality of the j-th test point in the test candidate patch, and M and N represent the number of reference points and test points in the local reference patch and the test candidate patch, respectively. A uniform quality distribution is used here, meaning that each reference point has the same initial quality, and each test candidate point also has the same initial quality.

[0175] Iterative unbalanced optimal transfer is performed by updating the scaling vectors of the local reference patch (reference side) and the test candidate patch (test side):

[0176] .

[0177] in, and These represent the scaling vectors for the local reference patch (reference side) and the test candidate patch (test side), respectively. and These represent the reference-side mass vector and the test-side mass vector, respectively. This indicates the matching response of the current test candidate patch to the local reference patch after scaling on the test side. This indicates the matching response of the current local reference patch to the test candidate patch after scaling on the reference side. , These are the Gibbs kernel matrix G and the scaling vector, respectively. and The product. Division and exponentiation in the formula are element-wise operations. To prevent division by zero of extremely small positive numbers, this iterative process allows for partial reference quality that cannot be fully matched by the test side, making it suitable for characterizing coverage gaps caused by missing parts or omissions.

[0178] The soft matching matrix from the reference point to the test candidate point is obtained:

[0179] .

[0180] in, Indicates from the reference point To test candidate point The quality of soft matching. If Larger, indicating a larger reference point Part of the quality can be tested at the test points Explanation; if The overall size is small, indicating that it is difficult to find a reasonable correspondence for this reference point in the point cloud to be detected. Reference point The quality of the observations interpreted at the test point is Accordingly, the mismatched reference quality Defined as:

[0181] .

[0182] Use the total unmatched reference quality as the anchor-level coverage for missing scores:

[0183] .

[0184] in, This represents the anchor-level coverage missing score corresponding to the l-th reference anchor point. This score represents the total reference quality in the local reference patch that is not interpreted by the point cloud to be detected. This score can be directly understood as the proportion of the local structure in the reference point cloud that cannot be interpreted by the point cloud to be detected. The higher the score, the more likely the reference area is to have missing parts, missing components, or missing local structures.

[0185] S304. Cover missing scores at the anchor point level in steps S302 and S303. The results are propagated to each test point in the point cloud to be tested through nearest neighbor interpolation, forming a point-level coverage of missing scores. The empty patch boundary score given in step S302 and the non-empty patch anchor score calculated in step S303 should together form an anchor-level score set, which is then interpolated to a point-level score in this step; in step S302, anchor points with empty reference patches are set to 0 or excluded, and anchor points with empty test patches are set to 1.

[0186] S400. Obtain the point-level relationship anomaly score based on the test-side anchor point relationship vector of the aligned point cloud to be detected and the local relationship residual between the reference point cloud and the corresponding reference-side relationship vector.

[0187] Assembly anomalies often manifest as changes in the relative relationships between components, rather than just single-point position offsets. In this invention, a relational residual branch is set up in this step, and the following process is used to obtain the local relational residual between the reference point cloud and the point cloud to be detected, in order to estimate the point-level relational anomaly score:

[0188] S401. Construct a nearest neighbor graph by sampling several test center anchor points in the point cloud to be detected, and find the nearest reference nearest neighbor center of the corresponding test center anchor point in the reference point cloud. Obtain the relationship vectors of the point cloud to be detected in the test side and the reference point cloud in the reference side nearest neighbor query as follows:

[0189] The relation residual branch samples the test center anchor point set in the point cloud to be detected. A nearest neighbor graph is constructed, and the test center anchor point uses the same sampling results as in step S201, and uses the same nearest neighbor test center anchor point in the reference point cloud as the feature branch. As the corresponding normal structure, the nearest neighbor query takes k = 8. For the nearest neighbor graph, the edge between the test center anchor point i and its j-th nearest neighbor center is... The relationship vector between the test side and the reference side is:

[0190] ;

[0191] .

[0192] This represents the relation vector of the point cloud to be detected in the nearest neighbor query on the test side. This is the relation vector of the reference point cloud in the reference-side nearest neighbor query. Indicates the anchor point of the test center for the point cloud to be tested. The m-th k-nearest neighbor center, Indicates the reference point cloud corresponding to the test center anchor point The m-th k-th nearest neighbor center is obtained, and the relationship graph uses l and m as node indices.

[0193] S402. Define the test center anchor point center-level relation residual as the residual of the side length ratio between the test center anchor point and the reference nearest neighbor center, and the residual of the direction of the relation vector, as the local relation residual. :

[0194] .

[0195] The first term is the side length proportional residual, which corresponds to the formula in... The nearest neighbor edge length is the Euclidean length of the edge connecting the test center anchor point and one of its K nearest neighbor centers in the nearest neighbor graph. The edge length ratio residual describes the stretching or compression of the local edge length by comparing the stretching or compression of the test-side nearest neighbor edge length and the reference-side nearest neighbor edge length. It is used to measure whether the local edge length between two center points in the test point cloud has been stretched or compressed relative to the corresponding edge length in the reference point cloud. The second term is the direction residual, corresponding to the formula in... The term describes directional consistency, measuring whether the directions of the test-side relation vector and the reference-side relation vector are consistent, calculated using cosine similarity. The more consistent the directions, the closer this term is to 0; the greater the directional deviation, the larger this term is. η is a dimensionless weighting coefficient of the directional residual relative to the side length residual, used to balance the numerical magnitude of the directional residual and the side length proportional residual; this value is a fixed weight, fixed at 0.5 in this embodiment. If the number of available centers is insufficient to form a stable neighborhood, zero relation residual is returned.

[0196] S403, convert the central-level relation residuals... Point-level relationship anomaly scores are obtained by interpolating the mean of nearest neighbor centers to test points in the point cloud to be detected. .

[0197] S500: The point-level local feature anomaly score, point-level coverage missing score, and point-level relationship anomaly score corresponding to each test point in the point cloud to be detected are fused together, and the object-level anomaly score is output as the detection result.

[0198] Specifically, the point-level local feature anomaly score Point-level coverage of missing scores Point-level relationship anomaly score Perform equal-weighted fusion:

[0199] .

[0200] Obtain the final point-level anomaly map of the point cloud to be detected. The average of the highest 10% of the point-level scores is taken as the output object-level anomaly score of the point cloud to be detected.

[0201] See Figure 2 This invention also discloses a three-dimensional industrial assembly anomaly detection system based on unbalanced optimal transmission and local relation residuals, including a point cloud alignment module 100, a point-level local feature anomaly extraction module 200, a point-level coverage loss estimation module 300, a point-level relation anomaly judgment module 400, and an object-level anomaly fusion module 500. Specifically, the point cloud alignment module 100 acquires a reference point cloud and a point cloud to be detected for the three-dimensional assembly object, aligns the reference point cloud and the point cloud to be detected to the same coordinate system, and then inputs them to the point-level local feature anomaly extraction module 200, the point-level coverage loss estimation module 300, and the point-level relation anomaly judgment module 400, respectively.

[0202] The point-level local feature anomaly extraction module 200 embeds a defect-sensitive feature extractor, which extracts local input features of the aligned reference point cloud and the point cloud to be detected, respectively. The extracted features are input into the pre-trained defect-sensitive feature extractor, and the corresponding defect-sensitive local features are output. The point-level local feature anomaly score is obtained based on the feature difference between the corresponding defect-sensitive local features of the reference point cloud and the point cloud to be detected.

[0203] The point-level coverage loss estimation module 300 constructs the local distribution in the neighborhood of the reference center anchor point in the aligned reference point cloud. It performs soft matching between the reference quality estimated by unbalanced optimal transmission and the local distribution corresponding to the point cloud to be detected. The point-level coverage loss score of the point cloud to be detected is obtained from the total unmatched reference quality.

[0204] The point-level relationship anomaly judgment module 400 obtains the point-level relationship anomaly score based on the local relationship residual between the test-side anchor point relationship vector of the aligned point cloud to be detected and the corresponding reference-side relationship vector of the reference point cloud.

[0205] The object-level anomaly fusion module 500 fuses the point-level local feature anomaly extraction module 200, the point-level coverage missing estimation module 300, and the point-level relationship anomaly judgment module 400 to output the point-level local feature anomaly score, the point-level coverage missing score, and the point-level relationship anomaly score corresponding to each test point in the point cloud to be detected, and outputs the object-level anomaly score as the detection result.

[0206] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0207] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a step or method.

[0208] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transport and local relation residuals, characterized in that, Includes the following steps: S100: Obtain the reference point cloud and the point cloud to be detected of the 3D assembly object, and align the reference point cloud and the point cloud to be detected to the same coordinate system. S200: Extract the local input features of the aligned reference point cloud and the point cloud to be detected, respectively, and input them into the pre-trained defect-sensitive feature extractor. Output the corresponding defect-sensitive local features. Obtain the point-level local feature anomaly score based on the feature difference between the corresponding defect-sensitive local features of the reference point cloud and the point cloud to be detected. S300. Construct the local distribution in the neighborhood of the reference center anchor point in the aligned reference point cloud. Soft match the reference quality with the local distribution corresponding to the point cloud to be detected by estimating the reference quality through unbalanced optimal transmission. The point-level coverage missing score of the point cloud to be detected is obtained from the total unmatched reference quality. S400. Obtain the point-level relationship anomaly score based on the test-side anchor point relationship vector of the aligned point cloud to be detected and the local relationship residual between the reference point cloud and the corresponding reference-side relationship vector. S500: The point-level local feature anomaly score, point-level coverage missing score, and point-level relationship anomaly score corresponding to each test point in the point cloud to be detected are fused together, and the object-level anomaly score is output as the detection result.

2. The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals according to claim 1, characterized in that: In step S100, aligning the reference point cloud and the point cloud to be detected to the same coordinate system includes the following sub-steps: S101. Normalize the reference point cloud and the point cloud to be detected based on their own centroids, and transform them to their respective local coordinate systems. S102. Select the outer anchor point and the middle anchor point based on the radial distance quantiles from the reference point cloud and the point cloud to be detected to their respective centroids in their respective local coordinate systems. S103. Determine the first principal axis direction based on the direction voting tensor of the outer anchor point, project the middle anchor point onto the orthogonal subspace of the first principal axis to determine the second principal axis direction, and cross-product the first principal axis direction and the second principal axis direction to obtain the third principal axis direction. S104. The reference point cloud and the point cloud to be detected are constructed with right-handed orthogonal bases based on their respective centroids and three principal axis directions, and mapped to their respective normalized coordinates. ICP is performed with the reference point cloud as the fixed end and the point cloud to be detected as the moving end to complete the coordinate system normalization, so that the reference point cloud and the point cloud to be detected are aligned in the same coordinate system.

3. The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals according to claim 1, characterized in that: The defect-sensitive feature extractor is based on a lightweight geometric feature encoder. It extracts local patch features of the input point cloud sample through the encoder, including local input channels, convolutional layers, stable geometric branches, defect-sensitive branches, gated fusion units, pooling layers, and projection heads. The local input channel generates a seven-dimensional local patch feature from the preprocessed point cloud sample. The first three dimensions are the relative coordinates of the point cloud sample after centering and radius normalization, the middle three dimensions are the normal vector of the point cloud sample, and the last dimension is the normalized radius of the point cloud sample. The convolutional layer maps the seven-dimensional local patch features into 64-dimensional point features through two 1×1 convolutions, which are then input into the stable geometric branch and the defect-sensitive branch respectively. The stable geometric branch outputs stable geometric branch output features through two depth-separable convolutional blocks with 64 channels and a kernel size of 3. The defect-sensitive branch adopts a three-scale parallel structure, and through... Convolutional fusion outputs defect-sensitive branch output features; The gated fusion unit performs weighted fusion of the stable geometric branch output features and the defect-sensitive branch output features to output a fused feature. The pooling layer concatenates the max pooling and average pooling results of the fused features, and obtains the patch descriptor of the local patch features through the projection head.

4. The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals according to claim 1, characterized in that: The pre-training of the defect-sensitive feature extractor uses a classification head to perform supervised optimization based on the local patch features output by the lightweight geometric feature encoder and the corresponding patch feature labels. The specific training process is as follows: Obtain pre-trained point cloud samples, and perform centering and radius normalization preprocessing based on the centroid of the point cloud samples themselves; The preprocessed point cloud samples are input into a lightweight geometric feature encoder, which outputs patch descriptors of local patch features. The classification head then uses classification loss, center constraint, and margin constraint to jointly optimize the defect-sensitive feature extractor based on the patch descriptors of the local patch features and the corresponding patch feature labels. The classification loss is the classification loss of the supervised classification head in distinguishing between normal local patch features and abnormal local patch features; The center constraint is based on the center loss that makes local patch features of the same category move closer to the center of the corresponding patch feature label category. The interval constraint is constrained by the distance difference between the local Patch feature and the normal and abnormal categories in the Patch feature label as the interval loss.

5. The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals according to claim 1, characterized in that: The calculation process for the point-level local feature anomaly score is as follows: S201. Sample local patch pairs corresponding to several positions in the reference point cloud and the point cloud to be detected, and input them into the pre-trained defect-sensitive feature extractor to obtain the defect-sensitive local features of the reference point cloud and the point cloud to be detected, respectively. S202. Calculate the center-level local feature anomaly score between the defect-sensitive local features at the sampling locations of the reference point cloud and the point cloud to be detected. : ； and To obtain the defect-sensitive local features of the reference point cloud and the point cloud to be detected at sampling position j, Represents the L2 norm; S203. The center-level local feature anomaly score is propagated to each test point in the point cloud to be detected by nearest neighbor interpolation to form a point-level local feature anomaly score.

6. The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals according to claim 1, characterized in that: Step S300 includes the following sub-steps: S301. Sample a local reference patch at the reference center anchor point in the reference point cloud, and construct a test candidate patch in the point cloud to be detected centered on the same reference center anchor point. S302. If there is a local reference patch that is empty, the coverage missing score of the corresponding reference center anchor point in the reference point cloud is set to 0. If any of the test candidate patches is empty, the coverage missing score of the corresponding reference center anchor point in the point cloud to be detected is set to 1. S303. Construct the squared distance matrix between the reference point and the test candidate point in the corresponding Patch from the non-empty local reference Patch and test candidate Patch. The median of the squared distances from each reference point in the local reference patch to the corresponding reference center anchor point is used as the local scale. The squared distance matrix is ​​normalized to a geometric cost matrix. Based on the geometric cost matrix, a Gibbs kernel is constructed to represent the soft matching similarity between the reference points in the local reference patch and the test points in the test candidate patch. Define the discrete quality of the reference point in the local reference patch and the test point in the test candidate patch in the corresponding patch. Iterate the unbalanced optimal transmission by updating the scaling vectors of the reference side of the local reference patch and the test side of the test candidate patch to obtain the soft matching matrix from the reference point to the test candidate point. Use the total unmatched reference quality as the anchor point level to cover the missing score. S304. The anchor-level coverage missing scores from steps S302 and S303 are propagated to each test point in the point cloud to be detected through nearest neighbor interpolation to form point-level coverage missing scores.

7. The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals according to claim 6, characterized in that: In step S301, a local reference patch for the reference point cloud and a test candidate patch for the point cloud to be detected are constructed using the K-nearest neighbor algorithm with a preset number of neighbors.

8. The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals according to claim 1, characterized in that: The point-level relationship anomaly score in step S400 is obtained through the following process: S401. Sample several test center anchor points in the point cloud to be detected to construct a nearest neighbor graph, and find the nearest reference nearest neighbor center of the corresponding test center anchor point in the reference point cloud. Obtain the relationship vectors of the point cloud to be detected on the test side and the reference point cloud on the reference side in the nearest neighbor query respectively. S402. Define the center-level relation residual of the test center anchor point as the local relation residual by using the residual of the side length ratio of the nearest neighbor side length between the test center anchor point and the reference nearest neighbor center and the direction residual of the relation vector. S403. The center-level relationship residual is interpolated to the test points in the point cloud to be detected by means of the nearest neighbor centers to obtain the point-level relationship anomaly score.

9. The three-dimensional industrial assembly anomaly detection method based on unbalanced optimal transmission and local relation residuals according to claim 1, characterized in that: In step S500, the point-level local feature anomaly score, the point-level coverage missing score, and the point-level relationship anomaly score are fused with equal weights to obtain the final point-level anomaly map of the point cloud to be detected. The average of the highest 10% of the point-level scores is taken as the output object-level anomaly score of the point cloud to be detected.

10. A three-dimensional industrial assembly anomaly detection system based on unbalanced optimal transport and local relation residuals, characterized in that, include: The point cloud alignment module acquires the reference point cloud and the point cloud to be detected of the 3D assembled object, and aligns the reference point cloud and the point cloud to be detected to the same coordinate system. The point-level local feature anomaly extraction module embeds a defect-sensitive feature extractor, which extracts local input features of the aligned reference point cloud and the point cloud to be detected, respectively. The extracted features are input into the pre-trained defect-sensitive feature extractor, and the corresponding defect-sensitive local features are output. The point-level local feature anomaly score is obtained based on the feature difference between the corresponding defect-sensitive local features of the reference point cloud and the point cloud to be detected. The point-level coverage loss estimation module constructs the local distribution in the neighborhood of the reference center anchor point in the aligned reference point cloud, and performs soft matching between the reference quality estimated by unbalanced optimal transmission and the local distribution corresponding to the point cloud to be detected. The point-level coverage loss score of the point cloud to be detected is obtained from the total unmatched reference quality. The point-level relationship anomaly detection module obtains the point-level relationship anomaly score based on the local relationship residual between the test-side anchor point relationship vector of the aligned point cloud to be detected and the corresponding reference-side relationship vector of the reference point cloud. The object-level anomaly fusion module fuses the point-level local feature anomaly scores, point-level coverage loss scores, and point-level relationship anomaly scores corresponding to each test point in the point cloud to be detected, and outputs the object-level anomaly score as the detection result.

Images