Three-dimensional defect detection method, system and device based on typical geometric alignment and multi-view adaptive rendering

Abstract

This invention discloses a 3D defect detection method, system, and device based on typical geometric alignment and multi-view adaptive rendering, comprising the following steps: S1, acquiring the initial 3D point cloud of the object to be detected, and performing unsupervised feature analysis on the initial 3D point cloud through typical geometric alignment to obtain a standardized aligned point cloud of the object to be detected; S2, converting the standardized aligned point cloud of the object to be detected into a set of geometric feature maps from multiple camera views; S3, inputting the geometric feature maps of the object to be detected from the camera views into a general 2D image anomaly detection network, outputting point-level anomaly scores, and then back-projecting them onto the corresponding surface points of the standardized aligned point cloud to obtain object-level anomaly indicators for 3D defects. This invention does not rely on the geometric structure and observation posture of the object to be detected, and can achieve robust anomaly defect detection on unaligned object 3D point clouds, achieving high-precision point-level anomaly localization.

Description

Technical Field

[0001] This invention discloses a three-dimensional defect detection method, system, and device based on typical geometric alignment and multi-view adaptive rendering, belonging to the field of three-dimensional computer vision and industrial defect detection technology. Background Technology

[0002] Industrial quality control is undergoing a paradigm shift from two-dimensional visual analysis to three-dimensional geometric perception. While two-dimensional anomaly detection is quite mature, its reliability is limited by the lack of spatial dimension, making it unable to distinguish depth-dependent structural defects such as minute deformations or warping. In contrast, three-dimensional point clouds provide the spatial geometric and physical priors required for high-precision detection. However, unlike the two-dimensional domain, which benefits from massive amounts of foundational models (such as CLIP and DINO), the three-dimensional domain faces a severe data scarcity problem, making it difficult to learn discriminative high-level representations in an unsupervised manner.

[0003] To address the aforementioned issues, projection-based 3D anomaly detection methods have become a key strategy. These methods utilize multi-view representations (MVRs) to geometrically render 3D images, leveraging the robust feature extraction capabilities of existing 2D detectors to detect 3D defect features. For example, Chinese patent application CN120765639A discloses a method and system for detecting wheel hub defects based on visual multi-features. However, existing projection paradigms suffer from a fundamental flaw: they primarily employ static view projection protocols, implicitly assuming that objects possess a consistent, pre-aligned, fixed pose. In real-world, unconstrained industrial environments, raw point clouds often exhibit arbitrary SO(3) rotations. In such cases, fixed-view rendering entangles pose changes with geometric changes, causing downstream detectors to misjudge normal pose shifts as structural anomalies, resulting in catastrophic distribution shifts and false alarms.

[0004] Therefore, there is an urgent need for a pose-insensitive framework that can achieve robust anomaly detection on misaligned 3D point clouds, decoupling geometry from observation pose. Summary of the Invention

[0005] The technical problem solved by this invention is to address the issue that pose offset changes affect the detection accuracy of projection-based 3D anomaly detection methods, and to provide a 3D defect detection method, system, and device based on typical geometric alignment and multi-view adaptive rendering.

[0006] This invention is achieved using the following technical solution:

[0007] This invention first discloses a 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering, comprising the following steps:

[0008] S1. Obtain the initial 3D point cloud of the object to be detected. Perform unsupervised feature analysis on the initial 3D point cloud through typical geometric alignment of the point cloud to eliminate the rotational variation of the 3D special orthogonal group and obtain the normalized aligned point cloud of the object to be detected.

[0009] S2. Project the normalized point cloud of the object to be detected onto multiple camera views, and convert the surface normals of the point cloud to the corresponding camera coordinate system for encoding, so as to obtain a set of geometric feature maps composed of surface normal maps of the object to be detected under multiple camera views.

[0010] S3. Input the set of geometric feature maps of the object to be detected in the camera view into the two-dimensional image anomaly detection network. The two-dimensional image anomaly detection network outputs point-level anomaly scores based on the encoded features and reconstructed features of the geometric feature maps. Then, the point-level anomaly scores are back-projected onto the corresponding surface of the normalized aligned point cloud to obtain the final object-level anomaly index of the object to be detected.

[0011] In the 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering of the present invention, the typical geometric alignment of point clouds in step S1 further includes the following sub-steps:

[0012] S11. Calculate the centroid of the initial three-dimensional point cloud. Subtract the three-dimensional coordinates of the initial three-dimensional point cloud from the centroid to obtain the zero-mean point set of the centered point cloud. Subtract the three-dimensional coordinates of each three-dimensional point in the initial three-dimensional point cloud from the three-dimensional coordinates of the centroid to obtain the zero-mean point set of the centered point cloud. Perform covariance matrix analysis on the zero-mean point set of the centered point cloud to obtain the geometric covariance matrix of the initial three-dimensional point cloud.

[0013] S12. Perform eigenvalue decomposition on the geometric covariance matrix to obtain the orthogonal principal axis matrix as the initial rotation hypothesis. For the random flipping of positive and negative directions caused by eigenvalue decomposition, perform sign disambiguation based on the skewness of the one-dimensional projection distribution of the zero-mean point set of the centralized point cloud along each orthogonal principal axis direction, and force the positive direction of eigenvalue decomposition to align with the heavier end of the distribution.

[0014] S13. Construct a rotation matrix using the disambiguated orthogonal principal axis matrix, and map the initial 3D point cloud of the object to be detected to a normalized aligned point cloud using the rotation matrix.

[0015] In the 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering of the present invention, further, in step S12, the geometric covariance matrix is... Perform eigenvalue decomposition U is composed of three orthogonal principal axes The eigenvector matrix formed That is, defining the initial rotation assumption of the initial 3D point cloud as the orthogonal principal axis matrix. It is a diagonal matrix formed by the corresponding eigenvalues. ,in The three eigenvalues ​​representing the variance of the point cloud along the directions of the corresponding eigenvectors in the geometric covariance matrix are categorized as follows: arrangement;

[0016] The symbol disambiguation introduces the following skewness-based symbol function:

[0017] ;

[0018] in, The eigenvalue decomposition yields the first... eigenvectors along orthogonal principal axes , Let N represent the i-th 3D point in the initial 3D point cloud, and N represent the number of 3D points in the initial 3D point cloud. The sign function is defined as follows: when the skewness of the centered point cloud projected along the k-th orthogonal principal axis is greater than zero. When less than zero, take When equal to zero .

[0019] In the 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering of the present invention, further, when the ratio of any two adjacent eigenvalues ​​in the geometric covariance matrix is ​​less than a set threshold, it is determined that there is a geometric degradation situation with unstable principal axis direction. In the case of geometric degradation, the corresponding unstable orthogonal principal axis is constrained to a predefined global reference direction, and then orthogonalized in combination with the remaining orthogonal principal axes to maintain the strict orthogonality of the rotation matrix and prevent coordinate system oscillation.

[0020] In the 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering of the present invention, further, in step S13, a rotation matrix is ​​constructed based on the disambiguated orthogonal principal axis matrix. The initial 3D point cloud of the object to be detected Transform and scale to within a unit sphere to form a canonical aligned point cloud unaffected by the original pose. :

[0021] ;

[0022] in, To normalize the point cloud to a scaling factor within a unit sphere, the scaling factor is determined based on the maximum Euclidean distance from each point in the rotated point cloud to the origin. The centroid of the initial 3D point cloud is given.

[0023] In the 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering of the present invention, step S2 further includes the following sub-steps:

[0024] S21. Uniformly sample V camera viewpoints on a unit sphere;

[0025] S22. Using a differentiable renderer, the normalized aligned point cloud is rendered from each camera viewpoint into a surface normal map under different camera viewpoints. In this process, the surface normals of the normalized aligned point cloud are converted to the corresponding camera coordinate system for encoding, thereby obtaining a geometric feature map related to the camera viewpoint.

[0026] S23. During the rendering process, explicitly record the mapping relationship between the pixel coordinates of the point cloud in the dense geometric feature map and the coordinates of the surface points of the normalized aligned point cloud.

[0027] In the three-dimensional defect detection method based on typical geometric alignment and multi-view adaptive rendering of the present invention, further, in step S21, the camera view is sampled on the Fibonacci mesh surrounding the unit sphere.

[0028] In the three-dimensional defect detection method based on typical geometric alignment and multi-view adaptive rendering of the present invention, further, in step S3, the two-dimensional image anomaly detection network extracts the encoded feature map and the corresponding reconstructed feature map of the geometric feature map under each camera view.

[0029] Based on the mapping of the correspondence between the pixel coordinates of the point cloud in the geometric feature map of each camera view and the surface points of the normalized aligned point cloud recorded in step S2, for the same 3D point in the normalized aligned point cloud, the initial encoded features and initial reconstructed features corresponding to the 3D point are extracted in the encoded feature map and reconstructed feature map of each camera view.

[0030] The initial encoded features of the same 3D point obtained from multiple camera views are arithmetically averaged, and the corresponding initial reconstructed features are arithmetically averaged to obtain the multi-view aggregated encoded features and multi-view aggregated reconstructed features of the 3D point.

[0031] The multi-view aggregated coding features and multi-view aggregated reconstructed features are normalized, the cosine distance between them is calculated, and the cosine distances obtained from multiple network layers of the two-dimensional image anomaly detection network are averaged to obtain the point-level anomaly score of the three-dimensional point.

[0032] The object-level anomaly index of the object to be detected is determined by the maximum value of the point-level anomaly score of each three-dimensional point in the object to be detected.

[0033] Furthermore, the present invention also discloses a 3D defect detection system based on typical geometric alignment and multi-view adaptive rendering, comprising:

[0034] The typical geometric alignment module for point clouds obtains the initial 3D point cloud of the object to be detected. It performs unsupervised feature analysis on the initial 3D point cloud through typical geometric alignment of point clouds, eliminates the rotational variation of the 3D special orthogonal group, and obtains the normalized aligned point cloud of the object to be detected.

[0035] The multi-view transformation module projects the normalized aligned point cloud of the object to be detected onto multiple camera views, transforms the surface normals of the point cloud into the corresponding camera coordinate system for encoding, and obtains a set of geometric feature maps composed of surface normal maps of the object to be detected under multiple camera views.

[0036] The detection module inputs the set of geometric feature maps of the object to be detected in the camera view into a two-dimensional image anomaly detection network. The two-dimensional image anomaly detection network outputs point-level anomaly scores based on the encoded and reconstructed features of the geometric feature maps. Then, the point-level anomaly scores are back-projected onto the corresponding surface of the normalized aligned point cloud to obtain the final object-level anomaly index of the object to be detected.

[0037] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the three-dimensional defect detection method based on typical geometric alignment and multi-view adaptive rendering described above.

[0038] The present invention, by adopting the above technical solution, has the following beneficial effects:

[0039] (1) The typical geometric alignment of three-dimensional point clouds proposed in this invention effectively decouples the internal geometric shape of the initial three-dimensional point cloud of the object to be detected from the external observation posture by using an unsupervised mechanism. It can unify and standardize the initial three-dimensional point cloud with arbitrary three-dimensional special orthogonal group (SO(3)) rotation to a regular coordinate system. By estimating its internal reference system through unsupervised feature analysis, the rotation variation of arbitrary three-dimensional special orthogonal group (SO(3)) is eliminated, which solves the problem that the existing projection-based method is limited by a rigid viewpoint.

[0040] (2) The present invention generates surface normal maps from multiple camera views as geometric feature maps of the object to be detected by adaptive normal projection. This mode is different from the traditional absolute depth map. It is not sensitive to changes in absolute distance and illumination, and can explicitly characterize local directional changes on the surface. Since high-frequency structural anomalies such as surface scratches, dents, and warping will cause local normal perturbations, which are transformed into significant geometric response differences in the camera view normal map, the feature response sensitivity of such high-frequency structural anomalies is significantly improved.

[0041] (3) This invention transforms the detection of high-dimensional and variable three-dimensional anomalies into a standard two-dimensional task. As a "universal adapter," it utilizes the closed-loop mapping of the positional relationship between two-dimensional geometric feature maps and three-dimensional point clouds to stably convert misaligned three-dimensional point clouds into visual data suitable for general two-dimensional image anomaly detection networks. Figure 1 This approach enables geometric representation, allowing large-scale 2D vision models to achieve zero-sample semantic transfer directly onto 3D data, achieving high-precision 3D geometric perception and detection performance of objects without modifying the model architecture.

[0042] In summary, the present invention provides a 3D defect detection method, system, and device based on typical geometric alignment and multi-view adaptive rendering. It does not depend on the geometric structure and observation posture of the object to be detected, and can achieve robust abnormal defect detection on the 3D point cloud of the unaligned object, thus achieving high-precision point-level anomaly localization.

[0043] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0044] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the technical solutions of this invention. The illustrative embodiments of this invention and their descriptions are used to explain the technical solutions of this invention and do not constitute an improper limitation of this invention.

[0045] Figure 1 This is a flowchart of the 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering disclosed in this invention.

[0046] Figure 2 This is a visualization of the effect of performing typical geometric alignment of the initial 3D point cloud of an object in this invention.

[0047] Figure 3 This is a schematic diagram comparing a projection view that has not been aligned using the classic geometric alignment method with a Cam-Normal projection view aligned using the classic geometric alignment method of the present invention. Detailed Implementation

[0048] Example

[0049] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the technical solutions of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0050] See Figure 1 The figure illustrates the 3D defect detection method of the present invention based on typical geometric alignment and multi-view adaptive rendering, which achieves robust anomaly detection on the 3D point cloud of the misaligned object to be detected. The method specifically includes the following steps:

[0051] S1. Obtain the initial three-dimensional point cloud of the object to be detected. Perform unsupervised feature analysis on the initial three-dimensional point cloud through typical geometric alignment of the point cloud to eliminate the rotational variation of the three-dimensional special orthogonal group SO(3) and obtain the normal aligned point cloud of the object to be detected.

[0052] Let the initial 3D point cloud set of any object to be detected be... ,in, This represents the number of 3D points in the initial 3D point cloud; the typical geometric alignment process of the point cloud corresponds to obtaining the normalization transformation that maps any input initial 3D point cloud to a unified standard reference system, and the normalization transformation includes at least translation, rotation and scale normalization.

[0053] The typical geometric alignment of the initial 3D point cloud in this step includes the following sub-steps:

[0054] S11. Calculate the centroid of the initial 3D point cloud. , The zero-mean point set of the centered point cloud is obtained by subtracting the three-dimensional coordinates of the initial 3D point cloud from the centroid of the point cloud. Specifically, the zero-mean point set of the centered point cloud is obtained by subtracting the three-dimensional coordinates of each 3D point in the initial 3D point cloud from the three-dimensional coordinates of the centroid of the point cloud. Covariance matrix analysis is then performed on the zero-mean point set of the centered point cloud to obtain the geometric covariance matrix of the initial 3D point cloud. Obtain the geometric covariance distribution of the initial 3D point cloud:

[0055] ;

[0056] Here, ⊤ represents the matrix transpose.

[0057] S12, Geometric covariance matrix Eigenvalue decomposition is performed to obtain the orthogonal principal axis matrix as the initial rotation hypothesis. To address the random flipping of positive and negative directions caused by eigenvalue decomposition, sign disambiguation is performed based on the skewness (third statistical moment) of the one-dimensional projection distribution of the zero-mean point set of the centralized point cloud along each orthogonal principal axis direction. This forces the positive direction of eigenvalue decomposition to align with the heavier end of the distribution, thus eliminating the ambiguity of viewpoint flipping.

[0058] In this sub-step, the geometric covariance matrix is... Perform eigenvalue decomposition to obtain U is composed of three orthogonal principal axes The eigenvector matrix formed , It is a diagonal matrix formed by the corresponding eigenvalues. ,in For the three eigenvalues ​​of the point cloud along the eigenvector directions corresponding to the geometric covariance matrix, the preferred order is... The arrangement and eigenvector matrix U are used to define the initial rotation assumption orthogonal principal axis matrix of the initial 3D point cloud.

[0059] To eliminate the inherent sign ambiguity in the principal component analysis of eigenvalue decomposition of the covariance matrix of the centered point cloud obtained in step S11 and to prevent the flipping of random perspectives, this step introduces a sign disambiguation strategy based on skewness (third statistical moment). The sign function for the sign disambiguation is as follows:

[0060] .

[0061] in, The eigenvalue decomposition yields the first... eigenvectors along orthogonal principal axes , Let N represent the i-th 3D point in the initial 3D point cloud, and N represent the number of 3D points in the initial 3D point cloud. The sign function takes the skewness of the centered point cloud projected along the k-th orthogonal principal axis as input, representing the skewness direction of the projected distribution. When the input is greater than zero, it is taken as... When less than zero, take When equal to zero This sign function unifies the directions of each orthogonal principal axis to eliminate the inherent sign ambiguity in principal component analysis.

[0062] Specifically, this step involves first projecting the centered initial 3D point cloud onto each orthogonal principal axis direction to obtain the corresponding one-dimensional projection distribution; then calculating the skewness (third statistical moment) of each projection distribution to characterize the degree of skewness in that direction; if the skewness (third statistical moment) is positive, the current principal axis direction remains unchanged; if the third statistical moment is negative, the principal axis direction is reversed, thereby unifying the positive principal axis direction to the side where the point cloud distribution is heavier.

[0063] Furthermore, when the ratio of any two adjacent eigenvalues ​​in the geometric covariance matrix is ​​less than a set threshold, it is determined that there is a geometric degradation situation where the corresponding principal axis direction is unstable; wherein, the set threshold value ranges from 1.05 to 1.50, preferably 1.2. When there are at least two eigenvalues ​​that are close to each other, resulting in a geometric degradation situation where the corresponding principal axis direction is unstable, the corresponding unstable orthogonal principal axis is constrained to a predefined global reference direction, and orthogonalized in combination with the other principal axes to maintain the strict orthogonality of the rotation matrix and prevent coordinate system oscillation.

[0064] S13. Construct a rotation matrix using the disambiguated orthogonal principal axis matrix. The initial 3D point cloud of the object to be detected is mapped to a normalized aligned point cloud by using a rotation matrix.

[0065] In step S13, a rotation matrix is ​​constructed based on the disambiguated orthogonal principal axis matrix. , Composed of three pairwise orthogonal principal axis vectors after sign disambiguation, it is used to rotate the centered 3D point cloud from the original coordinate system to the normalized coordinate system, thus transforming the initial 3D point cloud of the object to be detected. Transform and scale to within a unit sphere to form a canonical aligned point cloud unaffected by the original pose. :

[0066] .

[0067] in, Let the point cloud centroid be the initial 3D point cloud. To normalize the point cloud to a unit sphere, the scaling factor is determined based on the maximum Euclidean distance from each point in the rotated point cloud to the origin. The reciprocal of this maximum Euclidean distance is taken as the scaling factor to ensure that the entire transformed point cloud falls within the unit sphere. Figure 2 As shown in the diagram. The disambiguated eigenvectors refer to the orthogonal principal axis vectors after direction unification processing in step S12, and the unstable principal axes after global vector constraint correction in the case of geometric degradation, and the rotation matrix. It consists of three sets of pairwise orthogonal principal axis vectors.

[0068] S2. Project the standardized point cloud of the object to be detected onto multiple camera viewpoints, transform the surface normals of the point cloud into the corresponding camera coordinate system for encoding, and obtain a set of geometric feature maps composed of surface normal maps of the object to be detected under multiple camera viewpoints.

[0069] The 3D point cloud of the object to be detected, after being standardized and aligned in step S1, is converted into a set of geometric feature maps under multiple viewpoints. Since the 3D point cloud of the object to be detected has been mapped to a unified standardized reference system, under the condition of fixed spherical viewpoint sampling, the local geometric regions corresponding to each viewpoint have a more stable semantic correspondence, thereby reducing the projection distribution offset caused by the original pose difference. Furthermore, the set of geometric feature maps in this invention is preferably a set of camera view surface normal maps obtained under multiple camera viewpoints, which has the characteristics of being insensitive to changes in texture and lighting and being able to highlight local surface geometric changes.

[0070] This step is implemented through the following sub-steps:

[0071] S21. Uniformly sample V camera viewpoints on a unit sphere. Preferably, camera view sampling is performed on a Fibonacci grid surrounding a unit sphere, with the number of camera viewpoints V preferably being 27. Here, the unit sphere refers to the unit sphere that encloses the normalized, aligned point cloud after scale normalization in step S13. Since the normalized, aligned point cloud of the object under test has been regularized and aligned at this point, the fixed spherical sampling is mathematically equivalent to adaptive camera placement that dynamically tracks the original object's orientation.

[0072] S22. Using a differentiable renderer, such as but not limited to PyTorch3D, render the orthogonal aligned point cloud into surface normal maps (Cam-Normal) from various camera perspectives. The preset rendering resolution is preferably 224×224, resulting in a set of surface normal maps. , , To standardize and align the surface normal map of the point cloud under the v-th camera viewpoint, this surface normal map expresses the first derivative of the 3D point cloud on the surface of the object to be detected. After being transformed into the camera coordinate system, it provides viewpoint-related geometric cues and forms a more stable local geometric semantic correspondence under standardized alignment conditions. In this process, the surface normals of the standardized aligned point cloud are encoded in the corresponding camera coordinate system, transforming the local orientation information of the 3D surface into a 2D geometric representation related to the current observation camera viewpoint, thus obtaining a geometric feature map related to the camera viewpoint.

[0073] It should be noted that the normal color encoding of the same surface region under different camera viewpoints can change with the transformation of camera coordinates. Therefore, the surface normal map does not require the color to remain unchanged across viewpoints. The key is that, under the conditions of normal alignment and fixed viewpoint sampling, the surface normal map of each camera viewpoint can stably represent the local geometry of the object to be detected and provide semantically consistent geometric clues for subsequent two-dimensional anomaly detection.

[0074] S23. During the rendering process, explicitly record the coordinate correspondence mapping between the pixel coordinates of the point cloud in the dense geometric feature map and the surface point coordinates of the normalized aligned point cloud. The two-dimensional coordinates refer to the pixel coordinates in the geometric feature map under each camera view, and the three-dimensional coordinates refer to the corresponding surface point coordinates in the normalized aligned point cloud. This is so that the point-level anomaly score can be mapped to the corresponding surface point in the normalized aligned point cloud and then mapped back to the initial three-dimensional point cloud according to the correspondence.

[0075] S3. Input the set of geometric feature maps of the object to be detected in the camera view into the two-dimensional image anomaly detection network. The two-dimensional image anomaly detection network outputs point-level anomaly scores based on the encoded features and reconstructed features of the geometric feature maps. Then, the point-level anomaly scores are back-projected onto the corresponding surface of the normalized aligned point cloud to obtain the final object-level anomaly index of the object to be detected.

[0076] If the Dinomaly algorithm model based on the frozen parameters ViT-B / 14 (DINOv2) feature extraction is selected as the two-dimensional image anomaly detection network, after inputting the geometric feature map set, the encoded features and corresponding reconstructed features of the geometric feature map of each viewpoint are extracted first. Then, based on the difference between the encoded features and the reconstructed features, a point-level anomaly score of the geometric feature map under each camera viewpoint is generated. Then, using the coordinate correspondence mapping of the point cloud transformation to the geometric feature map set under the camera viewpoint in step S2, the point-level anomaly score is mapped back to the initial three-dimensional point cloud surface points according to the standard. The final object-level anomaly index of the object to be detected is calculated, and the final high-precision point-level anomaly localization score result is generated, thus completing the closed loop from the unaligned initial three-dimensional point cloud of the object to be detected to the accurate identification of anomalies and defects.

[0077] The specific process of this step is as follows: The two-dimensional image anomaly detection network extracts the encoded feature map and the corresponding reconstructed feature map from the geometric feature map under each camera view. Based on the difference between the encoded feature and the reconstructed feature at the same pixel position under a single camera view, a pixel-level anomaly score is obtained under that camera view, which is used to characterize the degree of anomaly at the corresponding pixel position in the geometric feature map under a single camera view.

[0078] Based on the mapping relationship between the pixel coordinates of the point cloud in the geometric feature map under each camera view recorded in step S2 and the surface points of the standardized aligned point cloud, for the same 3D point in the standardized aligned point cloud, the initial encoded features and initial reconstructed features corresponding to that 3D point are extracted in the encoded feature map and reconstructed feature map of each camera view. The 3D points in the standardized aligned point cloud and the 3D points in the initial 3D point cloud correspond one-to-one through the translation, rotation and scale normalization transformation relationship in step S1. The standardized aligned point cloud can represent the 3D point cloud distribution on the surface of the object to be detected.

[0079] The initial encoded features of the same 3D point obtained from multiple camera views are arithmetically averaged, and the corresponding initial reconstructed features are arithmetically averaged to obtain the multi-view aggregated encoded features and multi-view aggregated reconstructed features of the 3D point.

[0080] The multi-view aggregated coding features and multi-view aggregated reconstructed features are normalized, the cosine distance between them is calculated, and the cosine distances obtained from multiple network layers of the two-dimensional image anomaly detection network are averaged to obtain the point-level anomaly score of the three-dimensional point, which is used to characterize the degree of anomaly of the corresponding three-dimensional point on the three-dimensional point cloud surface.

[0081] The object-level anomaly index of the object to be detected is determined by the maximum value of the point-level anomaly score of each three-dimensional point in the object to be detected, and the object-level anomaly index characterizes the overall degree of anomaly of the object to be detected.

[0082] This invention also discloses a 3D defect detection system based on typical geometric alignment and multi-view adaptive rendering, including a point cloud typical geometric alignment module 100, a multi-view conversion module 200, and a detection module 300. Specifically, as follows... Figure 1 As shown, after obtaining the initial 3D point cloud of the object to be detected, it is input into the typical geometric alignment module 100. Unsupervised feature analysis is performed on the initial 3D point cloud through typical geometric alignment to eliminate rotational variations of the 3D special orthogonal groups, obtaining a standardized aligned point cloud of the object to be detected. The specific implementation process is as shown in step S1. The standardized aligned point cloud of the object to be detected is projected onto multiple camera views through the multi-view conversion module 200. The surface normals of the point cloud are converted to the corresponding camera coordinate system for encoding, resulting in a set of geometric feature maps composed of surface normal maps of the object to be detected under multiple camera views. The specific implementation process is as shown in step S2. The detection module 300s has a built-in general 2D image anomaly detection network. The set of geometric feature maps of the object to be detected under the camera views is input into the 2D image anomaly detection network. The 2D image anomaly detection network outputs point-level anomaly scores based on the encoded features and reconstructed features of the geometric feature maps. Then, the point-level anomaly scores are back-projected onto the corresponding surface of the standardized aligned point cloud to obtain the final object-level anomaly index of the object to be detected. The specific implementation process is as shown in step S3.

[0083] To verify the effectiveness of the method described above, this embodiment was tested on the publicly available MiniShift-3D 3D anomaly detection benchmark dataset. This dataset covers samples from multiple industrial categories and can be used to evaluate the performance of 3D anomaly detection in scenarios with subtle defects. Test results show that the method of this invention has good detection performance in the task of detecting anomalies in misaligned 3D point clouds.

[0084] The testing also included comparing the scheme containing the typical geometric alignment module of the point cloud with a control scheme that removed the typical geometric alignment module, while keeping the rest of the network structure and detection process consistent. The results show that adding the typical geometric alignment module further improves anomaly detection performance, indicating that this module can effectively reduce the interference of the original point cloud pose differences on subsequent projection representation and anomaly detection results, thereby improving detection stability and anomaly recognition performance. Figure 3 As shown, after the three-dimensional point cloud of the sample to be detected is aligned with the typical geometric alignment of the point cloud in this invention, different point cloud samples can present a more stable local geometric structure correspondence under the same viewpoint, thereby reducing the projection distribution offset caused by the original pose difference and providing a more stable input representation for subsequent two-dimensional anomaly detection.

[0085] This embodiment performs three-dimensional defect detection on the surface of an object to be detected based on a computer device. The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the three-dimensional defect detection method based on typical geometric alignment and multi-view adaptive rendering described above in this embodiment.

[0086] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a step or method.

[0087] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this application. The scope of this application is defined by the appended claims.

Claims

1. A 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering, characterized in that, Includes the following steps: S1. Obtain the initial 3D point cloud of the object to be detected. Perform unsupervised feature analysis on the initial 3D point cloud through typical geometric alignment of the point cloud to eliminate the rotational variation of the 3D special orthogonal group and obtain the normalized aligned point cloud of the object to be detected. S2. Project the normalized point cloud of the object to be detected onto multiple camera views, and convert the surface normals of the point cloud to the corresponding camera coordinate system for encoding, so as to obtain a set of geometric feature maps composed of surface normal maps of the object to be detected under multiple camera views. S3. Input the set of geometric feature maps of the object to be detected in the camera view into the two-dimensional image anomaly detection network. The two-dimensional image anomaly detection network outputs point-level anomaly scores based on the encoded features and reconstructed features of the geometric feature maps. Then, the point-level anomaly scores are back-projected onto the corresponding surface of the normalized aligned point cloud to obtain the final object-level anomaly index of the object to be detected.

2. The 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering according to claim 1, characterized in that: The typical geometric alignment of the point cloud in step S1 includes the following sub-steps: S11. Calculate the centroid of the initial three-dimensional point cloud. Subtract the three-dimensional coordinates of the initial three-dimensional point cloud from the centroid to obtain the zero-mean point set of the centered point cloud. Subtract the three-dimensional coordinates of each three-dimensional point in the initial three-dimensional point cloud from the three-dimensional coordinates of the centroid to obtain the zero-mean point set of the centered point cloud. Perform covariance matrix analysis on the zero-mean point set of the centered point cloud to obtain the geometric covariance matrix of the initial three-dimensional point cloud. S12. Perform eigenvalue decomposition on the geometric covariance matrix to obtain the orthogonal principal axis matrix as the initial rotation hypothesis. For the random flipping of positive and negative directions caused by eigenvalue decomposition, perform sign disambiguation based on the skewness of the one-dimensional projection distribution of the zero-mean point set of the centralized point cloud along each orthogonal principal axis direction, and force the positive direction of eigenvalue decomposition to align with the heavier end of the distribution. S13. Construct a rotation matrix using the disambiguated orthogonal principal axis matrix, and map the initial 3D point cloud of the object to be detected into a normalized aligned point cloud using the rotation matrix.

3. The 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering according to claim 2, characterized in that: In step S12, the geometric covariance matrix is... Perform eigenvalue decomposition U is composed of three orthogonal principal axes The eigenvector matrix formed That is, defining the initial rotation assumption of the initial 3D point cloud as the orthogonal principal axis matrix. It is a diagonal matrix formed by the corresponding eigenvalues. ,in The three eigenvalues ​​representing the variance of the point cloud along the directions of the corresponding eigenvectors in the geometric covariance matrix are categorized as follows: arrangement; The symbol disambiguation introduces the following skewness-based symbol function: ； in, The eigenvalue decomposition yields the first... eigenvectors along orthogonal principal axes , Let N represent the i-th 3D point in the initial 3D point cloud, and N represent the number of 3D points in the initial 3D point cloud. The sign function is defined as follows: when the skewness of the centered point cloud projected along the k-th orthogonal principal axis is greater than zero. When less than zero, take When equal to zero .

4. The 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering according to claim 3, characterized in that: When the ratio of any two adjacent eigenvalues ​​in the geometric covariance matrix is ​​less than a set threshold, it is determined that there is a geometric degradation case where the corresponding principal axis direction is unstable. In the case of geometric degradation, the corresponding unstable orthogonal principal axis is constrained to a predefined global reference direction, and then orthogonalized in combination with the other orthogonal principal axes to maintain the strict orthogonality of the rotation matrix and prevent coordinate system oscillation.

5. The 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering according to claim 3, characterized in that: In step S13, a rotation matrix is ​​constructed based on the disambiguated orthogonal principal axis matrix. The initial 3D point cloud of the object to be detected Transform and scale to within a unit sphere to form a canonical aligned point cloud unaffected by the original pose. : ； in, To normalize the point cloud to a scaling factor within a unit sphere, the scaling factor is determined based on the maximum Euclidean distance from each point in the rotated point cloud to the origin. The centroid of the initial 3D point cloud is denoted as .

6. The 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering according to claim 3, characterized in that: Step S2 includes the following sub-steps: S21. Uniformly sample V camera viewpoints on a unit sphere; S22. Using a differentiable renderer, the normalized aligned point cloud is rendered from each camera viewpoint into a surface normal map under different camera viewpoints. In this process, the surface normals of the normalized aligned point cloud are converted to the corresponding camera coordinate system for encoding, thereby obtaining a geometric feature map related to the camera viewpoint. S23. During the rendering process, explicitly record the mapping relationship between the pixel coordinates of the point cloud in the dense geometric feature map and the coordinates of the surface points of the normalized aligned point cloud.

7. The 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering according to claim 3, characterized in that: In step S21, the camera viewpoint is sampled on a Fibonacci grid surrounding a unit sphere.

8. The 3D defect detection method based on typical geometric alignment and multi-view adaptive rendering according to claim 3, characterized in that: In step S3, the two-dimensional image anomaly detection network extracts coded feature maps and corresponding reconstructed feature maps from the geometric feature maps under each camera viewpoint. Based on the mapping of the correspondence between the pixel coordinates of the point cloud in the geometric feature map of each camera view and the surface points of the normalized aligned point cloud recorded in step S2, for the same 3D point in the normalized aligned point cloud, the initial encoded features and initial reconstructed features corresponding to the 3D point are extracted in the encoded feature map and reconstructed feature map of each camera view. The initial encoded features of the same 3D point obtained from multiple camera views are arithmetically averaged, and the corresponding initial reconstructed features are arithmetically averaged to obtain the multi-view aggregated encoded features and multi-view aggregated reconstructed features of the 3D point. The multi-view aggregated coding features and multi-view aggregated reconstructed features are normalized, the cosine distance between them is calculated, and the cosine distances obtained from multiple network layers of the two-dimensional image anomaly detection network are averaged to obtain the point-level anomaly score of the three-dimensional point. The object-level anomaly index of the object to be detected is determined by the maximum value of the point-level anomaly score of each three-dimensional point in the object to be detected.

9. A 3D defect detection system based on typical geometric alignment and multi-view adaptive rendering, characterized in that, include: The typical geometric alignment module for point clouds obtains the initial 3D point cloud of the object to be detected. It performs unsupervised feature analysis on the initial 3D point cloud through typical geometric alignment of point clouds, eliminates the rotational variation of the 3D special orthogonal group, and obtains the normalized aligned point cloud of the object to be detected. The multi-view transformation module projects the normalized aligned point cloud of the object to be detected onto multiple camera views, transforms the surface normals of the point cloud into the corresponding camera coordinate system for encoding, and obtains a set of geometric feature maps composed of surface normal maps of the object to be detected under multiple camera views. The detection module inputs the set of geometric feature maps of the object to be detected in the camera view into a two-dimensional image anomaly detection network. The two-dimensional image anomaly detection network outputs point-level anomaly scores based on the encoded and reconstructed features of the geometric feature maps. Then, the point-level anomaly scores are back-projected onto the corresponding surface of the normalized aligned point cloud to obtain the final object-level anomaly index of the object to be detected.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the three-dimensional defect detection method based on typical geometric alignment and multi-view adaptive rendering as described in any one of claims 1-8.

Images