Large-size part online in-place quality inspection system and method based on three-dimensional point cloud

Abstract

The invention particularly relates to a large-size part quality inspection system and method based on three-dimensional point clouds, the system comprises a point cloud data in-place acquisition subsystem, a data online deviation calculation subsystem and a deviation data visualization subsystem, the point cloud data in-place acquisition subsystem acquires a complete three-dimensional point cloud through single-view reconstruction and multi-view splicing; the large-scale data online deviation calculation subsystem performs preprocessing, registration and BVH tree acceleration deviation calculation; and the deviation data visualization subsystem generates a deviation cloud picture and presents the deviation cloud picture through AR technology in a virtual-real fusion manner. According to the large-size part quality inspection system based on the three-dimensional point cloud and the large-size part online in-situ quality inspection method based on the three-dimensional point cloud, non-contact in-situ detection is achieved, the splicing precision and the data processing speed are improved, manual dependence is reduced, and the system and the method have been applied to rocket barrel body quality inspection. And the detection efficiency and accuracy are ensured.

Description

Technical Field

[0001] This invention relates to the field of digital 3D measurement technology, specifically to a quality inspection system and method for large-size parts based on 3D point clouds. Background Technology

[0002] Large-sized parts play a crucial role in modern industrial products, especially in high-precision fields such as aerospace, where high machining accuracy is typically required. In the quality inspection of large-sized parts, traditional dimensional inspection methods rely primarily on manual experience using simple and limited measuring tools. Existing tools (such as inside micrometers) employ single-point measurements and multiple averaging, resulting in low overall inspection accuracy and difficulty in accurately and comprehensively reflecting the quality of critical locations. Large-sized housings are bulky and require mounting on suitable measuring machines for manual inspection, consuming significant manpower and time, thus impacting inspection efficiency. Deviation data for non-conforming parts is manually plotted on the part surface based on worker observations using simple measuring tools; this deviation detection relies on subjective experience and manual operation, making it susceptible to human error. Therefore, existing methods are unsuitable for inspecting large-sized parts, and the interaction between the data representation methods and workers is inefficient.

[0003] Structured light grating 3D reconstruction technology has gained significant attention in computer vision and industrial fields in recent years. Its applications are continuously expanding, finding increasingly in-depth use and exploration in medical diagnostics, automotive manufacturing, architectural design, and many other fields, demonstrating broad applicability and becoming a vital force driving technological innovation and development in these areas. Compared to traditional detection methods, structured light grating 3D reconstruction technology offers advantages such as high detection accuracy, non-contact measurement, and good real-time performance. Therefore, in the inspection of large-size parts, structured light grating 3D reconstruction technology can easily obtain the 3D information of the surface, and then the acquired surface 3D information can be processed to obtain the inspection results of the large-size parts. Augmented reality technology is a technology that integrates virtual information with the real world. It uses key technologies such as computer vision, environmental perception, and graphics rendering to interact with and overlay virtual digital images, models, or information onto the real environment. Therefore, thanks to the unique advantages of augmented reality technology, the inspection results of large-sized parts can be directly superimposed and mapped onto the real parts in an intuitive way, which greatly enriches the presentation dimensions of inspection information. This allows operators to perceive and understand the inspection results in real-world environments, thereby significantly improving the efficiency and depth of interaction between operators and inspection results, and promoting the rapid identification and effective utilization of inspection information.

[0004] However, the high-precision single-view 3D information of part surfaces obtained by structured light grating 3D reconstruction technology lacks positioning features, making it impossible to obtain the overall 3D information of large-sized parts by stitching together point clouds from multiple perspectives based on the features of the parts themselves. Large-sized parts lack the positioning features required for augmented reality virtual-real registration, making it difficult to overlay virtual detection results with large-sized parts in real scenes to form a virtual-real fusion effect. The analysis and processing of large-scale data consumes a lot of computing resources and time, and there is a lack of an efficient model deviation calculation method to improve the data processing process in the detection method of 3D reconstruction for large-sized parts.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To address the engineering problems of lacking online in-situ inspection methods and relying on manual experience for inspection results in the production and manufacturing of large-sized parts, this invention provides a quality inspection system and method for large-sized parts based on three-dimensional point clouds.

[0007] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0008] According to a first aspect of the present invention, an online in-situ quality inspection system for large-size parts based on three-dimensional point clouds is provided, comprising a point cloud data in-situ acquisition subsystem, an online data deviation calculation subsystem, and a deviation data visualization subsystem; The point cloud data in-situ acquisition subsystem is used to acquire three-dimensional point cloud data of the part surface, including a single-view point cloud data reconstruction module and a multi-view point cloud data stitching module. The single-view point cloud data reconstruction module acquires single-view point cloud data of the part surface and the reconstruction result of the marker points under that view based on the structured light grating reconstruction principle. The multi-view point cloud data stitching module realizes large-size point cloud stitching based on the visual detection theory of artificial marker points. The online deviation calculation subsystem includes a data preprocessing module, a model registration module, and a deviation calculation module. The data preprocessing module performs noise reduction, sampling, and encapsulation processing on the large-size measurement model. The model registration module aligns the measurement model with the standard model. The deviation calculation module calculates the deviation between the actual part and the standard model. The deviation data visualization subsystem includes a deviation cloud map drawing module and an AR visualization module. The deviation cloud map drawing module constructs a visualized deviation model, and the AR visualization module realizes the virtual-real fusion of the deviation model and the actual parts.

[0009] In some exemplary embodiments, the stitching process of the multi-view point cloud data stitching module includes: To establish the Region of Interest (ROI) of the acquired image for the overlapping field of view of binocular reconstruction, the coordinates of the center of the marker point are identified within the region and the three-dimensional spatial coordinates of the local marker point are reconstructed. Based on the geometric invariance of the Euclidean distance in the marker point space, local marker points under different viewpoints are matched and transformed to a unified coordinate system; A weighted centroid fusion method based on corresponding points is used to construct global marker points. The weights are determined based on the cumulative number of transformations from each local coordinate system to the global coordinate system or the confidence level. By applying the bundle adjustment optimization method, a dual-objective joint optimization function for reprojection error and large-size fitting error is constructed to optimize the global marker point stitching and fusion error.

[0010] In some exemplary embodiments, the data preprocessing module's processing flow includes: removing outliers based on statistical filtering algorithms, completing background segmentation through spatial clustering and marker point geometric feature recognition, simplifying data using voxel grid downsampling or curvature-preserving sampling, and achieving patch encapsulation through Poisson surface reconstruction or front advancement methods; wherein, the voxel size for downsampling and the outlier removal coefficient can be dynamically adjusted according to the detection accuracy requirements.

[0011] In some exemplary embodiments, the model registration module includes coarse registration and fine registration processes: coarse registration is based on fitting the axis of revolution of the body of revolution with global marker points and is achieved by aligning the axis vector and matching the cross-sectional contour; fine registration adopts the point-to-surface ICP algorithm, randomly sampling point clouds on the surface of the standard model and iteratively optimizing the pose until convergence.

[0012] In some exemplary embodiments, the deviation calculation module constructs a bounding volume hierarchical tree using adaptive bounding box technology to perform nearest neighbor search. Specifically, it includes: selecting an arc-shaped bounding box or a regular bounding box based on the geometric features of the part, constructing a first-level overall model and a second-level hierarchical BVH subdivision based on the sub-regions of the body of revolution, accelerating the query process through two-level search, calculating the Euclidean distance between the measurement point and the nearest neighbor of the standard model and converting it into an RGB color chromatographic model. In some exemplary embodiments, the deviation cloud map drawing module maps the deviation values ​​of the part surface to a color gradient space composed of 16 continuously transitioning RGB reference colors, evenly divides the deviation range into 15 sub-intervals, dynamically generates the corresponding RGB color for each deviation value through linear interpolation, and assigns the corresponding vertex to the measurement model to form a three-dimensional color deviation cloud map.

[0013] In some exemplary embodiments, the AR visualization module reconstructs the surface markers of the part through binocular vision, matches the pre-stored global marker coordinate system to calculate the pose of the real part in the virtual space, and generates a projected image in real time by fusing the part pose based on the camera-projector calibration parameters. It bridges the real and virtual spaces through the Vuforia identifier code, drives the deviation model to align with the real part, and realizes the virtual-real fusion display of the deviation cloud map with the help of augmented reality glasses or a projector.

[0014] According to a second aspect of the present invention, an online in-situ quality inspection method for large-size parts based on three-dimensional point clouds is provided, comprising: Step 1: Set up an on-site inspection environment including 3D scanning equipment, server, projector and AR equipment, and set artificial auxiliary marking points on the surface of the part; Step 2: Obtain single-view point cloud data and marker point reconstruction results through the structured light grating reconstruction principle, and complete multi-view point cloud stitching based on manually assisted marker points to obtain the overall three-dimensional point cloud model of the part; Step 3: Perform data preprocessing on the overall 3D point cloud model, including denoising, background segmentation, downsampling, and patch encapsulation; Step 4: Align the pose of the preprocessed measurement model with the standard design model through coarse and fine registration; Step 5: Construct a bounding volume hierarchical tree structure to quickly find the nearest neighbor points between the measurement model and the standard model and calculate the deviation; Step Six: Draw a 3D color deviation cloud map based on the deviation data; Step 7: Project the deviation cloud map onto the surface of the part or the worker's field of vision using AR equipment to achieve a virtual-real fusion display of the quality inspection results.

[0015] In some exemplary implementations, during the multi-view point cloud stitching in step two, local marker points from different perspectives are matched based on the geometric invariance of the Euclidean distance in the marker point space. After constructing global marker points by fusing the centroids of corresponding points, the stitching error is optimized through a dual-objective joint optimization function.

[0016] In some exemplary embodiments, the bounding body hierarchy tree constructed in step five is a hierarchical BVH structure, with the first-level bounding box covering the entire model, and the second-level bounding boxes subdivided according to the axial, angular or radial dimensions of the body of revolution, and the selection of arc-shaped bounding boxes or conventional bounding boxes based on the geometric features of the parts.

[0017] The large-size part quality inspection system and method based on 3D point clouds provided by the embodiments of the present invention, based on the reconstruction and pose optimization of manually assisted features, achieves accurate and efficient acquisition of large-size overall 3D information through multi-view point cloud stitching. It improves the calculation of large-size part inspection results through data processing methods more suitable for large sizes, and realizes augmented reality visualization of inspection results based on auxiliary features, promoting the digitalization and intelligentization of large-size part inspection. Compared with the prior art, it has the following beneficial effects: 1) By proposing an online in-situ quality inspection system for large-size parts based on 3D point cloud, the advantages of optical 3D measurement technology and AR technology can be utilized to improve the efficiency of online in-situ quality inspection of large-size parts, reduce the cost of quality inspection, and complete the quality inspection process in a more intelligent and automated way, so as to ensure the quality and efficiency of parts production.

[0018] 2) By proposing an effective ROI region calculation method based on the principle of binocular reconstruction, the image processing area in part surface reconstruction and marker point reconstruction is reduced, the acquisition speed of part surface point cloud is improved, and the computational cost of 3D reconstruction is reduced.

[0019] 3) By proposing a marker pose optimization method, based on an improved bundle adjustment method, the cumulative stitching error of marker points for different types of parts is reduced and smoothed, thereby improving the accuracy and stability of multi-view point cloud stitching and enhancing the precision of part surface point cloud and quality inspection process.

[0020] 4) By proposing a novel coarse registration method for point cloud models, based on the geometric characteristics of the parts, multiple fast coarse registration methods are constructed for different types of parts, thereby improving the speed of the registration process and reducing the computational cost required for the registration process.

[0021] 5) By proposing a large-scale data nearest neighbor search method, based on the bounding volume hierarchical tree structure, reasonable and effective bounding volume structures are designed for different types of parts, accelerating the nearest neighbor search and deviation calculation process, and realizing the rapid online generation of quality inspection results based on the three-dimensional point cloud of the part surface.

[0022] 6) By proposing a part pose perception method, based on the identification and matching of marker points, the part pose can be acquired in virtual space.

[0023] 7) Through HoloLens 2 glasses, workers can be online in the real world and understand the quality inspection results simply by wearing the glasses.

[0024] 8) Using projector hardware, a color cloud map of the quality inspection results is projected onto the surface of the actual parts, and the results are understood in conjunction with the scale data.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0027] Figure 1 This is a framework for an online, in-situ quality inspection system for large-size parts based on 3D point clouds. Figure 2 Workflow diagram for single-view point cloud data reconstruction module; Figure 3 Workflow diagram for the multi-view point cloud data stitching module; Figure 4 Workflow diagram for the data preprocessing module; Figure 5 Workflow diagram for model registration module; Figure 6 Here is a flowchart of the deviation calculation module. Figure 7 A workflow diagram for drawing deviation cloud maps; Figure 8 Workflow diagram for the AR visualization module; Figure 9 AR visualization effect to simulate the inspection results of parts: (a) Projector view; (b) Deviation model projection image; (c) Deviation model AR projection image. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0029] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] In related technologies, traditional manual inspection of large-sized parts is time-consuming and its results are subject to subjective factors. Contact-based automated inspection requires suitable tooling and is limited by machine tool size, making it unsuitable for excessively large parts. Online, in-situ quality inspection of large-sized parts requires digitalization and automation, enabling real-time monitoring of the entire process from measurement to interaction, reducing part movement, and improving the efficiency of quality inspection for large-sized parts.

[0031] To address the engineering challenges of lacking online in-situ inspection methods and relying on manual experience for inspection results in the manufacturing of large-sized parts, this invention, based on optical 3D measurement technology, studies methods for acquiring overall measurement data and calculating deviation results for large-sized parts. It designs an efficient interaction method between deviation detection results and operators, providing a quality inspection system based on 3D point clouds. This system acquires 3D information about the part surface and compares this information with a standard design model to determine the error between the part and the design standard, helping quality inspectors understand the measurement results. Acquiring the 3D point cloud of the part surface includes the reconstruction of single-view point cloud data and the stitching process between multi-view point cloud data. In industrial measurement, manual auxiliary marking points are often used to improve the stitching accuracy of the overall point cloud of the part. The 3D point cloud of the part surface can effectively characterize the part's dimensions and surface defect information. Compared to point cloud data acquired by traditional contact measurement, optical measurement technology enables in-situ inspection, offering faster, lower-cost, and less data loss when acquiring 3D point cloud data. This quality inspection system compares the 3D point cloud of a part's surface with a digital standard model. After data processing, it calculates the deviation between the actual part and the design standard. The point cloud-based quality inspection method can quickly measure the error between various positions of the part and the design standard online, offering fast measurement speed and avoiding missed or incorrect inspections. The system also generates deviation cloud maps to help workers understand the measurement results calculated from the point cloud deviation. The deviation cloud maps, combined with a scale, improve the interaction between workers and pure data, reducing reliance on workers' quality inspection experience and improving the speed and accuracy of inspections. Based on the above-mentioned 3D point cloud measurement quality inspection concept, an online in-situ quality inspection system and method for large-size parts based on 3D point clouds are proposed and applied to the quality inspection of rocket launcher bodies.

[0032] refer to Figure 1 As shown, an online in-situ quality inspection system for large-size parts based on 3D point clouds consists of three subsystems: an in-situ point cloud data acquisition subsystem, including a single-view point cloud data reconstruction module and a multi-view point cloud data stitching module; a large-scale data online deviation calculation subsystem, including a data preprocessing module (denoising), a model registration module (aligning with the model), and a deviation calculation module (calculating deviation); and a deviation data visualization subsystem, including a deviation cloud map drawing module and an AR visualization module. The single-view point cloud data reconstruction module in the point cloud data in-situ acquisition subsystem is based on the structured light grating reconstruction principle to acquire single-view point cloud data of the part surface and reconstruct the marker points under that view. The multi-view point cloud data stitching module is based on the theory of visual detection of artificial marker points and realizes a large-size point cloud stitching method based on artificially assisted marker points: ROI regions of the acquired images are established for the overlapping fields of view of binocular reconstruction to improve the reconstruction speed of marker points; based on the geometric invariance of Euclidean distance in marker point space, local marker points reconstructed under different views are matched and transformed to a unified coordinate system; a weighted centroid fusion method based on corresponding points is designed to initially construct global marker points; and a bundle adjustment optimization method is applied to construct a dual-objective joint optimization function for reprojection error and large-size fitting error to reduce global marker point stitching and fusion errors and smooth the error peaks of adjacent views. The data preprocessing module in the large-scale online deviation calculation subsystem removes, samples, and encapsulates interference point clouds for large-size measurement models. In the model registration module, a coarse registration method based on fitting axes using global marker points is proposed to avoid sampling the entire measurement data and reduce the memory and time costs of registration calculation. The deviation calculation module constructs a data structure index tree for nearest neighbor search of large-scale measurement data and applies BVH tree index structures of different types of bounding boxes to reduce the memory cost of constructing an accurate nearest neighbor search tree and quickly filter out irrelevant 3D spaces. The deviation cloud map drawing module in the deviation data visualization subsystem assigns RGB parameter values ​​to the nearest neighbor deviation calculation results to construct a visualized deviation model. The AR visualization module, based on the positioning principle of marker points, conducts research on a virtual-real fusion registration method for deviation models lacking positioning features, specifically for large-sized parts. This involves using manually assisted marker points to perceive and locate the model's pose in real space, and using the 3D reconstruction information from these marker points to locate the model's pose in virtual space. Using both augmented reality glasses and a projector, a virtual-real positioning and registration method for the deviation model is established. Using the part surface as a carrier, AR virtual-real fusion of the deviation model cloud map and the actual large-sized part is performed. Ultimately, this achieves online, in-situ quality inspection of large-sized parts.

[0033] The single-view point cloud data reconstruction module projects phase-encoded structured light (such as a phase-shift + Gray code combined grating) onto the surface of the part via an optomechanical system. Deformed grating images are synchronously acquired by dual industrial cameras whose spatial positions have been jointly calibrated. Based on the dual-camera calibration parameters (R, T, K, D) and grating phase decoding technology, the absolute phase maps of the left and right camera views are calculated respectively. Using the same phase value constraint and epipolar geometric relationship, accurate matching of binocular pixels is achieved. Finally, the three-dimensional point cloud data of the part surface under the current acquisition view is reconstructed through triangulation.

[0034] The multi-view point cloud data stitching module includes the following processes: 1) Rapid Identification and Reconstruction of Local Markers. Using local viewpoint images (including manually marked points) acquired by a binocular camera system as input, a rapid ROI (Region of Interest) extraction method based on the reconstructed region is proposed, targeting the effective area (overlapping field of view) for binocular reconstruction. This method pre-calculates the region in the image that can be effectively reconstructed in 3D based on camera calibration parameters, placement angle, and a preset acquisition distance threshold, avoiding image processing of invalid areas. Image processing (such as binarization and edge extraction) is performed within the ROI region, and algorithms such as ellipse fitting are used to identify the center coordinates of the marked points with high accuracy and speed. The correspondence between the centers of corresponding circles in the left and right images is determined through binocular stereo matching technology. Finally, the 3D spatial coordinates of the marked points in this local coordinate system are reconstructed using the principle of binocular vision (parallax calculation).

[0035] 2) Global Coordinate System Establishment and Global Marker Point Construction. Using the reconstructed 3D coordinate sets of marker points from various local perspectives as input, and based on the geometric invariance of the Euclidean distance in marker point space, the same physical marker point (corresponding points) observed from different local perspectives is matched. By calculating the optimal rigid body transformation (rotation matrix and translation vector) between the matched point sets, all local marker points are initially transformed into a unified global coordinate system. A spatial feature matching algorithm (such as multi-point verification matching based on distance invariance) is used to solve the pose transformation matrix, obtaining that the same physical marker point may have multiple initially transformed coordinates (from different perspectives) in the global coordinate system.

[0036] 3) Fusion of Same-Name Marker Points and Global Pose Optimization. Multiple preliminary coordinates of the same physical marker point in the global coordinate system are used as input. These multiple preliminary global coordinates of the same physical marker point are fused to generate a unique globally optimal coordinate for that marker point. A weighted centroid fusion method is proposed, with weights determined based on the cumulative number of transformations (or confidence level) from each local coordinate system to the global coordinate system, effectively reducing fusion errors. To eliminate accumulated errors during the stitching process and improve robustness, an improved bundle adjustment method is used for global optimization. A novel dual-objective joint optimization function is constructed, targeting reprojection error and volumetric fitting error. The reprojection error objective refers to minimizing the deviation between the 3D coordinates of the marker point projected back to the imaging planes of each camera and the coordinates of the original observed image, while ignoring minor changes in camera intrinsic parameters. The volumetric fitting error refers to minimizing the deviation between the marker point coordinates and the distance to the initially fitted axis of the volumetric system using geometric constraints. This dual-objective optimization simultaneously constrains the camera pose parameters and the global coordinates of the marker points. While ensuring stitching accuracy, it effectively suppresses data divergence and unreasonable optimization directions, significantly improving the stability and convergence speed of the optimization process. This results in a high-precision set of global marker point 3D coordinates and the corresponding optimized camera pose parameters.

[0037] 4) Rapid Point Cloud Stitching Based on Global Markers. The input consists of single-view point cloud data, the 2D coordinates of the markers identified in the local view image, and the optimized 3D coordinates of the global markers. The 2D markers identified in the local view image are matched with the known 3D coordinates of the global markers. Using the successfully matched point pairs, the optimal rigid body transformation matrix (rotation matrix R and translation vector T) from the local view coordinate system to the global coordinate system is calculated. Using the optimized global markers as a stable reference, the transformation from local to global is directly solved, avoiding the error accumulation problem of traditional adjacent view stitching. The calculated transformation matrix (R, T) is applied to transform the local single-view point cloud data to the global coordinate system. After all view point clouds have been transformed, they are fused in the global coordinate system (e.g., removing overlapping points), finally outputting a complete and accurate global 3D point cloud model of a body of revolution.

[0038] The multi-view point cloud data stitching module effectively solves the problem of lacking stitching features on the surface of rotating parts. Global marker point construction and optimization significantly reduce stitching accumulation errors. Rapid ROI extraction improves local reconstruction efficiency, and weighted centroid fusion reduces errors at corresponding points. A dual-objective (reprojection + rotating body fitting) bundle adjustment method is used for global optimization, improving accuracy and robustness. The dual-objective bundle adjustment method enhances the robustness of the optimization process, directly calculating the local-to-global transformation using optimized global marker points, avoiding accumulated errors.

[0039] The data preprocessing module receives the original global stitched point cloud output from the multi-view point cloud stitching module. Through a series of optimization processes, it reduces the scale of measurement data and generates a clean, concise, and topologically complete 3D model to be analyzed, laying the foundation for subsequent deviation calculation. Its core processing flow includes point cloud denoising (outlier removal, background segmentation), data simplification (downsampling), and surface reconstruction (patch encapsulation). The outlier removal unit calculates the average distance and standard deviation of each point and its K nearest neighbors based on a statistical filtering algorithm, and identifies and removes noise points according to a dynamic threshold. The background segmentation unit separates and deletes auxiliary marker point clouds through spatial clustering segmentation and geometric feature identification of marker points, retaining the workpiece body point cloud. The downsampling unit uses voxel mesh downsampling or curvature-preserving sampling to retain key geometric features while controlling the data scale. The patch encapsulation unit converts the point cloud into a triangular mesh model using Poisson surface reconstruction or front-stepping method, establishes topological relationships, and compresses the data. The voxel size of the downsampling unit and the coefficients of the outlier removal unit can be dynamically adjusted according to the required detection accuracy.

[0040] The model registration module is responsible for aligning the pose of the measurement model (a high-precision point cloud of the surface of a body of revolution obtained by stitching together multi-view point clouds) with the standard model (design model), including two processes: coarse registration and fine registration. 1) Coarse registration: Based on the global marker points, fit the axis of the rotating body (PCA principal component analysis combined with gradient descent optimization), and achieve coarse registration by aligning the axis vectors—adjusting the axis direction of the measurement model and the standard model to be parallel and matching the cross-sectional contour—extracting the two-dimensional contour through the axis, analyzing the normal features of the contour points and matching them with the contour of the standard model, and calculating the axial displacement.

[0041] 2) Fine registration: The point-to-surface ICP algorithm is used to randomly sample point clouds on the surface of the standard model and iteratively optimize the pose until convergence.

[0042] The model's registration module avoids relying on manual feature localization for coarse registration. It resolves registration ambiguities caused by repetitive local features of the body of revolution through axis and contour analysis. The axis fitting based on marker points is significantly faster than global point cloud fitting. Fine registration improves ICP convergence efficiency with the good initial values ​​provided by coarse registration.

[0043] The deviation calculation module receives the registered part point cloud and standard model, constructs a bounding volume hierarchy tree (BVH) using adaptive bounding box technology, and efficiently performs nearest neighbor search: 1) Dynamic bounding box selection: Select an arc-shaped bounding box or a conventional bounding box based on the geometric features of the part (such as the axis of rotation) to optimize the space division.

[0044] 2) Layered BVH construction: The first-level bounding box covers the entire model, and the second-level bounding box is subdivided according to the sub-regions of revolution (axial / angular / radial) to reduce redundant calculations.

[0045] 3) Accelerate queries: Quickly filter irrelevant areas through spatial relationships, and improve the efficiency of large-scale data processing by combining two-level search (coarse filtering + fine matching).

[0046] 4) Deviation output: Calculate the Euclidean distance between the measurement point and the nearest neighbor of the standard model, and convert it into an RGB color chromatogram model to intuitively display the deviation distribution.

[0047] In this deviation calculation module, a bounding box hierarchical tree structure is constructed to accelerate the nearest neighbor detection process in deviation calculation. To address the spatial non-uniformity of the measurement point cloud on the surface of the rotating body, an intersection detection method for the bounding box is designed. This significantly improves the query efficiency for rotating body parts, achieving a speed improvement of over 40% compared to the traditional KD tree method.

[0048] The deviation cloud map drawing module is responsible for drawing the deviation cloud map of the part deviation calculation results. The color information of the points in the cloud map uses RGB information as parameters. Based on the scale design, the deviation at different positions on the part surface is assigned corresponding values ​​to construct the deviation cloud map of the part inspection results. The deviation value of the part surface is mapped to a preset color gradient space, which consists of 16 continuously transitioning RGB reference colors, covering the complete color spectrum from the maximum negative deviation to the maximum positive deviation. The deviation range is evenly divided into 15 sub-intervals, each sub-interval corresponding to the transition segment of two adjacent reference colors. For each deviation value, the position of its sub-interval is calculated, and linear interpolation is performed between the corresponding adjacent reference colors to dynamically generate the RGB color corresponding to the deviation value. The generated RGB color is assigned to the corresponding vertex of the measurement model to form a three-dimensional color cloud map that intuitively expresses the deviation distribution.

[0049] The AR visualization module is responsible for displaying the quality inspection results on the real parts by using the deviation cloud map in the server as a carrier on the surface of the parts. It reconstructs the marker points on the part surface through binocular vision, matches them with a pre-stored global marker point coordinate system, and calculates the pose of the real part in virtual space to achieve pose perception. Based on camera-projector calibration parameters, it fuses the part pose to generate a projected image in real time, which is directly projected onto the part surface. It bridges the real and virtual spaces through Vuforia identifiers, adjusting the identifier pose in the virtual scene to drive the alignment of the deviation model with the real part. This requires recognizing the pose of the AR device and the part in virtual space using a camera and 3D reconstruction methods, resulting in a virtual-real fusion of the deviation cloud map of the quality inspection results and the part in the worker's real field of vision.

[0050] In the quality inspection of large-sized parts, traditional manual inspection is time-consuming and the results are affected by subjective factors. Contact-based automated inspection requires the design of suitable tooling and is limited by machine tool size, making it impossible to inspect excessively large parts. Online in-situ quality inspection of large-sized parts requires digital and automated inspection, realizing the entire process from measurement to interaction in real time, reducing part movement, and improving the efficiency of quality inspection of large-sized parts.

[0051] As another aspect of the present invention, a method for quality inspection of large-size parts based on three-dimensional point clouds is provided, specifically including the following steps: Step 1: On-site equipment setup and deployment. In-situ quality inspection is implemented at the production site of the shell parts of a large piece of equipment. 3D scanning equipment, such as two high-resolution industrial cameras, is set up; a server is deployed for data storage and computation; a projector and HoloLens device are deployed for AR interaction; and manually assisted marker points are added.

[0052] Step Two: Acquisition of Overall Point Cloud of the Part. The point cloud data reconstruction and multi-view point cloud stitching program is initiated. A high-precision point cloud of the large-sized part surface is acquired using a structured light grating-based 3D reconstruction method. This includes high-precision camera calibration, ROI image region extraction, grating encoding / decoding, and point cloud calculation / generation. The multi-view point cloud data of the large-sized part is then stitched and optimized. Manually assisted marker points are used to improve the overall point cloud stitching accuracy, and the reconstructed 3D coordinates of the marker points are globally optimized to further reduce cumulative errors caused by stitching.

[0053] Step 3: Point Cloud Data Preprocessing. Start the data preprocessing program to accurately represent the point cloud data of the part surface by removing outliers, segmenting the point cloud background, performing appropriate downsampling, and optimizing the point cloud data encapsulation.

[0054] Step 4: Registration of the point cloud model with the standard model. Start the model registration program, automatically determine the geometric shape characteristics of the parts, and achieve coarse registration between models by aligning feature points, feature lines, and feature surfaces; randomly sample an appropriate number of points for ICP iterative fine registration to achieve fine registration between models.

[0055] Step 5: Part Deviation Calculation. Start the deviation calculation program, establish the bounding volume hierarchical tree data structure of the standard design model, find the nearest neighbor of the point in the measurement model in the standard model, calculate the distance, and store it in the database.

[0056] Step Six: Deviation Data Cloud Plotting. Start the deviation cloud plotting program, filter the deviation data, design an appropriate scale, assign RGB parameter values ​​to different deviation distance points, and plot a cloud map to visualize the measurement results.

[0057] Step 7: AR Visualization of Deviation Cloud Map. Launch the AR visualization interactive program. Workers wear AR helmets or turn on projection AR devices. In a real measurement scenario, based on auxiliary features or identification codes, the deviation cloud map of the detection results is accurately projected onto the part surface or into the worker's field of vision using the AR device.

[0058] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0059] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0060] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is defined only by the appended claims.

Claims

1. A quality inspection system for large-size parts based on three-dimensional point clouds, characterized in that, It includes a point cloud data in-situ acquisition subsystem, a data online deviation calculation subsystem, and a deviation data visualization subsystem; The point cloud data in-situ acquisition subsystem is used to acquire three-dimensional point cloud data of the part surface, including a single-view point cloud data reconstruction module and a multi-view point cloud data stitching module. The single-view point cloud data reconstruction module acquires single-view point cloud data of the part surface and the reconstruction result of the marker points under that view based on the structured light grating reconstruction principle. The multi-view point cloud data stitching module realizes large-size point cloud stitching based on the visual detection theory of artificial marker points. The online deviation calculation subsystem includes a data preprocessing module, a model registration module, and a deviation calculation module. The data preprocessing module performs noise reduction, sampling, and encapsulation processing on the large-size measurement model. The model registration module aligns the measurement model with the standard model. The deviation calculation module calculates the deviation between the actual part and the standard model. The deviation data visualization subsystem includes a deviation cloud map drawing module and an AR visualization module. The deviation cloud map drawing module constructs a visualized deviation model, and the AR visualization module realizes the virtual-real fusion of the deviation model and the actual parts.

2. The method according to claim 1, characterized in that, The stitching process of the multi-view point cloud data stitching module includes: To establish the Region of Interest (ROI) of the acquired image for the overlapping field of view of binocular reconstruction, the coordinates of the center of the marker point are identified within the region and the three-dimensional spatial coordinates of the local marker point are reconstructed. Based on the geometric invariance of the Euclidean distance in the marker point space, local marker points under different viewpoints are matched and transformed to a unified coordinate system; A weighted centroid fusion method based on corresponding points is used to construct global marker points. The weights are determined based on the cumulative number of transformations from each local coordinate system to the global coordinate system or the confidence level. By applying the bundle adjustment optimization method, a dual-objective joint optimization function for reprojection error and large-size fitting error is constructed to optimize the global marker point stitching and fusion error.

3. The method according to claim 1, characterized in that, The data preprocessing module's processing flow includes: removing outliers based on statistical filtering algorithms, completing background segmentation through spatial clustering and geometric feature recognition of marker points, simplifying data using voxel grid downsampling or curvature-preserving sampling, and achieving patch encapsulation through Poisson surface reconstruction or front advancement methods; wherein, the voxel size for downsampling and the outlier removal coefficient can be dynamically adjusted according to the detection accuracy requirements.

4. The method according to claim 1, characterized in that, The model registration module includes coarse registration and fine registration processes: coarse registration is based on fitting the axis of revolution of the body with global marker points and is achieved by aligning the axis vector and matching the cross-sectional contour; fine registration adopts the point-to-surface ICP algorithm, randomly sampling point clouds on the surface of the standard model and iteratively optimizing the pose until convergence.

5. The method according to claim 1, characterized in that, The deviation calculation module constructs a bounding volume hierarchical tree using adaptive bounding box technology to perform nearest neighbor search. Specifically, it includes: selecting an arc-shaped bounding box or a regular bounding box based on the geometric features of the part, constructing a first-level overall model and a second-level hierarchical BVH subdivision based on the sub-regions of the body of revolution, accelerating the query process through two-level search, calculating the Euclidean distance between the measurement point and the nearest neighbor of the standard model and converting it into an RGB color chromatographic model.

6. The method according to claim 1, characterized in that, The deviation cloud map drawing module maps the deviation values ​​of the part surface to a color gradient space composed of 16 continuously transitioning RGB reference colors, evenly divides the deviation range into 15 sub-intervals, dynamically generates the corresponding RGB color for each deviation value through linear interpolation, and assigns the corresponding vertex to the measurement model to form a three-dimensional color deviation cloud map.

7. The method according to claim 1, characterized in that, The AR visualization module reconstructs the surface markers of the parts through binocular vision, matches them with the pre-stored global marker coordinate system to calculate the pose of the real parts in the virtual space, and generates a projected image in real time by fusing the part pose based on the camera-projector calibration parameters. It bridges the real and virtual spaces through the Vuforia identifier code, drives the deviation model to align with the real parts, and realizes the virtual-real fusion display of the deviation cloud map with the help of augmented reality glasses or a projector.

8. A method for quality inspection of large-size parts based on three-dimensional point clouds, implemented using the system described in any one of claims 1-7, characterized in that, include: Step 1: Set up an on-site inspection environment including 3D scanning equipment, server, projector and AR equipment, and set artificial auxiliary marking points on the surface of the part; Step 2: Obtain single-view point cloud data and marker point reconstruction results through the structured light grating reconstruction principle, and complete multi-view point cloud stitching based on manually assisted marker points to obtain the overall three-dimensional point cloud model of the part; Step 3: Perform data preprocessing on the overall 3D point cloud model, including denoising, background segmentation, downsampling, and patch encapsulation; Step 4: Align the pose of the preprocessed measurement model with the standard design model through coarse and fine registration; Step 5: Construct a bounding volume hierarchical tree structure to quickly find the nearest neighbor points between the measurement model and the standard model and calculate the deviation; Step Six: Draw a 3D color deviation cloud map based on the deviation data; Step 7: Project the deviation cloud map onto the surface of the part or the worker's field of vision using AR equipment to achieve a virtual-real fusion display of the quality inspection results.

9. The method for quality inspection of large-size parts based on three-dimensional point clouds according to claim 8, characterized in that, In step two, when stitching multi-view point clouds, local marker points from different perspectives are matched based on the geometric invariance of the Euclidean distance in the marker point space. After constructing global marker points by fusing the centroids of corresponding points, the stitching error is optimized through a dual-objective joint optimization function.

10. The method for quality inspection of large-size parts based on three-dimensional point clouds according to claim 8, characterized in that, The bounding volume hierarchy tree constructed in step five is a hierarchical BVH structure. The first-level bounding box covers the entire model, and the second-level bounding boxes are subdivided according to the axis, angle or radial direction of the body of revolution. Arc-shaped bounding boxes or conventional bounding boxes are selected according to the geometric features of the parts.

Images