Three-dimensional scanning point cloud reconstruction method, system, device and medium for complex workpieces

Abstract

The application provides a three-dimensional scanning point cloud reconstruction method, system, device and medium for complex workpieces, comprising: processing initial scanning data of a complex workpiece to obtain a target point-normal set; generating a candidate scanning viewpoint set according to the set; performing visibility evaluation, registrability evaluation and robot executability screening on the viewpoint set to obtain a scanning viewpoint sequence; sequentially moving to each scanning pose according to the sequence, and controlling a three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point clouds, and obtaining overlapping sub-clouds according to the point clouds combined with a nearest neighbor constraint strategy; performing progressive continuous fine registration on adjacent frame point clouds according to the overlapping sub-clouds to obtain fine registration poses of each frame of point clouds relative to a target frame; and fusing multiple frames of local point clouds according to the poses to obtain fused point clouds; and the above method improves the completeness, splicing accuracy and automation degree of three-dimensional point cloud model reconstruction of complex workpieces.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional vision measurement and three-dimensional model reconstruction technology, and in particular to a method, system, device and medium for three-dimensional scanning point cloud reconstruction of complex workpieces. Background Technology

[0002] With increasing demands for dimensional accuracy, assembly consistency, and surface quality in high-end equipment manufacturing, aerospace, automotive manufacturing, energy equipment, precision molds, and industrial quality inspection, 3D digital measurement of complex workpieces has gradually become a crucial foundational technology for manufacturing process control, reverse engineering, quality inspection, and digital archiving. Complex workpieces typically possess characteristics such as free-form surfaces, deep cavities, holes and grooves, thin-walled edges, abrupt changes in local curvature, occluded areas, and repetitive geometric structures, making it difficult to fully represent their surface morphology using only a small number of discrete measurement points. Therefore, acquiring high-density point clouds using 3D vision measurement equipment and then stitching together multi-viewpoint point clouds to form a complete 3D model has become an important technical approach for the digital measurement of complex workpieces.

[0003] Traditional multi-view 3D scanning can be achieved through manual handheld scanning, turntable scanning, multi-camera fixed arrays, or automated robotic scanning. While manual handheld scanning offers high flexibility, the scanning path, overlapping areas, and acquisition quality rely heavily on operator experience, making it difficult to guarantee repeatability and automation. Turntable scanning is suitable for workpieces with relatively regular shapes and minimal or no obstructions, but for workpieces with deep cavities, sidewalls, back surfaces, inside holes and grooves, or in complex clamping conditions, simply relying on turntable rotation is insufficient to capture the entire surface. Multi-camera fixed arrays can acquire data simultaneously from multiple directions, but the system cost, installation space, and multi-camera calibration complexity are high, and their adaptability to workpieces of different sizes and orientations is limited. In contrast, mounting 3D scanning equipment at the end effector of an industrial robot—a "robot eye on the hand" 3D scanning method—allows the robot's multi-degree-of-freedom motion capabilities to adjust the camera's position and orientation, enabling the sensor to approach the workpiece surface according to a planned viewpoint. However, this "robot eye on the hand" 3D scanning method transforms the high-precision reconstruction of complex workpieces from a single-sensor imaging problem into a systemic issue determined by the sensor's field of view, workpiece geometry, robot motion accuracy, overlap between viewpoints, local point cloud quality, and registration convergence conditions. Especially in scenarios where a high-precision, small-field-of-view 3D camera is mounted at the robot's end effector, complete workpiece reconstruction requires multiple shots and multi-frame point cloud stitching. Furthermore, robot absolute positioning errors and stiffness errors can cause deviations in the initial positions of point clouds in each frame.

[0004] Therefore, how to unify viewpoint planning, robot executability, overlapping relationships of adjacent point clouds, continuous registration, and quality evaluation to become the key technical foundation for achieving high integrity and high precision 3D reconstruction of complex workpieces is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention aims to provide a method, system, device, and medium for 3D scanning point cloud reconstruction of complex workpieces. It addresses the issues of small field of view in high-precision 3D cameras and insufficient absolute positioning accuracy and stiffness of robots to directly guarantee seamless point cloud stitching. During the scanning viewpoint planning stage, the invention simultaneously considers workpiece surface coverage, robot executability, and the registrationability of adjacent point clouds. After acquisition, the robot pose is used as a coarse initial value to extract reliable overlapping sub-clouds from adjacent point clouds. Multi-scale, progressive continuous registration corrects the pose errors between point clouds in each frame. Simultaneously, based on registration quality, coverage, and local stitching residuals, it determines whether additional scanning is needed. This forms a closed-loop process of "viewpoint planning—robot scanning—overlapping sub-cloud extraction—continuous registration—quality evaluation—feedback additional scanning—point cloud fusion," improving the completeness, stitching accuracy, and automation level of 3D reconstruction of complex workpieces.

[0006] The first objective of this invention is to provide a method for reconstructing three-dimensional scanning point clouds of complex workpieces; The technical solution provided by this invention is as follows: A method for reconstructing 3D scanning point clouds of complex workpieces includes the following steps: The initial scan data of the complex workpiece is processed to obtain the target point-normal set; A candidate scan viewpoint set is generated based on the target point-normal set; The candidate scanning viewpoint set is subjected to visibility evaluation, registration evaluation, and robot executability screening to obtain a scanning viewpoint sequence; According to the scanning viewpoint sequence, the robot moves to each scanning pose in sequence and controls the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point cloud. The overlapping sub-cloud is obtained by combining the multiple frames of local point cloud with the nearest neighbor constraint strategy. Based on the overlapping sub-clouds, progressive continuous fine registration is performed on the point clouds of adjacent frames to obtain the fine registration pose of each frame point cloud relative to the target frame. The local point clouds of multiple frames are fused according to the finely matched pose to obtain a fused point cloud.

[0007] Preferably, the process of processing the initial scan data of the acquired complex workpiece to obtain the target point-normal set specifically includes: The input model is processed by unifying coordinates, unifying units, removing outliers, and downsampling to obtain the original target point set; Calculate the corresponding normal vector for the neighborhood point set in the original target point set to obtain the target point-normal set.

[0008] Preferably, generating a candidate scan viewpoint set based on the target point-normal set specifically includes: A set of candidate scanning viewpoints is generated based on the spatial location of the target point-normal set, surface normals, optimal working distance of the scanner, field of view, depth of field, allowable incident angle range, and robot end-effector posture constraints.

[0009] Preferably, the visibility evaluation of the candidate scanning viewpoint set specifically includes: The visibility of the candidate scanning viewpoint set is evaluated, and a viewpoint-target point visibility matrix is ​​established.

[0010] Preferably, the registrability evaluation of the candidate scanning viewpoint set specifically includes: Calculate the set of target points that are commonly visible to any two candidate viewpoints based on the visibility matrix; The overlap rate between viewpoints is obtained based on the set of target points. A viewpoint registration score is constructed based on the viewpoint overlap rate.

[0011] Preferably, the candidate scanning viewpoint set is subjected to robot executability screening, specifically including: The robot end-effector pose is obtained based on the calibration relationship between the robot base coordinate system, workpiece coordinate system, camera coordinate system, and end-effector coordinate system. The robot end-effector pose is subjected to inverse kinematics solution, joint limit judgment, singularity judgment, tool and workpiece collision detection, robot body and workpiece collision detection, robot and worktable collision detection, and approach path feasibility detection to eliminate unreachable, unsafe, or unstable viewpoints from the candidate scanning viewpoint set.

[0012] Preferably, the step of performing visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set to obtain a scanning viewpoint sequence specifically includes: After performing visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set, the scanning viewpoint sequence is solved under the constraints of coverage, number of viewpoints, scanning quality, robot motion cost, and registration of adjacent viewpoints.

[0013] Preferably, the step of sequentially moving to each scanning pose according to the scanning viewpoint sequence, controlling the 3D scanning sensor installed at the robot's end effector to acquire multiple frames of local point clouds, and obtaining overlapping sub-clouds based on the multiple frames of local point clouds combined with a nearest neighbor constraint strategy specifically includes: According to the scanning viewpoint sequence, the robot moves to each scanning pose in sequence and controls the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point cloud. Based on the robot's forward kinematics and hand-eye calibration relationship, a coarse initial pose is provided for the multi-frame local point cloud; Calculate the nearest neighbor distance from each point in the source point cloud to the target point cloud based on the coarse initial pose; An adaptive seed threshold is determined based on the intrinsic resolution of the nearest neighbor point cloud and the camera resolution. Points whose nearest neighbor distance is less than the adaptive seed threshold are used as candidate overlapping seeds, and unstable corresponding points are further eliminated by mutual nearest neighbor constraint. The radius is expanded using seed points that satisfy the nearest neighbor constraint to obtain overlapping sub-clouds of source and target point clouds. The overlapping sub-clouds include: the source point cloud overlapping sub-cloud and the target point cloud overlapping sub-cloud.

[0014] Preferably, the nearest neighbor constraint specifically refers to: If the nearest neighbor of a point in the source point cloud is a point in the target point cloud, and the nearest neighbor of a point in the target point cloud is also a point in the source point cloud, then the points in the source point cloud and the points in the target point cloud are considered to satisfy the nearest neighbor constraint.

[0015] Preferably, the step of performing progressive continuous fine registration of adjacent frame point clouds based on the overlapping sub-clouds to obtain the fine registration pose of each frame point cloud relative to the target frame specifically includes: After progressively optimizing the point clouds of adjacent frames at multiple scales based on a multi-scale voxel sequence from coarse to fine and a gradually shrinking registration distance threshold, the fine registration pose of each frame point cloud relative to the target frame is obtained.

[0016] Preferably, the step of fusing multiple frames of local point clouds based on the finely registered pose to obtain a fused point cloud specifically includes: Based on the finely matched pose, the local point clouds of multiple frames are subjected to repeated point merging, voxel downsampling, outlier removal, normal unification, boundary smoothing, and mesh reconstruction to obtain the fused point cloud.

[0017] Preferably, before fusing the multi-frame local point clouds according to the finely registered pose to obtain the fused point cloud, the method further includes: When insufficient target coverage, unscanned local areas, insufficient effective overlap between adjacent point clouds, registration quality index below the threshold, excessive loop closure error, or obvious gaps, ghosting, steps, and low-density areas are detected in the fused point cloud, feedback scanning is automatically triggered to regenerate candidate supplementary viewpoints.

[0018] The second objective of this invention is to provide a three-dimensional scanning point cloud reconstruction system for complex workpieces; The technical solution provided by this invention is as follows: A 3D scanning point cloud reconstruction system for complex workpieces includes: a target point set construction module, a candidate viewpoint generation module, a scanning viewpoint sequence solving module, an overlapping sub-cloud extraction module, a progressive continuous registration module, and a point cloud fusion output module. The target point set construction module is used to process the initial scan data of the acquired complex workpiece to obtain a target point-normal set. The candidate viewpoint generation module is used to generate a candidate scan viewpoint set based on the target point-normal set; The scanning viewpoint sequence solving module is used to perform visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set to obtain a scanning viewpoint sequence. The overlapping sub-cloud extraction module is used to move sequentially to each scanning pose according to the scanning viewpoint sequence, control the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point clouds, and obtain overlapping sub-clouds based on the multiple frames of local point clouds combined with the nearest neighbor constraint strategy. The progressive continuous registration module is used to perform progressive continuous fine registration on the point clouds of adjacent frames according to the overlapping sub-clouds, so as to obtain the fine registration pose of each frame point cloud relative to the target frame. The point cloud fusion output module is used to fuse multiple frames of local point clouds according to the finely registered pose to obtain a fused point cloud.

[0019] The third objective of this invention is to provide an electronic device; The technical solution provided by this invention is as follows: An electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor, the memory storing a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform any of the method steps of the three-dimensional scanning point cloud reconstruction method for complex workpieces.

[0020] A fourth objective of this invention is to provide a computer-readable storage medium; The technical solution provided by this invention is as follows: A computer-readable storage medium for storing a computer program for causing a computer to perform the steps of any one of the methods for reconstructing a three-dimensional scanning point cloud for complex workpieces.

[0021] Compared with existing technologies, this invention provides a method for reconstructing 3D scanning point clouds of complex workpieces, comprising the following steps: processing the initial scanning data of the complex workpiece to obtain a target point-normal set; generating a candidate scanning viewpoint set based on the target point-normal set; performing visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set to obtain a scanning viewpoint sequence; sequentially moving to each scanning pose according to the scanning viewpoint sequence, and controlling a 3D scanning sensor mounted on the end effector of the robot to acquire multiple frames of local point clouds, and obtaining overlapping sub-clouds based on the multiple frames of local point clouds combined with a nearest neighbor constraint strategy; and further... Progressive continuous fine registration is performed on adjacent frame point clouds based on the overlapping sub-clouds to obtain the fine registration pose of each frame point cloud relative to the target frame; the local point clouds of multiple frames are fused based on the fine registration poses to obtain the fused point cloud; the above method forms a closed-loop process of "viewpoint planning - robot scanning - overlapping sub-cloud extraction - continuous registration - quality evaluation - feedback scan - point cloud fusion", which improves the integrity, stitching accuracy and automation of 3D reconstruction of complex workpieces. It unifies the consideration of viewpoint planning, robot executability, overlapping relationship of adjacent point clouds, continuous registration and quality evaluation, and becomes the key technical foundation for realizing high integrity and high precision 3D reconstruction of complex workpieces.

[0022] The present invention also provides a three-dimensional scanning point cloud reconstruction system for complex workpieces. Since this system and the three-dimensional scanning point cloud reconstruction method for complex workpieces solve the same technical problem and belong to the same technical concept, they should have the same beneficial effects, and will not be described in detail here. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a method for reconstructing three-dimensional scanning point clouds of complex workpieces, provided in an embodiment of this application; Figure 2 This is a scanning execution diagram of the robot scanning system provided in an embodiment of this application; Figure 3 A flowchart for extracting overlapping sub-clouds from adjacent point clouds provided in this application embodiment; Figure 4 A schematic diagram illustrating the registration correction system error provided in an embodiment of this application; Figure 5A structural diagram of a three-dimensional scanning point cloud reconstruction system for complex workpieces provided in this application embodiment; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] like Figure 1 As shown, this embodiment of the invention provides a method for reconstructing three-dimensional scanning point clouds of complex workpieces, including the following steps: S1. Process the initial scan data of the complex workpiece to obtain the target point-normal set; S2. Generate a candidate scan viewpoint set based on the target point-normal set; S3. Perform visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set to obtain a scanning viewpoint sequence; S4. Move to each scanning pose sequentially according to the scanning viewpoint sequence, and control the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point cloud, and obtain overlapping sub-clouds based on the multiple frames of local point cloud combined with the nearest neighbor constraint strategy; S5. Perform progressive continuous fine registration on the point clouds of adjacent frames according to the overlapping sub-clouds to obtain the fine registration pose of each frame point cloud relative to the target frame. S6. Based on the finely matched pose, fuse the local point clouds of multiple frames to obtain a fused point cloud.

[0027] In step S1, a series of preprocessing operations are performed on the initial scanning data of the complex workpiece obtained by the 3D scanning device. The aim is to extract target points and their corresponding normal vectors that can characterize the geometric features of the workpiece surface from these raw data, thereby forming an accurate set of target point-normal pairs, laying the data foundation for subsequent viewpoint planning.

[0028] In step S2, based on the target point-normal set obtained in the previous step, a series of potential scanning positions and orientations that can effectively observe different areas of the target workpiece are generated. These positions and orientations together constitute an initial set of candidate scanning viewpoints.

[0029] In step S3, each viewpoint in the candidate scanning viewpoint set is evaluated and screened in a multi-dimensional and comprehensive manner. This includes: evaluating the visibility of the target area by the sensor from that viewpoint (whether it is occluded); evaluating the ease of registration between the data collected from that viewpoint and existing data (registration capability); and, in conjunction with the robot's kinematics model, determining whether the robot can safely and collision-free reach and stably maintain the scanning pose (robot executability). Through this series of evaluations and screenings, a reasonable and efficient scanning viewpoint execution sequence is finally optimized and determined. In this embodiment, to address the problem of the limited single scanning range of a small-field-of-view high-precision 3D camera, a multi-view scanning sequence is automatically generated, enabling the robot to collect local point clouds of complex workpieces from multiple positions and postures, thereby improving the scanning coverage completeness of deep cavities, sidewalls, holes, thin-walled edges, free-form surfaces, and occluded areas.

[0030] In step S4, the robot is controlled to move sequentially to each designated scanning pose according to a predetermined scanning viewpoint sequence. At each pose, a high-precision 3D scanning sensor mounted on the robot's end effector is precisely controlled to acquire local surface point cloud data of the workpiece from the current viewpoint. By acquiring multiple frames of such local point clouds, and using a nearest neighbor constraint strategy to identify and extract overlapping point clouds between adjacent frames, i.e., overlapping sub-clouds, these overlapping regions are crucial for subsequent point cloud stitching.

[0031] In step S5, the overlapping sub-clouds obtained in the previous step are used as the registration constraint region to perform a progressive and continuous fine registration algorithm (such as the iterative nearest point algorithm and its variants) on the local point clouds of two adjacent frames. This process gradually optimizes the relative pose transformation between point clouds, and finally calculates a high-precision registration pose transformation matrix relative to a selected reference frame (target frame) for each frame of local point clouds.

[0032] In step S6, based on the finely registered poses of each frame of point cloud calculated in step S5, all the finely registered multi-frame local point cloud data are integrated into a unified global coordinate system using a point cloud fusion algorithm (such as voxel meshing or Poisson reconstruction). By fusing the points in the overlapping areas, redundancy is eliminated, and missing data is filled in, ultimately generating a complete, continuous, and high-precision 3D point cloud digital model of the target complex workpiece.

[0033] Compared with existing technologies, the above method is designed for complex workpieces with features such as free-form surfaces, deep cavities, holes and grooves, thin walls, occluded edges, repetitive structures, or local curvature abrupt changes. By unifying automatic viewpoint planning, robot accessibility screening, inter-viewpoint registration evaluation, multi-frame point cloud continuous registration, and quality feedback scanning process, the scanning viewpoint can not only cover the surface of the workpiece under test, but also ensure that adjacent scanning point clouds have an effective overlapping area suitable for subsequent registration. This improves the completeness, stitching accuracy, and automation of 3D reconstruction of complex workpieces.

[0034] Furthermore, this method is applicable to automated 3D scanning and high-precision point cloud stitching of complex workpieces. These complex workpieces can be aero-engine blades, turbine blades, complex castings, precision molds, slotted structural parts, thin-walled shells, automotive parts, freeform surface parts, or other workpieces requiring multi-view scanning to obtain complete surface data. The 3D scanning sensor can be a structured light 3D camera, a blue light scanner, a laser line scanner, a binocular 3D vision sensor, or a surface array 3D camera. In this embodiment, a structured light 3D camera installed at the end effector of a six-axis industrial robot is used as an example to illustrate the robot's eye scanning system. This six-axis industrial robot can be replaced with other multi-degree-of-freedom industrial robots, five-axis motion platforms, robot and turntable combination platforms, or guide rail scanning platforms.

[0035] Preferably, the process of processing the initial scan data of the acquired complex workpiece to obtain the target point-normal set specifically includes: The input model is processed by unifying coordinates, unifying units, removing outliers, and downsampling to obtain the original target point set; Calculate the corresponding normal vector for the neighborhood point set in the original target point set to obtain the target point-normal set.

[0036] In practical applications, the coordinate relationship of the robot eye scanning system on the hand is first established, and the robot's base coordinate system is set as follows: The robot end-effector coordinate system is The 3D camera coordinate system is The workpiece coordinate system is ,like Figure 2 As shown, V(1) is viewpoint 1, V(2) is viewpoint 2, and V(3) is viewpoint 3. The transformation of the workpiece coordinate system relative to the robot base coordinate system is obtained through robot calibration or fixture calibration. The transformation between the camera coordinate system and the robot end-effector coordinate system is obtained through hand-eye calibration. or The robot moved to the... When scanning a viewpoint, the robot controller provides the end effector pose. Then the coarse pose of the camera in the robot's base coordinate system It can be represented as: ; When the workpiece model is in the workpiece coordinate system When indicated below, the candidate camera viewpoint It can be done The coordinates are transformed to the robot's base coordinate system for inverse kinematics solving and motion execution. These coordinate relationships are used for viewpoint planning, robot motion control, and subsequent coarse alignment of the point cloud.

[0037] During the model input stage, the prior model or original target point set (i.e., initial point cloud) of the workpiece to be measured is acquired. This prior model can be a CAD model, an STL triangular mesh model, an OBJ mesh model, a theoretical point cloud model, or an initial point cloud obtained through a single low-precision pre-scan. For CAD or mesh models, they can be converted into triangular meshes or sampled point clouds; for point cloud models, their 3D coordinates are directly read. The input model undergoes coordinate unification, unit unification, outlier removal, and downsampling processing to obtain the original target point set. : ; in, The first part represents the surface of the workpiece. One target point, Indicates the number of target points. Represented as the first Target points It belongs to the three-dimensional real number space.

[0038] Further based on the original target point set For each target point, the corresponding normal vector is calculated from the neighborhood point set. The target point-normal set is obtained. : ; Furthermore, the input point cloud can be voxelized, dividing the workpiece surface point cloud into several voxel elements. Each effective voxel element uses the geometric center of its internal points as the voxel center and the average direction of the normals of the points within that voxel as the voxel normal, resulting in a set of voxel centers and a set of voxel normals. This set of voxel centers and voxel normals can be used as the target point-normal set for subsequent viewpoint planning, or the downsampled original target point set can be used as the target point-normal set.

[0039] Let the side length of the voxel be... The system divides the point cloud on the workpiece surface into several voxel units. For each valid voxel unit containing no less than a preset threshold number of points, the geometric center of the points within that voxel is calculated as the voxel center, and the average direction of the normal vector of the points within that voxel is calculated as the voxel normal. This yields the set of voxel centers. : ; and voxel normal set :

[0040] in, Indicates the target point. Indicates the target point The corresponding normal vector; Indicates the number of effective voxels; express Target point All normal vectors belong to three-dimensional real space. The set of voxel centers and the set of voxel normals can be used as the target point-normal set in subsequent viewpoint planning. For thin-walled edges, holes, deep cavities, chamfers, and regions with abrupt curvature changes, the voxel size can be reduced or the local sampling density can be increased to ensure that these regions are fully considered in viewpoint planning.

[0041] Preferably, generating a candidate scan viewpoint set based on the target point-normal set specifically includes: A set of candidate scanning viewpoints is generated based on the spatial location of the target point-normal set, surface normals, optimal working distance of the scanner, field of view, depth of field, allowable incident angle range, and robot end-effector posture constraints.

[0042] In practical applications, when the input model is a triangular mesh model, the system can use the center of the triangular facet, the vertex of the triangular mesh, the sampling point of the surface, or the center of the voxel as the target unit, and determine the normal vector of the target point based on the normal of the triangular facet, the normal of the vertex, or the normal fitted to the local neighborhood. For thin walls, holes, grooves, edges, and regions with abrupt curvature changes, the sampling density can be increased to avoid these regions being ignored in subsequent viewpoint planning. Through the above processing, the system obtains the target point set, the target normal set, the workpiece bounding box, the workpiece geometric center, the workpiece local scale, the voxel index, and the area to be covered, providing basic data for subsequent candidate scanning viewpoint generation and visibility evaluation.

[0043] After completing the construction of the target point set, this invention generates a candidate scanning viewpoint set based on the spatial location of the target points, surface normals, optimal working distance of the scanner, field of view, depth of field, allowable incident angle range, and robot end-effector posture constraints.

[0044] Specifically, in the candidate scanning viewpoint generation stage, candidate viewpoints are generated based on the spatial location of the target point or voxel center, the normal vector, and the imaging parameters of the 3D scanning sensor. These imaging parameters include the optimal working distance. Working distance near end Working distance at a distance Field of view Field of view height Maximum permissible angle of incidence Depth of field range and number of camera yaw angle samples around the optical axis. For the target point and its normal vector Using the target point as the observation object, multiple observation directions are sampled within a cone-shaped region either in the opposite direction to or near the normal. And calculate the camera center position according to the optimal working distance: ; in, The direction of observation from the camera towards the target point. This represents the center position of the candidate camera. To accommodate different camera mounting postures and local workpiece shapes, multiple yaw angles can also be sampled around the camera's optical axis. This generates multiple pose candidates, resulting in a candidate viewpoint set. : ; in, For the first Camera center of candidate viewpoints; For the camera pose matrix, The yaw angle is about the optical axis; This represents the total number of viewpoints. In one implementation, candidate viewing directions are generated through cone sampling, where the cone angle can be set to... to The yaw angle around the optical axis can be sampled at 3 to 12 discrete angles. For deep cavities, sidewalls, the interior of apertures and slots, and occluded areas, the candidate viewpoint density can be appropriately increased to obtain more executable and registerable viewpoints in the subsequent screening stage. In this embodiment, the candidate viewpoints can be generated by offsetting the optimal working distance along the opposite direction of the target point normal, or by sampling multiple observation directions in a conical space near the target point normal and sampling multiple yaw angles around the camera optical axis to obtain candidate viewpoints that meet the requirements of different imaging postures. For deep cavities, sidewalls, apertures, thin-walled edges, and occluded areas, the candidate viewpoint density can be increased to improve the flexibility of subsequent viewpoint selection.

[0045] Preferably, the visibility evaluation of the candidate scanning viewpoint set specifically includes: The visibility of the candidate scanning viewpoint set is evaluated, and a viewpoint-target point visibility matrix is ​​established.

[0046] In practical applications, during the visibility evaluation phase, for each candidate viewpoint... and each target point The process involves determining whether the target point can be effectively scanned. Specifically, the target point is transformed from the workpiece coordinate system or robot base coordinate system to the camera coordinate system, resulting in: ; in, Indicates the i-th candidate scanning viewpoint; Indicate candidate viewpoints The corresponding position of the camera's optical center in the global coordinate system; Indicate candidate viewpoints The corresponding camera pose rotation matrix; Used to transform the target point from the global coordinate system to the camera coordinate system; Indicates the target point Coordinates in the camera coordinate system.

[0047] like satisfy:

[0048]

[0049]

[0050] The target point is then considered to be within the sensor's field of view and effective working distance. Further calculations are made of the angle between the target point's normal vector and the line-of-sight direction; if this angle is less than the maximum permissible angle of incidence... If the target point is not obscured by the workpiece itself, then the target point is determined to be within the candidate viewpoint. The viewpoint-target point visibility matrix is ​​thus established. : ; in, Indicate viewpoint Able to effectively scan target points ; This indicates that a scan could not be performed effectively. Visibility can be determined using methods such as depth buffering, ray detection, or local thickness windows to identify occlusion. When using a point cloud model, depth projection in the camera coordinate system can be used to preserve near endpoints within the same projected mesh, thus approximating the occlusion relationship. When using a triangular mesh model, ray intersection detection can be used to determine whether occlusion exists between the camera center and the target point.

[0051] Preferably, the registrability evaluation of the candidate scanning viewpoint set specifically includes: Calculate the set of target points that are commonly visible to any two candidate viewpoints based on the visibility matrix; The overlap rate between viewpoints is obtained based on the set of target points. A viewpoint registration score is constructed based on the viewpoint overlap rate.

[0052] In practical applications, during the registrationability evaluation stage, it is no longer sufficient to simply determine whether a single viewpoint can cover the target point; rather, it is further determined whether the local point clouds obtained after continuous scanning of two candidate viewpoints possess stable registration conditions. For two candidate viewpoints... and Calculate the set of target points that are visible to both based on the visibility matrix. : ; in, Indicate candidate viewpoints The visibility matrix.

[0053] And define candidate viewpoints set of visible target points : ; Two candidate viewpoints and Theoretical overlap rate between It can be represented as: ; in, Indicate candidate viewpoints The set of visible target points, defined in the same way as candidate viewpoints set of visible target points Consistent; when the theoretical overlap rate When the value is less than the preset threshold, it indicates that the theoretical overlap area between the two viewpoints is insufficient and they are not suitable as adjacent viewpoints in a continuous scanning sequence.

[0054] The preset threshold is denoted as It is not automatically calculated by a formula, but determined based on the minimum common visible area required for continuous scanning of point clouds between adjacent viewpoints and subsequent registration. Specifically, Indicates two candidate viewpoints and The proportion of the number of publicly visible target points to the smaller set of visible target points in both viewpoints can be used as a criterion for whether there is sufficient overlap between adjacent viewpoints. This paper uses this threshold as an empirical parameter, setting it based on the sensor's field of view size, target point sampling density, and the minimum overlap ratio required for stitching adjacent scanned point clouds. If... If the common observation area between two viewpoints is insufficient, they are considered unsuitable as adjacent viewpoints in a continuous scanning sequence. Furthermore, in practice, this area is set at 50%, not less than 40%.

[0055] Furthermore, regarding overlapping regions The geometric feature richness is calculated, which can be obtained based on the normal variance, curvature variance, local principal direction distribution, boundary point proportion, or point cloud feature descriptor discreteness of target points within the overlapping region. For example, the eigenvalue distribution of the normal vector covariance matrix within the overlapping region can be calculated; if the normal variation is rich, the region is more conducive to point cloud registration than a planar region. The viewpoint change penalty between two viewpoints, camera center distance, robot joint space motion cost, and path length can also be calculated to obtain a registration score between viewpoints. : ; in, Indicates the richness of geometric features in the overlapping region. Indicates the discernibility of the normal distribution in the overlapping region. Indicates a penalty for changes in perspective. Indicates the robot from the viewpoint Motion to viewpoint The cost, to The weighting coefficients are used to prioritize adjacent viewpoints that have both sufficient overlap and good geometric discernibility, thus providing more stable data conditions for subsequent continuous registration. This embodiment introduces a viewpoint registerability evaluation during the viewpoint planning stage. It not only determines whether the target surface is covered by candidate viewpoints but also further evaluates the effective overlap ratio, geometric features of the overlapping area, normal distribution, and viewing angle changes between adjacent viewpoints. This makes the resulting scanned viewpoint sequence more suitable for subsequent continuous point cloud registration, reducing the risk of registration failure due to insufficient overlap or weak features in the overlapping area after scanning.

[0056] Preferably, the candidate scanning viewpoint set is subjected to robot executability screening, specifically including: The robot end-effector pose is obtained based on the calibration relationship between the robot base coordinate system, workpiece coordinate system, camera coordinate system, and end-effector coordinate system. The robot end-effector pose is subjected to inverse kinematics solution, joint limit judgment, singularity judgment, tool and workpiece collision detection, robot body and workpiece collision detection, robot and worktable collision detection, and approach path feasibility detection to eliminate unreachable, unsafe, or unstable viewpoints from the candidate scanning viewpoint set.

[0057] In practical applications, during the robot executability screening stage, each candidate camera viewpoint is converted into the robot's end-effector target pose. Let the... The poses of the candidate camera viewpoints in the workpiece coordinate system are: Then its camera pose in the robot's base coordinate system is: ; The pose of the robot's end effector can be obtained based on the hand-eye alignment relationship: ; right Perform inverse kinematics solution to obtain one or more robot joint solutions. For each joint solution, the system determines whether it satisfies the joint constraint condition: ; The robot determines whether it is close to a singular configuration based on the robot's Jacobian matrix. For candidate viewpoints that satisfy joint constraints and singularity constraints, collision detection is further performed. Collision detection targets include collisions between the 3D scanning sensor or end-effector and the workpiece, collisions between the 3D scanning sensor or end-effector and the worktable, collisions between the robot link and the workpiece, collisions between the robot link and the end-effector, and collisions between the robot link and the worktable.

[0058] Furthermore, the robot links can be simplified as capsules, the workpiece and tool can be represented as triangular meshes or point cloud bounding volumes, and the worktable can be represented as axis-aligned bounding boxes. If a candidate viewpoint lacks a valid inverse kinematics solution, exceeds joint limits, approaches a singular configuration, or collides, the candidate viewpoint is eliminated. For candidate viewpoints that pass the above detection, the feasibility of the approach path from the safe point to the target point can be further detected to avoid collisions during the robot's approach to the scanned position. In this embodiment, inverse kinematics solution, joint limit judgment, singularity judgment, tool-workpiece collision detection, robot-workpiece collision detection, robot-worktable collision detection, and approach path feasibility detection are performed for each end-effector pose, eliminating viewpoints that are unreachable, unsafe to execute, or have unstable postures.

[0059] Preferably, the step of performing visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set to obtain a scanning viewpoint sequence specifically includes: After performing visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set, the scanning viewpoint sequence is solved under the constraints of coverage, number of viewpoints, scanning quality, robot motion cost, and registration of adjacent viewpoints.

[0060] In practical applications, after completing the visibility evaluation, registrationability evaluation, and robot executability screening of candidate viewpoints, the scanned viewpoint sequence is solved. This sequence differs from simply solving for an unordered set of viewpoints; in this embodiment, a scanned viewpoint sequence that satisfies continuous registration requirements is solved. During the scanned viewpoint sequence solving stage, coverage, number of viewpoints, imaging quality, registrationability of adjacent viewpoints, and robot motion cost are considered as comprehensive objectives to solve the scanned viewpoint sequence. : ; in, Indicates the first in the scan sequence One perspective, This indicates the number of selected viewpoints. The sequence can be solved using a combination of multi-starting-point greedy algorithms, greedy search with backtracking, and set coverage and path optimization. Specifically, during sequence construction, viewpoints with high single-viewpoint coverage quality, robot executability, and satisfactory imaging quality are first selected as starting viewpoints. Then, the marginal coverage gain of each candidate viewpoint is calculated from the set of uncovered target points, while the registration score and robot motion cost between the candidate viewpoint and the current end viewpoint are also calculated. For candidate viewpoints... Its incremental score can be expressed as: ; in, Indicates the percentage of newly added coverage points. Indicates the quality of image at the viewpoint. Indicates the viewpoint at the end of the current sequence. With candidate viewpoints Registration capability, This represents the cost of robot movement. to This represents the weighting coefficient. If a candidate viewpoint has a high new coverage rate, but its registerability with the current viewpoint is below a threshold... If the viewpoint is not directly used as the next viewpoint, then an intermediate transitional viewpoint or a viewpoint with higher overlap is preferred to ensure that the point clouds of adjacent frames in the continuous scanning sequence have sufficient overlap and stable registration conditions. The sequence generation process continues until the coverage reaches a preset threshold. The number of candidate viewpoints has reached the upper limit or the new coverage gain is less than the preset threshold.

[0061] The scan viewpoint sequence needs to satisfy the following constraints: ; ; ; ; ; in, This indicates the preset minimum coverage threshold; This represents the minimum allowable registration threshold between adjacent viewpoints; Indicates the maximum number of scan viewpoints that can be selected; This represents the minimum new coverage gain threshold; This represents the set of executable viewpoints that satisfy sensor imaging constraints, robot inverse kinematics constraints, joint constraint constraints, singularity constraints, and collision detection constraints.

[0062] Preferably, the step of sequentially moving to each scanning pose according to the scanning viewpoint sequence, controlling the 3D scanning sensor installed at the robot's end effector to acquire multiple frames of local point clouds, and obtaining overlapping sub-clouds based on the multiple frames of local point clouds combined with a nearest neighbor constraint strategy specifically includes: According to the scanning viewpoint sequence, the robot moves to each scanning pose in sequence and controls the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point cloud. Based on the robot's forward kinematics and hand-eye calibration relationship, a coarse initial pose is provided for the multi-frame local point cloud; Calculate the nearest neighbor distance from each point in the source point cloud to the target point cloud based on the coarse initial pose; An adaptive seed threshold is determined based on the intrinsic resolution of the nearest neighbor point cloud and the camera resolution. Points whose nearest neighbor distance is less than the adaptive seed threshold are used as candidate overlapping seeds, and unstable corresponding points are further eliminated by mutual nearest neighbor constraint. The radius is expanded using seed points that satisfy the nearest neighbor constraint to obtain overlapping sub-clouds of source and target point clouds. The overlapping sub-clouds include: the source point cloud overlapping sub-cloud and the target point cloud overlapping sub-cloud.

[0063] In practical applications, during the robot scanning execution phase, the selected sequence of scanning viewpoints is converted into robot motion commands. For each scanning viewpoint... Obtain the corresponding robot end-effector pose and joint angles. The robot plans a collision-free motion path from the previous scan pose to the current scan pose. To ensure scanning safety, approach and retreat points can be set before and after each scan point. After the robot reaches and stabilizes at the target scan pose, the 3D scanning sensor acquires a local point cloud from the current viewpoint. : ; in, Indicates the first In the first frame of the point cloud The coordinates of a point in the camera coordinate system express The coordinates belong to three-dimensional real space. Indicates the first The number of points in the frame point cloud. During acquisition, the robot's end-effector pose, joint angles, camera exposure parameters, scanning timestamps, and scanning viewpoint numbers can be recorded simultaneously. A coarse initial pose of the frame point cloud is obtained based on the robot's pose and the hand-eye calibration relationship. : ; The coarse initial pose is used for initial alignment between adjacent point clouds, but is not used as the final stitching result. In this embodiment, the robot pose and hand-eye calibration results are used as the coarse initial pose for point cloud registration, rather than directly as the final stitching result. By correcting the point cloud misalignment caused by robot absolute positioning error, end effector stiffness error, and calibration error through continuous fine registration between adjacent frame point clouds, gaps, ghosting, and local steps at the stitching boundary of multi-frame point clouds can be reduced, thereby improving the overall 3D reconstruction accuracy.

[0064] In the point cloud preprocessing stage, filtering, unit unification, voxel downsampling, and normal estimation are performed on the local point cloud of each frame. For point clouds... Outlier points can be removed using statistical filtering, voxel downsampling can be used to reduce the number of points while preserving the geometric shape, and K-nearest neighbors or radius neighborhood can be used to estimate the normal. The voxel downsampling size can be determined based on the camera resolution and target registration accuracy. For example, when the nominal point spacing of the 3D camera is... At that time, the preprocessed voxel size can be set to to The minimum voxel size in the fine-tuning stage of registration can be set to to Specific parameters can be adjusted based on workpiece size, point cloud density, and computing resources.

[0065] like Figure 3 As shown, in the initial alignment stage of adjacent point clouds, the first step is calculated based on the coarse initial pose. Frame point cloud to reference frame The coarse initial transformation. If the first... Frame and reference frame The coarse initial poses are respectively and Then the first Coarse initial relative transformation from frame to reference frame for: ; In direct-to-first-frame mode, reference frame In sequential registration mode, the reference frame The purpose of the coarse initial transformation is to control the robot pose error within the convergent range of the registration algorithm and to provide initial spatial relationships for subsequent extraction of overlapping sub-clouds.

[0066] In the overlapping sub-cloud extraction stage, the source point cloud According to coarse initial transformation Transform to reference point cloud Near the coordinate system, a coarsely aligned source point cloud is obtained. :

[0067] for Each point in In the reference point cloud Search for the nearest neighbor and calculate the nearest neighbor distance. : ; The overlap seed threshold is adaptively determined based on the nearest neighbor distance distribution. : ; in, , Represents the distance set of the first percentile value This indicates camera resolution or the intrinsic scale of the point cloud. This is the proportionality coefficient. It will satisfy... The source point is used as a candidate overlap seed. To reduce accidental nearest neighbors and non-true corresponding points, a mutual nearest neighbor constraint is further adopted: if the source point... The nearest neighbor is used as the reference point ,and The nearest neighbor in the source cloud is still 1. If a pair of points satisfies the nearest neighbor relationship, it is considered to satisfy this relationship, and such pairs are retained as reliable overlap seeds. This embodiment automatically extracts overlapping sub-clouds from adjacent frame point clouds based on coarse initial pose, and combines nearest neighbor distance, adaptive threshold, nearest neighbor constraint, and radius expansion to determine reliable overlapping regions. This ensures that fine registration is primarily performed within the true overlapping regions, reducing interference from non-overlapping regions, occlusion boundaries, outliers, and repetitive structures on the registration results. Figure 4 As shown.

[0068] After obtaining a reliable overlapping seed, perform neighborhood expansion centered on the seed point, setting the expansion radius as follows: for: ; in, The expansion coefficient, A value of 2 to 5 can be used. A radius neighborhood search is performed in both the source and reference point clouds, centered on the seed point, to obtain the source overlapping sub-cloud. Overlapping subclouds with reference This refers to the overlapping sub-clouds of the source point cloud and the overlapping sub-cloud of the target point cloud. If the number of overlapping seeds is less than a preset threshold, or the number of points in the expanded overlapping sub-cloud is insufficient, the point cloud pair is considered to have insufficient overlap. This result is marked as low-quality registration risk, and a rescan or replanning is triggered in the subsequent quality evaluation stage. This process can limit the fine registration to the vicinity of the true overlapping area, reducing the interference of non-overlapping areas on the nearest neighbor search and error minimization process.

[0069] Preferably, the step of performing progressive continuous fine registration of adjacent frame point clouds based on the overlapping sub-clouds to obtain the fine registration pose of each frame point cloud relative to the target frame specifically includes: After progressively optimizing the point clouds of adjacent frames at multiple scales based on a multi-scale voxel sequence from coarse to fine and a gradually shrinking registration distance threshold, the fine registration pose of each frame point cloud relative to the target frame is obtained.

[0070] In practical applications, during the progressive continuous registration stage, only the source overlapping sub-clouds are registered. Overlapping subclouds with reference Instead of directly performing global nearest neighbor iteration on the entire point cloud, fine registration is performed. This approach reduces the impact of non-overlapping regions, occlusion boundaries, thin-walled edges, and outliers on the registration results. The system employs a multi-scale registration strategy from coarse to fine. In this embodiment, a progressive continuous registration strategy from coarse to fine is used. Through multi-scale voxel processing and a gradually shrinking corresponding point search threshold, it balances convergence stability under large initial errors and final stitching accuracy. This is suitable for scenarios where the robot eye's initial pose has some errors during hand scanning but high-precision alignment of local point clouds is required.

[0071] Let the initial voxel scale be The scale attenuation coefficient is , No. Voxel scale of each registration stage for: ; in, This is the smallest voxel scale. The corresponding point search threshold is determined based on the average nearest neighbor distance of the point cloud at the current scale and a preset lower limit:

[0072] in, This represents the average nearest neighbor distance of the point cloud at the current scale. For threshold coefficient, The minimum distance threshold is used. Stable convergence results are first obtained at a larger voxel scale and a larger search threshold, and then the voxel scale and search threshold are gradually reduced to improve the final registration accuracy.

[0073] In each registration stage, the system can establish an optimization objective using either point-to-point error or point-to-area error. When using point-to-area error, the optimization objective is: ; in, This is the set of corresponding points established at the current scale. For points in the source overlapping subcloud, To reference the corresponding points in the overlapping sub-clouds, For reference point The normal vector. At the end of each stage, the system updates the current transformation and uses the updated result as the initial value for the next stage. After multiple stages, the th... Frame point cloud to reference frame Fine registration transformation The registration quality evaluation mechanism established in this embodiment comprehensively considers factors such as the median nearest neighbor distance, robust error scale, intrinsic point cloud scale, camera resolution, inlier ratio, and closed-loop error. It can automatically determine whether the registration results of adjacent frame point clouds are reliable, avoiding the need to judge the stitching quality based solely on a single mean square error or manual observation.

[0074] Furthermore, pose increment gating conditions can be set. If the rotation increment or translation increment of the fine registration result relative to the coarse initial transformation exceeds the reasonable range of robot error, the registration result is rejected and marked as abnormal.

[0075] In the process of multi-frame continuous registration, two registration modes can be used. The first mode is the direct-to-first-frame mode, which means that the first frame is the first frame. Frame to the The frames are registered to either the first frame or the global reference point cloud, resulting in: ; This mode is suitable when there is sufficient overlap between each frame and the first frame or the global reference point cloud, which helps to reduce the accumulation of chain errors.

[0076] The second type is the sequential registration mode, i.e., the first... First, register the frames to the first... The frames are then unified to the coordinate system of the first frame through a chain transformation: ; This mode is suitable for situations where the scanning viewpoint moves continuously around the workpiece, with sufficient overlap between adjacent frames but insufficient overlap between distant frames. For circular scanning paths, the system can add a first frame or repeat the scanning frame at the end and calculate the closed-loop error based on the difference between chained transformation and direct transformation. If the closed-loop error exceeds a preset threshold, it indicates that there is cumulative drift in multi-frame continuous registration, requiring triggering supplementary scanning, re-registration, or global optimization.

[0077] Preferably, the step of fusing multiple frames of local point clouds based on the finely registered pose to obtain a fused point cloud specifically includes: Based on the finely matched pose, the local point clouds of multiple frames are subjected to repeated point merging, voxel downsampling, outlier removal, normal unification, boundary smoothing, and mesh reconstruction to obtain the fused point cloud.

[0078] In practical applications, during the point cloud fusion stage, all local point clouds are unified to the workpiece coordinate system or robot base coordinate system based on the final pose of each frame of point cloud. Let the first... The final global pose of the frame point cloud is Then merge point clouds It can be represented as: ; in, Indicates the first Frame-specific point cloud.

[0079] During the fusion process, overlapping areas can undergo processes such as merging duplicate points, voxel downsampling, statistical outlier removal, normal unification, boundary smoothing, and density equalization. For quality inspection scenarios, the system can directly output a complete point cloud and perform deviation analysis with the CAD theoretical model. For reverse engineering or 3D reconstruction scenarios, triangular mesh reconstruction can be further performed, outputting STL, PLY, OBJ, or other 3D model files. It can also output the final transformation matrix of each frame's point cloud, registration quality indicators for each adjacent frame, scan re-scanning records, coverage statistics, and low-quality area records for subsequent quality traceability. This embodiment considers viewpoint coverage, robot executability, adjacent point cloud registration capability, and continuous point cloud registration quality in a unified manner. Compared to simple viewpoint planning methods, simple robot path planning methods, or simple point cloud post-processing registration methods, it can better adapt to complex workpiece multi-viewpoint high-precision 3D reconstruction tasks, improving stitching accuracy and system robustness while ensuring scan integrity.

[0080] In one specific embodiment, the workpiece to be measured is a blade-like workpiece with a free-form surface and thin-walled edges. First, the theoretical point cloud or mesh model of the blade is read and converted into a target point set for viewpoint planning. The optimal working distance of the 3D camera is set as follows: The field of view width is The field of view height is The maximum permissible angle of incidence is The camera resolution is Candidate viewpoints are generated based on the blade surface normal, and their visibility is evaluated. For curved areas of the blade, the density of candidate viewpoints is increased to ensure sufficient candidate viewpoints in areas with large curvature changes and thin-walled edges. The system calculates the theoretical overlap area between candidate viewpoints based on the visibility matrix and prioritizes the addition of viewpoint combinations with high overlap rates, good normal changes, and low robot motion costs to the scanning sequence. After the robot sequentially acquires multiple frames of local point clouds, a coarse initial pose is obtained based on the robot pose and hand-eye calibration results. Then, the relative pose between point clouds in each frame is corrected through overlapping sub-cloud extraction and progressive continuous registration, finally yielding a complete blade point cloud model.

[0081] In another specific embodiment, the workpiece under test is a complex casting with holes, grooves, steps, and locally repetitive structures. These workpieces have similar local structures, and directly relying on nearest-neighbor registration of the entire point cloud can easily lead to incorrect correspondences. During the viewpoint planning stage, hole / groove edges, step transition areas, and curvature abrupt change areas are designated as high-weight regions to increase the density of candidate viewpoints. When solving for the viewpoint sequence, the theoretical overlap area between adjacent viewpoints is required to contain a certain proportion of boundary points, normal change points, or local feature points. After scanning, an overlap seed is extracted based on the coarse initial pose, and overlapping sub-clouds are obtained through nearest-neighbor and neighborhood expansion. Since fine registration is only performed within reliable overlap areas, the interference of non-overlapping hole / groove edges and repetitive structures on registration is reduced. If the registration quality index of a certain hole / groove region is low, a supplementary scan viewpoint is automatically generated, the point cloud of that region is acquired from a more suitable viewing direction, and re-registered with the existing high-quality point cloud.

[0082] In yet another embodiment, a sequential registration mode is used to complete the circular scan. The robot moves around the workpiece starting from a first viewpoint, sequentially acquiring data. Frame to the Calculate the local point cloud of the frame, and add closed-loop scan frames adjacent to or repeating the first viewpoint at the end. Calculate sequentially. The chain cumulative transform is then obtained. Simultaneously, the direct registration transform between the first and last frames is calculated based on the closed-loop frame. If the rotation or translation error between the chain cumulative transform and the direct registration transform is less than a preset threshold, the consistency of the overall scan sequence is considered to meet the requirements; if it exceeds the threshold, possible low-quality adjacent frames are located based on the error distribution, and the corresponding regions are preferentially rescanned or re-registered.

[0083] Preferably, before fusing the multi-frame local point clouds according to the finely registered pose to obtain the fused point cloud, the method further includes: When insufficient target coverage, unscanned local areas, insufficient effective overlap between adjacent point clouds, registration quality index below the threshold, excessive loop closure error, or obvious gaps, ghosting, steps, and low-density areas are detected in the fused point cloud, feedback scanning is automatically triggered to regenerate candidate supplementary viewpoints.

[0084] In practical applications, during the feedback scan phase, the system automatically determines whether supplementary scanning is needed based on coverage and registration quality. When a target point is not covered by any valid viewpoint, some target points have fewer coverage times than a preset value, adjacent viewpoints have insufficient theoretical overlap, actual overlapping sub-cloud extraction fails, registration quality indicators are below a threshold, or gaps appear in the fused point cloud or local point density is insufficient, the system extracts the corresponding low-quality region as the supplementary scan target region. For uncovered regions, candidate viewpoints for supplementary scanning are directly generated based on the spatial location and normal vector of the uncovered target points. For low-quality stitched regions, the system determines the supplementary scan center based on points with large registration residuals, points with low inlier ratios, or regions affected by closed-loop errors. For low-overlap regions, the system generates transitional or bridging viewpoints between two existing high-quality viewpoints.

[0085] The generation method for supplementary candidate viewpoints is the same as that for initial candidate viewpoints, but its evaluation objective leans more towards achieving reliable overlap with existing high-quality point clouds. In other words, supplementary viewpoints must not only cover low-quality areas but also have sufficient theoretical overlap and geometric discriminability with at least one frame of already registered high-quality point clouds. Supplementary candidate viewpoints also undergo screening based on field of view, depth of field, incident angle, occlusion, robot inverse kinematics, joint constraints, singularities, and collision detection. After performing supplementary scanning, the robot registers the supplementary point cloud to the corresponding high-quality reference point cloud or inserts it into the continuous scan sequence, and recalculates the coverage, overlap, registration quality, and fused point cloud quality. This process can be iteratively executed until preset integrity and accuracy requirements are met, or the maximum number of supplementary scans is reached. This embodiment can automatically generate supplementary scanning viewpoints and perform supplementary scanning when insufficient coverage, insufficient overlap, or insufficient registration quality is detected. This makes scanning, registration, evaluation, and supplementary scanning form a closed-loop process, reducing manual supplementary scanning, manual teaching, and manual parameter adjustment, and improving the degree of automation in batch inspection of complex workpieces and online measurement scenarios.

[0086] like Figure 5 As shown, this embodiment of the invention provides a three-dimensional scanning point cloud reconstruction system for complex workpieces, including: a target point set construction module, a candidate viewpoint generation module, a scanning viewpoint sequence solving module, an overlapping sub-cloud extraction module, a progressive continuous registration module, and a point cloud fusion output module; The target point set construction module is used to process the initial scan data of the acquired complex workpiece to obtain a target point-normal set. The candidate viewpoint generation module is used to generate a candidate scan viewpoint set based on the target point-normal set; The scanning viewpoint sequence solving module is used to perform visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set to obtain a scanning viewpoint sequence. The overlapping sub-cloud extraction module is used to move sequentially to each scanning pose according to the scanning viewpoint sequence, control the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point clouds, and obtain overlapping sub-clouds based on the multiple frames of local point clouds combined with the nearest neighbor constraint strategy. The progressive continuous registration module is used to perform progressive continuous fine registration on the point clouds of adjacent frames according to the overlapping sub-clouds, so as to obtain the fine registration pose of each frame point cloud relative to the target frame. The point cloud fusion output module is used to fuse multiple frames of local point clouds according to the finely registered pose to obtain a fused point cloud.

[0087] In practical applications, the 3D scanning point cloud reconstruction system for complex workpieces includes a target point set construction module, a candidate viewpoint generation module, a scanning viewpoint sequence solving module, an overlapping sub-cloud extraction module, a progressive continuous registration module, and a point cloud fusion output module. The candidate viewpoint generation module is connected to both the target point set construction module and the scanning viewpoint sequence solving module; the overlapping sub-cloud extraction module is connected to both the scanning viewpoint sequence solving module and the progressive continuous registration module; and the point cloud fusion output module is connected to the progressive continuous registration module. The target point set construction module processes the initial scanning data of the acquired complex workpiece to obtain... After obtaining the target point-normal set, the target point-normal set is transmitted to the candidate viewpoint generation module. The candidate viewpoint generation module generates a candidate scan viewpoint set based on the target point-normal set and then transmits the candidate scan viewpoint set to the scan viewpoint sequence solving module. The scan viewpoint sequence solving module performs visibility evaluation, registration evaluation, and robot executability screening on the candidate scan viewpoint set to obtain the scan viewpoint sequence. This scan viewpoint sequence is then transmitted to the overlapping sub-cloud extraction module. The overlapping sub-cloud extraction module moves sequentially to each scan pose according to the scan viewpoint sequence and controls the 3D scanning sensor mounted on the robot's end effector to capture images. The system collects multiple frames of local point clouds and, based on these local point clouds and a nearest neighbor constraint strategy, obtains overlapping sub-clouds. These overlapping sub-clouds are then transmitted to a progressive continuous registration module. This module performs progressive continuous fine registration on adjacent frame point clouds based on the overlapping sub-clouds to obtain the finely registered poses of each frame relative to the target frame. These finely registered poses are then transmitted to a point cloud fusion output module. The point cloud fusion output module then fuses the multiple frames of local point clouds based on the finely registered poses to obtain a fused point cloud. Through the collaboration of these modules, the system addresses the challenges of small field of view in high-precision 3D cameras and insufficient absolute positioning accuracy and stiffness of robots to directly guarantee seamless point cloud mapping. To address the issue of stitching, the scanning viewpoint planning stage simultaneously considers workpiece surface coverage, robot executability, and the registrationability of adjacent point clouds. After acquisition, the robot pose is used as a coarse initial value to extract reliable overlapping sub-clouds from adjacent point clouds. Multi-scale, progressive continuous registration is used to correct the pose error between point clouds in each frame. At the same time, the registration quality, coverage, and local stitching residuals determine whether additional scanning is needed. This forms a closed-loop process of "viewpoint planning - robot scanning - overlapping sub-cloud extraction - continuous registration - quality evaluation - feedback additional scanning - point cloud fusion," which improves the completeness, stitching accuracy, and automation level of 3D reconstruction of complex workpieces.

[0088] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0089] Figure 6This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 20 specifically includes: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the three-dimensional scanning point cloud reconstruction method for complex workpieces disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment can specifically be an electronic computer.

[0090] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a three-dimensional scanning point cloud reconstruction channel for complex workpieces between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0091] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222 and data 223, etc., and the storage method can be temporary storage or permanent storage.

[0092] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the three-dimensional scanning point cloud reconstruction method for complex workpieces executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the three-dimensional scanning point cloud reconstruction device for complex workpieces from external devices, as well as data collected by its own input / output interface 25.

[0093] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0094] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for reconstructing three-dimensional scanning point clouds of complex workpieces. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0095] It should be understood that the use of terms such as "method," "apparatus," "unit," and / or "module" in this application is merely to distinguish one method of different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0096] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0097] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0098] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0099] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for reconstructing three-dimensional scanning point clouds of complex workpieces, characterized in that, Includes the following steps: The initial scan data of the complex workpiece is processed to obtain the target point-normal set; A candidate scan viewpoint set is generated based on the target point-normal set; The candidate scanning viewpoint set is subjected to visibility evaluation, registration evaluation, and robot executability screening to obtain a scanning viewpoint sequence; According to the scanning viewpoint sequence, the robot moves to each scanning pose in sequence and controls the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point cloud. The overlapping sub-cloud is obtained by combining the multiple frames of local point cloud with the nearest neighbor constraint strategy. Based on the overlapping sub-clouds, progressive continuous fine registration is performed on the point clouds of adjacent frames to obtain the fine registration pose of each frame point cloud relative to the target frame. The local point clouds of multiple frames are fused according to the finely matched pose to obtain a fused point cloud.

2. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, The process of processing the initial scan data of the complex workpiece to obtain the target point-normal set specifically includes: The input model is processed by unifying coordinates, unifying units, removing outliers, and downsampling to obtain the original target point set; Calculate the corresponding normal vector for the neighborhood point set in the original target point set to obtain the target point-normal set.

3. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, The step of generating a candidate scan viewpoint set based on the target point-normal set specifically includes: A set of candidate scanning viewpoints is generated based on the spatial location of the target point-normal set, surface normals, optimal working distance of the scanner, field of view, depth of field, allowable incident angle range, and robot end-effector posture constraints.

4. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, Visibility evaluation of the candidate scan viewpoint set specifically includes: The visibility of the candidate scanning viewpoint set is evaluated, and a viewpoint-target point visibility matrix is ​​established.

5. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 4, characterized in that, The registrationability evaluation of the candidate scanning viewpoint set specifically includes: Calculate the set of target points that are commonly visible to any two candidate viewpoints based on the visibility matrix; The overlap rate between viewpoints is obtained based on the set of target points. A viewpoint registration score is constructed based on the viewpoint overlap rate.

6. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, The robot executability screening of the candidate scanning viewpoint set specifically includes: The robot end-effector pose is obtained based on the calibration relationship between the robot base coordinate system, workpiece coordinate system, camera coordinate system, and end-effector coordinate system. The robot end-effector pose is subjected to inverse kinematics solution, joint limit judgment, singularity judgment, tool and workpiece collision detection, robot body and workpiece collision detection, robot and worktable collision detection, and approach path feasibility detection to eliminate unreachable, unsafe, or unstable viewpoints from the candidate scanning viewpoint set.

7. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, The process of performing visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set to obtain a scanning viewpoint sequence specifically includes: After performing visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set, the scanning viewpoint sequence is solved under the constraints of coverage, number of viewpoints, scanning quality, robot motion cost, and registration of adjacent viewpoints.

8. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, The process of sequentially moving to each scanning pose according to the scanning viewpoint sequence, controlling the 3D scanning sensor installed at the robot's end effector to acquire multiple frames of local point clouds, and obtaining overlapping sub-clouds based on the multiple frames of local point clouds combined with a nearest neighbor constraint strategy specifically includes: According to the scanning viewpoint sequence, the robot moves to each scanning pose in sequence and controls the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point cloud. Based on the robot's forward kinematics and hand-eye calibration relationship, a coarse initial pose is provided for the multi-frame local point cloud; Calculate the nearest neighbor distance from each point in the source point cloud to the target point cloud based on the coarse initial pose; An adaptive seed threshold is determined based on the intrinsic resolution of the nearest neighbor point cloud and the camera resolution. Points whose nearest neighbor distance is less than the adaptive seed threshold are used as candidate overlapping seeds, and unstable corresponding points are further eliminated by mutual nearest neighbor constraint. The radius is expanded using seed points that satisfy the nearest neighbor constraint to obtain overlapping sub-clouds of source and target point clouds. The overlapping sub-clouds include: the source point cloud overlapping sub-cloud and the target point cloud overlapping sub-cloud.

9. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 8, characterized in that, The nearest neighbor constraint specifically refers to: If the nearest neighbor of a point in the source point cloud is a point in the target point cloud, and the nearest neighbor of a point in the target point cloud is also a point in the source point cloud, then the points in the source point cloud and the points in the target point cloud are considered to satisfy the nearest neighbor constraint.

10. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, The step of performing progressive continuous fine registration of adjacent frame point clouds based on the overlapping sub-clouds to obtain the fine registration pose of each frame point cloud relative to the target frame specifically includes: After progressively optimizing the point clouds of adjacent frames at multiple scales based on a multi-scale voxel sequence from coarse to fine and a gradually shrinking registration distance threshold, the fine registration pose of each frame point cloud relative to the target frame is obtained.

11. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, The step of fusing multiple frames of local point clouds based on the finely matched pose to obtain a fused point cloud specifically includes: Based on the finely matched pose, the local point clouds of multiple frames are subjected to repeated point merging, voxel downsampling, outlier removal, normal unification, boundary smoothing, and mesh reconstruction to obtain the fused point cloud.

12. The method for reconstructing three-dimensional scanning point clouds of complex workpieces according to claim 1, characterized in that, Before fusing multiple frames of local point clouds based on the finely matched pose to obtain the fused point cloud, the method further includes: When insufficient target coverage, unscanned local areas, insufficient effective overlap between adjacent point clouds, registration quality index below the threshold, excessive loop closure error, or obvious gaps, ghosting, steps, and low-density areas are detected in the fused point cloud, feedback scanning is automatically triggered to regenerate candidate supplementary viewpoints.

13. A three-dimensional scanning point cloud reconstruction system for complex workpieces, characterized in that, include: The module includes a target point set construction module, a candidate viewpoint generation module, a scan viewpoint sequence solving module, an overlapping sub-cloud extraction module, a progressive continuous registration module, and a point cloud fusion output module. The target point set construction module is used to process the initial scan data of the acquired complex workpiece to obtain a target point-normal set. The candidate viewpoint generation module is used to generate a candidate scan viewpoint set based on the target point-normal set; The scanning viewpoint sequence solving module is used to perform visibility evaluation, registration evaluation, and robot executability screening on the candidate scanning viewpoint set to obtain a scanning viewpoint sequence. The overlapping sub-cloud extraction module is used to move sequentially to each scanning pose according to the scanning viewpoint sequence, control the three-dimensional scanning sensor installed at the end of the robot to collect multiple frames of local point clouds, and obtain overlapping sub-clouds based on the multiple frames of local point clouds combined with the nearest neighbor constraint strategy. The progressive continuous registration module is used to perform progressive continuous fine registration on the point clouds of adjacent frames according to the overlapping sub-clouds, so as to obtain the fine registration pose of each frame point cloud relative to the target frame. The point cloud fusion output module is used to fuse multiple frames of local point clouds according to the finely registered pose to obtain a fused point cloud.

14. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor, the memory storing a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The storage medium is used to store a computer program, the computer program being used to cause a computer to perform the method according to any one of claims 1-12.

Images