Robot joint module three-dimensional model construction design method based on twin modeling

Abstract

The application discloses a robot joint module three-dimensional model construction design method based on twin modeling, relates to the technical field of three-dimensional modeling, and comprises the following steps: performing curvature distribution feature sensitive reduction on original multi-view point clouds to obtain a sparse feature point cloud set; performing coarse registration on adjacent view point clouds according to an iterative closest point algorithm with normal vector consistency constraint to generate an initial global point cloud skeleton; performing local deformation compensation through a non-rigid deformation field to obtain a corrected point cloud skeleton; performing structure semantic segmentation based on joint motion degrees of freedom to divide component-level point cloud subsets; performing Poisson surface reconstruction on each subset to generate an initial triangular mesh model; and performing boundary fitting degree optimization assembly alignment according to a global pose reference system to generate an overall assembly three-dimensional model. The application realizes high-precision reconstruction of a joint module and effectively suppresses noise and deformation error accumulation.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, specifically to a method for constructing and designing 3D models of robot joint modules based on twin modeling. Background Technology

[0002] Acquiring multi-view point cloud data of robot joint modules using 3D scanning equipment and reconstructing their 3D models based on this data is a key approach for digital design, simulation analysis, and status monitoring of joint modules. In this approach, the massive amounts of raw point cloud data from different scanning perspectives typically contain a large amount of redundant information, especially in low-curvature flat regions such as planes and regular curved surfaces. Excessively dense sampling points contribute little to the accuracy of model reconstruction but significantly increase the computational burden of subsequent registration and surface reconstruction. How to effectively preserve information about high-curvature geometric features such as edges, chamfers, and hole boundaries on the joint module while reducing the scale of the point cloud data is a major challenge faced by existing sparsity processing methods. Conventional uniform or random downsampling strategies often indiscriminately remove points from feature regions, leading to the loss of key geometric details and directly compromising the accuracy of the final model.

[0003] Furthermore, after reconstructing the complete joint module point cloud skeleton from point cloud fragments from multiple perspectives, it is necessary to separate the point clouds corresponding to different rigid components and perform independent surface reconstruction. Finally, the mesh models of each component are assembled into a complete module. In traditional workflows, the assembly alignment of component meshes often relies on manually specifying matching point pairs or using simple nearest-point iterative stitching. This method is difficult to accurately handle situations where there are design gaps or minor assembly deviations between components. In the joint area, adjacent component meshes often exhibit overlapping phenomena with mutually penetrating boundaries, or leave uncontrolled gaps, disrupting the geometric continuity of the overall assembly model. How to achieve a tight and non-penetrating boundary fit between component meshes at the assembly joint is an urgent problem to be solved in 3D model construction. Summary of the Invention

[0004] The purpose of this invention is to provide a design method for constructing a three-dimensional model of a robot joint module based on twin modeling. In the point cloud simplification stage, differential sampling and edge repair guided by spatial curvature distribution are used to preserve the integrity of points in high curvature feature areas and compress redundant data in flat areas. Furthermore, by using boundary fit deviation measurement and joint pose fine-tuning in the component assembly stage, non-penetrating tight fit of adjacent components at the joint surface is achieved.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a design method for constructing a three-dimensional model of a robot joint module based on twin modeling. The method starts from the original point cloud data of the robot joint module from multiple perspectives, and constructs an overall assembly three-dimensional model with high geometric accuracy, complete component structure and accurate assembly relationship through steps such as feature sensitive simplification, consistency coarse registration, non-rigid deformation compensation, structural semantic segmentation, surface reconstruction and assembly alignment.

[0006] As a technical solution of this invention, feature-sensitive point cloud simplification processing based on spatial curvature distribution is performed on the original multi-view point cloud data of the robot joint module to obtain a sparse feature point cloud set of the joint module. In this process, for each point cloud data point, the spatial curvature metric is determined using the local surface fitting parameters in its neighborhood space, and the point cloud is divided into high-curvature feature point groups and low-curvature flat point groups accordingly. The high-curvature feature point groups are completely preserved, while the low-curvature flat point groups undergo downsampling based on spatial uniform sampling, thereby significantly reducing redundant data while effectively maintaining key geometric features such as the model's edges and ridges. Preferably, the edge void regions remaining after downsampling are repaired using radial basis function interpolation. The boundary points of the edge void regions are used as interpolation nodes to solve for the radial basis function weight coefficients, obtaining the spatial coordinates of the newly added point cloud data points inside the voids, ultimately resulting in a sparse feature point cloud set of the joint module with continuous point cloud distribution and complete feature preservation.

[0007] Based on the aforementioned sparse feature point cloud set of the joint module, iterative nearest-point coarse registration processing based on normal vector consistency constraints is performed on point cloud segments from adjacent viewpoints to generate an initial global point cloud skeleton. Specifically, first-viewpoint and second-viewpoint point cloud segments from adjacent viewpoints are extracted from the sparse feature point cloud set, and the normal vectors of the point cloud data points in each segment are calculated. Using the cosine of the angle between the two normal vectors as a constraint, a set of normal vector consistency matching point pairs is established, and an initial rotation and translation transformation matrix is ​​calculated based on this set of matching point pairs. This initial rotation and translation transformation matrix is ​​used as the initial value for the iterative nearest-point algorithm, and iterative registration is performed on the point cloud segments from the two viewpoints. In each iteration, the distance residual between the matching point pairs is recalculated based on the updated rotation and translation transformation matrix. Preferably, when the relative change between the root mean square value of the distance residual calculated in the current iteration and the root mean square value of the distance residual calculated in the previous iteration is less than a preset iteration termination threshold, the distance residual is determined to have converged, the iteration is stopped, and the initial global point cloud skeleton is output. This scheme provides a high-quality initial transformation estimate for the iterative nearest point algorithm through normal vector consistency constraints, which significantly improves the convergence speed and global consistency of coarse registration and avoids the algorithm getting trapped in local optima.

[0008] A deformation-corrected point cloud skeleton is obtained by performing local deformation compensation processing based on a non-rigid deformation field on the initial global point cloud skeleton. In this processing, multiple control nodes are extracted from the local deformation-sensitive regions of the initial global point cloud skeleton, and an initial deformation displacement vector is assigned to each control node. Based on the spatial adjacency relationship between the control nodes, a non-rigid deformation field interpolation model based on radial basis functions is constructed and applied to all point cloud data points in the global point cloud skeleton. Each point cloud data point obtains a corresponding deformation displacement compensation amount according to its distance weight to each control node. This deformation displacement compensation amount is used to offset and correct the spatial coordinates of each point cloud data point, and all corrected point cloud data points are recombined into a deformation-corrected point cloud skeleton. This method can effectively compensate for the non-rigid deviation of the point cloud skeleton caused by the cumulative error of multi-view stitching and the local elastic deformation of the structure, making the corrected point cloud skeleton closer to the true geometry of the joint module.

[0009] Based on the deformation-corrected point cloud skeleton, structural semantic segmentation based on joint motion degrees of freedom is performed to divide the rigid components in the joint module into component-level point cloud subsets. During implementation, the directions of motion degrees of freedom for each rigid component and the relative motion constraints between components are determined according to the preset kinematic parameters of the robot joint module. Based on this kinematic information, the theoretical segmentation boundary positions between each rigid component are marked on the deformation-corrected point cloud skeleton. Using the theoretical segmentation boundary positions as initial segmentation seeds, point cloud clustering segmentation based on region growing is performed on the deformation-corrected point cloud skeleton. Preferably, when performing region growing clustering, point cloud data points within a preset neighborhood range on both sides of the theoretical segmentation boundary position are used as initial seed points, and the angle between the normal vectors of the point cloud data points and the spatial Euclidean distance are used as joint growth criteria. After obtaining the initial component point cloud clustering results, the clustering boundaries are adjusted according to the preset geometric dimensions of each rigid component, and the adjusted point cloud data points belonging to the same rigid component are grouped into component-level point cloud subsets. This segmentation strategy is driven by joint motion semantic information, which can accurately separate different rigid parts with complex shapes and close spatial proximity, effectively avoiding the foreground / background adhesion and oversegmentation problems common in traditional unsupervised segmentation methods.

[0010] For each component-level point cloud subset, surface fitting based on Poisson surface reconstruction is performed to generate an initial triangular mesh model for each rigid component. For each component-level point cloud subset, its corresponding directed point set is first calculated, with each point carrying a normal vector pointing outwards from the component. Then, a Poisson equation is constructed between the gradient field of the indicator function and the normal vector of the directed point set, and this equation is solved to obtain an approximate solution for the indicator function. Triangular facets are extracted from the approximate solution of the indicator function using an isosurface extraction algorithm to generate an initial component triangular mesh. This initial component triangular mesh is then subjected to triangular facet homogenization processing based on mesh quality optimization, resulting in an initial triangular mesh model with a regular mesh distribution, smooth surface, and effective representation of the component's detailed features.

[0011] Based on the initial triangular mesh models of each rigid component and the global pose reference system in the deformation correction point cloud skeleton, component assembly alignment processing based on boundary fit optimization is performed to generate an overall assembled 3D model of the joint module. Position information of the assembly mating surfaces between each rigid component is extracted from the deformation correction point cloud skeleton, and the target assembly pose of each rigid component in the global pose reference system is determined accordingly. The initial triangular mesh model of each rigid component is transformed to its corresponding spatial position in the global pose reference system, resulting in a transformed component mesh model. Based on this, the boundary fit deviation of the transformed component mesh models of adjacent rigid components at the assembly mating surfaces is calculated. This deviation includes two indices: boundary gap distance and boundary overlap depth. Based on the boundary fit deviation, joint pose fine-tuning is performed on the transformed component mesh models of adjacent rigid components. This fine-tuning process is repeated until the boundary gap distance and boundary overlap depth between adjacent rigid components are simultaneously less than a preset tolerance threshold. Finally, all fine-tuned component mesh models are merged to form a high-quality overall assembled 3D model without through gaps or non-physical compression. This process effectively eliminates assembly ambiguity and boundary mismatch issues introduced by independent component modeling, ensuring the geometric consistency and physical rationality of the final assembly model.

[0012] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0013] In the point cloud simplification stage of the robot joint module, the local surface fitting parameters in the neighborhood space of each point cloud data point are calculated to obtain the spatial curvature metric value, thereby dividing all points into high-curvature feature point groups and low-curvature flat point groups. Points in the high-curvature feature point group are preserved without reduction, directly maintaining the original sampling density of structurally sensitive areas such as edges, hole boundaries, and chamfers. Points in the low-curvature flat point group are downsampled based on spatial uniform sampling, significantly reducing data redundancy. For edge voids remaining in local areas after downsampling, the radial basis function interpolation method is used, with the void boundary points as interpolation nodes for neighborhood interpolation repair, generating the spatial coordinates of new point cloud data points. This differentiated processing method ensures that while the overall data scale is compressed, the high-curvature regions affecting critical geometric accuracy do not suffer feature signal loss. The point cloud information density is guided towards the structural contours upon which the reconstruction model depends, enabling the mesh model in the surface reconstruction stage to more faithfully reproduce the fine shape of the original workpiece.

[0014] After obtaining the initial triangular mesh models of each rigid component, the assembly mating surface position information between the components is extracted from the deformation correction point cloud skeleton, and the target assembly pose of each component is determined in the global pose reference system accordingly. After completing the initial spatial position transformation, the boundary fit deviation of adjacent components at the assembly mating surface is specifically calculated, including two types of indicators: boundary gap distance and boundary overlap depth. Based on these two indicators, the mesh models of adjacent components are jointly pose-fine-tuned, driving the component mesh to converge towards a state where the boundary gap distance and boundary overlap depth of their joint area are simultaneously minimized through numerical optimization, until both are compressed to below the preset tolerance. Through the assembly alignment mechanism oriented towards boundary fit, the spatial relationship of the component meshes at the contact surface is precisely controlled, eliminating visible gaps between meshes and avoiding geometric contradictions caused by surface penetration. The final merged overall assembly 3D model has a continuous and tightly fitted topological morphology at the mating boundary. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0016] Figure 1 This is a flowchart of a design method for constructing a 3D model of a robot joint module based on twin modeling;

[0017] Figure 2 This is a flowchart of a simplified method for multi-view point cloud feature sensitivity for robot joint modules based on spatial curvature distribution;

[0018] Figure 3 This is a flowchart of the local deformation correction process for point cloud skeletons based on non-rigid deformation fields. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See Figure 1 This invention provides a method for constructing and designing a 3D model of a robot joint module based on twin modeling. The method involves performing feature-sensitive point cloud simplification processing based on spatial curvature distribution on the original multi-view point cloud data of the robot joint module to obtain a sparse feature point cloud set. Based on this sparse feature point cloud set, iterative nearest-point coarse registration processing based on normal vector consistency constraints is performed on point cloud segments from adjacent views to generate an initial global point cloud skeleton. Local deformation compensation processing based on a non-rigid deformation field is performed on the initial global point cloud skeleton to obtain a deformation-corrected point cloud skeleton. Based on the deformation-corrected point cloud skeleton, structural semantic segmentation processing based on joint motion degrees of freedom is performed to divide the component-level point cloud subsets corresponding to each rigid component in the joint module. Surface fitting processing based on Poisson surface reconstruction is performed on the component-level point cloud subsets to generate initial triangular mesh models for each rigid component. Based on the initial triangular mesh models of each rigid component and the global pose reference system in the deformation-corrected point cloud skeleton, component assembly alignment processing based on boundary fit optimization is performed to generate the overall assembled 3D model of the joint module.

[0021] Example 1

[0022] See Figure 2 The original multi-view point cloud data of the robot joint module is processed using feature-sensitive point cloud simplification based on spatial curvature distribution to obtain a sparse feature point cloud set of the joint module. In specific implementation, original multi-view point cloud data of the robot joint module collected from multiple perspectives is acquired. For each point in the original multi-view point cloud data, a spherical neighborhood space is determined with that point as the center and its radius as the neighborhood search radius. The neighborhood search radius is determined based on the average point spacing of the original multi-view point cloud data, and can be three to five times the average point spacing.

[0023] For neighborhood point cloud data points located within the neighborhood space, a local quadratic surface is fitted using the least squares method. The functional form of the local quadratic surface is: The local surface fitting parameters are obtained by solving the normal equations. , , , , , Based on the local surface fitting parameters, the Gaussian curvature metric or the average curvature metric of the point cloud data points is calculated as the spatial curvature metric of the point cloud data points. In one optional approach, the spatial curvature metric is taken as the Gaussian curvature metric, calculated using the formula... Calculated.

[0024] Based on the calculated spatial curvature metric values, all point cloud data points in the original multi-view point cloud data are divided into a high-curvature feature point group and a low-curvature flat point group. The division method is as follows: the spatial curvature metric values ​​of all point cloud data points are statistically analyzed, and the upper quartiles of the spatial curvature metric values ​​are obtained. These upper quartiles are set as the curvature division threshold. Point cloud data points with spatial curvature metric values ​​greater than the curvature division threshold are assigned to the high-curvature feature point group, while point cloud data points with spatial curvature metric values ​​not greater than the curvature division threshold are assigned to the low-curvature flat point group.

[0025] All point cloud data points in the high-curvature feature point group are retained without deletion. Point cloud data points in the low-curvature flat point group are downsampled using spatial uniform sampling to obtain an initial simplified point cloud set. The spatial uniform sampling downsampling operation includes: constructing a 3D spatial bounding box for the low-curvature flat point group; dividing the 3D spatial bounding box into uniformly sized cubic grids along the three coordinate axes; the side length of the cubic grids is determined based on the target simplification ratio and the point cloud density of the low-curvature flat point group; within each cubic grid, retaining the point cloud data point with the closest spatial Euclidean distance to the center point of the cubic grid, and discarding the remaining point cloud data points. The retained point cloud data points are then merged with the point cloud data points in the high-curvature feature point group to form the initial simplified point cloud set.

[0026] Based on the spatial distribution of each point cloud data point in the initial simplified point cloud set, edge void regions left at the surface boundaries due to downsampling operations are detected. The edge void region detection method is as follows: calculate the neighborhood projection distribution of each point cloud data point in the initial simplified point cloud set; when the neighborhood missing angle of a point cloud data point in a certain direction exceeds a preset angle threshold, the point cloud data point is marked as the boundary point of the edge void region; the region enclosed by the set of boundary points that are interconnected and form a closed loop is determined as the edge void region.

[0027] Edge repair processing based on neighborhood interpolation is performed on the edge hole regions remaining after downsampling. The edge repair processing uses radial basis function interpolation, with the boundary points of the edge hole regions as interpolation nodes. The spatial coordinates of newly added point cloud data points within the edge hole regions are obtained by solving the radial basis function weight coefficients. The basis function form used for radial basis function interpolation is as follows:

[0028]

[0029] in, This represents the spatial Euclidean distance between the point to be interpolated and the interpolation node; For shape control parameters, The value of is determined based on the scale of the edge void region, and is taken as the reciprocal of the average distance between adjacent boundary points within the edge void region. When solving for the radial basis function weight coefficients, a system of linear equations is constructed for each boundary point. Satisfies the interpolation conditions ,in This represents the total number of boundary points in the edge void region. For the first The radial basis function weight coefficients corresponding to each interpolation node are obtained by solving this system of linear equations. The desired location is generated within the edge void region using a uniform grid. The spatial coordinates of the newly added point cloud data points are obtained through... The calculated new point cloud data points are added to the initial simplified point cloud set to form a sparse feature point cloud set for the joint module.

[0030] Example 2

[0031] Based on the sparse feature point cloud set of the joint module, iterative nearest-point coarse registration processing based on normal vector consistency constraints is performed on the point cloud segments from adjacent viewpoints to generate an initial global point cloud skeleton. In specific implementation, the first-viewpoint point cloud segment and the second-viewpoint point cloud segment from adjacent viewpoints are extracted from the sparse feature point cloud set of the joint module according to the viewpoint acquisition order. The first-viewpoint point cloud segment is the set of point cloud data points acquired from the v-th viewpoint after feature-sensitive point cloud simplification, and the second-viewpoint point cloud segment is the set of point cloud data points acquired from the (v+1)-th viewpoint after feature-sensitive point cloud simplification.

[0032] For each point cloud data point in the first-view point cloud segment, calculate the corresponding first normal vector. The first normal vector is calculated as follows: for a point cloud data point in the first-view point cloud segment, obtain the neighborhood point set of that point cloud data point. The neighborhood point set consists of all neighboring points whose spatial Euclidean distance to the point cloud data point is within a preset neighborhood radius; perform eigenvalue decomposition on the covariance matrix of the neighborhood point set, and determine the eigenvector corresponding to the smallest eigenvalue as the first normal vector of that point cloud data point. For each point cloud data point in the second-view point cloud segment, calculate the corresponding second normal vector in the same way.

[0033] Using the cosine of the angle between the first and second normal vectors as a constraint, a set of point pairs with consistent normal vectors is established between the first-view and second-view point cloud segments. For a point cloud data point in the first-view point cloud segment, its first normal vector is denoted as... For a point cloud data point in a second-view point cloud segment, its second normal vector is denoted as... ,calculate and The cosine value of the angle between the two point cloud data points is used to determine the matching point pair. If the cosine value is greater than a preset cosine threshold, the two point cloud data points are considered as a pair with consistent normal vectors. If the cosine value is not greater than the preset cosine threshold, no matching relationship is established. The preset cosine threshold is set to 0.95, retaining only matching point pairs with consistent normal vector directions and excluding incorrect matches with large deviations in normal vectors.

[0034] Based on the set of matching point pairs with consistent normal vectors, the initial rotation and translation transformation matrix between the first-view and second-view point cloud segments is calculated. The coordinates of the matching point pairs in the set of matching point pairs with consistent normal vectors are decentered, a covariance matrix is ​​constructed, and singular value decomposition is performed. The rotation matrix components and translation vector components are extracted from the singular value decomposition results and combined to form the initial rotation and translation transformation matrix.

[0035] Using the initial rotation-translation transformation matrix as the initial value for the iterative nearest-neighbor algorithm, the first-view and second-view point cloud segments are iteratively registered. During each iteration, the following operations are performed: Based on the current rotation-translation transformation matrix, all point cloud data points in the first-view point cloud segment undergo coordinate transformation; for each point cloud data point in the first-view point cloud segment after coordinate transformation, the point cloud data point in the second-view point cloud segment with the nearest Euclidean distance is searched, establishing the set of matching point pairs for the current iteration; based on the set of matching point pairs for the current iteration, the updated rotation-translation transformation matrix is ​​calculated; based on the updated rotation-translation transformation matrix, the distance residual between the matching point pairs is recalculated, obtaining the root mean square value of the distance residual.

[0036] The criterion for convergence of the distance residual is determined as follows: Let the first... The root mean square value of the distance residual between the matching point pairs calculated in the next iteration is , No. The root mean square value of the distance residual between the matching point pairs calculated in the next iteration is ,calculate and The relative change between them is When the relative change Less than the preset iteration termination threshold When the distance residual converges, the iteration process terminates. A preset iteration termination threshold is used. Set as This ensures that the registration accuracy meets the requirements of convergence stability while avoiding excessive iterative computation overhead.

[0037] When the distance residual converges, the rotation and translation transformation matrix obtained in the last iteration is used as the final transformation matrix. The first-view point cloud fragment and the second-view point cloud fragment are merged into a single registered point cloud fragment under the final transformation matrix. The above coarse registration process is then performed on the point cloud fragments of all adjacent views in sequence. All registered point cloud fragments are then merged to output the initial global point cloud skeleton.

[0038] Example 3

[0039] See Figure 3 The initial global point cloud skeleton is processed using local deformation compensation based on a non-rigid deformation field to obtain a deformed point cloud skeleton. In specific implementation, multiple control nodes within the local deformation-sensitive region are extracted from the initial global point cloud skeleton. The initial global point cloud skeleton is obtained by fusing coarsely registered multi-view point cloud fragments. Local deformation may exist in overlapping areas of adjacent views due to accumulated registration errors. To determine the local deformation-sensitive region, a local neighborhood consistency residual is calculated for each point cloud data point in the initial global point cloud skeleton. The calculation method is as follows: for a point cloud data point in the initial global point cloud skeleton, the set of neighboring points within a preset neighborhood radius is obtained. A covariance matrix is ​​constructed for this neighborhood point set, and the minimum eigenvalue of the covariance matrix is ​​calculated. The ratio of the minimum eigenvalue to the mean distance of all point cloud data points in the neighborhood point set is used as the local neighborhood consistency residual for that point cloud data point. Connected regions formed by point cloud data points whose local neighborhood consistency residuals are greater than a preset residual threshold are marked as local deformation-sensitive regions. Spatially uniform sampling is performed within the local deformation-sensitive area, with the sampling points serving as control nodes. The number of control nodes is determined based on the volume and point cloud density of the local deformation-sensitive area, ensuring that the spatial Euclidean distance between adjacent control nodes is not less than the preset minimum sampling interval.

[0040] Assign an initial deformation displacement vector to each control node. For each control node, search the initial global point cloud skeleton for the set of nearest points that have the closest spatial Euclidean distance to the control node, calculate the offset vector between the control node and the centroid of the corresponding set of nearest points, and use this offset vector as the initial deformation displacement vector for the control node.

[0041] Based on the spatial adjacency relationships among multiple control nodes, a non-rigid deformation field interpolation model based on radial basis functions is constructed. The non-rigid deformation field interpolation model is constructed as follows: for each control node, its spatial coordinates are denoted as... The initial deformation displacement vector corresponding to this control node is denoted as , The range of values ​​is to , To control the total number of nodes; the radial basis function is selected as follows:

[0042]

[0043] in The spatial Euclidean distance between two points. The shape factor of the radial basis functions. The value is determined based on the average spatial Euclidean distance between control nodes. Determined as

[0044]

[0045] To ensure that the coverage of the radial basis functions matches the distribution density of the control nodes; construct the interpolation weight coefficient vector to solve the system of equations, for the There are 1 control node that satisfies the interpolation constraints:

[0046]

[0047] In the formula: Indicates the first The weight coefficient vector corresponding to each control node For the summation index, The range of values ​​is to ; Indicates the first The first control node and the first The radial basis function value is calculated using the spatial Euclidean distance between each control node as the independent variable. For the first The initial deformation displacement vector of each control node. The complete weight coefficient vector is obtained by solving this system of linear equations. The interpolation model for the non-rigid deformation field was constructed.

[0048] The non-rigid deformation field interpolation model is applied to all point cloud data points in the initial global point cloud skeleton, so that each point cloud data point obtains a corresponding deformation displacement compensation amount according to its distance weight to each control node. For any point cloud data point in the initial global point cloud skeleton, its original spatial coordinates are: The deformation displacement compensation amount of the point cloud data point is calculated based on the non-rigid deformation field interpolation model. The method is as follows:

[0049]

[0050] In the formula: This indicates that an application needs to be applied to the spatial coordinates. The deformation and displacement compensation amount of point cloud data points; This indicates that the point cloud data point is related to the first... Spatial coordinate difference vector between control nodes; This indicates that the point cloud data point is related to the first... Spatial Euclidean distance between control nodes; These are radial basis function values; It is the first The weight coefficient vectors of each control node are summed, and the contributions of all control nodes are accumulated to obtain the deformation displacement compensation vector corresponding to the point cloud data point.

[0051] The spatial coordinates of each point cloud data point in the initial global point cloud skeleton are offset and corrected based on the deformation displacement compensation amount. Offset correction is performed by adjusting the original spatial coordinates of the point cloud data points. Corresponding deformation displacement compensation amount This is achieved by adding vectors to obtain the offset-corrected spatial coordinates. All offset-corrected point cloud data points are recombined to form a deformation-corrected point cloud skeleton. The deformation-corrected point cloud skeleton eliminates local non-rigid deformations left over from the coarse registration process, providing a spatially accurate geometric reference for subsequent structural semantic segmentation and surface reconstruction.

[0052] Example 4

[0053] Based on the deformation-corrected point cloud skeleton, structural semantic segmentation based on joint motion degrees of freedom is performed to divide the component-level point cloud subsets corresponding to each rigid component in the joint module. In specific implementation, preset kinematic parameters of the robot joint module are obtained, including the joint axis direction, range of motion, and relative motion constraints between each rigid component. The motion degree of freedom direction corresponding to each rigid component is extracted from the preset kinematic parameters to determine the kinematic relationship of relative rotation or translation between adjacent rigid components along the motion degree of freedom direction.

[0054] Based on the directions of motion degrees of freedom and relative motion constraints, the theoretical segmentation boundary positions between each rigid component are marked in the deformation correction point cloud skeleton. The method for marking the theoretical segmentation boundary positions is as follows: for every two adjacent rigid components, a motion segmentation surface is determined between them; the motion segmentation surface is a plane or curved surface, perpendicular to the relative motion degrees of freedom directions of the two rigid components and passing through the joint motion center point; the set of point cloud data points corresponding to the motion segmentation surface in the deformation correction point cloud skeleton is marked as the theoretical segmentation boundary position.

[0055] Using the theoretical segmentation boundary as the initial segmentation seed, point cloud clustering segmentation based on region growing is performed on the deformation-corrected point cloud skeleton. The region growing process uses point cloud data points within a preset neighborhood range on both sides of the theoretical segmentation boundary as initial seed points. The neighborhood range includes all point cloud data points within eight times the average point spacing of the deformation-corrected point cloud skeleton. When performing point cloud clustering segmentation based on region growing on the deformation-corrected point cloud skeleton, the angle between the normal vectors of the point cloud data points and the spatial Euclidean distance are used as joint growth criteria. For each initial seed point, the angle between the normal vectors and the point cloud data points to be judged are calculated. When the angle between the normal vectors is less than a preset angle threshold and the spatial Euclidean distance is less than a preset distance threshold, the point cloud data points to be judged are included in the same growth region. When the angle between the normal vectors is not less than the preset angle threshold or the spatial Euclidean distance is not less than the preset distance threshold, the point cloud data points to be judged are not included in the current growth region. The preset angle threshold is set to... This is to cover the range of normal vector changes caused by surface curvature variations in the deformation-corrected point cloud skeleton. A preset distance threshold is set to five times the average point spacing of the deformation-corrected point cloud skeleton to balance cluster connectivity and region separation. The region growth process is as follows: starting from the initial seed point, it expands outward layer by layer, adding point cloud data points that meet the joint growth criterion to the current region and using them as new seed points to continue expanding outward until no new point cloud data points meet the joint growth criterion, resulting in an initial component point cloud cluster.

[0056] After completing region growing, all initial component point cloud clusters are merged into an initial component point cloud clustering result. The boundaries of the initial component point cloud clustering result are adjusted according to the preset geometric dimensions of each rigid component. The preset geometric dimensions include the theoretical length, diameter, boundary contour shape, and thickness of the rigid component. The boundary adjustment method is as follows: for each initial component point cloud cluster, the minimum bounding box size of the cluster is calculated. This minimum bounding box size is compared with the preset geometric dimensions of the corresponding rigid component. Point cloud data points in areas exceeding the preset geometric dimensions are removed, and data points are added to the deficient areas according to the boundary contour of the preset geometric dimensions. The addition of deficient areas is achieved by projecting along the normal direction of the theoretical boundary and sampling to generate point cloud data points. After adjustment, all point cloud data points belonging to the same rigid component are grouped into a component-level point cloud subset.

[0057] During boundary adjustment, a quantization correction step based on point cloud-to-geometric model matching is employed. For the first... Theoretical boundary surface of a rigid component Extract the initial component point cloud clusters that are related to the theoretical boundary surface. Boundary point cloud data points whose distances are within a preset range are used to form a boundary point cloud set; construct the objective function:

[0058]

[0059] in, This represents the optimization objective value for boundary adjustment; The first point in the boundary point cloud set represents the first... Spatial coordinates of point cloud data points The range of values ​​is to , The total number of points in the boundary point cloud set; Indicates the first Point cloud data points to the theoretical boundary surface The square of the signed distance value is used to measure the boundary gap; Indicates the first A point cloud data point penetrates the theoretical boundary surface The square of the penetration depth is used to measure the boundary overlap depth; To balance the weighting factors of boundary gaps and boundary overlaps, The range of values ​​is arrive The real number between them is determined based on the tolerance requirements for gaps and overlaps; when the gap requirements are more stringent... Value greater than When the requirements for overlap are more stringent Value less than By minimizing the objective function Fine-tuning the coordinates of the boundary point cloud set ensures that the boundary point cloud data points converge to the theoretical boundary surface. Nearby, the boundary adjustment of the point cloud for a single rigid component is completed. The above boundary adjustment operation is performed on the initial component point cloud clusters for all rigid components, and all point cloud data points belonging to the same rigid component after adjustment are grouped into component-level point cloud subsets. Each component-level point cloud subset corresponds to an independent rigid component in the joint module.

[0060] Example 5

[0061] A surface fitting process based on Poisson surface reconstruction is performed on the component-level point cloud subsets to generate initial triangular mesh models for each rigid component. In specific implementation, for each component-level point cloud subset obtained from structural semantic segmentation, its corresponding directed point set is calculated. The directed point set is calculated as follows: for each point cloud data point in the component-level point cloud subset, the initial normal vector direction of that point cloud data point is estimated using a neighborhood covariance analysis method. The initial normal vector direction is obtained by constructing the covariance matrix of the neighborhood point set centered on that point cloud data point, performing eigenvalue decomposition on the covariance matrix, and using the eigenvector corresponding to the smallest eigenvalue as the initial normal vector direction. After obtaining the initial normal vector direction, a globally consistent orientation adjustment is performed on all initial normal vector directions so that all normal vectors point outwards from the corresponding rigid component. The method for adjusting the orientation of the normal vector is as follows: Construct the global centroid of a component-level point cloud subset. For each point cloud data point, calculate the direction vector from that point to the global centroid. When the angle between the initial normal vector direction and this direction vector is greater than 90 degrees, flip the initial normal vector direction so that the adjusted normal vector points outward from the component. Combine the spatial coordinates of each point cloud data point with the adjusted normal vector pointing outward from the component to form a directed point set. Each point in the directed point set carries three-dimensional spatial coordinates and the corresponding normal vector information.

[0062] Based on the directed point set, a Poisson equation is constructed between the gradient field of the indicator function and the normal vector of the directed point set. The process of constructing the Poisson equation is as follows: Define an indicator function in 3D space, where the indicator function takes the value of one inside the component and zero outside the component. The gradient field of the indicator function is non-zero near the component boundary. Use the normal vector information from the directed point set as discrete samples of the gradient field of the indicator function to construct a vector field as an approximation of the gradient of the indicator function. Discretize the space containing the component-level point cloud subset into an octree structure. The maximum depth of the octree is determined based on the point density of the component-level point cloud subset, taking the maximum depth value such that the number of point cloud data points contained in each leaf node does not exceed ten. Define a basis function on each leaf node of the octree, using cubic spline basis functions. Construct the Poisson equation in matrix form. The unknowns of the Poisson equation are the indicator function values ​​at the octree nodes. The right-hand side of the Poisson equation is obtained by the projection integral of the normal vectors in the directed point set onto the basis functions of the octree nodes. Solving the Poisson equation yields an approximate solution for the indicator function of the component-level point cloud subset. The values ​​of the approximate indicator function solution at each node of the octree are obtained by solving a sparse linear system of equations. The conjugate gradient method is used to solve the linear system of equations, and the iterative convergence tolerance is set to... .

[0063] Triangular facets are extracted from the approximate solution of the indicator function using an isosurface extraction algorithm. The isosurface extraction algorithm employs the moving cube method, with the isosurface threshold set to the expected value of the indicator function at the component boundary. On each edge of the octagonal leaf node mesh, the intersection of the isosurface with the edge is determined based on the values ​​of the approximate solutions of the indicator functions at both ends. When one endpoint of an edge has an indicator function value greater than the isosurface threshold and the other less than the threshold, the intersection point is determined by linear interpolation. Based on a predefined topology connection table defined by the moving cube method, all intersection points are connected to form triangular facets, generating the initial component triangular mesh.

[0064] The initial component triangular mesh undergoes triangular facet homogenization based on mesh quality optimization. Mesh quality optimization includes two steps: vertex position smoothing and triangular facet shape optimization. Vertex position smoothing employs the Laplacian smoothing method. For each vertex in the initial component triangular mesh, the average position of all adjacent vertices within its ring neighborhood is calculated. The vertex's position is then shifted towards its average position by a preset smoothing step size, which is 20% of the distance between the vertex and the average position. Triangular facet shape optimization uses a joint optimization based on edge length and area. It iterates through each edge in the initial component triangular mesh. When the edge length exceeds a preset maximum edge length threshold, a new vertex is inserted at the midpoint of the edge, and the relevant triangular facets are re-divided. When the area of ​​a triangular facet exceeds a preset maximum area threshold, a new vertex is inserted at the centroid of the triangular facet, and the facets are re-divided. The preset maximum edge length threshold is set to twice the average point spacing of the component-level point cloud subset, and the preset maximum area threshold is set to four times the square of the average point spacing of the component-level point cloud subset. The homogenized triangular mesh is used as the initial triangular mesh model for the corresponding rigid component.

[0065] Based on the initial triangular mesh model of each rigid component and the global pose reference system in the deformation correction point cloud skeleton, component assembly alignment processing based on boundary fit optimization is performed to generate the overall assembly 3D model of the joint module. In specific implementation, the assembly mating surface position information between each rigid component is extracted from the deformation correction point cloud skeleton. The extraction method for the assembly mating surface position information is as follows: for adjacent first and second rigid components, the first and second assembly mating surfaces are fitted according to the spatial distribution of point cloud data points near the theoretical segmentation boundary position in the deformation correction point cloud skeleton; the geometric parameters of the first and second assembly mating surfaces are used as the assembly mating surface position information. Based on the assembly mating surface position information, the target assembly pose of each rigid component in the global pose reference system is determined, and the target assembly pose makes the first and second assembly mating surfaces relatively fit each other in the global pose reference system.

[0066] Based on the target assembly pose, the initial triangular mesh model of each rigid component is transformed to its corresponding spatial position in the global pose reference frame. For the first... Initial triangular mesh model of a rigid component By using rotation matrix and translation vector The coordinates of all vertices are transformed to the global pose reference frame to obtain the transformed part mesh model. The transformation process applies a 3D rigid body transformation, where the rotation matrix is ​​obtained from the quaternions or Euler angle parameters in the target assembly pose, and the translation vector is the position coordinates in the target assembly pose.

[0067] Calculate the boundary fit deviation of the transformed component mesh models of adjacent rigid components at the assembly mating surface. The boundary fit deviation includes the boundary gap distance and the boundary overlap depth. The boundary gap distance is calculated as follows: within the assembly mating surface region, sample from the boundary surface of the first transformed component mesh model. For each sampling point, calculate its nearest distance to the boundary surface of the second transformed component mesh model. The root mean square value of all sampling point distances constitutes the boundary gap distance. The boundary overlap depth is calculated as follows: within the assembly mating surface area, detect the area where the first and second transformed component mesh models spatially overlap. For each overlapping point in the overlapping area, calculate its depth value in the normal direction penetrating the boundary surface of the other component. The root mean square value of all depth values ​​constitutes the boundary overlap depth.

[0068] Based on the boundary fit deviation, joint pose fine-tuning is performed on the transformed mesh models of adjacent rigid components. Joint pose fine-tuning is achieved by constructing and solving an optimization problem, with the optimization objective being to minimize the boundary gap distance and boundary overlap depth between adjacent rigid components. The optimization variables are the six-DOF pose parameter fine-tuning amounts of the first and second rigid components, specifically the translation fine-tuning amounts along the three coordinate axes. , , And the rotational fine-tuning around the three coordinate axes , , The objective function to be optimized is:

[0069]

[0070] in, This indicates the optimization of the objective function value; The weighting coefficient for the boundary gap distance. Values ; The weighting coefficients for the boundary overlap depth. Values This allows the optimization process to pay equal attention to the deviations of both parts, but with a slight emphasis on gap control. This represents the total number of boundary gap sampling points within the assembly mating surface area. The index is the serial number of the sampling point in the boundary gap. The range of values ​​is to ; Indicates the first The shortest distance from each boundary gap sampling point to the boundary surface of the other component; This represents the total number of overlapping points in the overlapping region. The index is the sequence number of the overlapping points. The range of values ​​is to ; Indicates the first The depth value of each overlapping point penetrating the boundary surface of the other component in the normal direction. When solving the optimization problem, a gradient descent numerical optimization algorithm is used to iteratively update the pose parameters of adjacent rigid components until the objective function value is optimized. The change amplitude is less than the preset convergence threshold The iteration stops when the boundary gap distance and boundary overlap depth between adjacent rigid components are both less than the preset tolerance. The assembly alignment is then deemed to meet the accuracy requirements, with the preset tolerance set to one-tenth of the average point spacing of the component-level point cloud subset. All fine-tuned component mesh models are merged into a complete triangular mesh model under the global pose reference system, generating the overall assembly 3D model of the joint module.

[0071] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for constructing and designing 3D models of robot joint modules based on twin modeling, characterized in that, The method includes: The original multi-view point cloud data of the robot joint module is subjected to feature-sensitive point cloud simplification based on spatial curvature distribution to obtain a sparse feature point cloud set of the joint module, including: For each point cloud data point in the original multi-view point cloud data, calculate the local surface fitting parameters in its neighborhood space, and determine the spatial curvature metric value of each point cloud data point based on the local surface fitting parameters. Based on the spatial curvature metric value, divide all point cloud data points in the original multi-view point cloud data into a high curvature feature point group and a low curvature flat point group. Perform a retention operation on the point cloud data points in the high curvature feature point group, and perform a downsampling operation based on spatial uniform sampling on the point cloud data points in the low curvature flat point group to obtain an initial simplified point cloud set. Based on the spatial position distribution of each point cloud data point in the initial simplified point cloud set, perform edge repair processing based on neighborhood interpolation on the edge hole regions left after the downsampling operation to obtain the sparse feature point cloud set of the joint module. Based on the sparse feature point cloud set of the joint module, iterative nearest point coarse registration processing based on normal vector consistency constraint is performed on the point cloud segments under adjacent views to generate an initial global point cloud skeleton. The initial global point cloud skeleton is subjected to local deformation compensation processing based on a non-rigid deformation field to obtain a deformation-corrected point cloud skeleton. Based on the deformation-corrected point cloud skeleton, structural semantic segmentation processing based on joint motion degrees of freedom is performed to divide the component-level point cloud subsets corresponding to each rigid component in the joint module, including: Based on the preset kinematic parameters of the robot joint module, the directions of motion degrees of freedom corresponding to each rigid component in the joint module and the relative motion constraints between each rigid component are determined. Based on the directions of motion degrees of freedom and the relative motion constraints, the theoretical segmentation boundary positions between each rigid component are marked in the deformation correction point cloud skeleton. Using the theoretical segmentation boundary positions as initial segmentation seeds, point cloud clustering segmentation based on region growing is performed on the deformation correction point cloud skeleton to obtain the initial component point cloud clustering results. Based on the preset geometric dimension parameters of each rigid component, the boundaries of the initial component point cloud clustering results are adjusted, and all point cloud data points belonging to the same rigid component after adjustment are grouped into the component-level point cloud subset. Perform surface fitting processing based on Poisson surface reconstruction on the component-level point cloud subset to generate the initial triangular mesh model of each rigid component; Based on the initial triangular mesh model of each rigid component and the global pose reference system in the deformation correction point cloud skeleton, component assembly alignment processing based on boundary fit optimization is performed to generate the overall assembly 3D model of the joint module.

2. The method for constructing and designing a 3D model of a robot joint module based on twin modeling according to claim 1, characterized in that, Based on the sparse feature point cloud set of the joint module, iterative nearest-point coarse registration processing based on normal vector consistency constraints is performed on the point cloud segments under adjacent views to generate an initial global point cloud skeleton, including: Extract first-viewpoint cloud segments and second-viewpoint cloud segments from the sparse feature point cloud set of the joint module, and calculate the first normal vector of each point cloud data point in the first-viewpoint cloud segment and the second normal vector of each point cloud data point in the second-viewpoint cloud segment. Using the cosine of the angle between the first normal vector and the second normal vector as a constraint, a set of normal vector consistency matching point pairs is established between the first viewpoint point cloud segment and the second viewpoint point cloud segment; Based on the set of matching point pairs with consistent normal vectors, calculate the initial rotation and translation transformation matrix between the first viewpoint point cloud segment and the second viewpoint point cloud segment; Using the initial rotation and translation transformation matrix as the initial value of the iterative nearest point algorithm, the first viewpoint point cloud segment and the second viewpoint point cloud segment are iteratively registered. In each iteration, the distance residual between the matching point pairs is recalculated based on the updated rotation and translation transformation matrix until the distance residual converges, at which point the initial global point cloud skeleton is output.

3. The method for constructing and designing a 3D model of a robot joint module based on twin modeling according to claim 1, characterized in that, Performing local deformation compensation processing based on a non-rigid deformation field on the initial global point cloud skeleton yields a deformed point cloud skeleton, including: Multiple control nodes are extracted from the local deformation-sensitive region from the initial global point cloud skeleton, and an initial deformation displacement vector is assigned to each control node. Based on the spatial adjacency relationship between the multiple control nodes, a non-rigid deformation field interpolation model based on radial basis functions is constructed; The non-rigid deformation field interpolation model is applied to all point cloud data points in the initial global point cloud skeleton, so that each point cloud data point obtains the corresponding deformation displacement compensation amount according to its distance weight from each control node. The spatial coordinates of each point cloud data point in the initial global point cloud skeleton are offset and corrected according to the deformation displacement compensation amount, and all the offset-corrected point cloud data points are recombined into the deformation-corrected point cloud skeleton.

4. The method for constructing and designing a 3D model of a robot joint module based on twin modeling according to claim 1, characterized in that, Perform surface fitting processing based on Poisson surface reconstruction on the component-level point cloud subset to generate initial triangular mesh models for each rigid component, including: For each component-level point cloud subset, calculate its corresponding directed point set, where each point in the directed point set carries normal vector information pointing to the outside of the component. Based on the directed point set, a Poisson equation is constructed between the gradient field of the indicator function and the normal vector of the directed point set, and the Poisson equation is solved to obtain an approximate solution of the indicator function for the component-level point cloud subset; Triangular facets are extracted from the approximate solution of the indicator function using an isosurface extraction algorithm to generate an initial component triangular mesh; The initial triangular mesh is subjected to triangular facet homogenization processing based on mesh quality optimization to obtain the initial triangular mesh model.

5. The method for constructing and designing a 3D model of a robot joint module based on twin modeling according to claim 4, characterized in that, The process involves performing component assembly alignment processing based on boundary fit optimization, using the initial triangular mesh model of each rigid component and the global pose reference system in the deformation correction point cloud skeleton, to generate an overall assembled 3D model of the joint module, including: Extract the assembly docking surface position information between each rigid component from the deformation correction point cloud skeleton, and determine the target assembly pose of each rigid component in the global pose reference system based on the assembly docking surface position information. Based on the target assembly pose, the initial triangular mesh model of each rigid component is transformed to the corresponding spatial position under the global pose reference system to obtain the transformed component mesh model; Calculate the boundary fit deviation of the transformed component mesh model of adjacent rigid components at the assembly mating surface. The boundary fit deviation includes the boundary gap distance and the boundary overlap depth. Based on the boundary fit deviation, the transformed component mesh models of adjacent rigid components are jointly pose-fine-tuned until the boundary gap distance and boundary overlap depth between adjacent rigid components are both less than the preset tolerance. Then, all fine-tuned component mesh models are merged into the overall assembly 3D model.

6. The method for constructing and designing a 3D model of a robot joint module based on twin modeling according to claim 1, characterized in that, When performing edge repair processing based on neighborhood interpolation on the edge hole region left after the downsampling operation, the radial basis function interpolation method is adopted. The boundary points of the edge hole region are used as interpolation nodes, and the spatial coordinates of the newly added point cloud data points inside the edge hole region are obtained by solving the radial basis function weight coefficients.

7. The method for constructing and designing a 3D model of a robot joint module based on twin modeling according to claim 2, characterized in that, The criterion for determining the convergence of the distance residual is that the relative change between the root mean square value of the distance residual between the matching point pairs calculated in the current iteration and the root mean square value of the distance residual calculated in the previous iteration is less than a preset iteration termination threshold.

8. The method for constructing and designing a 3D model of a robot joint module based on twin modeling according to claim 1, characterized in that, When performing point cloud clustering segmentation based on region growth on the deformation-corrected point cloud skeleton, the point cloud data points within a preset neighborhood range on both sides of the theoretical segmentation boundary are used as the initial seed points for region growth, and the angle between the normal vectors of the point cloud data points and the spatial Euclidean distance are used as the joint growth criterion.

Images