A multi-view point cloud registration method

Abstract

This invention relates to the field of image recognition technology, specifically providing a multi-view point cloud registration method. In the coarse registration stage, the RANSAC algorithm is used to perform coarse registration of the source and target point clouds based on multi-dimensional features, obtaining a coarse registration transformation matrix. In the fine registration stage, the coarse registration transformation matrix is ​​used as the initial value, and point cloud registration is performed in at least two resolution spaces. Segmented iterative optimization is performed using an error loss function set for each resolution space. In the low-resolution space, an iterative nearest-point algorithm is used for rapid convergence, while in the remaining resolution spaces, a generalized iterative nearest-point algorithm is used for local geometric alignment optimization. After progressive optimization registration in multiple resolution spaces, the fine registration of the source and target point clouds is completed. This invention significantly improves the robustness of feature matching in low-overlap regions, balances registration speed and accuracy, and overcomes the limitations of traditional point cloud registration methods under low-overlap and non-ideal point cloud conditions.

Description

Technical Field

[0001] This invention belongs to the field of image recognition, and in particular relates to a multi-view point cloud registration method. Background Technology

[0002] In today's era, science and technology are developing rapidly, and their influence has permeated all walks of life, becoming a key force driving productivity improvement. For animal husbandry, scientific breeding is one of the core links in improving industrial efficiency, which is inseparable from the accurate and continuous monitoring of livestock growth. In the traditional model, livestock body size needs to be manually measured at specific intervals. This process has obvious drawbacks: on the one hand, it consumes a lot of manpower and time, seriously slowing down work efficiency; on the other hand, close contact with livestock, especially some wild or stressed animals, can easily bring safety risks to workers, making it quite dangerous.

[0003] In recent years, 3D reconstruction technology has developed rapidly, opening up a new path for addressing this dilemma faced by the livestock industry. Specifically, by strategically deploying three Time-of-Flight (TOF) cameras at fixed locations, they simultaneously capture 3D point cloud information of livestock from left, right, and top perspectives. Then, using 3D reconstruction technology, these fragmented point cloud data are integrated and stitched together to accurately construct a 3D model of the livestock. Based on the rich geometric features contained in this model, algorithms can be used to quickly and accurately calculate the livestock's body size information, achieving automated measurement of livestock body size, greatly reducing manpower and improving monitoring efficiency and accuracy.

[0004] Point cloud registration is a crucial step in 3D scene reconstruction, and its accuracy and efficiency directly affect the results of subsequent processing. While common multi-view point cloud registration systems are widely used in 3D reconstruction and target modeling, they still face several key challenges:

[0005] First, ICP (Iterative Closest Point) and its variants (such as Point-to-Plane ICP, GICP, etc.) are currently the most commonly used point cloud registration methods. However, ICP has a significant dependence on the initial pose: if the initial pose differences between the two sets of point clouds are too large or the overlapping area is too small, ICP can easily get trapped in a local optimum and fail to align correctly.

[0006] Second, point cloud feature extraction and matching are very susceptible to factors such as occlusion, sparse regions, and irregular surfaces under multi-view conditions, which can lead to unstable descriptors, decreased matching accuracy, false matching, or even failure to find enough effective corresponding points.

[0007] Third, RMSE is a commonly used indicator for evaluating registration results, mainly by calculating the average Euclidean distance between matched point pairs. However, it can only measure geometric errors and cannot reflect visual consistency; it is sensitive to outliers (mismatched point pairs) and lacks robustness.

[0008] Furthermore, current registration methods generally do not consider computational workload, have low computational efficiency, and are difficult to apply to real-time or large-scale data scenarios. As a result, many theoretical designs cannot be put into practical use and lack application value.

[0009] Therefore, how to improve the initial pose robustness of the registration algorithm under low overlap and non-ideal point cloud conditions, optimize the feature matching mechanism, and establish a multi-dimensional evaluation system suitable for live animal reconstruction, so as to achieve low-cost and high-precision three-dimensional reconstruction of bovine bodies, has become a key technical problem that urgently needs to be solved. Summary of the Invention

[0010] In view of this, the present invention aims to provide a multi-view point cloud registration method. It improves the traditional FPFH matching by using the RANSAC algorithm with multi-feature constraints, significantly enhances the robustness of feature matching in low-overlap regions, and achieves step-by-step alignment through multi-resolution segmented error optimization. This effectively balances the contradiction between registration speed and accuracy and solves the limitations of traditional point cloud registration methods under low-overlap and non-ideal point cloud conditions.

[0011] To achieve the above objectives, the technical solution created by this invention is implemented as follows:

[0012] This invention provides a multi-view point cloud registration method, comprising:

[0013] Multidimensional features of the source point cloud and the target point cloud are extracted, and the RANSAC algorithm is used to perform coarse registration of the source point cloud and the target point cloud based on the multidimensional features to obtain the coarse registration transformation matrix of the source point cloud and the target point cloud.

[0014] Using the coarse registration transformation matrix as the initial value, the source point cloud and the target point cloud are registered in at least two resolution spaces. The error loss function set for each resolution space is used for piecewise iterative optimization. In the low resolution space, the iterative nearest point algorithm is used for fast convergence, and in the other resolution spaces, the generalized iterative nearest point algorithm is used for local geometric alignment optimization. After progressive optimization registration in at least two resolution spaces, the fine registration of the source point cloud and the target point cloud is completed.

[0015] Preferably, the extraction of multidimensional features from the source point cloud and the target point cloud also includes:

[0016] Collect multi-view point clouds of live livestock and transform the multi-view point clouds to the same coordinate system;

[0017] Preprocessing is performed on multi-view point clouds, wherein the preprocessing includes at least one of point cloud filtering, point cloud downsampling, and point cloud smoothing;

[0018] The point cloud from any viewpoint is used as the target point cloud, and the point clouds from other viewpoints are used as the source point clouds.

[0019] Preferably, point cloud filtering employs statistical filtering and / or pass-through filtering.

[0020] Preferably, the point cloud smoothing uses the robust moving least squares method to remove noise and irregular point cloud data in multi-view point clouds.

[0021] Preferably, the multidimensional features include FPFH features, the local curvature of each point, and the normal vector of each point.

[0022] Preferably, the RANSAC algorithm is used to perform coarse registration of the source point cloud and the target point cloud based on multi-dimensional features to obtain the coarse registration transformation matrix of the source point cloud and the target point cloud, including:

[0023] FPFH feature registration is performed on the source point cloud and the target point cloud using KD-tree nearest neighbor search to generate an initial set of registered point pairs.

[0024] In the iterative framework of the RANSAC algorithm, a dual geometric constraint is applied to the randomly sampled initial registration point pairs. The dual geometric constraint includes: the local curvature difference of the initial registration point pair is less than a first threshold, and the angle between the normal vectors of the initial registration point pair is less than a second threshold.

[0025] After the iteration converges, the coarse registration transformation matrix of the source point cloud and the target point cloud is determined based on the inlier rate.

[0026] Preferably, at least two resolution spaces include a low-resolution space, a medium-resolution space, and a high-resolution space.

[0027] Preferably, the source point cloud and target point cloud are registered in at least two resolution spaces, and piecewise iterative optimization is performed using an error loss function set for each resolution space, including:

[0028] In the first registration stage, the source point cloud and the target point cloud are downsampled to a low-resolution space. The coarse registration transformation matrix is ​​used as the initial value. The source point cloud and the target point cloud after the first downsampling are registered by ICP to obtain the first fine matching transformation matrix.

[0029] In the second registration stage, the source point cloud and the target point cloud are downsampled to the medium resolution space. Using the first fine matching transformation matrix as the initial value, the source point cloud and the target point cloud after the second downsampling are registered by GICP to obtain the second fine matching transformation matrix.

[0030] In the third registration stage, the source point cloud and the target point cloud are downsampled to the medium resolution space for the second time, and then downsampled to the high resolution space for the third time. Using the second fine matching transformation matrix as the initial value, the source point cloud and the target point cloud after the third downsampling are registered through GICP to obtain the third fine matching transformation matrix.

[0031] Preferably, the error loss function set for the low-resolution space is:

[0032] ;

[0033] in, Represents the first point in the source point cloud. One point, Indicates the target point cloud and Corresponding points This represents a rigid body transformation used to align the source point cloud to the target point cloud. This indicates the calculation of Euclidean distance;

[0034] The error loss functions set for both medium resolution and high resolution spaces are:

[0035] ;

[0036] in, This represents the inverse of the covariance matrix, used to adjust the error weights in each direction.

[0037] Preferably, it also includes: evaluating the fine registration of the source point cloud and the target point cloud, wherein the evaluation index includes at least one of the following: the overlap rate of the source point cloud and the target point cloud, the root mean square error, and the minimum average distance of the surface, wherein the minimum average distance of the surface is calculated as follows:

[0038] The registered source point cloud and target point cloud are projected onto the same two-dimensional plane. This two-dimensional plane is then meshed. Spatial surface fitting is performed on the points belonging to the source point cloud and the points belonging to the target point cloud in each grid to obtain the local surface of the source point cloud and the local surface of the target point cloud. The minimum distance between the local surface of the source point cloud and the local surface of the target point cloud in each grid is determined, and the average of the minimum distances obtained from all grids is calculated to obtain the minimum average distance of the surface.

[0039] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0040] This invention addresses the problem of low overlap rate in live cow point clouds by replacing the PPFH matching method with the RANSAC algorithm and innovatively designing multi-feature constraints. During the registration process, not only are the FPFH feature histograms of key points calculated, but also multi-dimensional geometric features such as curvature and normal vector angles are integrated. The optimal matching point pairs are selected through RANSAC iteration, which significantly improves the robustness of feature matching in low-overlap areas. The innovative optimization of the coarse registration process greatly reduces the initial registration error and effectively avoids the mismatch problem of traditional methods on cow point clouds.

[0041] In the fine registration stage, this invention innovatively designs an optimization strategy based on a multi-resolution piecewise error function to address the non-rigid deformation and noise interference on the bovine body surface. Through multi-resolution stepwise registration, it first uses ICP for rapid convergence at low resolution, and then combines the probabilistic model of GICP to optimize local geometric alignment at high resolution. This effectively balances registration speed and accuracy, significantly reducing the final error of bovine body point cloud registration. At the same time, it greatly reduces the amount of registration computation and shortens the computation time compared to traditional registration methods.

[0042] Furthermore, this invention addresses the limitations of traditional RMSE registration evaluation metrics by proposing the use of the minimum average distance of the surface as an evaluation metric. By calculating the average projection distance from the registered point cloud to the target surface, it more comprehensively reflects geometric consistency and is particularly suitable for error evaluation in low-overlap regions. This metric achieves a more stable measurement of the registration quality of bovine point clouds. Attached Figure Description

[0043] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0044] Figure 1 This is a flowchart of a multi-view point cloud registration method provided according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of multi-view cow body point cloud transformation to the same coordinate system according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of left-side view point cloud data preprocessing provided according to an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the minimum distance between curved surfaces provided in an embodiment of the present invention;

[0048] Figure 5 This is a multi-view cow body point cloud registration image provided according to an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and do not constitute a limitation thereof. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0050] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined to form various implementations. Furthermore, the order of the steps or actions in the method description can be changed or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various orders in the specification and drawings are merely for the clear description of a particular embodiment and do not imply a mandatory order, unless otherwise stated that a particular order must be followed.

[0051] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0052] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0053] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0054] Please see Figure 1 In one embodiment of the present invention, a multi-view point cloud registration method is provided. Taking the scenario of low-cost 3D reconstruction of a live cow using a TOF camera as an example, the specific application process of this method is as follows:

[0055] S1: Extract multidimensional features from the source point cloud and the target point cloud, and use the RANSAC algorithm to perform coarse registration of the source point cloud and the target point cloud based on the multidimensional features to obtain the coarse registration transformation matrix of the source point cloud and the target point cloud.

[0056] S2: Using the coarse registration transformation matrix as the initial value, register the source point cloud and the target point cloud in at least two resolution spaces. Perform piecewise iterative optimization using the error loss function set for each resolution space. In the low resolution space, the iterative nearest point algorithm is used for fast convergence. In the other resolution spaces, the generalized iterative nearest point algorithm is used for local geometric alignment optimization. After progressive optimization registration in at least two resolution spaces, the fine registration of the source point cloud and the target point cloud is completed.

[0057] As an optional embodiment, before step S1, the method further includes: acquiring multi-view point clouds of the target, which is a live cow. Currently, due to cost constraints, the measurement of the body size of a live cow generally uses only three sensors to acquire images of the live cow from three different perspectives, thereby obtaining the point cloud of the third perspective, typically the left-view point cloud, the right-view point cloud, and the top-view point cloud. Since only three sensors are used, the overlapping area of ​​the point clouds between adjacent perspectives is limited. Traditional ICP algorithms, which are highly dependent on the initial pose and sufficient overlapping area, are prone to getting trapped in local optima under low overlap conditions, making it difficult to achieve global alignment. Moreover, the non-rigid motion characteristics and complex surface morphology of the live cow further exacerbate the occlusion and sparse distribution of the point cloud, causing the stability of matching methods based on local features (such as FPFH) to decrease and the mismatch rate to increase significantly.

[0058] like Figure 2As shown, before point cloud registration, the left-view point cloud, right-view point cloud, and top-view point cloud need to be transformed to the same coordinate system using an initial transformation matrix. Then, the multi-view point clouds are preprocessed, which includes at least one of point cloud filtering, point cloud downsampling, and point cloud smoothing. In this embodiment of the invention, as... Figure 3 As shown, after transforming the multi-view point cloud to the same coordinate system, point cloud filtering is first performed. Specifically, a strategy combining statistical filtering and pass-through filtering is adopted to remove noise and irrelevant points from the point cloud data.

[0059] Furthermore, a voxel meshing method is used to initially downsample the point cloud, simplifying the filtered point cloud with a larger voxel size, quickly reducing the number of points, and preserving the spatial distribution characteristics of the point cloud as much as possible while reducing the number of points.

[0060] Furthermore, Robust Moving Least Squares (RMLS) is employed to smooth the downsampled point cloud, removing noise and irregular point cloud data. RMLS not only effectively smooths noise and irregular structures on the point cloud surface, but also, due to its local fitting nature, preserves sharp geometric features and edges of key areas such as the spine and legs of a cow, avoiding the loss of structural features due to over-smoothing.

[0061] After multi-view point cloud preprocessing, any one of the left-view, right-view, or top-view point clouds is used as the target point cloud, and the remaining viewpoint point clouds are used as source point clouds. The target point cloud serves as the reference point cloud for registration, representing a desired target location. The source point cloud, also known as the original point cloud, is the point cloud data to be transformed and is the starting point in the registration process. The source point cloud is the point cloud data that is actively adjusted and aligned. The source point cloud needs to undergo transformation operations to achieve the best spatial matching with the target point cloud.

[0062] As an optional implementation, S1: Improve the traditional coarse registration stage by using the RANSAC algorithm based on multi-feature constraints to improve the traditional FPFH matching. It not only calculates the FPFH feature histogram of key points, but also integrates multi-dimensional geometric features such as local curvature and the angle between normal vectors.

[0063] Specifically, the multidimensional features include FPFH features, the local curvature of each point, and the normal vector of each point. This applies to the source point cloud. and target point cloud For each point in the source point cloud, calculate its local curvature and normal vector, since any point in the source point cloud... The calculation method of local curvature and normal vector is the same as that of any point in the target point cloud. The same applies below; only arbitrary points in the source point cloud are considered. Let's take the calculation method as an example. For any point... First, determine the distribution of its surrounding neighboring points, and the local covariance matrix. The formula for calculation is:

[0064] ;

[0065] in, Indicates the first The coordinate vector of a neighboring point in the neighborhood of a given point. denoted by , where represents the mean of all points in the neighborhood, and k represents the number of neighborhood points.

[0066] Further calculations yielded the local curvature. for:

[0067] ;

[0068] in, , , and These are the eigenvalues ​​of the local covariance matrix.

[0069] This further estimates the arbitrary point. The normal vector is:

[0070] .

[0071] in, express The three eigenvectors correspond to the three principal directions in the local space. The characteristic equation representing the covariance matrix, i.e., the calculation points normal vector Neighborhood covariance matrix The eigenvector corresponding to the smallest eigenvalue.

[0072] Furthermore, the FPFH algorithm is used to extract FPFH features from the point cloud based on the local coordinate system. FPFH features are 33-dimensional feature vectors:

[0073] .

[0074] in, Indicates the center point being described. express neighborhood points, SPFH This represents the feature histogram corresponding to each query point. This represents the number of points contained in the neighborhood of a given point.

[0075] After multi-feature extraction, FPFH feature registration is performed on the source point cloud and the target point cloud using KD-tree nearest neighbor search, that is, for each point in the source point cloud... Find the point in the target point cloud that is most similar to the FPFH feature. The initial set of registration point pairs generated by FPFH feature registration is:

[0076] ;

[0077] in, The source point cloud midpoint representing the most similar FPFH feature The initial registration point pair is formed by the target point cloud and the midpoint.

[0078] The RANSAC (Random Sample Consensus) algorithm is used to estimate the transformation between the source and target point clouds based on the aforementioned multidimensional features, achieving coarse registration. Within the iterative framework of the RANSAC algorithm, the following process is iteratively executed until convergence:

[0079] a. In each iteration of the RANSAC algorithm, four point pairs are sampled for iteration. After the RANSAC algorithm is executed, the initial corresponding point pairs are obtained. The geometric constraints are calculated for each pair of corresponding points, and then the registration point pairs that satisfy the double geometric constraints are selected.

[0080] The dual geometric constraints include: the local curvature difference of the initial registration point pair is less than a first threshold, and the angle between the normal vectors of the initial registration point pair is less than a second threshold.

[0081] The specific constraints to be applied are as follows:

[0082] ;

[0083] in, and These represent the local curvature difference between two points aligned to a set of registration points. Indicates the first threshold. and These represent the angle between the normal vectors of two points in a set of registration points. This indicates the second threshold.

[0084] b. Solve for the least-squares transformation matrix of the registration point pair using Singular Value Decomposition (SVD). for:

[0085] ;

[0086] c. Calculate the interior-point law when applying the current least squares transformation. for:

[0087] .

[0088] in, This indicates an indicator function that returns 1 if the condition within the parentheses is true, and 0 otherwise. This represents the distance threshold, and N represents the number of source point clouds.

[0089] Throughout the iteration process, the least-squares transformation matrix with the highest interior-point rate is consistently recorded. Once the iteration converges, the transformation matrix with the highest interior-point rate is selected as the final coarse registration transformation matrix T.

[0090] The criteria for determining the end of an iteration are: the iteration converges or the number of iterations reaches a preset upper limit.

[0091] As an optional embodiment, S2: The point pair search is accelerated by KD-Tree, and the matching is optimized by a multi-resolution piecewise error function. Specifically, the fine registration process is iteratively registered in at least two resolution spaces. In this embodiment of the invention, three registration stages are constructed, corresponding to the low-resolution space, the medium-resolution space, and the high-resolution space, respectively.

[0092] In the first registration stage, the source and target point clouds are downsampled to a low-resolution space, specifically downsampled to a size of 50mm. Using the coarse registration transformation matrix as the initial value, ICP is used to register the source and target point clouds after the first downsampling to obtain the first fine-matching transformation matrix. This process significantly reduces the amount of point cloud data and can quickly obtain a relatively coarse transformation matrix. While this process may contain significant errors, it can roughly align the source and target point clouds, preparing them for the next registration step.

[0093] In the second registration stage, the source and target point clouds are downsampled to a medium-resolution space, specifically downsampled to a size of 20mm. Using the first fine-matching transformation matrix as the initial value, GICP is used to register the second downsampled source and target point clouds, obtaining the second fine-matching transformation matrix. This process preserves more geometric structure information and uses the first fine-matching transformation matrix obtained in the first registration stage as the initial transformation value, while GICP is used to optimize local geometric matching, effectively improving registration accuracy.

[0094] In the third registration stage, the source and target point clouds are downsampled to a medium-resolution space in the second downsampling stage, and then downsampled to a high-resolution space in the third downsampling stage, specifically downsampling the source and target point clouds to a size of 5mm. Using the second fine-matching transformation matrix as the initial value, GICP is used to register the source and target point clouds after the third downsampling, obtaining the third fine-matching transformation matrix. This process fine-tunes the point cloud transformation, preserving more detailed information, and yields the final high-precision point cloud registration result, i.e., the third fine-matching transformation matrix.

[0095] The error loss function defined for low-resolution space is:

[0096] ;

[0097] The error loss functions set for both medium resolution and high resolution spaces are:

[0098] .

[0099] The specific execution flow of the above process is as follows:

[0100] (1) Accelerate the search for corresponding matching point pairs by using KD-Tree to speed up the search.

[0101] (2) Construct a piecewise error function in the multi-resolution space.

[0102] (3) Progressive registration optimization is performed in multi-resolution space through the above piecewise error function. When the resolution is low, the main body registration and alignment are completed quickly, and when the resolution is high, details are preserved and registration is performed.

[0103] Addressing the problems of traditional RMSE evaluation metrics:

[0104] The existing RMSE index, which is relied upon for registration quality assessment, is difficult to fully reflect the geometric consistency of the bovine 3D model. In particular, it is prone to cumulative errors in low-overlap areas. The insufficient computational efficiency of the registration algorithm also restricts its real-time performance in dynamic scenes.

[0105] This invention also optimizes the registration metrics for point clouds. Specific registration evaluation can employ one of the following: overlap rate between the source and target point clouds, root mean square error (RMSE), and minimum average distance across the surface; or a combination of these metrics using a weighted approach. The calculation methods for overlap rate and RMSE are both existing designs, briefly described below:

[0106] Overlap rate :

[0107] ;

[0108] in, Indicates the distance threshold. This is an indicator function; if the condition within the parentheses is met, the function value is 1; otherwise, it is 0. express Points in the source point cloud after point cloud transformation and registration If point At the threshold Find at least one inside If the points are overlapping, then the points are considered to be overlapping.

[0109] Root mean square error :

[0110] ;

[0111] Where N represents the number of matched point pairs, This represents the points in the source point cloud after point cloud transformation and registration. Points in the target point cloud The Euclidean distance between them.

[0112] Minimum average distance of curved surface The calculation method is as follows:

[0113] Registered source cloud The source point cloud is projected onto the same two-dimensional plane as the target point cloud Q. This two-dimensional plane is then meshed to obtain multiple meshes, and the source point cloud within each mesh is statistically analyzed. For points in the source point cloud Q and points in the target point cloud Q, spatial surface fitting is performed on the points belonging to the source point cloud and the points belonging to the target point cloud within each grid, respectively, to obtain the local surface of the source point cloud and the local surface of the target point cloud. The local surface is as follows:

[0114] ;

[0115] in, The spatial coordinates of a point in a point cloud. These are the coefficients obtained by solving the least squares method.

[0116] For each grid cell, determine the minimum distance between the local surfaces of the source point cloud and the target point cloud within that grid cell. :

[0117] ;

[0118] in, This represents the minimum value for each grid cell. For the local surface of the source point cloud, For the target point cloud local surface.

[0119] Further calculation of the average of the minimum distances obtained from all grids yields the minimum average distance of the surface. :

[0120] ;

[0121] Where N represents the number of grid cells that satisfy the fitting conditions.

[0122] The calculation of the minimum average distance of a curved surface is specifically implemented using a computer, such as... Figure 4 As shown.

[0123] To verify the effectiveness and progressiveness of the methods in the embodiments of the present invention, the present invention conducts experiments and comparative analyses, for example:

[0124] During the experiment, a Time-of-Flight (TOF) camera was used to simultaneously capture 3D point cloud information of a live cow from three perspectives: left, right, and top. The experimental platform consisted of an AMD Ryzen 5 5600H processor with Radeon Graphics 3.30 GHz, 16GB of RAM, and a 64-bit Windows 10 operating system. The development environment was PyCharm combined with the Open3D 0.19.0 point cloud resource library.

[0125] In the coarse registration stage, the multi-feature-based RANSAC algorithm is compared with PCA, FPFH, and FPFH+RANSAC coarse registration algorithms under the same development environment. In the fine registration stage, experiments are conducted using the point cloud fine registration algorithm under the framework of the proposed coarse registration, and compared with traditional fine registration algorithms. Due to the randomness and uncertainty of the RANSAC algorithm, the data measured by the RANSAC algorithm in this paper are the average values ​​of multiple runs. In the registration task of left-view point cloud and top-view point cloud, the left-view point cloud is selected within 100mm of the nearest point in the top-view point cloud, and the top-view point cloud is selected within 150mm of the nearest point in the left-view point cloud. In the registration task of right-view point cloud and top-view point cloud, the right-view point cloud is selected within 70mm of the nearest point in the top-view point cloud, and the top-view point cloud is selected within 100mm of the nearest point in the right-view point cloud. In measuring the evaluation metrics, points within 60mm and 70mm of each other were selected for the left-view and top-view point clouds, respectively, while points within 70mm and 100mm of each other were selected for the right-view and top-view point clouds. The overlap threshold was 40mm, and the surface size was set to 150mm × 150mm for the minimum average distance. Due to the inherent randomness and uncertainty of the RANSAC algorithm, the data measured by the RANSAC algorithm in the table below are the average values ​​from multiple runs.

[0126] After registration using the method proposed in this invention, the top-view point clouds, left-view point clouds, and right-view point clouds of a live cow can be registered. For specific results, please refer to [link / reference needed]. Figure 5 The registration effect is shown in three perspectives.

[0127] The measurement data are shown in Tables 1 and 2.

[0128] Table 1. Accuracy and time of different coarse registration algorithms on data acquisition.

[0129]

[0130] Ours represents experimental data using the method proposed in the embodiments of this invention.

[0131] Table 2. Accuracy and time of different registration algorithms in data acquisition.

[0132]

[0133] Experimental data show that the present invention performs optimally in coarse point cloud registration. In registration from the left to the top view point cloud, the algorithm presented in this paper achieves the highest overlap rate of 88.4%, with the best root mean square error (RMSE) of 10.6 mm and surface distance of 1.8 mm. In registration from the right to the top view point cloud, the overlap rate of 80.4% is second only to PCA, but the accuracy is superior. Although the computation time is slightly longer than some of the comparison methods, the accuracy improvement is significant, achieving a good balance between accuracy and efficiency, thus validating the effectiveness of the algorithm.

[0134] The Ours algorithm proposed in this paper demonstrates significant advantages in the fine registration stage. Although its overlap rate is lower than that of the ICP algorithm, it outperforms the key accuracy metrics. Furthermore, the high overlap rate of the ICP algorithm stems from its forced matching of closest points. In terms of accuracy, the root mean square error is reduced by 15-25%, and the surface distance is improved by 44%. Simultaneously, the algorithm's efficiency is significantly improved, taking only about one-quarter the time of ICP. This performance advantage arises from the good initial pose provided by coarse registration, avoiding the problem of the ICP algorithm easily getting trapped in local optima. Overall, the Ours algorithm achieves a better balance between accuracy and efficiency, validating the effectiveness of the joint coarse and fine registration framework.

[0135] The method of this invention has been practically deployed and applied. During the application process, a point cloud registration visualization design was implemented. The point cloud registration interface is a comprehensive visualization platform integrating multiple registration algorithms. Its development process closely combines relevant technologies and practical experience in the field of point cloud processing. The interface is built on the PyQt5 framework, uses Open3D for point cloud visualization rendering, and utilizes win32gui to achieve seamless embedding of Qt windows and Open3D visualization windows. The entire interface adopts a modular design approach, encapsulating various functional components such as preprocessing and registration algorithms into independent modules, ensuring code maintainability and facilitating subsequent functional expansion. The final visualized registration platform has a user-friendly interface and complete functions, providing a tool platform for point cloud registration research.

[0136] In summary, the above description is merely a preferred embodiment of this specification and is not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0137] The systems, apparatuses, modules, or units described in one or more of the above embodiments may be implemented by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer. Specifically, a computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0138] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0139] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A multi-view point cloud registration method, characterized in that, The method comprises the following steps: extracting multi-dimensional features of the source point cloud and the target point cloud, and performing coarse registration on the source point cloud and the target point cloud based on the multi-dimensional features by using a RANSAC algorithm to obtain a coarse registration transformation matrix of the source point cloud and the target point cloud; the multi-dimensional features include FPFH features, local curvatures of each point, and normal vectors of each point; taking the coarse registration transformation matrix as an initial value, performing registration of the source point cloud and the target point cloud in at least two resolution spaces, and performing segmented iterative optimization by using an error loss function set in each resolution space, wherein fast convergence is performed in a low-resolution space by using an iterative closest point algorithm, and local geometric alignment optimization is performed in the remaining resolution spaces by using a generalized iterative closest point algorithm, and the fine registration of the source point cloud and the target point cloud is completed after gradual optimization registration in the at least two resolution spaces; the at least two resolution spaces include a low-resolution space, a medium-resolution space, and a high-resolution space, and the registration of the source point cloud and the target point cloud in the at least two resolution spaces and the segmented iterative optimization by using the error loss function set in each resolution space comprise: constructing a first registration stage, first down-sampling the source point cloud and the target point cloud to a low-resolution space, taking the coarse registration transformation matrix as an initial value, and performing registration operation on the first down-sampled source point cloud and target point cloud by using ICP to obtain a first fine matching transformation matrix; constructing a second registration stage, second down-sampling the source point cloud and the target point cloud to a medium-resolution space, taking the first fine matching transformation matrix as an initial value, and performing registration operation on the second down-sampled source point cloud and target point cloud by using GICP to obtain a second fine matching transformation matrix; constructing a third registration stage, second down-sampling the source point cloud and the target point cloud to a medium-resolution space, third down-sampling the source point cloud and the target point cloud to a high-resolution space, taking the second fine matching transformation matrix as an initial value, and performing registration operation on the third down-sampled source point cloud and target point cloud by using GICP to obtain a third fine matching transformation matrix.

2. The multi-view point cloud registration method of claim 1, wherein, The method further comprises the following steps before the extraction of the multi-dimensional features of the source point cloud and the target point cloud: collecting multi-view point clouds of a target, and transforming the multi-view point clouds to the same coordinate system; performing preprocessing on the multi-view point clouds, wherein the preprocessing at least includes any one of point cloud filtering, point cloud down-sampling, and point cloud smoothing; and taking any view point cloud as the target point cloud and the remaining view point clouds as the source point cloud.

3. The multi-view point cloud registration method of claim 2, wherein, The point cloud filtering adopts statistical filtering and / or straight-through filtering.

4. The multi-view point cloud registration method of claim 2, wherein, The point cloud smoothing adopts a robust moving least squares method for removing noise and irregular point cloud data in the multi-view point clouds.

5. The multi-view point cloud registration method of claim 4, wherein, The coarse registration of the source point cloud and the target point cloud based on the multi-dimensional features by using the RANSAC algorithm comprises the following steps: performing FPFH feature registration on the source point cloud and the target point cloud by using KD tree nearest neighbor search to generate an initial registration point pair set; In the iteration framework of the RANSAC algorithm, double geometric constraints are imposed on the randomly sampled initial registration point pair, including that the local curvature difference of the initial registration point pair is less than a first threshold, and the normal vector angle of the initial registration point pair is less than a second threshold. After iteration convergence, a coarse registration transformation matrix of the source point cloud and the target point cloud is determined according to an inlier rate.

6. The multi-view point cloud registration method of claim 1, wherein, The error loss function of the low resolution spatial setting Is: ； in, Represents the first point in the source point cloud. One point, Indicates the target point cloud and Corresponding points This represents a rigid body transformation used to align the source point cloud to the target point cloud. This indicates the calculation of Euclidean distance; Error loss function for the medium resolution space and the high resolution space settings All: ； wherein, denotes the inverse of the covariance matrix, used to adjust the error weight for each direction.

7. The multi-view point cloud registration method of claim 1, wherein, Further comprising: An evaluation is performed on the fine registration of the source point cloud and the target point cloud, and the evaluation index at least includes one of an overlap rate, a root mean square error and a minimum average distance of a surface of the source point cloud and the target point cloud, wherein the minimum average distance of the surface is calculated in the following manner: The registered source point cloud and the target point cloud are projected in the same two-dimensional plane, the two-dimensional plane is gridded, and the points belonging to the source point cloud and the points belonging to the target point cloud in each grid are respectively fitted with a spatial surface to obtain a local surface of the source point cloud and a local surface of the target point cloud, the minimum distance between the local surface of the source point cloud and the local surface of the target point cloud in each grid is determined, and the average of the minimum distances obtained by all grids is calculated to obtain the minimum average distance of the surface.

Images