A point cloud registration method based on double downsampling and graph optimization
Through dual downsampling and graph optimization technology, combined with iterative distance-aware similarity matrix convolution and fast graph convolution neural network, the trade-offs of efficiency and accuracy in point cloud registration are solved, and efficient and accurate point cloud registration is achieved. .
Patent Information
- Application Number
- CN202411311715.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-09-20
AI Technical Summary
The existing point cloud registration method is low in robustness and accuracy, low computational efficiency, and has a large dependence on initial alignment results, making it difficult to perform between accuracy and efficiency. Effective trade-offs.
Using a method based on dual downsampling and graph optimization, point cloud data dimensionality reduction is carried out through the combination of rough sampling and brilliant sampling, and deep features are extracted using iterative distance-aware similarity matrix convolution and fast graph convolution neural network. Factor graph optimization algorithm for point cloud registration.
The computing efficiency and accuracy of point cloud registration are significantly improved, point cloud details are maintained, the global consistency and accuracy of registration results are improved, and the computational complexity is reduced.
Smart Images

Figure CN118840399B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of point cloud registration, and in particular to a point cloud registration method based on double down sampling and graph optimization. Background Art
[0002] Point cloud registration is a core technology in the field of computer vision and 3D modeling, which involves aligning multiple point cloud data sets into a unified coordinate system to form a complete 3D scene. With the widespread application of devices such as LiDAR, RGB-D cameras, and structured light sensors, it is becoming easier to obtain high-quality point cloud data. However, in practical applications, point cloud data is often affected by factors such as noise and uneven data density, which makes the accuracy and efficiency of point cloud registration an important research topic.
[0003] Existing point cloud registration methods mainly include feature-based matching algorithms, iterative closest point algorithms (ICP) and graph optimization methods. Feature-based matching algorithms rely on extracting feature points (such as normals, curvatures, etc.) in point clouds and performing registration by matching feature points. However, these methods may exhibit poor robustness and low accuracy when dealing with high-noise data or when point cloud density varies greatly. The ICP algorithm performs registration by iteratively optimizing the closest point matching relationship between the source point cloud and the target point cloud. Although it performs well in terms of accuracy, it has low computational efficiency and is highly dependent on the initial alignment results.
[0004] In the process of point cloud registration, it is usually necessary to strike a balance between computational efficiency and accuracy. Traditional downsampling methods such as voxel downsampling can effectively reduce the amount of point cloud data and improve the computational speed, but may lead to the loss of point cloud details, thus affecting the final registration accuracy. In order to make up for the shortcomings of voxel downsampling in detail preservation, some fine sampling techniques based on normal angles have been proposed in recent years, which can better preserve the geometric information of point clouds. However, these methods usually have high computational complexity and affect the overall efficiency.
[0005] In order to solve the problem of low computational efficiency and high dependence on initial alignment results. In recent years, point cloud registration methods based on deep learning have gradually emerged. These methods build deep neural networks to automatically learn feature representations from data, thereby achieving more robust and efficient point cloud registration. For example, the PointNet and PointNet++ models achieve end-to-end feature learning and registration by directly processing point cloud data. In addition, the IDAM (iterative distance-awaresimilarity matrix convolution) algorithm achieves fast initial registration through affine transformation. The IDAM module uses a distance-aware mechanism to update the similarity matrix of the point cloud in each iteration, thereby gradually optimizing the alignment effect of the point cloud. FastGCN (Fast Graph Convolutional Networks) uses graph convolutional networks to extract deeper features.
[0006] Graph optimization technology has been gradually applied in point cloud registration. By constructing a factor graph model, it combines the information of multiple registration steps for global optimization. Although graph optimization can effectively improve global consistency, its computational complexity and sensitivity to initial estimates remain challenges in practical applications. Summary of the invention
[0007] In order to solve the above-mentioned technical problems, the present invention provides a point cloud registration method based on double downsampling and graph optimization. By combining the double downsampling technology of coarse sampling and fine sampling, the computational efficiency is significantly improved while maintaining the details of the point cloud.
[0008] In order to solve the above technical problems, the technical solution proposed by the present invention is:
[0009] A point cloud registration method based on double downsampling and graph optimization, the specific steps are as follows:
[0010] Step S1, obtaining point cloud data as a source point cloud, each point in the point cloud data having coordinates in a three-dimensional space;
[0011] Step S2, performing double downsampling on the source point cloud and the target point cloud, the double downsampling includes coarse sampling and fine sampling, the coarse sampling adopts voxel downsampling, and the fine sampling adopts downsampling based on normal angle;
[0012] Step S3, using iterative distance-aware similarity matrix convolution for coarse registration, adjusting the position and direction of the source point cloud set in an iterative manner to achieve preliminary global alignment;
[0013] Step S4, using a fast graph convolutional neural network to extract deep features of the point cloud;
[0014] Step S5, applying an iterative closest point algorithm to the features extracted by the fast graph convolutional neural network;
[0015] Step S6, through factor graph optimization, the point cloud registration problem is modeled as a factor graph, the optimization objective function is defined, the error term in the factor graph is minimized, and the factor graph optimization algorithm is used to iteratively solve until the error is lower than a preset threshold.
[0016] As a further improvement of the above technical solution:
[0017] Preferably, in step S1, the point cloud data is first denoised, a neighborhood radius and a threshold are set, and the distance between each point and the neighborhood points is analyzed. If the number of points in the neighborhood of a point is less than the threshold, the point is a noise point.
[0018] Preferably, in step S2, the coarse sampling method is:
[0019] The side length of the voxel is set to a, and the coordinates of the i-th point in the point cloud are , calculate the index of the voxel , the formula is:
[0020] (1)
[0021] in, represents the voxel index of the i-th point in the point cloud in the x direction. represents the voxel index of the i-th point in the point cloud in the y direction. represents the voxel index of the i-th point in the point cloud in the z direction; x i is the coordinate of the i-th point on the x-axis, y i is the coordinate of the i-th point on the y-axis, z i is the coordinate of the i-th point on the z-axis; a is the side length of the voxel in the coarse sampling;
[0022] Assume that there are N points in a voxel. Voxel downsampling is to create a three-dimensional voxel grid based on the input point cloud data. Each voxel grid is represented by the centroid of all points in the grid. The calculation expression is:
[0023] (2)
[0024] Among them, x i, y i, z i are the three coordinates of the i-th point; N is the point in the voxel grid, is the center of mass of the voxel.
[0025] Preferably, in step S2, the downsampling based on the normal angle is to calculate the average value of the angles between a point and the normal vectors of all other points in the neighborhood, and if the average value of the angles exceeds a preset threshold, the point is regarded as a feature point and retained;
[0026] The retained feature points are used as selection points, and the normal vector η of the selection points and the normal vectors of other points in the domain are calculated. , the angle average formula is:
[0027] (3)
[0028] Among them, w represents the number of all points in the neighborhood, Represents the normal vector of the i-th point;
[0029] If the average of the angles between the normal vector of the selected point and the normal vectors of other points in the neighborhood exceeds the set threshold, the point represents a key feature in the point cloud.
[0030] Preferably, the iteration method of step S3 is:
[0031] S3-1, set Geometric features and Geometric features , the dimensions of both features are K, where P cs is the source point cloud set, Q ct For the target point cloud set, the distance enhanced feature tensor is formed at the nth iteration as follows:
[0032] (4)
[0033] in, Represents concatenation, which means that in the similarity matrix convolution, the geometric feature vectors of the source point cloud and the target point cloud are concatenated. and And Euclidean features are concatenated into a (2K+4)-dimensional vector as the input feature; · refers to four quantities: , , , ; T (n) The (2K+4)-dimensional vector of the (i, j) position is the point pair (p i ,q j ) is a combination of geometric features and Euclidean features; the four-dimensional Euclidean features include p i and q j The distance between them, and the distance from q j Point to p i The unit vector of (n)Each enhanced feature vector in encodes the joint information of the local shape and current relative position of two points;
[0034] S3-2, distance-expanded feature tensor T (n) is a 2D image with (2K+4) channels, at T (n) A 1×1 2D convolution is applied to the last layer, and a single-channel image of the same spatial size is output at the last layer, that is, a multi-layer perceptron is applied to the enhanced feature vector at each position; then the Softmax function is applied to each row of the single-channel image to obtain the similarity matrix, denoted as S (n) ; S (n) Each row of defines a normalized probability distribution;
[0035] S3-3, take S (n) The argmax of each row is obtained from the similarity matrix S (n) Find the point index in the target point cloud corresponding to the maximum similarity in each row of ,in is the source point cloud p i The target point cloud with the highest similarity score uses these corresponding point pairs to solve the following optimization problem to find the estimated rigid body transformation R (n) , t (n) :
[0036] (5)
[0037] Among them, R is the rotation matrix and t is the translation vector;
[0038] Calculated using the orthogonal Procrstes algorithm of singular value decomposition; before entering the next iteration, use R (n) and t (n) Transform the source point cloud to a new location.
[0039] Preferably, in step S4, the fast graph convolutional neural network includes multiple convolutional layers, which are:
[0040] (1) Input layer
[0041] Function: The input layer accepts node features of point cloud data;
[0042] Input: feature matrix of point cloud node;
[0043] Output: The input features are directly passed to the next layer;
[0044] (2) First layer of graph convolution
[0045] Function: The first layer of graph convolution is used to aggregate the feature vector of a node with the features of its neighboring nodes to extract local structural information;
[0046] Convolution operation: implemented through GCNConv, calculating the weighted feature sum of each node and its neighboring nodes;
[0047] Activation function: Use ReLU activation function;
[0048] Output: feature matrix after convolution and activation;
[0049] (3) Hidden layer
[0050] Function: The hidden layer is a stack of multiple graph convolutional layers, which is used to extract deeper features layer by layer;
[0051] Convolution operation: Each hidden layer continues to perform graph convolution operations to further aggregate features;
[0052] Activation function: Each hidden layer usually applies an activation function;
[0053] Output: The features output by the hidden layer gradually incorporate more global information;
[0054] (4) Output layer
[0055] Function: The output layer processes the final features and generates the final feature representation for subsequent point cloud registration tasks;
[0056] Convolution operation: The convolution of the last layer is usually used to reduce the feature dimension to the target dimension;
[0057] Output: Deep feature representation after the last layer of convolution.
[0058] Preferably, in step S5, specifically:
[0059] S5-1, first, the deep features in step S4 are used to form a source feature point cloud and a target feature point cloud, and the source feature point cloud is represented as P feature , the target feature point cloud is represented by Q feature ; Next, for each source feature point cloud feature point Find the nearest feature point in the target feature point cloud , and based on the set of matching point pairs Calculate the rigid body transformation matrix H to minimize the matching error between the source feature point cloud and the target feature point cloud;
[0060] S5-2, use the transformation matrix H to transform the feature points of the source feature point cloud to obtain the transformed point cloud; repeat the process of nearest point matching and transformation matrix calculation until the matching error converges to a preset threshold or reaches the maximum number of iterations.
[0061] Preferably, the transformation matrix H is determined by solving an optimization problem:
[0062] (7)
[0063] Among them, H is the transformation matrix, and the form of the transformation matrix H is , where R is the rotation matrix and t is the translation vector.
[0064] Preferably, step S6 includes the following contents:
[0065] S6-1, the point cloud registration problem is modeled as a factor graph. The factor graph consists of variable nodes and factor nodes, where each variable node represents the position and posture of the point cloud, and each factor node represents the constraint relationship between point clouds; the objective function is used to quantify the error of each factor node in the factor graph, and the optimization objective function is defined as the sum of the squares of the errors of all factor nodes:
[0066] (8)
[0067] Among them, r means that the source point cloud and the target point cloud are assumed to involve r point pairs in the registration process. is the error of the i-th factor node, h is the variable node to be optimized, including the position and posture of the point cloud; specifically, the error It can be expressed as:
[0068] (9)
[0069] Among them, R and t are the rotation matrix and translation vector that need to be optimized, is the feature point in the source feature point cloud, is the feature point in the target point cloud;
[0070] S6-2, using the error algorithm of S6-1 to optimize and adjust the values of the variable nodes to minimize the error term in the objective function; including calculating the residual of each factor node, and updating the position and posture of the target point cloud according to the residual, until the value of the objective function is lower than a preset threshold or the maximum number of iterations is reached;
[0071] The nonlinear optimization algorithm Gauss-Newton is used to minimize the error term and solve the rotation matrix R and the translation vector t;
[0072] S6-3, the Gauss-Newton algorithm updates the value of each variable node through iterative optimization. The optimization process is to calculate the Jacobian matrix to update the rotation matrix R and the translation vector t. After multiple iterative optimizations, the final point cloud registration result is obtained, that is, the globally optimal point cloud position and posture.
[0073] Compared with the prior art, the point cloud registration method based on double downsampling and graph optimization provided by the present invention has the following advantages:
[0074] The point cloud registration method based on double downsampling and graph optimization of the present invention aims to improve the computational efficiency and accuracy of point cloud registration at the same time. The method significantly improves the computational efficiency while maintaining the details of the point cloud by combining the double downsampling technology of coarse sampling and fine sampling; coarse sampling uses voxel downsampling to quickly reduce the number of point clouds, while fine sampling is based on the normal angle to better preserve the geometric features of the point cloud. At the same time, graph optimization technology is used for global optimization. The core of graph optimization is to model the point cloud registration problem as a graph model, in which nodes represent the position and posture of point clouds, and edges represent the constraint relationship between point clouds. The values of these nodes are adjusted by optimizing the objective function to achieve higher registration accuracy, further improving the global consistency and accuracy of the registration results. While reducing the computational complexity, the present invention effectively improves the accuracy of point cloud registration, providing an efficient and accurate solution for point cloud data processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] Figure 1 It is a simplified schematic diagram of laser radar collecting point cloud data in the present invention.
[0076] Figure 2 It is a structural diagram of the feature extraction module based on IDAM in the present invention.
[0077] Figure 3 It is the overall architecture diagram of the point cloud registration framework based on IDAM in the present invention.
[0078] Figure 4 It is a flowchart of the FastGCN point cloud feature in the present invention.
[0079] Figure 5 It is the basic framework diagram of the method of the present invention. DETAILED DESCRIPTION
[0080] The specific embodiments of the present invention are described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0081] like Figures 1 to 5As shown in the figure, the point cloud registration method based on double downsampling and graph optimization of the present invention first performs double downsampling on the source point cloud and the target point cloud to achieve effective dimensionality reduction of the point cloud data; then, the false positive corresponding pairs are reduced by the iterative distance-aware similarity matrix convolution (IDAM) module to perform preliminary global alignment; in the fine registration process, the coarsely registered data and the double downsampled data are combined, the feature information at different levels is fully utilized, the fast graph convolutional neural network (FastGCN) is used to extract deep features, and the iterative closest point algorithm (ICP) is applied to the deep features, and the precise registration of the point cloud is further improved through iterative optimization; finally, the point cloud registration problem is modeled as a factor graph through factor graph optimization, the optimization objective function is defined, the error term in the factor graph is minimized, and the factor graph optimization algorithm is used to iteratively solve until the error is lower than the preset threshold, thereby improving the global consistency and accuracy of the registration.
[0082] The point cloud registration method based on double downsampling and graph optimization of the present invention has the following specific steps:
[0083] Step S1, obtaining point cloud data through laser radar.
[0084] The sensor scans the object or environment and generates point cloud data, which is a collection of points, each with coordinates in three-dimensional space.
[0085] Furthermore, point cloud data is acquired by a LiDAR sensor, and the data includes the coordinates (x, y, z) of each point in three-dimensional space (where x represents the coordinate on the x-axis, y represents the coordinate on the y-axis, and z represents the coordinate on the z-axis), as well as the reflection intensity information of the point. The LiDAR device emits laser pulses and receives the reflected laser signals. The laser pulses will reflect back after encountering an object, and the device records the time of these reflected signals and calculates the distance from the laser pulse to the target. The LiDAR device continuously emits laser pulses and receives reflected signals in different directions and angles to obtain the three-dimensional coordinates of different points. The collection of these three-dimensional coordinate points is the point cloud data. The generated point cloud data is usually represented as a collection of points, each of which contains its coordinates (x, y, z) in three-dimensional space, as well as other information such as reflection intensity, color, etc.
[0086] The data resolution is 1 cm, and the point cloud data is generally high-density, suitable for detailed environment modeling. The point cloud file is stored in PLY format, which contains the spatial coordinates and intensity value of each point.
[0087] In addition, the data needs to be initially denoised during acquisition to reduce the impact of measurement errors on subsequent analysis. The denoising process is a statistical filter that determines whether a point is a noise point by analyzing the distance between each point and its neighborhood points. A neighborhood radius and a threshold are set. If the number of points in the neighborhood of a point is less than the threshold, then the point is considered a noise point and is removed.
[0088] In this embodiment, a point cloud data set is obtained by a laser radar sensor, and the source point cloud set (The subscript cs represents the source coordinate system, 1, 2, ..., n represents the number of each point in the source point cloud), where each point (Here i is the assumed i-th point, where x i It represents the coordinate of the i-th point on the x-axis, y i represents the coordinate of the i-th point on the y-axis, z i represents the coordinate of the i-th point on the z-axis) is a 3D coordinate in the source point cloud, and the target point cloud set (ct represents the target coordinate system, 1, 2, ..., m represents the number of each point in the target point cloud), the point cloud dataset used as the reference for registration, where each point (here j is the assumed jth point) is a 3D coordinate in the target point cloud.
[0089] The source point cloud is used as a reference or starting point cloud dataset in point cloud registration. It contains a set of points in three-dimensional space acquired from a specific perspective or position. It is usually the object to be "moved" or "transformed" and aligned with the target point cloud through a series of rotations and translations. The target point cloud is the "fixed" point cloud dataset in the registration task and is usually unchanged. The target point cloud serves as a benchmark for alignment during the registration process and is used to guide the transformation process of the source point cloud. By comparing the differences between the source point cloud and the target point cloud, the algorithm calculates how to adjust the source point cloud so that the source point cloud is as consistent as possible with the target point cloud.
[0090] Step S2, double downsampling the source point cloud and the target point cloud, the double downsampling includes coarse sampling and fine sampling. The coarse sampling adopts the voxel downsampling method, and the fine sampling adopts the downsampling based on the normal angle.
[0091] S2-1, perform coarse sampling.
[0092] Coarse sampling uses the voxel downsampling method. The side length of the voxel is set to a, and for each point in the point cloud (Assuming it is the coordinate of the i-th point), calculate the index of the voxel to which it belongs , the formula is:
[0093] (1)
[0094] in, represents the voxel index of the i-th point in the point cloud in the x direction. represents the voxel index of the i-th point in the point cloud in the y direction. represents the voxel index of the i-th point in the point cloud in the z direction; x i is the coordinate of the ith point on the x-axis, y i is the coordinate of the ith point on the y-axis, z i is the coordinate of the ith point on the z-axis; a is the side length of the voxel in the coarse sampling.
[0095] In this embodiment, the target point cloud and the source point cloud are processed in the same manner.
[0096] All points belonging to the same voxel are grouped together. The same voxel means that their indexes are the same. For each point in a voxel, calculate its mean as the representative point of the voxel, that is, the centroid. Suppose there are N points in a voxel, then the centroid P of the voxel is centroid (The subscript centroid means center).
[0097] Voxel downsampling creates a three-dimensional voxel grid based on the input point cloud data. Each voxel grid is represented by the centroid of all points in the grid. The calculation expression is:
[0098] (2)
[0099] Among them, x i, y i, z i are the three coordinates of the i-th point; N is the point in the voxel grid.
[0100] S2-2, perform fine sampling.
[0101] Fine sampling is based on the downsampling method of normal angle. The downsampling method based on normal angle is to calculate the average angle between a point and the normal vectors of all other points in its neighborhood. If the average value exceeds the preset threshold, the point is regarded as a feature point and retained.
[0102] Although voxel downsampling does not destroy the data structure of the point cloud, if the voxel grid is too large, the features of the point cloud will be lost. To this end, in this embodiment, a downsampling method based on the normal angle of the neighborhood point cloud is used to finely sample the coarse sampling points, further reducing the number of point clouds while retaining the original features of the point clouds.
[0103] In this embodiment, the retained feature point is used as the selected point, and the normal vector η of the selected point and the normal vectors of other points in its area are calculated. , the formula for the average angle θ is:
[0104] (3)
[0105] According to the preset threshold θ threshold , select the point with the larger normal angle with its neighboring points. "Large" means that the average angle between the normal vector of the selected point and the normal vectors of other points in its neighborhood exceeds the set threshold. These points usually represent the key features in the point cloud.
[0106] In this embodiment, key features refer to points in the point cloud that can significantly describe the change in geometric shape. They are usually located at edges, corners, protrusions or depressions. A point can represent a key feature because it plays an important descriptive role in the local geometric structure, especially through the difference between its normal vector and the normal vector of the neighboring points. A point can represent different key features because it can have multiple geometric features at the same time, for example, it can be located at both the edge and the corner.
[0107] The coarse sampling and fine sampling of the source point cloud and the target point cloud adopt the same method.
[0108] In step S3, iterative distance-aware similarity matrix convolution (IDAM) is used for coarse registration, and the position and orientation of the source point cloud set are adjusted in an iterative manner to achieve preliminary global alignment.
[0109] Specifically:
[0110] S3-1, assuming that there is a Geometric features and Geometric features , the dimensions of these two features are both K. The distance-enhanced feature tensor is formed at the nth iteration as follows: (4)
[0111] in, Represents concatenation, which means that in the similarity matrix convolution, the geometric feature vectors of the source point cloud and the target point cloud are concatenated. and And Euclidean features are concatenated into a (2K+4)-dimensional vector as the input feature; · refers to four quantities: , , , ; T (n) The (2K+4)-dimensional vector of the (i, j) position is the point pair (p i ,q j ) is a combination of geometric features and Euclidean features; the four-dimensional Euclidean features include p i and q j The distance between them, and the distance from q j Point to p iThe unit vector of (n) Each augmented feature vector in encodes the joint information of the local shapes of two points and their current relative positions; this is useful for computing the similarity score at each iteration.
[0112] S3-2, distance-expanded feature tensor T (n) is a 2D image with (2K+4) channels. In order to extract the similarity score of each point pair, (n) A series of 1×1 2D convolutions are applied to the image, and the last layer outputs a single-channel image of the same spatial size. This is equivalent to applying a multi-layer perceptron to the enhanced feature vector at each position. The Softmax function is then applied to each row of the single-channel image to obtain the similarity matrix, denoted as S (n), S (n) Each row of defines a normalized probability distribution. Indicates p i and q j The “similarity score” of the two pairs (the higher the more similar). It can also be interpreted as q j Yes i The similarity score is a mode used to evaluate the similarity between point clouds by comparing the distance and angle information between them.
[0113] S3-3, in order to find the corresponding pair, take S (n) The argmax of each row (argmax is used to obtain the similarity matrix S (n) The result is a set of corresponding pairs. ( Here it refers to the source point cloud p i The target point cloud with the highest similarity score) uses these corresponding point pairs to solve the following optimization problem to find the estimated rigid body transformation R (n) , t (n) .
[0114] (5)
[0115] This is a classic absolute orientation problem, which can be solved efficiently using the Orthogonal Procrstes algorithm with singular value decomposition (SVD). Then, before entering the next iteration, use R (n) , t (n) Transform the source point cloud to a new location.
[0116] Going a step further, for example, for a rotation matrix, we want the point p i With target point The difference is the smallest, take the difference between them, find the second norm, and then sum the squares of the differences at each point. It is expected to be minimized:
[0117] make
[0118]
[0119] again
[0120]
[0121] Since the result of the above formula is actually a scalar, the transpose of the whole formula is the same. The transpose of a scalar is the scalar itself, that is:
[0122]
[0123] The first two terms are constants, so we need to i and The goal is to minimize , so let the last item be the largest. That is, maximize the function F:
[0124]
[0125] The C matrix is actually the result of summing the covariance matrices before and after the coordinate point.
[0126] Perform SVD decomposition on the C matrix and get
[0127] make X=VU T ,So , which is a positive definite matrix.
[0128] Furthermore, for any positive definite matrix AA T and any orthogonal matrix B, have .
[0129] According to this theorem, we can know that for a positive definite symmetric matrix XC , and any 3x3 orthogonal matrix B, we have:
[0130]
[0131] This means that the matrix XC , after rotating it once, the trace of this matrix will be smaller than the original matrix anyway XC Trace of.
[0132] Therefore, for , the goal is to find the rotation matrix R to maximize F. This special Xcan maximize this trace because XC No matter how you transform it linearly afterwards, its trace will never exceed XC So actually this X That’s the R you’re looking for.
[0133] After obtaining the rotation matrix R, we can get the translation vector t, which is the relative displacement between the target point cloud and the rotated source point cloud.
[0134] In step S4, the fast graph convolutional neural network (FastGCN) is used to extract deep features of the point cloud.
[0135] Traditional graph convolutional networks (GCNs) perform convolution operations through adjacency matrices and feature matrices. However, as the size of the graph increases, the computational complexity and memory consumption increase rapidly, limiting the application of GCNs on large-scale graph data. FastGCN in this embodiment proposes a hierarchical sampling method that effectively reduces computational complexity and memory consumption.
[0136] Specifically:
[0137] The propagation formula of the GCN layer is expressed as:
[0138] (6)
[0139] In order to perform convolution operations, the adjacency matrix needs to be normalized. The degree matrix of the adjacency matrix is , is a modified version of the adjacency matrix B, in which self-connections are usually added (i.e., a diagonal matrix I such that each node is connected to itself), expressed as . yes The inverse matrix of the square root of is used for normalization to ensure that the influence of node degree is balanced during information transmission. (k) represents the node feature matrix of the kth layer, S (k+1) represents the output feature matrix of layer k+1, W (k) is the weight matrix of the kth layer, which is used to perform linear transformation, and σ is a nonlinear activation function.
[0140] FastGCN uses a hierarchical sampling method to randomly sample nodes and edges from each layer, reducing the amount of computation. For each layer, FastGCN randomly samples from the neighbors of the node to form a smaller subgraph. This includes randomly sampling from the nodes of the previous layer and performing convolution operations on the features of the sampled nodes.
[0141] FastGCN is a neural network based on graph convolution, which extracts features from graph structures through efficient sampling and convolution operations. The following is the structure of FastGCN, which is divided into multiple convolutional layers, each with a specific function.
[0142] (1) Input Layer
[0143] Function: The input layer accepts node features of point cloud data.
[0144] Input: Feature matrix of point cloud node.
[0145] Output: The input features are directly passed to the next layer.
[0146] (2) First GCN Layer
[0147] Function: The first layer of graph convolution is used to aggregate the feature vector of a node with the features of its adjacent nodes and extract local structural information.
[0148] Convolution operation: implemented through GCNConv, which calculates the weighted feature sum of each node and its neighboring nodes.
[0149] Activation function: Use ReLU activation function to increase the nonlinear ability of the network.
[0150] Output: Feature matrix after convolution and activation.
[0151] (3) Hidden Layers
[0152] Function: The hidden layer is a stack of multiple graph convolutional layers, which is used to extract deeper features layer by layer. Each layer captures more information in the graph structure, thereby enriching the feature representation.
[0153] Convolution operation: Each hidden layer continues to perform graph convolution operations to further aggregate features.
[0154] Activation function: Each hidden layer usually applies an activation function (ReLU).
[0155] Output: The features output by the hidden layer gradually incorporate more global information.
[0156] (4) Output Layer
[0157] Function: The output layer processes the final features and generates the final feature representation for subsequent point cloud registration tasks.
[0158] Convolution operation: The convolution in the last layer is usually used to reduce the feature dimension to the target dimension.
[0159] Output: Deep feature representation after the last layer of convolution.
[0160] First, the point cloud data is represented as a graph structure, where each point is used as a vertex, and the adjacency matrix is constructed using the k-nearest neighbor (k-NN) method. Next, the feature vector of each vertex is initialized, including the three-dimensional coordinates of the point and local geometric information. The local geometric information includes the distance from the point to the neighboring point, curvature, normal vector, etc. Then, a fast graph convolution layer is applied for feature extraction. By stacking multiple layers of fast graph convolution layers, the deep features of the point cloud are extracted layer by layer, and the final feature representation contains both the local information of the point cloud and captures the global structure.
[0161] As the number of convolutional layers increases, the feature representation in the network becomes more and more abstract. This is because each layer of convolution not only aggregates the node features of the current layer, but also gradually integrates the information transmitted from distant nodes. Therefore, deep features can better represent the high-order relationships and complex structures in the data. In the graph convolutional network, by stacking multiple layers of convolutional layers, deep features that contain both local information and global structure can be extracted layer by layer. Global structure refers to the overall features of the entire point cloud data, such as overall shape, topological structure, symmetry, etc. These features go beyond the local information of a single node and its neighbors, but reflect the global characteristics of the point cloud data. After multiple layers of graph convolution, the obtained deep features may represent the relative position of a point in the entire point cloud structure, and even capture some global patterns or shape features. The deep features extracted by FastGCN not only improve the efficiency of feature extraction, but also enhance the expressiveness of features, making the subsequent fine registration process more accurate and robust.
[0162] In step S5, the iterative closest point algorithm (ICP) is applied to the features extracted by FastGCN to further improve the precise registration of the point cloud through iterative optimization.
[0163] S5-1, first, use FastGCN to extract the deep features of the point cloud, form the source feature point cloud and the target feature point cloud, and represent the source feature point cloud as P feature , the target feature point cloud is represented by Q feature Next, for each source feature point cloud feature point Find the nearest feature point in the target feature point cloud , and based on the set of matching point pairs Calculate the rigid body transformation matrix H (including the rotation matrix R and the translation vector t) to minimize the matching error between the source feature point cloud and the target feature point cloud.
[0164] Specifically, the transformation matrix H is determined by solving the following optimization problem:
[0165] (7)
[0166] Among them, H is the transformation matrix, and the form of the transformation matrix H is , where R is the rotation matrix and t is the translation vector.
[0167] S5-2, then, the feature points of the source feature point cloud are transformed using the calculated transformation matrix H to obtain a transformed point cloud. The process of nearest point matching and transformation matrix calculation is repeated until the matching error converges to a preset threshold or reaches a maximum number of iterations.
[0168] Step S6, through factor graph optimization, the point cloud registration problem is modeled as a factor graph, the optimization objective function is defined, the error term in the factor graph is minimized, and the factor graph optimization algorithm is used to iteratively solve until the error is lower than a preset threshold, thereby improving the global consistency and accuracy of the registration.
[0169] S6-1, the point cloud registration problem is modeled as a factor graph. The factor graph consists of variable nodes and factor nodes, where each variable node represents the position and posture of the point cloud, and each factor node represents the constraint relationship between the point clouds. Assume that the source feature point cloud and the target feature point cloud involve r point pairs in the registration process. , the distance errors between these point pairs are added as factor nodes to the factor graph. The objective function is used to quantify the errors of each factor node in the factor graph, and the optimization objective function is defined as the sum of the squares of the errors of all factor nodes:
[0170] (8)
[0171] Among them, r is the assumption that the source point cloud and the target point cloud involve r point pairs in the registration process, is the error of the i-th factor node, and h is the variable node to be optimized, including the position and posture of the point cloud. Specifically, the error It can be expressed as:
[0172] (9)
[0173] Among them, R and t are the rotation matrix and translation vector that need to be optimized, is the feature point in the source feature point cloud, is the feature point in the target feature point cloud.
[0174] S6-2, using the error algorithm of S6-1 to optimize and adjust the values of the variable nodes to minimize the error term in the objective function. This process includes calculating the residual of each factor node and updating the position and posture of the target point cloud based on these residuals until the value of the objective function is lower than the preset threshold or the maximum number of iterations is reached.
[0175] The nonlinear optimization algorithm Gauss-Newton is used to minimize the error term and solve the rotation matrix R and the translation vector t.
[0176] S6-3, the Gauss-Newton algorithm updates the value of each variable node through iterative optimization to gradually reduce the error. The optimization process involves calculating the Jacobian matrix to update the rotation matrix R and the translation vector t:
[0177] The specific derivation process is as follows:
[0178] Let the objective function
[0179] (10)
[0180] make .
[0181] First find the first-order derivative:
[0182] (11)
[0183] (12)
[0184] in It is the element in the jth row and ith column of the Jacobian matrix, so the first-order derivative can be expressed as follows:
[0185] (13)
[0186] Second order derivative:
[0187] (14)
[0188] Here Indicates the index of the current iteration step.
[0189] Observe the result of the second-order derivative. and are all elements in the Jacobian matrix, and when the iteration point is far away from the target point, the error and its second-order derivative are very small and can be ignored, so its second-order derivative can be written as follows:
[0190] (15)
[0191] So After the second-order expansion, it can be written as follows:
[0192] (16)
[0193] Taking the derivative and making it equal to 0 gives:
[0194] (17)
[0195] It can be expressed as:
[0196] (18)
[0197] Where J is the Jacobian matrix of the error term, ε is the error term vector, h is the updated variable value, is the value of the variable before updating.
[0198] After multiple iterations of optimization, the final point cloud registration result is obtained, that is, the globally optimal point cloud position and posture. These optimization results are manifested as the minimum distance error between point clouds, which significantly improves the global consistency and accuracy of point cloud registration.
[0199] The above implementation cases are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, they are not intended to limit the present invention. Therefore, any simple modification, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.
Claims
1. A point cloud registration method based on double downsampling and graph optimization, characterized in that: The specific steps are as follows: Step S1, obtaining point cloud data as a source point cloud, each point in the point cloud data having coordinates in a three-dimensional space; Step S2, performing double downsampling on the source point cloud and the target point cloud, the double downsampling includes coarse sampling and fine sampling, the coarse sampling adopts voxel downsampling, and the fine sampling adopts downsampling based on normal angle; Step S3, using iterative distance-aware similarity matrix convolution for coarse registration, adjusting the position and direction of the source point cloud set in an iterative manner to achieve preliminary global alignment; Step S4, using a fast graph convolutional neural network to extract deep features of the point cloud; Step S5, applying an iterative closest point algorithm to the features extracted by the fast graph convolutional neural network; Step S6, modeling the point cloud registration problem as a factor graph through factor graph optimization, defining an optimization objective function, minimizing the error term in the factor graph, and using the factor graph optimization algorithm to iteratively solve until the error is lower than a preset threshold; In step S2, the coarse sampling method is: The side length of the voxel in the coarse sampling is set to , for the first The coordinates of the points , calculate the index of the voxel , the formula is: (1) in, Represents the first Points in voxel index in the direction, Represents the first Points in voxel index in the direction, Represents the first Points in voxel index in direction; It is Points in The coordinates on the axis, It is Points in The coordinates on the axis, It is Points in Coordinates on the axis; is the side length of the voxel in the coarse sampling; Assume that there are N points in a voxel. Voxel downsampling is to create a three-dimensional voxel grid based on the input point cloud data. Each voxel grid is represented by the centroid of all points in the grid. The calculation expression is: (2) in, , , They are The three coordinates of the point; N is the point within the voxel grid, is the center of mass of the voxel; The downsampling based on the normal angle is to calculate the average value of the angles between a point and the normal vectors of all other points in the neighborhood. If the average value of the angles exceeds a preset threshold, the point is regarded as a feature point and retained. The retained feature points are used as selection points, and the normal vectors of the selected points and other points in the domain are calculated. The angle average formula is: (3) in, represents the number of all points in the neighborhood, Indicates The normal vector of a point; If the average value of the angle between the normal vector of the selected point and the normal vectors of other points in the neighborhood exceeds the set threshold, then the point represents a key feature in the point cloud; The step S6 includes the following contents: S6-1, the point cloud registration problem is modeled as a factor graph. The factor graph consists of variable nodes and factor nodes, where each variable node represents the position and posture of the point cloud, and each factor node represents the constraint relationship between point clouds; the objective function is used to quantify the error of each factor node in the factor graph, and the optimization objective function is defined as the sum of the squares of the errors of all factor nodes: (8) in, It is assumed that the source point cloud and the target point cloud involve A point, It is The error of the factor node, is the variable node to be optimized, including the position and posture of the point cloud; specifically, the error It can be expressed as: (9) in, and are the rotation matrix and translation vector to be optimized, is the feature point in the source point cloud, is the feature point in the target point cloud; S6-2, using the error algorithm of S6-1 to optimize and adjust the values of the variable nodes to minimize the error term in the objective function; including calculating the residual of each factor node, and updating the position and posture of the target point cloud according to the residual, until the value of the objective function is lower than a preset threshold or the maximum number of iterations is reached; The nonlinear optimization algorithm Gauss-Newton is used to minimize the error term and solve the rotation matrix and translation vectors ; S6-3, the Gauss-Newton algorithm updates the value of each variable node through iterative optimization. The optimization process is to calculate the Jacobian matrix to update the rotation matrix and translation vectors ; After multiple iterative optimizations, the final point cloud registration result is obtained, that is, the globally optimal point cloud position and posture.
2. The point cloud registration method based on double downsampling and graph optimization according to claim 1, characterized in that: In step S1, the point cloud data is first denoised, a neighborhood radius and a threshold are set, and the distance between each point and the neighborhood points is analyzed. If the number of points in the neighborhood of a point is less than the threshold, the point is considered to be a noise point.
3. The point cloud registration method based on double downsampling and graph optimization according to claim 1, characterized in that: In step S3, the iteration method is: S3-1, set Geometric features and Geometric features , the dimensions of both features are K, where is the source point cloud set, is the target point cloud set, in The distance enhanced feature tensor is formed at the iteration as follows: (4) in, Represents concatenation, which means that in the similarity matrix convolution, the geometric feature vectors of the source point cloud and the target point cloud are concatenated. and And Euclidean features are concatenated into a (2K+4)-dimensional vector as the input feature; · refers to four quantities: , , , ; The (2K+4)-dimensional vector of the (i, j) position is the point pair A combination of geometric features and Euclidean features; four-dimensional Euclidean features include and The distance between point to The unit vector of Each enhanced feature vector in encodes the joint information of the local shape and current relative position of two points; S3-2, distance-expanded feature tensor is a 2D image with (2K+4) channels. A 1×1 2D convolution is applied to the last layer, and a single-channel image of the same spatial size is output at the last layer, that is, a multi-layer perceptron is applied to the enhanced feature vector at each position; then the Softmax function is applied to each row of the single-channel image to obtain the similarity matrix, which is recorded as ; Each row of defines a normalized probability distribution; S3-3, take The argmax of each row is obtained from the similarity matrix Find the point index in the target point cloud corresponding to the maximum similarity in each row of ,in For the source point cloud The target point cloud with the highest similarity score uses these corresponding point pairs to solve the following optimization problem to find the estimated rigid body transformation , : (5) in, is the rotation matrix, is the translation vector; Calculated using the orthogonal Procrstes algorithm of singular value decomposition; before entering the next iteration, use and Transform the source point cloud to a new location.
4. The point cloud registration method based on double downsampling and graph optimization according to claim 3, characterized in that: In step S4, the fast graph convolutional neural network includes multiple convolutional layers, namely: (1) Input layer Function: The input layer accepts node features of point cloud data; Input: feature matrix of point cloud node; Output: The input features are directly passed to the next layer; (2) First layer of graph convolution Function: The first layer of graph convolution is used to aggregate the feature vector of a node with the features of its neighboring nodes to extract local structural information; Convolution operation: implemented through GCNConv, calculating the weighted feature sum of each node and its adjacent nodes; Activation function: Use ReLU activation function; Output: feature matrix after convolution and activation; (3) Hidden layer Function: The hidden layer is a stack of multiple graph convolutional layers, which is used to extract deeper features layer by layer; Convolution operation: Each hidden layer continues to perform graph convolution operations to further aggregate features; Activation function: Each hidden layer usually applies an activation function; Output: The features output by the hidden layer gradually incorporate more global information; (4) Output layer Function: The output layer processes the final features and generates the final feature representation for subsequent point cloud registration tasks; Convolution operation: The convolution of the last layer is usually used to reduce the feature dimension to the target dimension; Output: Deep feature representation after the last layer of convolution.
5. The point cloud registration method based on double downsampling and graph optimization according to claim 4, characterized in that: In the step S5, specifically: S5-1, first, the deep features in step S4 are used to form a source feature point cloud and a target feature point cloud, and the source feature point cloud is represented as , the target feature point cloud is expressed as ; Next, for each source feature point cloud feature point Find the nearest feature point in the target feature point cloud , and based on the set of matching point pairs Calculate the rigid body transformation matrix , to minimize the matching error between the source feature point cloud and the target feature point cloud; S5-2, using the transformation matrix Transform the feature points of the source feature point cloud to obtain a transformed point cloud; The process of closest point matching and transformation matrix calculation is repeated until the matching error converges to a preset threshold or the maximum number of iterations is reached.
6. The point cloud registration method based on double downsampling and graph optimization according to claim 5, characterized in that: The transformation matrix This is determined by solving an optimization problem: (7) in, is the transformation matrix, the transformation matrix The form is ,in is the rotation matrix, is the translation vector.
Citation Information
Patent Citations
Point cloud matching method and system based on derivative-free optimization
CN118314180A
Cross-source point cloud registration
GB201714179D0
Method, apparatus, electronic device and readable storage medium for point cloud data processing
US20210358213A1
WGAN-based unsupervised multi-view three-dimensional point cloud joint registration method
WO2022165876A1
