Point cloud alignment method and device based on adaptive voxels and electronic equipment
A point cloud alignment method based on adaptive voxel partitioning and KD tree algorithm for identifying planar features solves the problems of low efficiency and low accuracy in point cloud data alignment, achieving more efficient point cloud data integration and higher accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-02
- Publication Date
- 2026-03-31
AI Technical Summary
Existing technologies suffer from low processing efficiency, poor real-time performance, and low accuracy when aligning point cloud data. In particular, the computational complexity is high and the dependence on the initial pose is strong during the alignment of multi-view point cloud data. Furthermore, deep learning solutions consume a lot of computational resources.
An adaptive voxel point cloud alignment method is adopted. By acquiring point cloud frames in different coordinate systems, dividing them into multiple voxel units, using the KD tree algorithm to identify planar features and fuse them, and calculating the alignment rotation matrix, redundant data and computation are reduced.
It improves the computational efficiency and accuracy of point cloud alignment, reduces dependence on the initial pose, lowers the computational resource requirements, and achieves more efficient point cloud data integration.
Smart Images
Figure CN121767544A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of three-dimensional imaging technology, and in particular to a point cloud alignment method and apparatus based on adaptive voxels, as well as an electronic device. Background Technology
[0002] With the rapid development of 3D imaging technology, 3D reconstruction technology has been widely applied in fields such as robotics, intelligent manufacturing, and heritage preservation. In robotics, 3D reconstruction is used to generate three-dimensional models of the environment to support navigation, path planning, or task execution. In intelligent manufacturing, 3D reconstruction can accurately measure the surface shape of objects for quality inspection and production line optimization. In heritage preservation, 3D scanning records the form of historical buildings or cultural relics, enabling digital preservation and display. 3D LiDAR, as a commonly used technology, captures depth information by emitting lasers and represents it as a point cloud, providing the basic data for 3D reconstruction. However, due to limitations in the scanning angle of the equipment and the complexity of the object's shape, a single scan is insufficient to obtain complete point cloud data. Therefore, it is usually necessary to scan from multiple perspectives and align the point clouds to create a complete 3D model. During this alignment process, aligning multiple point cloud data collected from different angles requires processing a large amount of data. Furthermore, the accuracy of alignment is relatively low because different point cloud data contain significantly different point cloud distributions.
[0003] Therefore, a technical solution is needed to improve the processing efficiency and accuracy of point cloud data during alignment. Summary of the Invention
[0004] This application provides a point cloud alignment method, apparatus, and electronic device based on adaptive voxels to address the shortcomings of existing technologies in terms of low processing efficiency, poor real-time performance, and low accuracy when aligning point clouds.
[0005] To achieve the above objectives, embodiments of this application provide a point cloud alignment method based on adaptive voxels, characterized by comprising: Two point cloud frames that need to be aligned are obtained. The two point cloud frames include a first point cloud frame based on a first coordinate system and a second point cloud frame based on a second coordinate system, wherein the first coordinate system is different from the second coordinate system. The first voxel of the first point cloud frame and the second voxel of the second point cloud frame are determined based on the point cloud density of the first point cloud frame and the second point cloud frame, so as to divide the first point cloud frame and the second point cloud frame into multiple first point cloud voxel units and multiple second point cloud voxel units, respectively. Using the KD tree algorithm, for each first point cloud voxel unit and second point cloud voxel unit, identify the first point cloud frame voxel unit and second point cloud frame voxel unit with planar distribution characteristics, and extract their planar features. The planar features of the first point cloud frame voxel unit and the second point cloud frame voxel unit are fused respectively to generate the first point cloud voxel feature and the second point cloud voxel feature. Based on the first point cloud voxel features and the second point cloud voxel features, calculate the alignment rotation matrix between the first point cloud frame and the second point cloud frame. Using the alignment and rotation matrix, the first point cloud frame is transformed into the second coordinate system of the second point cloud frame.
[0006] According to the adaptive voxel-based point cloud alignment method of the present application embodiment, determining the first voxel of the first point cloud frame and the second voxel of the second point cloud frame based on the point cloud density of the first point cloud frame and the second point cloud frame includes: For the first point cloud frame and the second point cloud frame, they are divided into multiple first point cloud frame voxel units and multiple second point cloud frame voxel units according to a predetermined first voxel size and a predetermined second voxel size, respectively. For each of the plurality of first point cloud frame voxel units and the plurality of second point cloud frame voxel units, the local point cloud density in that voxel unit is calculated using the following formula. in, It is the number of point clouds contained in the i-th voxel unit. It is the volume of the i-th voxel unit; Based on local point cloud density The adjustment factor is calculated for each voxel using the following formula. in, It is a positive parameter used to control the sensitivity of voxel size to the local point cloud density; The adjusted voxel size for each voxel unit is calculated using the following formula. Where represents the side length of the i-th voxel unit, and D is the diagonal length of the smallest rectangle containing the entire voxel unit point cloud, and D is calculated using the following formula. in, , , , , and These are the maximum and minimum values of a point in a voxel element along the x, y, and z axes of the coordinate system containing that voxel element.
[0007] According to the adaptive voxel-based point cloud alignment method of this application embodiment, the KD tree algorithm is used to identify first and second point cloud voxel units with planar distribution characteristics for each first and second point cloud voxel unit, and extract their planar features, including: For the point cloud set within the i-th voxel unit Calculate its covariance matrix using the following formula. ,in, It is the centroid of the point cloud within the i-th voxel unit; Calculate the covariance matrix using the following formula. eigenvalues and its eigenvectors Among them, eigenvalues Let represent the dispersion of the point cloud within the i-th voxel in different directions, and its eigenvectors. Indicates the main distribution direction of the point cloud within the i-th voxel unit; The eigenvalues within the i-th voxel unit The eigenvector corresponding to the smallest eigenvalue in the i-th voxel is determined as the normal vector of the plane corresponding to the point cloud in the i-th voxel.
[0008] According to the adaptive voxel-based point cloud alignment method of this application embodiment, the planar features of the first point cloud frame voxel unit and the second point cloud frame voxel unit are fused to generate the first point cloud voxel feature and the second point cloud voxel feature, respectively, including: For the first point cloud frame and the second point cloud frame voxel units, respectively calculate the distance between the centroid planes of the planes corresponding to each first point cloud frame voxel unit and the distance between the centroid planes of the planes corresponding to each second point cloud frame voxel unit. Using the following formulas, based on the distance between the normal vector of the plane corresponding to each first point cloud frame voxel unit and the centroid plane, and based on the distance between the normal vector of the plane corresponding to each second point cloud frame voxel unit and the centroid plane, the planes to be fused in the first and second point cloud frame voxel units are determined to form multiple sets of planes to be fused. , where represents The normal vector of the i-th plane, and and Let these be the centroid planes of two adjacent planes t and s, respectively. This represents the fusion threshold used to determine the planes that need to be merged. For each group of planes to be fused in the first point cloud frame, a plane is determined from the group of planes to be fused as the first plane of each group of planes to be fused in the first point cloud frame, and the eigenvalues and eigenvectors of the determined plane are used as the first point cloud voxel features of the group of planes to be fused. For each group of planes to be fused in the second point cloud frame, a plane is determined from the group of planes to be fused as the second plane of each group of planes to be fused in the second point cloud frame, and the eigenvalues and eigenvectors of the determined plane are used as the second point cloud voxel features of the group of planes to be fused.
[0009] According to the point cloud alignment method based on adaptive voxels according to embodiments of this application, calculating the alignment rotation matrix between the first point cloud frame and the second point cloud frame based on the first point cloud voxel features and the second point cloud voxel features includes: Using the first planes determined in each group of planes to be fused in the first point cloud frame, a target plane corresponding to the planes in the second point cloud frame is determined, wherein the target plane is the second plane determined in each group of planes to be fused in the second point cloud frame; Calculate the rotation axis r from the first plane to the second plane using the following formula. in, and These are the normal vectors of the two first planes, and These are the normal vectors of the two second planes, It is an intermediate variable used to calculate the rotation axis r. Calculate the rotation angle using the following formula. , in, It is the projection vector. and They are orthogonal vectors, and Indicates inner product operation; The rotation matrix R is obtained using the following formula. Where I is the identity matrix, It is the tensor product of the rotation axes, and It is an antisymmetric matrix.
[0010] According to the point cloud alignment method based on adaptive voxels according to the embodiments of this application, calculating the alignment rotation matrix between the first point cloud frame and the second point cloud frame based on the first point cloud voxel features and the second point cloud voxel features further includes: Select a third plane from a plurality of first planes, the third plane being related to the first plane. and Different and their normal vectors With the first plane and With a large angle difference, the geometric constraints are calculated using the following formula. in, It is the threshold for determining the third plane; Calculate the translation vector T from the first plane to the second plane using the following formula. Where x is an intermediate variable, A is the normal vector matrix, and d is the planar distance vector, and .
[0011] This application also provides a point cloud alignment device based on adaptive voxels, including: The acquisition module is used to acquire two point cloud frames that need to be aligned. The two point cloud frames include a first point cloud frame based on a first coordinate system and a second point cloud frame based on a second coordinate system, wherein the first coordinate system is different from the second coordinate system. The first determining module is used to determine the first voxel of the first point cloud frame and the second voxel of the second point cloud frame based on the point cloud density of the first point cloud frame and the second point cloud frame, so as to divide the first point cloud frame and the second point cloud frame into multiple first point cloud voxel units and multiple second point cloud voxel units respectively. The first identification module is used to use the KD tree algorithm to identify first point cloud voxel units and second point cloud voxel units with planar distribution characteristics for each first point cloud voxel unit and second point cloud voxel unit, and extract their planar features. The fusion module is used to fuse the planar features of the first point cloud frame voxel unit and the second point cloud frame voxel unit respectively to generate the first point cloud voxel feature and the second point cloud voxel feature. The first calculation module is used to calculate the alignment rotation matrix between the first point cloud frame and the second point cloud frame based on the first point cloud voxel features and the second point cloud voxel features. The conversion module is used to convert the first point cloud frame to the second coordinate system of the second point cloud frame using the alignment and rotation matrix.
[0012] This application also provides an electronic device, including: Memory, used to store programs; A processor is configured to run the program stored in the memory, wherein the program executes the adaptive voxel-based point cloud alignment method provided in the embodiments of this application.
[0013] This application also provides a computer-readable storage medium storing a computer program executable by a processor, wherein the program, when executed by the processor, implements the adaptive voxel-based point cloud alignment method provided in this application.
[0014] The adaptive voxel-based point cloud alignment method, apparatus, electronic device, and storage medium provided in this application acquire two point cloud frames based on different coordinate systems that need to be aligned. The corresponding voxels are determined based on the densities of the first and second point cloud frames, dividing the two point cloud frames into multiple first point cloud frame voxel units and multiple second point cloud frame voxel units respectively. A KD-tree algorithm is used to identify and extract planar features for each voxel unit. The extracted planar features are fused to generate voxel features for the first and second point cloud frames. Based on the generated voxel features, a rotation matrix is calculated to transform the first point cloud frame to the coordinate system of the second point cloud frame. Therefore, the point cloud alignment method of this application embodiment can determine the size of voxels based on the point cloud density within the point cloud frame, thereby dividing the point cloud within the point cloud frame into voxel units of a size corresponding to the point cloud density. Based on such voxel units, features reflecting planar information are extracted, and the planar features of each voxel unit are appropriately fused to obtain features that more accurately reflect planar information. The rotation matrix is then calculated, thereby reducing redundant data and computation while retaining key information, improving computational efficiency, and obtaining a rotation matrix that more accurately reflects the geometric relationship between planes, thus improving the accuracy of alignment.
[0015] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description
[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of the point cloud alignment method based on adaptive voxels according to this application. Figure 2This is a schematic diagram of the structure of an embodiment of the point cloud alignment device based on adaptive voxels according to this application. Figure 3 A schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation
[0017] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0018] With the rapid development of 3D imaging technology, 3D reconstruction technology has been widely applied in various fields such as robotics, intelligent manufacturing, and heritage preservation. In robotics, 3D reconstruction generates three-dimensional models of the environment, supporting navigation, path planning, and task execution. In intelligent manufacturing, 3D reconstruction can accurately measure the surface shape of objects for quality inspection and production line optimization. In heritage preservation, 3D scanning can record the form of historical buildings or cultural relics, enabling digital preservation and display. 3D LiDAR is a commonly used technology that acquires depth information by emitting lasers and converts it into point cloud data, providing the foundation for 3D reconstruction. However, due to the limitations of the scanning angle of the equipment and the complexity of the object's shape, a single scan is insufficient to capture complete point cloud data. Therefore, scanning from multiple perspectives is usually required, and point cloud alignment is used to integrate this data to create a complete 3D model. The core objective of point cloud alignment is to unify multi-view point clouds into the same coordinate system by calculating the optimal mapping relationship between different point cloud frames, thereby generating a complete 3D point cloud model. During point cloud alignment, not only is a large amount of data processing required, but the accuracy of point cloud alignment also faces challenges due to the significant differences in the distribution of point cloud data acquired from different perspectives.
[0019] To address these issues, the ICP algorithm has been proposed in existing technologies. It searches for the closest point pairs between the source and target point clouds and continuously optimizes the transformation matrix to minimize the corresponding points. However, the ICP algorithm suffers from high computational complexity, a tendency to enter local optima, and strong dependence on the initial pose. Existing technologies also propose using a base set of four coplanar source point clouds and searching for collinear candidate point pairs to establish the basis in the target point cloud for rigid body transformation calculation. This existing approach does not require denoising the point clouds and requires low point cloud overlap, but it is prone to four-point set mismatch and has high computational complexity. Furthermore, existing technologies have proposed using deep learning neural networks for point cloud alignment calculations. However, feature extraction-based deep learning alignment schemes require significant GPU computing resources, resulting in high computational costs and, in practical applications, still suffer from low computational efficiency and poor real-time performance.
[0020] Therefore, embodiments of this application provide a point cloud alignment method based on adaptive voxels. For example... Figure 1 As shown, Figure 1 A schematic flowchart of an embodiment of the point cloud alignment method based on adaptive voxels is shown. Figure 1 As shown in the figure, the point cloud alignment method based on adaptive voxels according to the embodiments of this application includes: S101, Obtain the two point cloud frames that need to be aligned.
[0021] In step S101, two point cloud frames that need to be aligned can be acquired. In this embodiment, the point cloud frames can be acquired directly from the lidar in real time, or the point cloud data to be aligned can be acquired from a cloud server. For example, the two point cloud frames acquired in step S101 may include a first point cloud frame based on a first coordinate system and a second point cloud frame based on a second coordinate system. Specifically, the first coordinate system is different from the second coordinate system.
[0022] S102, determine the first voxel of the first point cloud frame and the second voxel of the second point cloud frame according to the point cloud density of the first point cloud frame and the second point cloud frame, so as to divide the first point cloud frame and the second point cloud frame into multiple first point cloud voxel units and multiple second point cloud voxel units respectively.
[0023] In step S102, the first voxel and the second voxel for dividing the first and second point cloud frames can be determined based on the distribution density of the point clouds within the first and second point cloud frames obtained in step S101. For example, the first and second point cloud frames obtained in step S101 can be divided into multiple first point cloud frame voxel units and multiple second point cloud frame voxel units, respectively, with predetermined first and second voxel sizes. That is, the first and second point cloud frames can be divided, for example, uniformly with a predetermined size, as a basis for subsequent dynamic adjustment of the voxel size.
[0024] Then, for each of the multiple first point cloud frame voxel units and multiple second point cloud frame voxel units obtained by dividing the voxel by a predetermined voxel size, the local point cloud density in that voxel unit is calculated using formula (1). (1) in, It is the number of point clouds contained in the i-th voxel unit. It is the volume of the i-th voxel unit; Based on local point cloud density The adjustment factor is calculated for each voxel unit using formula (2). (2) in, It is a positive parameter used to control the sensitivity of voxel size to the local point cloud density; The adjusted voxel size of each voxel unit is calculated using formula (3). (3) Where represents the side length of the i-th voxel unit, and D is the diagonal length of the smallest rectangle containing the entire voxel unit point cloud, and D is calculated using formula (4). (4) in, , , , , and These are the maximum and minimum values of a point in a voxel element along the x, y, and z axes of the coordinate system containing that voxel element.
[0025] Therefore, in step S102, the appropriate partition size can be re-determined based on the density of the point cloud distribution in each voxel unit of a preset size, and the voxel units can be re-partitioned, so that, for example, the voxel units in areas with denser point cloud distribution are smaller, while the voxel units in areas with sparser point cloud distribution are larger, thereby improving the processing efficiency of point cloud processing.
[0026] S103. Using the KD tree algorithm, for each first point cloud voxel unit and second point cloud voxel unit, identify the first point cloud frame voxel unit and second point cloud frame voxel unit with planar distribution characteristics, and extract their planar features.
[0027] In step S103, the KD tree algorithm can be used to perform planar feature recognition on the first and second point cloud voxel units obtained in step S102, that is, to identify point cloud voxel units that contain multiple point clouds that can reflect or display planar distribution features, and extract their planar features.
[0028] For example, in the embodiments of this application, the point cloud set within the i-th unit of the first point cloud voxel unit or the second point cloud voxel unit can be first... The covariance matrix is calculated using formula (5). (5), among which, Let be the centroid of the point cloud within the i-th voxel unit. Then, calculate the covariance matrix using formula (6). eigenvalues and its eigenvectors (6) Among them, eigenvalues Let represent the dispersion of the point cloud within the i-th voxel in different directions, and its eigenvectors. This indicates the main distribution direction of the point cloud within the i-th voxel unit. The eigenvalues within the i-th voxel unit The eigenvector corresponding to the smallest eigenvalue in the i-th voxel is determined as the normal vector of the plane corresponding to the point cloud in the i-th voxel.
[0029] S104, the planar features of the first point cloud frame voxel unit and the second point cloud frame voxel unit are fused respectively to generate the first point cloud voxel feature and the second point cloud voxel feature. In this embodiment, since point cloud data is typically discrete, although the planes extracted from the voxel units obtained through step S103 are appropriately divided using dynamically determined voxel sizes in step S102, the planes extracted from the point cloud within the voxel units may still be small and fragmented, and the accuracy of the represented planes may not be high. Therefore, in this embodiment, in step S104, the distance between the centroid planes of the planes corresponding to the first point cloud frame voxel units and the distance between the centroid planes of the planes corresponding to the second point cloud frame voxel units can be calculated for the first point cloud frame voxel units and the second point cloud frame voxel units, respectively.
[0030] Then, using formula (7), based on the distance between the normal vector of the plane corresponding to each first point cloud frame voxel unit and the centroid plane, and based on the distance between the normal vector of the plane corresponding to each second point cloud frame voxel unit and the centroid plane, the planes to be fused in the first and second point cloud frame voxel units are determined to form multiple sets of planes to be fused. (7), where represents The normal vector of the i-th plane, and and Let these be the centroid planes of two adjacent planes t and s, respectively. This represents the fusion threshold for determining the planes to be fused. For each group of planes to be fused in the first point cloud frame, a plane is determined from that group of planes to be fused as the first plane of each group of planes to be fused in the first point cloud frame, and the eigenvalues and eigenvectors of the determined plane are used as the first point cloud voxel features of that group of planes to be fused.
[0031] For each group of planes to be fused in the second point cloud frame, a plane is determined from the group of planes to be fused as the second plane of each group of planes to be fused in the second point cloud frame, and the eigenvalues and eigenvectors of the determined plane are used as the second point cloud voxel features of the group of planes to be fused.
[0032] S105, based on the features of the first point cloud voxel and the second point cloud voxel, calculate the alignment rotation matrix between the first point cloud frame and the second point cloud frame.
[0033] In step S105, the alignment and rotation matrix for transforming the point cloud of the first point cloud frame to the coordinate system of the second point cloud frame can be calculated based on the first point cloud voxel features and the second point cloud voxel features obtained in step S104.
[0034] For example, in this embodiment of the application, the first planes determined in each group of planes to be fused in the first point cloud frame can be used to determine the target planes corresponding to the planes in the second point cloud frame, wherein the target planes are the second planes determined in each group of planes to be fused in the second point cloud frame.
[0035] Then, the rotation axis r from the first plane to the second plane can be calculated using the following formula (8). (8) in, and These are the normal vectors of the two first planes, and These are the normal vectors of the two second planes, It is an intermediate variable used to calculate the rotation axis r. Then, the rotation angle can be calculated using the following formula (9). , (9) in, It is the projection vector. and They are orthogonal vectors, and This indicates the inner product operation.
[0036] Finally, the rotation matrix R can be obtained using the following formula (10). (10) Where I is the identity matrix, It is the tensor product of the rotation axes, and It is an antisymmetric matrix.
[0037] Furthermore, to further determine the transformation relationships between point clouds, in this embodiment of the application, a third plane can be selected from a plurality of first planes, wherein the third plane and the first plane... and Different and their normal vectors With the first plane and Having a large angle difference can enhance the stability of matrix calculations. Specifically, the geometric constraints can be calculated using the following formula (11). (11) in, It is the threshold for determining the third plane; The translation vector T from the first plane to the second plane is calculated using the following formulas (12)-(14). （12） (13) (14) Where x is an intermediate variable, A is the normal vector matrix, and d is the planar distance vector, and .
[0038] S106, using an alignment rotation matrix, transform the first point cloud frame to the second coordinate system of the second point cloud frame.
[0039] In step S106, the rotation matrix obtained in step S105 can be used to transform the point cloud in the first point cloud frame into the coordinate system of the second point cloud frame, thereby achieving the alignment of the first point cloud frame and the second point cloud frame.
[0040] Therefore, the point cloud alignment method based on adaptive voxels provided in this application obtains two point cloud frames based on different coordinate systems that need to be aligned, determines the corresponding voxels according to the density of the first point cloud frame and the second point cloud frame, divides the two point cloud frames into multiple first point cloud frame voxel units and multiple second point cloud frame voxel units respectively, uses the KD tree algorithm to identify and extract the planar features of each point cloud voxel unit, fuses the extracted planar features to generate voxel features of the first point cloud frame and the second point cloud frame, and calculates the rotation matrix that transforms the first point cloud frame to the coordinate system of the second point cloud frame based on the generated voxel features. Therefore, the point cloud alignment method of this application embodiment can determine the size of voxels based on the point cloud density within the point cloud frame, thereby dividing the point cloud within the point cloud frame into voxel units of a size corresponding to the point cloud density. Based on such voxel units, features reflecting planar information are extracted, and the planar features of each voxel unit are appropriately fused to obtain features that more accurately reflect planar information. The rotation matrix is then calculated, thereby reducing redundant data and computation while retaining key information, improving computational efficiency, and obtaining a rotation matrix that more accurately reflects the geometric relationship between planes, thus improving the accuracy of alignment.
[0041] Example 2 This application also provides a point cloud alignment device based on adaptive voxels. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of an embodiment of the adaptive voxel-based point cloud alignment device provided in this application. The odometer device in this application embodiment may include: an acquisition module 21, a first determination module 22, a first identification module 23, a fusion module 24, a first calculation module 25, and a conversion module 26.
[0042] The acquisition module 21 can be used to acquire two point cloud frames that need to be aligned.
[0043] The acquisition module 21 can acquire two point cloud frames that need to be aligned. In this embodiment, the acquisition module 21 can directly acquire the collected point cloud frames from the LiDAR in real time, or it can acquire the point cloud data that needs to be aligned from the cloud server. For example, the two point cloud frames acquired by the acquisition module 21 may include a first point cloud frame based on a first coordinate system and a second point cloud frame based on a second coordinate system. Specifically, the first coordinate system is different from the second coordinate system.
[0044] The first determining module 22 can be used to determine the first voxel of the first point cloud frame and the second voxel of the second point cloud frame based on the point cloud density of the first point cloud frame and the second point cloud frame, so as to divide the first point cloud frame and the second point cloud frame into multiple first point cloud voxel units and multiple second point cloud voxel units respectively.
[0045] The first determining module 22 can determine the first voxel and the second voxel for dividing the first and second point cloud frames based on the distribution density of the point clouds within the first and second point cloud frames acquired by the acquiring module 21. For example, for the first and second point cloud frames acquired by the acquiring module 21, the first determining module 22 can first divide them into multiple first point cloud frame voxel units and multiple second point cloud frame voxel units with predetermined first and second voxel sizes, respectively. That is, the first and second point cloud frames can be first divided, for example, uniformly with a predetermined size, as a basis for subsequent dynamic adjustment of the voxel size.
[0046] Then, for each of the multiple first point cloud frame voxel units and multiple second point cloud frame voxel units obtained by dividing the voxel by a predetermined voxel size, the local point cloud density in that voxel unit is calculated using formula (1). (1) in, It is the number of point clouds contained in the i-th voxel unit. It is the volume of the i-th voxel unit; Based on local point cloud density The adjustment factor is calculated for each voxel unit using formula (2). (2) in, It is a positive parameter used to control the sensitivity of voxel size to the local point cloud density; The adjusted voxel size of each voxel unit is calculated using formula (3). (3) Where represents the side length of the i-th voxel unit, and D is the diagonal length of the smallest rectangle containing the entire voxel unit point cloud, and D is calculated using formula (4). (4) in, , , , , and These are the maximum and minimum values of a point in a voxel element along the x, y, and z axes of the coordinate system containing that voxel element.
[0047] Therefore, the first determining module 22 can redetermine the appropriate partition size based on the density of the point cloud distribution in each preset size voxel unit, and re-partition the voxel units, so that, for example, the voxel units in areas with denser point cloud distribution are smaller, while the voxel units in areas with sparser point cloud distribution are larger, thereby improving the processing efficiency of point cloud processing.
[0048] The first identification module 23 can be used to use the KD tree algorithm to identify first point cloud voxel units and second point cloud voxel units with planar distribution characteristics for each first point cloud voxel unit and second point cloud voxel unit, and extract their planar features.
[0049] The first identification module 23 can use the KD tree algorithm to perform planar feature recognition on the first point cloud voxel unit and the second point cloud voxel unit obtained by the first determination module 22, that is, to identify the point cloud voxel unit that contains multiple point clouds that can reflect or display planar distribution features, and extract its planar features.
[0050] For example, in this embodiment of the application, the first identification module 23 may first identify the point cloud set within the i-th unit of the first point cloud voxel unit or the second point cloud voxel unit. The covariance matrix is calculated using formula (5). (5), among which, Let be the centroid of the point cloud within the i-th voxel unit. Then, calculate the covariance matrix using formula (6). eigenvalues and its eigenvectors (6) Among them, eigenvalues Let represent the dispersion of the point cloud within the i-th voxel in different directions, and its eigenvectors. This indicates the main distribution direction of the point cloud within the i-th voxel unit. The eigenvalues within the i-th voxel unit The eigenvector corresponding to the smallest eigenvalue in the i-th voxel is determined as the normal vector of the plane corresponding to the point cloud in the i-th voxel.
[0051] The fusion module 24 can be used to fuse the planar features of the first point cloud frame voxel unit and the second point cloud frame voxel unit respectively to generate the first point cloud voxel feature and the second point cloud voxel feature. In this embodiment, since point cloud data is typically discrete, although the planes extracted from the voxel units obtained by the first identification module 23 are appropriately divided using dynamically determined voxel sizes by the first determination module 22, the planes extracted from the point cloud within the voxel units may still be small and fragmented, and the accuracy of the represented planes may not be high. Therefore, in this embodiment, the fusion module 24 can calculate the distance between the centroid planes of the planes corresponding to the first point cloud frame voxel units and the distance between the centroid planes of the planes corresponding to the planes of the first point cloud frame voxel units and the second point cloud frame voxel units, respectively.
[0052] Then, using formula (7), the fusion module 24 can determine the planes to be fused in the first and second point cloud frame voxel units respectively, based on the distance between the normal vector of the plane corresponding to each first point cloud frame voxel unit and the centroid plane, and based on the distance between the normal vector of the plane corresponding to each second point cloud frame voxel unit and the centroid plane. (7), where represents The normal vector of the i-th plane, and and Let these be the centroid planes of two adjacent planes t and s, respectively. This represents the fusion threshold for determining the planes to be fused. For each group of planes to be fused in the first point cloud frame, a plane is determined from that group of planes to be fused as the first plane of each group of planes to be fused in the first point cloud frame, and the eigenvalues and eigenvectors of the determined plane are used as the first point cloud voxel features of that group of planes to be fused.
[0053] For each group of planes to be fused in the second point cloud frame, the fusion module 24 can determine a plane from the group of planes to be fused as the second plane of each group of planes to be fused in the second point cloud frame, and use the eigenvalues and eigenvectors of the determined plane as the second point cloud voxel features of the group of planes to be fused.
[0054] The first calculation module 25 can be used to calculate the alignment rotation matrix between the first point cloud frame and the second point cloud frame based on the first point cloud voxel features and the second point cloud voxel features.
[0055] The first calculation module 25 can calculate the alignment and rotation matrix for transforming the point cloud of the first point cloud frame to the coordinate system of the second point cloud frame based on the first point cloud voxel features and the second point cloud voxel features obtained by the fusion module 24.
[0056] For example, in an embodiment of this application, the first calculation module 25 can use each first plane determined in each group of planes to be fused in the first point cloud frame to determine the target plane corresponding to the plane in the second point cloud frame, wherein the target plane is the second plane determined in each group of planes to be fused in the second point cloud frame.
[0057] Then, the first calculation module 25 can use the following formula (8) to calculate the rotation axis r from the first plane to the second plane. (8) in, and These are the normal vectors of the two first planes, and These are the normal vectors of the two second planes, It is an intermediate variable used to calculate the rotation axis r. Then, the rotation angle can be calculated using the following formula (9). , (9) in, It is the projection vector. and They are orthogonal vectors, and This indicates the inner product operation.
[0058] Finally, the rotation matrix R can be obtained using the following formula (10). (10) Where I is the identity matrix, It is the tensor product of the rotation axes, and It is an antisymmetric matrix.
[0059] Furthermore, to further determine the transformation relationships between point clouds, in this embodiment, the first calculation module 25 may also select a third plane from a plurality of first planes, wherein the third plane and the first plane... and Different and their normal vectors With the first plane and Having a large angle difference can enhance the stability of matrix calculations. Specifically, the geometric constraints can be calculated using the following formula (11). (11) in, It is the threshold for determining the third plane; The translation vector T from the first plane to the second plane is calculated using the following formulas (12)-(14). （12） (13) (14) Where x is an intermediate variable, A is the normal vector matrix, and d is the planar distance vector, and .
[0060] The transformation module 26 can be used to transform the first point cloud frame to the second coordinate system of the second point cloud frame using an alignment rotation matrix.
[0061] The conversion module 26 can use the rotation matrix obtained by the first calculation module 25 to convert the point cloud in the first point cloud frame to the coordinate system of the second point cloud frame, thereby achieving the alignment of the first point cloud frame and the second point cloud frame.
[0062] Therefore, the adaptive voxel-based point cloud alignment device provided in this application obtains two point cloud frames based on different coordinate systems that need to be aligned, determines the corresponding voxels according to the density of the first and second point cloud frames, divides the two point cloud frames into multiple first point cloud frame voxel units and multiple second point cloud frame voxel units respectively, uses the KD tree algorithm to identify and extract the planar features of each point cloud voxel unit, fuses the extracted planar features to generate voxel features of the first and second point cloud frames, and calculates the rotation matrix that transforms the first point cloud frame to the coordinate system of the second point cloud frame based on the generated voxel features. Therefore, the point cloud alignment method of this application embodiment can determine the size of voxels based on the point cloud density within the point cloud frame, thereby dividing the point cloud within the point cloud frame into voxel units of a size corresponding to the point cloud density. Based on such voxel units, features reflecting planar information are extracted, and the planar features of each voxel unit are appropriately fused to obtain features that more accurately reflect planar information. The rotation matrix is then calculated, thereby reducing redundant data and computation while retaining key information, improving computational efficiency, and obtaining a rotation matrix that more accurately reflects the geometric relationship between planes, thus improving the accuracy of alignment.
[0063] Example 3 The above describes the internal functionality and structure of the adaptive voxel-based point cloud alignment method, which can be implemented as an electronic device. Figure 3 A schematic diagram illustrating the structure of an embodiment of the electronic device provided in this application. (See attached diagram.) Figure 3 As shown, the electronic device includes a memory 31 and a processor 32.
[0064] Memory 31 is used to store programs. In addition to the programs described above, memory 31 can also be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.
[0065] The memory 31 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0066] Processor 32 is not limited to a processor (CPU), but may also be a graphics processing unit (GPU), a field-programmable gate array (FPGA), an embedded neural network processor (NPU), or an artificial intelligence (AI) chip. Processor 32 is coupled to memory 31 and executes the program stored in memory 31 to perform the adaptive voxel-based point cloud alignment method of Embodiment 1 described above.
[0067] Furthermore, such as Figure 3 As shown, the electronic device may also include other components such as a communication component 33, a power supply component 34, an audio component 35, and a display 36. Figure 3 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 3 The components shown.
[0068] Communication component 33 is configured to facilitate wired or wireless communication between electronic devices and other devices. The electronic devices can access wireless networks based on communication standards, such as WiFi, 3G, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 33 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 33 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0069] Power supply component 34 provides power to various components of the electronic device. Power supply component 34 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.
[0070] Audio component 35 is configured to output and / or input audio signals. For example, audio component 35 includes a microphone (MIC) configured to receive external audio signals when the electronic device is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 31 or transmitted via communication component 33. In some embodiments, audio component 35 also includes a speaker for outputting audio signals.
[0071] Display 36 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.
[0072] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A point cloud alignment method based on adaptive voxels, characterized in that, include: Two point cloud frames that need to be aligned are obtained. The two point cloud frames include a first point cloud frame based on a first coordinate system and a second point cloud frame based on a second coordinate system, wherein the first coordinate system is different from the second coordinate system. The first voxel of the first point cloud frame and the second voxel of the second point cloud frame are determined based on the point cloud density of the first point cloud frame and the second point cloud frame, so as to divide the first point cloud frame and the second point cloud frame into multiple first point cloud voxel units and multiple second point cloud voxel units, respectively. Using the KD tree algorithm, for each first point cloud voxel unit and second point cloud voxel unit, identify the first point cloud frame voxel unit and second point cloud frame voxel unit with planar distribution characteristics, and extract their planar features. The planar features of the first point cloud frame voxel unit and the second point cloud frame voxel unit are fused respectively to generate the first point cloud voxel feature and the second point cloud voxel feature. Based on the first point cloud voxel features and the second point cloud voxel features, calculate the alignment rotation matrix between the first point cloud frame and the second point cloud frame. Using the alignment and rotation matrix, the first point cloud frame is transformed into the second coordinate system of the second point cloud frame.
2. The point cloud alignment method based on adaptive voxels according to claim 1, characterized in that, The step of determining the first voxel of the first point cloud frame and the second voxel of the second point cloud frame based on the point cloud density of the first point cloud frame and the second point cloud frame includes: For the first point cloud frame and the second point cloud frame, they are divided into multiple first point cloud frame voxel units and multiple second point cloud frame voxel units according to a predetermined first voxel size and a predetermined second voxel size, respectively. For each of the plurality of first point cloud frame voxel units and the plurality of second point cloud frame voxel units, the local point cloud density in that voxel unit is calculated using the following formula. in, It is the number of point clouds contained in the i-th voxel unit. It is the volume of the i-th voxel unit; Based on local point cloud density The adjustment factor is calculated for each voxel using the following formula. in, It is a positive parameter used to control the sensitivity of voxel size to the local point cloud density; The adjusted voxel size for each voxel unit is calculated using the following formula. Where represents the side length of the i-th voxel unit, and D is the diagonal length of the smallest rectangle containing the entire voxel unit point cloud, and D is calculated using the following formula. in, , , , , and These are the maximum and minimum values of a point in a voxel element along the x, y, and z axes of the coordinate system containing that voxel element.
3. The point cloud alignment method based on adaptive voxels according to claim 2, characterized in that, The KD-tree algorithm is used to identify first and second point cloud voxel units with planar distribution characteristics for each first and second point cloud voxel unit, and to extract their planar features, including: For the point cloud set within the i-th voxel unit Calculate its covariance matrix using the following formula. ,in, It is the centroid of the point cloud within the i-th voxel unit; Calculate the covariance matrix using the following formula. eigenvalues and its eigenvectors Among them, eigenvalues Let represent the dispersion of the point cloud within the i-th voxel in different directions, and its eigenvectors. This indicates the main distribution direction of the point cloud within the i-th voxel unit; The eigenvalues within the i-th voxel unit The eigenvector corresponding to the smallest eigenvalue in the i-th voxel is determined as the normal vector of the plane corresponding to the point cloud in the i-th voxel.
4. The point cloud alignment method based on adaptive voxels according to claim 3, characterized in that, The step of fusing the planar features of the first point cloud frame voxel unit and the second point cloud frame voxel unit respectively to generate the first point cloud voxel feature and the second point cloud voxel feature includes: For the first point cloud frame and the second point cloud frame voxel units, respectively calculate the distance between the centroid planes of the planes corresponding to each first point cloud frame voxel unit and the distance between the centroid planes of the planes corresponding to each second point cloud frame voxel unit. Using the following formulas, based on the distance between the normal vector of the plane corresponding to each first point cloud frame voxel unit and the centroid plane, and based on the distance between the normal vector of the plane corresponding to each second point cloud frame voxel unit and the centroid plane, the planes to be fused in the first and second point cloud frame voxel units are determined to form multiple sets of planes to be fused. , where represents The normal vector of the i-th plane, and and Let these be the centroid planes of two adjacent planes t and s, respectively. This represents the fusion threshold used to determine the planes that need to be merged. For each group of planes to be fused in the first point cloud frame, a plane is determined from the group of planes to be fused as the first plane of each group of planes to be fused in the first point cloud frame, and the eigenvalues and eigenvectors of the determined plane are used as the first point cloud voxel features of the group of planes to be fused. For each group of planes to be fused in the second point cloud frame, a plane is determined from the group of planes to be fused as the second plane of each group of planes to be fused in the second point cloud frame, and the eigenvalues and eigenvectors of the determined plane are used as the second point cloud voxel features of the group of planes to be fused.
5. The point cloud alignment method based on adaptive voxels according to claim 4, characterized in that, The step of calculating the alignment rotation matrix between the first point cloud frame and the second point cloud frame based on the first point cloud voxel features and the second point cloud voxel features includes: Using the first planes determined in each group of planes to be fused in the first point cloud frame, a target plane corresponding to the planes in the second point cloud frame is determined, wherein the target plane is the second plane determined in each group of planes to be fused in the second point cloud frame; Calculate the rotation axis r from the first plane to the second plane using the following formula. in, and These are the normal vectors of the two first planes, and These are the normal vectors of the two second planes, It is an intermediate variable used to calculate the rotation axis r. Calculate the rotation angle using the following formula. , in, It is the projection vector. and They are orthogonal vectors, and Indicates inner product operation; The rotation matrix R is obtained using the following formula. Where I is the identity matrix, It is the tensor product of the rotation axes, and It is an antisymmetric matrix.
6. The point cloud alignment method based on adaptive voxels according to claim 5, characterized in that, The step of calculating the alignment rotation matrix between the first point cloud frame and the second point cloud frame based on the first point cloud voxel features and the second point cloud voxel features further includes: Select a third plane from a plurality of first planes, the third plane being related to the first plane. and Different and their normal vectors With the first plane and With a large angle difference, the geometric constraints are calculated using the following formula. in, It is the threshold for determining the third plane; Calculate the translation vector T from the first plane to the second plane using the following formula. Where x is an intermediate variable, A is the normal vector matrix, and d is the planar distance vector, and .
7. A point cloud alignment device based on adaptive voxels, characterized in that, include: The acquisition module is used to acquire two point cloud frames that need to be aligned. The two point cloud frames include a first point cloud frame based on a first coordinate system and a second point cloud frame based on a second coordinate system, wherein the first coordinate system is different from the second coordinate system. The first determining module is used to determine the first voxel of the first point cloud frame and the second voxel of the second point cloud frame based on the point cloud density of the first point cloud frame and the second point cloud frame, so as to divide the first point cloud frame and the second point cloud frame into multiple first point cloud voxel units and multiple second point cloud voxel units respectively. The first identification module is used to use the KD tree algorithm to identify first point cloud voxel units and second point cloud voxel units with planar distribution characteristics for each first point cloud voxel unit and second point cloud voxel unit, and extract their planar features. The fusion module is used to fuse the planar features of the first point cloud frame voxel unit and the second point cloud frame voxel unit respectively to generate the first point cloud voxel feature and the second point cloud voxel feature. The first calculation module is used to calculate the alignment rotation matrix between the first point cloud frame and the second point cloud frame based on the first point cloud voxel features and the second point cloud voxel features. The conversion module is used to convert the first point cloud frame to the second coordinate system of the second point cloud frame using the alignment and rotation matrix.
8. An electronic device, characterized in that, include: Memory, used to store programs; A processor for running the program stored in the memory to perform the adaptive voxel-based point cloud alignment method as described in any one of claims 1-6.