Point cloud fast compression method based on global optimal ICP algorithm

Abstract

The invention relates to a point cloud rapid compression method based on a global optimal ICP algorithm. The problem that a point cloud compression method in the prior art is high in calculation complexity and poor in compression precision is solved. The method comprises the following steps: S1, extracting feature points and performing a current frame action according to the number of the feature points; s2, setting an initial pose and a convergence condition; s3, judging a convergence result according to a calculation result and obtaining a fine registration relative transformation matrix; s4, obtaining new point cloud data through fine registration of the relative transformation matrix; and S5, checking point cloud data and evaluating similarity. The method has the advantages that the point cloud compression ratio is higher, and the matching success rate of the point cloud macro block is improved; and the calculation complexity of motion estimation is reduced by using a global ICP algorithm, and the calculation time consumption is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a point cloud fast compression method based on a global optimal ICP algorithm. Background Art

[0002] With the development of technologies such as 3D reconstruction and SLAM, the compression of point cloud data has become particularly important. The basic idea of ​​point cloud compression is to reduce redundant information and achieve efficient compression by removing redundancy in space, time and attributes. Through point cloud inter-frame compression technology, the storage density of point cloud data can be reduced, so that the amount of point cloud data that needs to be stored is greatly reduced, achieving the purpose of efficient compression. Point clouds include a large amount of data, and the storage and transmission of these point clouds are costly and time-consuming; in addition, the traditional point cloud compression method uses the iterative closest point ICP algorithm. Although it shows good performance in registration, it still faces many problems due to its iterative processing method and the non-convexity of the problem to be solved:

[0003] 1. As an iterative algorithm, the computational complexity of the ICP algorithm is relatively high. Especially when processing large-scale point clouds, computational efficiency is a bottleneck. In an environment with high noise and many outliers, the performance of the ICP algorithm will be greatly affected by the presence of interference points.

[0004] 2. The ICP algorithm requires a good initial value as the starting condition for iteration, otherwise it is easy to fall into the local minimum. The calculated solution may be far away from the true solution, resulting in a decrease in accuracy and an increase in time during the point cloud compression process. In addition, the program tracking may fail due to incorrect transformation matrix estimation, which limits its application in robot operating systems.

[0005] It can be seen that in the existing technology, the point cloud compression method still has the problems of high computational complexity and poor compression accuracy.

[0006] In order to solve the deficiencies in the prior art, people have conducted long-term explorations and proposed a variety of solutions. For example, a Chinese patent document discloses a point cloud geometry compression [CN201880057814.5], which includes multiple points, each point including spatial information for the point. The encoder is configured to sub-sample the points and determine the subdivision positions for the sub-sampled points. In addition, the encoder is configured to determine whether to include points, exclude points, or reposition points relative to the subdivision positions for the corresponding subdivision positions. The encoder encodes the spatial information for the sub-sampling points and encodes the inclusion / repositioning information of the subdivision position points to generate a compressed point cloud. The decoder recreates an original or approximate copy of the original point cloud based on the spatial information included in the compressed point cloud and the inclusion / repositioning information of the subdivision position.

[0007] The above scheme solves the problems of large data volume, high cost and time-consuming in the existing point cloud compression technology to a certain extent. However, the scheme still has many shortcomings, such as high computational complexity and poor compression accuracy of the point cloud compression method. Summary of the invention

[0008] The purpose of the present invention is to provide a point cloud fast compression method based on the global optimal ICP algorithm to improve the speed of inter-frame motion estimation and reduce the point cloud compression time in response to the above problems.

[0009] To achieve the above object, the present invention adopts the following technical scheme: a point cloud fast compression method based on the global optimal ICP algorithm, the method comprising the following steps:

[0010] S1, feature point extraction and current frame action according to the number of feature points;

[0011] S2, set the initial pose and convergence conditions;

[0012] S3, judging the convergence result according to the calculation result and obtaining the fine registration relative transformation matrix;

[0013] S4, obtaining new point cloud data by fine registration of relative transformation matrix;

[0014] S5. Point cloud data inspection and similarity assessment.

[0015] In the above-mentioned point cloud fast compression method based on the global optimal ICP algorithm, step S1 specifically includes the following steps:

[0016] S11, using the key frame, predicted frame and IMU original data of the 3D point cloud as the original input point cloud data;

[0017] S12, extracting ORB feature points of the key frame and the predicted frame 2D image by using an ORB feature point extraction algorithm.

[0018] In step S1, the current frame action judgment includes the following contents:

[0019] ①Pre-set the threshold range of the number of feature points;

[0020] ② If the number of ORB feature points of the two frames is less than the preset threshold range of the number of feature points and does not meet the threshold condition, it is determined that the current frame cannot be encoded, and the algorithm jumps to compressing the key frame;

[0021] ③ If the number of ORB feature points of the two frames is within the pre-set threshold range of the number of feature points and meets the threshold condition, it is determined that the current frame can be encoded, and the ORB feature points of the 2D image are mapped to the 3D space according to the camera internal parameters.

[0022] In step S2, the relative position of the camera of the IMU raw data is obtained and set as the initial position of the global optimal ICP algorithm; at the same time, the maximum number of iterations and the mean square error of the global optimal ICP algorithm are set as convergence conditions.

[0023] In step S3, the convergence condition judgment includes the following contents:

[0024] S31. In 3D space, use the global optimal ICP algorithm to calculate the relative pose of the three-dimensional ORB features of the two-frame point cloud;

[0025] S32, if the global optimal ICP calculation result converges, then obtain the coarse registration relative transformation matrix of the key frame and the current frame;

[0026] S33, substituting the coarse registration relative transformation matrix of the current frame as the initial matrix into the ICP algorithm, performing motion estimation on all point cloud data, and obtaining the fine registration relative transformation matrix between the two frames.

[0027] S34. If the calculation result does not converge in the global optimal ICP algorithm, the algorithm jumps to key frame compression. If all calculations are completed, the fine registration relative transformation matrix is ​​obtained, and the algorithm jumps to prediction frame compression.

[0028] In the above-mentioned point cloud fast compression method based on the global optimal ICP algorithm, step S4 specifically includes the following steps:

[0029] S41, integrating the precise registration transformation matrix T into the predicted frame part Ppart;

[0030] S42, using the matrix T to transform the predicted frame point cloud Pcloud so as to align it with the key frame point cloud Icloud;

[0031] S43. Obtain a new point cloud representation Pcloud.

[0032] In step S4, an octree data structure is used to store the key frame point cloud Icloud and the transformed predicted frame point cloud Pcloud.

[0033] In step S4, the minimum leaf node resolution of the octree is set to M, and the point cloud is divided into macroblocks of M×M×M size according to the octree.

[0034] In step S5, each macroblock in the converted prediction frame point cloud Pcloud is checked to determine whether there is a matching macroblock in the key frame point cloud Icloud. If there is a matching macroblock, a similarity evaluation is performed.

[0035] In the above-mentioned point cloud fast compression method based on the global optimal ICP algorithm, the similarity evaluation specifically includes the following contents:

[0036] S51, if the similarity between the two macroblocks meets the preset condition, it is considered that the macroblock of the prediction frame can be described by the position of the key frame macroblock,

[0037] S52. Record the index Key of the key frame macroblock in the prediction frame part Ppart.

[0038] S53: If the similarity between the two macroblocks does not meet the preset condition or there is no corresponding macroblock, the original data of the predicted frame macroblock is stored in the unpredictable part Ipart.

[0039] Compared with the existing technology, the advantages of the present invention are: in terms of compression efficiency, a higher point cloud compression ratio of up to 56.34 is achieved, by extracting ORB feature points and using IMU to estimate the camera posture, using an improved ICP algorithm to perform motion estimation on the point cloud, and transforming the point cloud to the same coordinate system before performing similarity detection, thereby improving the matching success rate of point cloud macroblocks; by introducing 2D images and IMU data, using IMU to estimate the camera posture, and using a global ICP algorithm, the computational complexity of motion estimation is reduced, and the computational time is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Flowchart of the motion estimation algorithm in the present invention;

[0041] Figure 2 This is a flow chart of the prediction frame compression program in the present invention. DETAILED DESCRIPTION

[0042] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] like Figure 1-2 As shown, a point cloud fast compression method based on the global optimal ICP algorithm includes the following steps:

[0044] S1, feature point extraction and current frame action according to the number of feature points;

[0045] S2, set the initial pose and convergence conditions;

[0046] S3, judging the convergence result according to the calculation result and obtaining the fine registration relative transformation matrix;

[0047] S4, obtaining new point cloud data by fine registration of relative transformation matrix;

[0048] S5. Point cloud data inspection and similarity assessment.

[0049] Wherein, step S1 specifically includes the following steps:

[0050] S11, using the key frame, predicted frame and IMU original data of the 3D point cloud as the original input point cloud data;

[0051] S12, extracting ORB feature points of the key frame and the predicted frame 2D image by using an ORB feature point extraction algorithm.

[0052] In step S1, the current frame action judgment includes the following contents:

[0053] ①Pre-set the threshold range of the number of feature points;

[0054] ② If the number of ORB feature points of the two frames is less than the preset threshold range of the number of feature points and does not meet the threshold condition, it is determined that the current frame cannot be encoded, and the algorithm jumps to compressing the key frame;

[0055] ③ If the number of ORB feature points of the two frames is within the pre-set threshold range of the number of feature points and meets the threshold condition, it is determined that the current frame can be encoded, and the ORB feature points of the 2D image are mapped to the 3D space according to the camera internal parameters.

[0056] In step S2, the relative position of the camera of the IMU raw data is obtained and set as the initial position of the global optimal ICP algorithm; at the same time, the maximum number of iterations and the mean square error of the global optimal ICP algorithm are set as convergence conditions.

[0057] In step S3, the convergence condition judgment includes the following contents:

[0058] S31. In 3D space, use the global optimal ICP algorithm to calculate the relative pose of the three-dimensional ORB features of the two-frame point cloud;

[0059] S32, if the global optimal ICP calculation result converges, then obtain the coarse registration relative transformation matrix of the key frame and the current frame;

[0060] S33, substituting the coarse registration relative transformation matrix of the current frame as the initial matrix into the ICP algorithm, performing motion estimation on all point cloud data, and obtaining the fine registration relative transformation matrix between the two frames.

[0061] S34. If the calculation result does not converge in the global optimal ICP algorithm, the algorithm jumps to key frame compression. If all calculations are completed, the fine registration relative transformation matrix is ​​obtained, and the algorithm jumps to prediction frame compression.

[0062] Step S4 specifically includes the following steps:

[0063] S41, integrating the precise registration transformation matrix T into the predicted frame part Ppart;

[0064] S42, using the matrix T to transform the predicted frame point cloud Pcloud so as to align it with the key frame point cloud Icloud;

[0065] S43. Obtain a new point cloud representation Pcloud.

[0066] In step S4, an octree data structure is used to store the key frame point cloud Icloud and the transformed predicted frame point cloud Pcloud.

[0067] In step S4, the minimum leaf node resolution of the octree is set to M, and the point cloud is divided into macroblocks of M×M×M size according to the octree.

[0068] In step S5, each macroblock in the converted prediction frame point cloud Pcloud is checked to determine whether there is a matching macroblock in the key frame point cloud Icloud. If there is a matching macroblock, a similarity evaluation is performed.

[0069] Similarity assessment specifically includes the following:

[0070] S51, if the similarity between the two macroblocks meets the preset condition, it is considered that the macroblock of the prediction frame can be described by the position of the key frame macroblock,

[0071] S52. Record the index Key of the key frame macroblock in the prediction frame part Ppart.

[0072] S53: If the similarity between the two macroblocks does not meet the preset condition or there is no corresponding macroblock, the original data of the predicted frame macroblock is stored in the unpredictable part Ipart.

[0073] Example

[0074] The present invention is applied to a mobile robot device, and a Realsense stereo camera capable of capturing 3D point clouds is mounted on the main body of the mobile robot device, and a six-axis IMU module capable of outputting acceleration and angular acceleration is mounted on the chassis of the mobile robot device, and both send data to an industrial computer via Ethernet.

[0075] The principle of this embodiment is:

[0076] The present invention controls a mobile robot device to move indoors, reads a 2D-RGB image synchronized with a point cloud frame from a stereo camera, and then extracts feature points of a prediction frame and a key frame RGB image through an ORB feature point extraction algorithm. If the number of ORB feature points of the two frames is small and does not meet a threshold condition, it is determined that the prediction frame cannot be encoded, and the prediction frame is compressed as a key frame; otherwise, the currently extracted ORB feature points are mapped to a three-dimensional coordinate system through depth data in a depth map.

[0077] The six-axis IMU data on the 3D camera is processed by the Kalman filter to predict the change in the camera posture. The IMU and the stereo camera are fixed rigidly connected and represent the posture of the stereo camera. By calculating the change in the posture of the IMU within two frames of point cloud time, the approximate relative posture of the two frames of point cloud is obtained, and this posture is used as the initial posture of the ICP coarse registration algorithm to provide better initial conditions for the registration process.

[0078] There are usually some outliers in the experimental data, which will greatly affect the performance of the ICP algorithm. Therefore, the outliers need to be pruned. Here, a simple pruning method is adopted, namely the introspection selection algorithm, which thoroughly sorts the residuals and uses the K smallest residuals.

[0079] Set the maximum number of iterations and mean square error of the global optimal ICP algorithm, and divide the search space into multiple sub-regions. Calculate a boundary for each sub-region. Calculate the upper and lower bounds of each sub-region, evaluate the priority of the sub-region based on the upper and lower bounds, and search the most promising region first.

[0080] In the selected sub-region, the ICP algorithm is used to perform local search to refine the transformation matrix, and the transformation matrix is ​​iteratively updated until the preset convergence condition or the maximum number of iterations is reached.

[0081] If the calculation result of the global optimal ICP algorithm converges, the coarse registration transformation matrix between the key frame and the predicted frame can be obtained, otherwise, the predicted frame is compressed.

[0082] The transformation matrix obtained by coarse matching is used as the initial value of the ICP fine registration algorithm. ICP registration is performed on all point clouds of the predicted frame and the key frame, and the transformation matrix of the fine registration of the two is calculated. By multiplying the transformation matrix on the point cloud of the previous frame and superimposing it with the point cloud of the next frame, the point cloud registration effect can be intuitively seen.

[0083] Finally, the point cloud is divided into multiple groups of macroblocks using the octree structure, the successfully matched macroblocks are replaced with the position index of the macroblock for compression, and the compressed point cloud data is transmitted to the operator's HoloLens for rendering, thereby obtaining an image that the user can directly view.

[0084] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A point cloud fast compression method based on the global optimal ICP algorithm, characterized in that: The method comprises the following steps: S1, feature point extraction and current frame action according to the number of feature points; S2, set the initial pose and convergence conditions; S3, judging the convergence result according to the calculation result and obtaining the fine registration relative transformation matrix; S4, obtaining new point cloud data by fine registration of relative transformation matrix; S5. Point cloud data inspection and similarity assessment.

2. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11, using the key frame, predicted frame and IMU original data of the 3D point cloud as the original input point cloud data; S12, extracting ORB feature points of the key frame and the predicted frame 2D image by using an ORB feature point extraction algorithm.

3. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 2 is characterized in that: In step S1, the current frame action judgment includes the following contents: ①Pre-set the threshold range of the number of feature points; ② If the number of ORB feature points of the two frames is less than the preset threshold range of the number of feature points and does not meet the threshold condition, it is determined that the current frame cannot be encoded, and the algorithm jumps to compressing the key frame; ③ If the number of ORB feature points of the two frames is within the pre-set threshold range of the number of feature points and meets the threshold condition, it is determined that the current frame can be encoded, and the ORB feature points of the 2D image are mapped to the 3D space according to the camera internal parameters.

4. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 2, characterized in that: In step S2, the relative position of the camera of the IMU raw data is obtained and set as the initial position of the global optimal ICP algorithm; at the same time, the maximum number of iterations and the mean square error of the global optimal ICP algorithm are set as convergence conditions.

5. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 4, characterized in that: In step S3, the convergence condition judgment includes the following contents: S31. In 3D space, use the global optimal ICP algorithm to calculate the relative pose of the three-dimensional ORB features of the two-frame point cloud; S32, if the global optimal ICP calculation result converges, then obtain the coarse registration relative transformation matrix of the key frame and the current frame; S33, substituting the coarse registration relative transformation matrix of the current frame as the initial matrix into the ICP algorithm, performing motion estimation on all point cloud data, and obtaining the fine registration relative transformation matrix between the two frames. S34. If the calculation result does not converge in the global optimal ICP algorithm, the algorithm jumps to key frame compression. If all calculations are completed, the fine registration relative transformation matrix is ​​obtained, and the algorithm jumps to prediction frame compression.

6. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 5, characterized in that: Step S4 specifically includes the following steps: S41, integrating the precise registration transformation matrix T into the predicted frame part Ppart; S42, using the matrix T to transform the predicted frame point cloud Pcloud so as to align it with the key frame point cloud Icloud; S43. Obtain a new point cloud representation Pcloud.

7. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 6, characterized in that: In step S4, an octree data structure is used to store the key frame point cloud Icloud and the transformed predicted frame point cloud Pcloud.

8. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 7, characterized in that: In step S4, the minimum leaf node resolution of the octree is set to M, and the point cloud is divided into macroblocks of M×M×M size according to the octree.

9. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 8, characterized in that: In step S5, each macroblock in the converted prediction frame point cloud Pcloud is checked to determine whether there is a matching macroblock in the key frame point cloud Icloud. If there is a matching macroblock, a similarity evaluation is performed.

10. The point cloud fast compression method based on the global optimal ICP algorithm according to claim 9, characterized in that: The similarity assessment specifically includes the following contents: S51, if the similarity between the two macroblocks meets the preset condition, it is considered that the macroblock of the prediction frame can be described by the position of the key frame macroblock, S52. Record the index Key of the key frame macroblock in the prediction frame part Ppart. S53: If the similarity between the two macroblocks does not meet the preset condition or there is no corresponding macroblock, the original data of the predicted frame macroblock is stored in the unpredictable part Ipart.

Images