A method and system for fast global motion matching of dynamic point clouds based on centroid search
Through the dynamic point cloud fast global motion matching method based on centroid point search, the problem of high computational complexity of global motion matrix in motion estimation of three-dimensional point cloud data is solved, and the encoding time complexity is reduced and encoding efficiency is improved.
Patent Information
- Application Number
- CN202210144056.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-17
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-02-17
AI Technical Summary
During the motion estimation process of three-dimensional point cloud data, the global motion matrix has high computational complexity and the calculation accuracy is affected.
The dynamic point cloud fast global motion matching method based on centroid point search is adopted. By extracting centroid point and searching the point cloud data, the complexity of point cloud matching is reduced, and global motion matrix calculation is applied in the LMS algorithm.
This significantly reduces the encoding time complexity, at least 58%, and slightly reduces the rate distortion performance, improving the encoding efficiency of point cloud sequences.
Smart Images

Figure CN114549870B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a motion estimation method and system in inter-frame prediction coding of three-dimensional point cloud data, belonging to the technical field of image processing. Background Art
[0002] The use of 3D point cloud data has begun to be popularized in various fields. For example, virtual / augmented reality, robotics, geographic information systems, and the medical field. With the development of 3D point cloud data and the continuous improvement of the accuracy and speed of 3D point cloud data scanning equipment (such as laser radar, Kinect body sensing camera), people can accurately obtain a large amount of point cloud data on the surface of objects. Often, the point cloud data in a scene can include hundreds of thousands of points. Such a large amount of point cloud data also brings challenges to computer storage and transmission. Therefore, the compression of 3D point cloud data has become a hot issue.
[0003] According to different application scenarios, point clouds can be divided into three types: static point clouds, dynamic point clouds, and dynamically acquired point clouds. MPEG (Moving Pictures Experts Group) has developed two point cloud compression technology standards: video-based 3D point cloud compression technology (V-PCC) for dynamic point clouds and geometry-based 3D point cloud compression technology (G-PCC) for static point clouds and dynamically acquired point clouds. Among them, V-PCC encodes by projecting 3D information onto a 2D plane, while G-PCC encodes directly in 3D space. Inter-EM (inter-exploration model) under the G-PCC coding framework is a technical standard mainly for encoding dynamically acquired point clouds. There is a lot of temporal redundancy between dynamically acquired point clouds, so motion estimation becomes an indispensable key technology in Inter-EM.
[0004] In TMC13v12 Inter-EM v 2.0, the motion estimation process is divided into the following six steps: In the first step, the current frame point cloud is first divided into road points and object points by two thresholds top_z_boundary and bottom_z_boundary on the z-axis. If the z value of the point cloud is greater than top_z_boundary or less than bottom_z_boundary, it is classified as an object point, otherwise it is classified as a road point; In the second step, the reference frame corresponding to the current frame searches for the associated points corresponding to all object points, and puts the object points with corresponding associated points into pc_likely_world; In the third step, pc_likely_world is sampled equidistantly to obtain 2000 points, and these points are put into pc_world_target; In the fourth step, the corresponding point pc_world_ref of pc_world_target is found in the reference frame; In the fifth step, the LMS algorithm is applied to pc_world_target and pc_world_ref to calculate the global motion matrix (Mat_GM); In the sixth step, Mat_GM is applied to the object points of the reference frame point cloud, and the motion estimation process is completed. When using the LMS algorithm to calculate the global motion matrix, each time the 2000 points in pc_world_target and the 2000 points in pc_world_ref need to be operated, which will cause two major problems: First, a large number of points will significantly increase the calculation complexity of Mat_GM. Second, pc_world_target contains a large number of redundant points, which will affect the calculation accuracy of Mat_GM. Summary of the invention
[0005] Aiming at the shortcoming of high computational complexity of global motion matrix in motion estimation process, the present invention proposes a fast global motion matching method of dynamic point cloud based on centroid search;
[0006] The present invention also provides a dynamic point cloud fast global motion matching system based on centroid point search.
[0007] Terminology explanation:
[0008] 1. Point cloud: refers to a set of vectors in a three-dimensional coordinate system. Point cloud data generally contains location information (X, Y, Z) and attribute information such as color, reflectivity and normal vector.
[0009] 2. Inter-EM v2.0 Software: According to different application scenarios, point clouds can be divided into three types: static point clouds, dynamic point clouds, and point clouds obtained dynamically. The Moving Pictures Experts Group (MPEG) has developed two point cloud compression technology standards: video-based 3D point cloud compression technology (V-PCC) for dynamic point clouds and geometry-based 3D point cloud compression technology (G-PCC) for static point clouds and point clouds obtained dynamically. Inter-EM v2.0 software is a tool for inter-frame prediction in the G-PCC framework. Inter-EM v2.0 software can use the information of previously encoded frames to predict the current frame information.
[0010] 3. kd-tree Search: A kd-tree (K-dimensional tree) is a binary tree that stores some K-dimensional data. Constructing a kd-tree on a K-dimensional data set represents a partition of the K-dimensional space formed by the data set, that is, each node in the tree corresponds to a K-dimensional hyper-rectangular region. Kd-tree search is a search algorithm based on the kd-tree. Before introducing the kd-tree search, the construction algorithm of the kd-tree is briefly introduced as follows:
[0011] (1) Select the dimension K with the largest variance in the K-dimensional data set, and then select the median m on this dimension as the benchmark to partition the data set, obtaining two subsets; at the same time, create a tree node to store the data.
[0012] (2) Repeat the process of step (1) for the two subsets until all subsets can no longer be partitioned; if a subset can no longer be partitioned, save the data in the subset to a leaf node.
[0013] After constructing the kd-tree, the kd-tree search algorithm is introduced as follows:
[0014] (1) Starting from the root node with the query data Q, access the kd-tree downward according to the comparison results between Q and each node until reaching a leaf node. Among them, the comparison between Q and a node refers to comparing the value of Q corresponding to the K dimension in the node with m. If Q(k) < m, access the left subtree; otherwise, access the right subtree. When reaching a leaf node, calculate the distance between Q and the data saved on the leaf node, record the data point corresponding to the minimum distance, denoted as the current "nearest neighbor point" Pcur and the minimum distance Dcur.
[0015] (2) Perform a backtracking operation to find the "nearest neighbor" that is closer to Q. That is, determine whether there are any points closer to Q in the unvisited branches, and the distance between them is less than Dcur. If the distance between Q and the unvisited branches under its parent node is less than Dcur, it is considered that there is data closer to Q in the branch, and the node is entered to perform the same search process as step (1). If a closer data point is found, it is updated to the current "nearest neighbor" Pcur, and Dcur is updated. If the distance between Q and the unvisited branches under its parent node is greater than Dcur, it means that there is no point closer to Q in the branch. The backtracking judgment process is performed from bottom to top until there is no branch closer to Q when backtracking to the root node. At this point, the kd-tree search algorithm ends.
[0016] The technical solution of the present invention is:
[0017] A method for fast global motion matching of dynamic point clouds based on centroid search includes the following steps:
[0018] Step 1: The point cloud of the current frame is divided into road points and object points by two thresholds top_z_boundary and bottom_z_boundary on the z-axis. If the z value of the point cloud is greater than top_z_boundary or less than bottom_z_boundary, it is classified as an object point, otherwise, it is classified as a road point;
[0019] Step 2: Search for the associated points corresponding to all object points in the reference frame corresponding to the current frame. The reference frame refers to the reconstructed point cloud frame output after a complete encoding and decoding process of the previous frame of point cloud. The associated points refer to the corresponding points of the points in the current frame in the reference frame. Put the object points with corresponding associated points into pc_likely_world;
[0020] Step 3: Sample pc_likely_world at equal intervals and put the sampled points into pc_world_target; specifically:
[0021] Customize an array pc_world_target_temp, put the points obtained by equidistant downsampling of pc_likely_world into pc_world_target_temp, divide pc_world_target_temp into blocks, keep only the centroid points in each block, and put these centroid points into pc_world_target;
[0022] Step 4: Find the corresponding point pc_world_ref of pc_world_target in the reference frame;
[0023] Step 5: Apply the LMS algorithm to pc_world_target and pc_world_ref to calculate the global motion matrix;
[0024] Step 6: Apply Mat_GM to the object points of the reference frame point cloud. At this point, the motion estimation process is completed.
[0025] Preferably, according to the present invention, the specific implementation process of step 2 includes:
[0026] Step 2.1: Use Inter-EMv2.0 software to create a cubic bounding box for the point cloud data of the current frame. This cubic bounding box encloses all the point cloud data of the current frame and divides the cubic bounding box into small cubes of motion_block_size×motion_block_size×motion_block_size.
[0027] Through Inter-EMv2.0 software, a cubic bounding box is established for the point cloud data of the reference frame. This cubic bounding box encloses all the point cloud data of the reference frame, and the cubic bounding box is divided into small cubes of (motion_block_size+th_dist)×(motion_block_size+th_dist)×(motion_block_size+th_dist); where motion_block_size and th_dist are parameters of Inter-EMv2.0 software, motion_block_size=512, th_dist=31;
[0028] Step 2.2: Traverse each small cube in the current frame in turn. If there is a point in the small cube M of the current frame, find the corresponding small cube M of the small cube in the reference frame. ref , if M ref There are also points inside, then the points in M have associated points in the reference frame, otherwise it is considered that the points in M have no associated points in the reference frame.
[0029] According to the preferred embodiment of the present invention, in step 3, pc_world_target_temp is divided into small cubes, specifically referring to:
[0030] Find the smallest and largest components of the three components of the position information (x, y, z) in pc_world_target_temp to form the minimum point: bbox.min = (min(x i ) i=0…N-1 ,min(y i )i=0…N-1 ,min(z i ) i=0…N-1 ) and maximum point: bbox.max = (max(x i ) i=0…N-1 ,max(y i ) i=0…N-1 ,max(z i ) i=0…N-1 ), where x i ,y i , z i Represents the location information of the i-th point, N is the number of points;
[0031] A cubic bounding box is established based on the minimum point and the maximum point to enclose the point cloud data in the entire array pc_world_target_temp. The cubic bounding box is divided into small cubes of motion_block_size×motion_block_size×motion_block_size starting from the minimum point coordinates, where motion_block_size is a parameter provided by the Inter-EMv2.0 software, motion_block_size=512.
[0032] According to the preferred embodiment of the present invention, in step 3, the centroid point in each small cube is found and put into pc_world_target, which specifically refers to:
[0033] ① If the number of point clouds in the small cube is less than 3, these points are discarded directly;
[0034] ② If the number of point clouds in the small cube is greater than or equal to 3, find the centroid point P in the small cube centroid ; Find the distance P through kd-tree search centroid The nearest point P c , and put P c Put it into pc_world_target;
[0035] Further preferably, the centroid point P centroid The calculation formula is shown in formula (I):
[0036]
[0037] In formula (I), X i is the position information of the midpoint of the small cube, and N is the number of midpoints of the small cube.
[0038] Preferably, according to the present invention, the specific implementation process of step 5 includes:
[0039] set up
[0040] Among them, x i ,y i , z i Represents the position information of the i-th point in pc_world_ref, x′ i , y′ i , z′ i Represents the position information of the i-th point in pc_world_target, N is the number of points, MV_unity is an adaptive parameter;
[0041] Define the correlation matrix M,
[0042] The inverse matrix of M is obtained by Gaussian elimination and converted into a Hermite matrix to obtain M -1 ;
[0043] Define the transformation matrix R,
[0044] Define the transformation matrix T, T1,
[0045] T1=M -1 R,
[0046]
[0047] At this time, the third row data T1[3] of the matrix T1 is multiplied by the parameter MV_unity to obtain T[3], that is, T[3]=MV_unity×T1[3];
[0048] At the first iteration, the initial value of Mat_GM is:
[0049] Define the intermediate variable Mat_GML, Mat_GML = Mat_GM;
[0050] Inter-EMv2.0 defines the number of iterations NLMS=10, which means that in the next iteration, the Mat_GM obtained in the previous iteration is assigned as the initial value to the intermediate variable Mat_GML;
[0051] Perform mathematical operations on Mat_GML and T to obtain Mat_GM. The specific process is as follows:
[0052] For the transformation part:
[0053] Set the initial value: l = 0; c = 0;
[0054] Mat_GM[l][c]=Mat_GML[l][0]×T[0][c]×Mat_GML[l][1]×T[1][c]+Mat_GML[l][2]×T[2][c];
[0055] L increases by 1, c increases by 1, and Mat_GM[l][c] is iterated until the iteration termination condition is reached, i.e., l<3 and c<3;
[0056] For the translation part:
[0057] Set the initial value: c = 0;
[0058] Mat_GM[3][c]=Mat_GML[3][0]×T[0][c]+Mat_GML[3][1]×T[1][c]+Mat_GML[3][2]×T[2][c]+T[3][c];
[0059] L increases by 1, c increases by 1, and Mat_GM[3][c] is iterated until the iteration termination condition is reached, that is, c<3.
[0060] According to the preferred embodiment of the present invention, in step 6,
[0061] P i [0] = Mat_GM[0][0] × B i [0]+Mat_GM[1][0]×B i [1]+Mat_GM[2][0]×B i [2]+Mat_GM[3][0]
[0062] P i [1] = Mat_GM[0][1] × B i [0]+Mat_GM[1][1]×B i [1]+Mat_GM[2][1]×B i [2]+Mat_GM[3][1]
[0063] P i [2] = Mat_GM[0][2] × B i [0]+Mat_GM[1][2]×B i [1]+Mat_GM[2][2]×B i [2]+Mat_GM[3][2]
[0064] Among them, B i is the position information of the object point in the reference frame point cloud, 0≤i≤N, N is the number of object points in the reference frame point cloud; B i [0] is point B iThe x-axis information, B i [1] is point B i The y-axis information, B i [2] is point B i z-axis information; Mat_GM is a matrix with four rows and three columns, P i is the position information of the point obtained after applying Mat_GM, P i [0] is point P i The x-axis information, P i [1] is point P i The y-axis information, P i [2] is point P i The z-axis information.
[0065] A fast global motion matching system of dynamic point cloud based on centroid search, comprising a road point and object point division unit, a correlation point search unit, a centroid extraction unit, a global motion matrix calculation unit and a global motion matrix application unit;
[0066] The road point and object point division unit is used to execute step 1; the associated point search unit is used to execute step 2; the centroid point extraction unit is used to execute step 3; the global motion matrix calculation unit is used to execute steps 4 and 5; and the global motion matrix application unit is used to execute step 6.
[0067] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a method for fast global motion matching of dynamic point clouds based on centroid point search are implemented.
[0068] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for fast global motion matching of dynamic point clouds based on centroid point search.
[0069] The beneficial effects of the present invention are:
[0070] Compared with the original method, under the condition that the rate-distortion performance is slightly reduced, the encoding time of the present invention is reduced by at least 58%, thereby reducing the encoding time complexity of the point cloud sequence. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 It is a flow chart of the method for fast global motion matching of dynamic point clouds based on centroid point search of the present invention.
[0072] Figure 2 It is a schematic diagram of the centroid extraction process. DETAILED DESCRIPTION
[0073] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.
[0074] Example 1
[0075] A fast global motion matching method for dynamic point clouds based on centroid search, such as Figure 1 As shown, the steps include:
[0076] Step 1: The current frame point cloud is divided into road points and object points by two thresholds on the z-axis, top_z_boundary and bottom_z_boundary. Top_z_boundary and bottom_z_boundary are two thresholds on the z-axis calculated based on each frame of point cloud data, which are equivalent to two parameters of each frame of point cloud and are two fixed values. If the z value of the point cloud is greater than top_z_boundary or less than bottom_z_boundary, it is classified as an object point, otherwise, it is classified as a road point;
[0077] Step 2: Search for the associated points corresponding to all object points in the reference frame corresponding to the current frame. The reference frame refers to the reconstructed point cloud frame output after a complete encoding and decoding process of the previous frame of point cloud. The associated points refer to the corresponding points of the points in the current frame in the reference frame. Put the object points with corresponding associated points into pc_likely_world. pc_likely_world is a vector variable name defined in Inter-EMv2.0 software. Vector is a dynamic array in C++ syntax that can store any type of data. That is to say, pc_likely_world is the name of an array used to store point cloud data that meets the above requirements. The specific implementation process includes:
[0078] Step 2.1: Use Inter-EMv2.0 software to create a cubic bounding box for the point cloud data of the current frame. This cubic bounding box encloses all the point cloud data of the current frame and divides the cubic bounding box into small cubes of motion_block_size×motion_block_size×motion_block_size.
[0079] Through Inter-EMv2.0 software, a cubic bounding box is established for the point cloud data of the reference frame. This cubic bounding box encloses all the point cloud data of the reference frame, and the cubic bounding box is divided into small cubes of (motion_block_size+th_dist)×(motion_block_size+th_dist)×(motion_block_size+th_dist); where motion_block_size and th_dist are parameters of Inter-EMv2.0 software, motion_block_size=512, th_dist=31;
[0080] Step 2.2: Traverse each small cube in the current frame in turn. If there is a point in the small cube M of the current frame, find the corresponding small cube M of the small cube in the reference frame. ref (The correspondence here means the same position information), if M ref There are also points inside, then the points in M have associated points in the reference frame, otherwise it is considered that the points in M have no associated points in the reference frame.
[0081] Step 3: Sample pc_likely_world at equal intervals and put the sampled points (2000) into pc_world_target; pc_world_target is a vector variable name defined in Inter-EMv2.0 software. Vector is a dynamic array in C++ syntax that can store any type of data. That is to say, pc_world_target is the name of an array used to store the sampled point cloud data. Specifically:
[0082] Customize an array pc_world_target_temp, put the points obtained by equidistant downsampling of pc_likely_world into pc_world_target_temp, divide pc_world_target_temp into blocks, keep only the centroid points in each block, and put these centroid points into pc_world_target;
[0083] In step 3, pc_world_target_temp is divided into small cubes, specifically:
[0084] Find the smallest and largest components of the three components of the position information (x, y, z) in pc_world_target_temp to form the minimum point: bbox.min = (min(x i ) i=0…N-1 ,min(y i) i=0…N-1 ,min(z i ) i=0…N-1 ) and maximum point: bbox.max = (max(x i ) i=0…N-1 ,max(y i ) i=0…N-1 ,max(z i ) i=0…N-1 ), where x i ,y i , z i Represents the location information of the i-th point, N is the number of points;
[0085] A cubic bounding box is established based on the minimum point and the maximum point to enclose the point cloud data in the entire array pc_world_target_temp. The cubic bounding box is divided into small cubes of motion_block_size×motion_block_size×motion_block_size starting from the minimum point coordinates, where motion_block_size is a parameter provided by the Inter-EMv2.0 software, motion_block_size=512.
[0086] In step 3, find the center of mass of each small cube and put it into pc_world_target, specifically:
[0087] ① If the number of point clouds in the small cube is less than 3, these points are discarded directly;
[0088] ② If the number of point clouds in the small cube is greater than or equal to 3, find the centroid point P in the small cube centroid ; Note that there may not be a data point at the centroid position. Therefore, the distance P is found through kd-tree search centroid The nearest point P c , and put P c Put it into pc_world_target; the process is as follows Figure 2 shown.
[0089] Center of mass point P centroid The calculation formula is shown in formula (I):
[0090]
[0091] In formula (I), X i is the position information of the midpoint of the small cube, and N is the number of midpoints of the small cube. No other operations in Inter-EMv2.0 will be changed.
[0092] Step 4: Find the corresponding point pc_world_ref of pc_world_target in the reference frame;
[0093] Step 5: Apply the LMS algorithm to pc_world_target and pc_world_ref to calculate the global motion matrix (Mat_GM); the specific implementation process includes:
[0094] set up
[0095] Among them, x i ,y i , z i Represents the position information of the i-th point in pc_world_ref, x′ i , y′ i , z′ i Represents the position information of the i-th point in pc_world_target, N is the number of points, MV_unity is an adaptive parameter; each frame of the point cloud has its own fixed value.
[0096] Define the correlation matrix M,
[0097] The inverse matrix of M is obtained by Gaussian elimination and converted into a Hermite matrix to obtain M -1 ; The definition of a Hermite matrix is: if the transposed conjugate matrix of a matrix is equal to itself, then the matrix is a Hermite matrix.
[0098] Define the transformation matrix R,
[0099] Define the transformation matrix T, T1,
[0100] T1=M -1 R,
[0101]
[0102] At this time, the third row data T1[3] of the matrix T1 is multiplied by the parameter MV_unity to obtain T[3], that is, T[3]=MV_unity×T1[3];
[0103] At the first iteration, the initial value of Mat_GM is:
[0104] Define the intermediate variable Mat_GML, Mat_GML = Mat_GM;
[0105] Inter-EMv2.0 defines the number of iterations NLMS=10, which means that in the next iteration, the Mat_GM obtained in the previous iteration is assigned as the initial value to the intermediate variable Mat_GML;
[0106] Perform mathematical operations on Mat_GML and T to obtain Mat_GM. The specific process is as follows:
[0107] For the transformation part:
[0108] Set the initial value: l = 0; c = 0;
[0109] Mat_GM[l][c]=Mat_GML[l][0]×T[0][c]×Mat_GML[l][1]×T[1][c]+Mat_GML[l][2]×T[2][c];
[0110] L increases by 1, c increases by 1, and Mat_GM[l][c] is iterated until the iteration termination condition is reached, i.e., l<3 and c<3;
[0111] For the translation part:
[0112] Set the initial value: c = 0;
[0113] Mat_GM[3][c]=Mat_GML[3][0]×T[0][c]+Mat_GML[3][1]×T[1][c]+Mat_GML[3][2]×T[2][c]+T[3][c];
[0114] L increases by 1, c increases by 1, and Mat_GM[3][c] is iterated until the iteration termination condition is reached, that is, c<3.
[0115] Step 6: Apply Mat_GM to the object points of the reference frame point cloud. At this point, the motion estimation process ends.
[0116] P i [0] = Mat_GM[0][0] × B i [0]+Mat_GM[1][0]×B i [1]+Mat_GM[2][0]×B i [2]+Mat_GM[3][0]
[0117] P i [1] = Mat_GM[0][1] × B i [0]+Mat_GM[1][1]×B i [1]+Mat_GM[2][1]×B i [2]+Mat_GM[3][1]
[0118] P i [2] = Mat_GM[0][2] × B i [0]+Mat_GM[1][2]×B i [1]+Mat_GM[2][2]×B i [2]+Mat_GM[3][2]
[0119] Among them, B i is the position information of the object point in the reference frame point cloud, 0≤i≤N, N is the number of object points in the reference frame point cloud; B i [0] is point B i The x-axis information, B i [1] is point B i The y-axis information, B i [2] is point B i z-axis information; Mat_GM is a matrix with four rows and three columns, P i is the position information of the point obtained after applying Mat_GM, P i [0] is point P i The x-axis information, P i [1] is point P i The y-axis information, P i [2] is point P i The z-axis information.
[0120] The effect of the present invention can be further illustrated by experiments. Table 1 is a diagram comparing the encoding time complexity and rate distortion performance of the present invention and the original Inter-EMv2.0 method.
[0121] The experiment tests the coding complexity and rate-distortion performance of Cat3 sequence under the condition of lossy geometry-lossy attribute in TMC13v12Inter-EMv2.0 coding framework. Table 1 compares the coding time complexity and rate-distortion performance of the present invention and the original method in Inter-EMv2.0.
[0122] Table 1
[0123]
[0124]
[0125] It can be seen from Table 1 that, compared with the original method, under the condition that the rate-distortion performance is slightly reduced, the encoding time of the present invention is reduced by 58%, which means that the present invention reduces the encoding time complexity of the point cloud sequence.
[0126] Example 2
[0127] A fast global motion matching system of dynamic point cloud based on centroid search, comprising a road point and object point division unit, a correlation point search unit, a centroid extraction unit, a global motion matrix calculation unit and a global motion matrix application unit;
[0128] The road point and object point division unit is used to execute step 1; the associated point search unit is used to execute step 2; the centroid point extraction unit is used to execute step 3; the global motion matrix calculation unit is used to execute steps 4 and 5; and the global motion matrix application unit is used to execute step 6.
[0129] Example 3
[0130] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for fast global motion matching of dynamic point clouds based on centroid point search described in Example 1 are implemented.
[0131] Example 4
[0132] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for fast global motion matching of dynamic point clouds based on centroid point search described in Example 1.
Claims
1. A fast global motion matching method for dynamic point clouds based on centroid search, characterized in that: The steps include: Step 1: The point cloud of the current frame is divided into road points and object points by two thresholds top_z_boundary and bottom_z_boundary on the z-axis. If the z value of the point cloud is greater than top_z_boundary or less than bottom_z_boundary, it is classified as an object point, otherwise, it is classified as a road point; Step 2: Search for the associated points corresponding to all object points in the reference frame corresponding to the current frame. The reference frame refers to the reconstructed point cloud frame output after a complete encoding and decoding process of the previous frame of point cloud. The associated points refer to the corresponding points of the points in the current frame in the reference frame. Put the object points with corresponding associated points into pc_likely_world; Step 3: Sample pc_likely_world at equal intervals and put the sampled points into pc_world_target; specifically: Customize an array pc_world_target_temp, put the points obtained by equidistant downsampling of pc_likely_world into pc_world_target_temp, divide pc_world_target_temp into blocks, keep only the centroid points in each block, and put these centroid points into pc_world_target; Step 4: Find the corresponding point pc_world_ref of pc_world_target in the reference frame; Step 5: Apply the LMS algorithm to pc_world_target and pc_world_ref to calculate the global motion matrix; Step 6: Apply Mat_GM to the object points of the reference frame point cloud. At this point, the motion estimation process is completed.
2. The method for fast global motion matching of dynamic point clouds based on centroid search according to claim 1, characterized in that: The specific implementation process of step 2 includes: Step 2.1: Use Inter-EMv2.0 software to create a cubic bounding box for the point cloud data of the current frame. This cubic bounding box encloses all the point cloud data of the current frame and divides the cubic bounding box into small cubes of motion_block_size×motion_block_size×motion_block_size. Through Inter-EMv2.0 software, a cubic bounding box is established for the point cloud data of the reference frame. This cubic bounding box encloses all the point cloud data of the reference frame, and the cubic bounding box is divided into small cubes of (motion_block_size+th_dist)×(motion_block_size+th_dist)×(motion_block_size+th_dist); where motion_block_size and th_dist are parameters of Inter-EMv2.0 software, motion_block_size=512, th_dist=31; Step 2.2: Traverse each small cube in the current frame in turn. If there is a point in the small cube M of the current frame, find the corresponding small cube M of the small cube in the reference frame. ref , if M ref There are also points inside, then the points in M have associated points in the reference frame, otherwise it is considered that the points in M have no associated points in the reference frame.
3. The method for fast global motion matching of dynamic point clouds based on centroid search according to claim 1, characterized in that: In step 3, pc_world_target_temp is divided into small cubes, specifically: Find the smallest and largest components of the three components of the position information (x, y, z) in pc_world_target_temp to form the minimum point: bbox.min = (min(x i ) i=0…N-1 ,min(y i ) i=0…N-1 ,min(z i ) i=0…N-1 ) and maximum point: bbox.max = (max(x i ) i=0…N-1 ,max(y i ) i=0…N-1 ,max(z i ) i=0…N-1 ), where x i ,y i , z i Represents the location information of the i-th point, N is the number of points; A cubic bounding box is established based on the minimum point and the maximum point to enclose the point cloud data in the entire array pc_world_target_temp. The cubic bounding box is divided into small cubes of motion_block_size×motion_block_size×motion_block_size starting from the minimum point coordinates, where motion_block_size is a parameter provided by the Inter-EMv2.0 software, motion_block_size=512.
4. The method for fast global motion matching of dynamic point clouds based on centroid search according to claim 1, characterized in that: In step 3, find the center of mass of each small cube and put it into pc_world_target, specifically: ① If the number of point clouds in the small cube is less than 3, these points are discarded directly; ② If the number of point clouds in the small cube is greater than or equal to 3, find the centroid point P in the small cube centroid ; Find the distance P through kd-tree search centroid The nearest point P c , and put P c Place it in pc_world_target.
5. The method for fast global motion matching of dynamic point clouds based on centroid search according to claim 4, characterized in that: Center of mass point P centroid The calculation formula is shown in formula (I): In formula (I), X i is the position information of the midpoint of the small cube, and N is the number of midpoints of the small cube.
6. The method for fast global motion matching of dynamic point clouds based on centroid search according to claim 1, characterized in that: The specific implementation process of step 5 includes: Among them, x i ,y i , z i Indicates the position information of the i-th point in pc_world_ref, x i 3, y i 3, z i 3 represents the position information of the i-th point in pc_world_target, N is the number of points, MV_unity is an adaptive parameter; Define the correlation matrix M, The inverse matrix of M is obtained by Gaussian elimination and converted into a Hermite matrix to obtain M -1 ; Define the transformation matrix R, Define the transformation matrix T, T1, T1=M -1 R, At this time, the third row data T1[3] of the matrix T1 is multiplied by the parameter MV_unity to obtain T[3], that is, T[3]=MV_unity×T1[3]; At the first iteration, the initial value of Mat_GM is: Define the intermediate variable Mat_GML, Mat_GML = Mat_GM; Inter-EMv2.0 defines the number of iterations NLMS=10, which means that in the next iteration, the Mat_GM obtained in the previous iteration is assigned as the initial value to the intermediate variable Mat_GML; Perform mathematical operations on Mat_GML and T to obtain Mat_GM. The specific process is as follows: For the transformation part: Set the initial value: l = 0; c = 0; Mat_GM[l][c]=Mat_GML[l][0]×T[0][c]×Mat_GML[l][1]×T[1][c]+Mat_GML[l][2]×T[2][c]; L increases by 1, c increases by 1, and Mat_GM[l][c] is iterated until the iteration termination condition is reached, i.e., l<3 and c<3; For the translation part: Set the initial value: c = 0; Mat_GM[3][c]=Mat_GML[3][0]×T[0][c]+Mat_GML[3][1]×T[1][c]+Mat_GML[3][2]×T[2][c]+T[3][c]; L increases by 1, c increases by 1, and Mat_GM[3][c] is iterated until the iteration termination condition is reached, that is, c<3.
7. The method for fast global motion matching of dynamic point clouds based on centroid search according to claim 1, characterized in that: In step 6, P i [0]=Mat_GM[0][0]×B i [0]+Mat_GM[1][0]×B i [1]+Mat_GM[2][0]×B i [2]+Mat_GM[3][0] P i [1]=Mat_GM[0][1]×B i [0]+Mat_GM[1][1]×B i [1]+Mat_GM[2][1]×B i [2]+Mat_GM[3][1] P i [2]=Mat_GM[0][2]×B i [0]+Mat_GM[1][2]×B i [1]+Mat_GM[2][2]×B i [2]+Mat_GM[3][2] Among them, B i is the position information of the object point in the reference frame point cloud, 0≤i≤N, N is the number of object points in the reference frame point cloud; B i [0] is point B i The x-axis information, B i [1] is point B i The y-axis information, B i [2] is point B i z-axis information; Mat_GM is a matrix with four rows and three columns, P i is the position information of the point obtained after applying Mat_GM, P i [0] is point P i The x-axis information, P i [1] is point P i The y-axis information, P i [2] is point P i The z-axis information.
8. A fast global motion matching system for dynamic point clouds based on centroid search, used to implement the fast global motion matching method for dynamic point clouds based on centroid search as described in any one of claims 1 to 7, characterized in that: It includes a road point and object point division unit, a correlation point search unit, a centroid point extraction unit, a global motion matrix calculation unit and a global motion matrix application unit; The road point and object point division unit is used to execute step 1; the associated point search unit is used to execute step 2; the centroid point extraction unit is used to execute step 3; the global motion matrix calculation unit is used to execute steps 4 and 5; and the global motion matrix application unit is used to execute step 6.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for fast global motion matching of dynamic point clouds based on centroid search as described in any one of claims 1-7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for fast global motion matching of dynamic point clouds based on centroid point search described in any one of claims 1 to 7 are implemented.
