A point cloud frame sequence compression method based on maximum change direction

Abstract

The present application relates to a kind of point cloud frame sequence compression methods based on maximum change direction, belong to point cloud compression, solve the problems of low compression efficiency and complex calculation in prior art dynamic point cloud compression method.The method comprises the following steps: based on the maximum change coordinate system and maximum change direction of adjacent point cloud frame in point cloud frame sequence are calculated based on multilayer projection method;Point cloud frame sequence is segmented based on the maximum change direction of adjacent point cloud frame;For each segment point cloud frame, according to the maximum change coordinate system of adjacent point cloud frame, calculate fusion coordinate system;Each point cloud frame of the segment point cloud frame is projected on the projection direction of the fusion coordinate system to obtain the complete projection image of each point cloud frame in each projection direction, and according to the similarity of the complete projection image of adjacent two point cloud frames in each projection direction, the segment point cloud frame is compressed.The efficient point cloud compression is realized.

Description

A point cloud frame sequence compression method based on the maximum change direction Technical Field

[0001] This invention relates to the field of point cloud compression technology, and in particular to a point cloud frame sequence compression method based on the direction of maximum change. Background Technology

[0002] Point clouds can be divided into two types: static point clouds and dynamic point clouds. A static point cloud consists of a set of points with three-dimensional positional information, each point possessing one or more attributes to store additional information. A dynamic point cloud consists of multiple consecutive static point clouds, where each static point cloud is called a frame of the dynamic point cloud. The positional information of points in a point cloud is generally represented by (x, y, z) coordinates in a Cartesian coordinate system, used to characterize the spatial topology and reconstruct the shape of three-dimensional objects. A common attribute of point clouds is color information, typically stored using RGB values ​​to store the red, green, and blue channels, or using intensity values ​​to record the amplitude of pulse returns to occupy the color field.

[0003] Point clouds play a crucial role in future real-time holographic human-computer interaction scenarios. Devices based on LiDAR and photogrammetry can rapidly acquire large-scale, high-frequency changing real-time point cloud information for human-computer interaction. However, due to the uneven distribution of computing and storage resources across networks, the primary requirements are to transmit data to the user end for display, to the computing center for processing, and to the storage center for storage. But point cloud data is massive, unstructured, and lacks metadata; direct transmission would consume enormous bandwidth, becoming a major constraint on point cloud transmission. Therefore, point cloud compression is necessary to reduce data volume.

[0004] Currently, compression methods for dynamic human-computer interaction point clouds can be broadly categorized into two types: compression methods based on 3D structures and compression methods based on 2D mappings. Compression methods based on 3D structures directly study the compression algorithm for the point cloud, using structured algorithms for traversal and matching. Most of these methods are based on octree data structures, and a common problem is that the computational cost increases exponentially with the depth of the octree. This problem is particularly severe for compressing high-precision, high-detail point cloud sequences. Compression methods based on 2D mappings basically project the point cloud data onto a 2D plane according to certain rules, and then use existing mature video encoding and decoding technologies to compress and encode the 2D mapping sequence of the dynamic human-computer interaction point cloud. Compared with 3D structure-based compression algorithms, these algorithms significantly improve compression efficiency in both time and space, achieving remarkable results. In 2017, the MPEG (Moving Picture Expert Group) under ISO / IEC JTC1 established relevant international standards and released TMC2 (Test Model Category 2) for dynamic human-computer interaction point cloud compression algorithms. They adopted a compression method based on two-dimensional mapping to compress point clouds, which has high compression efficiency. However, because the three-dimensional point cloud is mapped into two-dimensional space, the continuity of changes in three-dimensional space is completely destroyed, resulting in the loss of some information and low compression efficiency. Summary of the Invention

[0005] Based on the above analysis, the present invention aims to provide a point cloud frame sequence compression method based on the maximum change direction, in order to solve the problems of computational complexity and low compression efficiency of existing dynamic point cloud compression methods.

[0006] On one hand, embodiments of the present invention provide a point cloud frame sequence compression method based on the maximum change direction, comprising the following steps:

[0007] The maximum change coordinate system and maximum change direction of adjacent point cloud frames in a point cloud frame sequence are calculated based on the multi-layer projection method.

[0008] Segment the point cloud frame sequence based on the maximum change direction of adjacent point cloud frames;

[0009] For each point cloud frame, a fusion coordinate system is calculated based on the coordinate system of the maximum change between adjacent point cloud frames; each point cloud frame in the segment is projected onto the projection direction of the fusion coordinate system to obtain a complete projection image of each point cloud frame in each projection direction; the segment of point cloud frames is compressed based on the similarity of the complete projection images of two adjacent point cloud frames in each projection direction.

[0010] Based on further improvements to the above technical solution, the maximum change coordinate system and maximum change direction of adjacent point cloud frames in a point cloud frame sequence are calculated using a multi-layer projection algorithm, including:

[0011] S11. For the first point cloud frame in the point cloud frame sequence, establish a coordinate system with its centroid as the origin, which is parallel to the coordinate axes of the reference coordinate system and has the same direction as the first point cloud frame. Use the second point cloud frame as the current point cloud frame.

[0012] S12. Establish a coordinate system with the centroid of the current point cloud frame as the origin, which is parallel to the coordinate axes and in the same direction as the final projection coordinate system of the previous frame of the current point cloud frame. This coordinate system is the current projection coordinate system of the current point cloud frame.

[0013] S13. Based on the layered projection method, project the current point cloud frame onto the projection direction of the current projection coordinate system; calculate the chamfer distance between the current point cloud frame and the previous frame in each projection direction of the current projection coordinate system, and the projection direction with the largest chamfer distance is the maximum projection direction of the current projection coordinate system.

[0014] S14. Fix the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system, and rotate the two coordinate axes other than the fixed coordinate axis multiple times around the fixed coordinate axis according to the rotation step size. Each rotation yields a candidate coordinate system, thereby obtaining multiple candidate coordinate systems coaxial with the fixed coordinate axis of the current projection coordinate system.

[0015] S15. Based on the layered projection method, project the current point cloud frame onto the projection direction of each candidate coordinate system; calculate the chamfer distance of the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of each candidate coordinate system; if there is a chamfer distance in all projection directions that is greater than the chamfer distance of the maximum projection direction of the current projection coordinate system, then the coordinate system containing the coordinate axis of the projection direction with the largest chamfer distance in all projection directions and the coordinate axis of the maximum projection direction of the current projection coordinate system is the current projection coordinate system, and the projection direction with the largest chamfer distance in all projection directions is the maximum projection direction of the current projection coordinate system, and return to step S14;

[0016] Otherwise, the current projection coordinate system is the coordinate system with the maximum change from the previous frame to the current point cloud frame, and the maximum projection direction of the current projection coordinate system is the maximum change direction from the previous frame to the current point cloud frame; the current projection coordinate system is the final projection coordinate system of the current point cloud frame; the next point cloud frame is the current point cloud frame, and the process returns to step S12 until all point cloud frames have been traversed.

[0017] Furthermore, the point cloud frame sequence is segmented based on the maximum change direction of adjacent point cloud frames, including:

[0018] S211. Take the first point cloud frame of the point cloud frame sequence as the starting point cloud frame of the first segment; take the first segment as the current segment; take the second point cloud frame as the current point cloud frame;

[0019] S212. If the angle between the maximum change direction of the current point cloud frame and the maximum change direction of the starting point cloud frame of the current segment is less than the first threshold, then add the current point cloud frame to the current segment; otherwise, take the current point cloud frame as the starting point cloud frame of the new segment and take the new segment as the current segment; take the next point cloud frame of the current point cloud frame as the current frame and return to step S212. If there is no next point cloud frame, the segmentation ends.

[0020] Furthermore, based on the layered projection method, the current point cloud frame is projected onto the projection direction of the current projection coordinate system, including:

[0021] S131. Construct a set of points to be projected from all points in the current point cloud frame; let k = 1, L = 1;

[0022] S132. Use the k-th coordinate axis of the current projected coordinate system as the reference axis;

[0023] S133. Sort each point in the set of points to be projected in descending order of their values ​​on the reference coordinate axis. For each sorted point, if a corresponding point already exists in the Lth layer of the projection direction corresponding to the reference axis, then do not project that point. Otherwise, project that point onto the corresponding position in the Lth layer of the projection direction corresponding to the reference axis and delete that point from the set of points to be projected. If the set of points to be projected is empty, then the projection ends. Otherwise, proceed to step S134.

[0024] S134, k = k + 1. If k = 4, then L = L + 1. Set k to 1 and return to step S132; otherwise, return to step S132.

[0025] Further, the chamfer distance between the current point cloud frame and the previous frame in each projection direction of the current projection coordinate system is calculated, including:

[0026] For the current projection direction of the current projection coordinate system, find the points of the current point cloud frame projected onto the current projection direction, and form the first set of points;

[0027] Find the points in the previous frame of the current point cloud that are projected onto the current projection direction, and form a second set of points;

[0028] Calculate the chamfer distance between the first point set and the second point set to obtain the chamfer distance between the current point cloud frame and the previous frame in the current projection direction of the current projection coordinate system.

[0029] Furthermore, the chamfer distance d(S1,S2) is calculated using the following formula:

[0030]

[0031] Where S1 represents the first set of points, Let S1 represent the number of points, and S2 represent the second set of points. This indicates the number of points in S2. This represents the distance from point x to point y. This represents the distance from point y to point x.

[0032] Furthermore, each point cloud frame of this segment is projected onto the projection direction of the fused coordinate system to obtain a complete projected image of each point cloud frame in each projection direction, including:

[0033] The layered projection method is used to project each point of each point cloud frame in the segment onto the projection direction corresponding to the fusion coordinate system, resulting in a multi-layer block projection map of each point cloud frame in each projection direction corresponding to the fusion coordinate system.

[0034] By stitching together the multi-layered block projection images of each point cloud frame in each projection direction, a complete projection image of each point cloud frame in each projection direction is obtained.

[0035] Furthermore, the point cloud frame segment is compressed based on the similarity of the complete projected images of two adjacent point cloud frames in each projection direction, including:

[0036] The complete projected images of the first and last point cloud frames in the current segment of point cloud frames in each projection direction are keyframe images.

[0037] For other point cloud frames in the current point cloud frame segment, calculate the structural similarity between the point cloud frame and the complete projected image of the previous point cloud frame in each projection direction; if the structural similarity does not exceed the second threshold, then the complete projected image is used as the key frame image; all key frame images of the point cloud frame segment constitute the compressed image of the point cloud frame segment.

[0038] Furthermore, the structural similarity between the point cloud frame and the complete projected image of its predecessor point cloud frame in each projection direction is calculated using the following method:

[0039] For the current projection direction, according to the formula Calculate the SSIM value of the complete projected image of the point cloud frame and its predecessor point cloud frame on the kth image channel;

[0040] Where, μ k,a μ represents the element mean of the k-th channel of the complete projected image of the point cloud frame. k,b σ represents the mean of the elements in the k-th channel of the complete projected image of the previous point cloud frame;k,a σ represents the element variance of the k-th channel of the complete projected image of the point cloud frame. k,b σ represents the element variance of the k-th channel of the complete projected image of the previous point cloud frame. k,ab c1 and c2 represent the elemental covariance of the k-th channel of the complete projected image of the point cloud frame and the complete projected image of the previous point cloud frame, where c1 and c2 are constants.

[0041] The SSIM values ​​of all image channels are summed to obtain the structural similarity value of the point cloud frame and the complete projected image of the previous point cloud frame in the current projection direction.

[0042] Furthermore, for each point cloud frame, a fused coordinate system is calculated based on the coordinate system of the maximum change between adjacent point cloud frames, including:

[0043] For the current segment point cloud frame, shift all the maximum change coordinate system translation values ​​corresponding to the current segment point cloud frame to the origin so that they coincide with the origin of the reference coordinate system;

[0044] For each coordinate axis of each coordinate system with the largest change, assign it to the type corresponding to the reference coordinate with which it has the smallest angle;

[0045] The unit vectors of the coordinate axes corresponding to each type are summed to obtain the coordinate axes of the fused coordinate system corresponding to that type; thus, the fused coordinate system corresponding to the current segment point cloud frame is obtained.

[0046] Compared with the prior art, the present invention obtains the change direction of connected point cloud frames by calculating the maximum change coordinate system of adjacent point cloud frames in the point cloud frame sequence; for each point cloud frame segment, a fusion coordinate system is calculated based on the maximum change coordinate system of adjacent point cloud frames, and the point cloud frame is projected on the projection direction corresponding to the fusion coordinate system, thereby retaining the spatiotemporal consistency characteristics of the point cloud frame after projection and thus improving the compression efficiency.

[0047] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0048] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0049] Figure 1 is a flowchart of the point cloud frame sequence compression method based on the maximum change direction according to an embodiment of the present invention. Detailed Implementation

[0050] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0051] A specific embodiment of the present invention discloses a point cloud frame sequence compression method based on the maximum change direction, as shown in Figure 1, comprising the following steps:

[0052] S1. Calculate the maximum change coordinate system and maximum change direction of adjacent point cloud frames in the point cloud frame sequence based on the multi-layer projection method;

[0053] S2. Segment the point cloud frame sequence based on the maximum change direction of adjacent point cloud frames;

[0054] S3. For each point cloud frame segment, calculate the fusion coordinate system based on the coordinate system of the maximum change between adjacent point cloud frames; project each point cloud frame segment onto the projection direction of the fusion coordinate system to obtain the projection image of each point cloud frame in each projection direction; compress the point cloud frame segment based on the similarity of the projection images of two adjacent point cloud frames in each projection direction.

[0055] Compared with the prior art, the present invention obtains the change direction of connected point cloud frames by calculating the maximum change coordinate system of adjacent point cloud frames in the point cloud frame sequence; for each point cloud frame segment, a fusion coordinate system is calculated based on the maximum change coordinate system of adjacent point cloud frames, and the point cloud frame is projected on the projection direction corresponding to the fusion coordinate system, thereby retaining the spatiotemporal consistency characteristics of the point cloud frame after projection and thus improving the compression efficiency.

[0056] Specifically, step S1 calculates the maximum change coordinate system and maximum change direction of adjacent point cloud frames in the point cloud frame sequence based on the multi-layer projection algorithm, including steps S11-S15:

[0057] S11. For the first point cloud frame in the point cloud frame sequence, establish a coordinate system with its centroid as the origin, which is parallel to the coordinate axes of the reference coordinate system and has the same direction as the first point cloud frame. Use the second point cloud frame as the current point cloud frame.

[0058] S12. Establish a coordinate system with the centroid of the current point cloud frame as the origin, which is parallel to the coordinate axes and in the same direction as the final projection coordinate system of the previous frame of the current point cloud frame. This coordinate system is the current projection coordinate system of the current point cloud frame.

[0059] S13. Based on the layered projection method, project the current point cloud frame onto the projection direction of the current projection coordinate system; calculate the chamfer distance between the current point cloud frame and the previous frame in each projection direction of the current projection coordinate system, and the projection direction with the largest chamfer distance is the maximum projection direction of the current projection coordinate system.

[0060] S14. Fix the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system, and rotate the two coordinate axes other than the fixed coordinate axis multiple times around the fixed coordinate axis according to the rotation step size. Each rotation yields a candidate coordinate system, thereby obtaining multiple candidate coordinate systems coaxial with the fixed coordinate axis of the current projection coordinate system.

[0061] S15. Based on the layered projection method, project the current point cloud frame onto the projection direction of each candidate coordinate system; calculate the chamfer distance of the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of each candidate coordinate system; if there is a chamfer distance in all projection directions that is greater than the chamfer distance of the maximum projection direction of the current projection coordinate system, then the coordinate system containing the coordinate axis of the projection direction with the largest chamfer distance in all projection directions and the coordinate axis of the maximum projection direction of the current projection coordinate system is the current projection coordinate system, and the projection direction with the largest chamfer distance in all projection directions is the maximum projection direction of the current projection coordinate system, and return to step S14;

[0062] Otherwise, the current projection coordinate system is the coordinate system with the maximum change from the previous frame to the current point cloud frame, and the maximum projection direction of the current projection coordinate system is the maximum change direction from the previous frame to the current point cloud frame; the current projection coordinate system is the final projection coordinate system of the current point cloud frame; the next point cloud frame is the current point cloud frame, and the process returns to step S12 until all point cloud frames have been traversed.

[0063] It should be noted that the reference coordinate system is a reference coordinate system; for example, the world coordinate system can be used as the reference coordinate system.

[0064] During implementation, for the first frame of the point cloud sequence, a coordinate system parallel to and in the same direction as the coordinate axes of the reference coordinate system is directly established with its centroid as the origin, which is used as its final projection coordinate system. For the second to the last frame, the final projection coordinate system needs to be obtained according to steps S12-S15.

[0065] For example, for the i-th point cloud frame, firstly, a coordinate system is established with the centroid of the i-th point cloud frame as the origin, parallel to and aligned with the coordinate axes of the final projected coordinate system of the (i-1)-th point cloud frame. This is the first projected coordinate system of the i-th point cloud frame, using O... i X1Y1Z1 represents the current projected coordinate system.

[0066] For indivisible objects, such as a person in human-computer interaction, their movement is a continuous process. The direction of change at one moment and the direction of change at the next moment do not change much for most of the time. Therefore, the maximum direction of change from the (i-2)th point cloud frame to the (i-1)th point cloud frame is also the maximum direction of change from the (i-1)th point cloud frame to the ith point cloud frame for most of the time. Therefore, for the ith point cloud frame, we first establish a coordinate system that is parallel to and oriented in the same direction as the coordinate system of the final projection coordinate system of the (i-1)th point cloud frame. This allows us to find the maximum coordinate system and the maximum direction of change from the (i-1)th point cloud frame to the ith point cloud frame more quickly, reducing the amount of computation and improving efficiency.

[0067] The i-th point cloud frame is projected onto the current projection coordinate system O based on the layered projection method. i In the projection direction of X1Y1Z1, the current projection coordinate system O i X1Y1Z1 includes O i The direction of the X1 axis, O i The direction of the Z1 axis and O i The direction of the Y1 axis is used to calculate the chamfer distance between the current point cloud frame and the previous frame in each projection direction of the current projection coordinate system. The projection direction with the largest chamfer distance is the maximum projection direction of the current projection coordinate system.

[0068] For example, the current projected coordinate system O i The maximum projection direction of X1Y1Z1 is O. i The direction of the X1 axis, O i The direction of the X1 axis is not necessarily the direction of the maximum change from the (i-1)th point cloud frame to the ith point cloud frame. Therefore, it is necessary to check if there are other directions where the difference between the two frames is greater. At this point, O i With the X1 axis fixed, rotate the other two coordinate axes, O, according to the rotation step size. i Z1 and O i Y1 axis around O i Rotate the X1 axis multiple times to obtain the coordinates relative to the current projected coordinate system O. i X1Y1Z1's O i Multiple candidate coordinate systems are coaxial with the X1 axis. Based on a layered projection method, the i-th point cloud frame is projected onto the projection direction of each candidate coordinate system; the chamfer distance between the current point cloud frame and the previous frame in each projection direction of each candidate coordinate system is calculated. If there exists a chamfer distance greater than that of the current projection coordinate system O in all projection directions... i The maximum projection direction of X1Y1Z1 (O i The chamfer distance in the X1 axis direction is then used to determine the coordinate axis corresponding to the projection direction with the largest chamfer distance among all projection directions, and the coordinate axis corresponding to the largest projection direction in the current projection coordinate system (O). i The coordinate system containing the X1 axis (labeled as O)i (X2Y2Z2) is the current projection coordinate system, and the projection direction with the largest chamfer distance among all projection directions (e.g., O). i O in the X2Y2Z2 coordinate system i The Y2 axis direction is the maximum projection direction of the current projection coordinate system; O i O in the X2Y2Z2 coordinate system i The Y2 axis direction is not necessarily the direction of the greatest change from the (i-1)th point cloud frame to the ith point cloud frame. It is also necessary to check if there are other directions where the difference between the two frames is greater. Therefore, O... i O in the X2Y2Z2 coordinate system i With the Y2 axis fixed, rotate the other two coordinate axes around O according to the rotation step size. i Rotate along the Y2 axis to generate multiple new candidate coordinate systems. Follow the steps described above to find if there is a projection direction with a larger chamfer distance. Continue in this manner until there is no projection direction in the candidate coordinate systems with a chamfer distance larger than the chamfer distance of the maximum projection direction of the current projection coordinate system. At this point, the maximum projection direction of the current projection coordinate system is the maximum change direction from the previous frame to the current point cloud frame; the current projection coordinate system is the final projection coordinate system of the current point cloud frame. Then, start with the next point cloud frame as the current point cloud frame and calculate its maximum change direction according to the above process until all point cloud frames in the point cloud frame sequence have been traversed.

[0069] During implementation, the rotation step size can be determined based on the calculation speed and accuracy of the changing direction. If high calculation accuracy is required, the rotation step size can be set to a smaller number, such as 10 degrees. If high calculation speed is required, the rotation step size can be set to a larger number, such as 30 degrees.

[0070] Each coordinate system of each point cloud frame corresponds to multiple projection directions. In one embodiment of the present invention, one coordinate system corresponds to three projection directions. For example, for the j-th coordinate system O of the i-th point cloud frame... i X j Y j Z j The three projection directions are O i X j The direction of the axis, O i Z j The direction of the axis and O i Y j The direction of the axis.

[0071] Specifically, step S13, which projects the current point cloud frame onto the projection direction of the current projection coordinate system based on the layered projection method, includes:

[0072] S131. Construct a set of points to be projected from all points in the current point cloud frame; let k = 1, L = 1;

[0073] It should be noted that k represents the k-th coordinate axis of the j-th projected coordinate system, and L represents the first projection layer of the j-th projected coordinate system. Initially, k = 1 and L = 1.

[0074] S132. Use the k-th coordinate axis of the current projected coordinate system as the reference axis;

[0075] S133. Sort each point in the set of points to be projected in descending order of their values ​​on the reference coordinate axis. For each sorted point, if a corresponding point already exists in the Lth layer of the projection direction corresponding to the reference axis, then do not project that point. Otherwise, project that point onto the corresponding position in the Lth layer of the projection direction corresponding to the reference axis and delete that point from the set of points to be projected. If the set of points to be projected is empty, then the projection ends. Otherwise, proceed to step S134.

[0076] S134, k = k + 1. If k = 4, then L = L + 1. Set k to 1 and return to step S132; otherwise, return directly to step S132.

[0077] During implementation, for the current projection coordinate system of the current point cloud frame (e.g., the i-th point cloud frame), for example, the j-th coordinate system O... i X j Y j Z j The set of points to be projected is formed by all the points in the current point cloud frame.

[0078] Coordinate system O i X j Y j Z j The first coordinate axis is used as the reference axis, for example, the X-axis is used as the reference axis, and the projection direction is O. i X j The direction of the axis is determined by sorting each point in the set of points to be projected in descending order of their coordinate values ​​on the X-axis, prioritizing points with larger coordinate values ​​for projection. Then, following the sorted order, each point is checked sequentially to determine if it can be projected onto the O-axis. i X j The first layer in the projection direction of the axis.

[0079] The first point after sorting can necessarily be projected onto the first layer in the axial direction. Projecting this point onto O i X j The first layer in the axial direction, mark the point at O. i X j The projection position of the first layer along the axis is occupied, so this point is removed from the set of points to be projected. Starting from the second point, it is necessary to determine whether the point is in position O. i X jIf the projection position of the first layer in the axial direction is occupied, and if it is occupied (meaning a point is already projected onto that position), then the point will not be projected and will remain in the set of points to be projected. Otherwise, the point will be projected onto O. i X j The first layer in the axial direction, mark the point at O. i X j The projection position of the first layer in the axial direction is occupied, so this point is removed from the set of points to be projected.

[0080] After traversing the set of points to be projected once, if the set is empty, the projection ends; if the set is not empty, meaning there are points that have not been projected, then the current projection coordinate system O is changed. i X j Y j Z j The second coordinate axis is used as the reference axis, for example, the Y-axis is used as the reference axis, and the projection direction is O. i Y j The direction of the axis is determined by sorting each point in the set of points to be projected in descending order of their Y-axis coordinates, prioritizing points with larger coordinate values. Then, following the sorted order, each point is checked sequentially to determine if it can be projected onto point O. i Y j The first layer along the axis; see the previous section for the specific judgment process. After traversing the set of points to be projected, if the set of points to be projected is empty, the projection ends; if the set of points to be projected is still not empty, then the current projection coordinate system O is changed. i X j Y j Z j Using the third coordinate axis, the Z-axis, as the reference axis, the points in the set to be projected are projected onto the third projection direction O following the same process. i Z j On the first layer in the axial direction.

[0081] O i X j Y j Z j After iterating through all three coordinate axes as reference axes, some points may still not be projected due to occlusion. That is, the set of points to be projected is still not empty. So, the layer number is incremented by one, i.e., L is incremented by 1. Following the above process, the points in the set of points to be projected are projected onto the second layer of the three projection directions. If there are still unprojected points, the points in the set of points to be projected are projected onto the third layer of the three projection directions, and so on, until the set of points to be projected is empty.

[0082] Compared with the prior art, the present invention projects the point cloud onto multiple layers in the projection direction using a layered projection method, thereby taking into account the occlusion relationship of the points and thus providing a basis for accurately calculating the direction of maximum change.

[0083] Specifically, step S13 calculates the chamfer distance between the current point cloud frame and the previous frame in each projection direction of the current projection coordinate system, including:

[0084] For the current projection direction of the current projection coordinate system, find the current point cloud frame projected in the current projection direction to form the first set of points;

[0085] Find the points in the previous frame of the current point cloud that are projected onto the current projection direction, and form a second set of points;

[0086] Calculate the chamfer distance between the first point set and the second point set to obtain the chamfer distance between the current point cloud frame and the previous frame in the current projection direction of the current projection coordinate system.

[0087] It should be noted that for the current point cloud frame (the i-th point cloud frame), the current projection coordinate system (O) i X j Y j Z j If its preceding point cloud frame (the (i-1)th point cloud frame) exists in the current projection coordinate system (O), i X j Y j Z j A coordinate system in which all three coordinate axes are parallel and oriented in the same direction (labeled as O). i-1 X m Y m Z m Since the coordinate axes are parallel and have the same direction, the current projected coordinate system (O) i X j Y j Z j The three projection directions of ) and the coordinate system O i X m Y m Z m The directions are consistent, so we can directly find the points projected onto the current projection direction of the previous frame of the previous point cloud frame to form the second set of points.

[0088] If the previous point cloud frame (the (i-1)th point cloud frame) does not exist in the current projection coordinate system (O i X j Y j Z j In a coordinate system where all three coordinate axes are parallel and oriented in the same direction, the current projected coordinate system (O) is then... i X j Y j Z j The origin of the coordinate system is moved to the centroid of the previous point cloud frame (the (i-1)th point cloud frame) to obtain the coordinate system (marked as O). i-1 Xm Y m Z m ), and project the (i-1)th point cloud frame onto O according to the aforementioned layered projection method. i-1 X m Y m Z m In the projection direction, since O i X j Y j Z j Translation, therefore the current projected coordinate system (O) i X j Y j Z j The three projection directions of ) and the coordinate system O i X m Y m Z m The directions are consistent; find the points projected onto the current projection direction from the previous frame of the previous point cloud frame to form the second set of points.

[0089] For example, the current point cloud frame (the i-th point cloud frame) is in the current projection coordinate system O. i X j Y j Z j If 8 layers are projected downwards, then the first point set is formed by taking all the points on the 8 layers projected in the current point cloud frame in the direction. The points projected onto the previous point cloud frame (the (i-1)th point cloud frame) in the current point cloud frame are then considered. i-1 X m Y m Z m Points on all layers (e.g., 10 layers) in the corresponding direction constitute the second set of points.

[0090] Specifically, the chamfer distance d(S1,S2) between the first set of points and the second set of points is calculated using the following formula:

[0091]

[0092] Where S1 represents the first set of points, Let S1 represent the number of points, and S2 represent the second set of points. This indicates the number of points in S2. This represents the distance from point x to point y. This represents the distance from point y to point x.

[0093] The calculated chamfer distance between the first and second point sets represents the distance in the current projected coordinate system O. i X j Y j Z jBelow, the current point cloud frame (the i-th point cloud frame) and its previous point cloud frame are in O i X j The difference along the axis increases with the chamfer distance. The same process is used to calculate the difference between the current point cloud frame (the i-th point cloud frame) and its previous point cloud frame at O. i Y j Shaft and O i Z j Differences in the axial direction.

[0094] After calculating the coordinate system of maximum change between any two adjacent point cloud frames in the point cloud frame sequence, the point cloud frame sequence is segmented. Specifically, segmenting the point cloud frame sequence based on the direction of maximum change between adjacent point cloud frames includes:

[0095] S211. Take the first point cloud frame of the point cloud frame sequence as the starting point cloud frame of the first segment; take the first segment as the current segment; take the second point cloud frame as the current point cloud frame;

[0096] S212. If the angle between the maximum change direction of the current point cloud frame and the maximum change direction of the starting point cloud frame of the current segment is less than the first threshold, then add the current point cloud frame to the current segment; otherwise, use the current point cloud frame as the starting point cloud frame of a new segment, and use the new segment as the current segment. Take the next point cloud frame of the current point cloud frame as the current frame, return to step S212, and if there is no next point cloud frame, the segmentation ends.

[0097] In practice, the first threshold can be determined based on the required accuracy of the calculation, for example, it can be set to 15 degrees.

[0098] That is, frames with basically the same direction of change are grouped into a segment, thereby improving the compression ratio and the accuracy of compressed data. For each point cloud frame segment, the coordinate systems of the most changed points cloud frames are merged to facilitate point cloud compression.

[0099] Specifically, in step S3, for each point cloud frame, the fused coordinate system is calculated based on the coordinate system of the maximum change between adjacent point cloud frames, including:

[0100] S311. For the current segment point cloud frame, translate all the coordinate systems with the largest changes corresponding to the current segment point cloud frame to coincide with the origin of the reference coordinate system.

[0101] During implementation, the coordinate system of the maximum change of all adjacent point cloud frames in the current segment is translated to coincide with the origin O0 of the reference coordinate system O0X0Y0Z0.

[0102] S312. For each coordinate axis of each coordinate system with the largest change, classify it into the type corresponding to the reference coordinate axis with the smallest angle.

[0103] For each coordinate axis of the coordinate system with the largest change, determine the angle between it and the three coordinate axes of the reference coordinate system. If the current coordinate axis has the smallest angle with the O0X0 axis of the reference coordinate system, then the current coordinate axis is classified into the first category; if the current coordinate axis has the smallest angle with the O0Y0 axis of the reference coordinate system, then the current coordinate axis is classified into the second category; if the current coordinate axis has the smallest angle with the O0Z0 axis of the reference coordinate system, then the current coordinate axis is classified into the third category.

[0104] It should be noted that there may be a coordinate axis of the coordinate system with the maximum change that has the same and smallest angle with the two coordinate axes of the reference coordinate system. In this case, it is sufficient to ensure that the three coordinate axes of the current coordinate system with the maximum change belong to different categories.

[0105] S313. Sum the unit vectors of the coordinate axes corresponding to each type to obtain the coordinate axes of the fused coordinate system corresponding to that type; obtain the fused coordinate system corresponding to the current segment point cloud frame.

[0106] The unit vectors corresponding to the coordinate axes of the same type are summed to obtain the coordinate axes of the fused coordinate system. The fused coordinate system corresponding to the s-th segment of the point cloud frame sequence is... express.

[0107] Specifically, in step S3, each point cloud frame of the segment is projected onto the projection direction of the fusion coordinate system to obtain a complete projected image of each point cloud frame in each projection direction, including:

[0108] S321. Using the layered projection method, project each point of each point cloud frame in the segment of point cloud frames onto the projection direction corresponding to the fusion coordinate system to obtain a multi-layer block projection map of each point cloud frame in each projection direction corresponding to the fusion coordinate system.

[0109] During implementation, for the i-th point cloud frame in the s-th point cloud frame sequence, the coordinate system is first fused. Translate to the centroid of the point cloud frame to obtain the fused projection coordinate system corresponding to the i-th point cloud frame. The corresponding projection direction is The direction of the axis The direction of the axis and The direction in which it is located.

[0110] Since the fusion coordinate system of each point cloud frame is obtained by translating the fusion coordinate system corresponding to the current segment, the projection direction of each point cloud frame is the same as the projection direction of the fusion coordinate system corresponding to the current segment.

[0111] During implementation, the i-th point cloud frame in the s-th segment of the point cloud frame sequence is projected onto the fused projection coordinate system using the same method as in steps S131-S134. In the corresponding projection direction, the multi-layer block projection map of the i-th point cloud frame in the s-th point cloud frame sequence is obtained in each projection direction. For example, if the projection of the i-th point cloud frame is divided into 7 layers, then in Projection direction Projection direction and The projection directions correspond to the seven block projection diagrams.

[0112] It should be noted that each pixel in each projection map records the coordinates and attribute index of the point in the point cloud frame projected to that location in the fused projection coordinate system, represented as (x, y, z, a). If no point is projected to that location, the corresponding pixel is (0, 0, 0, 0). In practice, the attribute can be color.

[0113] S322. Stitch together the multi-layer block projection images of each point cloud frame in each projection direction to obtain the complete projection image of each point cloud frame in each projection direction.

[0114] After obtaining the multi-layer block projection map of each point cloud frame in the current segment point cloud frame sequence in each projection direction, for each point cloud frame, such as the i-th point cloud frame in the s-th segment point cloud frame sequence, the multi-layer block projection map in each projection direction is stitched together to obtain the complete projection image of the i-th point cloud frame in the s-th segment point cloud frame sequence on each projection map plane.

[0115] In implementation, assume that the s-th point cloud frame sequence contains 10 point cloud frames. For each block projection map of these 10 point cloud frames, a two-dimensional coordinate system is established with the point corresponding to the block projection map as the origin. The maximum absolute value of the coordinate values ​​in the two-dimensional coordinate system is the side length of the block projection map. The maximum side length of all block projection maps is L. max The dimension of the baseline block diagram is 2L. max ×2L max For example, if the maximum side length is 100, then the dimension of the baseline block map is 200x200. Each block projection map is expanded to the size of the baseline block map; that is, a two-dimensional coordinate system is established with the point in the block projection map corresponding to the origin of the projection coordinate system of the block projection map as the origin. The x-axis of the two-dimensional coordinate system is -2L. max Up to 2L max -2L on the Y-axis max Up to 2L max The positions within the range that have no projection points are filled with 0, resulting in a filled block projection map.

[0116] For the i-th point cloud frame, The i-th point cloud frame is obtained by stitching together the block projection images corresponding to the projection direction in order from the first layer to the last layer. A complete projected image along the projection direction; The i-th point cloud frame is obtained by stitching together the block projection images corresponding to the projection direction in order from the first layer to the last layer. A complete projected image along the projection direction; The i-th point cloud frame is obtained by stitching together the block projection images corresponding to the projection direction in order from the first layer to the last layer. A complete projected image along the projection direction.

[0117] Specifically, in step S3, the point cloud frame segment is compressed based on the similarity of the complete projected images of two adjacent point cloud frames in each projection direction, including:

[0118] S331. The complete projection images of the first and last point cloud frames in the current segment in each projection direction are keyframe images.

[0119] S332. For other point cloud frames in the current point cloud frame segment, calculate the structural similarity of the point cloud frame with the complete projected image of the previous point cloud frame in each projection direction; if the structural similarity does not exceed the second threshold, then the complete projected image is used as the key frame image; all key frame images of the point cloud frame segment constitute the compressed image of the point cloud frame segment.

[0120] For example, for the i-th point cloud frame, we obtain its position in... Projection direction Projection direction and The complete projected image along the projection direction is used to calculate the i-th point cloud frame. The complete projected image along the projection direction and the (i-1)th point cloud frame are in The structural similarity of the complete projected image in the projection direction, and the i-th point cloud frame in The complete projected image along the projection direction and the (i-1)th point cloud frame are in The structural similarity of the complete projected image in the projection direction, and the i-th point cloud frame in The complete projected image along the projection direction and the (i-1)th point cloud frame are in Structural similarity of complete projected images in the projection direction.

[0121] It needs to be explained that, The fused projection coordinate system is the fused coordinate system. The coordinate system obtained by translating to the centroid of the i-th point cloud frame in the s-th segment The fused projection coordinate system is the fused coordinate system. The coordinate system is obtained by translating to the centroid of the (i-1)th point cloud frame in the s-th segment. Therefore... Projection direction and If the projection directions are the same, it is called a fused coordinate system. of The direction of the axis; Projection direction and If the projection directions are the same, it is called a fused coordinate system. of axis; Projection direction and If the projection directions are the same, it is called a fused coordinate system. of axis.

[0122] If the i-th point cloud frame is in The complete projected image along the projection direction and the (i-1)th point cloud frame are in The structural similarity of the complete projected image in the projection direction is less than the second threshold, that is, the i-th point cloud frame and the (i-1)-th point cloud frame are similar. If the difference in the projection direction is large, then the i-th point cloud frame will be projected onto the i-th point cloud frame. The complete projected image in the projection direction is used as the keyframe image; otherwise, the i-th point cloud frame is discarded. A complete projected image in the projection direction.

[0123] Specifically, step S332 calculates the structural similarity between the point cloud frame and the complete projected image of the previous point cloud frame in each projection direction in the following manner:

[0124] S3321. For the current projection direction, according to the formula...

[0125] Calculate the SSIM value of the complete projected image of the point cloud frame and its predecessor point cloud frame on the kth image channel;

[0126] Where, μ k,a μ represents the element mean of the k-th channel of the complete projected image of the point cloud frame. k,b σ represents the mean of the elements in the k-th channel of the complete projected image of the previous point cloud frame; k,a σ represents the element variance of the k-th channel of the complete projected image of the point cloud frame. k,b σ represents the element variance of the k-th channel of the complete projected image of the previous point cloud frame. k,ab c1 and c2 represent the elemental covariance of the k-th channel of the complete projected image of the point cloud frame and the complete projected image of the previous point cloud frame, where c1 and c2 are constants.

[0127] It should be noted that each image channel, i.e., each coordinate and color index, corresponds to four channels.

[0128] S3322. Add the SSIM values ​​of all image channels to obtain the structural similarity value of the point cloud frame and the complete projected image of the previous point cloud frame in the current projection direction.

[0129] The SSIM values ​​calculated from the four channels are added together to obtain the structural similarity value between the point cloud frame and the previous point cloud frame in the current projection direction.

[0130] After obtaining the keyframe image of the s-th segment of the 3D point cloud, the fused coordinates, keyframe image, and point cloud frame number, centroid, and projection method corresponding to each keyframe image are used as the compressed data for the current point cloud segment.

[0131] Since complete projection images with high similarity are not used as keyframe images, fewer complete projection images are retained in directions with small changes, thus greatly reducing the amount of compressed data and significantly improving the compression ratio. Furthermore, more complete projection images are retained as keyframes in directions with large changes, thereby reducing compression distortion. When restoring the point cloud based on the compressed data, the restoration accuracy is high, and the information of the original point cloud frames is retained, resulting in higher compression accuracy.

[0132] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0133] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A point cloud frame sequence compression method based on the direction of maximum change, characterized in that, Includes the following steps: The maximum change coordinate system and maximum change direction of adjacent point cloud frames in the point cloud frame sequence are calculated based on the multi-layer projection method; the point cloud frame sequence is segmented based on the maximum change direction of adjacent point cloud frames. For each point cloud frame, a fusion coordinate system is calculated based on the coordinate system of the maximum change between adjacent point cloud frames; each point cloud frame in the segment is projected onto the projection direction of the fusion coordinate system to obtain a complete projection image of each point cloud frame in each projection direction; and the segment of point cloud frames is compressed based on the similarity of the complete projection images of two adjacent point cloud frames in each projection direction. The algorithm for calculating the maximum change coordinate system and maximum change direction of adjacent point cloud frames in a point cloud frame sequence based on a multi-layer projection algorithm includes: S11, for the first point cloud frame in the point cloud frame sequence, establishing a coordinate system with its centroid as the origin, parallel to and aligned with the coordinate axes of the reference coordinate system, as the final projection coordinate system of the first point cloud frame; taking the second point cloud frame as the current point cloud frame; S12, establishing a coordinate system with its centroid as the origin, parallel to and aligned with the coordinate axes of the final projection coordinate system of the previous frame, as the current projection coordinate system of the current point cloud frame; S13, projecting the current point cloud frame onto the projection direction of the current projection coordinate system based on the layered projection method; calculating the current point cloud frame and... S14. The chamfer distance of the previous frame of the current point cloud frame in each projection direction of the current projection coordinate system is calculated, and the projection direction with the largest chamfer distance is the maximum projection direction of the current projection coordinate system; S15. The coordinate axis corresponding to the maximum projection direction of the current projection coordinate system is fixed, and the two coordinate axes other than the fixed coordinate axis are rotated multiple times around the fixed coordinate axis according to the rotation step size. Each rotation yields a candidate coordinate system, thus obtaining multiple candidate coordinate systems coaxial with the fixed coordinate axis of the current projection coordinate system; S16. The current point cloud frame is projected onto the projection direction of each candidate coordinate system based on the layered projection method; the chamfer distance of the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of each candidate coordinate system is calculated. If there exists a chamfer distance among all projection directions that is greater than the chamfer distance of the maximum projection direction in the current projection coordinate system, then the coordinate system containing the coordinate axis corresponding to the projection direction with the largest chamfer distance and the coordinate axis corresponding to the maximum projection direction in the current projection coordinate system is the current projection coordinate system. The projection direction with the largest chamfer distance among all projection directions is the maximum projection direction in the current projection coordinate system, and the process returns to step S14. Otherwise, the current projection coordinate system is the coordinate system of the maximum change from the previous frame to the current point cloud frame, and the maximum projection direction of the current projection coordinate system is the maximum change direction from the previous frame to the current point cloud frame. The current projection coordinate system is the final projection coordinate system of the current point cloud frame. The following step is performed: The next point cloud frame is taken as the current point cloud frame, and the process returns to step S12 until all point cloud frames have been traversed. The point cloud frame segment is compressed based on the similarity of the complete projected images of two adjacent point cloud frames in each projection direction. This includes: the complete projected images of the first and last point cloud frames in the current segment are both keyframe images in each projection direction; for other point cloud frames in the current segment, the structural similarity of the complete projected image of the current point cloud frame to that of its preceding point cloud frame in each projection direction is calculated; if the structural similarity does not exceed a second threshold, the complete projected image is used as a keyframe image; all keyframe images of the current point cloud frame segment constitute the compressed image of the segment.

2. The point cloud frame sequence compression method based on the maximum change direction according to claim 1, characterized in that, The point cloud frame sequence is segmented based on the maximum change direction of adjacent point cloud frames, including: S211, taking the first point cloud frame of the point cloud frame sequence as the starting point cloud frame of the first segment; taking the first segment as the current segment; taking the second point cloud frame as the current point cloud frame; S212, if the angle between the maximum change direction of the current point cloud frame and the maximum change direction of the starting point cloud frame of the current segment is less than a first threshold, then the current point cloud frame is added to the current segment; otherwise, the current point cloud frame is taken as the starting point cloud frame of a new segment, and the new segment is taken as the current segment; taking the next point cloud frame of the current point cloud frame as the current frame, returning to step S212, and if there is no next point cloud frame, the segmentation ends.

3. The point cloud frame sequence compression method based on the maximum change direction according to claim 1, characterized in that, The layered projection method projects the current point cloud frame onto the projection direction of the current projection coordinate system, including: S131, constructing a set of points to be projected from all points of the current point cloud frame; let k=1, L=1; S132, using the k-th coordinate axis of the current projection coordinate system as the reference axis; S133, sorting each point in the set of points to be projected in descending order of its value on the reference coordinate axis; for each sorted point, if a corresponding point in the L-th layer of the projection direction corresponding to the reference axis already exists, then the point is not projected; otherwise, the point is projected onto the corresponding position in the L-th layer of the projection direction corresponding to the reference axis, and the point is removed from the set of points to be projected; if the set of points to be projected is empty, the projection ends; otherwise, proceed to step S134; S134, k=k+1, if k=4, then L=L+1, set k to 1, and return to step S132; otherwise, return to step S132.

4. The point cloud frame sequence compression method based on the maximum change direction according to claim 1, characterized in that, Calculate the chamfer distance between the current point cloud frame and the previous point cloud frame in each projection direction of the current projection coordinate system, including: for the current projection direction of the current projection coordinate system, find the points of the current point cloud frame projected in the current projection direction to form a first set of points; find the points of the previous point cloud frame projected in the current projection direction to form a second set of points; calculate the chamfer distance between the first set of points and the second set of points to obtain the chamfer distance between the current point cloud frame and the previous point cloud frame in the current projection direction of the current projection coordinate system.

5. The point cloud frame sequence compression method based on the maximum change direction according to claim 4, characterized in that, Calculate the chamfer distance using the following formula. ： in, Describes the first set of points. express The number of midpoints Describes the second set of points. express The number of midpoints This represents the distance from point x to point y. This represents the distance from point y to point x.

6. The point cloud frame sequence compression method based on the maximum change direction according to claim 1, characterized in that, Projecting each point cloud frame of the segment onto the projection direction of the fusion coordinate system yields a complete projection image of each point cloud frame in each projection direction. This includes: using a layered projection method to project each point of each point cloud frame of the segment onto the corresponding projection direction of the fusion coordinate system, resulting in a multi-layered block projection map of each point cloud frame in each projection direction of the fusion coordinate system; and stitching together the multi-layered block projection maps of each point cloud frame in each projection direction to obtain a complete projection image of each point cloud frame in each projection direction.

7. The point cloud frame sequence compression method based on the maximum change direction according to claim 1, characterized in that, The structural similarity between a point cloud frame and its previous point cloud frame in each projection direction is calculated as follows: For the current projection direction, according to the formula... Calculate the SSIM value of the complete projected image of the point cloud frame and its predecessor on the k-th image channel, respectively; where, This represents the mean value of the k-th channel of the complete projected image of the point cloud frame; This represents the mean value of the k-th channel of the complete projected image of the previous point cloud frame. This represents the element variance of the k-th channel of the complete projected image of the point cloud frame; This represents the element variance of the k-th channel of the complete projected image of the previous point cloud frame. This represents the element-wise covariance of the k-th channel of the complete projected image of the point cloud frame and the complete projected image of the previous point cloud frame. and The SSIM value is a constant; the SSIM values ​​of all image channels are added together to obtain the structural similarity value of the point cloud frame and the complete projected image of the previous point cloud frame in the current projection direction.

8. The point cloud frame sequence compression method based on the maximum change direction according to claim 1, characterized in that, For each point cloud frame, the fused coordinate system is calculated based on the coordinate systems of the largest changes in adjacent point cloud frames. This includes: for the current point cloud frame, shifting all the coordinate systems of the largest changes corresponding to the current point cloud frame to the origin so that they coincide with the origin of the reference coordinate system; for each coordinate axis of each largest change coordinate system, assigning it to the type corresponding to the coordinate axis of the reference coordinate system with the smallest angle to it; summing the unit vectors of the coordinate axes corresponding to each type to obtain the coordinate axes of the fused coordinate system corresponding to that type; and obtaining the fused coordinate system corresponding to the current point cloud frame.

Images