A dynamic point cloud compression and decompression system for three-dimensional objects

Abstract

A dynamic point cloud compression and decompression system for three-dimensional objects includes: a compression end, which calculates the maximum change coordinate system and maximum change direction of adjacent point cloud frames in a point cloud frame sequence of a three-dimensional object based on a multi-layer projection method; segments the point cloud frame sequence based on the maximum change direction of adjacent point cloud frames; calculates the fused projection coordinate system of each point cloud frame in each segment of the point cloud frame according to the maximum change coordinate system of the adjacent point cloud frames; projects each point cloud frame of each segment of the point cloud frame onto the projection direction of the corresponding fused projection coordinate system to obtain a complete projection image of each point cloud frame in each projection direction, and obtains a key frame image of each segment of the point cloud frame according to the similarity of the complete projection images of adjacent point cloud frames in each projection direction; uses the key frame image of each segment of the point cloud frame and the corresponding segment number, point cloud frame number, fused projection coordinate system and projection direction as compressed data of the point cloud sequence; and a decompression end, which decompresses the compressed data to obtain a three-dimensional point cloud frame sequence.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud compression and decompression, and in particular to a dynamic point cloud compression and decompression system for three-dimensional objects. Background Art

[0002] Point clouds can be divided into two types: static point clouds and dynamic point clouds. A static point cloud consists of a set of points with three-dimensional position information, each of which has one or more attributes for storing additional information. A dynamic point cloud consists of multiple consecutive static point clouds, each of which is called a frame of the dynamic point cloud. The position information of a point in a point cloud is generally the (x, y, z) coordinates in a Cartesian coordinate system, which are used to characterize the spatial topology and reconstruct the shape of three-dimensional objects. A common attribute of a point cloud is color information, usually using RGB values ​​to store the red, green, and blue channels, or using intensity values ​​to record the pulse return amplitude to occupy the color field.

[0003] Point clouds will play a crucial role in future real-time holographic scenarios, such as human-computer interaction. Devices based on LiDAR and photogrammetry can rapidly acquire large-scale, high-frequency, and dynamic point cloud information for real-time human-computer interaction. However, due to the uneven distribution of computing and storage resources across networks, transmission to user terminals for display, computing centers for processing, and storage centers for storage will become a key requirement. However, point cloud data is large, unstructured, and lacks metadata. Direct transmission consumes significant bandwidth, a major constraint on point cloud transmission. Point cloud compression is necessary to reduce the data size.

[0004] Currently, compression methods for dynamic point clouds can be roughly divided into two categories: compression methods based on three-dimensional structures and compression methods based on two-dimensional mapping. Compression methods based on three-dimensional structures directly study compression algorithms for point clouds, and perform traversal and matching through structured algorithms. Most of these methods are based on the octree data structure. A common problem is that as the depth of the octree increases, the computational cost of the algorithm increases exponentially. This problem is particularly serious for the compression of high-precision and high-detail point cloud sequences. The basic idea of ​​compression methods based on two-dimensional mapping is to project point cloud data onto a two-dimensional plane according to certain rules, and then use existing mature video encoding and decoding technology to compress and encode the two-dimensional mapping sequence of dynamic point clouds. Compared with compression algorithms based on three-dimensional structures, this type of algorithm has greatly improved compression efficiency in time and space, and can achieve significant results. In 2017, MPEG (Moving Picture Expert Group) under ISO / IEC JTC1 established relevant international standards and released TMC2 (Test Model Category 2) for dynamic human-computer interaction point cloud compression algorithms. They used a compression method based on two-dimensional mapping to compress point clouds, which has high compression efficiency. However, since the three-dimensional point cloud is mapped into two-dimensional space, the continuity of changes in the three-dimensional space is completely destroyed, resulting in the loss of some information in this type of compression method. The compression efficiency is low, the decompressed point cloud is highly distorted, and the compressed data volume is large, which is not convenient for transmission and storage. Summary of the Invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a dynamic point cloud compression and decompression system for three-dimensional objects, so as to solve the problems of low compression efficiency and high distortion rate of existing dynamic point cloud compression and decompression systems.

[0006] In one aspect, an embodiment of the present invention provides a dynamic point cloud compression and decompression system for a three-dimensional object, comprising:

[0007] On the compression side, the maximum change coordinate system and maximum change direction of adjacent point cloud frames in the point cloud frame sequence of the three-dimensional object are calculated based on the multi-layer projection method; the point cloud frame sequence is segmented based on the maximum change direction of adjacent point cloud frames; for each point cloud frame segment, the fused projection coordinate system of each point cloud frame is calculated based on the maximum change coordinate system of the adjacent point cloud frames; each point cloud frame of the segment is projected on the projection direction of the corresponding fused projection coordinate system to obtain the complete projection image of each point cloud frame in each projection direction, and the key frame image of each point cloud frame segment is obtained based on the similarity of the complete projection images of two adjacent point cloud frames in each projection direction;

[0008] The key frame images of each segment of the point cloud frame and the segment number, point cloud frame number, fusion projection coordinate system and projection direction corresponding to each key frame image are used as compressed data of the point cloud sequence of the three-dimensional object;

[0009] The decompression end decompresses the compressed data to obtain a three-dimensional point cloud frame sequence.

[0010] Based on a further improvement of the above technical solution, the decompression end decompresses the compressed data to obtain a three-dimensional point cloud frame sequence, including:

[0011] The linear interpolation method is used to calculate the non-key frame images and the corresponding point cloud frame numbers between the key frame images with the same segment number and the same projection direction;

[0012] For each point cloud frame number, the corresponding key frame image and / or non-key frame image is restored to three-dimensional points according to the corresponding fusion projection coordinate system to obtain the point cloud corresponding to the point cloud frame number; the point clouds corresponding to all point cloud frame numbers constitute a three-dimensional point cloud frame sequence.

[0013] Furthermore, the linear interpolation method is used to calculate the non-key frame images and the corresponding point cloud frame numbers between the key frame images with the same segment number and the same projection direction, including:

[0014] The key frame images with the same segment number and the same projection direction are sorted in ascending order according to the point cloud frame number as a group of key frame images;

[0015] For two adjacent keyframe images in each set of keyframe images, if the corresponding point cloud frame numbers are not continuous, the non-keyframe images between the two adjacent keyframe images are calculated according to the following formula:

[0016]

[0017] Among them, si represents the point cloud frame number corresponding to the first key frame image in two adjacent key frame images, sj represents the point cloud frame number corresponding to the second key frame image in two adjacent key frame images, and I si Represents the image data of the first key frame image, I sj Represents the image data of the second key frame image, |I sj -I si | represents the calculated image I si and|I sj The difference of the corresponding elements, sk represents the point cloud frame number between si and sj, si<sk<sj, P sk Represents the image data of the non-keyframe image corresponding to the point cloud frame number sk.

[0018] Furthermore, for each point cloud frame number, the corresponding key frame image and / or non-key frame image is subjected to three-dimensional point restoration according to the corresponding fusion projection coordinate system to obtain the point cloud corresponding to the point cloud frame number, including:

[0019] According to the relationship between the fusion projection coordinate system and the reference coordinate system, the point corresponding to each pixel in each key frame image and / or non-key frame image corresponding to the current point cloud frame number is converted to the reference coordinate system to obtain the point cloud corresponding to the current point cloud frame number.

[0020] Furthermore, for each point cloud frame, the fusion projection coordinate system of each point cloud frame is calculated according to the maximum change coordinate system of adjacent point cloud frames, including:

[0021] For the current segment point cloud frame, all the maximum change coordinate system translation values ​​corresponding to the current segment point cloud frame are made to coincide with the origin of the reference coordinate system;

[0022] For each coordinate axis of each maximum variation coordinate system, classify it into the type corresponding to the coordinate axis of the base coordinate system with the smallest angle;

[0023] The unit vectors of the coordinate axes corresponding to each type are added together to obtain the coordinate axes of the fused coordinate system corresponding to the type; and the fused coordinate system corresponding to the current segment point cloud frame is obtained;

[0024] The origin of the fusion coordinate system corresponding to the current segment point cloud frame is translated to the center of mass of each point cloud frame in the current segment point cloud frame to obtain the fusion projection coordinate system corresponding to each point cloud frame.

[0025] Furthermore, the compression end calculates the maximum change coordinate system and maximum change direction of adjacent point cloud frames in the point cloud frame sequence based on a multi-layer projection algorithm, including:

[0026] S11. For the first point cloud frame in the point cloud frame sequence, establish a coordinate system with its center of mass as the coordinate origin, which is parallel to and has the same direction as the coordinate axis of the reference coordinate system as the final projection coordinate system of the first point cloud frame; and take the second point cloud frame as the current point cloud frame;

[0027] S12, establishing a coordinate system with the center of mass of the current point cloud frame as the coordinate origin, which is parallel to and has the same direction as the coordinate axis of the final projection coordinate system of the previous frame of the current point cloud frame as the current projection coordinate system of the current point cloud frame;

[0028] S13, projecting the current point cloud frame onto the projection direction of the current projection coordinate system based on a layered projection method; calculating the chamfer distance between the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of the current projection coordinate system, and the projection direction with the largest chamfer distance is the maximum projection direction of the current projection coordinate system;

[0029] S14. Fix the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system, and rotate the two coordinate axes other than the fixed coordinate axis around the fixed coordinate axis multiple times according to the rotation step size, obtaining a candidate coordinate system with each rotation, thereby obtaining multiple candidate coordinate systems that are coaxial with the fixed coordinate axis of the current projection coordinate system;

[0030] S15. Project the current point cloud frame onto the projection direction of each candidate coordinate system based on the layered projection method; calculate the chamfer distance of the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of each candidate coordinate system; if there is a chamfer distance in all projection directions that is greater than the chamfer distance of the maximum projection direction of the current projection coordinate system, then the coordinate system where the coordinate axis corresponding to the projection direction with the largest chamfer distance among all projection directions and the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system are located is the current projection coordinate system, and the projection direction with the largest chamfer distance among all projection directions is the maximum projection direction of the current projection coordinate system, and the process returns to step S14;

[0031] Otherwise, the current projection coordinate system is the coordinate system with the maximum change from the previous frame of the current point cloud frame to the current point cloud frame, and the maximum projection direction of the current projection coordinate system is the direction of the maximum change from the previous frame of the current point cloud frame to the current point cloud frame; the current projection coordinate system is the final projection coordinate system of the current point cloud frame; take the next point cloud frame as the current point cloud frame, and return to step S12 until all point cloud frames are traversed.

[0032] Furthermore, the current point cloud frame is projected onto the projection direction of the current projection coordinate system based on a layered projection method, including:

[0033] S131, all points of the current point cloud frame constitute a set of points to be projected; let k = 1, L = 1;

[0034] S132, taking the kth coordinate axis of the current projection coordinate system as a reference axis;

[0035] S133. Sort each point in the set of points to be projected in descending order according to their values ​​on the reference coordinate axis. For each sorted point, if a projection point already exists at a corresponding position of the Lth layer in the projection direction corresponding to the reference axis, then the point is not projected. Otherwise, the point is projected at a corresponding position of the Lth layer in the projection direction corresponding to the reference axis and deleted from the set of points to be projected. If the set of points to be projected is empty, the projection ends. Otherwise, execute step S134.

[0036] S134, k=k+1. If k=4, then L=L+1, set k to 1, and return to step S132; otherwise, return to step S132.

[0037] Furthermore, the compression end projects each point cloud frame of the segment of point cloud frames onto the projection direction of the corresponding fusion projection coordinate system to obtain a complete projection image of each point cloud frame in each projection direction, including:

[0038] A layered projection method is used to project each point of each point cloud frame of the segment onto the projection direction corresponding to the corresponding fusion projection coordinate system, and a multi-layer block projection map of each point cloud frame in each projection direction corresponding to the fusion projection coordinate system is obtained;

[0039] The multi-layer block projection images of each point cloud frame in each projection direction are stitched together to obtain the complete projection image of each point cloud frame in each projection direction.

[0040] Furthermore, the compression end obtains the key frame image of each point cloud frame according to the similarity of the complete projection images of two adjacent point cloud frames in each projection direction, including:

[0041] The complete projection images of the first point cloud frame and the last point cloud frame in each projection direction of the current segment point cloud frame are both key frame images;

[0042] For other point cloud frames in the current segment of point cloud frames, the structural similarity of the complete projection image of the point cloud frame and its previous point cloud frame in each projection direction is calculated; if the structural similarity does not exceed the second threshold, the complete projection image is used as the key frame image.

[0043] Furthermore, the structural similarity of the complete projection image of the point cloud frame and its previous point cloud frame in each projection direction is calculated in the following way:

[0044] For the current projection direction, according to the formula Calculate the SSIM value of the complete projection image of the point cloud frame and its previous point cloud frame on the kth image channel respectively;

[0045] Among them, μ k,a Represents the element mean of the kth channel of the complete projection image of the point cloud frame; μ k,b Represents the element mean of the kth channel of the complete projection image of the previous point cloud frame of the point cloud frame; σ k,a Represents the element variance of the kth channel of the complete projection image of the point cloud frame; σ k,b Represents the element variance of the kth channel of the complete projection image of the previous point cloud frame of the point cloud frame, σ k,ab Represents the element covariance of the kth channel of the complete projection image of the point cloud frame and the complete projection image of the previous point cloud frame of the point cloud frame, c1 and c2 are constants;

[0046] The SSIM values ​​of all image channels are added together to obtain the structural similarity value of the complete projection image of the point cloud frame and its previous point cloud frame in the current projection direction.

[0047] Compared with the existing system, the present invention calculates the maximum change direction of the point cloud frame and the corresponding maximum change coordinate system at the compression end, segments the point cloud frame according to the maximum change direction, and fuses the coordinates of each segment according to the maximum change to obtain the movement direction of each point cloud frame, and projects in the projection direction of the fused projection coordinate system to more accurately evaluate the similarity between the point cloud frames. Compression is performed according to the similarity in each projection direction to improve the compression rate, and the amount of compressed data is smaller. Decompression is performed on the compressed data at the decompression end, so as to more accurately restore the point cloud sequence of the three-dimensional object, and the amount of data transmitted between the compression end and the decompression end is smaller, thereby improving the transmission efficiency.

[0048] In the present invention, the above-mentioned technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of the present invention will be described in the following description, and some advantages will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0050] Figure 1 A block diagram of a dynamic point cloud compression and decompression system for three-dimensional objects according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0052] A specific embodiment of the present invention discloses a dynamic point cloud compression and decompression system for three-dimensional objects, such as Figure 1 As shown, including:

[0053] At the compression end, the maximum change coordinate system and maximum change direction of adjacent point cloud frames in a point cloud frame sequence of a three-dimensional object are calculated based on a multi-layer projection method; the point cloud frame sequence is segmented based on the maximum change direction of adjacent point cloud frames; for each point cloud frame segment, the fused projection coordinate system of each point cloud frame is calculated based on the maximum change coordinate system of adjacent point cloud frames; each point cloud frame of the segment is projected in the projection direction of the fused projection coordinate system to obtain a complete projection image of each point cloud frame in each projection direction, and the key frame image of each point cloud frame segment is obtained based on the similarity of the complete projection images of two adjacent point cloud frames in each projection direction;

[0054] The key frame images of each segment of the point cloud frame and the segment number, point cloud frame number, fusion projection coordinate system and projection direction corresponding to each key frame image are used as compressed data of the point cloud sequence of the three-dimensional object;

[0055] The decompression end decompresses the compressed data to obtain a three-dimensional point cloud frame sequence.

[0056] Compared to existing systems, the present invention calculates the maximum change direction of a point cloud frame and the corresponding maximum change coordinate system at the compression end, segments the point cloud frame according to the maximum change direction, and fuses the coordinates of each segment based on the maximum change to obtain the motion direction of each point cloud frame segment. Projection is then performed in the projection direction of the fused projection coordinate system, thereby more accurately assessing the similarity between point cloud frames. Compression is performed based on the similarity in each projection direction, thereby improving the compression rate and reducing the amount of compressed data. Decompression is performed on the compressed data at the decompression end, thereby more accurately restoring the point cloud sequence of the three-dimensional object. Furthermore, the amount of data transmitted between the compression and decompression ends is reduced, thereby improving transmission efficiency. During implementation, the three-dimensional object can be an indivisible object such as a person or a car.

[0057] Specifically, the compression end calculates the maximum change coordinate system and maximum change direction of adjacent point cloud frames in the point cloud frame sequence based on a multi-layer projection algorithm, including steps S11-S15:

[0058] S11. For the first point cloud frame in the point cloud frame sequence, establish a coordinate system with its center of mass as the coordinate origin, which is parallel to and has the same direction as the coordinate axis of the reference coordinate system as the final projection coordinate system of the first point cloud frame; and take the second point cloud frame as the current point cloud frame;

[0059] S12, establishing a coordinate system with the center of mass of the current point cloud frame as the coordinate origin, which is parallel to and has the same direction as the coordinate axis of the final projection coordinate system of the previous frame of the current point cloud frame as the current projection coordinate system of the current point cloud frame;

[0060] S13, projecting the current point cloud frame onto the projection direction of the current projection coordinate system based on a layered projection method; calculating the chamfer distance between the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of the current projection coordinate system, and the projection direction with the largest chamfer distance is the maximum projection direction of the current projection coordinate system;

[0061] S14. Fix the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system, and rotate the two coordinate axes other than the fixed coordinate axis around the fixed coordinate axis multiple times according to the rotation step size, obtaining a candidate coordinate system with each rotation, thereby obtaining multiple candidate coordinate systems that are coaxial with the fixed coordinate axis of the current projection coordinate system;

[0062] S15. Project the current point cloud frame onto the projection direction of each candidate coordinate system based on the layered projection method; calculate the chamfer distance of the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of each candidate coordinate system; if there is a chamfer distance in all projection directions that is greater than the chamfer distance of the maximum projection direction of the current projection coordinate system, then the coordinate system where the coordinate axis corresponding to the projection direction with the largest chamfer distance among all projection directions and the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system are located is the current projection coordinate system, and the projection direction with the largest chamfer distance among all projection directions is the maximum projection direction of the current projection coordinate system, and the process returns to step S14;

[0063] Otherwise, the current projection coordinate system is the coordinate system with the maximum change from the previous frame of the current point cloud frame to the current point cloud frame, and the maximum projection direction of the current projection coordinate system is the direction of the maximum change from the previous frame of the current point cloud frame to the current point cloud frame; the current projection coordinate system is the final projection coordinate system of the current point cloud frame; take the next point cloud frame as the current point cloud frame, and return to step S12 until all point cloud frames are traversed.

[0064] It should be noted that the reference coordinate system is a reference coordinate system. For example, the world coordinate system can be used as the reference coordinate system.

[0065] During implementation, for the first frame of the point cloud sequence, a coordinate system parallel to and in the same direction as the coordinate axis of the reference coordinate system is directly established with its center of mass as the coordinate origin as its final projection coordinate system. For the second to last frames, it is necessary to obtain its final projection coordinate system according to steps S12-S15.

[0066] For example, for the i-th point cloud frame, first, the center of mass of the i-th point cloud frame is used as the coordinate origin to establish a coordinate system that is parallel to and has the same direction as the coordinate axis of the final projection coordinate system of the i-1-th point cloud frame as the first projection coordinate system of the i-th point cloud frame. i X1Y1Z1 represents the current projection coordinate system.

[0067] Since the movement of three-dimensional objects is a continuous process, the change direction of the previous moment and the change direction of the next moment do not change much most of the time. Therefore, the maximum change direction from the i-2th point cloud frame to the i-1th point cloud frame is also the maximum change direction from the i-1th point cloud frame to the i-th point cloud frame most of the time. Therefore, for the i-th point cloud frame, a coordinate system is first established that is parallel to the coordinate axis of the final projection coordinate system of the i-1th point cloud frame and has the same direction, so that the maximum change coordinate system and maximum change direction from the i-1th point cloud frame to the i-th point cloud frame can be found more quickly, reducing the amount of calculation and improving efficiency.

[0068] Based on the layered projection method, the i-th point cloud frame is projected into the current projection coordinate system O i In the projection direction of X1Y1Z1, the current projection coordinate system O i X1Y1Z1 including O i The direction of the X1 axis, O i The direction of the Z1 axis and the O i In the direction of the Y1 axis, calculate the chamfer distance between the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of the current projection coordinate system. The projection direction with the largest chamfer distance is the maximum projection direction of the current projection coordinate system.

[0069] For example, the current projected coordinate system O i The maximum projection direction of X1Y1Z1 is O i The direction of the X1 axis, O i The direction of the X1 axis is not necessarily the direction of the maximum change from the i-1 point cloud frame to the i-th point cloud frame, so it is necessary to find out whether there are other directions where the difference between the two frames is greater. i The X1 axis is fixed, and the other two coordinate axes, namely O i Z1 and O i Y1 axis around O i The X1 axis rotates multiple times to obtain the coordinate system O i X1Y1Z1O i The X1 axis is coaxial with multiple candidate coordinate systems. Based on the layered projection method, the i-th point cloud frame is projected in the projection direction of each candidate coordinate system; the chamfer distance of the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of each candidate coordinate system is calculated. If there is a chamfer distance greater than the current projection coordinate system in all projection directions of the candidate coordinate system, the chamfer distance is greater than the current projection coordinate system. i The maximum projection direction of X1Y1Z1 (O i X1 axis direction), then the coordinate axis corresponding to the projection direction with the largest chamfer distance in all projection directions and the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system (O i The coordinate system where the X1 axis is located (marked as O iX2Y2Z2) is the current projection coordinate system, and the projection direction with the largest chamfer distance among all projection directions (for example, O i O in the X2Y2Z2 coordinate system i Y2 axis direction) is the maximum projection direction of the current projection coordinate system; i O in the X2Y2Z2 coordinate system i The Y2 axis direction is not necessarily the direction of the maximum change from the i-1 point cloud frame to the i point cloud frame. It is also necessary to find out whether there are other directions where the difference between the two frames is greater. i O in the X2Y2Z2 coordinate system i The Y2 axis is fixed and the other two coordinate axes rotate around O according to the rotation step. i The Y2-axis is rotated to generate multiple new candidate coordinate systems. According to the above steps, it is found whether there is a projection direction with a larger chamfer distance. And so on, until there is no projection direction with a chamfer distance larger than the chamfer distance of the maximum projection direction of the current projection coordinate system in the candidate coordinate system. At this time, the maximum projection direction of the current projection coordinate system is the maximum change direction from the previous frame of the current point cloud frame to the current point cloud frame; the current projection coordinate system is the final projection coordinate system of the current point cloud frame; then start with the next point cloud frame as the current point cloud frame, and calculate its maximum change direction according to the above process until all point cloud frames in the point cloud frame sequence are traversed.

[0070] During implementation, the rotation step can be determined based on the calculation speed and accuracy of the change direction. If high calculation accuracy is required, the rotation step can be set to a smaller number, such as 10 degrees. If high calculation speed is required, the rotation step can be set to a larger number, such as 30 degrees.

[0071] Each coordinate system of each point cloud frame corresponds to multiple projection directions. In one embodiment of the present invention, one coordinate system corresponds to three projection directions. For example, for the jth coordinate system O of the i-th point cloud frame i X j Y j Z j , the three projection directions are O i X j Axis direction, O i Z j The direction of the axis and O i Y j The direction of the axis.

[0072] Specifically, in step S13, the current point cloud frame is projected onto the projection direction of the current projection coordinate system based on the layered projection method, including:

[0073] S131, all points of the current point cloud frame constitute a set of points to be projected; let k = 1, L = 1;

[0074] It should be noted that k represents the kth coordinate axis of the jth projection coordinate system, and L represents the first projection layer of the jth projection coordinate system. Initially, k=1 and L=1.

[0075] S132, taking the kth coordinate axis of the current projection coordinate system as a reference axis;

[0076] S133. Sort each point in the set of points to be projected in descending order according to their values ​​on the reference coordinate axis. For each sorted point, if a projection point already exists at a corresponding position of the Lth layer in the projection direction corresponding to the reference axis, then the point is not projected. Otherwise, the point is projected at a corresponding position of the Lth layer in the projection direction corresponding to the reference axis and deleted from the set of points to be projected. If the set of points to be projected is empty, the projection ends. Otherwise, execute step S134.

[0077] S134, k=k+1. If k=4, then L=L+1, set k to 1, and return to step S132; otherwise, directly return to step S132.

[0078] During implementation, for the current projection coordinate system of the current point cloud frame (e.g., the i-th point cloud frame), for example, the j-th coordinate system O i X j Y j Z j , all points of the current point cloud frame constitute the set of points to be projected.

[0079] Coordinate system O i X j Y j Z j The first coordinate axis is used as the reference axis, for example, the X axis is used as the reference axis. At this time, the projection direction is O i X j In the direction of the axis, each point in the set of points to be projected is sorted in descending order according to its value on the X axis, so that the points with larger coordinate values ​​are projected first. According to the order of the sorted points, each point is judged in turn whether it can be projected on O i X j Axis projection direction of layer 1.

[0080] The first point after sorting can be projected on the first layer of the axis direction. i X j The first layer in the axis direction, mark the point at O i X j The projection position of the first layer in the axis direction is occupied, and the point is deleted from the set of points to be projected. Starting from the second point, it is necessary to determine whether the point is in O i X jIs the projection position of the first layer in the axis direction occupied? If it is occupied, that is, there is a point projected at this position, then the point will not be projected and it will remain in the set of points to be projected. Otherwise, the point will be projected at O i X j The first layer in the axis direction, mark the point at O i X j The projection position of the first layer in the axial direction is occupied, and the point is deleted from the set of points to be projected.

[0081] After all the points to be projected are traversed, if the set of points to be projected is empty, the projection ends; if the set of points to be projected is not empty, that is, there are points that are not projected, the current projection coordinate system O i X j Y j Z j The second coordinate axis is used as the reference axis, for example, the Y axis is used as the reference axis. At this time, the projection direction is O i Y j In the direction of the axis, each point in the set of points to be projected is sorted in descending order according to its value on the Y axis, so that the points with larger coordinate values ​​are projected first. According to the order of the sorted points, each point is judged in turn whether it can be projected on O i Y j The first layer in the axis direction, the specific judgment process is as mentioned above. After all the projection point sets are traversed, if the projection point set is empty, the projection ends. If the projection point set is still not empty, the current projection coordinate system O i X j Y j Z j The third coordinate axis Z is used as the reference axis, and the points in the set of points to be projected are projected in the third projection direction O according to the same process. i Z j On the first layer in the axial direction.

[0082] O i X j Y j Z j After polling all three coordinate axes as reference axes, there may still be some points that have not been projected due to the existence of occlusion relationship, that is, the set of points to be projected is still not empty, then the number of layers is increased by one, that is, L is increased by 1, and the points in the set of points to be projected are projected on the second layer of the three projection directions according to the above process. If there are still some points that have not been projected, the points in the set of points to be projected are projected on the third layer of the three projection directions, and so on, until the set of points to be projected is empty.

[0083] Compared with the prior art, the present invention projects the point cloud onto multiple layers in the projection direction through a layered projection method, taking into account the occlusion relationship of the points, thereby providing a basis for accurately calculating the maximum change direction.

[0084] Specifically, in step S13, the chamfer distance between the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of the current projection coordinate system is calculated, including:

[0085] For the current projection direction of the current projection coordinate system, find the projection of the current point cloud frame in the current projection direction to form a first point set;

[0086] Find the point of the previous frame of the current point cloud frame projected in the current projection direction to form a second point set;

[0087] The chamfer distances between the first point set and the second point set are calculated to obtain the chamfer distances between the current point cloud frame and the previous frame of the current point cloud frame in the current projection direction of the current projection coordinate system.

[0088] It should be noted that for the current projection coordinate system (O i X j Y j Z j ), if the previous point cloud frame (i-1 point cloud frame) exists in the current projection coordinate system (O i X j Y j Z j ) is a coordinate system in which the three axes are parallel and in the same direction (marked as O i-1 X m Y m Z m ), since the coordinate axes are parallel and in the same direction, the current projected coordinate system (O i X j Y j Z j ) and the three projection directions of the coordinate system O i X m Y m Z m The direction is consistent, so we can directly find the point of the previous frame of the previous point cloud frame projected in the current projection direction to form the second point set.

[0089] If the previous point cloud frame (i-1 point cloud frame) does not exist in the current projection coordinate system (O i X j Y j Z j ) is a coordinate system in which the three coordinate axes are parallel and in the same direction. At this time, the current projection coordinate system (O i X j Y j Z j ) is moved to the center of mass of the previous point cloud frame (the i-1 point cloud frame) to obtain the coordinate system (marked as O i-1 Xm Y m Z m ), project the i-1 point cloud frame onto O according to the aforementioned layered projection method i-1 X m Y m Z m In the projection direction, since O i X j Y j Z j Translation, so the current projected coordinate system (O i X j Y j Z j ) and the three projection directions of the coordinate system O i X m Y m Z m The direction is consistent, and the point of the previous frame of the previous point cloud frame projected in the current projection direction is found to form the second point set.

[0090] For example, the current point cloud frame (i-th point cloud frame) is in the current projection coordinate system O i X j Y j Z j 8 layers are projected down, then all the points on the 8 layers projected in the direction of the current point cloud frame are taken to form the first point set. i-1 X m Y m Z m Points on all layers (eg, 10 layers) of the corresponding direction constitute a second point set.

[0091] Specifically, the chamfer distance d(S1, S2) between the first point set and the second point set is calculated using the following formula:

[0092]

[0093] Among them, S1 represents the first point set, represents the number of points in S1, S2 represents the second set of points, represents the number of points in S2, Represents the distance from point x to point y, Indicates the distance from point y to point x.

[0094] The calculated chamfer distance between the first point set and the second point set is expressed in the current projection coordinate system O i X j Y j Z j Under this condition, the current point cloud frame (i-th point cloud frame) and its previous point cloud frame are in Oi X j The difference in the axis direction, the larger the chamfer distance, the greater the difference. The same process is used to calculate the difference between the current point cloud frame (i-th point cloud frame) and its previous point cloud frame in O i Y j Axis and O i Z j Difference in axis direction.

[0095] After calculating the maximum change coordinate system of each two adjacent point cloud frames in the point cloud frame sequence, the point cloud frame sequence is segmented. Specifically, segmenting the point cloud frame sequence based on the maximum change direction of adjacent point cloud frames includes:

[0096] S211, taking the first point cloud frame of the point cloud frame sequence as the starting point cloud frame of the first segment; taking the first segment as the current segment; and taking the second point cloud frame as the current point cloud frame;

[0097] S212: If the angle between the maximum change direction of the current point cloud frame and the maximum change direction of the starting point cloud frame of the current segment is less than a first threshold, the current point cloud frame is added to the current segment. Otherwise, the current point cloud frame is used as the starting point cloud frame of a new segment, and the new segment is used as the current segment. The next point cloud frame after the current point cloud frame is used as the current frame, and the process returns to step S212. If there is no next point cloud frame, the segmentation process ends.

[0098] During implementation, the first threshold may be determined according to calculation accuracy requirements, for example, it may be set to 15 degrees.

[0099] That is, frames with basically the same change direction are grouped into one segment, thereby improving the compression rate and the accuracy of the compressed data. For each segment of point cloud frame, the maximum change coordinate system of its adjacent point cloud frames is fused to facilitate point cloud compression.

[0100] Specifically, for each point cloud frame in the compression end, the fusion projection coordinate system of each point cloud frame is calculated based on the maximum change coordinate system of adjacent point cloud frames, including:

[0101] S311 . For the current segment point cloud frame, translate all the maximum change coordinate systems corresponding to the current segment point cloud frame until the origin coincides with the origin of the reference coordinate system.

[0102] During implementation, the maximum change coordinate system of all adjacent point cloud frames in the current segment point cloud frame is translated to coincide with the origin O0 of the reference coordinate system O0X0Y0Z0.

[0103] S312 . For each coordinate axis of each maximum variation coordinate system, classify it into the type corresponding to the coordinate axis of the reference coordinate system that has the smallest angle with it.

[0104] For each coordinate axis of each maximum change coordinate system, determine the angle between it and the three coordinate axes of the reference coordinate system. If the angle between the current coordinate axis and the O0X0 axis of the reference coordinate system is the smallest, the current coordinate axis is classified as the first category; if the angle between the current coordinate axis and the O0Y0 axis of the reference coordinate system is the smallest, the current coordinate axis is classified as the second category; if the angle between the current coordinate axis and the O0Z0 axis of the reference coordinate system is the smallest, the current coordinate axis is classified as the third category.

[0105] It should be noted that there may be a coordinate axis of the maximum change coordinate system that has the same and smallest angle with two coordinate axes of the reference coordinate system. In this case, it is sufficient to ensure that the three coordinate axes of the current maximum change coordinate system belong to different categories.

[0106] S313. Add the unit vectors of the coordinate axes corresponding to each type to obtain the coordinate axes of the fused coordinate system corresponding to the type; and obtain the fused coordinate system corresponding to the current segment point cloud frame.

[0107] The unit vectors corresponding to the coordinate axes of the same type are summed to obtain the coordinate axes corresponding to the fusion coordinate system. The fusion coordinate system corresponding to the s-th segment point cloud frame sequence is expressed as express.

[0108] S314 , translating the origin of the fusion coordinate system corresponding to the current segment point cloud frame to the centroid of each point cloud frame in the current segment point cloud frame to obtain the fusion projection coordinate system corresponding to each point cloud frame.

[0109] During implementation, for the i-th point cloud frame in the s-th point cloud frame sequence, first the fusion coordinate system Translate to the center of mass of the point cloud frame to obtain the fusion projection coordinate system corresponding to the i-th point cloud frame The corresponding projection direction is Axis direction, Axis direction and Direction.

[0110] Since the fused projection coordinate system of each point cloud frame is obtained by translating the fused coordinate system corresponding to the segment, the projection direction of each point cloud frame is the same as the projection direction of the fused coordinate system.

[0111] Specifically, the compression end projects each point cloud frame of the segment of point cloud frames in the projection direction of the fused projection coordinate system to obtain a complete projection image of each point cloud frame in each projection direction, including:

[0112] S321, projecting each point of each point cloud frame of the segment onto a projection direction corresponding to the fused projection coordinate system using a layered projection method, to obtain a multi-layer block projection image of each point cloud frame in each projection direction corresponding to the fused projection coordinate system;

[0113] During implementation, the i-th point cloud frame in the s-th point cloud frame sequence is projected onto the fusion projection coordinate system in the same manner as in steps S131-S134. In the corresponding projection direction, the multi-layer block projection map of the i-th point cloud frame in the s-th point cloud frame sequence is obtained in each projection direction. For example, the projection of the i-th point cloud frame is divided into 7 layers, then Projection direction, Projection direction and The projection directions correspond to 7 block projection images.

[0114] It should be noted that each pixel in each layer of the projection image records the coordinate value and attribute index of the point in the point cloud frame projected at that location in the fused projection coordinate system, expressed as (x, y, z, a). If no point is projected at that location, the corresponding pixel at that location is (0, 0, 0, 0). In implementation, the attribute can be a color attribute.

[0115] S322 , stitching the multi-layer block projection images of each point cloud frame in each projection direction to obtain a complete projection image of each point cloud frame in each projection direction.

[0116] After obtaining the multi-layer block projection image of each point cloud frame in each projection direction in the current segment of the point cloud frame sequence, for each point cloud frame, for example, the i-th point cloud frame in the s-th segment of the point cloud frame sequence, the multi-layer block projection images in each projection direction are spliced ​​respectively to obtain the complete projection image of the i-th point cloud frame in the s-th segment of the point cloud frame sequence on each projection surface.

[0117] During implementation, it is assumed that the sth segment of the point cloud frame sequence contains 10 point cloud frames. For each block projection of these 10 point cloud frames, the origin of the corresponding projection coordinate system is the point corresponding to the block projection image to establish a two-dimensional coordinate system. The maximum absolute value of the coordinate value in the two-dimensional coordinate system is the side length of the block projection image. The maximum side length of all block projection images is L max , the dimension of the benchmark block map is 2L max ×2L max For example, if the maximum side length is 100, the dimension of the benchmark block image is 200x200. Each block projection image is expanded to the size of the benchmark block image, that is, the origin of the projection coordinate system corresponding to the block projection image is the point corresponding to the block projection image. The two-dimensional coordinate system is established with the X-axis of the two-dimensional coordinate system at -2L. max to 2L max , Y-axis -2L max to 2L max The positions without projection points in the range are filled with 0 to obtain the filled block projection map.

[0118] For the i-th point cloud frame, The projection direction corresponds to the block projection images, which are spliced ​​together in the order from the first layer to the last layer to obtain the i-th point cloud frame. The complete projected image in the projection direction; The projection direction corresponds to the block projection images, which are spliced ​​together in the order from the first layer to the last layer to obtain the i-th point cloud frame. The complete projected image in the projection direction; The projection direction corresponds to the block projection images, which are spliced ​​together in the order from the first layer to the last layer to obtain the i-th point cloud frame. The complete projected image in the projection direction.

[0119] Specifically, the compression end obtains the key frame image of each point cloud frame based on the similarity of the complete projection images of two adjacent point cloud frames in each projection direction, including:

[0120] S331, the complete projection images of the first point cloud frame and the last point cloud frame in each projection direction of the current segment point cloud frame are both key frame images;

[0121] S332. For other point cloud frames in the current segment of point cloud frames, calculate the structural similarity of the complete projection image of the point cloud frame and its previous point cloud frame in each projection direction; if the structural similarity does not exceed a second threshold, use the complete projection image as a key frame image.

[0122] For example, for the i-th point cloud frame, we get Projection direction, Projection direction and The complete projection image in the projection direction is calculated respectively for the i-th point cloud frame in The complete projection image in the projection direction is the same as the i-1 point cloud frame The structural similarity of the complete projection image in the projection direction, the i-th point cloud frame in The complete projection image in the projection direction is the same as the i-1 point cloud frame The structural similarity of the complete projection image in the projection direction, the i-th point cloud frame in The complete projection image in the projection direction is the same as the i-1 point cloud frame Structural similarity of the complete projection image in the projection direction.

[0123] Need to explain, Fusion projected coordinate system to fusion coordinate system The coordinate system obtained by translating to the center of mass of the i-th point cloud frame of the s-th segment, Fusion projected coordinate system to fusion coordinate system The coordinate system is obtained by translating to the centroid of the i-1th point cloud frame of the sth segment, so, Projection direction and The projection direction is the same direction, that is, the fusion coordinate system of The direction of the axis; Projection direction and The projection direction is the same direction, that is, the fusion coordinate system of axis; Projection direction and The projection direction is the same direction, that is, the fusion coordinate system of axis.

[0124] If the i-th point cloud frame is The complete projection image in the projection direction is the same as the i-1 point cloud frame The structural similarity of the complete projection image in the projection direction is less than the second threshold, that is, the i-th point cloud frame and the i-1-th point cloud frame are If the difference in projection direction is large, the i-th point cloud frame is The complete projection image in the projection direction is used as the key frame image, otherwise, the i-th point cloud frame is discarded. The complete projected image in the projection direction.

[0125] Specifically, step S332 calculates the structural similarity of the complete projection image of the point cloud frame and its previous point cloud frame in each projection direction in the following manner:

[0126] S3321. For the current projection direction, according to the formula

[0127] Calculate the SSIM value of the complete projection image of the point cloud frame and its previous point cloud frame on the kth image channel respectively;

[0128] Among them, μ k,a Represents the element mean of the kth channel of the complete projection image of the point cloud frame; μ k,b Represents the element mean of the kth channel of the complete projection image of the previous point cloud frame of the point cloud frame; σ k,a Represents the element variance of the kth channel of the complete projection image of the point cloud frame; σ k,b Represents the element variance of the kth channel of the complete projection image of the previous point cloud frame of the point cloud frame, σ k,ab Represents the element covariance of the kth channel of the complete projection image of the point cloud frame and the complete projection image of the previous point cloud frame of the point cloud frame, c1 and c2 are constants.

[0129] It should be noted that each image channel is each coordinate and color index, that is, there are four channels corresponding to each image channel.

[0130] S3322. Add the SSIM values ​​of all image channels to obtain a structural similarity value of the complete projection image of the point cloud frame and its previous point cloud frame in the current projection direction.

[0131] The SSIM values ​​calculated from the four channels are added together to obtain the structural similarity value of the complete projection image of the point cloud frame and its previous point cloud frame in the current projection direction.

[0132] After obtaining the keyframe images for the s-th point cloud frame, the keyframe images contained in the s-th point cloud frame, as well as the segment sequence number, point cloud frame sequence number, fused projection coordinate system, and projection direction corresponding to each keyframe image, are used as the compressed data for the current point cloud segment. Since complete projection images with high similarity are not used as keyframe images, that is, fewer complete projection images are retained in directions with smaller changes, the amount of compressed data is greatly reduced, and the compression rate is greatly improved. In addition, more complete projection images are retained as keyframes in directions with larger changes, thereby reducing compression distortion. When the point cloud is restored based on the compressed data, the restoration degree is higher, the information of the original point cloud frame is retained, and the compression accuracy is higher.

[0133] Specifically, the decompression end decompresses the compressed data to obtain a three-dimensional point cloud frame sequence, including:

[0134] S41, using a linear interpolation method to calculate non-key frame images and corresponding point cloud frame numbers between key frame images with the same segment number and the same projection direction;

[0135] S42. For each point cloud frame number, perform three-dimensional point restoration on the corresponding key frame image and / or non-key frame image according to the corresponding fusion projection coordinate system to obtain the point cloud corresponding to the point cloud frame number; the point clouds corresponding to all point cloud frame numbers constitute a three-dimensional point cloud frame sequence.

[0136] Specifically, in step S41, the linear interpolation method is used to calculate the non-key frame images and the corresponding point cloud frame numbers between the key frame images with the same segment number and the same projection direction, including:

[0137] S411, sorting the key frame images with the same segment number and the same projection direction in ascending order of the point cloud frame number as a group of key frame images;

[0138] S412: For two adjacent key frame images in each set of key frame images, if the corresponding point cloud frame serial numbers are not continuous, calculate the non-key frame image between the two adjacent key frame images according to the following formula:

[0139]

[0140] Among them, si represents the point cloud frame number corresponding to the first key frame image in two adjacent key frame images, sj represents the point cloud frame number corresponding to the second key frame image in two adjacent key frame images, and I si Represents the image data of the first key frame image, I sj Represents the image data of the second key frame image, |I sj -I si | represents the calculated image I si and|I sj The difference of the corresponding elements, sk represents the point cloud frame number between si and sj, si<sk<sj, P sk Represents the image data of the non-keyframe image corresponding to the point cloud frame number sk.

[0141] Since the fusion coordinate systems corresponding to point cloud frames of different segments are different, the calculation of non-key frame images (i.e., the image corresponding to the complete projection image discarded in step S332) needs to be obtained based on the key frame images of the same segment. When implementing, first sort the key frame images of each projection direction of the sth segment (segment number is s) in the order of the point cloud frame numbers from small to large. For example, the fusion coordinate system of In the axial projection direction, the point cloud frame numbers are 12, 15, and 19 (i.e., the 12th, 15th, and 19th point cloud frames), then the point cloud frames with point cloud frame numbers of 13, 14, 16, 17, and 18 (i.e., the 13th, 14th, 16th, 17th, and 18th point cloud frames) need to be calculated. Non-key frame image in the axis projection direction. For the 13th point cloud frame The non-key frame images in the axis projection direction need to be calculated based on the projection images of the 12th and 15th point cloud frames.

[0142] According to the formula Calculate the 13th point cloud frame at Non-keyframe image P in the axial projection direction sk Among them, si=12,sj=15,sk=13,I si For the 12th point cloud frame Key frame image in the axis projection direction, I sj For the 15th point cloud frame The key frame image in the axis projection direction. sj -I si Indicates that the 12th point cloud frame is The key frame image and the 15th point cloud frame in the axis projection direction The difference between the corresponding elements of the key frame image in the axis projection direction (calculated separately for each channel).

[0143] After calculating the non-key frame image, in step S42, for each point cloud frame number, the corresponding key frame image and / or non-key frame image is subjected to 3D point restoration according to the corresponding fusion projection coordinate system to obtain the point cloud corresponding to the point cloud frame number, including:

[0144] According to the relationship between the fusion projection coordinate system and the reference coordinate system, the point corresponding to each pixel in each key frame image and / or non-key frame image corresponding to the current point cloud frame number is converted to the reference coordinate system to obtain the point cloud corresponding to the current point cloud frame number.

[0145] For each point cloud frame, the image projected in each projection direction may be a keyframe image or a non-keyframe image calculated according to the above steps, and the corresponding projection coordinate system is a fused projection coordinate system. The coordinate information recorded in the keyframe image and non-keyframe image is their coordinate information under the corresponding fused projection coordinates. Therefore, after extracting the pixel information of the keyframe images and / or non-keyframes in all projection directions corresponding to the point cloud frame, the point cloud of the point cloud frame in the reference coordinate system can be restored based on the relationship between the fused projection coordinate system and the reference coordinate system. The point clouds corresponding to all point cloud frame numbers constitute a three-dimensional point cloud frame sequence, that is, the decompressed point cloud sequence is obtained.

[0146] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0147] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A dynamic point cloud compression and decompression system for three-dimensional objects, characterized in that: include: On the compression side, the maximum change coordinate system and maximum change direction of adjacent point cloud frames in the point cloud frame sequence of the 3D object are calculated based on the multi-layer projection method; The point cloud frame sequence is segmented based on the maximum change direction of adjacent point cloud frames; for each point cloud frame segment, the fusion projection coordinate system of each point cloud frame is calculated based on the maximum change coordinate system of adjacent point cloud frames; Project each point cloud frame of the segment in the projection direction of the corresponding fusion projection coordinate system to obtain a complete projection image of each point cloud frame in each projection direction, and obtain a key frame image of each point cloud frame according to the similarity of the complete projection images of two adjacent point cloud frames in each projection direction; The key frame images of each segment of the point cloud frame and the segment number, point cloud frame number, fusion projection coordinate system and projection direction corresponding to each key frame image are used as compressed data of the point cloud sequence of the three-dimensional object; The decompression end decompresses the compressed data to obtain a three-dimensional point cloud frame sequence.

2. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 1, characterized in that: The decompression end decompresses the compressed data to obtain a three-dimensional point cloud frame sequence, including: The linear interpolation method is used to calculate the non-key frame images and the corresponding point cloud frame numbers between the key frame images with the same segment number and the same projection direction; For each point cloud frame number, the corresponding key frame image and / or non-key frame image is restored to three-dimensional points according to the corresponding fusion projection coordinate system to obtain the point cloud corresponding to the point cloud frame number; the point clouds corresponding to all point cloud frame numbers constitute a three-dimensional point cloud frame sequence.

3. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 2, characterized in that: The linear interpolation method is used to calculate the non-key frame images and the corresponding point cloud frame numbers between key frame images with the same segment number and the same projection direction, including: The key frame images with the same segment number and the same projection direction are sorted in ascending order according to the point cloud frame number as a group of key frame images; For two adjacent keyframe images in each set of keyframe images, if the corresponding point cloud frame numbers are not continuous, the non-keyframe images between the two adjacent keyframe images are calculated according to the following formula: Among them, si represents the point cloud frame number corresponding to the first key frame image in two adjacent key frame images, sj represents the point cloud frame number corresponding to the second key frame image in two adjacent key frame images, and I si Represents the image data of the first key frame image, I sj Represents the image data of the second key frame image, |I sj -I si | represents the calculated image I si and|I sj The difference of the corresponding elements, sk represents the point cloud frame number between si and sj, si<sk<sj, P sk Represents the image data of the non-keyframe image corresponding to the point cloud frame number sk.

4. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 2, characterized in that: For each point cloud frame number, the corresponding key frame image and / or non-key frame image is restored to three-dimensional points according to the corresponding fusion projection coordinate system to obtain the point cloud corresponding to the point cloud frame number, including: According to the relationship between the fusion projection coordinate system and the reference coordinate system, the point corresponding to each pixel in each key frame image and / or non-key frame image corresponding to the current point cloud frame number is converted to the reference coordinate system to obtain the point cloud corresponding to the current point cloud frame number.

5. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 1, characterized in that: For each point cloud frame, the fusion projection coordinate system of each point cloud frame is calculated based on the maximum change coordinate system of adjacent point cloud frames, including: For the current segment point cloud frame, all the maximum change coordinate system translation values ​​corresponding to the current segment point cloud frame are made to coincide with the origin of the reference coordinate system; For each coordinate axis of each maximum variation coordinate system, classify it into the type corresponding to the coordinate axis of the base coordinate system with the smallest angle; The unit vectors of the coordinate axes corresponding to each type are added together to obtain the coordinate axes of the fused coordinate system corresponding to the type; and the fused coordinate system corresponding to the current segment point cloud frame is obtained; The origin of the fusion coordinate system corresponding to the current segment point cloud frame is translated to the center of mass of each point cloud frame in the current segment point cloud frame to obtain the fusion projection coordinate system corresponding to each point cloud frame.

6. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 1, characterized in that: The compression end calculates the maximum change coordinate system and maximum change direction of adjacent point cloud frames in the point cloud frame sequence based on a multi-layer projection algorithm, including: S11. For the first point cloud frame in the point cloud frame sequence, establish a coordinate system with its center of mass as the coordinate origin, which is parallel to and has the same direction as the coordinate axis of the reference coordinate system as the final projection coordinate system of the first point cloud frame; and take the second point cloud frame as the current point cloud frame; S12, establishing a coordinate system with the center of mass of the current point cloud frame as the coordinate origin, which is parallel to and has the same direction as the coordinate axis of the final projection coordinate system of the previous frame of the current point cloud frame as the current projection coordinate system of the current point cloud frame; S13, projecting the current point cloud frame onto the projection direction of the current projection coordinate system based on a layered projection method; calculating the chamfer distance between the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of the current projection coordinate system, and the projection direction with the largest chamfer distance is the maximum projection direction of the current projection coordinate system; S14. Fix the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system, and rotate the two coordinate axes other than the fixed coordinate axis around the fixed coordinate axis multiple times according to the rotation step size, obtaining a candidate coordinate system with each rotation, thereby obtaining multiple candidate coordinate systems that are coaxial with the fixed coordinate axis of the current projection coordinate system; S15. Project the current point cloud frame onto the projection direction of each candidate coordinate system based on the layered projection method; calculate the chamfer distance of the current point cloud frame and the previous frame of the current point cloud frame in each projection direction of each candidate coordinate system; if there is a chamfer distance in all projection directions that is greater than the chamfer distance of the maximum projection direction of the current projection coordinate system, then the coordinate system where the coordinate axis corresponding to the projection direction with the largest chamfer distance among all projection directions and the coordinate axis corresponding to the maximum projection direction of the current projection coordinate system are located is the current projection coordinate system, and the projection direction with the largest chamfer distance among all projection directions is the maximum projection direction of the current projection coordinate system, and the process returns to step S14; Otherwise, the current projection coordinate system is the coordinate system with the maximum change from the previous frame of the current point cloud frame to the current point cloud frame, and the maximum projection direction of the current projection coordinate system is the direction of the maximum change from the previous frame of the current point cloud frame to the current point cloud frame; the current projection coordinate system is the final projection coordinate system of the current point cloud frame; take the next point cloud frame as the current point cloud frame, and return to step S12 until all point cloud frames are traversed.

7. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 6, characterized in that: Project the current point cloud frame in the projection direction of the current projection coordinate system based on the layered projection method, including: S131, all points of the current point cloud frame constitute a set of points to be projected; let k = 1, L = 1; S132, taking the kth coordinate axis of the current projection coordinate system as a reference axis; S133. Sort each point in the set of points to be projected in descending order according to their values ​​on the reference coordinate axis. For each sorted point, if a projection point already exists at a corresponding position of the Lth layer in the projection direction corresponding to the reference axis, then the point is not projected. Otherwise, the point is projected at a corresponding position of the Lth layer in the projection direction corresponding to the reference axis and deleted from the set of points to be projected. If the set of points to be projected is empty, the projection ends. Otherwise, execute step S134. S134, k=k+1. If k=4, then L=L+1, set k to 1, and return to step S132; otherwise, return to step S132.

8. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 1, characterized in that: The compression end projects each point cloud frame of the segment into the projection direction of the corresponding fusion projection coordinate system to obtain a complete projection image of each point cloud frame in each projection direction, including: A layered projection method is used to project each point of each point cloud frame of the segment onto the projection direction corresponding to the corresponding fusion projection coordinate system, and a multi-layer block projection map of each point cloud frame in each projection direction corresponding to the fusion projection coordinate system is obtained; The multi-layer block projection images of each point cloud frame in each projection direction are stitched together to obtain the complete projection image of each point cloud frame in each projection direction.

9. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 1, characterized in that: The compression end obtains the key frame image of each point cloud frame based on the similarity of the complete projection images of two adjacent point cloud frames in each projection direction, including: The complete projection images of the first point cloud frame and the last point cloud frame in each projection direction of the current segment point cloud frame are both key frame images; For other point cloud frames in the current segment of point cloud frames, the structural similarity of the complete projection image of the point cloud frame and its previous point cloud frame in each projection direction is calculated; if the structural similarity does not exceed the second threshold, the complete projection image is used as the key frame image.

10. The dynamic point cloud compression and decompression system for three-dimensional objects according to claim 9, characterized in that: The structural similarity between the complete projection image of the point cloud frame and its previous point cloud frame in each projection direction is calculated in the following way: For the current projection direction, according to the formula Calculate the SSIM value of the complete projection image of the point cloud frame and its previous point cloud frame on the kth image channel respectively; Among them, μ k,a Represents the element mean of the kth channel of the complete projection image of the point cloud frame; μ k,b Represents the element mean of the kth channel of the complete projection image of the previous point cloud frame of the point cloud frame; σ k,a Represents the element variance of the kth channel of the complete projection image of the point cloud frame; σ k,b Represents the element variance of the kth channel of the complete projection image of the previous point cloud frame of this point cloud frame, σ k,ab Represents the element covariance of the kth channel of the complete projection image of the point cloud frame and the complete projection image of the previous point cloud frame of the point cloud frame, c1 and c2 are constants; The SSIM values ​​of all image channels are added together to obtain the structural similarity value of the complete projection image of the point cloud frame and its previous point cloud frame in the current projection direction.

Images