Dynamic code rate lossy point cloud reconstruction method and system

Abstract

The invention discloses a dynamic code rate lossy point cloud reconstruction method and system, and relates to the technical field of lossy point cloud reconstruction, and the method comprises the steps: obtaining original point cloud data; carrying out hidden feature coding and quantization processing on the original point cloud by using a hierarchical multi-scale encoder; performing importance map generation and hidden feature pruning on the hidden feature coding and quantization processing result, and transmitting the pruning result; and expanding the received pruning result by using a multi-branch structure through a multi-branch expansion decoder to perform point cloud reconstruction. According to the method, firstly, an original point cloud is encoded into features containing high-level geometric semantic information and bottom-level high-resolution shape information through a hierarchical multi-scale encoder and quantization operation, and transmission features are pruned by using an importance map generation module, so that dynamic code rate distribution of each point in the point cloud is realized; and finally, more detailed point cloud reconstruction is completed at a receiving end through a multi-branch expansion structure.

Description

Technical Field

[0001] The present invention relates to the technical field of lossy point cloud reconstruction, and more specifically to a dynamic bit rate lossy point cloud reconstruction method and system. Background Art

[0002] Point cloud data, as one of the most widely used three-dimensional data, contains a huge amount of information. As the acquisition of high-precision point clouds becomes more and more convenient, it occupies a large amount of computer storage space and generates a large amount of redundant data. In some practical application scenarios that require remote data processing, such as autonomous driving, remote sensing mapping, and long-distance human-computer interaction, the real-time point cloud sampled by the sensor needs to be transmitted to the remote model framework using the communication network to complete the corresponding task. However, directly storing and transmitting lossless high-precision point clouds with huge data volumes will seriously affect the data acquisition speed of the framework and bring pressure on hardware and communication resources. Therefore, it is necessary to streamline and compress the collected original point cloud data to meet the transmission bandwidth requirements and ensure the possibility of data sharing and the smooth progress of the task. Given that the compression ratio of lossless compression is generally small and it is difficult to meet the compression requirements of massive point cloud data, lossy compression is more suitable for solving the point cloud data compression problem, but lossy compression will inevitably cause information loss during the transmission and communication process, which brings about the need for further research on lossy compression and decompression algorithms for point cloud data.

[0003] According to the international data compression standards established by the Moving Picture Experts Group (MPEG), existing point cloud lossy compression technologies can be divided into two categories. One is geometry-based point cloud compression (G-PCC), which retains key points by encoding the geometric information of static point clouds to simplify data and models. The other is video-based point cloud compression (V-PCC), which mainly processes dynamic point clouds with time information.

[0004] Initially, tree structures were introduced to establish topological relationships between points and geometric information, solving the structured storage of point clouds and the corresponding compression problems. Octrees are used to divide the point cloud space, and binary occupancy codes are generated for each node to complete the compression. The TMC13 anchor and point cloud library (PCL) released by MPEG also use octrees, kd trees and other structures to establish connection information between points, and have become classic benchmark methods in the field of G-PCC. Although compression methods based on tree structures are relatively easy to implement, the compression effect is greatly limited as the tree depth decreases.

[0005] On the other hand, some mapping-based methods have gradually emerged to solve the point cloud compression problem. Three-dimensional point clouds are converted into two-dimensional images or videos through different transformation methods, and then combined with existing mature image or video compression technologies (such as JPEG or HEVC, etc.) to complete subsequent compression. However, these methods contain many artificially designed components, have high computational complexity, and only have good effects on a certain type of point cloud, and have great application limitations.

[0006] In recent years, due to the successful performance of deep learning technology in image compression, deep convolutional neural networks have demonstrated powerful feature expression capabilities and have been applied to solve point cloud compression problems. However, due to the sparse distribution of point clouds and the need for parameter setting during voxelization, such methods lose a lot of information in the input data and the time-consuming data preprocessing work brings a huge computational burden to the hardware devices.

[0007] In addition, the PointNet framework, which can directly process the original point cloud, is used to encode key point features for compressed transmission, avoiding the division of voxel grids, and further proposing a generative model to improve the network's point cloud feature expression ability and reconstruction generation effect. This type of point representation-based method can better adapt to the nature of point cloud data and retain more effective information. However, the above work cannot achieve dynamic bit rate allocation for each point in the point cloud during the compression process, and the compression quality of the model and the details of decompression and reconstruction at the receiving end also need to be further improved.

[0008] Therefore, how to propose a dynamic bit rate lossy point cloud reconstruction method and system, encode the original point cloud into features containing high-level geometric semantic information and underlying high-resolution shape information through a hierarchical multi-scale feature extraction structure, and quantize it, and then use the importance map generation module to prune the transmission features, and transmit them after dynamic bit rate allocation to each point in the point cloud, and finally complete a more detailed point cloud reconstruction through a multi-branch expansion structure at the receiving end is a problem that technical personnel in this field need to solve urgently. Summary of the invention

[0009] In view of this, the present invention provides a method and system for reconstructing a point cloud with lossy dynamic bit rate, which encodes the original point cloud into features containing high-level geometric semantic information and low-level high-resolution shape information through a hierarchical multi-scale feature extraction structure, and quantizes the features. Then, an importance map generation module is used to prune the transmission features, and the dynamic bit rate of each point in the point cloud is allocated before transmission. Finally, a more detailed point cloud reconstruction is completed at the receiving end through a multi-branch expansion structure. In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0010] A dynamic bit rate lossy point cloud reconstruction method, comprising:

[0011] Get the original point cloud data;

[0012] Use a hierarchical multi-scale encoder to encode and quantize the original point cloud;

[0013] Generate importance graphs and perform latent feature pruning on the results of latent feature encoding and quantization, and transmit the pruning results;

[0014] The multi-branch expansion decoder uses a multi-branch structure to expand the received pruning results for point cloud reconstruction.

[0015] Optionally, the step of using a hierarchical multi-scale encoder to perform latent feature encoding and quantization processing on the original point cloud includes:

[0016] S11: Starting state, inputting the original point cloud data P = {p1, p2, p3, ...} and the hierarchical multi-scale encoder E;

[0017] S12: Sample the first level input sampling point set P1={p 11 ,p 12 ,p 13 ,...}, then divide each sampling point and the points within the sphere with a radius of r1 around it into local areas, and then extract the corresponding features of each local area through multiple 1*1 convolutional layers and maximum pooling layers to obtain the first-level features E1 of the hierarchical multi-scale encoder E;

[0018] S13: Repeat S12 until feature extraction of all levels in the hierarchical multi-scale encoder E is completed;

[0019] S14: Use interpolation operations and 1*1 convolutional layers to adjust the feature dimensions of different levels to the same size and apply sigmoid function activation;

[0020] S15: concatenate features at different levels to generate hierarchical coding latent features F = {f1, f2, f3, ...};

[0021] S16: Perform quantization rounding operation Q and output quantized latent features

[0022] Optionally, generating an importance map and performing latent feature pruning on the latent feature encoding and quantization processing results includes:

[0023] S21: Starting state, obtaining the hierarchical encoding latent feature F, quantizing the latent feature and importance branch network S;

[0024] S22: Input the hierarchical encoding latent feature F into the importance branch network S to generate an importance graph S(P) = {s1, s2, s3…};

[0025] S23: Perform quantization and rounding operation on the importance map S(P) to generate a quantized importance map To determine the bit allocation of latent features;

[0026] S24: Utilize the formula Generate importance mask M(f) = {m1, m2, m3, ...}; where To quantify the importance of The i-th position in ;

[0027] S25: Complete hidden feature pruning: Output transmission hidden features

[0028] Optionally, the value range of each element in the importance map is [0,1], which indicates the importance of the information of each point position in the latent feature map. Points with relatively larger values ​​will retain more feature dimensions during the pruning process. For points with a value of 0, no information is transmitted, but their positions are inferred through context information.

[0029] Optionally, the importance branch network S is based on MLP structure, consists of multiple fully connected layers, and has an output size of M*1, which represents the importance of information at each point in the latent feature map.

[0030] Optionally, the step of performing point cloud reconstruction by using a multi-branch expansion decoder to expand the received pruning result using a multi-branch structure includes:

[0031] S31: Starting state, input the received transmission hidden features and a multi-branch expansion decoder Dec;

[0032] S32: Use 1*1 convolutional layer to process the transmission hidden features And according to the expansion ratio r1 of the feature expansion layer in the first coordinate restoration structure, the feature map size is adjusted to obtain the expanded feature

[0033] S33: Processing using a coordinate reconstruction module consisting of multiple fully connected layers Generate a reconstructed point cloud of this level

[0034] S34: Repeat S32-S33 until all coordinates in the multi-branch expansion decoder Dec are restored;

[0035] S35: Output the final reconstructed point cloud

[0036] Optionally, the multi-branch expansion decoder Dec is composed of a plurality of stacked coordinate restoration structures, each of which uses a 1×1 convolution and a resizing operation to expand the input feature from a size of N×D to U×V, where U and V represent the expansion ratio and the number of input points, respectively.

[0037] Optionally, point cloud reconstruction modeling is also included:

[0038] Among them, P represents point cloud information, E represents encoding operation, Q represents quantization operation, M represents importance coding mask, Dec represents decompression process, and Q(·) represents quantized point cloud information;

[0039] Use rate-distortion loss to adjust network parameters;

[0040] L = θD + R, where θ is the balance weight coefficient, D i , R represent distortion loss and rate loss respectively.

[0041] Optionally, the distortion loss and rate loss include:

[0042]

[0043] Among them, the number of points contained in the original point cloud and the reconstructed point cloud are n and m respectively, S i is the corresponding component of the importance map S.

[0044] Optionally, a dynamic bit rate lossy point cloud reconstruction system comprises:

[0045] Acquisition module: used to obtain original point cloud data;

[0046] Hidden feature encoding and quantization processing module: used to perform hidden feature encoding and quantization processing on the original point cloud using a hierarchical multi-scale encoder;

[0047] Importance map generation and latent feature pruning module: used to generate importance maps and prune latent features based on latent feature encoding and quantization processing results, and transmit the pruning results;

[0048] Point cloud reconstruction module: used to reconstruct the point cloud by expanding the received pruning results with a multi-branch structure through a multi-branch expansion decoder.

[0049] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method and system for reconstructing a dynamic bit rate lossy point cloud, which has the following beneficial effects:

[0050] The present invention proposes a method for reconstructing a lossy point cloud with a dynamic bit rate, including: obtaining the original point cloud data; using a hierarchical multi-scale encoder to perform latent feature encoding and quantization processing on the original point cloud; generating an importance map and pruning the latent features for the results of the latent feature encoding and quantization processing, and transmitting the pruning results; and reconstructing the point cloud by expanding the received pruning results with a multi-branch structure through a multi-branch expansion decoder. The present invention first encodes the original point cloud into features containing high-level geometric semantic information and underlying high-resolution shape information through a hierarchical multi-scale encoder and quantization operations, and uses an importance map generation module to prune the transmission features to achieve dynamic bit rate allocation for each point in the point cloud, and finally completes a more detailed point cloud reconstruction at the receiving end through a multi-branch expansion structure; the feature encoding and reception reconstruction quality for the lossy point cloud environment in the existing technical solutions needs to be strengthened, and the encoding features of each sampling point in the point cloud are transmitted at a fixed bit rate. The present invention further improves the quality of feature encoding and improves the details of the reconstructed point cloud at the receiving end. At the same time, the bit rate can be dynamically allocated according to the situation, and the feature ratio of key detection points can be increased when the bandwidth is limited, and the information of other points can be suppressed, so as to achieve better compression transmission and data recovery. The present invention has dynamic scalability. By setting the parameters of the hierarchical multi-scale encoder and the multi-branch expansion encoder, point cloud feature compression encoding and reconstruction of different qualities can be completed, and dynamic selection can be made according to the specific requirements of subsequent tasks. Low-resolution encoding and reconstruction generate relatively blurred point clouds with low computational complexity and fast speed, which are more suitable for situations where there are no strict requirements on target details. High-resolution encoding and reconstruction are smoother and finer, but relatively time-consuming, and are suitable for fine-grained task requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0052] Figure 1 A schematic flow chart of a dynamic bit rate lossy point cloud reconstruction method provided by the present invention.

[0053] Figure 2 This is a flow chart of the encoding and quantization processing of high-quality point cloud features provided by the present invention.

[0054] Figure 3 The present invention provides a flowchart for importance map generation and latent feature pruning.

[0055] Figure 4 This is a schematic diagram of the structure of a multi-branch extended decoder Dec provided by the present invention.

[0056] Figure 5 This is a flow chart of multi-branch expansion decoding provided by the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] The embodiment of the present invention discloses a dynamic bit rate lossy point cloud reconstruction method, comprising:

[0059] Get the original point cloud data;

[0060] Use a hierarchical multi-scale encoder to encode and quantize the original point cloud;

[0061] Generate importance graphs and perform latent feature pruning on the results of latent feature encoding and quantization, and transmit the pruning results;

[0062] The multi-branch expansion decoder uses a multi-branch structure to expand the received pruning results for point cloud reconstruction.

[0063] In a specific implementation, a dynamic bit rate lossy point cloud geometric information compression and decompression method based on multi-scale encoding and multi-resolution reconstruction is provided. Figure 1 As shown in the figure, there are three main steps: Step 1. Use the hierarchical multi-scale encoder to encode the original point cloud into high-quality features that combine high-level geometric semantic information and low-level high-resolution shape information and perform quantization. Step 2. Use the importance map generation module to generate the importance map, complete feature pruning, reasonably allocate more bits to key points, and delete the feature dimensions of non-key points. Step 3. Use the multi-branch expansion decoder to expand the received features through the multi-branch structure to complete a more detailed and accurate point cloud reconstruction.

[0064] In a specific implementation, step 1: encoding and quantization of high-quality point cloud features

[0065] The encoder itself can be regarded as an analysis function that can transform the input point cloud into latent features: f = f e(P). The generation of high-quality point cloud features helps to improve the quality of lossy compression of point clouds and reduce information loss. The PointNet++ network directly uses a hierarchical structure to recursively capture the local and overall information of the original point cloud in the form of point element representation, and has good feature encoding capabilities for point cloud data. Therefore, the present invention designs a hierarchical multi-scale encoder E based on it. The hierarchical multi-scale encoder E will divide the point cloud into layers. When processing each layer, the D-FPS method is first applied to sample the points contained in this layer from the previous layer of point cloud set, and then. Each sampling point and the points within the spherical domain with a radius of r around it are divided into local areas. Subsequently, the corresponding features are extracted from each local area through multiple 1*1 convolutional layers and maximum pooling layers. The above steps are the same as the pointnet++ network structure, and the high-level features with strong geometric semantics and the low-level features with more detailed shape information in the network are aggregated together to generate higher quality feature representations. A rounding function is also introduced to realize the discrete quantization of the encoder output latent features to reduce the feature storage and transmission pressure caused by floating-point expression, such as Figure 2 As shown, the specific steps are as follows:

[0066] Step 1-1: Starting state, input the original point cloud data P = {p1, p2, p3, ...} and the hierarchical multi-scale encoder E;

[0067] Step 1-2: Sample the first level input sampling point set P1={p 11 ,p 12 ,p 13 ,...}, and then divide each sampling point and the points within the sphere with a radius of r1 around it into local areas. Then, multiple 1*1 convolutional layers and maximum pooling layers are used to extract the corresponding features of each local area to obtain the first-level features E1 of the hierarchical multi-scale encoder E;

[0068] Step 1-3: Repeat 1-2 until the feature extraction of all levels in the hierarchical multi-scale encoder E is completed;

[0069] Step 1-4: Use interpolation operations and 1*1 convolutional layers to adjust the feature dimensions of different levels to the same size and apply sigmoid function activation;

[0070] Step 1-5: Concatenate features at different levels to generate hierarchical encoding latent features F = {f1, f2, f3, ...};

[0071] Step 1-6: Perform quantization and rounding operation Q to output quantized latent features

[0072] In a specific implementation, step 2: importance map generation and latent feature pruning

[0073] The quantized latent feature dimension obtained after the hierarchical multi-scale encoder and quantizer processing is a constant length, and the uniform code length will lead to more background information in the reconstructed point cloud generated based on it, resulting in more loss of foreground key information, which will have an adverse impact on the performance of subsequent target detection tasks. In fact, the importance and encoding difficulty of each point in the original point cloud are different. In the case of limited bandwidth, it is necessary to retain more foreground target key point information, which is more conducive to high-quality targeted reconstruction of point clouds and the smooth completion of subsequent tasks.

[0074] Therefore, the point cloud compression stage needs to make further improvements based on the hierarchical multi-scale encoder to prune the non-key point information with less information in the latent features, so that the key point features have more weight. This allocation strategy can control the bit rate more reasonably, help improve compression performance and avoid wasting bandwidth resources.

[0075] In view of this, the present invention designs an importance map generation module to generate an importance map to achieve dynamic bitrate allocation of quantized latent features. The value range of each element in the importance map is [0,1], which represents the information importance of each point position in the latent feature map. Points with relatively larger values ​​will retain more feature dimensions during the pruning process, while points with a value of 0 will not transmit information, but infer their positions through context information. Figure 3 As shown in Figure 2, the steps for importance map generation and latent feature pruning are as follows:

[0076] Step 2-1: Starting state, prepare the hierarchical encoding latent features F obtained in the previous step, and quantize the latent features And the importance branch network S, the importance branch network S is based on the MLP structure, composed of multiple fully connected layers, and the output size is M*1, which indicates the importance of the information at each point in the latent feature map;

[0077] Step 2-2: Input the hierarchical encoding latent feature F into the importance branch network S to generate the importance map S(P) = {s1, s2, s3…};

[0078] Step 2-3: Quantize and round the importance map S(P) to generate a quantized importance map To determine the bit allocation of latent features;

[0079] Step 2-4: Generate the importance mask map M(f) = {m1, m2, m3, ...} using the following formula;

[0080]

[0081] in, To quantify the importance of The i-th position in ;

[0082] Step 2-5: Complete hidden feature pruning: Output transmission hidden features

[0083] In a specific implementation, step 3: multi-branch expansion decoding

[0084] The decoder itself can be viewed as a synthesis function that reconstructs the point cloud using the received latent features: The fully connected layer is usually used to complete the above decoding task due to its simple structure. However, there is a lot of information compression in the encoding and quantization process of point cloud data. Directly reconstructing the point cloud using only a few fully connected layers will result in insufficient details. Therefore, the present invention designs a multi-branch expansion decoder, which corresponds to the hierarchical compression structure in the encoder. By stacking multiple coordinate restoration structures, it can generate reconstructed point clouds of different resolutions from low to high, gradually improve local detail information, and finally achieve more refined decoding. Each coordinate restoration structure includes a feature expansion layer and a coordinate reconstruction module. Figure 4 As shown, the multi-branch extended decoding in the present invention is as follows Figure 5 As shown, the specific steps are as follows:

[0085] Step 3-1: Starting state, input the received transmission hidden features and a multi-branch expansion decoder Dec; the multi-branch expansion decoder Dec is composed of multiple coordinate restoration structures stacked together, each of which uses 1×1 convolution and resize operations to expand the input features from size N×D to U×V, where U and V represent the expansion ratio and the number of input points, respectively;

[0086] Step 3-2: Use 1*1 convolutional layer to process the transmission hidden features And according to the expansion ratio r1 of the feature expansion layer in the first coordinate restoration structure, the feature map size is adjusted to obtain the expanded feature

[0087] Step 3-3: Use a coordinate reconstruction module consisting of multiple fully connected layers Generate a reconstructed point cloud of this level

[0088] Step 3-4: Repeat 3-2 and 3-3 until all coordinates in the multi-branch expansion decoder Dec are restored;

[0089] Step 3-5: Output the final reconstructed point cloud

[0090] In a specific implementation, it also includes constructing a loss function

[0091] The point cloud compression and decompression method can be modeled as follows:

[0092]

[0093] Where P represents PointCloud information, E represents encoding operation, Q represents quantization operation, M represents importance coding mask, Dec represents decoding or decompression process, and Q(·) represents quantized point cloud information.

[0094] Before performing compression and decompression tasks, it is also necessary to use a loss function to train it to optimize the network weight parameters. For lossy compression methods, the original input reduces the transmission bits through operations such as encoding and quantization during the compression process, but it brings information loss, resulting in a certain degree of distortion in the reconstruction of the receiving end compared to the original point cloud. The degree of distortion can explain the compression and decompression quality of the original point cloud by the method. At the same time, the bit rate represents the bandwidth occupied by the hidden features during the transmission process, which can explain the degree of compression and simplification of the original point cloud by the method. Distortion and bit rate are two adversarial evaluation criteria for measuring the performance of lossy compression and decompression methods. A reduction in the degree of distortion means that the hidden features contain more information about the original data, but the bit rate also increases, and vice versa. The two cannot achieve optimality at the same time, and a trade-off is required. Therefore, during the training process, the present invention uses the following rate-distortion loss to adjust the network parameters to obtain better overall performance.

[0095] L = θD + R;

[0096] Among them, θ is the balance weight coefficient, D i , R represent distortion loss and rate loss respectively, as follows:

[0097]

[0098] Among them, the number of points contained in the original point cloud and the reconstructed point cloud are n and m respectively, S i is the corresponding component of the importance map S.

[0099] In a specific implementation, a dynamic bit rate lossy point cloud reconstruction system includes:

[0100] Acquisition module: used to obtain original point cloud data;

[0101] Hidden feature encoding and quantization processing module: used to perform hidden feature encoding and quantization processing on the original point cloud using a hierarchical multi-scale encoder;

[0102] Importance map generation and latent feature pruning module: used to generate importance maps and prune latent features based on latent feature encoding and quantization processing results, and transmit the pruning results;

[0103] Point cloud reconstruction module: used to reconstruct the point cloud by expanding the received pruning results with a multi-branch structure through a multi-branch expansion decoder.

[0104] Taking into account the data remote communication needs and the data storage and transmission pressure of massive point clouds in actual scenarios, the present invention proposes a dynamic code rate lossy point cloud compression and decompression method based on multi-scale coding and multi-resolution reconstruction to build a more friendly lossy point cloud environment. This method integrates the high-level semantic information and underlying shape information in the point cloud features at the data sending end to form multi-scale latent features, and designs an importance map generation module to dynamically adjust the code rate to retain more key information in the transmission codeword, and realizes effective transmission after further compression through quantization. Subsequently, at the receiving end, a multi-branch expansion structure is used to gradually generate a restored point cloud containing more detailed information from low resolution to high resolution, ensuring the accuracy of data reconstruction.

[0105] For example, in a vehicle-road collaborative autonomous driving network, each vehicle compresses the point cloud data collected by the lidar and transmits it to the roadside equipment. The roadside equipment aggregates the information of all vehicles to form dynamic three-dimensional road data at the road network level, and can use algorithms to schedule the operation of each vehicle. Point cloud compression is required when the point cloud collected by the vehicle is transmitted to the roadside equipment through the wireless vehicle network. The present invention constructs a dynamically adjustable hierarchical multi-scale encoder and a multi-branch expansion decoder, which can complete the compression encoding and reconstruction of point cloud features of different qualities, and can be dynamically selected according to the specific requirements of subsequent tasks; it can dynamically allocate the transmission bit rate according to the importance of each point in the scene and the local information content around it, so as to more effectively avoid information loss in the case of limited bandwidth, and ensure the point cloud compression performance and data quality at the receiving end.

[0106] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0107] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic bit rate lossy point cloud reconstruction method, characterized in that: include: Get the original point cloud data; Use a hierarchical multi-scale encoder to encode and quantize the original point cloud; Generate importance graphs and perform latent feature pruning on the results of latent feature encoding and quantization, and transmit the pruning results; The multi-branch expansion decoder uses a multi-branch structure to expand the received pruning results for point cloud reconstruction.

2. A dynamic bit rate lossy point cloud reconstruction method according to claim 1, characterized in that: The use of a hierarchical multi-scale encoder to perform latent feature encoding and quantization processing on the original point cloud includes: S11: Starting state, inputting the original point cloud data P = {p1, p2, p3, ...} and the hierarchical multi-scale encoder E; S12: Sample the first level input sampling point set P1={p 11 ,p 12 ,p 13 ,...}, then divide each sampling point and the points within the sphere with a radius of r1 around it into local areas, and then extract the corresponding features of each local area through multiple 1*1 convolutional layers and maximum pooling layers to obtain the first-level features E1 of the hierarchical multi-scale encoder E; S13: Repeat S12 until feature extraction of all levels in the hierarchical multi-scale encoder E is completed; S14: Use interpolation operations and 1*1 convolutional layers to adjust the feature dimensions of different levels to the same size and apply sigmoid function activation; S15: concatenate features at different levels to generate hierarchical coding latent features F = {f1, f2, f3, ...}; S16: Perform quantization rounding operation Q and output quantized latent features 3. The method for reconstructing a dynamic bit rate lossy point cloud according to claim 1, characterized in that: The generating of importance graph and pruning of latent features based on latent feature encoding and quantization processing results includes: S21: Starting state, obtaining the hierarchical encoding latent feature F, quantizing the latent feature and importance branch network S; S22: Input the hierarchical encoding latent feature F into the importance branch network S to generate an importance graph S(P) = {s1, s2, s3…}; S23: Perform quantization and rounding operation on the importance map S(P) to generate a quantized importance map To determine the bit allocation of latent features; S24: Utilize the formula Generate importance mask M(f) = {m1, m2, m3, ...}; where To quantify the importance of The i-th position in ; S25: Complete hidden feature pruning: Output transmission hidden features 4. The method for reconstructing a dynamic bit rate lossy point cloud according to claim 3, characterized in that: The value range of each element in the importance map is [0,1], which is used to indicate the importance of information at each point in the latent feature map. Points with relatively larger values ​​will retain more feature dimensions during the pruning process. For points with a value of 0, no information is transmitted, but their positions are inferred through context information.

5. The method for reconstructing a dynamic bit rate lossy point cloud according to claim 3, characterized in that: The importance branch network S is based on MLP structure, consists of multiple fully connected layers, and has an output size of M*1, which represents the importance of information at each point in the latent feature map.

6. The method for reconstructing a dynamic bit rate lossy point cloud according to claim 1, characterized in that: The step of performing point cloud reconstruction by using a multi-branch expansion decoder to expand the received pruning result using a multi-branch structure includes: S31: Starting state, input the received transmission hidden features and a multi-branch expansion decoder Dec; S32: Use 1*1 convolutional layer to process the transmission hidden features And according to the expansion ratio r1 of the feature expansion layer in the first coordinate restoration structure, the feature map size is adjusted to obtain the expanded feature S33: Processing using a coordinate reconstruction module consisting of multiple fully connected layers Generate a reconstructed point cloud of this level S34: Repeat S32-S33 until all coordinates in the multi-branch expansion decoder Dec are restored; S35: Output the final reconstructed point cloud 7. The method for reconstructing a dynamic bit rate lossy point cloud according to claim 6, characterized in that: The multi-branch expansion decoder Dec is composed of a plurality of stacked coordinate restoration structures, each of which uses 1×1 convolution and resizing operations to expand the input features from a size of N×D to U×V, where U and V represent the expansion ratio and the number of input points, respectively.

8. The method for reconstructing a dynamic bit rate lossy point cloud according to claim 1, characterized in that: It also includes point cloud reconstruction modeling: Among them, P represents point cloud information, E represents encoding operation, Q represents quantization operation, M represents importance coding mask, Dec represents decompression process, and Q(·) represents quantized point cloud information; Use rate-distortion loss to adjust network parameters; L = θD + R, where θ is the balance weight coefficient, D i , R represent distortion loss and rate loss respectively.

9. The method for reconstructing a dynamic bit rate lossy point cloud according to claim 8, characterized in that: The distortion loss and rate loss include: Among them, the number of points contained in the original point cloud and the reconstructed point cloud are n and m respectively, S i is the corresponding component of the importance map S.

10. A dynamic bit rate lossy point cloud reconstruction system, characterized in that: include: Acquisition module: used to obtain original point cloud data; Hidden feature encoding and quantization processing module: used to perform hidden feature encoding and quantization processing on the original point cloud using a hierarchical multi-scale encoder; Importance map generation and latent feature pruning module: used to generate importance maps and prune latent features based on latent feature encoding and quantization processing results, and transmit the pruning results; Point cloud reconstruction module: used to reconstruct the point cloud by expanding the received pruning results with a multi-branch structure through a multi-branch expansion decoder.

Images