Point cloud data processing method and device, electronic equipment and medium

Abstract

The invention provides a point cloud data processing method and device, electronic equipment and a medium. The problem that existing point cloud data is challenging in the aspect of attribute compression is effectively solved. The method comprises the following steps: performing multi-level division on point cloud data to obtain a first multi-resolution grid; quantizing the latent variable based on the geometric information of the latent variable, constructing different types of context information based on the quantized latent variable, and constructing an entropy model based on the different types of context information; processing the different types of context information based on the entropy model to obtain a prediction result, and coding the quantized latent variable based on the prediction result to obtain bit stream information; and decoding the bit stream information to obtain a second multi-resolution grid, obtaining corresponding point data based on a latent variable of the second multi-resolution grid, splicing point cloud feature information carried by the point data, and reconstructing point cloud data according to the spliced point cloud feature information.

Description

Technical Field

[0001] The present application relates to the technical fields of artificial intelligence and point cloud compression. Specifically, it relates to a method, device, electronic device and medium for processing point cloud data. Background Art

[0002] With the development of information technology, traditional two-dimensional information sources can no longer meet the growing immersive needs of humans, and three-dimensional representation forms are becoming increasingly popular. As a popular three-dimensional data format, point clouds have been widely used in many fields, such as augmented / virtual reality (AR / VR), autonomous driving, and digital cultural heritage protection.

[0003] Despite the increasing applications of point clouds, their characteristic of being composed of a large number of sparsely distributed three-dimensional points poses significant challenges for their efficient storage and transmission. Therefore, compression is essential to realize the practical applications of point clouds. Generally, a point cloud mainly consists of two parts: geometric coordinates (x, y, z) and corresponding attribute information, such as color, normal, and reflectivity. And attribute compression is more challenging, not only because attribute data usually requires more storage space, but also because it depends on the irregular and sparse structural characteristics inherent in the underlying geometric structure. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide a method, device, electronic device and medium for processing point cloud data, which effectively solves the challenging problems existing in the attribute compression of existing point cloud data.

[0005] In a first aspect, a method for processing point cloud data provided by an embodiment of the present application includes:

[0006] Performing attribute preprocessing on the obtained point cloud data to obtain preprocessed point cloud data, and performing multi-level division on the preprocessed point cloud data to obtain a first multi-resolution grid; the first multi-resolution grid has multiple latent variables;

[0007] Quantifying the latent variables based on the geometric information of the latent variables, constructing different types of context information based on the quantified latent variables, and constructing an entropy model based on the different types of context information;

[0008] Processing the different types of context information based on the entropy model to obtain a prediction result, and encoding the quantified latent variables based on the prediction result to obtain bitstream information;

[0009] Decode the bitstream information to obtain a second multi-resolution grid, obtain corresponding point data based on the latent variables of the second multi-resolution grid, splice the point cloud feature information carried by the point data, and reconstruct the point cloud data according to the spliced point cloud feature information.

[0010] Combined with the first aspect, the embodiments of the present application provide a first possible implementation manner of the first aspect, wherein, the constructing different types of context information based on the quantized latent variables includes:

[0011] Determine the type of context information based on the characteristics of the latent variables;

[0012] Determine the corresponding context construction method based on the type to construct different types of context information.

[0013] Combined with the first aspect, the embodiments of the present application provide a second possible implementation manner of the first aspect, wherein, the type at least includes the intra-layer type;

[0014] The determining the corresponding context construction method based on the type to construct different types of context information includes:

[0015] Select the potential context information of the target latent variable in a preset manner;

[0016] Filter out the target potential context information from the potential context information and sort the target potential context information to obtain the intra-layer context information.

[0017] Combined with the first aspect, the embodiments of the present application provide a third possible implementation manner of the first aspect, wherein, the type at least includes the inter-layer type;

[0018] The determining the corresponding context construction method based on the type to construct different types of context information includes:

[0019] Decode the latent variables of all levels before the target level to obtain the target context information of the target level;

[0020] Generate the target inter-layer context information based on the geometric information of the latent variables of the target level and the hierarchical context information of the target level.

[0021] Combined with the first aspect, the embodiments of the present application provide a fourth possible implementation manner of the first aspect, wherein, the quantizing the latent variables based on the geometric information of the latent variables includes:

[0022] Determine the corresponding quantization strategy based on the geometric information of the latent variables;

[0023] Perform targeted quantization on the latent variables through the corresponding quantization strategy.

[0024] In combination with the first aspect, an embodiment of the present application provides a fifth possible implementation manner of the first aspect, wherein the quantization strategy at least includes an intra-layer adaptive quantization strategy;

[0025] Performing targeted quantization on the latent variable through the corresponding quantization strategy includes:

[0026] Determining the importance level of the latent variable based on the trilinear interpolation coefficients of the latent variable;

[0027] Matching the importance level with the standard level category to obtain a corresponding matching result, and determining the quantization step size based on the matching result.

[0028] In combination with the first aspect, an embodiment of the present application provides a sixth possible implementation manner of the first aspect, wherein splicing the point cloud feature information carried by the points includes:

[0029] Retrieving a plurality of decoded vertices associated with the point in the target resolution, and querying the point cloud feature vectors of the plurality of decoded vertices;

[0030] Splicing the point cloud feature vectors of the plurality of decoded vertices to obtain complete point cloud feature information.

[0031] In a second aspect, an embodiment of the present application provides a point cloud data processing device, and the device includes:

[0032] A partitioning module, configured to perform attribute preprocessing on the acquired point cloud data to obtain preprocessed point cloud data, and perform multi-level partitioning on the preprocessed point cloud data to obtain a first multi-resolution grid; the first multi-resolution grid has a plurality of latent variables;

[0033] A quantization module, configured to quantify the latent variable based on the geometric information of the latent variable, construct different types of context information based on the quantified latent variable, and construct an entropy model based on the different types of context information;

[0034] An encoding module, configured to process the different types of context information based on the entropy model to obtain a prediction result, and encode the quantified latent variable based on the prediction result to obtain bitstream information;

[0035] A reconstruction module, configured to decode the bitstream information to obtain a second multi-resolution grid, obtain corresponding point data based on the latent variable of the second multi-resolution grid, splice the point cloud feature information carried by the point data, and reconstruct the point cloud data according to the spliced point cloud feature information.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of any one of the point cloud data processing methods are executed.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any one of the point cloud data processing methods are executed.

[0038] For a point cloud data processing method according to an embodiment of the present application, the method first performs attribute preprocessing on the acquired point cloud data to obtain preprocessed point cloud data, and performs multi-level division on the preprocessed point cloud data to obtain a first multi-resolution grid; the first multi-resolution grid has multiple latent variables; secondly, quantize the latent variables based on the geometric information of the latent variables, construct different types of context information based on the quantized latent variables, and construct an entropy model based on the different types of context information; then process the different types of context information based on the entropy model to obtain a prediction result, and encode the quantized latent variables based on the prediction result to obtain bitstream information; finally, decode the bitstream information to obtain a second multi-resolution grid, obtain corresponding point data based on the latent variables of the second multi-resolution grid, splice the point cloud feature information carried by the point data, and reconstruct the point cloud data according to the spliced point cloud feature information, thereby realizing processing based on the sparse structure and irregular aspects of the point cloud data. In this paper, a novel quantization is designed to cope with the imbalance of the importance of grid latent variables. In addition, a novel entropy model is also designed in this paper to achieve efficient compression, reduce the impact of the sparse structure and irregular characteristics of the point cloud data on point cloud compression, and also reduce the space occupied by attribute compression, and solve the challenging problems existing in the attribute compression of existing point cloud data. Description of the Drawings

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained according to these drawings without creative efforts.

[0040] Figure 1 The flowchart of the first point cloud data processing method provided by the embodiment of the present application is shown;

[0041] Figure 2Shows a schematic diagram of encoding input point cloud data provided by an embodiment of the present application;

[0042] Figure 3 (a)-(b) show a schematic diagram of in-layer quantization provided by an embodiment of the present application;

[0043] Figure 4 Shows a decoding block diagram provided by an embodiment of the present application;

[0044] Figure 5 Shows a schematic diagram of the structural details of a three-dimensional sparse convolutional network provided by an embodiment of the present application;

[0045] Figure 6 Shows a schematic diagram of the structure of a first point cloud data processing device provided by an embodiment of the present application;

[0046] Figure 7 Shows a schematic diagram of the structure of a first electronic device provided by an embodiment of the present application. Detailed implementation manners

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present application show operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical context relationships may be reversed in order or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0048] In addition, the described embodiments are only some embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application to be protected, but only represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the protection scope of the present application.

[0049] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated hereinafter, but does not exclude adding other features.

[0050] The characteristic of current point cloud data being composed of a large number of sparsely distributed three-dimensional points poses a significant challenge to its efficient storage and transmission. Therefore, compression is essential to realize the practical application of point clouds. Generally, point clouds mainly consist of two parts: geometric coordinates (x, y, z) and corresponding attribute information, such as color, normal, and reflectivity. Attribute compression is more challenging not only because attribute data usually requires more storage space but also because it depends on the irregular and sparse structural characteristics inherent in the underlying geometric structure.

[0051] Based on this, the embodiments of the present application provide a method, device, electronic device, and medium for processing point cloud data, which will be described below through embodiments.

[0052] Embodiment 1

[0053] To facilitate the understanding of this embodiment, first, a method for processing point cloud data disclosed in the embodiments of the present application will be introduced in detail. As Figure 1 shown in the flowchart of a method for processing point cloud data, a method for processing point cloud data provided by the present application includes:

[0054] S101. Perform attribute preprocessing on the acquired point cloud data to obtain the preprocessed point cloud data, and perform multi-level partitioning on the preprocessed point cloud data to obtain a first multi-resolution grid; the first multi-resolution grid has multiple latent variables;

[0055] S102. Quantify the latent variables based on the geometric information of the latent variables, construct different types of context information based on the quantified latent variables, and construct an entropy model based on the different types of context information;

[0056] S103. Process the different types of context information based on the entropy model to obtain a prediction result, and encode the quantified latent variables based on the prediction result to obtain bitstream information;

[0057] S104. Decode the bitstream information to obtain a second multi-resolution grid, obtain corresponding point data based on the latent variables of the second multi-resolution grid, splice the point cloud feature information carried by the point data, and reconstruct the point cloud data according to the spliced point cloud feature information.

[0058] In step S101, attribute preprocessing is performed on the acquired point cloud data, where the point cloud data is represented by P. The point cloud data includes geometric information and various attributes, and the attributes include color, reflectivity, etc. Here, taking the color attribute as an example, the RGB space is transformed into the YUV color space to facilitate subsequent attribute encoding, obtaining the preprocessed point cloud data, and performing multi-level partitioning on the preprocessed point cloud data, that is, using an octree to partition the three-dimensional space according to the geometric characteristics of the point cloud data P, as Figure 2 shown in the box within the context of Figure 2 . An octree is a widely adopted data structure for efficiently representing three-dimensional space data. In each layer, it recursively divides the space into eight smaller cubic sub-volumes, and each sub-volume corresponds to a node. The occupancy of each sub-volume is encoded by a bit value indicating whether it contains points, so that large empty regions can be represented compactly. This recursive partitioning will continue until the required resolution is reached. Therefore, the representation accuracy can be controlled by adjusting the depth of the tree, and multiple levels of detail LOD are generated for the point cloud data P.

[0059] Through the octree structure, a hierarchical geometric representation of the point cloud can be obtained. Each sub-volume in the octree has eight vertices, and these vertices are regarded as latent variables of the implicit neural representation. By performing trilinear interpolation on these vertices, an implicit representation can be generated for all points within the sub-volume. This method realizes the multi-resolution representation of point cloud attributes and provides a structured and flexible way to describe point cloud attributes. This hierarchical structure can capture details at each level, thereby generating a learnable multi-resolution feature grid, expressed as:

[0060]

[0061] where b is the lowest resolution level and L is the highest resolution level. Using the first multi-resolution grid structure for interpolation, the implicit neural representation of the points in the point cloud data P at different resolutions can be obtained. Through an end-to-end deep learning method, the latent variables are made to fit the original point cloud attribute information, converting the storage of the original point cloud attribute features into the storage of latent.

[0062] In step S102, directly storing the latent using 32-bit floating-point numbers consumes too many bits, so it must be quantized. However, the sparsity and irregularity of the point cloud data P result in different importance for each latent variable. Therefore, this paper designs two adaptive quantization strategies, namely, intra-layer adaptive quantization strategy and inter-layer adaptive quantization strategy, to adaptively adjust the quantization step size of each latent according to the geometric structure, thereby adjusting the distortion of each latent. Different types of context information are constructed based on the quantized latent variables, and an entropy model is constructed based on the different types of context information. That is, the different types of context information are the input of the entropy model. The entropy model is based on an MLP and is combined with an arithmetic encoder. The parameters of the entropy model are μ and σ. The schematic diagram of the construction of the entropy model is as Figure 2 shown. The first multi-resolution grid constitutes the main part of the bitstream. After quantizing the grid to an integer, an efficient entropy model is crucial for further reducing redundancy. Although it is feasible to use a single entropy model to process all grid levels, this paper finds that designing different entropy models for each level can better capture the unique statistical characteristics of each layer, thereby achieving better results. Due to the inherent sparsity and irregularity of the point cloud, these grids also exhibit similar characteristics. This application designs an entropy model that processes layers hierarchically, combines intra-layer and inter-layer context information, and optimizes the effect of redundancy reduction.

[0063] In a specific implementation process of step S102, there is an embodiment: quantizing the latent variable based on the geometric information of the latent variable includes:

[0064] S10211. Determine the corresponding quantization strategy based on the geometric information of the latent variable;

[0065] S10212. Quantize the latent variable specifically through the corresponding quantization strategy.

[0066] In steps S10211 - S10212, based on the structure of the first multi - resolution grid and the geometric information of the latent variable, it can be seen that there are two aspects related to the latent variable latent, namely between - layer and within - layer. Since floating - point latent variables are not conducive to calculation, corresponding quantization strategies are set for different aspects. That is, the floating - point latent variables are quantized into integer - point latent variables. For the between - layer aspect, an inter - layer adaptive quantization strategy is set. For the within - layer aspect, an intra - layer adaptive quantization strategy is set. To meet different requirements for compression ratio and reconstruction quality in practical applications, the quantization step size can be adjusted at each level. When a higher compression ratio is needed, the details at higher levels become less important, so a larger quantization step size can be used for them; while the lower levels are more important and a smaller quantization step size is needed. When higher reconstruction quality is required, it is also important to retain the details at higher levels, which requires a smaller quantization step size for higher levels. That is, different quantization strategies correspond to different quantization step sizes. Through the corresponding quantization strategies, targeted quantization is performed on the corresponding aspects of the latent variable to produce a more obvious quantization effect.

[0067] In the specific implementation process of step S10212, there is an embodiment as follows:

[0068] The quantization strategy at least includes an intra - layer adaptive quantization strategy;

[0069] The targeted quantization of the latent variable through the corresponding quantization strategy includes:

[0070] S102121. Determine the importance level of the latent variable based on the trilinear interpolation coefficients of the latent variable;

[0071] S102122. Match the importance level with the standard level categories to obtain the corresponding matching result, and determine the quantization step size based on the matching result.

[0072] In steps S102121 - S102122, when the quantization strategy is the intra - layer adaptive quantization strategy, that is, the aspect to be quantized at this time is the within - layer aspect of the latent variable. Within the same level l, due to the sparsity and irregularity of the point cloud data P, the latent variable latent does not necessarily have the same importance. Different grids may contain different numbers of points. As shown in Figure 3 (a), the two latents have different importance levels because the number of points they affect is different. Latent β affects far more points than latent α, so the former is more important. However, even if each latent affects the same number of points, their importance levels are also different because of the relative positions of the latents with respect to each point. As shown in Figure 3(As shown in (b), the latent variables closer to the three-dimensional points have a greater effect and higher importance. To quantify the importance of each latent variable, this paper uses the interpolation weights of the eigenvectors from the vertices of the first multi-resolution grid to the point data during the trilinear interpolation process. For each latent variable, it can appear as a shared vertex of multiple sub-volumes in the octree structure. Therefore, it affects all points within each sub-volume containing this vertex. This paper classifies the latent variables according to the trilinear interpolation coefficients of each latent variable into three importance categories, with each category corresponding to a quantization step size, that is, there is a positive correlation between the trilinear interpolation coefficient and the degree of importance, as Figure 3 shown by the thickness of the arrows in (b), which represents the numerical value of the degree of importance. That is, based on the magnitude of the trilinear interpolation coefficient, the corresponding numerical value of the degree of importance can be determined, and the matching result of the degree of importance and the standard degree category can be obtained. That is, the standard degree category corresponds to the pre-divided numerical value of the degree of importance. Based on the matching result, the standard degree category corresponding to the latent variable can be known, and then the corresponding quantization step size can be further determined. The higher the importance, the smaller the quantization step size, so as to make the distortion as small as possible, and then improve the quantization effect based on the corresponding quantization step size.

[0073] When the quantization strategy is the inter-layer adaptive quantization strategy, that is, when the inter-layer aspect of the latent variable is to be quantized at this time, a set of learnable quantization step sizes is introduced to allocate appropriate quantization step sizes for each level l, so that the model can be adjusted according to the importance of different levels. The quantized latent variable at level l is calculated by formula (2):

[0074]

[0075] where x l represents the original latent variable at level l. A larger Δ l value introduces a larger quantization error and is applicable to less important levels; while a smaller Δ l value can retain more information for key levels. This adaptive quantization strategy dynamically adjusts the precision of different levels and achieves the best balance between the compression rate and the reconstruction quality.

[0076] In the specific implementation process of step S102, there is another embodiment: constructing different types of context information based on the quantized latent variables, including:

[0077] S10221. Determine the type of context information based on the characteristics of the latent variable;

[0078] S10222. Determine the context construction method corresponding to the type to construct different types of context information.

[0079] In steps S10221 - S10222, the inherent sparsity and irregularity of the point cloud data P result in different information contained in the latent variables at different positions. That is, according to the type of context information determined by the latent variables, according to Figure 2 the boxes in the level of detail of the point cloud in the multi - resolution grid construction in, there are two types of context types, namely the intra - layer type and the inter - layer type, and the context construction methods corresponding to the types are determined. The intra - layer type corresponds to the intra - layer construction method, and the inter - layer type corresponds to the inter - layer construction method to construct different types of context information and have corresponding construction effects. This application uses two types of context information, intra - layer and inter - layer, to perform more accurate prediction on the latent to be encoded, thereby reducing the number of context information to be encoded and further ensuring the compression effect.

[0080] In the specific implementation process of step S10222, there is an embodiment: the type at least includes the intra - layer type;

[0081] The determining the context construction method corresponding to the type to construct different types of context information includes:

[0082] S1022211. Select the potential context information of the target latent variable according to a preset method;

[0083] S1022212. Screen out the target potential context information from the potential context information and sort the target potential context information to obtain the intra - layer context information.

[0084] In steps S1022211 - S1022212, let the latent variable for which the context is to be constructed be the target latent variable, then set as the l - th layer target latent variable to be encoded, and select its surrounding K×K×K neighborhood as the potential context information. To control the number of parameters and adapt to the sparsity and irregularity of the point cloud data P, this application selects the N nearest decoded neighbors, that is, selects the target potential context information, and arranges the target potential context information in ascending order of distance. This distance - based sorting can organize the context information according to reliability, partially compensating for the limitations of the multi - layer perceptron (MLP) in using the spatial structure. If the available neighbors are less than N, the context vector is completed by zero padding. In addition, the K×K×K neighborhood defines the context dependence range of each latent variable, thus realizing the parallel decoding of completely independent latent variables to accelerate the process, and adjusting this parameter can adjust the decoding speed and performance according to requirements.

[0085] In the specific implementation process of step S10222, there is another embodiment: the type at least includes the inter - layer type;

[0086] The determination is based on the context construction method corresponding to the type to construct context information of different types, including:

[0087] S1022221. Decode the latent variables of all levels before the target level to obtain the target context information of the target level;

[0088] S1022222. Generate target inter-level context information based on the geometric information of the latent variables of the target level and the hierarchical context information of the target level.

[0089] In steps S1022221 - S1022222, the context information construction method of the inter-level type is different from that of the intra-level type. The context information of the inter-level type is related to decoding. In order to make full use of the previously decoded information while decoding efficiently, this article adopts a decoding strategy from coarse to fine. The latent variables in different levels of LOD represent information of different resolutions. Therefore, the latent variables at the same spatial position should convey consistent information among different LODs. This article starts decoding from the lower level of LOD and allows it to provide hierarchical context for higher levels of LOD. When decoding the latent variables of the l-th level, all the latent variables from the previous levels b, b + 1, …, l - 1 have been decoded and can be used as context references. For each latent variable in the l-th level, this article uses the geometric information of its corresponding vertices to perform trilinear interpolation on the latent variables decoded in the previous levels to generate inter-level context information, that is, the inter-level context information of the target level where the latent variable is located uses the information of its previous level, that is, lower-resolution information, to strengthen the prediction.

[0090] In step S103, the encoder and decoder used by the entropy model described in this application are range coders. Based on the entropy model, the different types of context information are processed to obtain a prediction result. Through the processing of the entropy model, the redundancy and correlation between this context information and the latent variables to be encoded can be analyzed, where the latent variables to be encoded are the quantized latent variables, thereby providing a basis for subsequent data compression or encoding. The prediction result is the Laplace distribution parameter value. According to the Laplace distribution, the prediction probability is obtained. Based on the prediction result, the quantized latent variables are encoded, that is, the accuracy of the prediction result will directly affect the efficiency and quality of subsequent encoding. That is, according to the decoded same-level neighborhood and the context information of the previously decoded lower-resolution levels, the quantized latent variables in the first multi-resolution grid are encoded to obtain bitstream information. The encoding method can specifically be entropy encoding. The bitstream information is a compact data format that is easy to store and transmit. It contains all the necessary information of the original quantized latent variables but removes the redundancy and unnecessary parts, thereby achieving efficient compression.

[0091] In step S104, decoding is the inverse process of the encoding process. Decoding is performed to obtain the decoded multi-resolution grid, and the bitstream information is decoded to obtain the second multi-resolution grid. The second multi-resolution grid corresponds to, or even has the same relationship as, the first resolution grid. In this application, a novel three-dimensional sparse convolutional network is sampled to replace the traditional MLP as the final reconstruction module. As shown in Figure 4 shown, where the structural details of the three-dimensional sparse convolutional network are as shown in Figure 5 shown. Compared with the MLP, the three-dimensional sparse convolutional network makes full use of the spatial neighborhood information in the point cloud, has a larger receptive field, and can integrate neighborhood information. After decoding to obtain the second multi-resolution grid, corresponding point data is obtained based on the latent variables of the second multi-resolution grid. The point data also carries some additional feature information obtained through learning. The specific method of linear interpolation is trilinear interpolation. The point cloud feature information carried by the point data is spliced. The point cloud feature information is represented by a feature vector, and the point cloud data is reconstructed according to the spliced point cloud feature information. The spliced feature information is combined into a complete point cloud data set that can be used for visualization or further processing. Through reconstruction, a visually coherent and geometrically accurate reconstructed point cloud data can be obtained. Among them, the three-dimensional coordinates p i of the point cloud data have their reconstructed colors obtained by the reconstruction module g φ and are represented by formula (5):

[0092]

[0093] Based on the reconstruction module described in formula (5), the reconstruction quality is significantly improved.

[0094] In the specific implementation process of step S104, there is an embodiment: splicing the point cloud feature information carried by the points, including:

[0095] S1041. Retrieve the multiple decoded vertices associated with the point in the target resolution and query the point cloud feature vectors of the multiple decoded vertices;

[0096] S1042. Splice the point cloud feature vectors of the multiple decoded vertices to obtain the complete point cloud feature information.

[0097] In steps S1041 - S1042, for each point data i in the point cloud, whose three-dimensional coordinates are p i , based on the second multi-resolution grid, determine the voxel containing p i in the octree at resolution level l, and retrieve its associated eight decoded vertices Then, through trilinear interpolation of these vertices, query the feature vector at this coordinate, which is represented by formula (3):

[0098]

[0099] Subsequently, the feature vectors of all resolution levels are concatenated together to form the complete feature information of point p i , which is represented by formula (4):

[0100] f(p i ) = concat[f b (p i ), f b+1 (p i ),..., f L (p i )] (4)

[0101] This application is different from other traditional deep learning methods based on the autoencoder architecture. It adopts a method based on implicit neural representation and uses a compact small model to individually fit a single point cloud frame. Therefore, there is no need to consider generalization, making it more robust. Table 1 shows the quantitative comparison of BD-rate, indicating that the BD-rate is significantly reduced on various datasets. Specifically, on the MVUB and Owlii datasets, the method in this paper saves 23.45% (22.57%) and 15.86% (17.77%) of the bitrate in the Y (YUV) space respectively, significantly outperforming G-PCC TMC13v26 and highlighting the efficiency in point cloud attribute compression. In addition, this paper also compares the proposed method with the autoencoder-based method SparsePCAC, and it can be seen that the method in this paper has a very large performance improvement compared to SparsePCAC and achieves a very large proportion of bitrate savings in all point cloud samples.

[0102] Table 1 Bitrate savings comparison table of the present invention relative to other methods

[0103]

[0104] Example 2

[0105] This application also provides a point cloud data processing device. As shown in Figure 6 is a block diagram of a point cloud data processing device. The functions implemented by this point cloud data processing device correspond to the steps of executing a point cloud data processing method on a terminal device. This device can be understood as a component of a server including a processor. The device includes:

[0106] A partitioning module 601, configured to perform attribute preprocessing on the acquired point cloud data to obtain preprocessed point cloud data, and perform multi-level partitioning on the preprocessed point cloud data to obtain a first multi-resolution grid; the first multi-resolution grid has multiple latent variables;

[0107] A quantization module 602, configured to quantize the latent variable based on the geometric information of the latent variable, construct different types of context information based on the quantized latent variable, and construct an entropy model based on the different types of context information;

[0108] An encoding module 603, configured to process the different types of context information based on the entropy model to obtain a prediction result, and encode the quantized latent variable based on the prediction result to obtain bitstream information;

[0109] A reconstruction module 604, configured to decode the bitstream information to obtain a second multi-resolution grid, obtain corresponding point data based on the latent variable of the second multi-resolution grid, splice the point cloud feature information carried by the point data, and reconstruct point cloud data according to the spliced point cloud feature information.

[0110] In a feasible implementation manner, the quantization module includes:

[0111] A first determination module, configured to determine the type of context information based on the characteristics of the latent variable;

[0112] A construction module, configured to determine the context construction method corresponding to the type to construct different types of context information.

[0113] In a feasible implementation manner, the quantization module further includes:

[0114] A selection module, configured to select potential context information of the target latent variable in a preset manner;

[0115] A sorting module, configured to filter out target potential context information from the potential context information and sort the target potential context information to obtain intra-layer context information.

[0116] In a feasible implementation manner, the quantization module also includes:

[0117] A decoding module, configured to decode the latent variables of all levels before the target level to obtain the target context information of the target level;

[0118] A generation module, configured to generate target inter-layer context information based on the geometric information of the latent variable of the target level and the hierarchical context information of the target level.

[0119] In a feasible implementation manner, the quantization module further includes:

[0120] A second determination module, configured to determine a corresponding quantization strategy based on the geometric information of the latent variable;

[0121] A first quantization module, configured to perform targeted quantization on the latent variable through the corresponding quantization strategy.

[0122] In a feasible implementation, the quantization module further includes:

[0123] A third determination module, configured to determine the importance degree of the latent variable based on the trilinear interpolation coefficient of the latent variable;

[0124] A matching module, configured to match the importance degree with a standard degree category to obtain a corresponding matching result, and determine a quantization step size based on the matching result.

[0125] In a feasible implementation, the reconstruction module includes:

[0126] A retrieval module, configured to retrieve multiple decoded vertices associated with the point in the target resolution, and query the point cloud feature vectors of the multiple decoded vertices;

[0127] A splicing module, configured to splice the point cloud feature vectors of the multiple decoded vertices to obtain complete point cloud feature information.

[0128] Embodiment 3

[0129] The present application further provides an electronic device, as Figure 7 shown, including: a processor 701, a memory 702, and a bus 703. The memory 702 stores machine-readable instructions executable by the processor 701. When the electronic device runs, the processor 701 communicates with the memory 702 through the bus 703. When the machine-readable instructions are executed by the processor 701, the steps of any one of the point cloud data processing methods are executed.

[0130] Embodiment 4

[0131] The present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any one of the point cloud data processing methods are executed.

[0132] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the method embodiments, and will not be elaborated herein. In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical functional division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed with each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.

[0133] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0135] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0136] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A point cloud data processing method, characterized in that: The method comprises: Performing attribute preprocessing on the acquired point cloud data to obtain preprocessed point cloud data, and performing multi-level division on the preprocessed point cloud data to obtain a first multi-resolution grid; the first multi-resolution grid has a plurality of latent variables; quantizing the latent variable based on geometric information of the latent variable, constructing different types of context information based on the quantized latent variable, and constructing an entropy model based on the different types of context information; Processing the different types of context information based on the entropy model to obtain a prediction result, and encoding the quantized latent variable based on the prediction result to obtain bitstream information; The bit stream information is decoded to obtain a second multi-resolution grid, corresponding point data is obtained based on latent variables of the second multi-resolution grid, point cloud feature information carried by the point data is spliced, and point cloud data is reconstructed according to the spliced ​​point cloud feature information.

2. The method according to claim 1, characterized in that The different types of context information are constructed based on the quantified latent variables, including: determining a type of contextual information based on characteristics of the latent variable; A context construction method corresponding to the type is determined to construct different types of context information.

3. The method according to claim 2, characterized in that The types include at least intra-layer types; The determining is based on the context construction method corresponding to the type to construct different types of context information, including: Select the potential context information of the target latent variable according to a preset method; Target potential context information is filtered out from the potential context information, and the target potential context information is sorted to obtain intra-layer context information.

4. The method according to claim 2, characterized in that: The types include at least interlayer types; The determining is based on the context construction method corresponding to the type to construct different types of context information, including: Decode the latent variables of all levels before the target level to obtain the target context information of the target level; The target layer context information is generated based on the geometric information of the latent variables at the target layer and the hierarchical context information at the target layer.

5. The method according to claim 1, characterized in that The quantizing the latent variable based on the geometric information of the latent variable comprises: Determining a corresponding quantization strategy based on the geometric information of the latent variable; The latent variables are quantified in a targeted manner through corresponding quantification strategies.

6. The method according to claim 5, characterized in that The quantization strategy at least includes an intra-layer adaptive quantization strategy; The targeted quantification of the latent variables by using corresponding quantification strategies includes: determining the importance of the latent variable based on trilinear interpolation coefficients of the latent variable; The importance level is matched with the standard level category to obtain a corresponding matching result, and a quantization step size is determined based on the matching result.

7. The method according to claim 1, characterized in that The point cloud feature information carried by the points is stitched together, including: Retrieving a plurality of decoded vertices associated with the point in the target resolution, and querying point cloud feature vectors of the plurality of decoded vertices; Concatenate the point cloud feature vectors of multiple decoded vertices to obtain complete point cloud feature information.

8. A point cloud data processing device, characterized in that: The device comprises: A partitioning module is used to perform attribute preprocessing on the acquired point cloud data to obtain preprocessed point cloud data, and to perform multi-level partitioning on the preprocessed point cloud data to obtain a first multi-resolution grid; the first multi-resolution grid has a plurality of latent variables; A quantization module, configured to quantize the latent variable based on the geometric information of the latent variable, construct different types of context information based on the quantized latent variable, and construct an entropy model based on the different types of context information; An encoding module, configured to process the different types of context information based on the entropy model to obtain a prediction result, and to encode the quantized latent variable based on the prediction result to obtain bitstream information; A reconstruction module is used to decode the bit stream information to obtain a second multi-resolution grid, obtain corresponding point data based on latent variables of the second multi-resolution grid, splice point cloud feature information carried by the point data, and reconstruct the point cloud data based on the spliced ​​point cloud feature information.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the point cloud data processing method according to any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the point cloud data processing method according to any one of claims 1 to 7 are executed.

Images