Efficient point cloud processing method based on graph convolutional network and implicit neural representation

Abstract

The invention relates to the technical field of point cloud data, in particular to an efficient point cloud processing method based on a graph convolutional network and implicit neural representation, and two coordinate-based neural networks are used for processing geometric and attribute data of a point cloud respectively. A first network generates occupancy states of voxels and a second network generates attributes of occupancy voxels. According to the method, voxelization is carried out on the input point cloud, the space of the input point cloud is divided into a plurality of blocks, and only voxels in non-empty blocks are processed, so that the calculation efficiency is remarkably improved. According to the method, the graph convolutional network is introduced for learning and processing the spatial relationship between voxels in the point cloud data, so that the data can be effectively compressed while geometric details are kept. Besides, the neural network parameters are quantized and coded by using a deep content adaptive binary arithmetic coding method, so that the storage requirements are reduced, the reconstruction quality of the compressed data is ensured, and efficient transmission and storage of the data are also realized in a decoding stage.

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud data, and in particular to an efficient point cloud processing method based on graph convolutional networks and implicit neural representation. Background Art

[0002] Point cloud data is becoming increasingly important in many application areas, such as autonomous driving, augmented reality / virtual reality (AR / VR), digital twins, and robotics. This is because point cloud data can realistically describe three-dimensional objects and scenes. Point cloud data usually consists of a large number of three-dimensional points, each of which is defined by its geometric position and attributes such as color, normal, and reflectivity. With the advancement of sensing technology, large-scale point cloud data that can capture high-resolution spatial position and attribute information has become possible. However, the huge volume and unstructured nature of point cloud data pose severe challenges to storage and transmission, requiring a large amount of storage memory or bandwidth.

[0003] Traditional point cloud compression methods, such as octrees, triangular meshes, and 3D to 2D projections, have addressed these issues to some extent. These methods mainly rely on segmenting and projecting the geometry and attributes of point cloud data to reduce the amount of data. In recent years, deep learning techniques have been applied to point cloud compression, using an autoencoder architecture to directly process point cloud data, converting the input point cloud into a latent representation through an encoder, and reconstructing the input through a decoder. Although these learning-based methods outperform traditional methods in performance, they usually require large point cloud datasets for network training, and their performance on test data depends largely on the selection of training data. Summary of the invention

[0004] The purpose of the present invention is to provide an efficient point cloud processing method based on graph convolutional networks and implicit neural representation, reduce the storage and transmission requirements of point cloud data, and improve the efficiency and quality of compression and reconstruction.

[0005] To achieve the above object, the present invention provides an efficient point cloud processing method based on graph convolutional network and implicit neural representation, comprising the following steps:

[0006] Step 1: Point cloud data preprocessing and voxelization, quantizing the point cloud data into voxel grids and dividing the space into multiple small cubes;

[0007] Step 2: Generate the occupancy state of voxels. Use the first coordinate-based neural network to generate the occupancy state of each voxel.

[0008] Step 3: Generate attributes of occupied voxels using a second coordinate-based neural network.

[0009] Step 4: Quantize and encode the neural network parameters, and quantize the trained neural network parameters;

[0010] Step 5: Decode and reconstruct the geometry and attributes of the point cloud, restore the neural network parameters through the transmitted encoding parameters, and use two neural networks to reconstruct the geometry and attribute parts of the point cloud.

[0011] Optionally, the space in step 1 is further divided into 2 T ×2 T ×2 T small cubes, T is the parameter for the voxel space division, which determines the size of the small cubes. Each small cube contains 2 N-T ×2 N-T ×2 N-T Individual voxels.

[0012] Optionally, the execution process of step 2 includes the following steps:

[0013] First, the enhanced feature representation is processed by the graph convolutional network, and then the output is the occupancy probability p;

[0014] p=f θ (x)

[0015] Among them, f θ It is a composite neural network based on coordinate and graph convolutional networks. θ represents the geometric components of the original point cloud, and the input is the spatial coordinate x of the voxel.

[0016] According to the given threshold τ, determine whether the voxel is occupied:

[0017]

[0018] Where V represents the set of voxels in all non-empty cubes, The reconstructed geometry for all occupied voxels.

[0019] Optionally, in step 3, a second coordinate-based neural network g is used Φ Generate the attributes of the occupied voxels, Φ represents the attribute component of the original point cloud, and inputs the spatial coordinates of the occupied voxels Output corresponding attributes

[0020]

[0021] Optionally, the position encoding function in step 4 is defined as:

[0022] γ(x)=(x,sin(2 0 πx),cos(2 0 πx),…,sin(2 L-1πx),cos(2 L-1 πx))

[0023] Where L is the number of encoding layers, indicating the total number of sine and cosine functions of different frequencies, used to provide multi-frequency representation of position information;

[0024] The quantification formula is:

[0025]

[0026] Where k represents the original neural network parameters, Δ is the quantization step size, and round(·) means rounding to the nearest integer.

[0027] Optionally, a deep content adaptive binary arithmetic coding method is used for encoding and decoding. The key is to build a probability model and perform binary arithmetic coding, where each weight is assigned a specific probability model according to its context, that is, the surrounding or background information associated with the current data, covering other weights in the same layer or weights in the same position. By analyzing these context information, the probability of occurrence of symbols is dynamically adjusted to build a probability model that adapts to the current data; after the probability model is built, the quantized weights are encoded using binary arithmetic coding. The encoding of each weight depends on its probability distribution calculated by the context model. The specific formula for building the probability model and binary arithmetic coding is:

[0028] C(ω)=-log2P(q i |C)

[0029]

[0030] Where C(ω) is the encoding length of the weight ω, P(q i |C) is the probability of weight ω calculated based on the context model C, q i are the quantized weights, μ and σ 2 are the mean and variance calculated based on context C;

[0031] The decoding process is the inverse operation of the encoding process, and the formula is as follows:

[0032] q i =Decode(P(q i |C))

[0033] Among them, Decode(.) is the arithmetic decoding function.

[0034] Optionally, in step 5, different thresholds are set to optimize the reconstruction quality of the voxel occupancy state, and an appropriate threshold τ is selected to distinguish between occupied and unoccupied voxels in the space;

[0035] The judgment formula for the occupied state is:

[0036]

[0037] Optionally, the geometric loss function in step 5 adopts an α-balanced focal loss function, and the focal loss function expression is:

[0038]

[0039] where α is the balancing factor, is the predicted occupancy probability.

[0040] Optionally, the attribute loss function in step 5 uses a method based on Euclidean distance to quantify the difference between the predicted attribute and the true attribute. The loss function is defined as the squared Euclidean distance between the predicted attribute value and the nearest neighbor true attribute value, and the formula is:

[0041]

[0042] in is the predicted attribute, c nearest is the true property of the nearest neighbor.

[0043] The present invention provides an efficient point cloud processing method based on graph convolutional network and implicit neural representation, which uses two coordinate-based neural networks to process the geometry and attribute data of the point cloud respectively. The first network generates the occupancy state of the voxel, and the second network generates the attribute of the occupied voxel. By voxelizing the input point cloud and dividing its space into multiple blocks, only the voxels in those non-empty blocks are processed, thereby significantly improving the computational efficiency. The key point of the present method is to introduce a graph convolutional network (GCN) for learning and processing the spatial relationship between voxels in the point cloud data, which helps to effectively compress the data while maintaining the geometric details. In addition, by quantizing the neural network parameters and encoding them using the deep content adaptive binary arithmetic coding method (DeepCABAC), the present invention not only reduces the storage requirements, but also ensures the reconstruction quality of the compressed data. In the decoding stage, the neural network parameters obtained by decoding can accurately reconstruct the geometry and attributes of the point cloud, and realize the efficient transmission and storage of data. The experimental results show that the method of the present invention is superior to the existing octree method and other deep learning-based methods in terms of geometry and attribute compression, and has high versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a schematic diagram of the steps of an efficient point cloud processing method based on graph convolutional network and implicit neural representation of the present invention.

[0046] Figure 2 It is a schematic diagram of the specific execution flow of the present invention.

[0047] Figure 3 It is a schematic diagram of the structure of the neural network of the present invention. DETAILED DESCRIPTION

[0048] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0049] See also Figure 1 The present invention provides an efficient point cloud processing method based on graph convolutional network and implicit neural representation, comprising the following steps:

[0050] S1: Point cloud data preprocessing and voxelization, quantizing the point cloud data into voxel grids and dividing the space into multiple small cubes;

[0051] S2: Generate the occupancy state of voxels, using the first coordinate-based neural network to generate the occupancy state of each voxel;

[0052] S3: Generate properties of occupied voxels, using a second coordinate-based neural network to generate properties of occupied voxels;

[0053] S4: quantize and encode the neural network parameters, and quantize the trained neural network parameters and;

[0054] S5: Decode and reconstruct the geometry and attributes of the point cloud, restore the neural network parameters through the transmitted encoding parameters, and use two neural networks to reconstruct the geometry and attribute parts of the point cloud.

[0055] The following is a further explanation based on the specific execution steps:

[0056] In step S1, each cube is represented as a 3D voxel space. Assuming that the initial point cloud is voxelized at N-bit resolution, the voxel space should contain 2 N ×2 N ×2 N To optimize processing efficiency, the space is further divided into 2 T ×2 T ×2 T small cubes, T is the parameter for the voxel space division, which determines the size of the small cubes. Each small cube contains 2 N-T ×2 N-T ×2 N-T Individual voxels.

[0057] In step S2 and step S3, the present invention proposes a point cloud compression framework, including two coordinate-based neural networks:

[0058] The first neural network (geometric network) is used to generate the occupancy state of voxels. It receives the spatial coordinates of the voxels as input and outputs the occupancy probability of the voxels. This network uses an implicit neural representation method combined with a graph convolutional network (GCN) to learn the spatial relationship between voxels, thereby improving data compression and reconstructed geometric accuracy.

[0059] The second neural network (the attribute network) is used to determine the properties of the occupied voxels, such as color or reflectivity. This network also receives the spatial coordinates of the voxels and outputs the corresponding attribute values, allowing fine control over the visual and physical properties of the reconstruction process.

[0060] Step S2 is to generate the occupancy state of the voxel, see Figure 2 "Generating the occupancy state of voxels" section, Figure 3 The "network f θ (x)” section.

[0061] Network structure: A composite neural network () based on coordinates and graph convolutional networks (GCN) is used to generate the occupancy state of each voxel. The neural network contains graph convolution layers, position encoding, fully connected layers, ReLU activation functions, layer normalization, and residual connections.

[0062] The specific steps include:

[0063] 1. Graph convolutional layer: extracts effective feature representation of nodes for point cloud classification, segmentation and other downstream tasks.

[0064] 2. Position encoding: Encode the input voxel coordinate x to capture the spatial position information. The position encoding function is:

[0065] γ(x)=(x,sin(2 0 πx),cos(2 0πx),…,sin(2 L-1 πx),cos92 L-1 πx))

[0066] Where L is the number of frequencies.

[0067] 3. Network processing: The position-encoded input passes through multiple layers of fully connected layers and ReLU activation functions, each layer is followed by layer normalization, and finally the final occupancy state output p is obtained through residual connection.

[0068] p=f θ (γ9x)

[0069] 4. Occupancy status judgment: According to the given threshold τ, determine whether the voxel is occupied.

[0070]

[0071] Where V represents the set of voxels in all non-empty cubes, The reconstructed geometry for all occupied voxels.

[0072] Step S3 specifically generates the attributes of the occupied voxels, see Figure 2 Generate Occupied Voxel Properties section, Figure 3 "Network g Φ "part.

[0073] Network structure: Using a second coordinate-based neural network g Φ Generates attributes of occupied voxels. The neural network contains position encoding, fully connected layers, ReLU activation functions, layer normalization, and residual connections.

[0074] Specific steps:

[0075] 1. Nearest neighbor attribute mapping: For each occupied voxel, the nearest attribute value is found from the attribute data via the nearest neighbor method.

[0076] 2. Network processing: will occupy voxel coordinates and the nearest neighbor attribute c nearest As the output, it passes through multiple layers of fully connected layers and ReLU activation function, each layer is normalized, and finally the final attribute output is obtained through residual connection

[0077]

[0078] Step S4, quantize and encode the neural network parameters, see Figure 2 The "Quantizing and Encoding Neural Network Parameters" section of .

[0079] Parameter quantization: Quantize the trained neural network parameters θ and Φ, converting continuous value parameters into discrete values ​​to reduce the number of storage bits.

[0080]

[0081] Where Δ is the quantization step size and k is the neural network parameter.

[0082] The parameters of the neural network model are quantized and given a fixed step size Δ to facilitate efficient data encoding and transmission. The step size Δ given by the quantization process is efficiently encoded through the Deep Content Adaptive Binary Arithmetic Coding (DeepCABAC) method. Through this quantization process, the continuous numerical parameters of the network can be converted into discrete quantized values, thereby reducing the complexity of model storage.

[0083] Furthermore, the DeepCABAC method is used to encode the quantized parameters, converting the continuous or floating-point precision neural network parameters into discrete integer values.

[0084] Each weight of the model is assigned a probability model based on its context, which may include other weights in the same layer, weights at the same position, etc. The probability model can be expressed as:

[0085]

[0086] Among them, q i are the quantized weights, μ and σ 2 are the mean and variance calculated based on the context C.

[0087] For each quantized integer, DeepCABAC builds a context model based on the parameters’ positions in the neural network and their relationships with other parameters.

[0088] The quantized weights are encoded using binary arithmetic coding. The encoding of each weight depends on its probability distribution calculated by the context model. The encoding process can use the following formula:

[0089] C(ω)=-log2P(q i |C)

[0090] Where C(ω) is the encoding length of the weight ω, P(q i |C) is the probability of the weight ω calculated according to the context model C. Specifically, C is a model used to understand the contextual relationship between data (for example, the value of the neural network weight depends not only on itself, but may also be affected by other weights in the same layer or adjacent layers), mainly used to discover and utilize hidden rules in the data.

[0091] Through DeepCABAC, the quantized parameters are effectively compressed while maintaining the integrity of important information, thereby significantly reducing the transmission and storage costs of model parameters while ensuring model performance.

[0092] In addition, a sampling strategy for controlling the proportion of occupied voxels was adopted when training the network. This strategy balances the number of occupied and unoccupied voxels by selectively extracting voxels from point cloud data, thereby improving the efficiency of network learning and the generalization ability of the model.

[0093] Step S5, decode and reconstruct the geometry and attributes of the point cloud, see Figure 2 The "Decode and reconstruct point cloud geometry and attributes" section of

[0094] Parameter decoding: The quantized neural network parameters θ and Φ are restored through DeepCABAC decoding. The decoding process of DeepCABAC is the inverse operation of the encoding process. Its goal is to recover the quantized weights of the neural network from the compressed binary string and further convert these weights back to floating point format to reconstruct the original network model.

[0095] Binary arithmetic decoding, the first step to recover the quantized weights from the binary string. This step uses the same probability model as the encoding to decode the data.

[0096] q i =Decode(P(q i |C))

[0097] Among them, q i is the decoded quantization weight, P(q i |C) is a weighted probability model based on context C, and Decode(.) is an arithmetic decoding function.

[0098] Dequantize to convert the quantized integer back to its original floating point form.

[0099] ω=q i ×Δ

[0100] Where ω is the recovered floating point weight, q i is the decoded quantization weight, and Δ is the quantization step used in the original quantization process.

[0101] Since DeepCABAC uses dynamic context models during encoding, these models must also be updated and maintained during decoding to maintain decoding accuracy. After each weight is decoded, the context needs to be updated according to the newly decoded weight value in order to accurately decode subsequent weights.

[0102] C new =UpdateContext(C,qi )

[0103] Among them, C new is the updated context, q i is the decoded weight, and UpdateContext is the function that updates the context.

[0104] The geometric loss function in the reconstruction process uses an α-balanced focal loss function to enhance the model's ability to learn edges or unclear features, especially for processing unevenly distributed voxel data. The focal loss function expression is:

[0105]

[0106] where α is the balancing factor, is the predicted occupancy probability.

[0107] The attribute loss function uses a method based on Euclidean distance to quantify the difference between the predicted attribute and the true attribute. Specifically, the loss function is defined as the squared Euclidean distance between the predicted attribute value and the nearest neighbor true attribute value, and the formula is:

[0108]

[0109] in is the predicted attribute, c nearest is the true property of the nearest neighbor.

[0110] DeepCABAC decodes and reconstructs the geometry and attributes of the point cloud, restores the neural network parameters θ and Φ through the transmitted encoding parameters, and then uses these two neural networks to reconstruct the geometry and attribute parts of the point cloud.

[0111] Reconstruct geometry: Use the decoded neural network parameters θ to reconstruct the geometric part of the point cloud.

[0112]

[0113] Reconstruct attributes: Use the decoded neural network parameters Φ to reconstruct the attribute part of the point cloud.

[0114]

[0115] In summary, the present invention has the following beneficial effects:

[0116] A unified point cloud data processing framework is proposed, which integrates graph convolutional networks (GCN) and implicit neural representation technology, enabling it to efficiently process the geometric and attribute information of point clouds at the same time. This method significantly improves the accuracy and quality of reconstruction while maintaining a high data compression rate, and shows significant advantages in geometric compression and joint compression compared to the existing G-PCC standard.

[0117] The present invention introduces graph convolutional network (GCN) and implicit neural representation (INR) in point cloud compression, providing new research direction and technical means for point cloud compression.

[0118] The method of the present invention is not dependent on a specific data set and can be applied to various types of point cloud data. By training a single point cloud, efficient point cloud compression and reconstruction are achieved, and excellent performance can be demonstrated on different data sets.

[0119] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. An efficient point cloud processing method based on graph convolutional networks and implicit neural representation, characterized in that: The following steps are involved: Step 1: Point cloud data preprocessing and voxelization, quantizing the point cloud data into voxel grids and dividing the space into multiple small cubes; Step 2: Generate the occupancy state of voxels. Use the first coordinate-based neural network to generate the occupancy state of each voxel. Step 3: Generate attributes of occupied voxels using a second coordinate-based neural network. Step 4: Quantize and encode the neural network parameters, and quantize the trained neural network parameters; Step 5: Decode and reconstruct the geometry and attributes of the point cloud, restore the neural network parameters through the transmitted encoding parameters, and use two neural networks to reconstruct the geometry and attribute parts of the point cloud.

2. The efficient point cloud processing method based on graph convolutional network and implicit neural representation according to claim 1, characterized in that: In step 1, the space is further divided into 2 T ×2 T ×2 T small cubes, T is the parameter for the voxel space division, which determines the size of the small cubes. Each small cube contains 2 N-T ×2 N-T ×2 N-T Individual voxels.

3. The efficient point cloud processing method based on graph convolutional network and implicit neural representation according to claim 2, characterized in that: The execution process of step 2 includes the following steps: First, the enhanced feature representation is processed by the graph convolutional network, and then the output is the occupancy probability p; p=f θ (x) Among them, f θ It is a composite neural network based on coordinate and graph convolutional networks. θ represents the geometric components of the original point cloud, and the input is the spatial coordinate x of the voxel. According to the given threshold τ, determine whether the voxel is occupied: Where V represents the set of voxels in all non-empty cubes, The reconstructed geometry for all occupied voxels.

4. The efficient point cloud processing method based on graph convolutional network and implicit neural representation according to claim 3, characterized in that: In step 3, a second coordinate-based neural network g is used Φ Generate the attributes of the occupied voxels, Φ represents the attribute component of the original point cloud, and inputs the spatial coordinates of the occupied voxels Output corresponding attributes 5. The efficient point cloud processing method based on graph convolutional network and implicit neural representation according to claim 4, characterized in that: The position encoding function in step 4 is defined as: γ(x)=(x,sin(2 0 πx),cos(2 0 πx),…,sin(2 L-1 πx),cos(2 L-1 πx)) Where L is the number of encoding layers, indicating the total number of sine and cosine functions of different frequencies, used to provide multi-frequency representation of position information; The quantification formula is: Where k represents the original neural network parameters, Δ is the quantization step size, and round(·) means rounding to the nearest integer.

6. The efficient point cloud processing method based on graph convolutional network and implicit neural representation according to claim 5, characterized in that: The key to encoding and decoding using deep content adaptive binary arithmetic coding is to build a probability model and perform binary arithmetic coding, where each weight is assigned a specific probability model according to its context, that is, the surrounding or background information associated with the current data, covering other weights in the same layer or weights in the same position. By analyzing this context information, the probability of occurrence of symbols is dynamically adjusted to build a probability model that adapts to the current data; after the probability model is built, the quantized weights are encoded using binary arithmetic coding. The encoding of each weight depends on its probability distribution calculated by the context model. The specific formula for building the probability model and binary arithmetic coding is: C(ω)=-log2P(q i |C) Where C(ω) is the encoding length of the weight ω, exp is a function of the shape of the probability distribution, and P(q i |C) is the probability of weight ω calculated based on the context model C, q i are the quantized weights, μ and σ 2 are the mean and variance calculated based on context C; The decoding process is the inverse operation of the encoding process, and the formula is as follows: q i =Decode(P(q i |C)) Among them, Decode(.) is the arithmetic decoding function.

7. The efficient point cloud processing method based on graph convolutional network and implicit neural representation according to claim 6, characterized in that: In step 5, different thresholds are set to optimize the reconstruction quality of the voxel occupancy state, and the occupied and unoccupied voxels in the space are distinguished by selecting a suitable threshold τ; The judgment formula for the occupied state is:

8. The efficient point cloud processing method based on graph convolutional network and implicit neural representation according to claim 7, characterized in that: The geometric loss function in step 5 adopts the α-balanced focal loss function, and the focal loss function expression is: where α is the balancing factor, is the predicted occupancy probability.

9. The efficient point cloud processing method based on graph convolutional network and implicit neural representation according to claim 8, characterized in that: The attribute loss function in step 5 uses a method based on Euclidean distance to quantify the difference between the predicted attribute and the true attribute. The loss function is defined as the squared Euclidean distance between the predicted attribute value and the nearest neighbor true attribute value. The formula is: in is the predicted attribute, c nearest is the true property of the nearest neighbor.

Images