Quantization error correction device, quantization error correction method, and program
The quantization error correction device uses DNN architectures to analyze point groups at higher resolution, addressing the challenge of quantization errors in existing methods by accurately determining the occupied status of smaller divisions, enhancing decoding accuracy without increasing codes, and supporting both dense and sparse point groups.
Patent Information
- Application Number
- JP2022054279
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-03-29
- Publication Date
- 2025-01-29
- Estimated Expiration
- 2042-03-29
AI Technical Summary
Existing quantum error correction methods, such as the 8-neighborhood method, struggle to accurately distinguish overlapping points in a point group, leading to quantization errors that cannot be effectively reduced without increasing the number of codes, and are limited by the depth of the hierarchical structure, affecting the accuracy of point group decoding.
A quantization error correction device and method that utilizes feature extraction and estimation units to analyze point groups at a higher resolution than the quantization interval, employing DNN architectures like MCNN and decoder networks to accurately determine the occupied status of smaller divisions of the point group, thereby correcting quantization errors without increasing the number of codes.
The solution effectively reduces quantization errors in point groups by accurately estimating and correcting the occupied status of smaller divisions, improving the accuracy of point group decoding without increasing the number of codes, and is applicable to both dense and sparse point groups.
Smart Images
Figure 0007623976000002 
Figure 0007623976000003 
Figure 0007623976000004
Abstract
Description
[Technical field]
[0001] The present invention relates to a quantization error correction device, a quantization error correction method, and a program. [Background technology]
[0002] For example, a widely used technique for encoding and decoding a point group based on the octree method is described in Non-Patent Document 1. The octree method used in this technique is premised on quantization of the point group to be encoded.
[0003] The quantization of a point cloud is a process in which a three-dimensional space is divided into voxels of an arbitrary size, and the position coordinates of points existing inside each voxel are replaced with the position coordinates representing the voxel. Here, the position coordinates representing the voxel can be set to any position inside the voxel, and can be set to the center or vertex of the voxel, for example.
[0004] In the octree method, a voxel is divided into eight parts, and a process of recursively encoding an occupancy state that indicates whether or not a point exists in each divided voxel (occupied or unoccupied) is performed.
[0005] In point quantization, if multiple points exist within the same voxel, their coordinates are all quantized to the same value. Such points are called duplicate points.
[0006] In the octree method, since it is not possible to distinguish between overlapping points, it is necessary to encode them as a single point and to separately encode information indicating the number of overlapping points for each voxel. During decoding, the overlapping points can be decoded by duplicating the points decoded by the octree method by the corresponding number of duplicates.
[0007] Here, in order to reduce the amount of code required for the information representing the number of overlapping points, one of the overlapping points may be left and the rest may be deleted. This process is called deleting overlapping points.
[0008] The point cloud encoding and decoding technique based on the octree method allows lossless compression of quantized point clouds. However, since the information lost by quantization cannot be restored, an error occurs between the encoded point cloud and the decoded point cloud.
[0009] In the octree method, the position information of the point cloud is expressed in a hierarchical tree structure, and the deeper the hierarchy, the more the error can be reduced, but the more the amount of code increases. In other words, by changing the number of levels used to express the octree, the balance between the error and the amount of code can be adjusted.
[0010] Furthermore, a technique is being considered for reducing the error without increasing the amount of code by performing post-processing on the decoded point group. Note that the number of overlapping points does not represent the number of occupied voxels among the voxels divided into eight. For example, if the number of overlapping points is eight, there is a possibility that one of the eight voxels has eight points, or that each of the eight voxels has one point. Therefore, the number of overlapping points is not necessarily useful for reducing the error.
[0011] Non-Patent Document 2 discloses a technique for fine-tuning the coordinates of points encoded and decoded based on the octree method using Deep Neural Networks (DNN) called Coordinate Refinement Module (CRM). This makes it possible to reduce quantization errors without increasing the amount of code.
[0012] In Non-Patent Document 3, N pixels are arranged in each of the x, y, and z directions (total of N 3 A super-resolution technology has been disclosed that uses DNNs to reconstruct a higher-resolution volume from a volume composed of 1000 voxels (10000 voxels). This makes it possible to reduce quantization errors without increasing the amount of code. Non-Patent Document 4 discloses a technology for recovering an object surface from a point cloud using a DNN that represents a 3D shape using an implicit function representation. By applying this technology, it is possible to realize voxel super-resolution, which recovers a high-resolution volume from a low-resolution volume. Therefore, it is possible to reduce quantization errors without increasing the amount of code.
[0013] Non-Patent Document 5 discloses a super-resolution technique for point clouds acquired by LiDAR. In this technique, the point cloud is first projected onto a range image, the range image is super-resolved by a DNN, and then the range image is back-projected onto the point cloud to super-resolve the point cloud. [Prior art documents] [Non-patent literature]
[0014] [Non-Patent Document 1] G-PCC codec description v12, ISO / IEC JTC1 / SC29 / WG7 N0151 [Non-Patent Document 2] Zizheng Que, Guo Lu, and Dong Xu, "VoxelContext-Net: An Octree based Framework for Point Cloud Compression," In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2021 [Non-Patent Document 3] Skylar W. Wurster, et al, "Deep Hierarchical Super-Resolution for Scientific Data Reduction and Visualization," arXiv preprint arXiv:2107.00462, 2021 [Non-Patent Document 4] Songyou Peng, et al, "Convolutional occupancy networks," In Proceedings of the European Conference on Computer Vision, 2020 [Non-Patent Document 5] Tixiao Shan, et al, "Simulation-based lidar super-resolution for ground vehicles," Robotics and Autonomous Systems, vol. 134, 2020 [Non-Patent Document 6] Christopher Choy, JunYoung Gwak, and Silvio Savarese. "4D spatiotemporal convnets: Minkowski convolutional neural networks." In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2019 [Non-Patent Document 7] Haozhe Xie, et al. "Pix2vox: Context-aware 3d reconstruction from single and multi-view images." Proceedings of the IEEE / CVF International Conference on Computer Vision. 2019 [Non-Patent Document 8] Peng-Shuai Wang, et al. "Adaptive O-CNN: A patch-based deep representation of 3D shapes," ACM Transactions on Graphics (TOG), vol. 37, no.6, 2018 Summary of the Invention [Problem to be solved by the invention]
[0015] However, the CRM disclosed in Non-Patent Document 2 predicts a single offset for each voxel and performs fine adjustment by adding the offset to the position coordinates of points present inside the voxel. Therefore, even after such fine adjustment, overlapping points have the same coordinates. Therefore, when multiple points exist inside a voxel, there is a problem that it is not possible to reduce the quantization error individually. In addition, even when the overlapping points are deleted, there is a problem that it is not possible to reduce the quantization error individually for the multiple points before deletion.
[0016] In addition, the technology disclosed in Non-Patent Document 3 restores a high-resolution volume using features extracted from the entire input volume. Therefore, the occupancy state of each voxel in the input volume is not necessarily maintained in the high-resolution volume. For example, data may be restored to the position of an empty voxel in the input volume, or an empty voxel may be restored to the position of a voxel having data in the input volume. Therefore, when the accuracy of super-resolution is low, there is a problem that the error with the point cloud to be encoded may increase.
[0017] In addition, the technology disclosed in Non-Patent Document 4 requires information representing the surface of a target object in order to learn a model. When the point cloud is dense, it is easy to construct a surface from the point cloud, but it is difficult in the case of a sparse point cloud such as that acquired by LiDAR. Therefore, there is a problem that such a technology cannot necessarily be used for a sparse point cloud.
[0018] Furthermore, in the technology disclosed in Non-Patent Document 5, a high-resolution range image is restored using features extracted from an input range image. Therefore, a restoration error occurs in the high-resolution range image, which causes an error in the position coordinates of the back-projected point cloud. As a result, there is a problem that the error with the point cloud to be encoded may increase.
[0019] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a quantization error correction device, a quantization error correction method, and a program that can reduce quantization error without increasing the amount of code by performing post-processing on a point group that has been encoded and decoded using an encoding technique including quantization processing. [Means for solving the problem]
[0020] A first feature of the present invention is summarized as a quantization error correction device configured to correct quantization error of a quantized point cloud, comprising: a feature extraction unit configured to extract features for each point from the point cloud; and an occupancy estimation unit configured to estimate, from the features, an occupancy state of an area divided at intervals smaller than the quantization interval of the quantized point cloud.
[0021] A second feature of the present invention is a quantization error correction method for correcting quantization error of a quantized point cloud, comprising the steps of: extracting features for each point from the point cloud; and estimating, from the features, an occupancy state of an area divided at intervals smaller than the quantization interval of the quantized point cloud.
[0022] A third feature of the present invention is summarized as a program for causing a computer to function as a quantization error correction device configured to correct quantization errors of a quantized point cloud, the quantization error correction device having a feature extraction unit configured to extract features for each point from the point cloud, and an occupancy estimation unit configured to estimate, from the features, an occupancy state of an area divided at intervals smaller than the quantization interval of the quantized point cloud. Effect of the Invention
[0023] According to the present invention, it is possible to provide a quantization error correction device, a quantization error correction method, and a program that can reduce quantization error without increasing the amount of code by performing post-processing on a point group that has been encoded and decoded using an encoding technique that includes a quantization process. [Brief description of the drawings]
[0024] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a point cloud processing system 10 according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of functional blocks of a quantization error correction device 300 according to an embodiment. [Diagram 3] FIG. 3 is a schematic diagram of a voxel. [Figure 4] FIG. 4 is a flowchart showing an example of the operation of the quantization error correction device 300 according to an embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of functional blocks of a quantization error correction device 300 according to the first modification. [Figure 6] FIG. 6 is a diagram showing an example of the configuration of a point cloud processing system 10 according to the second modified example. [Figure 7] FIG. 7 is a diagram showing an example of functional blocks of a quantization error correction device 300 according to the third modification. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0025] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, and various variations including combinations with other existing components are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.
[0026] (First embodiment) A point cloud processing system 10 according to a first embodiment of the present invention will be described below with reference to Figures 1 to 4. Figure 1 is a diagram showing a point cloud processing system 10 according to the embodiment.
[0027] As shown in FIG. 1, a point cloud processing system 10 includes a point cloud encoding device 100, a point cloud decoding device 200, and a quantization error correction device 300.
[0028] The point cloud encoding device 100 is configured to generate encoded data (bit stream) by encoding an input point cloud. The point cloud decoding device 200 is configured to generate a decoded point cloud by decoding the bit stream. The quantization error correction device 300 is configured to generate a corrected point cloud by correcting the decoded point cloud.
[0029] Here, such a bit stream may be transmitted from the point group encoding device 100 to the point group decoding device 200 via a transmission path. Also, the bit stream may be stored in a storage medium and then provided from the point group encoding device 100 to the point group decoding device 200.
[0030] Moreover, the decoded point group may be transmitted from the point group decoding device 200 to the quantization error correction device 300 via a transmission path. Moreover, the decoded point group may be stored in a storage medium and then provided from the point group decoding device 200 to the quantization error correction device 300.
[0031] (Quantization error correction device 300) The quantization error correction device 300 according to this embodiment will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the quantization error correction device 300 according to this embodiment.
[0032] 2, the quantization error correction device 300 includes a feature extraction unit 310, an occupancy estimation unit 320, and a model learning unit 330. These functional blocks are realized by executing a program that causes a computer installed in the quantization error correction device 300 to function.
[0033] Each functional block of the quantization error correction device 300 according to this embodiment will be described below.
[0034] [Feature extraction unit 310] The feature extractor 310 is configured to extract point-wise features from the input point cloud.
[0035] In this embodiment, the input point cloud is a point cloud quantized using voxels, in which case the feature extraction unit 310 may be configured to extract features using any DNN capable of extracting features point-wise.
[0036] For example, the feature extraction unit 310 may be configured to extract features using Minkowski Convolutional Neural Networks (MCNN) as described in Non-Patent Document 6.
[0037] Here, MCNN is configured to first assign points to voxels and extract the above-mentioned features by convolution of the voxels. Therefore, by matching the size of the voxels used in quantizing the input point cloud with the size of the voxels used in MCNN, points and voxels can be associated one-to-one. This makes it possible to extract features for each voxel, i.e., for each point.
[0038] In addition, MCNN employs a sparse convolution method that convolves only occupied voxels, enabling efficient feature extraction even for large-scale point clouds such as those acquired by LiDAR.
[0039] That is, the feature extraction unit 310 may be configured to extract point-wise features by convolving only the non-zero elements using a sparse convolution method.
[0040] [Occupancy estimation unit 320] The occupancy estimation unit 320 is configured to estimate the occupancy state of voxels having a higher resolution than the voxels used for the quantization for each point, where the occupancy state is expressed in two ways: occupied or unoccupied.
[0041] In this embodiment, the voxels obtained by dividing the voxels used for quantization into M are assumed to be high-resolution voxels. The method of dividing the voxels may be a method of dividing at equal intervals along the x-axis, y-axis, and z-axis, or another method may be used.
[0042] In other words, the occupancy estimation unit 320 is configured to estimate the occupancy state of an area divided at intervals smaller than the quantization intervals of the quantized point group from the characteristics of each point.
[0043] A schematic diagram of a voxel is shown in Fig. 3. In Fig. 3, a cube drawn with a thick line represents a voxel used for quantization, and a cube drawn with a thin line represents a high-resolution voxel. In the example of Fig. 3, a voxel is divided into two parts at equal intervals along the x-axis, y-axis, and z-axis, resulting in M=8.
[0044] The occupancy estimation unit 320 is configured to estimate M occupancy states corresponding to the M voxels using an arbitrary DNN architecture from the features extracted by the feature extraction unit 310.
[0045] One example of a DNN architecture is a fully connected neural network.
[0046] In addition, there are two possible approaches to estimating the M occupancy states: classification and regression.
[0047] Here, for classification, the occupancy states of M voxels are 2 M However, since it is known that there are points inside the voxels used for quantization, it is not possible for all of them to be unoccupied. Therefore, there are two classes, excluding the one where all are unoccupied. M For each of the -1 classes, the DNN outputs a prediction of the probability that the above features belong, and the occupancy state corresponding to the class with the highest probability is the estimated result.
[0048] That is, the occupancy estimation unit 320 may be configured to estimate the occupancy state by classifying into classes represented by a combination of occupancy states of multiple voxels (areas divided at intervals smaller than the quantization interval of the quantized point cloud).
[0049] On the other hand, in the case of regression, the output of the DNN is a prediction of M possible occupancy states. Here, occupancy is represented as "1" and non-occupancy is represented as "0", and the prediction of the occupancy state is a scalar value in the range of 0 to 1.
[0050] The occupancy estimation unit 320 may be configured to estimate the occupancy state by performing binary classification using a threshold for each of the M types of outputs.
[0051] That is, the occupancy estimation unit 320 may be configured to estimate the occupancy state by determining predicted values of the occupancy states of M voxels using a threshold value.
[0052] Here, the threshold value may be any scalar value in the range of 0 to 1. However, since it is known that there are points inside the voxels used for quantization, it is not possible for all of them to be unoccupied. Therefore, in order to guarantee that at least one voxel is occupied, a dynamic threshold T d may also be used.
[0053]
number
[0054] In the case of regression, the occupancy estimation unit 320 may also use a DNN architecture based on deconvolution using a 3D structure.
[0055] An example of such a DNN architecture is the decoder architecture described in Non-Patent Document 7. Such a decoder takes the above features as input and reconstructs a volume of M voxels. Since each voxel contains a prediction of its occupancy, such a decoder can estimate its occupancy by classifying them as binary using a threshold.
[0056] In this embodiment, the occupancy estimation unit 320 may be configured to estimate M occupancy states corresponding to M voxels using point-by-point features as input.
[0057] In other words, the occupancy estimator 320 may be configured to estimate the occupancy state by deconvolving pointwise features.
[0058] The occupancy estimation unit 320 may also use a DNN architecture that predicts octree codes hierarchically.
[0059] As an example of such a DNN architecture, the decoder architecture described in Non-Patent Document 8 can be considered. Such a decoder predicts an octree code for one layer using the above-mentioned features as input. Then, such a decoder extracts features from the predicted octree code through a deconvolution layer and a convolution layer, and predicts an octree code for the next layer from the extracted features. In this way, such a decoder can estimate the occupancy state of a high-resolution voxel by predicting the octree code hierarchically up to a specified depth.
[0060] The approach to predict the octree code may be classification or regression. In this embodiment, the occupancy estimation unit 320 may be configured to estimate M occupancy states corresponding to M voxels using point-by-point features as input. Here, M is the number of voxels that can be represented using an octree.
[0061] In other words, the occupancy estimation unit 320 may be configured to estimate the occupancy state of each voxel by predicting an octree code hierarchically.
[0062] In addition, the occupancy estimation unit 320 may be configured to estimate the occupancy state by combining a fully connected neural network, a DNN architecture based on deconvolution, or a DNN architecture that hierarchically predicts an octree code.
[0063] After estimating the occupancy state for each point, the occupancy estimation unit 320 is configured to restore the point cloud using the occupancy state. That is, the occupancy estimation unit 320 is configured to generate points at position coordinates representative of the high-resolution voxels estimated to be occupied, and replace the original points with these points. This restored point cloud is defined as the output point cloud.
[0064] [Model Learning Part 330] The model training unit 330 is configured to train a model by calculating a loss between the true value and the prediction of the above-mentioned occupancy state. For example, the model training unit 330 may be configured to train a model parameter of the DNN so as to minimize such a loss.
[0065] The model learning unit 330 is configured to create a set of input points and true values of occupancy states by quantizing a point cloud quantized using high-resolution voxels using lower-resolution voxels. Here, it is assumed that high-resolution voxels can be constructed by dividing a low-resolution voxel into M pieces. At this time, the occupancy states of M high-resolution voxels can be associated with each point of the point cloud quantized using the low-resolution voxel.
[0066] In this embodiment, a point group quantized using low-resolution voxels is used as an input point group, and the occupancy state of high-resolution voxels corresponding to each point of the input point group is used as the true value of the occupancy state.
[0067] When using a classification approach, the model training unit 330 can use a cross-entropy loss function as a loss function. However, when the number of data belonging to each class is significantly biased, the cross-entropy loss function may not improve the classification accuracy. To address this issue, the model training unit 330 may use a balanced cross-entropy loss function or a focal loss function.
[0068] On the other hand, when a regression approach is used, the model training unit 330 can use a mean squared error function or a binary cross-entropy loss function as a loss function. However, with the binary cross-entropy loss function, if there is a large bias in the number of "1"s and "0"s, there is a risk that the regression accuracy will not improve. To address this issue, the model training unit 330 may use a weighted binary cross-entropy loss function.
[0069] In addition, when using a DNN architecture that estimates an octree code, the model learning unit 330 may use the structural loss function or patch loss function described in Non-Patent Document 8.
[0070] That is, when calculating the above-mentioned loss, the model learning unit 330 may be configured to change the coefficients of the loss function using the distribution of training data across classes represented by a combination of the occupancy states of multiple voxels or the distribution of training data across two classes representing the voxel occupancy states.
[0071] An example of the operation of the quantization error correction device 300 according to this embodiment will be described below with reference to FIG.
[0072] As shown in FIG. 4, in step S101, the quantization error correction device 300 extracts features for each point from the quantized point group.
[0073] In step S102, the quantization error correction device 300 estimates, from the extracted features, the occupancy state of regions (voxels) divided at intervals smaller than the quantization intervals of the quantized point group.
[0074] In step S103, the quantization error correction device 300 reconstructs the point cloud using the estimated occupancy state, and outputs the reconstructed point cloud as an output point cloud.
[0075] In step S104, the quantization error correction device 300 calculates the loss between the estimated result of the above-mentioned occupancy state and the true value, and learns the model.
[0076] According to this embodiment, by estimating the occupancy state of voxels with a higher resolution than the voxels used for quantization, it is possible to appropriately correct the quantization error of the point group without increasing the amount of code.
[0077] (Change example 1) Hereinafter, a first modified example of the present invention will be described with reference to FIG. 5, focusing on differences from the above-described first embodiment.
[0078] The quantization error correcting device 300 according to the present modified example 1 is configured to hold the trained model parameters in the storage unit 340. Therefore, the quantization error correcting device 300 does not need to execute the process of the model training unit 330 described above.
[0079] As shown in FIG. 4, in a quantization error correction device 300 according to this modified example, a feature extraction section 310 and an occupancy estimation section 320 perform processing using trained model parameters stored in a storage section 340 .
[0080] Here, such model parameters may be stored in advance in any storage medium and then transmitted to the quantization error correction device 300 before encoding and decoding, or may be transmitted to the quantization error correction device 300 simultaneously with the bit stream obtained by encoding, or may be transmitted to the quantization error correction device 300 during the decoding of the point cloud from the bit stream.
[0081] (Change example 2) Hereinafter, a second modified example of the present invention will be described with reference to FIG. 6, focusing on differences from the first embodiment described above.
[0082] The point cloud processing system 10 according to this modified example is configured to fine-tune the model parameters while performing encoding.
[0083] As shown in FIG. 6, a point cloud processing system 10 according to this modified example includes a point cloud encoding device 100, a point cloud decoding device 200, a first quantization error correction device 300A, and a second quantization error correction device 300B.
[0084] The first quantization error correction unit 300A is configured to fine-tune the model using the input point cloud.
[0085] The second quantization error correction unit 300B is configured to generate a correction point cloud by correcting the decoded point cloud using the fine-tuned model parameters.
[0086] In the point cloud processing system 10 of this modified example, the fine-tuned model parameters may be transmitted to the second quantization error correction device 300B while the point cloud encoding device 100 is encoding the input point cloud, or may be transmitted to the second quantization error correction device 300B while the point cloud encoding device 100 is transmitting a bit stream obtained by encoding the input point cloud to the point cloud decoding device 200, or may be transmitted to the second quantization error correction device 300B while the point cloud decoding device 200 is decoding the point cloud from the bit stream.
[0087] (Change example 3) Hereinafter, a third modified example of the present invention will be described with reference to FIG. 7, focusing on differences from the above-described first embodiment.
[0088] In this modification, as shown in FIG. 7, the output points generated from the input points by the quantization error correcting device 300 are regarded as the input points, so that the quantization error can be corrected multiple times.
[0089] That is, as shown in FIG. 7, the quantization error correction device 300 according to this modified example may be configured to recursively correct the quantization error by inputting the point group reconstructed using the above-mentioned occupancy state to the feature extraction unit 310.
[0090] The quantization error correcting device 300 according to this modified example may execute the model learning unit 330 each time the quantization error is corrected once, or may execute the model learning unit 330 after the quantization error is corrected multiple times.
[0091] Furthermore, the quantization error correction device 300 according to this modified example may be configured to learn model parameters in advance individually according to the size of the voxel that quantizes the point cloud, and when correcting the quantization error, execute the feature extraction unit 310 and the occupancy estimation unit 320 using model parameters corresponding to the size of the voxel that quantizes the input point cloud. With this configuration, it is possible to use model parameters suitable for the input point cloud.
[0092] Furthermore, the above-mentioned point group encoding device 100, point group decoding device 200, and quantization error correction device 300 may be realized as a program that causes a computer to execute each function (each process).
[0093] In the above embodiments, the present invention has been described by taking the example of application to the quantization error correction device 300. However, the present invention is not limited to such an example, and can be similarly applied to a system having each function of the quantization error correction device 300. [Industrial Applicability]
[0094] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which makes it possible to contribute to Goal 9 of the Sustainable Development Goals (SDGs) led by the United Nations, which is to "build resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]
[0095] 10...Point cloud processing system 100...Point cloud encoding device 200...Point cloud decoding device 300…Quantization error correction device 310…Feature extraction unit 320...occupancy estimation section 330…Model Learning Section
Claims
1. 1. A quantization error correction device configured to correct a quantization error of a quantized point cloud, comprising: a feature extraction unit configured to extract point-wise features from the point cloud; and an occupancy estimation unit configured to estimate, from the features, an occupancy state of an area divided at intervals smaller than the quantization interval of the quantized point group.
2. 2. The quantization error correction device according to claim 1, wherein the feature extraction unit is configured to extract the features by convolving only non-zero elements using a sparse convolution method.
3. 3. The quantization error correction device according to claim 1, wherein the occupancy estimation unit is configured to estimate the occupancy state by classification into classes represented by combinations of occupancy states of a plurality of the regions.
4. The quantization error correction device according to any one of claims 1 to 3, characterized in that the occupancy estimation unit is configured to estimate the occupancy state by determining a predicted value of the occupancy state of the region using a threshold value.
5. 5. The quantization error correction device according to claim 4, wherein the occupancy estimation unit is configured to replace the threshold with the maximum value when the maximum value of the predicted value of the occupancy state of the divided area is less than the threshold.
6. The quantization error correction device according to any one of claims 1 to 5, characterized in that the occupancy estimation unit is configured to estimate the occupancy state by projecting the features using a fully connected neural network.
7. 7. The quantization error correction device according to claim 1, wherein the occupancy estimation unit is configured to estimate the occupancy state by deconvolving the features.
8. 8. The quantization error correction device according to claim 1, wherein the occupancy estimation unit is configured to estimate the occupancy state by hierarchically predicting an octree code.
9. The quantization error correction device according to any one of claims 1 to 8, further comprising a model learning unit configured to learn a model by calculating a loss between the estimated result of the occupancy state and a true value.
10. The quantization error correction device according to claim 9, characterized in that, when calculating the loss, the model learning unit changes a coefficient of a loss function using a distribution of training data across classes represented by a combination of occupancy states of a plurality of the regions or a distribution of training data across two classes representing the occupancy states.
11. The quantization error correction device according to any one of claims 1 to 10, characterized in that the quantization error is recursively corrected by inputting a point group reconstructed using the occupancy state to the feature extraction unit.
12. A quantization error correction method for correcting a quantization error of a quantized point group, comprising the steps of: extracting point-wise features from the point cloud; and estimating, from the features, an occupancy state of an area divided at intervals smaller than the quantization interval of the quantized point group.
13. A program for causing a computer to function as a quantization error correction device configured to correct a quantization error of a quantized point cloud, the program comprising: The quantization error correction device comprises: a feature extraction unit configured to extract point-wise features from the point cloud; and an occupancy estimation unit configured to estimate, from the features, an occupancy state of an area divided at intervals smaller than the quantization interval of the quantized point cloud.