Point group decoding device, point group decoding method and program

Abstract

To perform a scalable decoding with a limited output number to achieve a score not larger than a specified score.SOLUTION: A score group decoder 200 has a geometric information decoding unit 2010 formed to decode the score of each layer of the octree structure or the difference in the scores of the layers of the octree structure.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a point group decoding device, a point group decoding method, and a program. [Background technology]

[0002] Non-Patent Documents 1 and 2 disclose a technique for decoding 3D position (geometry) information of a point cloud compressed by recursively performing octree division, and a technique for decoding attribute information of points corresponding to the decoded point cloud positions as needed.

[0003] Furthermore, Non-Patent Document 3 discloses, as one of the functions of Non-Patent Document 1 and Non-Patent Document 2, a scalable decoding technology that decodes point groups of different resolutions in a scalable manner by decoding the Octree structure only up to an intermediate resolution, as shown in FIG. 13.

[0004] Such scalable decoding technology allows low-resolution point clouds to be decoded in a scalable manner without decoding the entire bitstream, and can be used for applications such as thumbnails. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Text of ISO / IEC 23090-9 DIS Geometry-based PCC w19088 [Non-patent document 2] G-PCC codec description v6, ISO / IEC JTC1 / SC29 / WG11 w19091 [Non-patent document 3] Spatial scalability support for G-PCC, ISO / IEC JTC1 / SC29 / WG11 m47352 Summary of the Invention [Problem to be solved by the invention]

[0006] Generally, when considering an application such as a viewer for viewing point clouds generated using free viewpoint video technology, real-time rendering becomes impossible on a device with limited computing resources if too many points are input.

[0007] Therefore, in order to perform real-time rendering, there are cases where it is desirable to suppress the score after decoding to a certain level or less using the scalable decoding function disclosed in Non-Patent Document 3.

[0008] However, in the specifications described in Non-Patent Document 1, when implementing the scalable decoding disclosed in Non-Patent Document 3, although it is possible to decode the top n layers of the Octree structure, the score of the (n+1)th layer from the top cannot be known until decoding is performed, which poses a problem in that it is not possible to decode within a specified score.

[0009] In this regard, although the score can be found by decoding the (n+1)th layer, there is a problem in that decoding the (n+1)th layer imposes a large processing load.

[0010] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a point cloud decoding device, a point cloud decoding method, and a program that can perform scalable decoding by limiting the number of output points so that the number of points is equal to or less than a specified number of points. [Means for solving the problem]

[0011] A first feature of the present invention is summarized as a point cloud decoding device including a geometric information decoding unit configured to decode the number of points in each layer of an octree structure or the difference between the number of points in each layer of the octree structure.

[0012] A second feature of the present invention is summarized as a point cloud decoding device comprising a geometric information decoding unit configured to decode the number of points in each layer of an octree structure or the difference between the number of points in each layer of the octree structure, wherein the geometric information decoding unit is configured to decode m (m is an integer equal to or greater than 1) defined as a syntax, and not to record the number of points or the difference between the points in the first m layers.

[0013] A third feature of the present invention is summarized as a point cloud decoding device comprising: a tree synthesis unit configured to perform scalable decoding up to a layer m (m is an integer equal to or greater than 1) layers below an input layer; and an LoD calculation unit configured to calculate an LoD (Level of Detail) based on geometric information of the (m+1)th layer.

[0014] A fourth aspect of the present invention is summarized as a point group decoding method, comprising a step of decoding the number of points in each layer of an octree structure or a difference between the number of points in each layer of the octree structure.

[0015] A fifth feature of the present invention is a program for use in a point cloud decoding device, which causes a computer to execute a step of decoding the number of points in each layer of an octree structure or the difference between the number of points in each layer of the octree structure. [Effects of the Invention]

[0016] According to the present invention, it is possible to provide a point cloud decoding device, a point cloud decoding method, and a program that can perform scalable decoding by limiting the number of output points so that the number of points is equal to or less than a specified number of points. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a diagram illustrating an example of the configuration of a point cloud processing system 10 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of a point group decoding device 200 according to an embodiment. [Figure 3]10 is a diagram illustrating an example of the configuration of coded data (bit stream) received by a geometric information decoding unit 2010 of a point cloud decoding device 200 according to an embodiment. [Figure 4] 1 is an example of a syntax configuration of GPS2011 according to an embodiment. [Figure 5] 10 is an example of a syntax configuration of GPS2012A / 2012B according to an embodiment. [Figure 6] 10 is an example of a syntax configuration of GPS2012A / 2012B according to an embodiment. [Figure 7] FIG. 2 is a diagram illustrating control data decoded by a geometric information decoding unit 2010 of the point cloud decoding device 200 according to an embodiment. [Figure 8] FIG. 2 is a diagram illustrating control data decoded by a geometric information decoding unit 2010 of the point cloud decoding device 200 according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating control data decoded by an attribute information decoding unit 2060 of the point cloud decoding device 200 according to an embodiment. [Figure 10] 1 is an example of a syntax configuration of APS2061 according to an embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of processing details of an LoD calculation unit 2090 of the point cloud decoding device 200 according to an embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of processing details of an LoD calculation unit 2090 of the point cloud decoding device 200 according to an embodiment. [Figure 13] FIG. 1 is a diagram illustrating a conventional technique. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0019] (First embodiment) A point cloud processing system 10 according to a first embodiment of the present invention will be described below with reference to Figures 1 to 12. Figure 1 is a diagram showing a point cloud processing system 10 according to the present embodiment.

[0020] As shown in FIG. 1, the point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0021] The point cloud encoding device 100 is configured to generate encoded data (bitstream) by encoding an input point cloud signal, and the point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bitstream.

[0022] The input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud, such as color information and reflectance of each point.

[0023] Here, the bit stream may be transmitted from the point group encoding device 100 to the point group decoding device 200 via a transmission path. Alternatively, the bit stream may be stored in a storage medium and then provided from the point group encoding device 100 to the point group decoding device 200.

[0024] (Point Cloud Decoding Device 200) The point group decoding device 200 according to this embodiment will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the point group decoding device 200 according to this embodiment.

[0025] The point cloud decoding device 200 has a function of receiving the bit stream generated by the point cloud encoding device 100 as input and decoding the position information and attribute information of the point cloud.

[0026] 2, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT (Region Adaptive Hierarchical Transform) unit 2080, an LoD (Level of Detail) calculation unit 2090, an inverse lifting unit 2100, and an inverse color transformation unit 2110. The following describes the detailed functions of each unit in the functional block diagram shown in FIG.

[0027] The geometric information decoding unit 2010 is configured to receive as input a bit stream relating to geometric information (geometric information bit stream) from among the bit streams output from the point group encoding device 100, and to decode the syntax.

[0028] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0029] The tree synthesis unit 2020 is configured to take as input the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating at which node in the tree structure described below the point group exists, and generate the position of the point (tree information) indicating in which area within the space to be decoded the point exists.

[0030] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree information synthesis unit 2020.

[0031] For example, when decoding three-dimensional point cloud data of an object, if the point cloud is densely distributed on the surface of the object, approximate surface information is used to represent the area where the point cloud exists by approximating it with a small plane, rather than decoding each individual point cloud.

[0032] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information using, for example, a method called "Trisoup." Specific processing of "Trisoup" can be, for example, the methods described in Non-Patent Document 1 and Non-Patent Document 2. Furthermore, when decoding a sparse point cloud acquired by Lidar or the like, this processing can be omitted.

[0033] The geometric information reconstruction unit 2040 is configured to reconstruct the geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point group to be decoded based on the tree information generated by the tree information synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0034] The inverse coordinate transformation unit 2050 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform it from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

[0035] The attribute information decoding unit 2060 is configured to receive as input a bit stream relating to attribute information (attribute information bit stream) from among the bit streams output from the point group encoding device 100, and to decode the syntax.

[0036] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0037] Moreover, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

[0038] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and the quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, to generate inverse quantized residual information.

[0039] The dequantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090, depending on the features of the point group to be decoded. The control data decoded by the attribute information decoding unit 2060 specifies which unit the information is to be output to.

[0040] The RAHT unit 2080 is configured to receive as input the dequantized residual information generated by the dequantized residual information and the geometric information generated by the geometric information reconstruction unit 2040, and to decode the attribute information of each point using a type of Haar transform called RAHT (inverse Haar transform in the decoding process). Specific processing of the RAHT can be performed using the methods described in Non-Patent Document 1 and Non-Patent Document 2, for example.

[0041] The LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 as an input and generate an LoD.

[0042] LoD is information for defining the reference relationship (reference point and referenced point) to realize predictive coding, such as predicting attribute information of another point from attribute information of another point and encoding or decoding the prediction residual.

[0043] In other words, LoD is information that defines a hierarchical structure in which each point contained in geometric information is classified into multiple levels, and the attributes of points belonging to higher levels are encoded or decoded using the attribute information of points belonging to lower levels.

[0044] As a specific method for determining the LoD, for example, the methods described in Non-Patent Documents 1 and 2 may be used.

[0045] The inverse lifting unit 2100 is configured to decode the attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse-quantized residual information generated by the inverse-quantized residual information. Specific processing of the inverse lifting can be, for example, the methods described in Non-Patent Document 1 and Non-Patent Document 2.

[0046] The inverse color conversion unit 2110 is configured to perform inverse color conversion processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information to be decoded is color information and color conversion has been performed on the point group encoding device 100 side. Whether or not to perform such inverse color conversion processing is determined by the control data decoded by the attribute information decoding unit 2060.

[0047] The point cloud decoding device 200 is configured to decode and output attribute information of each point in the point cloud through the above processing.

[0048] Below, the parts of the point group decoding device 200 that are specific to the present invention will be described.

[0049] (Geometric Information Decoding Unit 2010) The control data decoded by the geometric information decoding unit 2010 will be described below with reference to FIGS.

[0050] FIG. 4 shows an example of the structure of coded data (bit stream) received by the geometric information decoding unit 2010. In FIG.

[0051] First, the bitstream may include a GPS2011. GPS2011 is an abbreviation for Geometry Parameter Set, and is a set of control data related to decoding of geometric information. Specific examples will be described later. Each GPS2011 includes at least GPS ID information for identifying each GPS2011 when there are multiple GPS2011s.

[0052] Second, the bitstream may include GSH2012A / 2012B. GSH2012A / 2012B stands for Geometry Slice Header and is a collection of control data corresponding to a slice, which will be described later. Specific examples will be described later. GSH2012A / 2012B includes at least GPS ID information for specifying the GPS2011 corresponding to each GSH2012A / 2012B.

[0053] Third, the bitstream may include slice data 2013A / 2013B following the GSH 2012A / 2012B. The slice data 2013A / 2013B includes data that encodes geometric information. An example of the slice data 2013A / 2013B is an occupancy code, which will be described later.

[0054] As described above, the bit stream is configured such that each slice data 2013A / 2013B corresponds to one GSH 2012A / 2012B and one GPS 2011.

[0055] As described above, in GSH2012A / 2012B, which GPS2011 to refer to is specified by GPS ID information, so a common GPS2011 can be used for multiple slice data 2013A / 2013B.

[0056] In other words, it is not necessary to transmit GPS 2011 for each slice. For example, as shown in Fig. 3, the bitstream may be configured such that GPS 2011 is not coded immediately before GSH 2012B and slice data 2013B.

[0057] Note that the configuration in Figure 3 is merely an example. As long as each slice data 2013A / 2013B is configured to correspond to GSH2012A / 2012B and GPS2011, elements other than those described above may be added as components of the bit stream. For example, the bit stream may include a sequence parameter set (SPS). Similarly, the bit stream may be shaped into a configuration different from that shown in Figure 3 when transmitted.

[0058] Figure 4 shows an example of the syntax configuration of GPS2011.

[0059] Note that the syntax names explained below are merely examples. If the syntax functions explained below are similar, the syntax names may be different.

[0060] The GPS 2011 may include GPS ID information (gps_geom_parameter_set_id) for identifying each GPS 2011.

[0061] The GPS 2011 may include a flag (inferred_direct_coding_mode_enabled_flag) for controlling ON / OFF of IDCM (inferred direct coding mode), which will be described later, in the tree synthesis unit 2020 .

[0062] Non-patent documents 1 and 2 disclose a method (implicitQtBt) for performing quadtree or binary tree partitioning instead of octree partitioning. Based on this method, GPS2011 may include a flag (gps_implicit_geom_partition_flag) indicating whether or not to perform quadtree or binary tree partitioning (QtBt) in the tree synthesis unit 2020, as described in Non-patent documents 1 and 2.

[0063] For example, if the value of gps_implicit_geom_partition_flag is "1", it may be defined that "QtBt" is performed, and if the value of gps_implicit_geom_partition_flag is "0", it may be defined that only "Octree" is performed.

[0064] GPS2011 may include a flag (geom_recording_point_num_flag) that controls whether or not to record the number of points at each level when the tree structure is decoded.

[0065] If such scores should not be notified, it is possible to prevent recording of such scores by setting geom_recording_point_num_flag to OFF. Generally, recording such scores leads to an increase in data size, so geom_recording_point_num_flag can be switched ON / OFF depending on the user's purpose.

[0066] Also, considering that there are use cases for scalable decoding that benefit from notifying the scores of each layer, document A, "[New Proposal] On interaction between implicit QTBT and Scalable lifting (ISO / IEC JTC1 / SC29 / WG11 m53497)," proposes that scalable decoding and implicit QTBT be used exclusively, since using them together can cause problems. Therefore, if gps_implicit_geom_partition_flag is ON, geom_recording_point_num_flag may be set to OFF.

[0067] The Descriptor column in Figure 4 indicates how each syntax is coded. ue(v) indicates an unsigned zeroth-order exponential-Golomb code, and u(1) indicates a 1-bit flag.

[0068] Figure 5 shows an example of the syntax configuration of GSH2012A / 2012B.

[0069] The GSH2012A / 2012B may include a syntax (gsh_geometry_parameter_set_id) for specifying the GPS2012 corresponding to the GSH2012A / 2012B.

[0070] When the value of gps_implicit_geom_partition_flag is "1" (i.e., "ON") in the GPS2012 corresponding to the GSH2012A / 2012B, the GSH2012A / 2012B may additionally include control data related to ImplicitQtBt.

[0071] For example, the control data related to ImplicitQtBt includes gsh_log2_root_nodesize_s, gsh_log2_root_nodesize_t_minus_s, gsh_log2_root_nodesize_v_minus_t, and the like shown in FIG.

[0072] Furthermore, the GSH2012A / 2012B may include a syntax (gsh_point_num_per_depth[i]) that indicates the number of points at each level of the tree to be synthesized by the tree synthesis unit 2020, as shown in FIG.

[0073] gsh_point_num_per_depth[i] may be specified to always take a value greater than or equal to 0. gsh_point_num_per_depth[i] may be coded using, for example, an unsigned zeroth-order exponential-Golomb code, or coded using a pre-specified number of bits.

[0074] Furthermore, since there is one node at the top level of the tree structure, and the score of the node at the bottom level of the tree structure is calculated by subtracting the total score from the sum of the scores of the nodes other than the bottom level, the scores of the node at the top level and the node at the bottom level are not included in gsh_point_num_per_depth[i] and may be calculated by the tree synthesis unit 2020 after decoding the geometric information, as they are calculable.

[0075] Furthermore, the score of each layer in the tree structure may be recorded as a difference value from the score of the layer saved immediately before. In this case, since the difference value may be a negative value, the score may be recorded as a signed Golomb code se(v).

[0076] Furthermore, the score recorded here may be an approximate value rather than an exact score, in order to reduce the amount of information. As a result, there is no need to write the exact score, and the score information can be written with a small amount of information. However, since there will be an error with the actual score, there is a possibility that the score will exceed the specified score, as described below in terms of "decrypting so as to fit within the specified score."

[0077] In addition, since it is considered that there are few applications for decoding extremely low-resolution points, when geom_recording_point_num_flag is ON (for example, when the value of geom_recording_point_num_flag is "1"), the geometric information decoding unit 2010 may skip the mth layer from the top layer without recording points.

[0078] In this case, when geom_recording_point_num_flag is ON, the geometric information decoding unit 2010 may be configured to skip and not record the points (or point difference) of the first m layers based on m specified as syntax, and to record the points (or point difference) from the m+1th layer onwards in gsh_point_num_per_depth[i].

[0079] For example, when m=5, the geometric information decoding unit 2010 may be configured not to record the scores (or the differences in scores) for the first to fifth layers in gsh_point_num_per_depth[i], but to record the scores (or the differences in scores) for the sixth layer and beyond in order starting from gsh_point_num_per_depth[0].

[0080] An example of a syntax configuration in such a case is shown in Fig. 6. In Fig. 6, m is recorded as a syntax named gsh_recording_start_layer.

[0081] In the example of Figure 6, gsh_recording_start_layer is shown as an unsigned zeroth-order exponential Golomb code, but considering that the number of layers is unlikely to become extremely large, it may be recorded as a fixed-length s-bit Descriptor.

[0082] It should be noted that the part that records the scores for each layer does not necessarily have to be GSH2012A / 2012B; for example, if it is guaranteed that there is one slice, the scores for each layer may be recorded in GPS2011.

[0083] In the case of scalable decoding, since the LoD structure formed by the LoD calculation unit 2090 and each layer of the Octree structure are identical, the score of each such layer may be recorded as the score of the LoD structure in the ASH (Attribute Slice Header) or APS described below.

[0084] (Tree Synthesis Department 2020) The control data decoded by the geometric information decoding unit 2010 will be described with reference to FIGS.

[0085] The tree synthesis unit 2020 is configured to obtain the position of points indicating in which regions of the decoding target space points exist by decoding the tree structure with the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating in which node of the tree structure described later the point cloud exists as inputs.

[0086] The tree synthesis unit 2020 is configured to obtain the position of such points by recursively repeating the division of the decoding target space defined as a cube into finer 2×2×2 cubes. At this time, the tree synthesis unit 2020 sequentially calculates in which regions of the 2×2×2 the nodes are formed by referring to an 8-bit occupancy code for each node.

[0087] Here, as shown in FIG. 15, when implementing the scalable decoding function, according to Non-Patent Document 1, a parameter (SkipOctreeLayers) indicating how many layers from the bottom of the Octree structure are skipped is given from outside the point cloud decoding device 200. As shown in FIG. 7, based on SkipOctreeLayers, it is determined up to which upper layer decoding is performed.

[0088] As a result, although the resolution of the point cloud decoded by the point cloud decoding device 20 based on SkipOctreeLayers can be determined in a scalable manner, as shown in FIG. 8, it is not possible to know the number of points when decoding up to the next layer (number of points C in FIG. 8).

[0089] Therefore, for example, when it is desired to perform scalable decoding by stopping the process so that the number of point clouds after decoding is S or less, if it is T points (T < S) at the time of "number of points 1 + number of points A + number of points B" in FIG. 8, without decoding the layer of number of points C, it cannot be determined whether including the layer of number of points C will exceed S points or not including the layer of number of points C will not exceed S points.

[0090] However, decoding the layer of number of points C will result in waste of computational resources as such.

[0091] Therefore, in this embodiment, the geometric information decoding unit 2010 notifies the number of points in each layer, and by knowing the number of points in the next layer before decoding the next layer, it is possible to perform decoding processing so that the number of points is equal to or less than S points without performing decoding. This is not limited to setting the number of points as a threshold, and can be similarly thought of when specifying a ratio, such as limiting decoding to less than 50% of the total number of points.

[0092] When decoding such an octree structure, the techniques described in Non-Patent Document 1 and Non-Patent Document 2 introduce DCM (Direct Coding Mode). DCM is a tool that improves compression efficiency when there are only a few nodes linked under a certain node, such as one or two points, by directly encoding the positions where the points exist and decoding them using a point cloud decoding device 200, rather than writing occupancy codes. In particular, Non-Patent Document 1 and Non-Patent Document 2 introduce Inferred DCM (IDCM), which implicitly determines whether to perform DCM based on surrounding nodes.

[0093] (Attribute information decoding unit 2060) The control data decoded by the attribute information decoding unit 2060 will be described below with reference to FIGS.

[0094] FIG. 9 shows an example of the structure of coded data (bit stream) received by the geometric information decoding unit 2060.

[0095] First, the bitstream may include APS2061. APS2061 is an abbreviation for Attribute Parameter Set, and is a set of control data related to the decoding of attribute information. A specific example will be described later.

[0096] The attribute information may include color information of the point cloud as well as reflectance information of the point cloud, and multiple APS2061 may be prepared for each type of attribute. Each APS2061 includes at least APS ID information for identifying each APS2061 when multiple APS2061 exist.

[0097] Second, the bitstream may include ASH2062A / 2062B. ASH2062A / 2062B stands for Attribute Slice Header and contains control data corresponding to each slice. Specific examples will be described later. ASH2062A / 2062B includes at least APS ID information for specifying the APS2061 corresponding to each ASH2062A / 2062B.

[0098] Third, the bitstream may include slice data 2063A / 2063B following ASH 2062A / 2062B. The slice data 2063A / 2063B includes data in which attribute information is coded.

[0099] As described above, the bitstream is configured such that each slice data 2063A / 2063B corresponds to one ASH 2062A / 2062B and one APS 2061.

[0100] As described above, in ASH2062A / 2062B, which APS2061 to refer to is specified by APS id information, so a common APS2061 can be used for a plurality of slice data 2063A / 2063B.

[0101] 9 is merely an example. As long as the ASH 2062A / 2062B and the APS 2061 correspond to each slice data 2063A / 2063B, elements other than those described above may be added as components of the bitstream. For example, the bitstream may include a sequence parameter set (SPS).

[0102] Similarly, when transmitted, the data may be reshaped into a configuration different from that shown in FIG. 9. Furthermore, the data may be combined with the bit stream decoded by the geometric information decoding unit 2010 and transmitted as a single bit stream. For example, slice data 2013A and 2063A, and slice data 2013B and 2063B may each be treated as a single slice data, with GSH2012A and ASH2062A, and GSH2012B and ASH2062B arranged immediately before each slice, respectively. Furthermore, in this case, GPS2011 and APS2061 may be arranged before each GSH and ASH.

[0103] Figure 10 shows an example of the syntax configuration of APS2061.

[0104] The APS 2061 may include APS id information (aps_attr_parameter_set_id) for identifying each APS 2061.

[0105] The APS 2061 may include information (attr_coding_type) indicating a decoding method for the attribute information. For example, it may be specified that when the value of attr_coding_type is "1", the inverse lifting unit 2100 performs variable weighted lifting prediction, when the value of attr_coding_type is "0", the RAHT unit 2080 performs RAHT, and when the value of attr_coding_type is "2", the inverse lifting unit 2100 performs lifting prediction with a fixed weight.

[0106] When the value of attr_coding_type is "2", that is, when lifting prediction with fixed weights is performed in the inverse lifting unit 2100, APS2061 may include a flag (lifting_scalability_enabled_flag) indicating whether to apply scalable lifting (a lifting method for scalable decoding disclosed in non-patent document 3).

[0107] In this embodiment, when lifting_scalability_enabled_flag is "0", scalable lifting is not performed, and when lifting_scalability_enabled_flag is "1", scalable lifting is performed.

[0108] In this embodiment, the purpose of specifying the number of points in each layer of the Octree structure in the syntax is to know the number of points in the n+1th layer that have not been decoded when this scalable decoding is performed. For this reason, when scalable lifting is not used (when lifting_scalability_enabled_flag=0), geom_recording_point_num_flag may be specified to be always set to "0".

[0109] (LoD calculation unit 2090) An example of the processing performed by the LoD calculation unit 2090 will be described below with reference to FIGS.

[0110] The LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 as an input and generate the LoD.

[0111] Methods for generating an LoD structure are mentioned in Non-Patent Document 1 and Non-Patent Document 2, but when performing scalable decoding as shown in Fig. 15, the number of points in each layer of the LoD structure needs to match the number of points in each layer of the Octree structure. This is to ensure that when performing scalable decoding, the number of points matches for the Geometry (Octree) structure and the Attribute (LoD) structure, no matter how many layers are skipped, as shown in Fig. 11.

[0112] In order to achieve a match in the number of points between the Octree structure and the LoD structure, in scalable lifting, the LoD calculation unit 2090 is configured to generate an LoD based on the Octree structure.

[0113] For reference, the method for generating the LoD structure during encoding will be described below. Specifically, as shown in FIG. 12, the point cloud obtained by the geometric information reconstruction unit 2040 is placed at the bottom layer of the LoD structure (here, "bottom" refers to the direction in which points are dense / the downward direction of the pyramid), and nodes at positions with the same parent node are grouped together, from which one point is selected as a representative for the LoD in the upper layer.

[0114] The points that are not selected are left in that layer. By repeating this process, the number of points selected in the upper layer will match the number of points in the layer at the same depth in the Octree structure. When the point cloud decoding device 200 performs scalable decoding, LoDs are generated in order from the middle layer toward the top in Figure 12. Although quantization errors occur in terms of position, the LoD structure itself, which determines which points to raise to the top to construct the LoD, can be constructed in the same way even when decoding from the middle layer.

[0115] When forming such an LoD structure, the method of selecting the points to be selected as the top may be as described in Non-Patent Documents 1 and 2, in which the points with the smallest / largest Morton codes are selected based on the order of the Morton codes. Alternatively, as described in Document B, "[G-PCC] CE13.15 report on LoD generation with distance from centroid for spatial scalability (ISO / IEC JTC1 / SC29 / WG11 m53288)," the method may be used in which the centroid is calculated within a group belonging to the same parent node and the point closest to the centroid is selected.

[0116] Furthermore, as mentioned above, when generating LoD based on the center of gravity, if there are two points, the center of gravity is always located midway between the two points, making it difficult to select a point close to the center of gravity. In such cases, it is possible to select one of the points depending on the order of the Morton code, but this does not necessarily mean that the optimal point can be selected.

[0117] Therefore, when performing scalable decoding, the tree synthesis unit 2020 may decode up to a layer one level deeper than the layer specified by SkipOctreeLayers, and if the center of gravity is in an intermediate position, refine the generation of LoDs by selecting the point with the larger number of points linked to each point one level below as the higher LoD.

[0118] Alternatively, a method of selecting a point closest to the center of gravity of a point two layers below, or a method of selecting a node closest to a center of gravity calculated by weighting the center of gravity of the point one layer below and the center of gravity of the point two layers below, may be adopted. Furthermore, the tree synthesis unit 2020 may decode not only the data one layer below but also up to m layers below and refine the generation of LoD based on the geometric information.

[0119] Furthermore, the above-described point group encoding device 100 and point group decoding device 200 may be realized as a program that causes a computer to execute each function (each step).

[0120] In each of the above embodiments, the present invention has been described using the application of the point cloud encoding device 100 and the point cloud decoding device 200 as an example, but the present invention is not limited to such an example and can be similarly applied to a point cloud encoding / decoding system having the functions of the point cloud encoding device 100 and the point cloud decoding device 200. [Explanation of symbols]

[0121] 10...Point cloud processing system 100...Point cloud encoding device 200...Point cloud decoding device 2010…Geometric Information Decoding Department 2020…Tree Synthesis Department 2030…Approximate surface synthesis part 2040...Geometric information reconstruction unit 2050...Inverse coordinate transformation section 2060...Attribute information decoding unit 2070...Inverse quantization section 2080…RAHT Department 2090…LoD calculation section 2100...Reverse lifting section 2110...Color inverse converter

Claims

1. A point cloud decoding device, comprising: A point cloud decoding device characterized by comprising a geometric information decoding unit configured to decode a flag that controls whether to decode the number of points in each layer of an octree structure or the difference between the number of points in each layer of the octree structure, and to decode the number of points in each layer or the difference between the number of points in each layer only when the flag is ON.

2. A point cloud decoding device, A point cloud decoding device characterized by comprising a geometric information decoding unit configured to decode the number of points in each layer of the tree structure or the difference between the number of points in each layer of the tree structure only when the geometric information of the point cloud data to be decoded is encoded in a tree structure.

3. decoding a flag that controls whether to decode the score of each layer of the octree structure or the difference between the scores of each layer of the octree structure; and decoding the points of each layer or the difference between the points of each layer only when the flag is ON.

4. A point cloud decoding method characterized by having a step of decoding the number of points in each layer of the tree structure or the difference between the number of points in each layer of the tree structure only when the geometric information of the point cloud data to be decoded is encoded in a tree structure.

5. A program for use in a point group decoding device, the program being installed on a computer: decoding a flag that controls whether to decode the score of each layer of the octree structure or the difference between the scores of each layer of the octree structure; and decoding the score of each layer or the difference between the scores of each layer only when the flag is ON.

6. A program for use in a point cloud decoding device, comprising: A program characterized by executing a step of decoding the number of points in each layer of the tree structure or the difference between the number of points in each layer of the tree structure only when the geometric information of the point cloud data to be decoded is encoded in a tree structure.

Images