Point group decoding device, point group decoding method and program

Abstract

To improve coding efficiency.SOLUTION: A point group decoding device 200 includes: a geometric information decoding unit 2010 for decoding parameters relating to a sub-sampling interval; and an approximate surface synthesis unit 2030 for generating a reconstructed point group of a target slice or node, determining a sub-sampling interval on the basis of parameters relating to the sub-sampling interval, and sub-sampling the reconstructed point group of the target slice or node on the basis of the sub-sampling interval.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a point group decoding device, a point group decoding method, and a program. [Background technology]

[0002] Non-Patent Document 1 discloses a geometry information encoding technique called Trisoup. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] G-PCC Future Enhancement, ISO / IEC JTC1 / SC29 / WG11 N19328 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the method of Non-Patent Document 1 has a problem in that Trisoup can only be executed with a fixed node size for each slice.

[0005] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a point cloud decoding device, a point cloud decoding method, and a program that, in Trisoup with multiple node sizes, decodes the Trisoup node size for each region, so that Trisoup nodes of the same node size are locally adjacent, making it easier to utilize correlation in the spatial direction and improving coding efficiency. [Means for solving the problem]

[0006] A first feature of the present invention is a point cloud decoding device comprising: a geometric information decoding unit that decodes parameters related to a subsampling interval; and an approximate surface synthesis unit, wherein the approximate surface synthesis unit generates a reconstructed point cloud of a target slice or node, determines a subsampling interval based on the parameters related to the subsampling interval, and subsamples the reconstructed point cloud of the slice or node based on the subsampling interval.

[0007] A second feature of the present invention is summarized as a point cloud decoding method comprising the steps of: decoding parameters related to a subsampling interval; generating a reconstructed point cloud of a target slice or node; determining a subsampling interval based on the parameters related to the subsampling interval; and subsampling the reconstructed point cloud of the slice or node based on the subsampling interval.

[0008] A third feature of the present invention is a program that causes a computer to function as a point cloud decoding device, the point cloud decoding device comprising: a geometric information decoding unit that decodes parameters related to a subsampling interval; and an approximate surface synthesis unit, wherein the approximate surface synthesis unit generates a reconstructed point cloud of a target slice or node, determines a subsampling interval based on the parameters related to the subsampling interval, and subsamples the reconstructed point cloud of the slice or node based on the subsampling interval. [Effects of the Invention]

[0009] According to the present invention, in Trisoup with multiple node sizes, by decoding the Trisoup node size for each region, Trisoup nodes of the same node size are locally adjacent, making it easier to utilize correlation in the spatial direction, and it is possible to provide a point cloud decoding device, a point cloud decoding method, and a program that can improve coding efficiency. [Brief explanation of the drawings]

[0010] [Figure 1]FIG. 1 is a diagram showing an example of the configuration of a point cloud processing system 10 according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of functional blocks of a point group decoding device 200 according to an embodiment. [Figure 3] FIG. 3 is a diagram showing an example of the configuration of coded data (bit stream) received by the geometric information decoding unit 2010 of the point cloud decoding device 200 according to an embodiment. [Figure 4] FIG. 4 is a diagram showing an example of the syntax configuration of GPS2011. [Figure 5] FIG. 5 is a diagram showing an example of the syntax configuration of GSH2012. [Figure 6] FIG. 6 is a diagram showing an example of the syntax configuration of GSH2012. [Figure 7] FIG. 7 is a flowchart showing an example of processing in the tree synthesis unit 2020 of the point group decoding device 200 according to an embodiment. [Figure 8] FIG. 8 is a flowchart showing an example of processing by the approximate surface synthesis unit 2030 of the point cloud decoding device 200 according to an embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the process of step S703 shown in FIG. [Figure 10] FIG. 10 is a flowchart showing an example of the process of step S704 shown in FIG. [Figure 11] FIG. 11 is a flowchart showing a specific example of a method for generating mask information. [Figure 12] FIG. 12 is a diagram illustrating an example of generating segments and mask values. [Figure 13] FIG. 13 is a flowchart showing an example of the process of step S705 shown in FIG. [Figure 14] FIG. 14 is a flowchart showing an example of the process of step S706 shown in FIG. [Figure 15] FIG. 15 is a flowchart showing an example of the process of step S708 shown in FIG. [Figure 16A]FIG. 16A is a diagram for explaining an example of the process in step S705 in FIG. [Figure 16B] FIG. 16B is a diagram for explaining an example of the process in step S706 in FIG. [Figure 16C] FIG. 16C is a diagram for explaining an example of the process in step S708 in FIG. [Figure 17] FIG. 17 is a flowchart showing an example of the process of step S709 shown in FIG. [Figure 18] FIG. 18 is a flowchart showing an example of the process of step S706 shown in FIG. [Figure 19] FIG. 19 is a flowchart showing an example of processing by the approximate surface synthesis unit 2030 of the point cloud decoding device 200 according to an embodiment. [Figure 20] FIG. 20 is a diagram for explaining an example of processing by the approximate surface synthesis unit 2030 of the point cloud decoding device 200 according to an embodiment. [Figure 21] FIG. 21 is a diagram showing an example of functional blocks of the point group encoding device 100 according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the components in the following embodiments can be appropriately replaced with existing components, etc., and various variations, including combinations with other existing components, are possible. Therefore, the description of the following embodiments does not limit the content of the invention described in the claims.

[0012] (First embodiment) A point cloud processing system 10 according to a first embodiment of the present invention will be described below with reference to Figures 1 to 21. Figure 1 is a diagram showing a point cloud processing system 10 according to the present embodiment.

[0013] As shown in FIG. 1, the point cloud processing system 10 includes a point cloud encoding device 100 and a point cloud decoding device 200.

[0014] The point cloud encoding device 100 is configured to generate encoded data (bitstream) by encoding an input point cloud signal, and the point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bitstream.

[0015] The input point cloud signal and the output point cloud signal are composed of position information and attribute information of each point in the point cloud, such as color information and reflectance of each point.

[0016] Here, the bit stream may be transmitted from the point group encoding device 100 to the point group decoding device 200 via a transmission path. Alternatively, the bit stream may be stored in a storage medium and then provided from the point group encoding device 100 to the point group decoding device 200.

[0017] (Point Cloud Decoding Device 200) The point group decoding device 200 according to this embodiment will be described below with reference to Fig. 2. Fig. 2 is a diagram showing an example of functional blocks of the point group decoding device 200 according to this embodiment.

[0018] As shown in Figure 2, the point cloud decoding device 200 has a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, an LoD calculation unit 2090, an inverse lifting unit 2100, and an inverse color transformation unit 2110.

[0019] The geometric information decoding unit 2010 is configured to receive as input a bit stream relating to geometric information (geometric information bit stream) from among the bit streams output from the point group encoding device 100, and to decode the syntax.

[0020] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0021] The tree synthesis unit 2020 is configured to take as input the control data decoded by the geometric information decoding unit 2010 and an occupancy code indicating at which node in the tree described below the point group exists, and generate tree information indicating in which area in the space to be decoded the point exists.

[0022] The tree synthesis unit 2020 may be configured to perform decoding of the occupancy code within itself.

[0023] This process divides the space to be decoded into rectangular prisms, determines whether a point exists within each rectangular prism by referencing the occupancy code, divides the rectangular prism in which the point exists into multiple rectangular prisms, and then generates tree information by recursively repeating the process of referencing the occupancy code.

[0024] Here, inter prediction may be used when decoding such occupancy code.

[0025] In this embodiment, a method called "Octree" can be used, which recursively divides the above-mentioned rectangular parallelepiped into octries by treating it as a cube, and a method called "QtBt" can be used, which divides the rectangular parallelepiped into quadtrees and binary trees in addition to the octtree division. Whether or not to use "QtBt" is transmitted as control data from the point cloud encoding device 100.

[0026] In this embodiment, the method of decoding geometric information by dividing the above-mentioned cube by "Octree" until it becomes 1x1x1 in size is specifically called "Octree only."

[0027] Also, even when "Octree" is used in combination with "QtBt", the method of decoding geometric information by dividing the above-mentioned rectangular parallelepiped using only "Octree" and "QtBt" until it becomes 1x1x1 in size may also be called "Octree only".

[0028] Alternatively, when the control data specifies that predictive coding is to be used, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree structure determined by the point cloud encoding device 100.

[0029] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using the tree information generated by the tree synthesis unit 2020, and to decode the point cloud based on the approximate surface information.

[0030] For example, when decoding three-dimensional point cloud data of an object, if the point cloud is densely distributed on the surface of the object, approximate surface information is used to represent the area where the point cloud exists by approximating it with a small plane, rather than decoding each individual point cloud.

[0031] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information and decode the point cloud using a method called "Trisoup," for example. A specific processing example of "Trisoup" will be described later. Furthermore, when decoding a sparse point cloud acquired by Lidar or the like, this processing can be omitted.

[0032] The geometric information reconstruction unit 2040 is configured to reconstruct the geometric information (position information in the coordinate system assumed by the decoding process) of each point of the point cloud data to be decoded based on the tree information generated by the tree synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0033] The inverse coordinate transformation unit 2050 is configured to receive the geometric information reconstructed by the geometric information reconstruction unit 2040 as input, transform it from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal, and output position information.

[0034] The attribute information decoding unit 2060 is configured to receive as input a bit stream relating to attribute information (attribute information bit stream) from among the bit streams output from the point group encoding device 100, and to decode the syntax.

[0035] The decoding process is, for example, a context-adaptive binary arithmetic decoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0036] Moreover, the attribute information decoding unit 2060 is configured to decode the quantized residual information from the decoded syntax.

[0037] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantized residual information decoded by the attribute information decoding unit 2060 and the quantization parameter, which is one of the control data decoded by the attribute information decoding unit 2060, to generate inverse quantized residual information.

[0038] The dequantized residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090, depending on the features of the point group to be decoded. The control data decoded by the attribute information decoding unit 2060 specifies which unit the information is to be output to.

[0039] The RAHT unit 2080 is configured to receive as input the inverse-quantized residual information generated by the inverse quantization unit 2070 and the geometric information generated by the geometric information reconstruction unit 2040, and to decode the attribute information of each point using a type of Haar transform (inverse Haar transform in the decoding process) called RAHT (Region Adaptive Hierarchical Transform). As a specific process of RAHT, for example, the method described in Non-Patent Document 1 can be used.

[0040] The LoD calculation unit 2090 is configured to receive the geometric information generated by the geometric information reconstruction unit 2040 as an input and generate an LoD (Level of Detail).

[0041] LoD is information for defining the reference relationship (reference point and referenced point) to realize predictive coding, such as predicting attribute information of another point from attribute information of another point and encoding or decoding the prediction residual.

[0042] In other words, LoD is information that defines a hierarchical structure in which points contained in geometric information are classified into multiple levels, and the attributes of points belonging to lower levels are encoded or decoded using the attribute information of points belonging to higher levels.

[0043] As a specific method for determining the LoD, for example, the method described in Non-Patent Document 1 mentioned above may be used.

[0044] The inverse lifting unit 2100 is configured to decode attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse-quantized residual information generated by the inverse quantization unit 2070. As a specific process of inverse lifting, for example, the method described in the above-mentioned Non-Patent Document 1 can be used.

[0045] The inverse color conversion unit 2110 is configured to perform inverse color conversion processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100, when the attribute information to be decoded is color information and color conversion has been performed on the point group encoding device 100 side. Whether or not to perform such inverse color conversion processing is determined by the control data decoded by the attribute information decoding unit 2060.

[0046] The point cloud decoding device 200 is configured to decode and output attribute information of each point in the point cloud through the above processing.

[0047] (Geometric Information Decoding Unit 2010) The control data decoded by the geometric information decoding unit 2010 will be described below with reference to FIGS.

[0048] FIG. 3 shows an example of the structure of coded data (bit stream) received by the geometric information decoding unit 2010. In FIG.

[0049] First, the bitstream may include a GPS2011. The GPS2011 is also called a geometry parameter set and is a set of control data related to decoding of geometric information. Specific examples will be described later. Each GPS2011 includes at least GPS ID information for identifying each GPS2011 when there are multiple GPS2011s.

[0050] Second, the bitstream may include GSH2012A / 2012B. GSH2012A / 2012B is also called a geometry slice header or geometry data unit header, and is a collection of control data corresponding to a slice, which will be described later. Although the term "slice" will be used in the following description, slice can also be interpreted as a data unit. Specific examples will be described later. GSH2012A / 2012B includes at least GPS ID information for specifying the GPS2011 corresponding to each GSH2012A / 2012B.

[0051] Third, the bitstream may include slice data 2013A / 2013B following the GSH 2012A / 2012B. The slice data 2013A / 2013B includes data that encodes geometric information. An example of the slice data 2013A / 2013B is an occupancy code, which will be described later.

[0052] As described above, the bit stream is configured such that each slice data 2013A / 2013B corresponds to one GSH 2012A / 2012B and one GPS 2011.

[0053] As described above, in GSH2012A / 2012B, which GPS2011 to refer to is specified by GPS ID information, so a common GPS2011 can be used for multiple slice data 2013A / 2013B.

[0054] In other words, it is not necessary to transmit GPS 2011 for each slice. For example, as shown in Fig. 3, the bitstream may be configured such that GPS 2011 is not coded immediately before GSH 2012B and slice data 2013B.

[0055] 3 is merely an example. As long as GSH 2012A / 2012B and GPS 2011 correspond to each slice data 2013A / 2013B, elements other than those described above may be added as components of the bit stream.

[0056] For example, as shown in Fig. 3, the bitstream may include a sequence parameter set (SPS) 2001. Similarly, when transmitted, the bitstream may be shaped into a configuration different from that shown in Fig. 3. Furthermore, the bitstream may be combined with a bitstream decoded by an attribute information decoding unit 2060 (described later) and transmitted as a single bitstream.

[0057] Figure 4 shows an example of the syntax configuration of GPS2011.

[0058] Note that the syntax names explained below are merely examples. If the syntax functions explained below are similar, the syntax names may be different.

[0059] The GPS 2011 may include GPS ID information (gps_geom_parameter_set_id) for identifying each GPS 2011.

[0060] The Descriptor column in Figure 4 indicates how each syntax is coded. ue(v) indicates an unsigned zeroth-order exponential Golomb code, and u(1) indicates a 1-bit flag.

[0061] The GPS 2011 may include a flag (trisoup_enabled_flag) that controls whether or not Trisoup is used in the approximate surface synthesis unit 2030 .

[0062] For example, it may be defined that Trisoup is not used when the value of trisoup_enabled_flag is "0", and that Trisoup is used when the value of trisoup_enabled_flag is "1".

[0063] The geometric information decoding unit 2020 may be configured to additionally decode the following syntax when Trisoup is used, that is, when the value of trisoup_enabled_flag is “1”.

[0064] Note that trisoup_enabled_flag may be included in SPS2001 instead of GPS2011.

[0065] GPS2011 may include a flag (trisoup_multilevel_enabled_flag, first flag) that controls whether or not Trisoup is allowed on multiple levels.

[0066] For example, when the value of trisoup_multilevel_enabled_flag is "0", multi-level trisoup is not allowed, i.e., single-level trisoup is performed, and when the value of trisoup_multilevel_enabled_flag is "1", multi-level trisoup is allowed.

[0067] If the syntax is not included in GPS2011, the value of the syntax may be regarded as the value for performing Trisoup at a single level, i.e., "0".

[0068] Note that trisoup_multilevel_enabled_flag may be defined to be included in SPS2001 instead of GPS2011. In this case, if trisoup_multilevel_enabled_flag is not included in SPS2001, the value of the syntax may be considered to be the value for performing trisoup at a single level, i.e., "0".

[0069] 5 and 6 show an example of the syntax configuration of GSH 2012. As mentioned above, GSH is also called GDUH (Geometry Data Unit Header).

[0070] The geometric information decoding unit 2020 may be configured to additionally decode the following syntax when multi-level Trisoup is permitted, that is, when the value of trisoup_multilevel_enabled_flag is "1".

[0071] When GSH2012 allows multi-level trisoups, it may include syntax (log2_trisoup_max_node_size_minus2) to specify the maximum trisoup node size.

[0072] The syntax may be expressed as a value obtained by converting the maximum actual Trisoup node size into a logarithm with base 2. Furthermore, the syntax may be expressed as a value obtained by subtracting 2 from the logarithm with base 2 with respect to the maximum actual Trisoup node size.

[0073] GSH2012 may include syntax (log2_trisoup_min_node_size_minus2) to specify a minimum Trisoup node size when allowing multi-level Trisoups.

[0074] The syntax may be expressed as a value obtained by converting the actual minimum Trisoup node size to a logarithm with base 2. Furthermore, the syntax may be expressed as a value obtained by converting the actual minimum Trisoup node size to 4 (=2 2 ), converted to a logarithm with base 2 and then expressed as the value after subtracting 2.

[0075] In addition, the value of this syntax may be restricted to be greater than or equal to 0 and less than or equal to log2_trisoup_max_node_size_minus2.

[0076] In this case, trisoup_depth may be defined as trisoup_depth=log2_trisoup_max_node_size_minus2−log2_trisoup_min_node_size_minus2+1 as shown in FIG.

[0077] Furthermore, instead of directly decoding the minimum Trisoup node size and the maximum Trisoup node size, depth values ​​corresponding to the maximum Trisoup node size and the minimum Trisoup node size in Octree processing, which will be described later, may be decoded.

[0078] For example, if the maximum depth (the depth at which all nodes are 1x1x1 in size) is 10, the minimum Trisoup node size is 4 (=22 ), the maximum Trisoup node size is 16 (=2 4 ), you can decode a Depth value of 8 corresponding to the minimum Trisoup node size and a Depth value of 6 corresponding to the maximum Trisoup node size.

[0079] When GSH2012 allows multi-level trisoup, it may include a syntax (log2_trisoup_ctu_size_minus2) that specifies the size of a node (hereinafter referred to as CTU) that decodes the trisoup node size.

[0080] Such syntax may be expressed as a base 2 logarithmic value.

[0081] Furthermore, this syntax limits the actual Trisoup node size to a minimum of 4 (=2 2 ), converted to a logarithm with base 2 and then expressed as the value after subtracting 2.

[0082] Furthermore, the value of such syntax may be restricted to be equal to or greater than the maximum Trisoup node size.

[0083] The value of such syntax may also be expressed as the value obtained by converting the CTU size to its logarithm in base 2 minus the value obtained by converting the maximum Trisoup node size to its logarithm in base 2.

[0084] The geometric information decoding unit 2020 may be configured to additionally decode the following syntax when multi-level Trisoup is not permitted, that is, when the value of trisoup_multilevel_enabled_flag is "0".

[0085] As described above, the geometric information decoding unit 2020 in this embodiment may be configured to decode the maximum Trisoup node size, which is the maximum value of the node size to which the above-mentioned Trisoup is applied, and the minimum Trisoup node size, which is the minimum value of the node size to which Trisoup is applied.

[0086] As described above, the geometric information decoding unit 2020 in this embodiment may be configured to decode the predetermined size (CTU size) as a value equal to or larger than the maximum Trisoup node size.

[0087] With this configuration, it becomes possible to select a node size between the maximum Trisoup node size and the minimum Trisoup node size for each region (CTU) of a predetermined size.

[0088] As described above, the geometric information decoding unit 2020 in this embodiment may be configured to set the above-mentioned maximum Trisoup node size to a predetermined size value.

[0089] With this configuration, it is possible to omit decoding of the CTU size, thereby reducing the amount of coding and the amount of processing.

[0090] GSH2012 may include syntax (log2_trisoup_node_sizeTrisoup_node_size_minus2) to specify the Trisoup node size when multi-level Trisoup is not allowed and Trisoup is used.

[0091] Such syntax may be expressed as a base 2 logarithm of the actual Trisoup node size.

[0092] Furthermore, such syntax may be expressed as a base 2 logarithm of the actual Trisoup node size, followed by subtracting 2.

[0093] In this case, trisoup_depth may be defined as trisoup_depth=1, as shown in FIG.

[0094] GSH2012 may include syntax (trisoup_sampling_value_minus1) to control the subsampling interval of the reconstruction points when using Trisoup.

[0095] Alternatively, instead of such syntax, a threshold (trisoup_sampling_threshold) indicating the maximum number of points after subsampling may be decoded as shown in Fig. 6. A specific subsampling method based on these syntax values ​​will be described later.

[0096] GSH2012 may include a syntax (trisoup_vertex_number_bits) that specifies the precision (number of bits) of the vertex position of Trisoup (described later). For example, if the value of this syntax is 2, it means that the vertex position is 2 bits, that is, it can take on four values: 0, 1, 2, and 3.

[0097] Here, as described above, trisoup_vertex_number_bits may always transmit only one value regardless of the value of trisoup_depth, or the number of bits to be decoded may be changed depending on the value of trisoup_depth.

[0098] For example, trisoup_vertex_number_bits corresponding to each trisoup_depth may be decoded. In other words, the same number of trisoup_vertex_number_bits as the trisoup_depth may be decoded. Specifically, for example, when trisoup_depth is 2, two types of values ​​of trisoup_vertex_number_bits may be transmitted.

[0099] GSH2012 may include a flag (trisoup_centroid_vertex_residual_flag) indicating whether to decode the centroid residual of Trisoup vertices (described later). For example, the flag may be defined so that a value of 1 indicates that the centroid residual is decoded, and a value of 0 indicates that the centroid residual is not decoded.

[0100] Here, as described above, trisoup_centroid_vertex_residual_flag may always transmit only one value regardless of the value of trisoup_depth, or the number of bits to be decoded may be changed depending on the value of trisoup_depth.

[0101] For example, trisoup_centroid_vertex_residual_flag corresponding to each trisoup_depth may be decoded. In other words, the same number of trisoup_centroid_vertex_residual_flag as the trisoup_depth may be decoded. Specifically, for example, when trisoup_depth is 2, two types of values ​​of trisoup_centroid_vertex_residual_flag may be transmitted.

[0102] When using Trisoup and allowing Trisoup at multiple levels, GSH2012 may include a flag (unique_segments_exist_flag[i]) for each hierarchy i (i=0,...,trisoup_depth-1) indicating whether unique segments exist in the target hierarchy.

[0103] For example, if the value of unique_segments_exist_flag[i] is "1", it means that there is at least one unique segment in tier i. If the value of unique_segments_exist_flag[i] is "0", it means that there is no unique segment in tier i.

[0104] If a unique segment exists in the target layer for each layer i (i = 0, ..., trisoup_depth-1), i.e., if the value of unique_segments_exist_flag[i] is "1", GSH2012 may additionally include syntax (num_unique_segments_bits_minus1[i]) indicating the number of bits of the syntax indicating the number of unique segments in the target layer and syntax (num_unique_segments_minus1[i]) indicating the number of unique segments in the target layer.

[0105] Here, for both num_unique_segments_bits_minus1[i] and num_unique_segments_minus1[i], the original value minus "1" may be coded as the syntax value.

[0106] (Tree Synthesis Department 2020) The processing of the tree merging unit 2020 will be described below with reference to Fig. 7. Fig. 6 is a flowchart showing an example of processing in the tree merging unit 2020. Note that the following describes an example in which trees are merged using "Octree."

[0107] In step S601, the tree synthesis unit 2020 checks whether the processing for all depths has been completed. Note that the depth number may be included as control data in the bitstream transmitted from the point group encoding device 100 to the point group decoding device 200.

[0108] The tree synthesis unit 2020 calculates the node size of the target depth. In the case of an "Octree," the node size of the first depth may be defined as "2 to the power of the number of depths." In other words, if the number of depths is N, the node size of the first depth may be defined as 2 to the Nth power.

[0109] In addition, the node size at the second and subsequent depths may be defined by decreasing the number N by 1. That is, the node size at the second depth may be defined as "2 to the (N-1) power," the node size at the third depth may be defined as "2 to the (N-2) power," and so on.

[0110] Alternatively, since the node size is always defined as a power of 2, the value of the exponent part (N, N-1, N-2, etc.) can simply be considered as the node size. In the following explanation, the node size refers to the value of the exponent part of the length of one side of the node.

[0111] For simplicity, the following description will be given taking as an example a case where the node shape is a cube, that is, a case where all sides of the node have the same length.

[0112] When using QtBt, that is, when the node shape is a rectangular parallelepiped and the length of each side of the node varies in each axial direction (x, y, z), the length of the shortest side of the three directions can be considered the node size.Similarly, the length of the longest side of the three directions can be considered the node size.

[0113] Here, when the flag (trisoup_enabled_flag) that controls whether to use Trisoup indicates that Trisoup is to be used, that is, when the value of trisoup_enabled_flag is "1," the tree synthesis unit 2020 may change the number of depths to be processed based on the value of the syntax (log2_trisoup_min_node_size_minus2) that specifies the minimum value of the Trisoup node size or the syntax (log2_Trisoup_node_size_minus2) that specifies the Trisoup node size. In such a case, for example, it may be defined as follows:

[0114] Processing Depth = Total Depth - (Minimum) Trisoup Node Size Here, the minimum Trisoup node size can be defined as (log2_trisoup_min_node_size_minus2+2), for example. Similarly, the Trisoup node size can be defined as (log2_Trisoup_node_size_minus2+2).

[0115] In this case, if the processing for all the processing depths has been completed, the tree merging unit 2020 proceeds to step S609, and if not, the tree merging unit 2020 proceeds to step S602.

[0116] In other words, if (number of processing depths - n) = 0, the tree merging unit 2020 proceeds to step S609, and if (number of processing depths - n) > 0, the tree merging unit 2020 proceeds to step S602.

[0117] Furthermore, the tree merging unit 2020 may determine that Trisoup is applied to all nodes having the node size (N-number of processing depths) when proceeding to step S609.

[0118] In step S602, the tree synthesis unit 2020 determines whether or not it is necessary to decode Trisoup_node_size, which will be described later, at the target depth.

[0119] For example, if "Trisoup at multiple levels is allowed (the value of trisoup_multilevel_enabled_flag is "1")" and "the node size (Nn) of the target depth is CTU size," the tree synthesis unit 2020 may determine that "decoding of Trisoup_node_size is necessary."

[0120] In this embodiment, the "node that decodes the Trisoup node size" is called a CTU (Coding Tree Unit). The name CTU is merely an example, and any other name may be used as long as it is a "node that decodes the Trisoup node size."

[0121] Furthermore, if the above conditions are not met, the tree synthesis unit 2020 may determine that "decoding of Trisoup_node_size is not necessary."

[0122] Here, the maximum Trisoup node size can be defined as, for example, (log2_trisoup_max_node_size_minus2+2).

[0123] Similarly, the minimum Trisoup node size can be defined as, for example, (log2_trisoup_min_node_size_minus2+2).

[0124] Once the above-mentioned determination is complete, the tree merging unit 2020 proceeds to step S603.

[0125] In step S603, the tree merging unit 2020 determines whether or not processing of all nodes included in the target depth has been completed.

[0126] If it is determined that processing of all nodes at the target depth has been completed, the tree merging unit 2020 proceeds to step S601 and performs processing of the next depth.

[0127] On the other hand, if processing of all nodes at the target depth has not been completed, the tree merging unit 2020 proceeds to step S604.

[0128] In step S604, the tree synthesis unit 2020 checks whether or not it is necessary to decode the Trisoup_node_size determined in step S602.

[0129] Alternatively, step S602 may be omitted, and at the timing of processing step S604, it may be determined whether or not the Trisoup_node_size needs to be decoded using a method similar to that of step S602.

[0130] If it is determined that decoding of Trisoup_node_size is necessary, the tree merging unit 2020 proceeds to step S605, and if it is determined that decoding of Trisoup_node_size is not necessary, the tree merging unit 2020 proceeds to step S606.

[0131] In step S605, the tree synthesis unit 2020 decodes Trisoup_node_size.

[0132] Trisoup_node_size is information indicating the size at which Trisoup is applied to a descendant node obtained by recursively dividing the CTU using Octree or QtBt.

[0133] For example, if the CTU size is 5 (=2 5 =32) and the decrypted Trisoup_node_size is 2 (=2 2 = 4), it may mean that Trisoup is applied to each node (node ​​size 2) obtained when the CTU is divided three times (= 5 - 2) by an Octree.

[0134] The node size to which the decrypted Trisoup is applied (hereinafter referred to as Trisoup node size) is stored as additional information of the node. Note that the initial value of the Trisoup node size is 0 (=2 0 =1) and overwrite it with the decrypted value in step S605.

[0135] After decoding Trisoup_node_size, the tree synthesis unit 2020 proceeds to step S606.

[0136] In step S606, the tree merging unit 2020 checks the value of the Trisoup node size stored as internal information of the node.

[0137] If Trisoup is to be applied to the target node, that is, if the size of the node is equal to the Trisoup node size stored as internal information of the node, the tree merging unit 2020 proceeds to step S607.

[0138] If Trisoup is not applied to the target node, that is, if the size of the node differs from the Trisoup node size stored as internal information of the node, the tree merging unit 2020 proceeds to step S608.

[0139] In step S607, the tree merging unit 2020 stores the target node as a node to which Trisoup is applied, i.e., as a Trisoup node. No further node division using "Octree" is applied to this target node. The tree merging unit 2020 then proceeds to step S603 to process the next node.

[0140] In step S608, the tree synthesis unit 2020 decodes information called an occpancy code.

[0141] In the case of an "Octree," the occpancy code is information that indicates whether the point to be decoded is contained in each of the child nodes when the target node is divided in half in each of the x, y, and z axis directions and divided into eight nodes (called child nodes).

[0142] For example, the occpancy code may assign one bit of information to each child node, and if the one bit of information is "1", it may be defined that the point to be decoded is contained within the child node, and if the one bit of information is "0", it may be defined that the point to be decoded is not contained within the child node.

[0143] When decoding such occpancy code, the tree synthesis unit 2020 may estimate in advance the probability that the point to be decoded exists in each child node, and entropy decode the bits corresponding to each child node based on that probability.

[0144] In step S608, the tree synthesis unit 2020 may store the position information of the nodes to which Trisoup has not been applied in a one-dimensional array or the like for each depth, for use in step S901 described later.

[0145] Specifically, the tree merging unit 2020 may store position information of the target node (node ​​before division) in step S608. For example, the tree merging unit 2020 may store the coordinate values ​​of the point closest to the origin among the vertices of the target node (rectangular parallelepiped) for each depth in a one-dimensional array, and this information may be used by the approximate surface merging unit 2030, which will be described later.

[0146] If the size of the rectangular prism (the length of the sides in the x-axis, y-axis, and z-axis directions) is determined for each depth, the position information of the node can be restored by identifying the coordinate values ​​and depth of the point closest to the origin within the node as described above.

[0147] Similarly, the point cloud encoding device 100 may perform entropy encoding.

[0148] In step S608, the Trisoup node size stored as internal information of the node may be stored as internal information of each child node of the node. In other words, the Trisoup node size may be inherited by the child node. This allows the value of the Trisoup node size decoded in step S605 to be propagated to the descendant nodes.

[0149] (Approximate surface synthesis section 2030) An example of the processing of the approximate surface synthesis unit 2030 will be described below with reference to FIGS.

[0150] FIG. 8 is a flowchart showing an example of the processing performed by the approximate surface synthesis unit 2030.

[0151] As shown in FIG. 8, in step S701, the approximate surface synthesis unit 2030 determines whether or not processing has been completed at all trisoup_depths.

[0152] If the processing has been completed for all trisoup_depth, the process proceeds to step S709. If the processing has not been completed for all trisoup_depth, the process proceeds to step S702.

[0153] In step S702, the approximate surface synthesis unit 2030 determines whether the Trisoup node size corresponding to the trisoup_depth is equal to the maximum Trisoup node size.

[0154] If they are equal, the operation proceeds to step S704, and if they are not equal, that is, if the Trisoup node size corresponding to the trisoup_depth is smaller than the maximum Trisoup node size, the operation proceeds to step S703.

[0155] In step S702, the approximate surface synthesis unit 2030 acquires and integrates the vertex positions for each node.

[0156] In step S703, the approximate surface synthesis unit 2030 generates segments corresponding to the "interpolated vertices (vertices on which interpolation processing has been performed)" generated in step S708, which will be described later. Fig. 9 is a flowchart showing an example of the processing in step S703.

[0157] An example of the process in step S703 will be described below with reference to FIG.

[0158] As shown in FIG. 9, in step S801, the approximate surface synthesis unit 2030 determines whether or not processing has been completed for all of the "interpolated vertices" generated in step S708, which will be described later.

[0159] If so, the operation proceeds to step S805 and ends processing. If not, the operation proceeds to step S802 to process the next "interpolated vertex."

[0160] In step S802, the approximate surface synthesis unit 2030 determines on which direction (x, y, or z direction) the "interpolated vertex" exists as a point on a segment.

[0161] For example, as described below, if the vertices of a segment and Trisoup are located at a position 0.5 away from the integer coordinate position, then in the x-direction segment, only the x-coordinate of the "interpolated vertex" will be an integer value (x.0), and the y-coordinate and z-coordinate will be decimal values ​​(x.5).

[0162] Similarly, in segments in the y direction, only the y coordinate is an integer value, and in segments in the z direction, only the z coordinate is an integer value, so the approximate surface synthesis unit 2030 can determine the direction of the segment in which the ``interpolated vertex'' should exist by checking which axis direction coordinate is an integer.

[0163] In addition, when coordinate values ​​are stored in integer format, for example, when the true coordinate value is doubled and stored as an integer value to represent 0.5, the approximate surface synthesis unit 2030 can determine whether the coordinate value in each axis direction is an integer or a decimal (x.5) based on whether the lowest bit is 1.

[0164] The approximate surface synthesis unit 2030 stores the result of this determination, and proceeds to the next step S803.

[0165] In step S803, the approximate surface synthesis unit 2030 determines whether the "interpolated vertex" exists on an edge in the current Trisoup node size. For example, the approximate surface synthesis unit 2030 can make this determination in the following manner. (1) First, the approximate surface synthesis unit 2030 quantizes the coordinate values ​​of the x, y, and z axes of the "interpolated vertices" into integers.

[0166] Specifically, the approximate surface synthesis unit 2030 adds 0.5 to each coordinate value and then rounds down the decimal points.

[0167] Here, when the coordinate values ​​are stored as integer values ​​that are twice the true values, the approximate surface synthesis unit 2030 adds 1, divides by 2, and truncates to the nearest integer.

[0168] Alternatively, if the coordinate values ​​are stored as integer values ​​that are twice the true values, the approximate surface synthesis unit 2030 adds 1 and then shifts them one bit to the right.

[0169] Note that although the case where integers are converted for all of the x, y, and z axis directions has been described here, the approximate surface synthesis unit 2030 may convert integers only for axes other than the "direction" determined in step S802.

[0170] For example, when the above-mentioned "direction" is the x-axis, the approximate surface synthesis unit 2030 only needs to convert the y-coordinate and z-coordinate into integers. (2) Secondly, the approximate surface synthesis unit 2030 checks whether the coordinate values ​​after integer conversion of the axes other than the "direction" determined in step S802 are multiples of the Trisoup node size.

[0171] For example, the approximate surface synthesis unit 2030 determines whether the above-mentioned "direction" is the x-axis direction and the value of the Trisoup node size is 8 (=2 3 ), check whether the integerized y and z coordinates are multiples of 8.

[0172] In the process (2) above, if the coordinate values ​​of all axes other than the "direction" after integer conversion are multiples of the Trisoup node size, the approximate surface synthesis unit 2030 determines that the "interpolated vertex" exists on an edge of the current Trisoup node size, and proceeds to step S804.

[0173] If this is not the case, that is, if the coordinate value in at least one axis direction is not a multiple of the Trisoup node size, the approximate surface synthesis unit 2030 determines that the "interpolated vertex" does not exist on an edge in the current Trisoup node size, and proceeds to step S801 to process the next "interpolated vertex."

[0174] In step S804, the approximate surface synthesis unit 2030 generates a segment corresponding to the "interpolated vertex" using the coordinates of the "interpolated vertex" converted into integers in step S803.

[0175] Here, a segment may be composed of three elements: coordinates (x, y, z) of the start point, coordinates (x, y, z) of the end point, and the vertex position on the segment (0 to Trisoup node size - 1).

[0176] The above-mentioned starting point can be obtained, for example, by quantizing the coordinate values ​​of the "interpolated vertex" converted into integers by the Trisoup node size.

[0177] Specifically, for example, if the logarithm of the Trisoup node size with base 2 is TrisoupNodeSizeLog2, it can be derived by performing the following calculation on the coordinate values ​​of the "interpolated vertices".

[0178] Coordinate value after quantization = (coordinate value>>TrisoupNodeSizeLog2)< <TrisoupNodeSizeLog2 Here, >> means a right bit shift, and << means a left bit shift.

[0179] When proceeding to the processing of step S804, it is clear that the coordinate values ​​of the axes other than the "direction" determined in step S802 are already multiples of the Trisoup node size, so in step S804, the approximate surface synthesis unit 2030 may perform the above-mentioned quantization processing only on the coordinate values ​​of the axis corresponding to the "direction" determined in step S802.

[0180] Next, the coordinates of the end point can be calculated by adding the value of the Trisoup node size to the coordinate value of the axis corresponding to the "direction" determined in step S802 for the coordinates of the start point.

[0181] Finally, the vertex position on the segment can be derived by subtracting the coordinate value of the starting point on the axis corresponding to the above-mentioned "direction" from the coordinate value of the "interpolated vertex" converted to an integer on the axis corresponding to the "direction" determined in step S802.

[0182] After the above processing is performed, the operation proceeds to step S801 to process the next "interpolated vertex."

[0183] As described above, after the approximate surface synthesis unit 2030 generates segments from the "interpolated vertices" in step S703 of FIG. 8, the process proceeds to step S704.

[0184] In step S704, the approximate surface synthesis unit 2030 collects neighbor information to be used for decoding the vertices in the next step S705.

[0185] 10 is a flowchart showing an example of the process of step S704. An example of the process of step S704 will be described below with reference to FIG.

[0186] As shown in Fig. 10, in step S901, the approximate surface synthesis unit 2030 generates mask information. Fig. 11 shows a specific example of a method for generating mask information.

[0187] When generating mask information, as shown in FIG. 11, the approximate surface synthesis unit 2030 determines in step S1001 whether or not processing has been completed for all Trisoup nodes in the Trisoup node size.

[0188] If completed, the operation proceeds to step S1003; if not completed, the operation proceeds to step S1002.

[0189] In step S1002, the approximate surface synthesis unit 2030 generates 36 segments and mask values ​​corresponding to each segment for the Trisoup node.

[0190] An example of generating segments and mask values ​​is shown in Fig. 12. The shaded area in Fig. 12 indicates an example of the Trisoup node, each line segment indicates an example of a segment, and the numbers written on the segments indicate an example of a mask value.

[0191] Since it is difficult to provide examples of mask values ​​for all segments, examples of mask values ​​are provided only for some segments. However, in reality, mask values ​​are set for all segments as follows:

[0192] First, the approximate surface synthesis unit 2030 generates four segments at adjacent positions with larger coordinate values ​​than the node and four segments at adjacent positions with smaller coordinate values ​​for each of the 12 sides of the Trisoup node in the x, y, and z directions, generating a total of 36 segments as shown in Figure 12.

[0193] Here, each segment has information about its start point and end point, but since there are no vertices, information about the vertex positions is not held.

[0194] Next, an example of setting the mask value for each segment will be described.

[0195] The approximate surface synthesis unit 2030 sets a mask value in which only one of the lower four bits of the mask value is 1 and the rest are 0 for the segments corresponding to the 12 sides of the Trisoup node.

[0196] For example, as shown in FIG. 12, the approximate surface synthesis unit 2030 may set mask values ​​of 1, 2, 4, and 8 for segments in the x-axis direction.

[0197] Similarly, the approximate surface synthesis unit 2030 may set mask values ​​of 1+(1<<13), 2+(1<<13), 4+(1<<13), and 8+(1<<13) for segments in the y-axis direction.

[0198] Similarly, the approximate surface synthesis unit 2030 may set mask values ​​of 1+(1<<14), 2+(1<<14), 4+(1<<14), and 8+(1<<14) for segments in the z-axis direction.

[0199] When set in this way, if any of the first to fourth bits of the mask value is 1, it can be determined that the node corresponding to the segment is a Trisoup node.

[0200] Next, the approximate surface synthesis unit 2030 may set mask values ​​of 16, 32, 64, and 128, respectively, for the segments (12 segments in total, four in each of the x, y, and z directions) corresponding to adjacent nodes whose coordinate values ​​increase from the Trisoup node in the x, y, and z directions, as shown in Figure 12.

[0201] With this setting, if any of the 5th to 8th bits of the mask value is 1, it can be determined that the adjacent node with the smaller coordinate value for the segment is the Trisoup node.

[0202] Next, the approximate surface synthesis unit 2030 may set mask values ​​of 256, 512, 1024, and 2048, respectively, for segments (12 segments in total, four in each of the x, y, and z directions) corresponding to adjacent nodes whose coordinate values ​​are smaller than the Trisoup node in the x, y, and z directions, as shown in Figure 12.

[0203] In this case, if any of the 9th to 12th bits of the mask value is 1, it can be determined that the adjacent node with a larger coordinate value for the segment is the Trisoup node.

[0204] As described above, after generating the segments and the corresponding mask values, the approximate surface synthesis unit 2030 proceeds to step S1001 to process the next Trisoup node.

[0205] Here, the nodes to which Trisoup was not applied at the depth, which were saved in step S608 above, may be added to the nodes to be processed in step S1001 as the nodes to be processed in this step.

[0206] For "nodes to which Trisoup was not applied at the relevant depth", Trisoup is not applied at the relevant depth, so no Trisoup vertices or the like are generated for the node itself, but this is an area where Trisoup should be applied at depths greater than the relevant depth. Therefore, when generating a mask for "nodes to which Trisoup was not applied at the relevant depth" in step S1002, a mask where "any of the first to fourth bits of the mask value is 1" as described above is not set, and only a mask where "any of the fifth to eighth bits of the mask value is 1" and a mask where "any of the ninth to twelfth bits of the mask value is 1" are set.

[0207] In step S1003, the approximate surface synthesis unit 2030 determines whether or not processing of all segments corresponding to the "interpolated vertices" generated in step S703 has been completed.

[0208] If completed, the operation proceeds to step S1005, and the mask generation process of step S901 ends. If not completed, the operation proceeds to step S1004.

[0209] In step S1004, the approximate surface synthesis unit 2030 generates, for the segment corresponding to the "interpolated vertex," a mask value that indicates that the "interpolated vertex" exists in the segment.

[0210] The approximate surface synthesis unit 2030 may set the mask value to, for example, 1<<15. When such a value is set, if the 16th bit of the mask value is 1, it can be determined that an "interpolated vertex" exists in the segment.

[0211] After generating the mask value, the approximate surface synthesis unit 2030 proceeds to step S1003 to process the next segment.

[0212] Note that the mask values ​​generated in steps S1002 and S1004 are both powers of 2. By doing so, when combining mask values ​​of segments that exist in the same position in step S905 described later, the mask values ​​can be easily combined by performing a bit operation (logical sum).

[0213] Furthermore, depending on whether each bit of the mask value synthesized in this way is 1 or 0, information about the segment (whether it is a Trisoup node, whether there is a Trisoup node adjacent to it, whether there is an "interpolated vertex" in the segment, etc.) can be obtained, as described above.

[0214] After generating the mask information as described above, the approximate surface synthesis unit 2030 proceeds to step S902.

[0215] In step S902, the approximate surface synthesis unit 2030 collects and sorts all of the segments generated in step S901 and the segments corresponding to the "interpolated vertices."

[0216] The approximate surface synthesis unit 2030 sorts the segments based on, for example, the coordinates of the start point and the coordinates of the end point of each segment. By performing this processing, it is possible to rearrange the segments so that segments having the same start point and end point are consecutive.

[0217] After completing this sorting, the approximate surface synthesis unit 2030 proceeds to step S903.

[0218] In step S903, the approximate surface synthesis unit 2030 determines whether or not processing of all segments after sorting in step S902 has been completed.

[0219] If completed, the operation proceeds to step S909, and the neighbor information collection process of step S704 ends. If not completed, the operation proceeds to step S904.

[0220] In step S904, the approximate surface synthesis unit 2030 determines whether the segment is in the same position as the segment processed immediately before.

[0221] Specifically, the approximate surface synthesis unit 2030 determines whether or not both the coordinates of the start point and the coordinates of the end point of the segment are the same as those of the segment processed immediately before.

[0222] If the position is the same as the previously processed segment, the operation proceeds to step S905. If the position is not the same as the previously processed segment, the operation proceeds to step S906.

[0223] In step S905, the approximate surface synthesis unit 2030 integrates the mask value corresponding to the segment generated in step S901 with the mask values ​​of the segments at the same position processed up to that point.

[0224] For example, when the mask values ​​corresponding to each segment are configured as powers of 2 as described in step S901, the approximate surface synthesis unit 2030 can integrate the masks by taking the logical sum of the mask values ​​of each segment.

[0225] After the approximate surface synthesis unit 2030 integrates the masks, the process proceeds to step S907.

[0226] In step S906, the approximate surface synthesis unit 2030 stores the segment processed immediately before the current segment as a unique segment if the segment satisfies a predetermined condition A.

[0227] For example, the predetermined condition A may be that the segment processed immediately before is a segment corresponding to a Trisoup node.

[0228] For example, if at least one of the four lowest bits of the mask value integrated in step S905 is 1, it is clear that the segment processed immediately before is a segment corresponding to the Trisoup node.

[0229] The approximate surface synthesis unit 2030 may store, as information about the unique segment, the coordinates of the start point, the coordinates of the end point, information about the interpolation points described in step S907 below, information about adjacent segments described in step S908 below, and the mask value integrated in step S905. This information is used in the vertex decoding process in step S705.

[0230] The approximate surface synthesis unit 2030 saves information about the unique segments, and then initializes various information (information about interpolation points, information about adjacent segments, integrated mask values, etc.).

[0231] Furthermore, if the predetermined conditions are not satisfied, the approximate surface synthesis unit 2030 only performs initialization. After the above processing is completed, the operation proceeds to step S907.

[0232] In step S907, if the predetermined condition B is satisfied, the approximate surface synthesis unit 2030 stores information about the interpolation points.

[0233] For example, the predetermined condition B may be that the mask value of the segment indicates that the segment has an "interpolated vertex." Specifically, the predetermined condition B may be that the 16th bit of the mask value is 1.

[0234] The approximate surface synthesis unit 2030 saves the coordinates of the "interpolated vertices" when predetermined condition B is satisfied. Specifically, the approximate surface synthesis unit 2030 saves the values ​​of the "vertex positions on the segment" generated in step S804. The values ​​saved here are saved as information about the unique segment in step S906.

[0235] If the segment is not determined to be a unique segment in step S906 (if the predetermined condition A in step S906 is not satisfied), the information about the interpolation points of the segment saved in step S907 is discarded.

[0236] In addition, the approximate surface synthesis unit 2030 may prepare a separate array to store the values ​​of the "vertex positions on the segment," and in this step store an index value for identifying the value corresponding to the segment in that array instead of the coordinate value itself.

[0237] After the above processing is completed, the operation proceeds to step S908.

[0238] In step S908, the approximate surface synthesis unit 2030 stores information about the adjacent segments.

[0239] Specifically, for example, the approximate surface synthesis unit 2030 stores the start coordinates of the segment, and information for identifying segments having the same start coordinates or end coordinates (for example, segment index, etc.).

[0240] For example, there are four segments with the same start point coordinates as the segment in question, and four segments with the same end point coordinates as the start point coordinates of the segment in question, in the same axial direction as the segment in question, and four segments in the axial direction perpendicular to the segment in question. The information saved here is saved as information about unique segments in step S906.

[0241] If the segment is not determined to be a unique segment in step S906 (if the predetermined condition A in step S906 is not satisfied), the information on the adjacent segments of the segment generated in step S908 is discarded.

[0242] After the above processing is completed, the operation proceeds to step S903 to process the next segment.

[0243] As described above, after collecting the neighboring information in step S704, the approximate surface synthesis unit 2030 proceeds to step S705.

[0244] In step S705, the approximate surface synthesis unit 2030 decodes the vertices of the Trisoup at the trisoup_depth.

[0245] 13 is a flowchart showing an example of a method for decoding vertices in step S705. An example of the processing in step S705 will be described below with reference to FIG.

[0246] As shown in FIG. 13, in step S1201, the approximate surface synthesis unit 2030 determines whether or not the processing has been completed for all the unique segments generated in step S703.

[0247] If the processing has been completed for all unique segments, the operation proceeds to step S1207, and the processing of step S703 ends. If the processing has not been completed for all unique segments, the operation proceeds to step S1202.

[0248] In step S1202, the approximate surface synthesis unit 2030 determines whether or not an "interpolated vertex" exists in the unique segment.

[0249] Whether an "interpolated vertex" exists can be determined based on the integrated mask value saved in step S906. If an "interpolated vertex" exists, the operation proceeds to step S1203. If an "interpolated vertex" does not exist, the operation proceeds to step S1204.

[0250] In step S1203, the approximate surface synthesis unit 2030 saves the "vertex positions on the segment" saved in step S906 as the vertex positions in the unique segment.

[0251] In other words, the approximate surface synthesis unit 2030 sets the "vertex position on the segment" saved in step S906 as the decoded value of the vertex position in the unique segment.

[0252] Furthermore, the approximate surface synthesis unit 2030 sets the decoded value of "vertex presence / absence" in the unique segment to "vertex present."

[0253] That is, step S1203 is a process of implicitly determining the vertex positions and the information indicating the presence or absence of a vertex using the interpolated values, instead of decoding the information indicating the vertex positions and the presence or absence of a vertex from the bitstream.

[0254] After the above processing is completed, the operation proceeds to step S1201 to process the next unique segment.

[0255] In step S1204, the approximate surface synthesis unit 2030 decodes, from the bitstream, information indicating whether or not a vertex exists in the unique segment.

[0256] The information indicating the presence or absence of a vertex may be a 1-bit flag, and may be entropy coded in the point cloud coding device 100. The approximate surface synthesis unit 2030 may prepare a plurality of probability models for entropy coding (coding device) / decoding (decoding device) and select one according to the context. The context may be set based on the mask value saved and integrated in step S906, for example.

[0257] After the approximate surface synthesis unit 2030 decodes the information indicating the presence or absence of a vertex, the process proceeds to step S1205.

[0258] In step S1205, the approximate surface synthesis unit 2030 determines whether or not the unique segment has a vertex.

[0259] If a vertex exists based on the information indicating the presence or absence of the vertex decoded in step S1204, the approximate surface synthesis unit 2030 proceeds to step S1206. If such a vertex does not exist, the approximate surface synthesis unit 2030 proceeds to step S1201 and proceeds to processing the next unique segment.

[0260] In step S1206, the approximate surface synthesis unit 2030 decodes the "vertex position on the segment" for the unique segment in which it is determined that a vertex exists. Here, the vertex position may be entropy coded. When performing entropy coding (encoding device) / decoding (decoding device), the approximate surface synthesis unit 2030 may prepare multiple probability models and select one according to the context. The context may be set, for example, based on the information of the adjacent segments saved in step S906.

[0261] Here, the vertex positions may be decoded with the bit precision specified by the syntax (trisoup_vertex_number_bits) that specifies the precision (number of bits) of the vertex positions of Trisoup described above.

[0262] For example, if trisoup_vertex_number_bits is 2, it may be decoded to take four values: 0, 1, 2, and 3. Furthermore, if trisoup_vertex_number_bits is decoded for each depth, it may be decoded with bit precision corresponding to that depth.

[0263] In addition, the bit precision specified by trisoup_vertex_number_bits is used only at the depth corresponding to the minimum Trisoup node size (largest depth), and thereafter, the bit precision may be increased by 1 as the node size increases by 1 (as the depth decreases by 1).

[0264] For example, if there are two types of node sizes (two levels: minimum and maximum), the vertex position may be decoded using the value of trisoup_vertex_number_bits at the depth corresponding to the minimum Trisoup node size, and the value of trisoup_vertex_number_bits plus 1 at the depth corresponding to the maximum Trisoup node size, as the bit precision of the corresponding depth.

[0265] After decoding the vertex positions, the approximate surface synthesis unit 2030 proceeds to step S1201 to process the next unique segment.

[0266] As described above, after the approximate surface synthesis unit 2030 decodes the vertices in step S705, the process proceeds to step S706.

[0267] In step S706, the approximate surface synthesis unit 2030 generates a reconstructed point group based on the vertices decoded in step S705. Fig. 14 is a flowchart showing an example of a method for generating a reconstructed point group. An example of the processing in step S706 will be described below with reference to Fig. 14.

[0268] As shown in FIG. 14, in step S1301, the approximate surface synthesis unit 2030 determines whether or not processing has been completed for all Trisoup nodes at the trisoup_depth.

[0269] If the process has been completed for all Trisoup nodes, the operation proceeds to step S1306, where the process ends. If the process has not been completed for all Trisoup nodes, the operation proceeds to step S1302.

[0270] In step S1302, first, the approximate surface synthesis unit 2030 identifies unique segments corresponding to each side (segment) of the Trisoup node. For example, the approximate surface synthesis unit 2030 can identify unique segments whose start point coordinates and end point coordinates are the same as those of each side (segment) of the Trisoup node, from among the unique segments processed in step S705.

[0271] Second, if a vertex exists in the identified unique segment, the approximate surface synthesis unit 2030 adds the vertex to the reconstructed point group in the following procedure. (1) The approximate surface synthesis unit 2030 shifts the position of the segment by −0.5. Specifically, the approximate surface synthesis unit 2030 subtracts 0.5 from the coordinate values ​​of the start point coordinate and the end point coordinate of the segment, except for the axis along the direction of the segment.

[0272] For example, if the start coordinates of a segment are (x, y, z) = (12, 100, 32), the node size is 4, and the direction of the segment is the x-axis, the end coordinates will be (16, 100, 32), which is the x-coordinate of the start point plus the node size.

[0273] On the other hand, the coordinates other than the x-axis, that is, the y and z coordinates in this example, are each reduced by 0.5, so the start and end coordinates become (12, 99.5, 31.5) and (16, 99.5, 31.5), respectively. (2) The approximate surface synthesis unit 2030 adds the "vertex position on the segment" of the segment to the above-mentioned starting point coordinates. For example, in the above example, if the "vertex position on the segment" is 2, the result is (14, 99.5, 31.5). (3) The approximate surface synthesis unit 2030 adds 0.5 to the coordinate values ​​calculated in (2) above, except for the coordinates of the axes along the direction of the segment. For example, in the above example, the result is (14, 100, 32). The same result can be obtained by adding 2, the vertex position on the unique segment, to the starting coordinates (12, 100, 32) of the unique segment before executing the procedure in (1) above, so the approximate surface synthesis unit 2030 may perform the calculation in this manner. (4) If the coordinates calculated in (3) above exist inside the Trisoup node, the approximate surface synthesis unit 2030 generates a point at the coordinates calculated in (3) above and adds it to the reconstructed point group.

[0274] Here, the inside of the Trisoup node is a point whose x, y, and z coordinate values ​​are within the range of the start point of the Trisoup node (the point with the smallest x, y, and z coordinates) + Trisoup node size - 1.

[0275] For example, if the starting coordinates of the Trisoup node are (x, y, z) = (12, 100, 32) and the Trisoup node size is 4, points in the ranges (12 to 15, 100 to 103, 32 to 35) are considered to be inside the Trisoup node.

[0276] In this case, if the coordinates calculated in (3) above are (14, 100, 32), these coordinates are determined to be inside the Trisoup node, and the approximate surface synthesis unit 2030 adds the point at these coordinates to the reconstructed point cloud. Figure 16A shows such a result when projected onto the xy plane.

[0277] On the other hand, for example, if the starting point coordinates of the Trisoup node are (x, y, z) = (12, 96, 32), points in the ranges (12 to 15, 96 to 99, 32 to 35) are inside the Trisoup node, so in this case, it is determined that (14, 100, 32) does not exist inside the Trisoup node, and the coordinates calculated in (3) above are not added to the reconstructed point group, and this operation ends the processing. Figure 16B shows an example of such a case.

[0278] After the above processing is completed, the operation proceeds to step S1303.

[0279] In step S1303, the approximate surface synthesis unit 2030 calculates the initial coordinates of the centroid from the vertices corresponding to the Trisoup node.

[0280] For example, the initial coordinates of the centroid can be calculated by averaging the x, y, and z components of all vertex positions of the Trisoup node. Note that if the Trisoup node has three or fewer vertices, the centroid calculation may be omitted.

[0281] After this process is completed, the operation proceeds to step S1304.

[0282] In step S1304, the approximate surface synthesis unit 2030 sorts the vertices and determines the projection plane. Specifically, for example, the approximate surface synthesis unit 2030 can sort the vertices and determine the projection plane in the following procedure. (1) The approximate surface synthesis unit 2030 projects the vertices onto the xy plane. This process is equivalent to extracting the x and y coordinate values ​​from the coordinates of each vertex.

[0283] Since the vertices are originally on the sides of the nodes, in the projected plane, each vertex will be on the side of a square or rectangle that the nodes are projected onto. Since a square is also a type of rectangle, the following explanation will be given assuming that the shape of the node after projection is a rectangle. (2) The approximate surface synthesis unit 2030 sorts the points on the sides of the rectangle onto which the nodes are projected in a clockwise or counterclockwise order.

[0284] Furthermore, the approximate surface synthesis unit 2030 assigns a temporary index value (0, 1, 2, . . . ) to each vertex in the sorted order. (3) The approximate surface synthesis unit 2030 defines a triangle defined by three points: two adjacent vertices in sorted order and the centroid, and calculates the area of ​​this triangle. The area of ​​the triangle can be calculated, for example, by generating vectors starting from the centroid and pointing to the coordinates of the two vertices, and then using the cross product of these vectors.

[0285] In this way, the approximate surface synthesis unit 2030 calculates and sums the areas of triangles for all combinations of two adjacent vertices in sorted order: (0, 1), (1, 2), . . . (N, 0).

[0286] When there are only three vertices, the approximate surface synthesis unit 2030 calculates the area of ​​the triangle formed by the three vertices without using a centroid. This area is the area on the projected plane. (4) The approximate surface synthesis unit 2030 performs the above steps (1) to (3) on the xz plane and the yz plane in the same way, and selects the plane with the largest area calculated in the above step (3) as the projection plane. At this time, the sorting order and the provisional index assigned in the above step (2) are adopted as the final sorting order and index.

[0287] As described above, after sorting the vertices and determining the projection surface, the approximate surface synthesis unit 2030 proceeds to step S1305.

[0288] In step S1305, the approximate surface synthesis unit 2030 generates a reconstructed point group by using the triangles generated in step S1304 and ray tracing.

[0289] Specifically, the approximate surface synthesis unit 2030 generates the reconstructed point group in the following procedure. (1) First, the approximate surface synthesis unit 2030 generates a triangle based on the index and centroid determined in step S1304, similar to step S1304. However, while the approximate surface synthesis unit 2030 generated a triangle on a two-dimensional plane in step S1304, in step S1305 the approximate surface synthesis unit 2030 generates a triangle in three-dimensional space using all of the x, y, and z coordinates of each vertex. (2) Secondly, the approximate surface synthesis unit 2030 defines the normal vector of the projection surface (if the projection surface is an xy plane, the vector in the z direction) and places it on the projection surface.

[0290] For example, the approximate surface synthesis unit 2030 sets the initial position to the point on the projection surface that is closest to the origin of the node shape (that is, the rectangle). (3) Third, the approximate surface synthesis unit 2030 determines whether or not the normal vector intersects with the triangle generated in (1) above as the norm of the normal vector is increased, and if so, calculates the coordinates of the intersection.

[0291] For example, the approximate surface synthesis unit 2030 can realize such processing by using a general ray tracing method, etc. Fig. 16A shows an example of a triangle configuration. (4) Fourth, if the coordinates where the normal vector calculated in (3) above intersects with the triangle exist within the Trisoup node, the approximate surface synthesis unit 2030 generates a point at that coordinate position and adds it to the reconstructed point group. (5) Fifth, the approximate surface synthesis unit 2030 repeats the above steps (3) and (4) for each of the integer coordinate positions within the node (rectangle) on the projection surface.

[0292] In this case, the interval at which the normal vectors are arranged may be 1 (i.e., all integer coordinate positions). Here, the interval of 1 is the smallest possible interval. An example of a reconstructed point group generated from the triangle in FIG. 16A is shown in FIG. 16B.

[0293] After generating the reconstruction points for the Trisoup node in the above manner, the approximate surface synthesis unit 2030 proceeds to step S1301 to process the next Trisoup node.

[0294] As described above, in step S706, the approximate surface synthesis unit 2030 generates the reconstructed point group, and then the process proceeds to step S707.

[0295] In step S707, the approximate surface synthesis unit 2030 determines whether the Trisoup node size at the trisoup_depth is the minimum Trisoup node size.

[0296] If it is the minimum Trisoup node size, the operation proceeds to step S701. Otherwise, that is, if the Trisoup node size at the trisoup_depth is larger than the minimum Trisoup node size, the operation proceeds to step S708.

[0297] In step S708, the approximate surface synthesis unit 2030 interpolates vertices on the segments of the nodes in the minimum Trisoup node size based on the vertices corresponding to the Trisoup node size in trisoup_depth decoded in step S705.

[0298] Fig. 15 is a flowchart showing an example of the processing of step S708. An example of the processing of step S708 will be described below with reference to Fig. 15. Note that the processing of Fig. 15 is almost the same as the processing of Fig. 14, and the same processing is denoted by the same reference numerals. Only the differences from Fig. 14 will be described below.

[0299] As shown in FIG. 15, in step S1402, the approximate surface synthesis unit 2030 saves the vertices of the Trisoup node as "interpolated vertices."

[0300] Specifically, the approximate surface synthesis unit 2030 stores the coordinate values ​​obtained after steps (1) and (2) described in step S1302 have been performed as "interpolated vertices."

[0301] For example, in step S1302, the approximate surface synthesis unit 2030 saves the coordinate values ​​(14, 99.5, 31.5) as the "interpolated vertex." This is because the position of the segment is defined to be shifted by 0.5 from the integer coordinate position.

[0302] To store this value as an integer, the approximate surface synthesis unit 2030 may store it as a value obtained by doubling the true coordinate value. In the above example, the approximate surface synthesis unit 2030 may save the value (28, 199, 63) as the "interpolated vertex."

[0303] In step S1405, the approximate surface synthesis unit 2030 changes the position at which the normal vector is placed in procedure (2) of step S1305 so that it is placed at a position where a segment exists when the Trisoup node is divided by the minimum Trisoup node size.

[0304] If the intersecting coordinates calculated in the same manner as in step (3) of step S1305 are on a segment obtained by dividing the Trisoup node into the minimum Trisoup node size, the approximate surface synthesis unit 2030 stores the coordinate values ​​as "interpolated vertices."

[0305] If the coordinate values ​​are doubled and converted into integers and stored in step S1402, the approximate surface synthesis unit 2030 also stores the doubled coordinate values ​​in step S1405.

[0306] The approximate surface synthesis unit 2030 saves the "interpolated vertices" generated by the above process and uses them in step S703 of the next trisoup_depth.

[0307] Furthermore, if the trisoup_depth is greater than 0, i.e., if the Trisoup node size at the trisoup_depth is not the maximum Trisoup node size, there will be "interpolated vertices" generated at a larger node size, but the approximate surface synthesis unit 2030 does not delete the set of these "interpolated vertices," but saves the newly generated "interpolated vertices" in step S1402 by adding them to the above set.

[0308] As shown in FIG. 16C, the approximate surface synthesis unit 2030 may store only the "interpolated vertices" that exist on the surface of the Trisoup node.

[0309] For convenience, step S706 and step S708 have been described as separate processes, but since they have much in common, they may be performed simultaneously.

[0310] For example, if the conditions of step S707 are satisfied in step S1302, the approximate surface synthesis unit 2030 may also perform step S1402, and similarly, if the conditions of step S707 are satisfied in step S1305, the approximate surface synthesis unit 2030 may also perform step S1405.

[0311] In step S709, the reconstructed point group in the slice generated by the above method is subsampled. Fig. 17 is a flowchart showing an example of the processing in step S709. An example of the processing in step S709 will be described below with reference to Fig. 17.

[0312] 17, in step S1701, the approximate surface synthesis unit 2030 sorts the reconstruction points included in the reconstruction point group. This sorting is a process of rearranging the reconstruction points using a predetermined method.

[0313] For example, the approximate surface synthesis unit 2030 may sort based on the coordinate values ​​of each point. Alternatively, for example, the approximate surface synthesis unit 2030 may sort in ascending order of the x coordinate of each point.

[0314] At this time, for example, if there are multiple points with the same x coordinate, the approximate surface synthesis unit 2030 may sort them in ascending order of y coordinate. Furthermore, if there are multiple points with the same x coordinate and y coordinate, the approximate surface synthesis unit 2030 may sort them in ascending order of z coordinate. After performing this sorting, the approximate surface synthesis unit 2030 proceeds to step S1702.

[0315] In step S1702, the approximate surface synthesis unit 2030 determines the subsampling interval S based on the syntax (trisoup_sampling_value_minus1) that controls the subsampling interval of the reconstruction points described in Figure 5, or the threshold (trisoup_sampling_threshold) that indicates the maximum number of points after subsampling described in Figure 6.

[0316] For example, the approximate surface synthesis unit 2030 may set as S the value of the syntax that controls the sub-sampling interval of the reconstruction points.

[0317] Also, for example, the approximate surface synthesis unit 2030 may set the minimum value of the sub-sampling interval of the reconstruction points to 1, and may set S to the value obtained by adding 1 to the value of the syntax that controls the sub-sampling interval of the reconstruction points.

[0318] Next, an example of a method for determining S using a threshold value indicating the maximum number of points after subsampling will be described.

[0319] Here, when the number of points included in the reconstructed point group (before subsampling) is N, the approximate surface synthesis unit 2030 may set the smallest positive integer that satisfies the following relationship as S. In this case, the minimum value of S is 1.

[0320] N / S≦trisoup_sampling_threshold Alternatively, the approximate surface synthesis unit 2030 may modify the following equation and set S to the smallest positive integer that satisfies the following relationship:

[0321] N≦S×trisoup_sampling_threshold As described above, after the approximate surface synthesis unit 2030 determines the sub-sampling interval S, the process proceeds to step S1703.

[0322] In step S1703, the approximate surface synthesis unit 2030 performs sub-sampling.

[0323] For example, when an index i (i=0, 1, 2, . . . , N-1) is assigned to each point in the order sorted in step S1701, the approximate surface synthesis unit 2030 may perform subsampling by retaining only points where i%S=0 and discarding other points. Here, % is an operator that calculates the remainder.

[0324] After completing the sub-sampling, the approximate surface synthesis unit 2030 proceeds to step S1704 and ends the process.

[0325] As described above, according to the above configuration, a reconstructed point group having an arbitrary number of points or less can be easily generated by first decoding a point group at a fine density and then subsampling it.

[0326] Furthermore, with this configuration, by determining the subsampling interval S based on the syntax value transmitted from the point cloud encoding device 100 and performing subsampling, the point cloud encoding device 100 can set the above-mentioned syntax value taking into account the subsampling density between different slices and the subjective quality, which has the advantage of being able to control the subjective quality of the reconstructed point cloud.

[0327] After completing the subsampling process in step S709, the approximate surface synthesis unit 2030 proceeds to step S710 and ends the process.

[0328] As described above, the approximate surface synthesis unit 2030 in this embodiment may be configured to decode, for each node of a predetermined size, the node size for applying Trisoup to descendant nodes obtained by recursively dividing the node using an Octree.

[0329] Furthermore, as described above, the approximate surface synthesis unit 2030 in this embodiment may be configured to perform context-adaptive decoding using information on the presence or absence of vertices and the vertex positions of spatially adjacent decoded segments when decoding the presence or absence of vertices and the vertex positions of each segment that constitutes a Trisoup node.

[0330] Furthermore, as described above, the approximate surface synthesis unit 2030 in this embodiment may be configured to perform context-adaptive decoding using information on the presence / absence of vertices and vertex positions of segments that constitute a node of the same size as the Trisoup node, among decoded segments that are spatially adjacent to the Trisoup node, when decoding the presence / absence of vertices and vertex positions of each segment that constitutes the Trisoup node.

[0331] With this configuration, an appropriate Trisoup node size is selected for each local region according to the characteristics of the point cloud, and Trisoup nodes of the same size are adjacent within the local region, thereby improving coding efficiency by utilizing spatial correlation.

[0332] Although FIG. 8 illustrates an example in which subsampling is performed in slice units, such subsampling processing may be performed for each node.

[0333] An example of processing when subsampling is performed for each node will be described with reference to Fig. 18. Fig. 18 is the processing of Fig. 14 with step S709, which is subsampling processing, added.

[0334] As shown in FIG. 18, the approximate surface synthesis unit 2030 performs the processes of S1302 to S1305 for the node, and generates a reconstructed point group for the node.

[0335] Thereafter, the approximate surface synthesis unit 2030 performs subsampling processing in step S709 on the reconstructed point group at the node.

[0336] Basically, the same processing can be realized by replacing the reconstruction point group of the slice in question in the description of the processing in step S709 with the reconstruction point group of the node in question.

[0337] In determining the sub-sampling interval in step S1702, the approximate surface synthesis unit 2030 determines the sub-sampling interval S based on the syntax (trisoup_sampling_value_minus1) that controls the sub-sampling interval.

[0338] As described above, the point cloud decoding device 200 in this embodiment may be configured to include a geometric information decoding unit 2010 that decodes parameters related to the subsampling interval, and an approximate surface synthesis unit 2030 that generates a reconstructed point cloud for the slice or node, determines the subsampling interval based on the parameters related to the subsampling interval S described above, and subsamples the reconstructed point cloud for the slice or node based on the subsampling interval S.

[0339] With this configuration, it is possible to set parameters on the point cloud encoding device 100 side taking into consideration the subjective quality of the reconstructed point cloud, and it is also possible to keep the reconstructed point cloud to an arbitrary number of points or less.

[0340] Here, the arbitrary number of points is, for example, the upper limit of the number of reconstruction points per slice specified by the standard, and must be observed to ensure interoperability between devices that meet the standard.

[0341] Furthermore, as described above, the approximate surface synthesis unit 2030 in this embodiment may be configured to perform subsampling by sorting the reconstructed point group of the above-mentioned slice or the node in a predetermined manner, assigning indices to the reconstructed point group in the sorted order, retaining only points whose index value is 0 when divided by the subsampling interval, and discarding other points.

[0342] With this configuration, points can be subsampled spatially uniformly from the reconstructed point group, thereby suppressing degradation of subjective image quality due to subsampling.

[0343] Furthermore, as described above, the point cloud decoding device 200 in this embodiment may be configured to decode a syntax that controls the subsampling interval of the reconstruction points as a parameter related to the subsampling interval S, and the approximate surface synthesis unit 2030 may be configured to perform subsampling using the value of the syntax that controls the subsampling interval S of the reconstruction points or a value obtained by adding 1 to the value of the syntax that controls the subsampling interval S of the reconstruction points as the subsampling interval S.

[0344] With this configuration, the point group decoding device 200 can omit the calculation related to deriving the sub-sampling interval S.

[0345] Furthermore, as described above, the point cloud decoding device 200 in this embodiment may be configured to decode a threshold value indicating the maximum number of points after subsampling as a parameter related to the subsampling interval S, and the approximate surface synthesis unit 2030 may be configured to perform subsampling by using the smallest value of S that is equal to or greater than the number of points in the reconstructed point cloud of the slice when the threshold value indicating the maximum number of points after subsampling is multiplied by a positive integer S as the subsampling interval.

[0346] With this configuration, the point group decoding device 200 can explicitly determine how many points or less the reconstructed point group should have, which facilitates operation verification.

[0347] In the above, we have explained an example of interpolating a vertex of a smaller node size from a vertex of a larger node size at the boundary of Trisoup nodes of different sizes using Figure 8, but conversely, we can also use a vertex of a smaller node size to replace a vertex of a larger node size.

[0348] 19 and 20, an example of a process for replacing a vertex of a larger node size with a vertex of a smaller node size at a Trisoup node boundary of different sizes will be described. Note that the same processes as those in FIG. 8 are assigned the same reference numerals and will not be described again.

[0349] As shown in FIG. 19, in step S721, the approximate surface synthesis unit 2030 interpolates vertices at the minimum node size.

[0350] Here, the basic interpolation method is the same as the method described in step S708 of Fig. 8 and in Fig. 15 and Fig. 16. The only difference is that in step S708, interpolated vertices are generated only on the surface of the original node as shown in Fig. 16C, but in step S721, interpolated vertices are also generated inside the original node as shown in Fig. 20.

[0351] In this way, the approximate surface synthesis unit 2030 generates vertices in units of the minimum Trisoup node size, regardless of the original Trisoup node size.

[0352] In step S722, the approximate surface synthesis unit 2030 integrates the vertices, using as input the vertices in each segment of the minimum node size interpolated in step S721 and the vertices decoded in the minimum Trisoup node size.

[0353] In this case, if the node sizes differ between adjacent nodes, there may be a plurality of vertices interpolated from a larger node size and a vertex decoded at the minimum Trisoup node size for a given segment.

[0354] For example, the approximate surface synthesis unit 2030 can integrate these vertices by keeping only one point per segment and discarding the others.

[0355] Specifically, for example, when there are multiple vertices, the approximate surface synthesis unit 2030 can integrate them by leaving only the vertex generated from the smallest Trisoup node size among them.

[0356] As described above, the approximate surface synthesis unit 2030 integrates the vertices of each segment into one, and then proceeds to the process of generating a reconstructed point group in step S723.

[0357] In step S723, the approximate surface synthesis unit 2030 generates a reconstructed point group based on the vertices integrated in step S722.

[0358] At this time, by steps S721 and S722, the vertices of the Trisoup correspond to each segment in the minimum Trisoup node size, so that a reconstructed point group for the entire slice can be generated by performing the processing of FIG. 14 in the minimum Trisoup node size.

[0359] (Point cloud encoding device 100) The point group encoding device 100 according to this embodiment will be described below with reference to Fig. 21. Fig. 21 is a diagram showing an example of functional blocks of the point group encoding device 100 according to this embodiment.

[0360] As shown in FIG. 21, the point cloud encoding device 100 includes a coordinate transformation unit 1010, a geometric information quantization unit 1020, a tree analysis unit 1030, an approximate surface analysis unit 1040, a geometric information encoding unit 1050, a geometric information reconstruction unit 1060, a color conversion unit 1070, an attribute transfer unit 1080, a RAHT unit 1090, an LoD calculation unit 1100, a lifting unit 1110, an attribute information quantization unit 1120, and an attribute information encoding unit 1130.

[0361] The coordinate conversion unit 1010 is configured to convert the three-dimensional coordinate system of the input point cloud into any other coordinate system. For example, the coordinate conversion may involve rotating the input point cloud to convert the x, y, and z coordinates of the input point cloud into any s, t, and u coordinates. Alternatively, as a variation of the conversion, the coordinate system of the input point cloud may be used as is.

[0362] The geometric information quantization unit 1020 is configured to quantize the position information of the input point group after coordinate transformation and remove points with overlapping coordinates. Note that when the quantization step size is 1, the position information of the input point group and the position information after quantization match. In other words, when the quantization step size is 1, it is equivalent to not performing quantization.

[0363] The tree analysis unit 1030 is configured to receive position information of the quantized point group as input, and to generate an occupancy code indicating at which node in the encoding target space a point exists, based on a tree structure described below.

[0364] In this process, the tree analysis unit 1030 is configured to recursively divide the encoding target space into rectangular parallelepipeds to generate a tree structure.

[0365] If a point exists within a rectangular parallelepiped, a tree structure can be generated by recursively dividing the rectangular parallelepiped into multiple rectangular parallelepipeds until the rectangular parallelepiped reaches a predetermined size. Each such rectangular parallelepiped is called a node. Each rectangular parallelepiped generated by dividing a node is called a child node, and the occupancy code is expressed as 0 or 1 to indicate whether the child node contains a point.

[0366] As described above, the tree analysis unit 1030 is configured to generate occupancy codes while recursively dividing nodes until a predetermined size is reached.

[0367] In this embodiment, a method called "Octree" can be used, which recursively performs octree division on the above-mentioned rectangular parallelepiped, always treating it as a cube, and a method called "QtBt" can be used, which performs quadtree division and binary tree division in addition to octree division.

[0368] Here, whether or not to use "QtBt" is transmitted to the point cloud decoding device 200 as control data.

[0369] Alternatively, it may be specified to use predicitive coding using an arbitrary tree structure. In such a case, the tree analysis unit 1030 determines the tree structure, and the determined tree structure is transmitted to the point cloud decoding device 200 as control data.

[0370] For example, the tree-structured control data may be configured so that it can be decoded using the procedure described in FIG.

[0371] The approximate surface analyzer 1040 is configured to generate approximate surface information using the tree information generated by the tree analyzer 1030 .

[0372] For example, when decoding three-dimensional point cloud data of an object, if the point cloud is densely distributed on the surface of the object, approximate surface information is used to represent the area where the point cloud exists by approximating it with a small plane, rather than decoding each individual point cloud.

[0373] Specifically, the approximate surface analysis unit 1040 may be configured to generate approximate surface information using, for example, a method called "Trisoup." Also, when decoding a sparse point cloud acquired by Lidar or the like, this process can be omitted.

[0374] The geometric information encoding unit 1050 is configured to generate a bitstream (geometric information bitstream) by encoding syntax such as the occupancy code generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040. Here, the bitstream may include, for example, the syntax described in Figures 4 and 5.

[0375] The encoding process is, for example, a context-adaptive binary arithmetic coding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the position information.

[0376] The geometric information reconstruction unit 1060 is configured to reconstruct the geometric information of each point of the point cloud data to be encoded (the coordinate system assumed by the encoding process, i.e., the position information after the coordinate transformation in the coordinate transformation unit 1010) based on the tree information generated by the tree analysis unit 1030 and the approximate surface information generated by the approximate surface analysis unit 1040.

[0377] The color conversion unit 1070 is configured to perform color conversion when the input attribute information is color information. The color conversion does not necessarily have to be performed, and whether or not the color conversion process is to be performed is coded as part of the control data and transmitted to the point cloud decoding device 200.

[0378] The attribute transfer unit 1080 is configured to correct the attribute values ​​based on the position information of the input point cloud, the position information of the point cloud after reconstruction by the geometric information reconstruction unit 1060, and the attribute information after color change by the color conversion unit 1070, so as to minimize distortion of the attribute information.

[0379] The RAHT unit 1090 is configured to receive the attribute information transferred by the attribute transfer unit 1080 and the geometric information generated by the geometric information reconstruction unit 1060 as input, and to generate residual information for each point using a type of Haar transform called RAHT (Region Adaptive Hierarchical Transform). As a specific example of the RAHT processing, the method described in the above-mentioned document 2 can be used.

[0380] The LoD calculation unit 1100 is configured to receive the geometric information generated by the geometric information reconstruction unit 1060 as an input and generate an LoD (Level of Detail).

[0381] LoD is information for defining the reference relationship (reference point and referenced point) to realize predictive coding, such as predicting attribute information of another point from attribute information of another point and encoding or decoding the prediction residual.

[0382] In other words, LoD is information that defines a hierarchical structure in which points contained in geometric information are classified into multiple levels, and the attributes of points belonging to lower levels are encoded or decoded using the attribute information of points belonging to higher levels.

[0383] As a specific method for determining the LoD, for example, the method described in the above-mentioned document 2 may be used.

[0384] The lifting unit 1110 is configured to generate residual information by performing a lifting process using the LoD generated by the LoD calculation unit 1100 and the attribute information after attribute transfer by the attribute transfer unit 1080.

[0385] As a specific example of the lifting process, the method described in the document (Text of ISO / IEC 23090-9 DIS Geometry-based PCC, ISO / IEC JTC1 / SC29 / WG11 N19088) may be used.

[0386] The attribute information quantization unit 1120 is configured to quantize the residual information output from the RAHT unit 1090 or the lifting unit 1110. Here, a quantization step size of 1 is equivalent to no quantization being performed.

[0387] The attribute information encoding unit 1130 is configured to perform encoding processing using the quantized residual information and the like output from the attribute information quantization unit 1120 as syntax, and to generate a bit stream related to the attribute information (attribute information bit stream).

[0388] The encoding process is, for example, a context-adaptive binary arithmetic encoding process. Here, for example, the syntax includes control data (flags and parameters) for controlling the decoding process of the attribute information.

[0389] Through the above processing, the point cloud encoding device 100 is configured to perform encoding processing using the position information and attribute information of each point in a point cloud as input, and to output a geometric information bit stream and an attribute information bit stream.

[0390] Furthermore, the above-described point group encoding device 100 and point group decoding device 200 may be realized as a program that causes a computer to execute each function (each step).

[0391] In each of the above embodiments, the present invention has been described using the application of the point cloud encoding device 100 and the point cloud decoding device 200 as an example, but the present invention is not limited to such an example and can be similarly applied to a point cloud encoding / decoding system having the functions of the point cloud encoding device 100 and the point cloud decoding device 200. [Industrial Applicability]

[0392] According to this embodiment, for example, it is possible to improve the overall service quality in video communication, which will contribute to the achievement of Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Develop resilient infrastructure, promote sustainable industrialization and foster innovation." [Explanation of symbols]

[0393] 10...Point cloud processing system 100...Point cloud encoding device 1010... Coordinate conversion section 1020...Geometric information quantization section 1030...Tree analysis section 1040…Approximate surface analysis section 1050...Geometric information encoding unit 1060...Geometric information reconstruction unit 1070...Color conversion section 1080...Attribute transfer section 1090…RAHT Department 1100...LoD calculation section 1110...Lifting section 1120...Attribute information quantization section 1130...Attribute information encoding unit 200...Point cloud decoding device 2010…Geometric Information Decoding Department 2020…Tree Synthesis Department 2030…Approximate surface synthesis part 2040...Geometric information reconstruction unit 2050...Inverse coordinate transformation section 2060...Attribute information decoding unit 2070...Inverse quantization section 2080…RAHT Department 2090…LoD calculation section 2100...Reverse lifting section 2110...Color inverse converter

Claims

1. A point cloud decoding device, comprising: a geometry information decoding unit that decodes parameters related to the sub-sampling interval; an approximate surface synthesis unit; The approximate surface synthesis unit generating a reconstructed point cloud for a slice or node of interest; determining a sub-sampling interval based on a parameter related to the sub-sampling interval; A point cloud decoding device, characterized in that it subsamples the reconstructed point cloud of the slice or node based on the subsampling interval.

2. The approximate surface synthesis unit sorting the reconstructed point cloud of the slice or node in a predetermined manner; assigning an index to the reconstructed point cloud in the sorted order; The point cloud decoding device according to claim 1, characterized in that subsampling is performed by retaining only points whose remainder when the index value of each point is divided by the subsampling interval is 0, and discarding other points.

3. the geometric information decoding unit decodes, as the parameter related to the subsampling interval, a syntax for controlling a subsampling interval of the reconstruction points; The point cloud decoding device according to claim 2, characterized in that the approximate surface synthesis unit performs the subsampling using a value of a syntax that controls the subsampling interval of the reconstruction points or a value obtained by adding 1 to the value of a syntax that controls the subsampling interval of the reconstruction points as the subsampling interval.

4. the geometric information decoding unit decodes a threshold value indicating a maximum number of points after the subsampling as a parameter related to the subsampling interval; The point cloud decoding device described in claim 2, characterized in that the approximate surface synthesis unit performs the subsampling using the smallest integer value that is greater than or equal to the number of points in the reconstructed point cloud of the slice as the subsampling interval when a threshold value indicating the maximum number of points after the subsampling is multiplied by a positive integer.

5. A point cloud decoding method, comprising: decoding parameters relating to the sub-sampling interval; generating a reconstructed point cloud of a slice or node of the object; determining a sub-sampling interval based on a parameter related to the sub-sampling interval; and subsampling the reconstructed point cloud of the slice or node based on the subsampling interval.

6. A program that causes a computer to function as a point group decoding device, The point group decoding device includes: a geometry information decoding unit that decodes parameters related to the sub-sampling interval; an approximate surface synthesis unit; The approximate surface synthesis unit generating a reconstructed point cloud for a slice or node of interest; determining a sub-sampling interval based on a parameter related to the sub-sampling interval; A program for subsampling a reconstructed point cloud of the slice or node based on the subsampling interval.

Images