Point cloud decoding apparatus, point cloud decoding method, and program

Abstract

The point cloud decoding apparatus (200) of the present invention includes a tree synthesis unit (2020), which is configured to decode the position information of each point in the point cloud data of the decoding object and assign the position information index in ascending order according to the decoding order of the position information.

Description

Technical Field

[0001] This invention relates to a point cloud decoding device, a point cloud decoding method, and a program. Background Technology

[0002] Non-patent document 1 discloses a technique for sorting points in a point cloud data according to the location information after decoding the location information of each point and before decoding the attribute information of each point in the point cloud data.

[0003] Existing technical documents

[0004] Non-patent literature

[0005] Non-patent document 1: ISO / IEC 23090-9DIS Geometry-based PCC (Point Cloud Compression) text, ISO / IEC JTC1 / SC29 / WG11 N19088

[0006] Non-patent literature 2: [G-PCC] [New Proposal] Predictive Geometry Coding, International Organization for Standardization / International Electrotechnical Commission JTC1 / SC29 / WG11 m51012 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] However, in the existing technology, when sorting the points of the point cloud data according to the location information, there is a problem that the attribute information cannot be decoded if the decoding of the location information of all the points in the point cloud is not completed.

[0009] Furthermore, in the existing technology, when the sorting process is omitted, there is a problem that the processing order of point clouds in the decoding process of attribute information is not specified.

[0010] Therefore, the present invention was made in view of the above problems, and its object is to provide a point cloud decoding device, point cloud decoding method and program, which can establish a unique correspondence between the decoded position information and the attribute information by uniquely determining the order of decoding the attribute information even when no sorting process is performed when decoding the attribute information.

[0011] Methods for solving problems

[0012] The first feature of the present invention is a point cloud decoding device, the main feature of which is that it has a tree synthesis unit, which is configured to decode the position information of each point of the point cloud data of the decoding object, assign indexes of each point of the point cloud data in ascending order according to the decoding order of the position information, and store the indexes in a one-dimensional array.

[0013] The second feature of the present invention is a point cloud decoding method, which includes: a step of decoding the position information of each point in the point cloud data of the decoding object; a step of assigning indexes of each point in the point cloud data in ascending order according to the decoding order of the position information; and a step of storing the indexes in a one-dimensional array.

[0014] The third feature of the present invention is a program used by a point cloud decoding device, the purpose of which is to enable a computer to perform: a step of decoding the position information of each point in the point cloud data of the decoding object; a step of assigning an index to each point in the point cloud data in ascending order according to the decoding order of the position information; and a step of storing the index in a one-dimensional array.

[0015] Invention Effects

[0016] According to the present invention, a point cloud decoding device, a point cloud decoding method and a program can be provided, which can establish a unique correspondence between the decoded position information and the attribute information even when no sorting process is performed when decoding the attribute information, by uniquely determining the order of decoding the attribute information.

[0017] Brief description of the attached diagram

[0018] Figure 1 This is a diagram illustrating an example of the structure of a point cloud processing system 10 according to one embodiment.

[0019] Figure 2 This is a diagram illustrating an example of the functional blocks of a point cloud decoding device 200 according to one embodiment.

[0020] Figure 3 This is an example of the structure of the encoded data (bit stream) received by the geometric information decoding unit 2010 of the point cloud decoding apparatus 200 according to one embodiment.

[0021] Figure 4 This is an example of the syntax structure of GPS2011 as described in one implementation.

[0022] Figure 5 This is a flowchart illustrating an example of the processing of the tree synthesis unit 2020 of the point cloud decoding apparatus 200 according to one embodiment.

[0023] Figure 6 It means Figure 5The flowchart shows an example of node information decoding processing in step S506.

[0024] Figure 7 It means Figure 5 The flowchart shows an example of node information decoding processing in step S506.

[0025] Figure 8 It means Figure 6 A diagram illustrating an example of the method for setting the mask planar in step S601.

[0026] Figure 9 This is an example of the structure of the encoded data (bit stream) received by the attribute information decoding unit 2060 of the point cloud decoding apparatus 200 according to one embodiment.

[0027] Figure 10 This is an example of the syntax structure of APS2061 involved in one implementation.

[0028] Figure 11 This is a flowchart illustrating an example of the processing steps of the LoD calculation unit 2090 of the point cloud decoding apparatus 200 according to one embodiment. Detailed Implementation

[0029] The following is a reference to the appendix. Figure 1 The embodiments of the present invention will be described below. Furthermore, the constituent elements in the following embodiments can be appropriately substituted with known constituent elements, and various modifications, including combinations with other known constituent elements, are possible. Therefore, the inventive content described in the technical solutions is not limited to the embodiments described below.

[0030] (First Embodiment)

[0031] The following is for reference Figures 1 to 11 The point cloud processing system 10 according to the first embodiment of the present invention will be described. Figure 1 This is a diagram illustrating the point cloud processing system 10 described in the embodiments of this invention.

[0032] like Figure 1 As shown, the point cloud processing system 10 has a point cloud encoding device 100 and a point cloud decoding device 200.

[0033] The point cloud encoding device 100 is configured to generate encoded data (bit stream) by encoding the input point cloud signal. The point cloud decoding device 200 is configured to generate an output point cloud signal by decoding the bit stream.

[0034] Furthermore, the input and output point cloud signals are composed of the position and attribute information of each point within the point cloud. Attribute information includes, for example, the color and reflectivity of each point.

[0035] Here, the bit stream can be transmitted from the point cloud encoding device 100 to the point cloud decoding device 200 via a transmission path. Alternatively, the bit stream can be provided from the point cloud encoding device 100 to the point cloud decoding device 200 after being stored in a storage medium.

[0036] (Point cloud decoding device 200)

[0037] The following is for reference Figure 2 The point cloud decoding device 200 involved in this embodiment will be described. Figure 2 This is a diagram illustrating an example of the functional blocks of the point cloud decoding device 200 according to this embodiment.

[0038] like Figure 2 As shown, the point cloud decoding device 200 includes a geometric information decoding unit 2010, a tree synthesis unit 2020, an approximate surface synthesis unit 2030, a geometric information reconstruction unit 2040, an inverse coordinate transformation unit 2050, an attribute information decoding unit 2060, an inverse quantization unit 2070, a RAHT unit 2080, a LoD calculation unit 2090, an inverse lifting unit 2100, and an inverse color transformation unit 2110.

[0039] The geometric information decoding unit 2010 is configured to decode the syntax of the geometric information-related bit stream (geometric information bit stream) in the bit stream output from the point cloud encoding device 100.

[0040] Decoding processing is, for example, context-adaptive binary arithmetic decoding. Here, for example, the syntax includes control data (flags, parameters) for controlling the decoding process of position information.

[0041] The tree synthesis unit 2020 is configured to generate tree information in which region of the decoded object space the point exists in, based on the control data decoded by the geometric information decoding unit 2010 and the occupancy code (described later) indicating which node the point cloud exists in.

[0042] This process can recursively divide the decoded object space into cuboids, determine whether a point exists in each cuboid by referring to the occupancy code, divide the cuboid with points into multiple cuboids, and generate tree information by referring to the occupancy code.

[0043] In this embodiment, a method called "Octree" can be used, in which the above-mentioned cuboid is recursively divided into cubes by an octree; and a method called "QtBt" can be used, in which quadtree and binary tree divisions are performed in addition to octree division. Whether to use "QtBt" is transmitted as control data from the point cloud encoding device 100 side.

[0044] Alternatively, if specified using predictive coding as described later based on control data, the tree synthesis unit 2020 is configured to decode the coordinates of each point based on an arbitrary tree structure determined in the point cloud coding device 100.

[0045] The approximate surface synthesis unit 2030 is configured to generate approximate surface information using tree information generated by the tree information synthesis unit 2020.

[0046] For approximate surface information, such as when decoding the three-dimensional point cloud data of an object, in cases where the point cloud is densely distributed on the object's surface, the area where the point cloud exists is approximately represented on a small plane without decoding each point cloud individually.

[0047] Specifically, the approximate surface synthesis unit 2030 can generate approximate surface information, for example, using a method called "Trisoup". As a specific process of "Trisoup", the method described in Non-Patent Document 1 can be used, for example. Furthermore, this process can be omitted when decoding sparse point clouds obtained by LiDAR or the like.

[0048] The geometric information reconstruction unit 2040 is configured to reconstruct the geometric information (position information in the coordinate system assumed by the decoding process) of each point in the point cloud data of the decoding object based on the tree information generated by the tree information synthesis unit 2020 and the approximate surface information generated by the approximate surface synthesis unit 2030.

[0049] The inverse coordinate transformation unit 2050 is configured to transform the input geometric information reconstructed by the geometric information reconstruction unit 2040 from the coordinate system assumed by the decoding process to the coordinate system of the output point cloud signal and output its position information.

[0050] The attribute information decoding unit 2060 is configured to decode the syntax of the attribute information-related bit stream (attribute information bit stream) in the bit stream output from the point cloud encoding device 100.

[0051] Decoding processing is, for example, context-adaptive binary arithmetic decoding. Here, for example, the syntax contains control data (flags and parameters) for controlling the decoding process of attribute information.

[0052] Furthermore, the attribute information decoding unit 2060 is configured to quantize the residual information based on the decoded syntax.

[0053] The inverse quantization unit 2070 is configured to perform inverse quantization processing based on the quantization completion residual information decoded by the attribute information decoding unit 2060 and the quantization parameters, which are one of the control data decoded by the attribute information decoding unit 2060, and generate inverse quantization completion residual information.

[0054] After inverse quantization, the residual information is output to either the RAHT unit 2080 or the LoD calculation unit 2090 based on the characteristics of the point cloud of the decoded object. The output location is specified by control data decoded by the attribute information decoding unit 2060.

[0055] The RAHT unit 2080 is configured to decode the attribute information of each point of the input inversely quantized residual information generated from the inversely quantized residual information and the geometric information generated by the geometric information reconstruction unit 2040 using a type of Haar transform (inverse Haar transform in decoding processing) called RAHT (Region Adaptive Hierarchical Transform). As a specific RAHT process, for example, the method described in Non-Patent Document 1 can be used.

[0056] The LoD calculation unit 2090 is configured to generate the LoD (Level of Detail) of the geometric information input by the geometric information reconstruction unit 2040.

[0057] LoD is information used to define a reference relationship (reference point and referenced point), which is used to perform prediction encoding such as predicting the attribute information of another point based on the attribute information of one point, and encoding or decoding the prediction residual.

[0058] In other words, LoD defines hierarchical information, such as classifying points in geometric information into multiple levels, and using the attribute information of points belonging to higher levels to encode or decode the attributes of points belonging to lower levels.

[0059] As a specific method for determining LoD, for example, the method described in Non-Patent Document 1 can be used.

[0060] The inverse lifting unit 2100 is configured to decode the attribute information of each point based on the hierarchical structure defined by the LoD, using the LoD generated by the LoD calculation unit 2090 and the inverse quantization completed residual information. As a specific process for inverse lifting, for example, the method described in Non-Patent Document 1 can be used.

[0061] The inverse color transformation unit 2110 is configured to perform inverse color transformation processing on the attribute information output from the RAHT unit 2080 or the inverse lifting unit 2100 when the attribute information of the decoded object is color information and color transformation has been performed on the point cloud encoding device 100 side. Whether to perform the inverse color transformation processing is determined by the control data decoded by the attribute information decoding unit 2060.

[0062] The point cloud decoding device 200 is configured to decode and output the attribute information of each point in the point cloud through the above processing.

[0063] (Geometric Information Decoding Department 2010)

[0064] The following uses Figures 3-4 The control data decoded by the geometric information decoding unit 2010 will be explained.

[0065] Figure 3 This is an example of the structure of encoded data (bit stream) received by the geometric information decoding unit 2010.

[0066] First: The bitstream can also contain GPS2011. GPS2011, also known as a geometric parameter set, is a collection of control data related to the decoding of geometric information. Specific examples will be explained later. Each GPS2011 contains at least one GPS ID information used to identify each one in the presence of multiple GPS2011s.

[0067] Second: The bit stream can also contain GSH2012A / 2012B. GSH2012A / 2012B, also known as the geometric slice header or geometric data unit header, is a collection of control data corresponding to the slices described later. Hereafter, the term "slice" will be used, but it can also be referred to as a "data unit." Specific examples will be given later. GSH2012A / 2012B contains at least GPS ID information specifying the GPS2011 corresponding to each GSH2012A / 2012B.

[0068] Third: The bitstream can also include slice data 2013A / 2013B after GSH2012A / 2012B. Slice data 2013A / 2013B contains data encoded with geometric information. As an example of slice data 2013A / 2013B, the occupancy codes described later can be listed.

[0069] As described above, the bit stream becomes a structure where GSH2012A / 2012B and GPS2011 correspond to each slice data 2013A / 2013B, respectively.

[0070] As mentioned above, since the GPS id information in GSH2012A / 2012B specifies which GPS2011 to reference, it is possible to use a common GPS2011 for multiple slice data 2013A / 2013B.

[0071] In other words, GPS2011 does not necessarily need to transmit every slice. For example, as Figure 3 As shown, it can also be configured to not encode GPS2011 before GSH2012B and slice data 2013B, such a bit stream structure.

[0072] also, Figure 3 The structure described above is merely one example. Any structure corresponding to GSH2012A / 2012B and GPS2011 with slice data 2013A / 2013B can also have additional elements added as constituent elements of the bit stream. For example, the bit stream can also contain Sequence Parameter Sets (SPS). Furthermore, similarly, it can be shaped during transmission to... Figure 3 Different structures. Moreover, it can also be combined with the bit stream decoded by the attribute information decoding unit 2060 (described later) and transmitted as a single bit stream.

[0073] Figure 4 This is an example of the syntax structure of GPS2011.

[0074] Furthermore, the syntax names described below are merely examples. The syntax names may differ as long as the functions of the syntaxes described below are the same.

[0075] GPS2011 may contain GPS ID information (gps_geom_parameter_set_id) used to identify each GPS2011.

[0076] also, Figure 4 The descriptor field indicates how each syntax is encoded. ue(v) represents a codeless zero-order exponent Golomb code, and u(1) represents a 1-bit flag.

[0077] GPS2011 may include a flag (gps_predictive_mode_enabled_flag) for controlling whether predictive coding is used by the tree synthesis unit 2020.

[0078] For example, when the value of gps_predictive_mode_enabled_flag is "1", it can be defined as using predictive coding, and when the value of gps_predictive_mode_enabled_flag is "0", it can be defined as not using predictive coding.

[0079] GPS2011 may include a flag (geometry_planar_mode_flag) for controlling whether the tree synthesis unit 2020 uses planar mode.

[0080] For example, when the value of geometry_planar_mode_flag is "1", it can be defined as using planar mode, and when the value of geometry_planar_mode_flag is "0", it can be defined as not using planar mode.

[0081] GPS2011 may include a flag (geom_tree_coded_axis_list_present_flag) for controlling whether "QtBt" is performed by the tree synthesis unit 2020.

[0082] For example, if the value of geom_tree_coded_axis_list_present_flag is "1", it can be defined as performing "QtBt", and if the value of geom_tree_coded_axis_list_present_flag is "0", it can be defined as performing only "Octree".

[0083] Alternatively, with geom_tree_coded_axis_list_present_flag set to "1", it can be defined to also allow cubes other than cubes as node shapes (i.e., also use "QtBt"). With geom_tree_coded_axis_list_present_flag set to "0", it can be defined to only allow cubes as node shapes (i.e., only use "Octree").

[0084] In addition to the above definition, any definition that controls whether "QtBt" can be used will suffice.

[0085] Furthermore, geometry_planar_mode_flag and geom_tree_coded_axis_list_present_flag can be defined to decode only when predictive coding is not used.

[0086] (Tree Synthesis Department 2020)

[0087] use Figures 5-8 This illustrates an example of the processing in the Tree Synthesis Unit 2020. Figure 5 This is a flowchart illustrating an example of the processing of the Tree Synthesis Unit 2020.

[0088] like Figure 5 As shown, in step S501, the tree synthesis unit 2020 determines whether to apply predictive coding.

[0089] For example, the tree synthesis unit 2020 can refer to the value of gps_predictive_mode_enabled_flag to determine whether to apply predictive coding.

[0090] If the tree synthesis unit 2020 determines that predictive coding is to be applied, it proceeds to step S502; if it determines that predictive coding is not to be applied, it proceeds to step S504.

[0091] In step S502, the tree synthesis unit 2020 determines whether the decoding of the position information of all point cloud data contained in the corresponding slice is complete.

[0092] The processing in step S502 can, for example, involve transmitting information indicating the number of point cloud data contained in the slice to the GSH in advance, and comparing the number of point cloud data with the number of data that has been processed to determine whether the processing of all points is complete.

[0093] If the tree synthesis unit 2020 has completed decoding of the location information of all point cloud data (the location information of all points in the point cloud), it proceeds to step S507 and ends the process. On the other hand, if the tree synthesis unit 2020 has not completed decoding of the location information of all point cloud data (the location information of all points in the point cloud), it proceeds to step S503.

[0094] In step S503, the tree synthesis unit 2020 decodes the location information of a point in the point cloud data (the location information of a point in the point cloud) through predictive coding.

[0095] Predictive coding is a unit that decodes the position information of point cloud data by decoding the difference between the predicted position information based on the decoded point cloud data and the position information of the point cloud data, and then adding the two together. A specific method can be implemented, for example, by the method described in Non-Patent Document 2.

[0096] In step S503, the tree synthesis unit 2020 decodes the location information of one point in the point cloud data, and then returns to the processing in step S502.

[0097] When the tree synthesis unit 2020 determines that predictive coding is being applied, it can decode the location information of all point cloud data by repeatedly executing steps S502 and S503.

[0098] Here, the tree synthesis unit 2020 can assign an index to each point of the point cloud data obtained by decoding the position information in ascending order according to the decoding order in step S503. Moreover, the tree synthesis unit 2020 can store the index in a one-dimensional array data Order[i].

[0099] As described above, the tree synthesis unit 2020 can be configured to have a predictive coding unit that serves as a decoding method for location information, assign an index to each point of the point cloud data based on the decoding order in the predictive coding, and store the index in a one-dimensional array.

[0100] By setting the structure in this way, even if the LoD calculation unit 2090 described later does not perform sorting processing, the order of decoding the attribute information is uniquely determined, thus making the decoded position information uniquely correspond to the attribute information.

[0101] Next, the handling of cases where predictive coding is not applied will be explained.

[0102] In step S504, the tree synthesis unit 2020 confirms whether the tree structure using "Octree" or "QtBt" has completed all Depth processing.

[0103] The Depth value of the slice can be pre-transmitted by GSH, for example. The tree synthesis unit 2020 can determine whether all Depth processing has been completed by comparing the Depth value of the slice with the already processed Depth values.

[0104] If the tree synthesis unit 2020 has completed the processing of all Depths, it proceeds to step S507 and ends the processing. On the other hand, if the tree synthesis unit 2020 has not completed the processing of all Depths, it proceeds to step S505.

[0105] In step S505, the tree synthesis unit 2020 determines whether the processing of all nodes within the Depth is complete.

[0106] The number of nodes contained in this Depth can be calculated based on the result of the decoding process of the node information in the previously processed Depth (step S506). In addition, the number of nodes contained in the initially processed Depth is 1.

[0107] If the tree synthesis unit 2020 has completed the processing of all nodes contained in the Depth, it returns to the processing in step S504. On the other hand, if the tree synthesis unit 2020 has not completed the processing of all nodes, it proceeds to the processing in step S506.

[0108] In step S506, the tree synthesis unit 2020 divides the node into child nodes in an eight-segment manner when using "Octree" and divides the node into child nodes in any of the eight-segment, four-segment, or two-segment manner when using "QtBt". Based on information such as occupancy code, the occupancy map indicating whether point cloud data is contained in each child node is decoded.

[0109] For example, a value of "1" in the raster map can be defined as the existence of point cloud data within the corresponding child node. Conversely, a value of "0" in the raster map can be defined as the absence of point cloud data within the corresponding child node or the absence of the child node itself.

[0110] Therefore, the tree synthesis unit 2020 can determine the number of nodes in the next processing depth by counting the number of child nodes with a value of "1" in the raster map within the same depth. A specific example of the processing in step S506 will be described later.

[0111] After generating the raster map for a node, the tree synthesis unit 2020 returns to the processing in step S505.

[0112] As described above, the tree synthesis unit 2020 decodes the position information of the point cloud data by recursively segmenting nodes until the node size is 1×1×1 while repeating the processing of steps S504 to 506 without applying predictive coding.

[0113] Here, each point in the point cloud data obtained by decoding the position information can be assigned an index in ascending order according to the decoding order in step S506, and stored in a one-dimensional array data Order[i] according to the index order.

[0114] Furthermore, as described above, the tree synthesis unit 2020 can decode the position information of each point in the point cloud data of the decoding object, and assign an index to each point in ascending order according to the order of the decoded position information. Moreover, the tree synthesis unit 2020 can be configured to store the index in a one-dimensional array.

[0115] By setting the structure in this way, even if the LoD calculation unit 2090 described later does not perform sorting processing, the order of decoding the attribute information is uniquely determined, thus making the decoded position information uniquely correspond to the attribute information.

[0116] Figure 6 This is a flowchart illustrating an example of node information decoding processing in step S506.

[0117] In step S601, the tree synthesis unit 2020 determines whether a planar mode can be applied to each axis direction of the node (e.g., each of the x-axis, y-axis, and z-axis).

[0118] If the tree synthesis unit 2020 can apply the planar mode, it proceeds to step S602. On the other hand, if the tree synthesis unit 2020 cannot apply the planar mode, it proceeds to step S603.

[0119] The tree synthesis unit 2020 can, for example, determine whether a planar mode can be applied by first deriving an eligible_planar_flag that indicates whether a planar mode can be applied in each axial direction, and by referring to the value of the eligible_planar_flag.

[0120] For example, when the value of eligible_planar_flag is "1", it can be defined as being able to apply planar mode in this axis direction of the node, and when the value of eligible_planar_flag is "0", it can be defined as not being able to apply planar mode in this axis direction of the node.

[0121] Here, if the axis direction of this node is not divided by the application of "QtBt", the corresponding eligible_planar_flag value can be set to "0", that is, set to not apply planar mode.

[0122] That is, it can be configured to generate a flag indicating whether a planar pattern can be applied in each axial direction of the node. In the case that the node is split by a quadtree or a binary tree, the value of the flag indicating whether the corresponding planar pattern can be applied in the axial direction where no split has occurred is set to a value indicating that the planar pattern cannot be applied.

[0123] Here, various methods can be used to determine whether the axis direction of the node is segmented. For example, as shown in Non-Patent Document 1, it can be determined by default according to predetermined rules and node shape, or it can be explicitly determined on the point cloud encoding device 100 side and transmitted via bit stream.

[0124] For example, information about which axis direction to split in each Depth can be transmitted to GSH as a 3-bit syntax (each bit corresponding to a different axis direction). For example, it can be defined that axes with a bit value of "1" are split, and axes with a bit value of "0" are not split.

[0125] Here, for example, in the Depth to which the node belongs, if the bit representation corresponding to the axis direction is "0 (= "no split"), the value of the corresponding eligible_planar_flag can be set to "0", that is, set to not apply planar mode.

[0126] In step S602, the tree synthesis unit 2020 decodes the syntax associated with the planar pattern. For example, the tree synthesis unit 2020 decodes is_planar_flag and plane_position as described in Non-Patent Document 1. After decoding the syntax, the tree synthesis unit 2020 proceeds to step S603.

[0127] In step S603, the tree synthesis unit 2020 sets the mask plane.

[0128] The mask plane is an 8-bit bitmask. Each bit corresponds to a child node when the node is divided into 2 parts along each of the three axes (e.g., x-axis, y-axis, z-axis). For example, a bit value of "1" can be defined as the absence of a corresponding child node or the absence of point cloud data in the corresponding child node. Similarly, a bit value of "0" can be defined as the potential presence of point cloud data in the corresponding child node.

[0129] Here, the tree synthesis unit 2020 can set a mask plane in the axial direction of the segment that does not generate "QtBt" so that when viewed in the axial direction, the bit corresponding to the child node of the plane (Lower plane) that belongs to the coordinates closer to the origin (0) is "0", and the bit corresponding to the child node of the plane (Upper plane) that belongs to the coordinates farther from the origin is "1".

[0130] That is, the tree synthesis unit 2020 can be configured to generate mask planes corresponding to the eight child nodes of each node. When the node is divided by a quadtree or a binary tree, the value of the mask plane corresponding to the child node of the plane that belongs to the side of the undivided axis direction origin is set to "0", and the value of the mask plane corresponding to the child node of the plane that belongs to the side of the undivided axis direction origin is set to "1".

[0131] exist Figure 8 The specific example is shown below.

[0132] For example, such as Figure 8 As shown in (a), when the node is partitioned by Qt (quadtree) so that it is not partitioned in the z-axis direction, for the mask plane, as follows: Figure 8 As shown in (a), the bit corresponding to the Lower plane in the z-axis direction is "0", and the bit corresponding to the Upper plane in the z-axis direction is "1". Furthermore, in Figure 8 The labels a to h in (a) that are recorded as positions are... Figure 8 The positions of the child nodes recorded in (c) correspond to each other.

[0133] Similarly, as Figure 8 As shown in (b), when the Bt (divided by the bisection wood) is not divided into the x-axis and y-axis directions, only the bit corresponding to the child node of the Lower plane belonging to the x-axis and y-axis directions is "0", and all other bits are "1".

[0134] As described above, after setting the mask plane, the tree synthesis unit 2020 proceeds to step S604.

[0135] In step S604, the tree synthesis unit 2020 decodes the raster map. Here, the raster map is decoded using the mask plane described above and the occupancy code contained in the bit stream transmitted from the point cloud encoding device 100.

[0136] Similar to the mask plane described above, the raster map consists of 8 bits. Also as with the mask plane, each bit corresponds to a child node in the case of dividing the node into 2 parts along three axes (e.g., x-axis, y-axis, z-axis).

[0137] Here, the tree synthesis unit 2020 decodes the corresponding bit of the raster map of the child node whose value is "1" in the mask plane as "0" (no point cloud data exists in this child node).

[0138] On the other hand, the tree synthesis unit 2020 decodes the occupancy code of the child node whose value is "0" in the mask plane. When the value of the occupancy code is "1 (point cloud data exists in the child node)," the corresponding raster map bit is set to "1"; when the value of the occupancy code is "0 (point cloud data does not exist in the child node)," the corresponding raster map bit is set to "0."

[0139] That is, the tree synthesis unit 2020 can be configured to decode the raster map indicating whether point cloud data exists in each child node of the node. For the child node whose mask plane value is "1", the value of the corresponding raster map is set to "0". For the child node whose mask plane value is "0", the value of the raster map is set with reference to the value of the occupancy code contained in the bit stream.

[0140] With this structure, the control of "QtBt" can be implemented using a mask plane set up for planar mode, thus simplifying the design of the specification and the device based on this specification.

[0141] Furthermore, the values ​​of the mask plane described above are just one example, and the meanings of the "0" and "1" bits described above can be changed.

[0142] As described above, the tree synthesis unit 2020 is capable of decoding the 8-bit raster map corresponding to each node. After decoding the raster map, the tree synthesis unit 2020 proceeds to step S605 and ends the processing.

[0143] In addition, the above steps can also be performed as follows: Figure 7 The order of the changes is as described in the document. Specifically, the tree synthesis unit 2020 can first perform mask plane setting and raster map decoding, and then perform a determination of whether the plane mode can be used and a decoding process of the syntax associated with the plane mode.

[0144] In this case, the determination of whether a planar pattern can be utilized and the decoding of the syntax associated with the planar pattern are performed separately for each child node of the node, rather than for the node itself. Therefore, step S701 is added, and the tree synthesis unit 2020 performs loop processing until the processing of all child nodes that have been decoded from the raster map to have point cloud data is completed.

[0145] (Attribute Information Decoding Unit 2060)

[0146] The following uses Figure 9 and Figure 10 The control data decoded by the attribute information decoding unit 2060 will be explained.

[0147] Figure 9 This is an example of the structure of encoded data (bit stream) received by the attribute information decoding unit 2060.

[0148] First: Bitstreams can also contain APS2061. APS2061, also known as an attribute parameter group, is a collection of control data related to the decoding of attribute information. Specific examples will be explained later. Each APS2061 contains at least an APS ID, used to identify each one when multiple APS2061s exist.

[0149] Second: The bitstream can also contain ASH2062A / 2062B. ASH2062A / 2062B, also known as the attribute slice header, is a collection of control data corresponding to the slices described later. Specific examples will be explained later. ASH2062A / 2062B at least contains APS id information to specify the APS2061 corresponding to each ASH2062A / 2062B.

[0150] Third: The bitstream can also include slice data 2063A / 2063B after ASH2062A / 2062B. Slice data 2063A / 2063B contains data encoded with attribute information.

[0151] As described above, the bit stream becomes a structure where ASH2062A / 2062B and APS2061 correspond to each slice data 2063A / 2063B, respectively.

[0152] As described above, since the APS id information in ASH2062A / 2062B specifies which APS2061 to reference, it is possible to use a common APS2061 for multiple slice data 2063A / 2063B.

[0153] In other words, APS2061 does not necessarily need to transmit every slice. For example, as Figure 9 As shown, it can also be configured as a bitstream structure where APS2061 is not encoded before ASH2062B and slice data 2063B.

[0154] also, Figure 9 The structure described above is just one example. Any structure corresponding to ASH2062A / 2062B and APS2061 with slice data 2063A / 2063B can also have additional elements added as constituent elements of the bit stream. For example, the bit stream can also contain sequence parameter sets (SPS).

[0155] Similarly, it can also be shaped into AND during transmission. Figure 9 Different structures are possible. Furthermore, the bitstream decoded by the geometric information decoding unit 2010 can be combined with the bitstream to form a single bitstream for transmission. For example, slice data 2013A and 2063A, and slice data 2013B and 2063B can be processed as single slice data, resulting in a structure where GSH2012A and ASH2062A, and GSH2012B and ASH2062B are configured before each slice, respectively. Additionally, GPS2011 and APS2061 can be configured before each GSH and ASH in this case.

[0156] Figure 10 This is an example of the syntax structure of APS2061.

[0157] APS2061 may contain APS id information (aps_attr_parameter_set_id) used to identify each APS2061.

[0158] APS2061 may include information (attr_coding_type) representing the decoding method of attribute information. For example, it may be specified that when the value of attr_coding_type is "0", variable weighted boosting prediction is performed in the inverse boosting unit 2100; when the value of attr_coding_type is "1", RAHT is performed in the RAHT unit 2080; and when the value of attr_coding_type is "2", boosting prediction with fixed weights is performed in the inverse boosting unit 2100.

[0159] APS2061 may include a flag indicating whether to apply scalable lifting when the value of attr_coding_type is "2", i.e., when lifting prediction is performed in the inverse lifting unit 2100 under fixed weights.

[0160] It can be specified that scalability boosting is not applied when the value of lifting_scalability_enabled_flag is "0", and it can be specified that scalability boosting is applied when the value of lifting_scalability_enabled_flag is "1".

[0161] Alternatively, it can be specified that when the value of lifting_scalability_enabled_flag is "1", that is, when applying scalable enhancements, skipOctreeLayers can be obtained as an external parameter when the point cloud decoding device 200 is executed, as described above.

[0162] Additionally, it can be specified that when the above "QtBt" is applied, that is, when the value of geom_tree_coded_axis_list_present_flag is "1", scalability enhancement will not be applied, i.e., the value of lifting_scalability_enabled_flag must be "0".

[0163] That is, the geometric information decoding unit 2010, which decodes the flag indicating whether "QtBt" is applied to the control, and the attribute information decoding unit 2060, which decodes the flag indicating whether "scalable enhancement" is applied to the control, can also be limited to not applying "scalable enhancement" when "QtBt" is applied.

[0164] In this way, by limiting the use of "QtBt" in an exclusive manner and by making it extensible, the combination of features can be reduced, thus making the specification simple and easy to install.

[0165] For example, in the case of scalable enhancement, by restricting it to a certain application "Octree", in the processing of scalable enhancement, the node shape must be a cube, and the processing can be performed on the premise that each partition must be an octree partition. Therefore, compared with the case where the node shape is a cuboid and the partition types are allowed to be quadtree and binary tree partitions in addition to octree partitions, the specification can be simplified.

[0166] APS2061 may include a syntax (lifting_num_detail_levels_minus1) that specifies the number of LoD levels calculated by the LoD calculation unit 2090. Since the value of LoD needs to be an integer value greater than 1, this syntax can also be an integer value greater than 0, and the value of this syntax plus "1" is used as the number of LoD levels.

[0167] APS2061 may include a flag (lifting_morton_sort_skip_enabled_flag) that controls whether the LoD computing unit 2090 performs sorting processing.

[0168] For example, it can be defined that sorting is performed when the value of lifting_morton_sort_skip_enabled_flag is "0", and sorting is not performed when the value of lifting_morton_sort_skip_enabled_flag is "1".

[0169] Alternatively, for example, it can be defined that APS2061 includes the lifting_morton_sort_skip_enabled_flag only if the value of lifting_num_detail_levels_minus1 mentioned above is "0", that is, if the number of LoD levels is "1".

[0170] You can define a constraint that the value of lifting_morton_sort_skip_enabled_flag must be "0" when the value of gps_predictive_mode_enabled_flag is "0", that is, when predictive coding is not applied.

[0171] That is, the geometric information decoding unit 2010, which decodes the flag indicating whether predictive coding can be applied, and the attribute information decoding unit 2060, which decodes the flag indicating whether to perform sorting processing based on the position information after decoding the point cloud data before decoding the attribute information of the point cloud data of the decoding object, can be configured to have a restriction that when the flag indicating whether predictive coding can be applied indicates that predictive coding cannot be applied, the value of the flag indicating whether to perform sorting processing must be set to a value indicating a certain "sorting processing" value.

[0172] Furthermore, the "no sorting process" function in the attribute information decoding enables low-latency processing on the point cloud encoding device 100 side (shortening the delay time from the start of the input point cloud data of the encoded object until the corresponding point cloud data is output to the bit stream). This function is particularly effective when combined with predictive encoding that sequentially decodes position information.

[0173] Therefore, by limiting the "do not perform sorting" function to only be combined with predictive coding, the effectiveness of the function can be maintained while being easy to design and verify.

[0174] (LoD Computing Department 2090)

[0175] The following uses Figure 11 Here is an example of the processing content of the LoD computing unit 2090.

[0176] Figure 11 This is an example of a flowchart for the LoD computing unit 2090.

[0177] In step S1101, the LoD calculation unit 2090 performs a process (sorting) to rearrange the position information of each point in the point cloud output from the geometric information reconstruction unit 2040.

[0178] Specifically, for example, the LoD calculation unit 2090 can update the one-dimensional array data Order[i], which is an input of the index of the location information of each point cloud data generated by the tree synthesis unit 2020 and stored in ascending order, according to the result of sorting based on the location information. The LoD calculation unit 2090 can also sort the location information in ascending order, for example, according to the Morton code that can be generated from the location information.

[0179] Alternatively, it can be configured so that when the flag controlling whether to perform sorting processing (lifting_morton_sort_skip_enabled_flag) is obtained, indicating that sorting processing is not performed, for example, in the example above, when the value of lifting_morton_sort_skip_enabled_flag is "1", the LoD calculation unit 2090 does not perform the sorting processing.

[0180] It can be stipulated that when the above sorting process is not performed, the index of each point in the point cloud data is granted according to the decoding order of the tree synthesis unit 2020. Moreover, it can be stipulated that the index of each point in the point cloud data is stored in a one-dimensional array data Order[i] in ascending order of the index.

[0181] It can be stipulated that, if the above sorting process is not performed and the tree synthesis unit 2020 determines that predictive coding is applied, the index of each point in the point cloud data is assigned according to the decoding order of the predictive coding in the tree synthesis unit 2020. Moreover, it can be stipulated that the index of each point in the point cloud data is stored in a one-dimensional array data Order[i] in ascending order of the index.

[0182] That is, the LoD calculation unit 2090 can be configured to determine whether to perform sorting based on the position information after decoding the point cloud data of the decoded object before decoding the attribute information. If sorting is not performed, the attribute information is decoded based on the order of the indices stored in the one-dimensional array.

[0183] With this structure, even without performing the sorting process described above, the order in which the attribute information is decoded is uniquely determined, thus establishing a unique correspondence between the decoded position information and the attribute information.

[0184] As described above, after performing or omitting the sorting process described above, the LoD calculation unit 2090 proceeds to step S1102.

[0185] In step S1102, the LoD calculation unit 2090 calculates the LoD of the location information of each point. Known methods can be used to calculate the LoD, therefore detailed explanation is omitted.

[0186] After calculating the LoD of each point cloud data, the LoD calculation unit 2090 proceeds to step S1103.

[0187] In step S1103, the LoD calculation unit 2090 performs a k-nearest neighbor search on each node, targeting nodes whose LoD is higher than that node's. The specific k-nearest neighbor search method can be any known method, therefore a detailed explanation is omitted.

[0188] After performing k-nearest neighbor search on all nodes, the LoD calculation unit 2090 proceeds to step S1104 and ends the process.

[0189] In addition, the point cloud encoding device 100 and point cloud decoding device 200 described above can also be implemented by a program that enables a computer to execute each function (each process).

[0190] Furthermore, in the above embodiments, the present invention has been described using the application of the present invention to the point cloud encoding device 100 and the point cloud decoding device 200 as examples. However, the present invention is not limited to the examples described, and can also be applied to point cloud encoding / decoding systems having the functions of the point cloud encoding device 100 and the point cloud decoding device 200.

[0191] Symbol Explanation

[0192] 10: Point cloud processing system; 100: Point cloud encoding device; 200: Point cloud decoding device; 2010: Geometric information decoding unit; 2020: Tree synthesis unit; 2030: Approximate surface synthesis unit; 2040: Geometric information reconstruction unit; 2050: Inverse coordinate transformation unit; 2060: Attribute information decoding unit; 2070: Inverse quantization unit; 2080: RAHT unit; 2090: LoD calculation unit; 2100: Inverse lifting unit; 2110: Inverse color transformation unit.

Claims

1. A point cloud decoding device, characterized in that, have: The tree synthesis unit is configured to receive an encoded bitstream including occupancy code syntax, decode the position information of each point in the point cloud data that is the object of decoding from the bitstream, and assign indexes to each point in the point cloud data in ascending order according to the decoding order of the position information.

2. The point cloud decoding device according to claim 1, characterized in that, The tree synthesis part is configured to store the index in a one-dimensional array.

3. The point cloud decoding device according to claim 1 or 2, characterized in that, The tree synthesis unit is configured to decode the location information through predictive coding, and assign the index to each point of the point cloud data based on the decoding order of the location information in the predictive coding.

4. The point cloud decoding device according to any one of claims 1 to 2, characterized in that, The device includes a LoD (LoD) calculation unit configured to determine whether to perform a process of sorting the points of the point cloud data based on the location information before decoding the attribute information of each point in the point cloud data. If the sorting process is not performed, the attribute information is decoded based on the order of the index.

5. The point cloud decoding apparatus according to any one of claims 3, characterized in that, The device includes a LoD (LoD) calculation unit configured to determine whether to perform a process of sorting the points of the point cloud data based on the location information before decoding the attribute information of each point in the point cloud data. If the sorting process is not performed, the attribute information is decoded based on the order of the index.

6. The point cloud decoding device according to claim 3, characterized in that, have: A geometric information decoding unit is configured to decode a flag indicating whether the predictive coding can be applied; and The attribute information decoding unit is configured to decode a flag that controls whether to perform processing based on the location information to sort the points of the point cloud data before decoding the attribute information of each point in the point cloud data. When the value of the flag controlling whether the predictive coding can be applied indicates that predictive coding cannot be applied, the value of the flag controlling whether the sorting process is performed is set to a value indicating that the sorting process is performed.

7. The point cloud decoding device according to claim 6, characterized in that, have: The LoD calculation unit is configured to decode the attribute information based on the index order if the sorting process is not performed based on the value of a flag that controls whether the sorting process is performed.

8. A point cloud decoding method, characterized in that, have: The process of decoding the position information of each point in point cloud data, which is the object of decoding, from an encoded bit stream including occupancy code syntax; The process of assigning indexes of each point in the point cloud data in ascending order according to the decoding sequence of the location information; as well as The process of storing the index in a one-dimensional array.

9. A computer program product comprising a program used by a point cloud decoding device, characterized in that, The program causes the computer to execute: The process of decoding the position information of each point in point cloud data, which is the object of decoding, from an encoded bit stream including occupancy code syntax; The process of assigning indexes of each point in the point cloud data in ascending order according to the decoding sequence of the location information; as well as The process of storing the index in a one-dimensional array.

10. A computer-readable storage medium storing a program for causing a processor to execute the point cloud decoding method according to claim 8.

Images