Decoding method, encoding method, decoding device, and encoding device

Abstract

A decoding method receives a bit stream in which a plurality of three-dimensional points each having first attribute information and second attribute information are encoded (S401), and predicts the first attribute information with reference to the second attribute information (S402). For example, the predicted first attribute information may have the same three-dimensional point as the referenced second attribute information. For example, the predicted first attribute information and the referenced second attribute information may have different three-dimensional points. For example, the first attribute information and the second attribute information may be stored in the first component and the second component, respectively.

Description

Technical Field

[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device. Background Art

[0002] In a wide range of fields such as computer vision, map information, surveillance, infrastructure inspection, or image distribution, which are used for autonomous operation of cars or robots, devices and services that effectively utilize three-dimensional data are expected to become more widespread in the future. Three-dimensional data is obtained by various methods such as distance sensors such as laser rangefinders, stereo cameras, or a combination of multiple monocular cameras.

[0003] As one of the methods of expressing three-dimensional data, there is a method called point cloud that represents the shape of a three-dimensional structure by a group of points in a three-dimensional space. In the point cloud, the position and color of the point group are stored. Although it is predicted that point cloud will become the mainstream method of expressing three-dimensional data, the amount of data of the point group is very large. Therefore, in the accumulation or transmission of three-dimensional data, like two-dimensional dynamic images (for example, there are MPEG4AVC or HEVC standardized by MPEG), compression based on the amount of encoded data must be performed.

[0004] In addition, compression of point clouds is partially supported by a public library (Point Cloud Library) that performs point cloud association processing.

[0005] In addition, there is known a technology for searching and displaying facilities located around a vehicle using three-dimensional map data (for example, refer to Patent Document 1).

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: International Publication No. 2014 / 020663 Summary of the invention

[0009] Problems to be solved by the invention

[0010] In the encoding and decoding of three-dimensional data, it is desirable to reduce the amount of encoded data.

[0011] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device capable of reducing the amount of encoded data.

[0012] Means for solving problems

[0013] A decoding method according to one aspect of the present disclosure receives a bit stream in which a plurality of three-dimensional points each having first attribute information and second attribute information are encoded, and predicts the first attribute information with reference to the second attribute information.

[0014] A coding method according to one embodiment of the present invention is a coding method for generating a bit stream by encoding a plurality of three-dimensional points each having first attribute information and second attribute information, predicting the first attribute information with reference to the second attribute information, and encoding the first attribute information using the predicted result.

[0015] Effects of the Invention

[0016] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can reduce the amount of encoded data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a diagram showing an example of the structure of point cloud data in Implementation Example 1.

[0018] Figure 2 This is a block diagram showing the structure of the three-dimensional data encoding device according to the first embodiment.

[0019] Figure 3 This is a flowchart of the encoding process of attribute information in implementation mode 1.

[0020] Figure 4 This is a block diagram showing the structure of the three-dimensional data decoding device according to the first embodiment.

[0021] Figure 5 This is a flowchart of the decoding process of attribute information in the first embodiment.

[0022] Figure 6 This is a block diagram showing the structure of the attribute information encoding unit according to the first embodiment.

[0023] Figure 7 This is a block diagram showing the structure of the attribute information decoding unit according to the first embodiment.

[0024] Figure 8 This is a diagram showing a reference relationship in the first example of the prediction process according to the first embodiment.

[0025] Fig. 9 This is a diagram showing a reference relationship in the second example of the prediction process according to the first embodiment.

[0026] Fig.10 This is a diagram for explaining the prediction process based on the prediction mode according to the first embodiment.

[0027] Fig.11This is a diagram showing the structure of TLV data in the first embodiment.

[0028] Fig.12 This is a diagram showing the structure of a bit stream in implementation mode 1.

[0029] Fig.13 This is embodiment 1 Fig.13 This is a diagram showing a syntax example of a TLV unit (TLV_unit).

[0030] Fig.14 This is a diagram showing a syntax example of attribute_parameter in implementation mode 1.

[0031] Fig.15 This is a diagram showing a syntax example of a subordinate attribute data unit header according to the first embodiment.

[0032] Fig.16 This is a diagram showing a syntax example of a dependent attribute data unit according to the first embodiment.

[0033] Fig.17 This is a diagram showing a syntax example of a dependent attribute data unit according to the first embodiment.

[0034] Fig.18 This is a flowchart of the encoding process of attribute information in implementation mode 1.

[0035] Fig.19 This is a flowchart of the decoding process of attribute information in the first embodiment.

[0036] Fig. 20 This is a diagram showing a syntax example of frame_attr_param_SEI according to the first embodiment.

[0037] Fig.21 This is a diagram showing a syntax example of angle_partial_access_SEI according to the first embodiment.

[0038] Fig. 22 This is a flowchart of the decoding process when partial decoding is performed in the first embodiment.

[0039] Fig.23 This is a block diagram showing the structure of a three-dimensional data encoding device according to the second embodiment.

[0040] Fig.24 This is a diagram showing an example of merging sub-components in implementation mode 2.

[0041] Fig.25 This is a block diagram showing the structure of a three-dimensional data decoding device according to the second embodiment.

[0042] Fig.26 This is a diagram showing an example of a reference relationship in inter-sub-component prediction according to the second embodiment.

[0043] Fig. 27 This is a diagram showing an example of a reference relationship in inter-sub-component prediction according to the second embodiment.

[0044] Fig.28 This is a diagram showing an example of a reference relationship in inter-sub-component prediction according to the second embodiment.

[0045] Fig.29 This is a flowchart of the attribute information encoding process according to the second embodiment.

[0046] Fig.30 This is a flowchart of the decoding process of attribute information in the second embodiment.

[0047] Fig.31 is a flowchart of the decoding process.

[0048] Fig.32 is a flowchart of the encoding process. DETAILED DESCRIPTION

[0049] A three-dimensional data decoding method according to one embodiment of the present disclosure receives a bit stream formed by encoding a plurality of three-dimensional points each having first attribute information and second attribute information, and predicts the first attribute information with reference to the second attribute information. Sometimes there is a correlation between the attribute information. Therefore, by predicting the first attribute information with reference to the second attribute information, it is possible to reduce the amount of data of the first attribute information. Thus, the amount of data processed by the decoding device can be reduced.

[0050] A three-dimensional data decoding method according to one embodiment of the present disclosure receives a bit stream and decodes the bit stream, wherein the bit stream includes: encoded attribute information of a plurality of three-dimensional points, each of the plurality of three-dimensional points having first attribute information and second attribute information; and meta-information indicating that the first attribute information is predicted with reference to the second attribute information. Sometimes there is a correlation between attribute information. Therefore, by predicting the first attribute information with reference to the second attribute information, it is possible to reduce the amount of data of the first attribute information. The decoding device uses the meta-information to appropriately decode the bit stream with the reduced amount of data. In addition, the amount of data processed by the decoding device can be reduced.

[0051] For example, the first attribute information to be predicted and the second attribute information to be referenced may be possessed by the same three-dimensional point. By predicting the first attribute information of the three-dimensional point with reference to the second attribute information of the three-dimensional point, it is possible to reduce the amount of data of the first attribute information. Thus, the amount of data processed by the decoding device can be reduced.

[0052] For example, the first attribute information to be predicted and the second attribute information to be referred to may be possessed by different three-dimensional points.

[0053] For example, the first attribute information and the second attribute information may be stored in a first component and a second component, respectively.

[0054] For example, the first quantization step size used for the first attribute information may be larger than the second quantization step size used for the second attribute information. By using a smaller quantization step size for the second attribute information referenced by other attribute information, degradation of the second attribute information can be suppressed, and the accuracy of prediction of the first attribute information that references the second attribute information can be improved. In addition, by using a larger quantization step size for the first attribute information, the amount of data can be reduced. Thus, the coding efficiency can be improved while degradation of the information as a whole can be suppressed.

[0055] For example, the first attribute information and the second attribute information may be generated by respectively decoding the first coded attribute information and the second coded attribute information, and the second attribute information may be losslessly compressed. By using lossless compression on the second attribute information referenced by other attribute information, degradation of the second attribute information can be suppressed, and the accuracy of prediction of the first attribute information that references the second attribute information can be improved. Thus, the coding efficiency can be improved while degradation of the information as a whole can be suppressed.

[0056] For example, each of the first component and the second component may include a first dimensional element and a second dimensional element, the first dimensional element of the first component is predicted with reference to the first dimensional element of the second component, and the second dimensional element of the first component is predicted with reference to the second dimensional element of the second component. Thus, when there is correlation between the same dimensional elements between components, the amount of data can be reduced by utilizing the correlation.

[0057] For example, the first component may include a first dimensional element and a second dimensional element, the first dimensional element may be predicted with reference to the second component, and the second dimensional element may be predicted with reference to the first dimensional element. Depending on the attribute information, sometimes there is a correlation between different components, and sometimes there is a correlation between the dimensional elements constituting the component. Therefore, by using the correlation between components and the correlation between dimensional elements, the amount of data can be reduced.

[0058] For example, the bitstream may include information about the second prediction mode applied to the second component instead of the first prediction mode applied to the first component. This can reduce the amount of data because the information about the first component can be reduced.

[0059] For example, the bitstream may include a first residual value of the first attribute information, the first residual value is a difference between a third residual value of the first attribute information and a second residual value of the second attribute information, the third residual value is a difference between a first value of the first attribute information and a first predicted value of the first attribute information, and the second residual value is a difference between a second value of the second attribute information and a second predicted value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the residual of the first attribute information and the residual of the second attribute information.

[0060] For example, the bitstream may include a first residual value of the first attribute information, the first residual value being a difference between a first value of the first attribute information and a second value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the value of the first attribute information and the value of the second attribute information.

[0061] For example, the bitstream may include a first residual value of the first attribute information, the first residual value being the difference between the first value of the first attribute information and the second predicted value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the value of the first attribute information and the predicted value of the second attribute information. In addition, since there is no need to calculate the predicted value of the first attribute information, the amount of processing can be reduced.

[0062] For example, the bitstream may include flag information indicating whether the second attribute information can be referenced in the prediction of the first attribute information. Thus, since the decoding device can refer to the flag information to determine whether the second attribute information can be referenced in the prediction of the first attribute information, the bitstream can be decoded appropriately.

[0063] For example, the bitstream may include coefficient information for calculating the first predicted value of the first attribute information. By using the coefficient information, the amount of residual data can be reduced.

[0064] For example, the bitstream may include a first data unit storing attribute information predicted with reference to other attribute information and a second data unit storing attribute information not predicted with reference to other attribute information.

[0065] For example, the first attribute information may include RGB values, the second attribute information may include reflectance values, and at least one of the RGB values ​​may be predicted with reference to the reflectance values. Thus, the amount of data can be reduced by utilizing the correlation between color and reflectance, which are usually correlated.

[0066] The encoding method of this embodiment is an encoding method for generating a bit stream by encoding a plurality of three-dimensional points each having first attribute information and second attribute information, predicting the first attribute information with reference to the second attribute information, and encoding the first attribute information using the predicted result. Sometimes there is a correlation between attribute information. Therefore, by predicting the first attribute information with reference to the second attribute information, the amount of data of the first attribute information can be reduced.

[0067] In addition, a decoding device according to one embodiment of the present invention comprises a processor and a memory, wherein the processor uses the memory to receive a bit stream obtained by encoding a plurality of three-dimensional points each having first attribute information and second attribute information, and predicts the first attribute information with reference to the second attribute information.

[0068] In addition, an encoding device of one embodiment of the present invention is an encoding device that generates a bit stream by encoding multiple three-dimensional points, each of which has first attribute information and second attribute information, and comprises: a processor; and a memory; the processor uses the memory to predict the first attribute information with reference to the second attribute information, and uses the predicted result to encode the first attribute information.

[0069] In addition, these general or specific methods can be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, or by any combination of systems, methods, integrated circuits, computer programs, and recording media.

[0070] Hereinafter, the embodiments are described in detail with reference to the accompanying drawings. In addition, the embodiments described below all represent a specific example of the present disclosure. The numerical values, shapes, materials, constituent elements, configuration positions of constituent elements, connection methods, steps, order of steps, etc. shown in the following embodiments are examples and do not limit the subject matter of the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements not recorded in the independent claims are described as arbitrary constituent elements.

[0071] (Implementation Method 1)

[0072] Hereinafter, a three-dimensional data encoding device and a three-dimensional data decoding device according to this embodiment will be described. The three-dimensional data encoding device generates a bit stream by encoding three-dimensional data. The three-dimensional data decoding device generates three-dimensional data by decoding the bit stream.

[0073] Three-dimensional data is, for example, three-dimensional point group data (also referred to as point group data). A point group is a collection of multiple three-dimensional points, representing the three-dimensional shape of an object. Point group data includes position information and attribute information of multiple three-dimensional points. The position information represents the three-dimensional position of each three-dimensional point. In addition, position information is sometimes referred to as geometry information. For example, position information is represented by an orthogonal coordinate system or a polar coordinate system.

[0074] The attribute information indicates, for example, color information, reflectance, transmittance, infrared information, normal vector, time information, etc. One three-dimensional point may have a single type of attribute information or may have multiple types of attribute information.

[0075] The attribute information has different dimensions depending on the type of attribute. For example, color information has three elements (three dimensions) such as RGB or YCbCr. Reflectivity has one element (one dimension). In addition, attribute information has different restrictions such as the range of possible values, bit depth, or whether it can take negative values, depending on the type of attribute.

[0076] In addition, the multiple types of attribute information are not limited to types with different information categories such as color information and reflectivity, but also include types with different formats such as RGB and YCbCr, and types with different information contents such as color information when observed from different viewpoints. In addition, in the following, different types of attribute information may be simply recorded as different attribute information, etc.

[0077] In addition, three-dimensional data is not limited to point group data, and can also be other three-dimensional data such as mesh data. Mesh data (also called three-dimensional mesh data) is a data format used by CG (Computer Graphics) to represent the three-dimensional shape of an object as a collection of surface information. For example, mesh data contains point group information (such as vertex information). Therefore, for this point group information, the same method as the method corresponding to the point group data can be applied.

[0078] In conventional three-dimensional data encoding and decoding systems, a plurality of types of attribute information are encoded and decoded independently.

[0079] In this embodiment, when encoding a plurality of different attribute information, the three-dimensional data encoding device refers to other attribute information when encoding a certain attribute information, thereby being able to reduce the amount of encoding.

[0080] For example, the three-dimensional data encoding device sets the first attribute information as reference attribute information and the second attribute information as subordinate attribute information. The three-dimensional data encoding device first encodes the first attribute information, and then, in the encoding of the second attribute information, makes a prediction with reference to the encoding result of the first attribute information. In this way, the amount of encoding can be reduced. The encoding result of the first attribute information is, for example, a value after the first attribute information is reconstructed, specifically, a value obtained by decoding the encoded first attribute information. In addition, the three-dimensional data encoding device generates metadata indicating that the first attribute information is reference attribute information and the second attribute information is subordinate attribute information, and stores the metadata in the bitstream.

[0081] The three-dimensional data decoding device obtains metadata from the bit stream. In addition, the three-dimensional data decoding device first decodes the first attribute information, and then performs prediction by referring to the decoding result of the first attribute information when decoding the second attribute information.

[0082] Furthermore, the generation and transmission of metadata may be omitted, and the three-dimensional data encoding device and the three-dimensional data decoding device may fixedly process that the first attribute information is the reference attribute information.

[0083] More specifically, for example, when a three-dimensional point is observed from a plurality of viewpoints, since the illumination method and observation of light differ at each viewpoint, the three-dimensional point has a different color value at each viewpoint. Figure 1 is a diagram showing an example of the structure of such point cloud data. Figure 1 As shown, each point has position information, first color information and second color information. Here, the first color information and the second color information are attribute information, for example, color information corresponding to different viewpoints.

[0084] In such a point group, the correlation between the first color information and the second color information may be high. Therefore, when the second color information is encoded using the above method, differential information with a small data size may be generated by using the first color information for prediction. As a result, the amount of encoding may be reduced.

[0085] Furthermore, the 3D data encoding device transmits an identifier indicating that the first attribute information is reference attribute information and the second attribute information is dependent attribute information. Thus, the 3D data decoding device can obtain the identifier from the bit stream and perform decoding based on the identifier to determine the dependent relationship.

[0086] The following describes the configurations of the three-dimensional data encoding device and the three-dimensional data decoding device according to the present embodiment. Figure 23D data encoding device 100 is a block diagram showing the structure of the present embodiment. 3D data encoding device 100 generates encoded attribute information and dependency information by encoding a plurality of attribute information (first to second attribute information). 3D data encoding device 100 includes attribute information encoding unit 101. Figure 2 , only the processing unit for encoding the attribute information is described. The three-dimensional data encoding device 100 may include other processing units such as a position information encoding unit for encoding the position information.

[0087] Figure 3 1 is a flowchart of the encoding process of attribute information performed by the three-dimensional data encoding device 100. First, the attribute information encoding unit 101 encodes the first attribute information (S101). Here, the attribute information encoding unit 101 encodes the plurality of attribute information into one attribute component respectively. For example, the attribute information encoding unit 101 encodes the first attribute information into the first component and encodes the second attribute information into the second component.

[0088] In addition, after encoding the first attribute information, the attribute information encoding unit 101 encodes the second attribute information with reference to the first attribute information (S102). Specifically, the attribute information encoding unit 101 calculates the predicted value by making a prediction with reference to the encoded information of the first attribute information. The attribute information encoding unit 101 encodes the second attribute information using the predicted value. For example, the attribute information encoding unit 101 calculates the difference between the second attribute information and the predicted value, that is, the predicted residual (residual value). In addition, the attribute information encoding unit 101 generates the encoded attribute information by performing arithmetic coding (entropy coding) on ​​the predicted residual. In addition, arithmetic coding may not be performed.

[0089] The encoding information of the first attribute information is, for example, a reconstructed value of the first attribute information, specifically, a value obtained by decoding the encoded first attribute information. Alternatively, the encoding information of the first attribute information may be information used for encoding the first attribute information.

[0090] Here, first attribute information referenced by other attribute information is referred to as reference attribute information, and attribute components including reference attribute information are referred to as reference components. In addition, second attribute information that refers to other attribute information is referred to as dependent attribute information, and attribute components including dependent attribute information are referred to as dependent components.

[0091] Furthermore, the attribute information coding unit 101 stores dependency information indicating that the second attribute information is dependent on the first attribute information in the bitstream (coded data) (S103). In other words, the dependency information is meta-information indicating that the second attribute information is predicted with reference to the first attribute information.

[0092] Figure 43D data decoding device 200 is a block diagram showing the structure of the present embodiment. 3D data decoding device 200 generates a plurality of attribute information (first to second attribute information) by decoding the coded attribute information and dependency information contained in the bit stream. The 3D data decoding device 200 includes an attribute information decoding unit 201. Figure 4 , only the processing unit for decoding the attribute information is described. The three-dimensional data decoding apparatus 200 may include other processing units such as a position information decoding unit for decoding the position information.

[0093] Figure 5 2 is a flowchart of a decoding process of attribute information performed by the three-dimensional data decoding device 200. The attribute information decoding unit 201 decodes a plurality of attribute information. Specifically, the attribute information decoding unit 201 obtains dependency information indicating that the second attribute information is a subordinate of the first attribute information from the bit stream (S111). Next, the attribute information decoding unit 201 decodes the first attribute information and the second attribute information based on the dependency information.

[0094] Specifically, the attribute information decoding unit 201 decodes the first attribute information (S112). Next, the attribute information decoding unit 201 decodes the second attribute information with reference to the decoded information of the first attribute information (S113). Specifically, the attribute information decoding unit 201 calculates the predicted value by predicting with reference to the decoded information of the first attribute information. The attribute information decoding unit 201 decodes the second attribute information using the predicted value. For example, the attribute information decoding unit 201 obtains the prediction residual (difference) between the second attribute information and the predicted value from the bit stream. In addition, when the second attribute information is arithmetic-coded, the attribute information decoding unit 201 obtains the prediction residual by entropy decoding the coded attribute information. The attribute information decoding unit 201 restores the second attribute information by adding the obtained prediction residual to the predicted value.

[0095] Furthermore, the decoded information of the first attribute information is, for example, a value obtained by decoding (reconstructing) the first attribute information. Furthermore, the decoded information of the first attribute information may be information used for decoding the first attribute information.

[0096] Next, the configuration of the attribute information encoding unit 101 and the process of encoding attribute information will be described. Here, an example will be described in which the attribute information encoding unit 101 is an encoding unit that performs encoding using LoD (LoD based attribute encoder).

[0097] Figure 61 is a block diagram showing the configuration of the attribute information coding unit 101. The attribute information coding unit 101 includes a LoD generation unit 111, a neighbor search unit 112, a prediction unit 113, a residual calculation unit 114, a quantization unit 115, an inverse quantization unit 116, a reconstruction unit 117, a memory 118, and an arithmetic coding unit 119.

[0098] The LoD generation unit 111 generates a LoD hierarchy using position (geometry) information of a plurality of three-dimensional points. The neighbor search unit 112 searches for three-dimensional points (neighboring points) adjacent to the object point to be processed using the LoD generation result and distance information between three-dimensional points.

[0099] The prediction unit 113 generates a predicted value of the attribute information of the object point. Specifically, the prediction unit 113 generates a predicted value of the reference attribute information of the object point with reference to the attribute information of the adjacent point different from the object point. In addition, the predicted value of the subordinate attribute information of the object point is generated using the encoding information of the reference attribute information of the object point. The residual calculation unit 114 generates a prediction residual (residual value) which is the difference between the attribute information and the predicted value. Specifically, the residual calculation unit 114 generates the difference between the reference attribute information and the predicted value of the reference attribute information as the prediction residual of the reference attribute information. The residual calculation unit 114 generates the difference between the subordinate attribute information and the predicted value of the subordinate attribute information as the prediction residual of the subordinate attribute information.

[0100] The quantization unit 115 quantizes the prediction residual. The inverse quantization unit 116 inversely quantizes the quantized prediction residual. The reconstruction unit 117 adds the prediction value to the inversely quantized prediction residual to generate a decoded value of the attribute information of the target point.

[0101] The memory 118 stores the encoded information as attribute information (decoded values) of a plurality of encoded and decoded three-dimensional points. The decoded values ​​stored in the memory 118 are used for subsequent prediction of the three-dimensional points by the prediction unit 113. In addition, the decoded values ​​of the reference attribute information of the object point are used for prediction of the subordinate attribute information of the object point by the prediction unit 113.

[0102] The arithmetic coding unit 119 performs arithmetic coding on the quantized prediction residual. Alternatively, the arithmetic coding unit 119 may binarize the quantized prediction residual and perform arithmetic coding on the binarized prediction residual.

[0103] Different encoding parameters or quantization values ​​(also called quantization parameters or quantization steps) can be used when encoding a dependent component (dependent attribute information) and when encoding a reference component (reference attribute information). For example, the reference component is referenced by other components, so it is likely to be of high importance. Therefore, the reference component can be reversibly compressed, and the dependent component can be irreversibly compressed. For example, irreversible compression is an encoding process that includes a quantization process, and reversible compression is an encoding process that does not include a quantization process. Alternatively, the quantization step size for the reference component can be smaller than the quantization step size for the dependent component. Thus, it is possible to improve the encoding efficiency while suppressing the degradation of the information as a whole. That is, the three-dimensional data encoding device 100 classifies the original attribute information into reference components and dependent components, selects encoding parameters suitable for the reference components and the dependent components, respectively, and encodes them. Thus, it is possible to expect an improvement in encoding efficiency.

[0104] Next, the configuration of the attribute information decoding unit 201 and the process of decoding the attribute information will be described. Here, an example will be described in which the attribute information decoding unit 201 is a decoding unit that performs decoding using LoD (LoD based attribute decoder).

[0105] Figure 7 2 is a block diagram showing the configuration of the attribute information decoding unit 201 . The attribute information decoding unit 201 includes a LoD generation unit 211 , an adjacent search unit 212 , a prediction unit 213 , an arithmetic decoding unit 214 , an inverse quantization unit 215 , a reconstruction unit 216 , and a memory 217 .

[0106] The LoD generation unit 211 generates LoD using decoded position information of a plurality of three-dimensional points. The neighbor search unit 212 searches for three-dimensional points (neighboring points) adjacent to the object point to be processed using the LoD generation result and distance information between three-dimensional points.

[0107] The prediction unit 213 generates a predicted value of the attribute information of the object point. Specifically, the prediction unit 213 generates a predicted value of the reference attribute information of the object point by referring to the attribute information of the neighboring point different from the object point. In addition, the predicted value of the dependent attribute information of the object point is generated using the encoding information of the reference attribute information of the object point.

[0108] The arithmetic decoding unit 214 generates a prediction residual (quantized prediction residual) by arithmetic decoding the encoding attribute information included in the bit stream. The inverse quantization unit 215 inversely quantizes the arithmetic decoded prediction residual (quantized prediction residual). The reconstruction unit 216 generates a decoded value by adding the prediction value to the inverse quantized prediction residual.

[0109] The memory 217 stores decoded values ​​of a plurality of decoded 3D points. The decoded values ​​stored in the memory 217 are used for prediction of subsequent 3D points by the prediction unit 213. In addition, the decoded values ​​of the reference attribute information of the object point are used for prediction of the dependent attribute information of the object point by the prediction unit 213.

[0110] Next, the prediction process will be described in detail. First, the first example of the prediction process will be described. Figure 8 : is a diagram showing the reference relationship in the first example of prediction processing. In this example, the object point has component A and component B, component A is the reference component, and component B is the subordinate component. Component A and component B are both color information, and each includes three dimensional elements (also called sub-components) of green (G), blue (B), and red (R).

[0111] In the first example, inter-component prediction is used for all dimensional elements included in the subordinate attribute information. In inter-component prediction, in order to reduce the amount of coding, the prediction residual of the attribute value (the value of the attribute information) of the subordinate component of the object point and the predicted value based on the attribute value of the reference component of the object point is calculated, and the prediction residual is encoded.

[0112] exist Figure 8 In the example, the arrow indicates the dimensional element of the reference target of the encoding information in the encoding of the attribute component. The dimensional element at the starting point of the arrow refers to the dimensional element at the end point. In this example, each of the multiple dimensional elements included in the subordinate component refers to the same dimensional element included in the reference component. That is, the G element of the subordinate component refers to the G element of the reference component, the B element of the subordinate component refers to the B element of the reference component, and the R element of the subordinate component refers to the R element of the reference component.

[0113] Here, the attribute value of point N of component A is set to (R AN , G AN , B AN ), set the attribute value of point N of component B to (R BN , G BN , B BN ), the predicted value of point N of component B uses the prediction coefficient (coeff ia (R), coeff ia (G), coeff ia (B)), use (R AN· coeff ia (R), G AN· coeff ia (G), B AN· coeff ia(R)). That is, the predicted value of point N of component B is a value obtained by multiplying the attribute value of component A by the prediction coefficient. In addition, the prediction coefficient is derived using a prescribed method. For example, a plurality of candidate values ​​of the prediction coefficient are used to derive the prediction value and the residual, and the candidate value with the least amount of coding is selected as the prediction coefficient. In addition, for example, information indicating the prediction coefficient is stored in a bit stream and sent to a three-dimensional data decoding device.

[0114] The difference between the attribute value of component A and the predicted value is the prediction residual of component B (resi(R BN )、resi(G BN )、resi(B BN )) is expressed by the following formula.

[0115] resi(G BN )=G BN -G AN· coeff ia (G)

[0116] resi(B BN )=B BN -B AN· coeff ia (B)

[0117] resi(R BN )=R BN -R AN· coeff ia (R)

[0118] On the other hand, in the decoding process, the attribute value (R BN , G BN , B BN )Use the decoded prediction residual (resi(R BN )、resi(G BN )、resi(B BN ))、Prediction coefficient (coeff ia (R), coeff ia (G), coeff ia (B)), and the attribute value of component A (R AN , G AN , B AN ), expressed by the following formula.

[0119] G BN = resi(G BN )+G AN· coeff ia (G)

[0120] B BN = resi(BBN )+B AN· coeff ia (B)

[0121] R BN = resi(R BN )+R AN· coeff ia (R)

[0122] By using the above method, when the correlation between two pieces of attribute information is high, the prediction residual can be reduced, thereby reducing the amount of coding.

[0123] Furthermore, when there are three or more pieces of attribute information, two or more pieces of attribute information may refer to one piece of reference attribute information.

[0124] Next, a second example of the prediction process will be described. Fig. 9 This is a diagram showing the reference relationship in the second example of prediction processing. In the second example, inter-component prediction is used for a part of the multiple dimensional elements included in the subordinate component, and inter-sub-component prediction is used for the other part. Inter-sub-component prediction is a prediction method in which the object dimensional element of the processing object refers to other dimensional elements belonging to the same component as the object dimensional element.

[0125] Specifically, the representative dimensional element (in Fig. 9 In the example shown, the G element refers to the representative dimensional element (G element) of the reference component (component A). In addition, dimensional elements other than the representative dimensional element among the plurality of dimensional elements included in the subordinate component (in Fig. 9 In the example shown, the B element and the R element refer to the representative dimensional element included in the subordinate component.

[0126] In this case, if the attribute value of the reference component at point N is (R AN , G AN , B AN ), the attribute value of the subordinate component is (R BN , G BN , B BN ), then the prediction residual (resi(R AN )、resi(G AN )、resi(B AN ))Use the inter-subcomponent prediction coefficient (coeff Ais (R), coeff Ais (B)) is expressed as follows.

[0127] resi(G AN )=G AN

[0128] resi(B AN )=B AN -G AN· coeff Ais (B)

[0129] resi(R AN )=R AN -G AN· coeff Ais (R)

[0130] The prediction residual of the dependent component (resi(R BN )、resi(G BN )、resi(B BN ))Use the inter-subcomponent prediction coefficient (coeff Bis (R), coeff Bis (B)) and the inter-component prediction coefficient (coeff ia (G)) is expressed as follows.

[0131] resi(G BN )=G BN -G AN· coeff ia (G)

[0132] resi(B BN )=B BN -G BN· coeff Bis (B)

[0133] resi(R BN )=R BN -G BN· coeff Bis (R)

[0134] That is, in this example, the G element of component A is not predicted. In addition, the B element and the R element of component A are predicted between sub-components with reference to the G element of component A. The G element of component B is predicted between components with reference to the G element of component A. In addition, the B element and the R element of component B are predicted between sub-components with reference to the G element of component B.

[0135] In addition, similarly to the above-mentioned prediction coefficient, the inter-sub-component prediction coefficient and the inter-component prediction coefficient are derived using a predetermined method. In addition, for example, information indicating the prediction coefficient is stored in a bit stream and transmitted to the three-dimensional data decoding device.

[0136] On the other hand, in the decoding process, if the prediction residual of the reference component at point N is (resi(R AN)、resi(G AN ) and resi(B AN )), the prediction residual of the dependent component is (resi(R BN )、resi(G BN ) and resi(B BN )), then the attribute value of the reference component (R AN , G AN and B AN ) uses the inter-subcomponent prediction coefficient (coeff Ais (R) and coeff Ais (B)) is expressed as follows.

[0137] G AN = resi(G AN )

[0138] B AN = resi(B AN )+G AN· coeff Ais (B)

[0139] R AN = resi(R AN )+G AN· coeff Ais (R)

[0140] The attribute value of the dependent component (R BN , G BN , B BN ) uses the inter-subcomponent prediction coefficient (coeff Bis (R), coeff Bis (B)) and the inter-component prediction coefficient (coeff ia (G)) is expressed as follows.

[0141] G BN = resi(G BN )+G AN· coeff ia (G)

[0142] B BN = resi(B BN )+G BN· coeff Bis (B)

[0143] R BN = resi(R BN )+G BN· coeff Bis (R)

[0144] In this way, the amount of coding can be reduced by combining inter-subcomponent prediction of prediction residuals that can be reduced when the correlation between dimensional elements is high and inter-component prediction of prediction residuals that can be reduced when the correlation between dimensional elements between components is high.

[0145] Next, a third example of prediction processing is described. In the third example, the prediction value of the dependent attribute information is derived with reference to the prediction value of the reference attribute information. In addition, the prediction value of the reference attribute information is derived using a prediction based on a prediction model. That is, the prediction value of the dependent attribute information is derived with reference to the prediction value of the reference attribute information predicted with reference to the attribute information of one or more adjacent points different from the object point.

[0146] Fig.10 is a diagram for explaining prediction processing based on prediction mode, that is, prediction processing with reference to neighboring points. Fig.10 As shown, the predicted value of the reference attribute information is calculated based on multiple adjacent points (nearby points) adjacent to (close to) the object point. For example, any attribute value of the multiple adjacent points is set as the predicted value. Alternatively, the average value of the multiple attribute values ​​of the multiple adjacent points is set as the predicted value. In addition, regardless of the number of dimensional elements contained in the attribute information, the same predicted value can be used for all dimensional elements contained in the attribute information for one object point.

[0147] For example, the three-dimensional data encoding device selects one of a plurality of prediction modes and calculates a prediction value based on the selected prediction mode. In addition, the three-dimensional data encoding device stores information (pred_mode) indicating the selected prediction mode in a bit stream.

[0148] The plurality of prediction modes include, for example, the following modes. When prediction mode = 0, the three-dimensional data encoding device Fig.10 The predicted value is determined by taking the average of the attribute value of the first neighboring point, the attribute value of the second neighboring point, and the attribute value of the third neighboring point. For example, the first neighboring point is the encoded 3D point closest to the object point, the second neighboring point is the encoded 3D point second closest to the object point, and the third neighboring point is the encoded 3D point third closest to the object point.

[0149] When prediction mode = 1, the 3D data encoding device determines the attribute value of the first adjacent point as the prediction value. When prediction mode = 2, the 3D data encoding device determines the attribute value of the second adjacent point as the prediction value. When prediction mode = 3, the 3D data encoding device determines the attribute value of the third adjacent point as the prediction value.

[0150] For example, the three-dimensional data encoding device calculates a prediction value and a prediction residual using these multiple prediction modes, and selects a prediction mode with the smallest encoding amount.

[0151] In addition, for example, the attribute value of the neighboring point to be referenced is an attribute value of the same attribute component as the attribute component of the processing object included in the object point. For example, when the object point and the neighboring point each have a first attribute component and a second attribute component, the first attribute component of the neighboring point is referenced in the prediction of the first attribute component of the object point, and the second attribute component of the neighboring point is referenced in the prediction of the second attribute component of the object point.

[0152] For example, the three-dimensional data encoding device selects a prediction mode for each three-dimensional point, and stores information (pred_mode) indicating the prediction mode in the bitstream for each three-dimensional point. Note that it is not always necessary to store the pred_mode of each point in the bitstream, and it may be based on a specified condition that can be determined in both the encoding device and the decoding device. When the condition is met, the three-dimensional data encoding device uses a predetermined prediction mode (for example, pred_mode=0) without storing pred_mode in the bitstream. In this case, the three-dimensional data decoding device uses the above-mentioned predetermined prediction mode when the condition is met. In addition, the number of selectable prediction modes may be limited when the specified condition is met.

[0153] The three-dimensional data encoding device refers to the predicted value of component A in predicting the attribute value of component B. Here, an example of encoding in the order of R, G, and B is shown, but the encoding order may be reversed.

[0154] When the predicted value of the reference component determined by pred_mode is set to (pred(R AN )、pred(G AN )、pred(B AN )), the prediction residual (resi(R AN )、resi(G AN )、resi(B AN )) is expressed as follows.

[0155] resi(R AN )=R AN -pred(R AN )

[0156] resi(G AN )=G AN -pred(G AN )

[0157] resi(B AN )=B AN -pred(B AN )

[0158] The prediction residual of the dependent component (resi(R BN )、resi(G BN )、resi(B BN )) is expressed as follows.

[0159] resi(R BN )=R BN -pred(R AN )

[0160] resi(G BN )=G BN -pred(G AN )

[0161] resi(B BN )=B BN -pred(B AN )

[0162] In this case, information indicating the prediction mode of the reference component is stored in the bitstream, but information indicating the prediction mode of the dependent component is not stored in the bitstream.

[0163] In addition, the three-dimensional data decoding device calculates the prediction value of the reference component using the prediction mode indicated by the decoded pred_mode. AN )、pred(G AN )、pred(B AN )), the prediction residual of the decoded reference component is set to (resi(R AN )、resi(G AN )、resi(B AN )), the prediction residual of the decoded dependent component is set to (resi(R BN )、resi(G BN )、resi(B BN )) In the case of AN , G AN , B AN ) is expressed as follows.

[0164] R AN = resi(R AN )+pred(R AN )

[0165] G AN = resi(G AN )+pred(G AN )

[0166] B AN = resi(BAN )+pred(B AN )

[0167] The attribute value of the dependent component (R BN , G BN , B BN ) is shown below.

[0168] R BN = resi(R BN )+pred(R AN )

[0169] G BN = resi(G BN )+pred(G AN )

[0170] B BN = resi(B BN )+pred(B AN )

[0171] That is, the property value of the dependent component is calculated using the prediction value obtained when decoding the reference component and the prediction residual of the dependent component decoded from the bit stream.

[0172] In this way, by using the prediction value used in the reference component as the prediction value of the subordinate component, it is not necessary to encode the information (pred_mode) representing the prediction mode with respect to the subordinate component. As a result, the amount of coding can be reduced. In addition, the processing time for calculating the prediction mode with respect to the subordinate component can be reduced. In addition, when generating the prediction residual, the prediction value calculated above can also be multiplied by a coefficient. For example, the coefficient is pre-calculated in a manner that makes the prediction residual smaller. In addition, the coefficient can also be stored in the bitstream.

[0173] In addition, here, an example of using the prediction value of the reference component for the subordinate component is described, but the prediction model used in the reference component can also be used to calculate the prediction value of the subordinate component. That is, the reference component and the subordinate component can make the prediction model common. Specifically, when predicting the reference component, the prediction model is determined, and the prediction model is used to determine the prediction value of the reference component based on the attribute value of the reference component of the adjacent point. Furthermore, using this prediction model, the prediction value of the subordinate component is determined according to the attribute value of the subordinate component of the adjacent point. The difference between the attribute value of the reference component and the prediction value of the reference component is calculated as the prediction residual of the reference component, and the difference between the attribute value of the subordinate component and the prediction value of the subordinate component is calculated as the prediction residual of the subordinate component.

[0174] Even in this case, it is not necessary to encode information (pred_mode) indicating the prediction mode for the dependent component, so the amount of encoding can be reduced.

[0175] Next, a fourth example of prediction processing is described. In the fourth example, the prediction mode and prediction value are not shared between component A and component B. The prediction mode and prediction value are determined for component A and component B respectively, and the respective prediction residuals are calculated. The difference between the two calculated prediction residuals is encoded as the prediction residual of component B.

[0176] Specifically, the three-dimensional data encoding device determines the prediction mode of component A for each three-dimensional point, calculates the prediction value based on the determined prediction mode, and derives the first prediction residual using the calculated prediction value. Similarly, the three-dimensional data encoding device determines the prediction mode of component B for each three-dimensional point, calculates the prediction value based on the determined prediction mode, and derives the second prediction residual using the calculated prediction value.

[0177] In addition, the three-dimensional data encoding device stores pred_mode of each component of each three-dimensional point in the bit stream.

[0178] In addition, the three-dimensional data encoding device directly encodes the first prediction residual for the reference component. On the other hand, the three-dimensional data encoding device uses the first prediction residual of the reference component to predict the second prediction residual of the subordinate component to generate a third prediction residual. For example, the third prediction residual is the difference between the first prediction residual and the second prediction residual.

[0179] For example, the first prediction residual (resi(R AN ), resi(G AN ), resi(B AN )) is expressed as follows.

[0180] resi(R AN )=R AN -pred(R AN )

[0181] resi(G AN )=G AN -pred(G AN )

[0182] resi(B AN )=B AN -pred(B AN )

[0183] The second prediction residual of the subordinate component (resi(R BN )、resi(G BN )、resi(B BN )) is shown below.

[0184] resi(R BN )=R BN-pred(R BN )

[0185] resi(G BN )=G BN -pred(G BN )

[0186] resi(B BN )=B BN -pred(B BN )

[0187] The third prediction residual of the subordinate component (resi2(R BN )、resi2(G BN )、resi2(B BN )) is shown below.

[0188] resi2(R BN )=resi(R BN )-resi(R AN )

[0189] resi2(G BN )=resi(G BN )-resi(G AN )

[0190] resi2(B BN )=resi(B BN )-resi(B AN )

[0191] In addition, in calculating the third prediction residual, at least one of the first prediction residual and the second prediction residual may be multiplied by a coefficient. For example, the coefficient may be pre-calculated so as to reduce the third prediction residual. In addition, the coefficient may be stored in the bitstream.

[0192] On the other hand, the 3D data decoding apparatus derives a second prediction residual of the dependent component using the decoded first prediction residual of the reference component and the decoded third prediction residual of the dependent component. The 3D data decoding apparatus restores the attribute value of the dependent component using the derived second prediction residual.

[0193] For example, the attribute value of the reference component (R AN , G AN , B AN ) is expressed as follows.

[0194] R AN = resi(R AN )+pred(R AN )

[0195] G AN= resi(G AN )+pred(G AN )

[0196] B AN = resi(B AN )+pred(B AN )

[0197] The second prediction residual of the subordinate component (resi(R BN )、resi(G BN )、resi(B BN )) is shown below.

[0198] resi(R BN )=resi2(R BN )-pred(R BN )

[0199] resi(G BN )=resi2(G BN )-pred(G BN )

[0200] resi(B BN )=resi2(B BN )-pred(B BN )

[0201] The attribute value of the dependent component (R BN , G BN , B BN ) is shown below.

[0202] R BN = resi(R BN )+pred(R BN )

[0203] G BN = resi(G BN )+pred(G BN )

[0204] B BN = resi(B BN )+pred(B BN )

[0205] In addition, in the above, the case where the prediction mode and the prediction value are not shared between the reference component and the subordinate component is described, but as described in the third example, the prediction mode and the prediction value can also be shared. That is, the prediction value of the reference component can also be used in the calculation of the second prediction residual of the subordinate component. Thereby, the processing of the prediction in the subordinate component can be reduced. In addition, by not encoding the prediction mode, the amount of coding can be reduced. In addition, it is also possible to share the prediction mode instead of sharing the prediction value.

[0206] In addition, as another method, the three-dimensional data encoding device may encode the difference between the prediction value of the subordinate component and the residual of the reference component as the encoded data of the subordinate component. That is, the three-dimensional data encoding device calculates a first prediction value based on the prediction mode for the reference component, calculates a first residual that is a difference between the attribute information of the reference component and the first prediction value, and encodes the first residual as the encoded data of the reference component. Next, the three-dimensional data encoding device calculates a second prediction value based on the prediction mode for the subordinate component, and encodes the difference between the second prediction value and the first residual as the encoded data of the subordinate component.

[0207] Next, the structure of a bit stream (coded data) generated by the three-dimensional data coding device will be described. Fig.11 This is a diagram showing the structure of TLV data (Type Length Value) included in a bit stream.

[0208] TLV data (TLV unit) includes Type, Length and Value. Type and Length are the header of TLV data, and Type includes the identifier of the data stored in Value. Value is the payload of TLV data, including the encoded data.

[0209] Fig.12 is a diagram showing an example of the structure of a bit stream. Fig.12 As shown, the bit stream contains SPS, GPS, APS, GDU header, GDU, ADU header, ADU, DADU header and DADU.

[0210] SPS (Sequence Parameter Set) is metadata (parameter set) common to multiple frames. APS (Attribute Parameter Set) is metadata (parameter set) related to the encoding of attribute information (reference attribute information and subordinate attribute information). GPS (Geometry Parameter Set) is metadata (parameter set) related to the encoding of location information. For example, APS and GPS are metadata common to multiple frames.

[0211] GDU is a data unit of coded data of position information (geometry data unit). ADU is a data unit of coded data of attribute information (reference attribute information) (attribute data unit). DADU is a data unit of coded data of subordinate attribute information (subordinate attribute data unit). GDU header, ADU header and DADU header are headers (control information) of GDU, ADU and DADU respectively. Fig.12 The respective data shown are stored in a plurality of TLV units.

[0212] Fig.13 is a diagram showing an example of the syntax of a TLV unit (TLV_unit). Fig.13 As shown, the TLV unit includes type and length. In addition, the TLV unit includes data corresponding to the type.

[0213] Specifically, the encoded data of the normal attribute component includes a header of an attribute data unit (attribute_data_unit_header()) and an attribute data unit (attribute_data_unit_data()). The type of the TLV unit is represented as "attribute_data_unit".

[0214] Here, the encoding data of a general attribute component refers to the encoding data of a reference component, which is encoded data with reference to a corresponding geometry data unit without referring to other attribute data units.

[0215] On the other hand, the coded data after being coded with reference to other attribute components includes a header of a dependent attribute data unit (dependent_attribute_data_unit_header()) and a dependent attribute data unit (dependent_attribute_data_unit_data()). The type of the TLV unit is represented as "dependent_attribute_data_unit". The coded data coded with reference to the corresponding geometry data unit and the corresponding attribute data unit of the reference target is stored in the dependent attribute data unit.

[0216] In addition, the encoded data of the position information component includes a header of the geometry data unit (geometry_data_unit_header()) and a geometry data unit (geometry_data_unit_data()). The type of the TLV unit is represented as "geometry_data_unit".

[0217] In addition, here, an example of identifying attribute data units and subordinate attribute data units by TLV type is shown, but attribute data units can also be represented by type, and information indicating whether the attribute data unit is a normal attribute data unit or a subordinate attribute data unit is stored in the attribute data unit header.

[0218] In addition, the first color information and the second color information are each encoded as an attribute component using an encoding method of any attribute information such as LoD based attribute or Transform based attribute. That is, in the above description, an example of using LoD based attribute is described, but the method of this embodiment can also be applied when using Transform based attribute.

[0219] LoD based attribute is one of the conversion methods using LoD (Level of Detail), and is a method of calculating prediction residuals. In addition, LoD is a method of hierarchizing three-dimensional points based on position information, and is a method of hierarchizing three-dimensional points based on the distance (density) between points.

[0220] Transform based attribute is, for example, RAHT (Region Adaptive Hierarchical Transform). RAHT is a method that uses the position information of three-dimensional points to transform attribute information. By applying RAHT transform or Haar transform to attribute information, coding coefficients (high-frequency components and low-frequency components) of each level are generated, and these values ​​or the predicted residuals of the coding coefficients of adjacent nodes are quantized and entropy encoded.

[0221] That is, when using the Transform based attribute, as with the LoD based attribute, the method of encoding the residual using the attribute information, coding coefficient or prediction residual of the reference component can be applied to the prediction of the attribute information, coding coefficient or prediction residual of the subordinate component.

[0222] In other words, the correlation used in the above-mentioned prediction processing can be (1) the correlation between the first attribute information and the second attribute information of a three-dimensional point, or (2) the correlation between multiple first coding coefficients obtained by transforming (RAHT or Haar) multiple first attribute information of multiple three-dimensional points and multiple second coding coefficients obtained by transforming (RAHT or Haar) multiple second attribute information of multiple three-dimensional points, or (3) the correlation between the first prediction residuals generated by performing inter-frame or intra-frame prediction on multiple first coding coefficients and the second prediction residuals generated by performing inter-frame or intra-frame prediction on multiple second coding coefficients.

[0223] Here, in a transform such as RAHT transform or Haar transform, multiple coding coefficients are generated by transforming the attribute information of multiple three-dimensional points. Therefore, it is possible that the correlation between the coding coefficient of the subordinate component (e.g., the first attribute information) and the coding coefficient of the reference component (e.g., the second attribute information) is not limited to the two attribute information of the same three-dimensional point, but includes the correlation between two attribute information of different three-dimensional points.

[0224] That is, the correlation between two attribute information of a 3D point is not limited to that of a 3D point, and the correlation between a plurality of first attribute information of a plurality of 3D points and a plurality of second attribute information of the plurality of 3D points may also be used. In other words, the correlation between the first attribute information of a first 3D point included in the plurality of 3D points and the second attribute information of a second 3D point included in the plurality of 3D points and different from the first 3D point may also be used.

[0225] In addition, the first color information is encoded as a three-dimensional first attribute component, and the second color information is encoded as a three-dimensional second attribute component. The encoded data of the first attribute component is stored in the ADU, and the encoded data of the second attribute component encoded with reference to the first attribute information is stored in the DADU. The attribute component identifiers attr_id=0 and attr_id=1 are stored in the ADU header and the DADU header, respectively.

[0226] The SPS includes an attribute identifier (attribute_type=color) indicating that the first attribute component and the second attribute component are color information, attribute component identifiers (attr_id) of the first attribute component and the second attribute component, and information indicating the number of dimensions of each of the first attribute component and the second attribute component.

[0227] Next, a syntax example of attribute_parameter included in SPS is described. Fig.14 This is a diagram showing a syntax example of attribute_parameter(i).

[0228] attribute_parameter(i) is a parameter regarding the i-th attribute information, and information indicating a dependency relationship between a plurality of attribute information is stored in attribute_parameter.

[0229] Specifically, when attr_param_type = 4, attribute_parameter includes reference_attr_id[i]. attr_param_type = 4 indicates that the attribute component depends on other components (is a dependent component).

[0230] Here, setting num_attribute_parameter to 0 or more and enabling attribute_param_type = 4 can be said to enable a flag indicating that the attribute component depends on other components. In addition, this can be said to enable a flag indicating whether the underlying attribute information can be restored by the attribute component alone.

[0231] In addition, num_attribute_parameter is included in the SPS and indicates the number of attribute_parameters included in the SPS.

[0232] reference_attr_id indicates the attribute component identifier of the reference (dependent target) component.

[0233] When the attribute component of attr_id=1 is a subordinate component that makes the attribute component of attr_id=0 a reference component, the attribute_parameter of attr_id=1 includes attr_param_type=4 and attr_id of the reference target component (ie, 1).

[0234] Furthermore, the SPS may include a flag indicating whether the sequence includes dependent attribute information or whether the sequence may include dependent attribute information.

[0235] Fig.15 This is a diagram showing an example of the syntax of a dependent attribute data unit header (dependent_attribute_data_unit_header). The dependent attribute data unit header includes reference_attr_slice_id, attr_id, reference_attr_id, inter_attribute_pred_enabled, predmode_reference_flag, and inter_attribute_pred_coeff[i][c].

[0236] reference_attr_slice_id indicates the slice ID of the attribute component of the reference target. Here, the slice ID is an identifier of the slice of the point group. Therefore, the slice ID of the attribute component of the reference target is the same as the slice ID of the position information of the reference target and the slice ID of the subordinate component of the processing object. Therefore, reference_attr_slice_id can indicate the slice ID of the position information of the reference target or the slice ID of the subordinate component of the processing object. attr_id indicates the index of the attribute component.

[0237] reference_attr_id is the same as reference_attr_id of attribute_parameter, and indicates a component identifier of an attribute component that refers to a target (dependent target) component. reference_attr_id is included in at least one of the SPS and the dependent attribute data unit header.

[0238] inter_attribute_pred_enabled indicates whether inter-component prediction is enabled. For example, a value of 1 indicates enabled, and a value of 0 indicates disabled.

[0239] predmode_reference_flag indicates whether to refer to the prediction mode (pred_mode) of the reference component.

[0240] When inter_attribute_pred_enabled is 1, the subordinate attribute data unit header includes inter_attribute_pred_coeff[i][c]. Here, Fig.15 The lod_max_levels shown indicates the number of LoD layers. num_dimension indicates the number of dimensional elements contained in the attribute component, that is, the number of dimensions.

[0241] inter_attribute_pred_coeff represents the prediction coefficient. Fig.15 As shown, the prediction coefficients are stored per LoD layer and per sub-component (dimension).

[0242] Note that the prediction coefficients may be common to multiple LoD layers, may be common to multiple subcomponents, or may be common to multiple LoD layers and multiple subcomponents. That is, prediction coefficients may be generated for each of more than one LoD, or may be generated for each of more than one dimensional element.

[0243] In addition, inter_attribute_pred_coeff_diff, which represents the difference of the prediction coefficient, may be used instead of inter_attribute_pred_coeff. For example, the inter_attribute_pred_coeff_diff at the beginning of the LoD represents the difference between the predetermined initial value and the prediction coefficient of the processing object. The second inter_attribute_pred_coeff_diff from the beginning of the LoD represents the difference between the prediction coefficient at the beginning of the LoD and the prediction coefficient of the processing object (the second one). For the third and subsequent ones, the difference between the prediction coefficient immediately before and the prediction coefficient of the processing object is used in the same way as the second one. Therefore, the amount of data of the prediction coefficient sent can be reduced.

[0244] In addition, when a common prediction coefficient is used for a data unit, information indicating the prediction coefficient may be stored in the header of the data unit. In addition, when a common prediction coefficient is used for a plurality of data units, information indicating the prediction coefficient may be stored in the APS. In addition, when a common prediction coefficient is used for a sequence, information indicating the prediction coefficient may be stored in the SPS.

[0245] In addition, Fig.15 , an example is shown in which control signals such as reference_attr_id, inter_attribute_pred_enabled, and predmode_reference_flag are shared by multiple LoDs and multiple dimensional elements, but at least one of these control signals may be set for each LoD or may be set for each dimensional element.

[0246] Next, a syntax example of the dependent attribute data unit is described. Fig.16 It means use Figure 8 as well as Fig. 9 FIG. 1 is a diagram showing an example of the syntax of a dependent attribute data unit (dependent_attribute_data_unit) in the case of the first example or the second example of the prediction process. Fig.16 As shown, the subordinate attribute data unit includes resi.

[0247] exist Fig.16 In , point_count indicates the number of points included in the data unit. num_dimension indicates the number of dimensional elements included in the attribute component, that is, the number of dimensions. In addition, resi is generated for each point and each dimensional element.

[0248] Resi represents the prediction residual. Specifically, as shown in the first and second examples of the prediction processing described above, resi represents the residual (difference) between the prediction value calculated at each point of the data unit with reference to the reference subcomponent or other subcomponents in the attribute component of the processing object and the subcomponent of the processing object.

[0249] Furthermore, when resi of any subcomponent is resi=0 at a plurality of consecutive points, the dependent attribute data unit may include zero_run_length indicating the number of consecutive 0 points instead of resi.

[0250] In order to make arithmetic coding of resi advantageous, arithmetic coding may be performed after decomposing resi or changing the value of resi according to conditions of other subcomponents. In addition, making arithmetic coding advantageous means, for example, reducing the amount of data after arithmetic coding.

[0251] Fig.17 It means use Fig.10 FIG. 1 is a diagram showing an example of the syntax of a dependent attribute data unit (dependent_attribute_data_unit) in the case of the third example or the fourth example of the prediction process. Fig.16 As shown, the subordinate attribute data unit includes pred_mode and resi (or resi2).

[0252] pred_mode is set for each point and each subcomponent, and indicates the prediction mode used for the subcomponent. In addition, when the prediction mode (pred_mode) of the reference component is not referenced (predmode_reference_flag = 0), the dependent attribute data unit includes pred_mode. On the other hand, when the prediction mode (pred_mode) of the reference component is referenced (predmode_reference_flag = 1), the dependent attribute data unit does not include pred_mode.

[0253] In the case of the third example, the dependent attribute data unit includes resi generated for each point and each dimensional element. resi indicates the residual (difference) between the predicted value generated using the referenced pred_mode or the pred_mode included in the dependent attribute data unit and the subcomponent to be processed.

[0254] In the case of the fourth example, the dependent attribute data unit includes resi2 generated for each point and each dimensional element. Resi2 is the residual (difference) between the residual of the subcomponent to be processed and the residual of the reference subcomponent, generated by using the referenced pred_mode or the pred_mode included in the dependent attribute data unit.

[0255] Furthermore, when resi of any subcomponent is resi=0 at a plurality of consecutive points, the dependent attribute data unit may include zero_run_length indicating the number of consecutive 0 points instead of resi.

[0256] In order to make arithmetic coding of resi advantageous, resi may be decomposed into specific bits, or arithmetic coding may be performed on the basis of deforming a plurality of resi of a plurality of subcomponents so as to be combined into one data. In addition, arithmetic coding being advantageous means, for example, reducing the amount of data after arithmetic coding. In addition, pred_mode and resi of a specific subcomponent may also be combined into one data.

[0257] Fig.18 1 is a flowchart of the encoding process of attribute information (attribute component) performed by the three-dimensional data encoding device. First, the three-dimensional data encoding device determines whether the target component of the attribute component to be processed is a dependent component (S121).

[0258] If the target component is a dependent component (Yes in S121), the 3D data encoding device stores the reference component ID (e.g., reference_attr_id) and information indicating the dependency relationship (e.g., attr_param_type=4) in metadata (S122). The 3D data encoding device encodes the target component with reference to the reference component (S123).

[0259] On the other hand, when the target component is not a dependent component (No in S121), the three-dimensional data encoding device encodes the target component alone without referring to other attribute components (S124).

[0260] Fig.19 Flowchart of decoding processing of attribute information (attribute component) by a 3D data decoding device. First, the 3D data decoding device obtains (decodes) a reference component ID (eg, reference_attr_id) and information indicating dependency (eg, attr_param_type) from metadata included in a bitstream (S131).

[0261] Next, the three-dimensional data decoding device determines whether the target component of the attribute component to be processed is a dependent component (S132). For example, the three-dimensional data decoding device makes this determination based on the information indicating the dependency relationship acquired in step S131.

[0262] When the target component is a dependent component (Yes in S132 ), the three-dimensional data decoding apparatus decodes the target component with reference to the reference component ( S133 ).

[0263] On the other hand, when the target component is not a dependent component (No in S132), the three-dimensional data decoding apparatus decodes the target component alone without referring to other attribute components (S134).

[0264] also, Fig.18 as well as Fig.19 The processing shown may be performed in frame units or in slice units.

[0265] The following describes a modified example of the method for encoding the dependent attribute information. For example, as the above-mentioned method for deriving the prediction coefficient, the following method may also be used. The three-dimensional data encoding device uses each of the candidate values ​​of the plurality of prediction coefficients and calculates the sum of the residuals of the plurality of points for each candidate value. The three-dimensional data encoding device may also determine the candidate value with the smallest calculated sum as the prediction coefficient.

[0266] As another derivation method, the three-dimensional data encoding device calculates the sum of the dependent attribute information of the plurality of points and the sum of the reference attribute information of the plurality of points. The three-dimensional data encoding device may also use each of the plurality of candidate values ​​of the prediction coefficient, calculate the difference between the product value of the candidate value and the sum of the reference attribute information and the sum of the dependent attribute information for each candidate value, and determine the constraint with the minimum difference as the prediction coefficient.

[0267] In addition, in a mode in which irreversible encoding is performed (for example, an encoding mode that includes quantization processing), the three-dimensional data encoding device can use the input attribute values ​​before encoding to calculate the prediction coefficients when deriving the prediction coefficients, or can use the decoded values ​​after encoding and decoding (after quantization and inverse quantization) to calculate the prediction coefficients in order to improve accuracy.

[0268] For example, when encoding attribute components of three or more color information, the three-dimensional data encoding device may set any one of the color information as a reference component and encode the other attribute components with reference to the same reference component. In this case, the three-dimensional data decoding device can decode the other attribute components by first decoding the reference component. Therefore, random access when decoding any attribute component can be improved.

[0269] In addition, a plurality of reference components may be set, thereby possibly improving random accessibility when the number of attribute components is large.

[0270] In addition, when the number of dimensions of the reference component and the subordinate component is different, a specific sub-component (dimensional element) included in the reference component may be referenced. In this case, information that can identify which sub-component is referenced may be stored in the bitstream. Alternatively, which sub-component is referenced may be predetermined.

[0271] In addition, in the above description, an example of storing attribute_parameter in SPS is shown, but it can also be stored in APS or SEI. For example, in the case where the dependency changes for each frame, the dependency information (such as attribute_parameter) can also be stored in the SEI of each frame. For example, frame_attr_param_SEI can be cited as the SEI of each frame.

[0272] Furthermore, when dependency information is stored in both the SPS and the SEI of each frame, the three-dimensional data decoding apparatus preferentially uses the SEI of each frame.

[0273] Fig. 20 : is a diagram showing a syntax example of frame_attr_param_SEI in this case. In this example, attribute_idx specifies the attribute component to which the dependency information (attribute_parameter) is applied. That is, dependency information is specified for each attribute component. In addition, Fig. 20 The sps_idx shown is the index of the SPS, and the frame_idx is the index of the frame. num_parameter indicates the number of dependency information (attribute_parameter).

[0274] In addition, common transformation information may be specified for all attribute components. For example, the transformation information may be included in a SEI common to all attribute components, such as attribute_structure_SEI. In this case, the three-dimensional data decoding device can determine the dependency relationship between multiple attribute components by parsing attribute_structure_SEI.

[0275] In addition, SEI for partial decoding may also be defined. Fig.21 This is a diagram showing a syntax example of the SEI (angle_partial_access_SEI).

[0276] For example, when encoding a plurality of color information different for each viewpoint, the SEI includes viewpoint (angle) information and an attribute component identifier (attr_id). Furthermore, for a dependent component (dependent_flag=1), an attribute component identifier (reference_attr_id) of a reference component is indicated.

[0277] also, Fig.21 The num_angle shown indicates the number of viewpoints, and angle indicates a viewpoint. dependent_flag indicates whether the attribute component is a dependent component.

[0278] Although not shown in the figure, information indicating the conversion method such as pre-processing and post-processing (for example, offset value and scaling value, etc.) may be included in SEI. In addition, such information only needs to be included in at least one of SPS and SEI.

[0279] Fig. 22 is a flowchart of the decoding process when partial decoding is performed. For example, in the system layer Fig. 22 Processing shown.

[0280] First, the 3D data decoding device decodes and analyzes the SEI for partial decoding (S211). Next, the 3D data decoding device analyzes the viewpoint information and obtains the attribute component identifier (ID) of the attribute information of the viewpoint to be decoded (S212). Next, when the attribute component is a dependent component, the 3D data decoding device obtains the attribute component identifier of the reference component (S213). Next, the 3D data decoding device searches for the attribute data unit, extracts the data unit having the obtained attribute component identifier, and decodes the extracted data unit (S214).

[0281] In this way, the three-dimensional data decoding apparatus obtains the attribute component identifier (ID) of the attribute component (reference component and dependent component) of a specific viewpoint from the bit stream, and extracts the data unit using the attribute component identifier.

[0282] In addition, the extracted data units may not be decoded, but a bitstream or file may be reconstructed from the extracted data units. For example, the edge requests a bitstream or file containing attribute information of a specific viewpoint from the server. The server may generate a bitstream or file of attribute information extracted from a specific viewpoint through the above processing, and send the bitstream or file to the edge. In this way, the amount of data sent can be reduced, and thus the communication time may be shortened.

[0283] (Implementation Method 2)

[0284] In this embodiment, a method of encoding sub-components in a combined manner will be described. Fig.23 3D data encoding device 300 of this embodiment is a block diagram showing the structure of the 3D data encoding device 300. The 3D data encoding device 300 generates coded attribute information and coded merge information by encoding a plurality of attribute information (first to second attribute information). The 3D data encoding device 300 includes a merge unit 301 and an attribute information encoding unit 302. Fig.23 , only the processing unit for encoding the attribute information is described. The three-dimensional data encoding device 300 may also include other processing units such as a position information encoding unit for encoding the position information.

[0285] The integration unit 301 integrates a plurality of attribute information (first to second attribute information) to generate integrated attribute information.

[0286] Fig.24 is a diagram showing an example of merging of subcomponents. Fig.24 As shown, for example, the attribute information includes color information (first attribute information) having three-dimensional elements and reflectance information (second attribute information) having one-dimensional elements. In this case, the merging unit 301 merges the color information and the reflectance information to generate merged attribute information having four-dimensional elements.

[0287] The attribute information encoding unit 302 generates encoded attribute information by encoding the merged attribute information as one attribute component. Fig.24 In the example shown, the attribute information encoding unit 302 encodes the merged attribute information into one attribute component having four-dimensional sub-components. Figure 2 The attribute information encoding unit 101 shown is the same.

[0288] In addition, the attribute information encoding unit 302 generates merge information as metadata related to merging. Fig.24 In the example shown, the attribute information encoding unit 302 generates merge information indicating that merged attribute information is generated by combining color information and reflectance information. In addition, the attribute information encoding unit 302 generates encoded merge information by encoding the merge information. In addition, the three-dimensional data encoding device 300 generates a bit stream including the encoded attribute information and the encoded merge information.

[0289] Fig.25 3D data decoding device 400 according to the present embodiment is a block diagram showing the structure of the 3D data decoding device 400. The 3D data decoding device 400 generates a plurality of attribute information (first to second attribute information) by decoding a bit stream. The 3D data decoding device 400 includes an attribute information decoding unit 401 and a separation unit 402. Fig.25, only the processing unit for decoding the attribute information is described. The three-dimensional data decoding apparatus 400 may also include other processing units such as a position information decoding unit for decoding the position information.

[0290] The attribute information decoding unit 401 generates merge attribute information by decoding the coded attribute information included in the bitstream. In addition, the attribute information decoding unit 401 generates merge information by decoding the coded merge information included in the bitstream.

[0291] The separation unit 402 generates a plurality of attribute information (first to second attribute information) by separating the merged attribute information using the merged information. Fig.24 In the illustrated example, the separation unit 402 restores the color information (first attribute information) and the reflectance information (second attribute information).

[0292] Here, multiple attribute information of a certain point may be correlated with each other even when the types of attribute information are different. For example, color information and reflectivity may be correlated. In addition, a combination of correlated attribute information may be, for example, color information, reflectivity, transmittance, and infrared information.

[0293] In this way, by merging and encoding a plurality of highly correlated attribute information, metadata can be shared, thereby reducing the amount of data. For example, by encoding the merged plurality of sub-components using the same prediction mode, the amount of data representing the prediction mode can be reduced. In addition, for example, prediction processing based on inter-sub-component prediction can also be performed between the elements of the merged attribute information. That is, inter-sub-component prediction cannot be applied to attribute information with a dimension of 1, but by merging a plurality of attribute information, inter-sub-component prediction can be applied, and the overall amount of coding can be reduced compared to the case where each of the plurality of attribute information is encoded separately.

[0294] Next, a coding method and a signaling method for combining attribute information will be described. First, a method for indicating that an object attribute component of a coding target is attribute information obtained by combining a plurality of attribute information will be described.

[0295] For example, an attribute type indicating attribute information including a plurality of attribute information may be defined in attribute_type indicating the type of attribute information included in the SPS, such as "integration". Alternatively, attribute_type indicating an attribute type of a plurality of attribute information before integration may be stored in the SPS.

[0296] For example, integrated_attr_id[i] indicating the attribute component identifier of the attribute component to be merged may be stored in SPS or attribute_param(). In this case, sub_component_type indicating the type and order of subcomponents constituting the attribute component may be predefined and stored in SPS or attribute_param().

[0297] For example, sub_component_type=0 represents (r, G, B, R), sub_component_type=1 represents (G, B, R, r), and sub_component_type=2 represents (r, R, G, B).

[0298] The num_dimension indicating the number of dimensions of the attribute component indicates the number of dimensions obtained by adding up the numbers of dimensions of the components before merging. For example, when merging three-dimensional color and one-dimensional reflectance, the number of dimensions of the merged attribute component is 4.

[0299] Next, a method for encoding the merged attribute component is described. For example, an attribute component obtained by merging attribute components of dimensions m and n is encoded as a component of dimension m+n, and the encoded data is stored in attribute_data_unit_data(). Here, m and n are arbitrary natural numbers.

[0300] By encoding a plurality of attribute information as one attribute component, one data unit can be used, thereby reducing the amount of header data.

[0301] Furthermore, the three-dimensional data encoding device may encode or not encode the attribute components that are merged and have a dimension of 0.

[0302] Next, a method for decoding the merged attribute information and restoring it to the original attribute components is described. The three-dimensional data decoding device can decode the bit stream generated by the method shown above as the merged attribute components. The three-dimensional data decoding device can refer to the merged target component identifier (integrated_attr_id[i]) stored in the SPS or attribute_param() to grasp the merged source attribute component identifier, so it can restore the merged attribute components to the original multiple attribute components.

[0303] Next, a first example of prediction processing for combining attribute information is described. In the first example, a prediction encoding method based on a prediction mode is used as an encoding method for combining attribute information. In addition, an example is described in which a color having three-dimensional sub-components of R, G, and B and a reflectance r having one-dimensional sub-component are combined and encoded as four-dimensional components. As a prediction encoding method, a method using Fig.10 etc. for predictive processing.

[0304] Assume that the merged attribute information at point N is (R N , G N , B N and r N ), the prediction value determined by the prediction mode (pred_mode) is (pred(R N )、pred(G N )、pred(B N ) and pred(r N )), then the prediction residual (resi(R N )、resi(G N )、resi(B N ) and resi(r N )) is expressed as follows.

[0305] resi(R N )=R N -pred(R N )

[0306] resi(G N )=G N -pred(G N )

[0307] resi(B N )=B N -pred(B N )

[0308] resi(r N )=r N -pred(r N )

[0309] In addition, if the prediction residual of the decoded merged attribute information at point N in the three-dimensional data decoding device is set to (resi(R N )、resi(G N )、resi(B N )、resi(r N )), the predicted value determined by the prediction model is set to (pred(R N )、pred(G N )、pred(BN )、pred(r N )), then merge the attribute information (R N , G N , B N 、r N ) is expressed as follows.

[0310] R N = resi(R N )+pred(R N )

[0311] G N = resi(G N )+pred(G N )

[0312] B N = resi(B N )+pred(B N )

[0313] r N = resi(r N )+pred(r N )

[0314] Here, since pred_mode is common to all subcomponents in the attribute component, pred_mode can be made common to more subcomponents by merging multiple attribute components. That is, the number of pred_modes sent from the 3D data encoding device to the 3D data decoding device can be reduced, thereby reducing the amount of code.

[0315] In addition, for example, for attribute information with a dimension of 1 such as reflectance, the pred_mode of each dimension at one point is 1, whereas for attribute information with a dimension of 3 such as color information, the pred_mode of each dimension at one point is 1 / 3, so it can be said that the smaller the dimension, the worse the transmission efficiency of pred_mode. Therefore, the dimension can be increased by merging attributes, and the number of pred_modes of each dimension at one point is reduced, so the transmission efficiency of pred_mode can be improved.

[0316] Next, a second example of prediction processing for merging attribute information is described. In the second example, inter-subcomponent prediction is used as a coding method for merging attribute information. In the prediction coding method, in order to reduce the amount of coding, the prediction residual between the attribute value to be coded and the predicted value is coded. As the predicted value, for example, the attribute value of another subcomponent in the attribute component is used.

[0317] Fig.26 , Fig. 27 as well as Fig.28 is a diagram showing an example of a reference relationship (dependency relationship) in inter-subcomponent prediction. Fig.26 , Fig. 27 or Fig.28 The reference relationship between the sub-components shown in the figure is used for prediction. The arrows in the figure represent the reference relationship of the prediction, the starting point of the arrow is the subordinate sub-component, and the end point of the arrow is the reference sub-component. In addition, an example of color information being RGB is shown here, but the color information can also be other color information such as YCbCr.

[0318] exist Fig.26 In the example shown, the merge attribute information at point N is set to (R N , G N , B N 、r N ), set the reference subcomponent to G N In the case of (B N , R N 、r N ) can be used to predict G N The prediction residual (resi(B)) of the subcomponent to be predicted in the three-dimensional data encoding device is calculated by multiplying the prediction coefficients (coeff(B), coeff(R), coeff(r)). N )、resi(R N ) and resi(r N )) is expressed as follows.

[0319] resi(G N )=G N

[0320] resi(B N )=B N -G N ·coeff(B)

[0321] resi(R N )=R N -G N ·coeff(R)

[0322] resi(r N )=r N -G N ·coeff(r)

[0323] exist Fig. 27 In the example shown, the reference subcomponent is r and the prediction residual is expressed as follows.

[0324] resi(r N )=r N

[0325] resi(G N )=G N -r N ·coeff(G)

[0326] resi(B N )=B N -r N ·coeff(B)

[0327] resi(R N )=R N -r N ·coeff(R)

[0328] In addition, the number of dependent subcomponents is not limited to one, but may also be multiple. Fig.28 In the example shown, B and R refer to G, and G refers to r. In this way, a dual prediction reference relationship can be used. Fig.28 In the example shown, the prediction residual is expressed as follows.

[0329] resi(r N )=r N

[0330] resi(B N )=B N -G N ·coeff(B)

[0331] resi(R N )=R N -G N ·coeff(R)

[0332] resi(G N )=G N -r N ·coeff(G)

[0333] In addition, the reference relationship shown here is an example, and reference relationships other than these may also be used.

[0334] In addition, if Fig.26 As shown, in the three-dimensional data decoding device, the prediction residual of the decoded merged attribute information at point N is set to (resi(B N )、resi(R N )、resi(r N )), set the reference subcomponent to G N In the case of (B N , R N 、r N ) can be used to predict G NThe merged attribute information (G N , B N , R N 、r N ) is expressed as follows.

[0335] G N = resi(G N )

[0336] B N = resi(B N )+G N· coeff(B)

[0337] R N = resi(R N )+G N· coeff(R)

[0338] r N = resi(r N )+G N· coeff(r)

[0339] exist Fig. 27 In the example shown, the merge attribute information is expressed as follows.

[0340] r N = resi(r N )

[0341] G N = resi(G N )+r N· coeff(G)

[0342] B N = resi(B N )+r N· coeff(B)

[0343] R N = resi(R N )+r N· coeff(R)

[0344] exist Fig.28 In the example shown, the merge attribute information is expressed as follows.

[0345] r N = resi(r N )

[0346] B N = resi(B N )+GN· coeff(B)

[0347] R N = resi(R N )+G N· coeff(R)

[0348] G N = resi(G N )+r N· coeff(G)

[0349] As described above, for example, by making predictions with reference to highly correlated sub-components, the prediction residual of the coded object can be reduced, thereby reducing the amount of coding. In addition, for attribute information with a dimension of 1 such as reflectivity, inter-sub-component prediction cannot be used, but by merging the attribute information to make it multidimensional, inter-sub-component prediction can be used to reduce the amount of coding.

[0350] Next, a third example of prediction processing for merged attribute information is described. In the third example, as a coding method for merged attribute information, prediction based on prediction mode and inter-subcomponent prediction are used in combination. Fig.26 The reference relationship shown is an example of the case of a reference relationship for prediction between sub-components.

[0351] First, the first residual is calculated for G, B, R, and r respectively by prediction based on the prediction mode. Fig.26 As shown, for B, R, and r, the difference between the first residual of the processing object and the first residual of the reference target, i.e., the second residual, is calculated by referring to the inter-subcomponent prediction of G. In this case, the second residual (resi(G N )、resi(B N )、resi(R N )、resi(r N )) is expressed as follows.

[0352] resi(G N )=G N -pred(G N )

[0353] resi(B N )={B N -pred(B N )}-resi(G N )·coeff(B)

[0354] resi(R N )={R N -pred(R N )}-resi(GN )·coeff(R)

[0355] resi(r N )={r N -pred(r N )}-resi(G N )·coeff(r)

[0356] In addition, the merge attribute information (G N , B N , R N 、r N ) is expressed as follows.

[0357] G N = resi(G N )+pred(G N )

[0358] B N ={resi(B N )+pred(B N )}+resi(G N )·coeff(B)

[0359] R N ={resi(R N )+pred(R N )}+resi(G N )·coeff(R)

[0360] r N ={resi(r N )+pred(r N )}+resi(G N )·coeff(r)

[0361] In addition, here, Fig.26 The reference relationship is explained as an example, but in Fig. 27 or Fig.28 The same processing can be applied to the reference relationships shown in the figure. In addition, the same processing can be applied to reference relationships other than these.

[0362] Fig.29Flowchart of the encoding process of attribute information performed by a three-dimensional data encoding device. First, the three-dimensional data encoding device determines whether to merge multiple attribute information (attribute components) of the processing object (S301). For example, the three-dimensional data encoding device may also determine to merge multiple attribute information when the multiple attribute information is a group of attribute information of a predetermined category (for example, color information and reflectivity). Alternatively, the three-dimensional data encoding device may also determine whether to merge multiple attribute information based on the number of dimensions of each of the multiple attribute information or the total number of dimensions. For example, it may also be determined to merge when the attribute information includes a number of dimensions less than a threshold value, or when the total number of dimensions is less than a threshold value.

[0363] When the three-dimensional data encoding device determines that a plurality of attribute information is to be merged (Yes in S301), the three-dimensional data encoding device generates merged attribute information by merging the plurality of attribute information (S302). In addition, the three-dimensional data encoding device stores (encodes) the merged attribute information in metadata (S303) and encodes the merged attribute information (S304).

[0364] On the other hand, when the three-dimensional data encoding device determines that the plurality of attribute information are not to be combined (No in S301), the three-dimensional data encoding device encodes the plurality of attribute information as separate attribute components.

[0365] Fig.30 1 is a flowchart of a decoding process of attribute information of a 3D data decoding device. First, the 3D data decoding device obtains (decodes) merge information from metadata included in a bit stream (S311). Next, the 3D data decoding device generates attribute information by decoding the encoded attribute information included in the bit stream (S312).

[0366] Next, the three-dimensional data decoding device determines whether the decoded attribute information is merged attribute information based on the merge information (S313). If the decoded attribute information is merged attribute information (Yes in S313), the three-dimensional data decoding device restores the original multiple attribute information by separating the decoded attribute information (S314). On the other hand, if the decoded attribute information is not merged attribute information (No in S313), the three-dimensional data decoding device directly outputs the decoded attribute information.

[0367] Notice, Fig.29 as well as Fig.30 The processing shown may be performed in frame units or in slice units.

[0368] In addition, in the above description, the example of combining three-dimensional attribute information and one-dimensional attribute information is mainly described, but the number of dimensions of the combined attribute information can be arbitrary. In other words, the three-dimensional data encoding device can combine m-dimensional attribute information and n-dimensional attribute information to generate m+n-dimensional attribute components. Here, m and n are arbitrary natural numbers.

[0369] In addition, the three-dimensional data encoding device may also generate n+x-dimensional third attribute information by merging the x-dimensional subcomponents of a part of the m-dimensional first attribute information and the n-dimensional second attribute information. In this case, the three-dimensional data encoding device encodes the fourth attribute information composed of the remaining mx-dimensional subcomponents of the first attribute information and the n+x-dimensional third attribute information.

[0370] In addition, the three-dimensional data encoding device may separate and merge three three-dimensional color information, i.e., color1 = (R1, G1, B1), color2 = (R2, G2, B2), and color3 = (R3, G3, B3), to generate component A = (R1, R2, R3), component B = (G1, G2, G3), and component C = (B1, B2, B3), and encode the generated component A, component B, and component C. In this way, by merging sub-components with high correlation, the accuracy of prediction between sub-components and prediction between components is improved, so it can be expected that the encoding efficiency will be improved.

[0371] In addition, here, an example of performing prediction within a component after merging a plurality of attribute information is described, but the inter-component prediction described in Embodiment 1 may be used to apply the same prediction (reference relationship) to a plurality of attribute information before merging. Fig.26 In the example shown, the reference from r to G can be achieved through inter-component prediction.

[0372] (Summarize)

[0373] As described above, the decoding device (three-dimensional data decoding device) of Embodiment 1 performs Fig.31 The decoding device receives a bit stream (S401) in which a plurality of three-dimensional points each having first attribute information and second attribute information are encoded, and predicts the first attribute information (e.g., subordinate attribute information) with reference to the second attribute information (e.g., reference attribute information) (S402). For example, the decoding device decodes the first attribute information using the predicted result (prediction value). For example, the decoding device calculates a first prediction value of the first attribute information with reference to the second attribute information. The decoding device restores the first attribute information by adding the first residual of the first attribute information contained in the encoded attribute information to the first prediction value.

[0374] Here, sometimes there is a correlation between the attribute information. Therefore, by predicting the first attribute information with reference to the second attribute information, it is possible to reduce the amount of data of the first attribute information. Thus, the amount of data to be processed by the decoding device can be reduced.

[0375] In addition, the decoding device receives a bit stream and decodes the bit stream. The bit stream includes: encoded attribute information of a plurality of three-dimensional points, each of the plurality of three-dimensional points having first attribute information and second attribute information; and meta information (such as dependency information or attr_param_type), indicating that the first attribute information is predicted with reference to the second attribute information. For example, the decoding device decodes the first attribute information using the meta information. For example, the decoding device predicts the first attribute information with reference to the second attribute information.

[0376] Here, sometimes there is a correlation between the attribute information. Therefore, by predicting the first attribute information with reference to the second attribute information, it is possible to reduce the data amount of the first attribute information. The decoding device can appropriately decode the bit stream with the reduced data amount using the meta information. In addition, the amount of data processed by the decoding device can be reduced.

[0377] For example, the predicted first attribute information and the referenced second attribute information are possessed by the same 3D point. Thus, by predicting the first attribute information of the 3D point with reference to the second attribute information of the 3D point, it is possible to reduce the amount of data of the first attribute information. Thus, the amount of data processed by the decoding device can be reduced.

[0378] For example, the predicted first attribute information and the referenced second attribute information have different 3D points. That is, the first 3D point has the predicted first attribute information, and the second 3D point different from the first 3D point has the referenced second attribute information.

[0379] For example, the first attribute information and the second attribute information are respectively stored in a first component (eg, a dependent component) and a second component (eg, a reference component).

[0380] For example, the first quantization step size for the first attribute information is larger than the second quantization step size for the second attribute information. By using a smaller quantization step size for the second attribute information referenced by other attribute information, it is possible to suppress degradation of the second attribute information and improve the accuracy of prediction of the first attribute information that references the second attribute information. In addition, for the first attribute information, by using a larger quantization step size, the amount of data can be reduced. Thus, it is possible to improve coding efficiency while suppressing degradation of the information as a whole.

[0381] For example, the first attribute information and the second attribute information are generated by respectively decoding the first coded attribute information and the second coded attribute information, and the second attribute information is losslessly compressed (reversibly compressed). For example, the first attribute information is irreversibly compressed.

[0382] Thus, by using lossless compression on the second attribute information referenced by other attribute information, degradation of the second attribute information can be suppressed, and the accuracy of prediction of the first attribute information that references the second attribute information can be improved. Thus, the coding efficiency can be improved while degradation of the information as a whole can be suppressed.

[0383] For example, Figure 8 As shown in FIG. 1 , each of the first component and the second component includes a first dimensional element and a second dimensional element, and the first dimensional element of the first component is predicted with reference to the first dimensional element of the second component, and the second dimensional element of the first component is predicted with reference to the second dimensional element of the second component. Thus, when there is a correlation between the same dimensional elements between components, the amount of data can be reduced by utilizing the correlation.

[0384] For example, Fig. 9 As shown in FIG. 1 , the first component includes the first dimensional element and the second dimensional element, the first dimensional element is predicted with reference to the second component, and the second dimensional element is predicted with reference to the first dimensional element. Depending on the attribute information, sometimes there is a correlation between different components, and sometimes there is a correlation between the dimensional elements constituting the component. Therefore, the amount of data can be reduced by using the correlation between components and the correlation between dimensional elements.

[0385] For example, the bitstream does not include information about the first prediction mode applied to the first component (eg, pred_mode), but includes information about the second prediction mode applied to the second component (eg, pred_mode). Thus, since information about the first component can be reduced, the amount of data can be reduced.

[0386] For example, as described in the fourth example of prediction processing, the bitstream includes a first residual value of the first attribute information, the first residual value is the difference between the third residual value of the first attribute information and the second residual value of the second attribute information, the third residual value is the difference between the first value of the first attribute information and the first predicted value of the first attribute information, and the second residual value is the difference between the second value of the second attribute information and the second predicted value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the residual of the first attribute information and the residual of the second attribute information.

[0387] For example, as described in the first and second examples of the prediction process, the bitstream includes a first residual value of the first attribute information, and the first residual value is the difference between the first value of the first attribute information and the second value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the value of the first attribute information and the value of the second attribute information.

[0388] For example, as described in the first and third examples of prediction processing, the bitstream includes a first residual value of the first attribute information, and the first residual value is the difference between the first value of the first attribute information and the second predicted value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the value of the first attribute information and the predicted value of the second attribute information. In addition, since there is no need to calculate the predicted value of the first attribute information, the amount of processing can be reduced.

[0389] For example, the bitstream includes a first residual value of the first attribute information, and the first residual value is the difference between the first predicted value of the first attribute information and the second residual value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the predicted value of the first attribute information and the residual of the second attribute information.

[0390] For example, the bitstream includes flag information (e.g., inter_attribute_pred_enabled) indicating whether the second attribute information can be referenced in the prediction of the first attribute information. Thus, the decoding device can refer to the flag information to determine whether the second attribute information can be referenced in the prediction of the first attribute information, and can thus appropriately decode the bitstream.

[0391] For example, the first component includes a first dimension element and a second dimension element, and the bitstream includes first flag information indicating whether the second attribute information can be referenced in the prediction of the first dimension element, and second flag information indicating whether the second attribute information can be referenced in the prediction of the second dimension element. Thus, whether the second attribute information can be referenced can be switched for each dimension element, and thus the amount of data can be reduced.

[0392] For example, the bitstream includes coefficient information (for example, inter_attribute_pred_coeff) for calculating the first predicted value of the first attribute information. By using the coefficient information, the amount of residual data can be reduced.

[0393] For example, the bitstream includes a first data unit (eg, DADU) storing attribute information predicted with reference to other attribute information, and a second data unit (eg, ADU) storing attribute information not predicted with reference to other attribute information.

[0394] For example, Fig. 27As shown, the first attribute information includes RGB values, the second attribute information includes reflectance values, and at least one of the RGB values ​​is predicted with reference to the reflectance values. Thus, the amount of data can be reduced by utilizing the correlation between color and reflectance, which are usually correlated.

[0395] For example, the decoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0396] In addition, the encoding device (three-dimensional data encoding device) of this embodiment performs Fig.32 The processing shown. The encoding device generates a bit stream by encoding a plurality of three-dimensional points each having first attribute information and second attribute information. The encoding device predicts the first attribute information with reference to the second attribute information (S411), and encodes the first attribute information using the result of the prediction (S412). For example, the encoding device generates a bit stream by encoding the first attribute information. For example, the encoding device encodes the first attribute information using the result of the prediction (prediction value). For example, the encoding device calculates a first prediction value of the first attribute information of the three-dimensional point with reference to the second attribute information of the three-dimensional point. The encoding device calculates a first residual as a difference between the first value of the first attribute information and the first prediction value, and encodes the first residual.

[0397] Here, there may be a correlation between the attribute information. Therefore, by predicting the first attribute information with reference to the second attribute information, it is possible to reduce the data amount of the first attribute information.

[0398] In addition, the encoding device obtains attribute information of a plurality of three-dimensional points each having first attribute information and second attribute information, and encodes the first attribute information and the second attribute information of the plurality of three-dimensional points to generate a bitstream. The bitstream includes the encoded attribute information of the plurality of three-dimensional points and meta-information (e.g., dependency information or attr_param_type) indicating that the first attribute information is predicted with reference to the second attribute information. For example, the encoding device stores the meta-information in the bitstream. For example, the encoding device predicts the first attribute information with reference to the second attribute information.

[0399] Here, there may be a correlation between the attribute information. Therefore, by predicting the first attribute information with reference to the second attribute information, it is possible to reduce the data amount of the first attribute information.

[0400] For example, the predicted first attribute information and the referenced second attribute information are possessed by the same 3D point. Thus, by predicting the first attribute information of the 3D point with reference to the second attribute information of the 3D point, it is possible to reduce the amount of data of the first attribute information. Thus, the amount of data processed by the decoding device can be reduced.

[0401] For example, the predicted first attribute information and the referenced second attribute information are possessed by different 3D points. That is, a first 3D point has the predicted first attribute information, and a second 3D point different from the first 3D point has the referenced second attribute information.

[0402] For example, the first attribute information and the second attribute information are respectively stored in a first component (eg, a dependent component) and a second component (eg, a reference component).

[0403] For example, the first quantization step size for the first attribute information is larger than the second quantization step size for the second attribute information. By using a smaller quantization step size for the second attribute information referenced by other attribute information, it is possible to suppress degradation of the second attribute information and improve the accuracy of prediction of the first attribute information that references the second attribute information. In addition, for the first attribute information, by using a larger quantization step size, the amount of data can be reduced. Thus, it is possible to improve coding efficiency while suppressing degradation of the information as a whole.

[0404] For example, the first attribute information and the second attribute information are generated by respectively decoding the first coded attribute information and the second coded attribute information, and the second attribute information is losslessly compressed (reversibly compressed). For example, the first attribute information is irreversibly compressed.

[0405] Thus, by using lossless compression on the second attribute information referenced by other attribute information, degradation of the second attribute information can be suppressed, and the accuracy of prediction of the first attribute information that references the second attribute information can be improved. Thus, the coding efficiency can be improved while degradation of the information as a whole can be suppressed.

[0406] For example, Figure 8 As shown in FIG. 1 , each of the first component and the second component includes a first dimensional element and a second dimensional element, and the first dimensional element of the first component is predicted with reference to the first dimensional element of the second component, and the second dimensional element of the first component is predicted with reference to the second dimensional element of the second component. Thus, when there is a correlation between the same dimensional elements between components, the amount of data can be reduced by utilizing the correlation.

[0407] For example, Fig. 9 As shown in FIG. 1 , the first component includes the first dimensional element and the second dimensional element, the first dimensional element is predicted with reference to the second component, and the second dimensional element is predicted with reference to the first dimensional element. Depending on the attribute information, sometimes there is a correlation between different components, and sometimes there is a correlation between the dimensional elements constituting the component. Therefore, the amount of data can be reduced by using the correlation between components and the correlation between dimensional elements.

[0408] For example, the bitstream does not include information about the first prediction mode applied to the first component (e.g., pred_mode), but includes information about the second prediction mode applied to the second component (e.g., pred_mode). For example, the encoding device does not store information about the first prediction mode applied to the first component (e.g., pred_mode) in the bitstream, but stores information about the second prediction mode applied to the second component (e.g., pred_mode) in the bitstream. Thus, since the information about the first component can be reduced, the amount of data can be reduced.

[0409] For example, as described in the fourth example of prediction processing, the bitstream includes a first residual value of the first attribute information, the first residual value is the difference between the third residual value of the first attribute information and the second residual value of the second attribute information, the third residual value is the difference between the first value of the first attribute information and the first predicted value of the first attribute information, and the second residual value is the difference between the second value of the second attribute information and the second predicted value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the residual of the first attribute information and the residual of the second attribute information.

[0410] For example, as described in the first and second examples of the prediction process, the bitstream includes a first residual value of the first attribute information, and the first residual value is the difference between the first value of the first attribute information and the second value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the value of the first attribute information and the value of the second attribute information.

[0411] For example, as described in the first and third examples of prediction processing, the bitstream includes a first residual value of the first attribute information, and the first residual value is the difference between the first value of the first attribute information and the second predicted value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the value of the first attribute information and the predicted value of the second attribute information. In addition, since there is no need to calculate the predicted value of the first attribute information, the amount of processing can be reduced.

[0412] For example, the bitstream includes a first residual value of the first attribute information, and the first residual value is the difference between the first predicted value of the first attribute information and the second residual value of the second attribute information. Thus, the amount of data is reduced by encoding the difference between the predicted value of the first attribute information and the residual of the second attribute information.

[0413] For example, the bitstream includes flag information (e.g., inter_attribute_pred_enabled) indicating whether the second attribute information can be referenced in the prediction of the first attribute information. For example, the encoding device stores flag information (e.g., inter_attribute_pred_enabled) indicating whether the second attribute information can be referenced in the prediction of the first attribute information in the bitstream. Thus, the decoding device can refer to the flag information to determine whether the second attribute information can be referenced in the prediction of the first attribute information, and thus can appropriately decode the bitstream.

[0414] For example, the first component includes a first dimensional element and a second dimensional element, and the bitstream includes first flag information indicating whether the second attribute information can be referenced in the prediction of the first dimensional element, and second flag information indicating whether the second attribute information can be referenced in the prediction of the second dimensional element. For example, the encoding device stores the first flag information and the second flag information in the bitstream. Thus, it is possible to switch whether the second attribute information can be referenced for each dimensional element, thereby possibly reducing the amount of data.

[0415] For example, the bitstream includes coefficient information (eg, inter_attribute_pred_coeff) for calculating the first predicted value of the first attribute information. For example, the encoding device stores the coefficient information in the bitstream. By using the coefficient information, the amount of residual data can be reduced.

[0416] For example, the bitstream includes a first data unit (eg, DADU) storing attribute information predicted with reference to other attribute information, and a second data unit (eg, ADU) storing attribute information not predicted with reference to other attribute information.

[0417] For example, Fig. 27 As shown, the first attribute information includes RGB values, the second attribute information includes reflectance values, and at least one of the RGB values ​​is predicted with reference to the reflectance values. Thus, the amount of data can be reduced by utilizing the correlation between color and reflectance, which are usually correlated.

[0418] For example, the encoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.

[0419] In addition, the decoding device (three-dimensional data decoding device) of embodiment 2 is a decoding device that decodes the encoded attribute information of a three-dimensional point, generates a component by decoding the encoded component of the encoded attribute information, and divides the component into at least two sub-components.

[0420] Thus, the decoding device can restore the original two sub-components from the component obtained by combining the two sub-components. In addition, the amount of control signal (metadata) data is reduced by combining the two sub-components. Thus, the amount of data processed by the decoding device can be reduced.

[0421] Furthermore, the decoding device receives a bit stream including meta-information indicating that a component of the attribute information of the three-dimensional point includes a plurality of sub-components, and divides the component into the plurality of sub-components according to the meta-information.

[0422] Thus, the decoding device can restore the original multiple sub-components from the component into which the multiple sub-components are combined. In addition, the amount of control signal (metadata) data is reduced by combining the multiple sub-components. Thus, the amount of data processed by the decoding device can be reduced.

[0423] For example, the predicted value of the first dimensional element of the component is calculated with reference to the second dimensional element of the component. By combining, the number of dimensional elements contained in the same component increases. As a result, the number of candidates for reference targets in the case of reference between dimensional elements increases, which may improve coding efficiency.

[0424] For example, the bitstream includes information about the prediction mode common to all dimensional elements included in the component. By reducing the number of components by combining, the amount of this information can be reduced. Therefore, the amount of data can be reduced.

[0425] For example, the component includes a reflectance value and an RGB value, and the reflectance value is used to predict the G value of the RGB value. Thus, the amount of data can be reduced by utilizing the correlation between the color and the reflectance, which are generally correlated.

[0426] For example, the G value is referred to in the prediction of the R value and the B value in the RGB value. Thus, by utilizing the correlation of a plurality of colors of three-dimensional points that generally have correlation, the amount of data can be reduced.

[0427] Furthermore, the encoding device (three-dimensional data encoding device) of the second embodiment is an encoding device that encodes attribute information of three-dimensional points, combines multiple components of multiple attribute information into one component, and encodes the combined one component.

[0428] Thus, the data amount of the control signal (metadata) is reduced by combining the two sub-components.

[0429] In addition, the encoding device encodes the attribute information of the three-dimensional point, and generates a bit stream including the encoded attribute information and meta-information indicating that a component of the attribute information of the three-dimensional point includes a plurality of sub-components.

[0430] Thus, the data amount of the control signal (metadata) is reduced by combining a plurality of sub-components.

[0431] For example, the predicted value of the first dimensional element of the component is calculated with reference to the second dimensional element of the component. By combining, the number of dimensional elements contained in the same component increases. As a result, the number of candidates for reference targets in the case of reference between dimensional elements increases, which may improve coding efficiency.

[0432] For example, the bitstream includes information about the prediction mode common to all dimensional elements included in the component. By reducing the number of components by combining, the amount of this information can be reduced. Therefore, the amount of data can be reduced.

[0433] For example, the component includes a reflectance value and an RGB value, and the reflectance value is used to predict the G value of the RGB value. Thus, the amount of data can be reduced by utilizing the correlation between the color and the reflectance, which are generally correlated.

[0434] For example, the G value is referred to in the prediction of the R value and the B value in the RGB value. Thus, by utilizing the correlation of a plurality of colors of three-dimensional points that generally have correlation, the amount of data can be reduced.

[0435] As mentioned above, the three-dimensional data encoding device (encoding device) and the three-dimensional data decoding device (decoding device) according to the embodiment and the modified example of the present disclosure have been described, but the present disclosure is not limited to the embodiment.

[0436] In addition, each processing unit included in the three-dimensional data encoding device and the three-dimensional data decoding device of the above-mentioned embodiment is typically implemented as an integrated circuit, that is, LSI. They may be implemented as a single chip individually or in part or in whole.

[0437] In addition, the integrated circuit is not limited to LSI, and can also be realized by a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI.

[0438] In addition, in each of the above embodiments, each component may be formed by dedicated hardware, or implemented by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0439] In addition, the present disclosure may also be implemented as a three-dimensional data encoding method (encoding method) or a three-dimensional data decoding method (decoding method) executed by a three-dimensional data encoding device (encoding device) and a three-dimensional data decoding device (decoding device).

[0440] In addition, the division of the functional blocks in the block diagram is an example, and multiple functional blocks can also be implemented as one functional block, or one functional block can be divided into multiple blocks, or a part of the function can be moved to other functional blocks. In addition, a single hardware or software can process the functions of multiple functional blocks with similar functions in parallel or time-sharing.

[0441] In addition, the order of executing each step in the flowchart is for illustrative purposes to specifically explain the present disclosure, and may be an order other than the above. In addition, a part of the above steps may be executed simultaneously (in parallel) with other steps.

[0442] In the above, one or more embodiments of a three-dimensional data encoding device and a three-dimensional data decoding device are described based on the embodiments, but the present disclosure is not limited to the embodiments. As long as it does not deviate from the main purpose of the present disclosure, various modifications that can be thought of by those skilled in the art are implemented in the embodiments, and the embodiments constructed by combining the constituent elements of different embodiments can also be included in the scope of one or more embodiments.

[0443] Industrial availability

[0444] The present disclosure can be applied to a three-dimensional data encoding device and a three-dimensional data decoding device.

[0445] Description of Reference Numerals

[0446] 100, 300 3D data encoding device

[0447] 101, 302 Attribute information encoding unit

[0448] 111 LoD Generation Department

[0449] 112 Adjacency Search Department

[0450] 113 Forecasting Department

[0451] 114 Residual Calculation Unit

[0452] 115 Quantitative Department

[0453] 116 Inverse Quantization Unit

[0454] 117 Reconstruction Department

[0455] 118 Memory

[0456] 119 Arithmetic Coding Unit

[0457] 200, 400 3D data decoding device

[0458] 201, 401 Attribute information decoding unit

[0459] 211 LoD Generation Department

[0460] 212 Adjacency Search Department

[0461] 213 Forecasting Department

[0462] 214 Arithmetic decoding unit

[0463] 215 Inverse Quantization Unit

[0464] 216 Reconstruction Department

[0465] 217 Memory

[0466] 301 Merger Department

[0467] 402 Separation Department

Claims

1. A decoding method, receiving a bit stream in which a plurality of three-dimensional points each having first attribute information and second attribute information are encoded, The first attribute information is predicted with reference to the second attribute information.

2. A decoding method, Receive bit stream, decoding the bit stream, The bitstream comprises: Encoded attribute information of a plurality of three-dimensional points, each of the plurality of three-dimensional points having first attribute information and second attribute information; and The meta information indicates that the first attribute information is predicted with reference to the second attribute information.

3. The decoding method according to claim 1 or 2, The first attribute information to be predicted and the second attribute information to be referred to are possessed by the same three-dimensional point.

4. The decoding method according to claim 1 or 2, The first attribute information to be predicted and the second attribute information to be referred to are possessed by different three-dimensional points.

5. The decoding method according to claim 1 or 2, The first attribute information and the second attribute information are stored in a first component and a second component, respectively.

6. The decoding method according to claim 1 or 2, A first quantization step size for the first attribute information is larger than a second quantization step size for the second attribute information.

7. The decoding method according to claim 1 or 2, The first attribute information and the second attribute information are generated by decoding the first coded attribute information and the second coded attribute information, respectively. The second attribute information is losslessly compressed.

8. The decoding method according to claim 5, Each of the first component and the second component includes a first dimensional element and a second dimensional element, predicting the first dimensional element of the first component with reference to the first dimensional element of the second component, The second dimensional element of the first component is predicted with reference to the second dimensional element of the second component.

9. The decoding method according to claim 5, The first component includes a first dimensional element and a second dimensional element, predicting the first dimensional element with reference to the second component, The second dimensional element is predicted with reference to the first dimensional element.

10. The decoding method according to claim 5, In the bitstream, does not contain information about a first prediction mode applied to said first component, Rather, it contains information about a second prediction mode applied to the second component.

11. The decoding method according to claim 1 or 2, The bitstream includes a first residual value of the first attribute information, The first residual value is a difference between a third residual value of the first attribute information and a second residual value of the second attribute information. The third residual value is the difference between the first value of the first attribute information and the first predicted value of the first attribute information. The second residual value is a difference between a second value of the second attribute information and a second predicted value of the second attribute information.

12. The decoding method according to claim 1 or 2, The bitstream includes a first residual value of the first attribute information, The first residual value is a difference between a first value of the first attribute information and a second value of the second attribute information.

13. The decoding method according to claim 1 or 2, The bitstream includes a first residual value of the first attribute information, The first residual value is a difference between a first value of the first attribute information and a second predicted value of the second attribute information.

14. The decoding method according to claim 1 or 2, The bitstream includes flag information indicating whether the second attribute information can be referred to in prediction of the first attribute information.

15. The decoding method according to claim 1 or 2, The bitstream includes coefficient information for calculating a first prediction value of the first attribute information.

16. The decoding method according to claim 1 or 2, The bitstream includes a first data unit storing attribute information predicted with reference to other attribute information, and a second data unit storing attribute information not predicted with reference to other attribute information.

17. The decoding method according to claim 1 or 2, The first attribute information includes RGB values, The second attribute information includes a reflectance value, At least one of the RGB values ​​is predicted with reference to the reflectance value.

18. A method of encoding, generating a bit stream by encoding a plurality of three-dimensional points each having first attribute information and second attribute information, In the encoding method, predicting the first attribute information by referring to the second attribute information, The first attribute information is encoded using a result of the prediction.

19. A decoding device comprising: Processor; and Memory, The processor uses the memory, receiving a bit stream in which a plurality of three-dimensional points each having first attribute information and second attribute information are encoded, The first attribute information is predicted with reference to the second attribute information.

20. An encoding device, generating a bit stream by encoding a plurality of three-dimensional points each having first attribute information and second attribute information, The encoding device comprises: Processor; and Memory, The processor uses the memory, predicting the first attribute information by referring to the second attribute information, The first attribute information is encoded using a result of the prediction.

Images