Decoding method, encoding method, decoding device, and encoding device

Abstract

This decoding method comprises: receiving control information (S301) and decoding first attribute information regarding a first attribute of a three-dimensional point in accordance with the received control information (S302), wherein the received control information indicates that the first attribute depends on a second attribute of the three-dimensional point. For example, the control information may indicate that the first attribute information is a difference between the first attribute and the second attribute. For example, the control information may include at least one of offset information and scale information to be applied to the difference. For example, the offset information may be individually provided for each of a plurality of attributes of the three-dimensional point.

Description

Decoding method, encoding method, decoding device, and encoding device

[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device.

[0002] In the future, devices and services that utilize 3D data are expected to become widespread in a wide range of fields, including computer vision for autonomous operation of automobiles or robots, map information, surveillance, infrastructure inspection, video distribution, etc. 3D data can be acquired in a variety of ways, including distance sensors such as range finders, stereo cameras, or a combination of multiple monocular cameras.

[0003] One method of representing three-dimensional data is a representation called a point cloud, which represents the shape of a three-dimensional structure using a group of points in three-dimensional space. In a point cloud, the position and color of the points are stored. Point clouds are expected to become the mainstream method of representing three-dimensional data, but point clouds have a very large amount of data. Therefore, when storing or transmitting three-dimensional data, it is essential to compress the amount of data by encoding, just as with two-dimensional moving images (examples include MPEG-4 AVC or HEVC standardized by MPEG).

[0004] Furthermore, compression of point clouds is partially supported by a public library (Point Cloud Library) that performs processing related to point clouds.

[0005] Furthermore, a technique is known in which three-dimensional map data is used to search for and display facilities located around a vehicle (see, for example, Patent Document 1).

[0006] International Publication No. 2014 / 020663

[0007] In encoding and decoding three-dimensional data, it is desirable to be able to reduce the amount of encoded data.

[0008] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device that can reduce the amount of encoded data.

[0009] A decoding method according to one aspect of the present disclosure includes receiving control information, decoding first attribute information relating to a first attribute of a three-dimensional point according to the received control information, and the received control information indicating that the first attribute depends on a second attribute of the three-dimensional point.

[0010] An encoding method according to one aspect of the present disclosure encodes first attribute information relating to a first attribute of a three-dimensional point, and transmits the encoded first attribute information and control information, wherein the control information indicates that the first attribute depends on a second attribute of the three-dimensional point.

[0011] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can reduce the amount of encoded data.

[0012] FIG. 1 is a diagram showing an example of the configuration of point cloud data according to an embodiment. FIG. 2 is a block diagram showing the configuration of a three-dimensional data encoding device according to an embodiment. FIG. 3 is a block diagram showing the configuration of a three-dimensional data decoding device according to an embodiment. FIG. 4 is a diagram showing an example of information input / output to / from an attribute information conversion unit according to an embodiment. FIG. 5 is a diagram showing an example of the syntax of an SPS according to an embodiment. FIG. 6 is a diagram showing an example of the syntax of attribute_parameter according to an embodiment. FIG. 7 is a diagram showing the configuration of a bitstream according to an embodiment. FIG. 8 is a diagram showing the configuration of an attribute component according to an embodiment. FIG. 9 is a diagram showing an example of information stored in an SPS according to an embodiment. FIG. 10 is a diagram showing an example of information stored in an SPS according to an embodiment. FIG. 11 is a block diagram showing the configuration of an attribute information encoding unit according to an embodiment. FIG. 12 is a block diagram showing the configuration of an attribute information decoding unit according to an embodiment. FIG. 13 is a diagram showing an example of the syntax of attribute_parameter according to an embodiment. FIG. 14 is a flowchart of attribute information encoding processing according to an embodiment. FIG. 15 is a flowchart of attribute information decoding processing according to an embodiment. Fig. 16 is a diagram showing an example of the syntax of frame_attr_param_SEI according to an embodiment. Fig. 17 is a diagram showing an example of the syntax of angle_partial_access_SEI according to an embodiment. Fig. 18 is a flowchart of a decoding process when performing partial decoding according to an embodiment. Fig. 19 is a flowchart of a three-dimensional data decoding process according to an embodiment. Fig. 20 is a flowchart of a three-dimensional data encoding process according to an embodiment.

[0013] A decoding method according to one aspect of the present disclosure includes receiving control information, decoding first attribute information related to a first attribute of a three-dimensional point in accordance with the received control information, and decoding first attribute information related to a first attribute of the three-dimensional point, the first attribute being dependent on a second attribute of the three-dimensional point. The dependency of the first attribute on the second attribute may potentially reduce the amount of data in the first attribute information. A decoding device can appropriately decode coded data with reduced data amount using the control information. Furthermore, the amount of data handled by the decoding device can be reduced.

[0014] For example, the control information may indicate that the first attribute information is the difference between the first attribute and the second attribute. By having the first attribute information be the difference between the first attribute and the second attribute, it is possible to reduce the amount of data for the first attribute information. For example, when multiple colors are associated with a three-dimensional point as attributes, the multiple colors will be approximately the same depending on the region in which the three-dimensional point exists. In this case, the first attribute information will have a value close to 0 depending on the region, reducing the amount of data. Therefore, control information may be provided for each region.

[0015] For example, the control information may include at least one of offset information and scale information applied to the difference. By performing conversion using at least one of the offset information and scale information, it is possible to reduce the amount of data in the first attribute information. For example, when the first attribute and the second attribute satisfy a linear function relationship, the value of the first attribute information becomes close to 0.

[0016] For example, at least one of the offset information and the scale information may be individually provided for each of the plurality of pieces of attribute information of the three-dimensional point, thereby allowing at least one of the offset information and the scale information to be set for each attribute, thereby reducing the amount of data in the first attribute information.

[0017] For example, the first attribute information may be stored in a first component that is different from a second component in which second attribute information relating to the second attribute is stored.

[0018] For example, the number of first dimensions representing the first component may be different from the number of second dimensions representing the second component, thereby reducing the amount of data in the first attribute information even when the number of dimensions of the first attribute and the second attribute are different.

[0019] For example, the first attribute and the second attribute may represent a first color and a second color, respectively, of the 3D point, and each of the first color and the second color may be represented in multiple dimensions, thereby reducing the amount of data by exploiting dependencies between multiple colors of the 3D point that may be correlated.

[0020] For example, the first attribute and the second attribute may indicate the color and reflectance of the three-dimensional point, respectively, and the first dimensional number representing the color may be three, and the second dimensional number representing the reflectance may be one. This allows the amount of data to be reduced by utilizing the dependency between color and reflectance, which may be correlated. For example, when the reflectance is substantially equal to the minimum value, the color of the three-dimensional point having that reflectance is close to black, and when the reflectance is substantially equal to the maximum value, the color of the three-dimensional point having that reflectance is close to white. Therefore, if the encoding device and the decoding device agree that the first attribute information is decoded into information indicating black or white based on the reflectance, the amount of data may be reduced.

[0021] Furthermore, since there may be consecutive positions with maximum or minimum reflectance, control information indicating that reflectance and color are dependent on each other may be provided for each region.

[0022] For example, the decoding method may further decode second attribute information related to the second attribute, and convert the decoded first attribute information based on the decoded second attribute information, thereby restoring the original first attribute from the first attribute information whose data amount has been reduced by converting the first attribute information based on the second attribute information.

[0023] For example, the first attribute information may be a difference between the first attribute and the second attribute, and the conversion may restore the first attribute from the first attribute information and the second attribute information.

[0024] The encoding method according to the present embodiment encodes first attribute information relating to a first attribute of a three-dimensional point, and transmits the encoded first attribute information and control information, the control information indicating that the first attribute depends on a second attribute of the three-dimensional point. By making the first attribute dependent on the second attribute, it is possible to reduce the amount of data of the first attribute information.

[0025] For example, the control information may indicate that the first attribute information is a difference between the first attribute and the second attribute. By indicating that the first attribute information is a difference between the first attribute and the second attribute, it is possible to reduce the amount of data of the first attribute information.

[0026] For example, the control information may include at least one of offset information and scale information to be applied to the difference. By performing conversion using at least one of the offset information and scale information, it may be possible to reduce the amount of data of the first attribute information.

[0027] For example, at least one of the offset information and the scale information may be provided individually for each of a plurality of attributes of the three-dimensional point, thereby allowing at least one of the offset information and the scale information to be set for each attribute information, thereby reducing the amount of data of the first attribute information.

[0028] For example, the first attribute information may be stored in a first component that is different from a second component in which second attribute information relating to the second attribute is stored.

[0029] For example, the number of first dimensions representing the first component may be different from the number of second dimensions representing the second component, thereby reducing the amount of data in the first attribute information even when the number of dimensions of the first attribute and the second attribute are different.

[0030] For example, the first attribute and the second attribute may represent a first color and a second color, respectively, of the 3D point, and each of the first color and the second color may be represented in multiple dimensions, thereby reducing the amount of data by exploiting dependencies between multiple colors of the 3D point that may be correlated.

[0031] For example, the first attribute and the second attribute may indicate the color and reflectance of the three-dimensional point, respectively, and the first number of dimensions representing the color may be 3, and the second number of dimensions representing the reflectance may be 1. This makes it possible to reduce the amount of data by utilizing the dependency between color and reflectance, which may be correlated.

[0032] In addition, a decoding device according to one aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to receive control information and decodes first attribute information relating to a first attribute of a three-dimensional point according to the received control information, and the received control information indicates that the first attribute depends on a second attribute of the three-dimensional point.

[0033] Moreover, an encoding device according to one aspect of the present disclosure includes a processor and a memory, wherein the processor uses the memory to encode first attribute information relating to a first attribute of a three-dimensional point, and transmits the encoded first attribute information and control information, wherein the control information indicates that the first attribute depends on a second attribute of the three-dimensional point.

[0034] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0035] Hereinafter, the embodiments will be described in detail with reference to the drawings. Note that each of the embodiments described below represents a specific example of the present disclosure. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not recited in independent claims will be described as optional components.

[0036] (Embodiment) A three-dimensional data encoding device and a three-dimensional data decoding device according to this embodiment will be described below. The three-dimensional data encoding device generates a bit stream by encoding three-dimensional data. The three-dimensional data decoding device generates three-dimensional data by decoding the bit stream.

[0037] The three-dimensional data is, for example, three-dimensional point cloud data (also called point cloud data). A point cloud is a collection of multiple three-dimensional points and represents the three-dimensional shape of an object. The point cloud data includes position information and attribute information of multiple three-dimensional points. The position information indicates the three-dimensional position of each three-dimensional point. Note that the position information may also be called geometry information. For example, the position information is expressed in a Cartesian coordinate system or a polar coordinate system.

[0038] The attribute information indicates, for example, color information, reflectance, transmittance, infrared information, normal vector, time information, etc. One 3D point may have a single piece of attribute information or may have multiple types of attribute information.

[0039] The number of dimensions of attribute information varies depending on the type of attribute. For example, color information has three elements (three dimensions) such as RGB or YCbCr. Reflectance has one element (one dimension). Furthermore, attribute information may have different restrictions depending on the type of attribute, such as the range of values ​​that can be taken, bit depth, or whether negative values ​​can be taken.

[0040] Note that multiple types of attribute information are not limited to different types of information such as color information and reflectance, but also include different formats such as RGB and YcbCr, and different information contents such as color information when viewed from different viewpoints. In the following, different types of attribute information may also be simply referred to as different attribute information, etc.

[0041] The three-dimensional data is not limited to point cloud data, but may be other three-dimensional data such as mesh data. Mesh data (also called three-dimensional mesh data) is a data format used in CG (Computer Graphics) and represents the three-dimensional shape of an object as a collection of surface information. For example, mesh data includes point cloud information (e.g., vertex information). Therefore, methods similar to those for point cloud data can be applied to this point cloud information.

[0042] In conventional three-dimensional data encoding and decoding systems, multiple types of attribute information are encoded and decoded independently.

[0043] In this embodiment, when encoding multiple types of attribute information, the three-dimensional data encoding device converts at least one of the multiple types of attribute information using another attribute information, and encodes the converted information, thereby reducing the amount of encoded data.

[0044] For example, the point cloud data includes first attribute information and second attribute information, and the three-dimensional data encoding device generates third attribute information, which is difference information, by subtracting the first attribute information from the second attribute information for each point.

[0045] The three-dimensional data encoding device generates a bitstream by encoding the first attribute information and the third attribute information, respectively. The three-dimensional data encoding device also generates metadata (control information) indicating that the third attribute information is information generated by subtracting the first attribute information from the second attribute information, and stores the metadata in the bitstream.

[0046] The three-dimensional data decoding device decodes the first attribute information and the third attribute information, and restores the second attribute information using the decoded first attribute information, third attribute information, and metadata.

[0047] More specifically, for example, when a 3D point is observed from multiple viewpoints, the way the light hits and the appearance differs for each viewpoint, and therefore the 3D point has a different color value for each viewpoint. Figure 1 is a diagram showing an example of the configuration of such point cloud data. As shown in Figure 1, each point has position information, first color information, and second color information. Here, the first color information and the second color information are attribute information, such as color information corresponding to different viewpoints.

[0048] In such a point cloud, there is a possibility that the correlation between the first color information and the second color information is high. Therefore, by subtracting the first color information from the second color information using the above method, it is possible to generate difference information (third attribute information) having a smaller data size than the second color information. Therefore, the three-dimensional data encoding device may be able to reduce the amount of encoded data by encoding the difference information as the third attribute information instead of the second color information.

[0049] Furthermore, the three-dimensional data encoding device stores in the bitstream metadata indicating that the third attribute information is a difference between the first color information and the second color information, thereby enabling the three-dimensional data decoding device to restore the second color information from the first color information and the difference information based on the metadata.

[0050] The configurations of a three-dimensional data encoding device and a three-dimensional data decoding device according to this embodiment will be described below. FIG. 2 is a block diagram showing the configuration of a three-dimensional data encoding device 100 according to this embodiment. The three-dimensional data encoding device 100 generates encoded attribute information and encoding conversion information by encoding multiple pieces of attribute information (first to third attribute information). This three-dimensional data encoding device 100 includes an attribute information conversion unit 101 and an attribute information encoding unit 102. Note that FIG. 2 only shows the processing units related to encoding of attribute information. The three-dimensional data encoding device 100 may also include other processing units, such as a position information encoding unit that encodes position information.

[0051] The attribute information conversion unit 101 converts multiple pieces of attribute information (first attribute information, second attribute information, and third attribute information) into multiple pieces of converted attribute information. That is, the attribute information conversion unit 101 converts target attribute information included in the multiple pieces of attribute information by using reference attribute information included in the multiple pieces of attribute information that is different from the target attribute information. In addition, the attribute information conversion unit 101 generates conversion information that is information related to the conversion of the attribute information.

[0052] The attribute information conversion unit 101 generates conversion information based on the correlation between multiple pieces of attribute information. The conversion information is provided to reduce the data amount of converted attribute information. The conversion information includes, for example, information specifying reference attribute information used to convert target attribute information. The conversion information also includes information indicating the type of conversion. The information indicating the type of conversion is, for example, information specifying an arithmetic expression for conversion using target attribute information and reference attribute information. Note that the attribute information conversion unit 101 may generate conversion information based on the dependency between multiple pieces of attribute information instead of the correlation between multiple pieces of attribute information.

[0053] The attribute information conversion unit 101 may perform the conversion process using two pieces of attribute information out of the plurality of pieces of attribute information, or may perform the conversion process using three or more pieces of attribute information out of the plurality of pieces of attribute information. That is, the attribute information conversion unit 101 may convert the target attribute information using two or more pieces of reference attribute information. That is, the attribute information conversion unit 101 may generate one piece of converted attribute information using three or more pieces of attribute information. Furthermore, the plurality of pieces of converted attribute information output from the attribute information conversion unit 101 may or may not include attribute information that is not converted.

[0054] The attribute information encoding unit 102 encodes each of the multiple pieces of converted attribute information as one attribute component. The attribute information encoding unit 102 stores the encoded attribute information including the multiple attribute components in a bitstream. The attribute information encoding unit 102 also generates encoded conversion information by encoding conversion information for the multiple pieces of converted attribute information and identification information indicating a correspondence between the multiple pieces of converted attribute information and the multiple attribute components, and stores the generated encoded conversion information in the bitstream. For example, the identification information is information indicating a correspondence between each piece of converted attribute information and an identification number (attribute component identifier) ​​of the attribute component. The identification number is set, for example, based on the ordinal number of the attribute component in the bitstream.

[0055] FIG. 3 is a block diagram showing the configuration of a three-dimensional data decoding device 200 according to this embodiment. The three-dimensional data decoding device 200 generates multiple pieces of attribute information (first to third attribute information) by decoding the coded attribute information and coded conversion information included in the bitstream. This three-dimensional data decoding device 200 includes an attribute information decoding unit 201 and an attribute information inverse conversion unit 202. Note that FIG. 3 only shows the processing units related to the decoding of the attribute information. The three-dimensional data decoding device 200 may also include other processing units, such as a position information decoding unit that decodes position information.

[0056] The attribute information decoding unit 201 generates a plurality of pieces of converted attribute information by decoding the coded attribute information, and also generates conversion information and identification information by decoding the coded conversion information.

[0057] The attribute information inverse conversion unit 202 generates a plurality of pieces of attribute information (first to third attribute information) by inversely converting the plurality of pieces of converted attribute information based on the conversion information.

[0058] Next, an example of conversion processing in which difference information between the first color information and the second color information is generated will be described. Fig. 4 is a diagram showing an example of information input to and output from the attribute information conversion unit 101A in this case. The attribute information conversion unit 101A is an example of the attribute information conversion unit 101 shown in Fig. 2.

[0059] In this example, the attribute information conversion unit 101A does not perform conversion processing on the first color information for the multiple points that make up the point cloud, and outputs the first color information as is to the attribute information encoding unit 102 as the first converted attribute information.

[0060] For example, at point n=i, A(i) is output as is, where A(i) is the first color information of point i, and i is the identifier of the point (three-dimensional point).

[0061] On the other hand, for a plurality of points constituting the point cloud, the first color information is subtracted from the second color information to calculate first difference information.

[0062] For example, at point n=i, the first difference information C(i)=B(i)−A(i)=(R Ｂｉ -R Ａｉ , G Ｂｉ -G Ａｉ , B Ｂｉ -B Ａｉ ) is calculated, where B(i) is the second color information of point i. Ａｉ , G Ａｉ , B Ａｉ are the red, green, and blue components of the first color information, respectively, and R Ｂｉ , G Ｂｉ , B Ｂｉ are the red, green, and blue components of the second color information, respectively. That is, in this example, the first difference information has three-dimensional components.

[0063] Furthermore, the attribute information conversion unit 101A generates conversion information indicating that the first difference information was generated by subtracting the value of the first color information from the second color information.

[0064] Furthermore, when generating the first difference information, the attribute information conversion unit 101A sets the scale value S=(S ０ , S １ , S ２ ) and the scaling operation is a multiplication or division with an offset value O=(O ０ , O １ , O ２ ) may be used to perform at least one of an offset process, which is addition or subtraction.

[0065] For example, the first difference information C(i)=B(i)−A(i)×S+O=(R Ｂｉ -R Ａｉ ×S ０ +O ０ , G Ｂｉ -G Ａｉ ×S １ +O １ , B Ｂｉ -B Ａｉ ×S ２ +O ２ ) may be calculated.

[0066] This may allow the value of the difference information to be made smaller by the scale or offset value depending on the distribution of values ​​between attributes, or the mean or median of the attribute.

[0067] Furthermore, when the difference information is a negative value, the difference information can be converted to a positive value by adding offset information, which allows the attribute information encoding unit 102 to encode the difference information even if the attribute information encoding unit 102 does not support encoding of negative values.

[0068] The attribute information conversion unit 101A may determine the offset value or scale value based on predetermined setting information such as the bit depth of the attribute information. Alternatively, the attribute information conversion unit 101A may investigate the range of attribute information values ​​in the point cloud and determine the offset value or scale value based on the range. Alternatively, the attribute information conversion unit 101A may investigate the offset value or scale value that minimizes the value of the difference information. In other words, the attribute information conversion unit 101A may calculate difference information for each combination of offset value and scale value, and select the combination of offset value and scale value that provides the smallest value of difference information.

[0069] For example, an offset value or a scale value is set for each type of attribute information. A common offset value or a scale value may be used for point clouds of multiple frames, or an individual offset value or a scale value may be used for each frame. Alternatively, an individual offset value or a scale value may be used for each slice, which is a processing unit (group) into which a frame is divided.

[0070] Furthermore, when attribute information has elements in multiple dimensions, an offset value or a scale value may be set for each dimension, or a common offset value or a scale value may be set for all dimensions.

[0071] 4, third color information D may be input to the attribute information conversion unit 101A, and difference information E (=D-A) between the third color information D and the first color information A may be output. In this case, the offset value or scale value used to calculate difference information C may be the same as or different from the offset value or scale value used to calculate difference information E. Using different offset values ​​or scale values ​​for each piece of attribute information may further reduce the amount of encoded data.

[0072] Furthermore, the attribute information encoding unit 102 encodes the transformation information including the offset information indicating the offset value or the scale value, and stores the encoded transformation information in the bitstream.

[0073] Furthermore, while FIG. 4 shows an example in which difference information between two pieces of color information is calculated, difference information between attribute information of different information types, such as difference information between color information and reflectance, may also be calculated. In this case, the number of dimensions of the two pieces of attribute information for which difference information is calculated may be different. For example, difference information may be derived by subtracting three-dimensional color information A from one-dimensional reflectance I. For example, in this case, difference information C(i)=I(i)-A(i)=(I ｉ -R Ａｉ , I ｉ -G Ａｉ , I ｉ -B Ａｉ ) is calculated.

[0074] Note that the one-dimensional reflectance I may be subtracted from the three-dimensional color information A. In this case, an offset value or a scale value may also be used.

[0075] Furthermore, the two pieces of color information may have different color matrices, such as RGB and YUV or YCbCr. In this case, any combination of elements may be used to calculate the difference. For example, when calculating the difference between RGB and YUV, the difference between the R component and the Y component may be calculated, or the difference between the R component and the U component or the V component may be calculated.

[0076] Here, there may be a correlation between color information and reflectance. In such a case, by encoding the difference information between the color information and reflectance, the amount of encoded data can be reduced and encoding efficiency can be improved. Note that the combination of attribute information for calculating the difference is not limited to the above, and the present invention can be applied to any combination of correlated attribute information. An example of a combination of correlated attribute information is color information, reflectance, transmittance, and infrared information.

[0077] Although the above description has been given using an example of encoding attribute information of a point cloud for one frame, this method can also be applied to encoding attribute information of point clouds for multiple frames. Furthermore, this method can also be applied to encoding attribute information of point clouds for multiple frames using temporal prediction. That is, the first color information may be a first residual that is the difference between the first color information of the target frame and the first color information of the reference frame, the second color information may be a second residual that is the difference between the second color information of the target frame and the second color information of the reference frame, and the first difference information may be the difference between the first residual and the second residual. Furthermore, data for one frame may be divided into multiple slice data, and encoding may be performed for each slice data.

[0078] Next, an example of syntax related to the above processing will be described. Fig. 5 is a diagram showing an example of the syntax of SPS. SPS (Sequence Parameter Set) is metadata (parameter set) that is common to multiple frames.

[0079] This SPS includes sps_idx, common_information(), attribute_type, instance_id, num_dimension, num_attribute_parameter, attribute_parameter(i), and attribute_info().

[0080] sps_idx is an identifier of the SPS. common_information() is additional information related to the entire sequence. attribute_type, instance_id, num_dimension, num_attribute_parameter, and attribute_parameter(i) are set for each attribute information.

[0081] Attribute_type is an identifier for attribute information. Instance_id is an identifier for identifying instances of the same attribute type. Num_dimension indicates the number of dimensions of attribute information. Attribute_info() is other additional information related to attribute information.

[0082] The attribute_parameter() includes various types of additional information (metadata) for each attribute information and has a general-purpose format. The num_attribute_parameter indicates the number of attribute_parameters.

[0083] 6 is a diagram showing an example of the syntax of attribute_parameter(i). attribute_parameter(i) includes attr_param_type, attribute_source_offset_num_bits, attribute_source_offset, attribute_source_scale_num_bits, attribute_source_scale, and attribute_source_scale_frac_bits.

[0084] attr_param_type indicates the type of attribute_parameter. For example, when attribute_param_type=3, attribute_parameter(i) includes attribute_source_offset_num_bits, attribute_source_offset, attribute_source_scale_num_bits, attribute_source_scale, and attribute_source_scale_frac_bits.

[0085] Here, setting num_attribute_parameter to 0 or greater and enabling attribute_param_type=3 is equivalent to setting a flag indicating whether or not conversion information is included to valid (including conversion information).

[0086] Attribute_source_offset_num_bits, attribute_source_offset, attribute_source_scale_num_bits, attribute_source_scale, and attribute_source_scale_frac_bits are offset information relating to the offset value and scale value.

[0087] Attribute_source_offset indicates an offset value for restoring the decoded attribute information to the original (source) attribute information.

[0088] Attribute_source_scale indicates a scale value for restoring the decoded attribute information to the original attribute information.

[0089] attribute_source_offset_num_bits indicates the number of bits of the offset value (attribute_source_offset). Note that if the number of bits indicated by attribute_source_offset_num_bits is 0, this means that offset processing is not performed (i.e., offset = 0). In other words, attribute_source_offset_num_bits has the function of a flag that indicates whether or not to indicate an offset value (whether or not to perform offset processing).

[0090] attribute_source_scale_num_bits indicates the number of bits of the scale value (attribute_source_scale). If the number of bits indicated by attribute_source_scale_num_bits is 0, it means that scaling processing is not performed (i.e., scale = 1). In other words, attribute_source_scale_num_bits has the function of a flag that indicates whether or not to indicate a scale value (whether or not to perform scaling processing).

[0091] attribute_source_scale_frac_bits indicates the number of bits required to represent the decimal point of the scale value. The scale value is calculated using this attribute_source_scale_frac_bits and the following (Equation 1).

[0092]

[0093] Next, an example of encoding the color information, difference information, and transformation information will be described. The first color information and the first difference information shown in Fig. 4 are each encoded as one attribute component using any encoding method for attribute information, such as LoD-based attribute or Transform-based attribute.

[0094] The LoD-based attribute is a conversion method using Level of Detail (LoD) and is a method of calculating a prediction residual. The LoD is a method of layering three-dimensional points according to position information, and layering three-dimensional points according to the distance (denseness) between points.

[0095] The transform-based attribute is, for example, a RAHT (Region Adaptive Hierarchical Transform) method. RAHT is a method of transforming attribute information using position information of three-dimensional points, and by applying a Haar transform or the like to the attribute information, high-frequency components and low-frequency components of each layer are generated, and these values ​​are subjected to quantization, entropy coding, or the like.

[0096] Furthermore, the first color information is encoded as a three-dimensional first attribute component, and the first difference information is encoded as a three-dimensional second attribute component, where the first difference information is attribute information that has been transformed and has a reduced data amount, as described above.

[0097] 7 is a diagram showing the structure of a bit stream. As shown in FIG. 7, the bit stream includes SPS, GPS, APS(0), APS(1), Geom(0), Attr(0), and Attr(1).

[0098] The coded data of the first attribute component (first color information) is stored in data unit Attr(0). The data unit header of Attr(0) stores the attribute component identifier attr_id=0. The coded data of the second attribute component (first difference information) is stored in data unit Attr(1). The data unit header of Attr(1) stores the attribute component identifier attr_id=1.

[0099] 8 is a diagram showing the configuration of the first attribute component and the second attribute component. As shown in FIG. 8, each of the first attribute component and the second attribute component has three-dimensional elements.

[0100] Furthermore, metadata related to the encoding of attribute information is stored in the APS. APS (Attribute Parameter Set) is metadata (parameter set) related to the encoding of attribute information, with APS(0) being metadata related to the first attribute component and APS(1) being metadata related to the second attribute component. For example, APS is metadata common to multiple frames.

[0101] Here, an APS is provided for each attribute component, but if the same encoding parameters are used for two attribute components, one common APS may be provided for the two attribute components, and the one APS may be referenced by the two attribute components.

[0102] Geom(0) is encoded data of location information. GPS (Geometry Parameter Set) is metadata (parameter set) related to encoding of location information. For example, GPS is metadata common to multiple frames.

[0103] The transformation information is stored in at least one of the SPS and the SEI (Supplemental Enhancement Information). The SEI is extended information that stores parameters (optional parameters) that may not necessarily be used during decoding.

[0104] 9 is a diagram showing an example of information stored in an SPS. The SPS includes an attribute identifier (attribute_type = color) indicating that a first attribute component (Attribute0) is color information, an attribute component identifier (attr_id = 0) of the first attribute component, and the number of dimensions of the first attribute component (num_dimension = 3). The SPS also includes an attribute identifier (attribute_type = color difference) indicating that a second attribute component (Attribute1) is color information difference information, an attribute component identifier (attr_id = 1) of the second attribute component, and the number of dimensions of the second attribute component (num_dimension = 3).

[0105] Here, the attribute identifier of the second attribute component indicates that the attribute component (attribute information) is differential information of color information, but it may also indicate that the attribute component (attribute information) is converted attribute information, or that the attribute component (attribute information) is differential information of two pieces of attribute information.

[0106] Note that an attribute component may be indicated as difference information (or converted attribute information) using another method. For example, the attribute identifier may indicate that the attribute component is color information (attribute_type=color), and another syntax may be used to indicate that the information is difference information relative to reference color information. FIG. 10 is a diagram showing another example of information stored in the SPS in this case. Note that FIG. 10 is an example of information when the syntax shown in FIG. 13 (described later) is used. The example shown in FIG. 10 differs from the example shown in FIG. 9 in the information of the second attribute component (Attribute1). Specifically, the attribute identifier indicates that the second attribute component is color information (attribute_type=color), and information indicating that the second attribute component is converted information (attr_param_type=4, reference_attr_id=0, dependent_type=0) is added. Details of this information will be described later in the description of FIG.

[0107] Furthermore, the order in which the information of the attribute component is presented in the SPS may be defined as attr_id, and the attr_id may not be included in the SPS. In other words, the three-dimensional data decoding device may determine the attr_id of the attribute component based on the order in which the information of the attribute component is presented in the SPS.

[0108] Next, a description will be given of the configuration of the attribute information encoder 102 and the process of encoding difference information. Note that, here, an example will be described in which the attribute information encoder 102 is an encoder that performs encoding using LoD (LoD-based attribute encoder).

[0109] 11 is a block diagram showing the configuration of the attribute information encoding unit 102. The attribute information encoding unit 102 includes an LoD generation unit 111, an adjacent search unit 112, a prediction unit 113, a residual calculation unit 114, a quantization unit 115, an inverse quantization unit 116, a reconstruction unit 117, a memory 118, and an arithmetic encoding unit 119.

[0110] The LoD generation unit 111 generates an LoD hierarchy using position (geometry) information of a plurality of 3D points. The neighbor search unit 112 searches for 3D points (neighboring points) near a target point to be processed using the LoD generation result and distance information between the 3D points.

[0111] The prediction unit 113 generates a predicted value of the difference information. For example, the prediction unit 113 generates the predicted value using difference information of encoded adjacent points. The residual calculation unit 114 generates a prediction residual, which is the difference between the difference information and the predicted value.

[0112] The quantization unit 115 quantizes the prediction residual of the difference information. The inverse quantization unit 116 inversely quantizes the quantized prediction residual. The reconstruction unit 117 adds the predicted value and the inversely quantized prediction residual to generate a decoded value in which the prediction residual of the target point is decoded. The memory 118 stores difference information (decoded values) of multiple coded and decoded 3D points. The decoded values ​​stored in the memory 118 are used by the prediction unit 113 to predict subsequent 3D points.

[0113] The arithmetic coding unit 119 arithmetically codes the quantized prediction residual. Note that the arithmetic coding unit 119 may binarize the quantized prediction residual and arithmetically code the binarized prediction residual.

[0114] Note that different encoding parameters or quantization values ​​(quantization parameters) may be used when encoding differential information and when encoding reference components (reference attribute information) referenced by other attribute components. For example, reference components may be referenced by other components and therefore may be of high importance. Therefore, the reference components may be losslessly compressed, and the differential information may be lossily compressed. This can improve encoding efficiency while suppressing degradation of the overall information. In other words, the three-dimensional data encoding device may classify multiple attribute information to be encoded into reference components and differential information, and select encoding parameters appropriate for each of the reference components and differential information for encoding. This can be expected to improve encoding efficiency. Note that the reference components may be attribute components whose attribute information is not encoded as differential information (the attribute information is encoded as is).

[0115] Although an example of encoding differential information has been described here, the same encoding process as above can also be applied when encoding converted attribute information other than differential information, or when encoding attribute information that is not converted.

[0116] Next, a description will be given of the configuration of the attribute information decoding unit 201 and the process of decoding difference information. Note that, here, an example will be described in which the attribute information decoding unit 201 is a decoding unit that performs decoding using LoD (LoD based attribute decoder).

[0117] 12 is a block diagram showing the configuration of the attribute information decoding unit 201. The attribute information decoding unit 201 includes an LoD generation unit 211, an adjacent search unit 212, a prediction unit 213, an arithmetic decoding unit 214, an inverse quantization unit 215, a reconstruction unit 216, and a memory 217.

[0118] The LoD generator 211 generates LoDs using decoded position information of multiple 3D points. The neighbor searcher 212 searches for 3D points (neighboring points) near the target point to be processed using the LoD generation result and distance information between the 3D points.

[0119] The prediction unit 213 generates a predicted value of the difference information. For example, the prediction unit 213 generates a predicted value using difference information of adjacent points that have already been decoded.

[0120] The arithmetic decoding unit 214 generates prediction residuals (quantized prediction residuals) by arithmetically decoding encoded difference information included in the bitstream. The inverse quantization unit 215 inverse quantizes the arithmetically decoded prediction residuals (quantized prediction residuals). The reconstruction unit 216 generates decoded values ​​by adding predicted values ​​and dequantized prediction residuals. The memory 217 stores difference information (decoded values) of multiple decoded 3D points. The decoded values ​​stored in the memory 217 are used by the prediction unit 213 to predict subsequent 3D points.

[0121] Furthermore, although an example of decoding differential information has been described here, the same decoding process as above can also be applied when decoding converted attribute information other than differential information, or when decoding attribute information that is not converted.

[0122] Next, a modified example of the syntax of the conversion information will be described. Fig. 13 is a diagram showing an example of the syntax of attribute_parameter(i). The attribute_parameter shown in Fig. 13 is a parameter related to the i-th attribute information and includes conversion information. In this example, the conversion information indicates whether multiple attribute components are in a dependent relationship and indicates that dependency relationship.

[0123] Specifically, the attribute_parameter includes a reference_attr_id and a dependent_type when the attribute_param_type is 4. These pieces of information indicate that the attribute component depends on other components (is a dependent component).

[0124] Here, setting num_attribute_parameter to 0 or greater and enabling attribute_param_type=4 can be said to be equivalent to enabling a flag indicating that the attribute component is dependent on other components. This can also be said to be equivalent to enabling a flag indicating whether the original attribute information can be restored by the attribute component alone.

[0125] The "reference_attr_id" indicates the attribute component identifier of the reference (dependency) component. The "dependent_type" indicates how the attribute component depends on the reference component. For example, if "dependent_type" = 0, the attribute component is differential information for the reference component. A value other than 0 may be used to indicate other dependencies.

[0126] In addition, when attr_param_type=4 is defined to indicate that the attribute component is difference information for the reference component, dependent_type does not need to be included in the bitstream.

[0127] Reference_offset_num_bits, reference_offset, reference_scale_num_bits, reference_scale, and reference_scale_frac_bits are offset information relating to the offset value and scale value.

[0128] The reference_offset indicates an offset value for restoring the decoded attribute information to the original (source) attribute information.

[0129] Reference_scale indicates a scale value for restoring the decoded attribute information to the original attribute information.

[0130] Reference_offset_num_bits indicates the number of bits of the offset value (reference_offset). Note that if the number of bits indicated by reference_offset_num_bits is 0, this means that no offset processing is performed (i.e., offset = 0). In other words, reference_offset_num_bits functions as a flag indicating whether or not to indicate an offset value (whether or not to perform offset processing).

[0131] Reference_scale_num_bits indicates the number of bits of the scale value (reference_scale). If the number of bits indicated by reference_scale_num_bits is 0, it means that scaling processing is not performed (i.e., scale = 1). In other words, reference_scale_num_bits has the function of a flag that indicates whether or not to indicate a scale value (whether or not to perform scaling processing).

[0132] The reference_scale_frac_bits indicates the number of bits required to represent the decimal point of the scale value. The scale value S is calculated using the reference_scale_frac_bits and the following (Equation 2).

[0133]

[0134] Using the above syntax, the example SPS signal shown in Fig. 10 is generated. This makes it possible to notify the three-dimensional data decoding device that an attribute component having attr_id = 1 is difference information with respect to an attribute component having attr_id = 0.

[0135] The three-dimensional data decoding device decodes the conversion information and determines, based on the conversion information, that an attribute component having attr_id = 1 is difference information relative to an attribute component having attr_id = 0. The three-dimensional data decoding device can then restore the original attribute information using the difference information of the first attribute component after decoding and the attribute information of the second attribute component. In other words, by analyzing reference_attr_id and dependent_type, the three-dimensional data decoding device can uniquely determine a method for restoring attribute information using the attribute components after decoding.

[0136] For example, the attribute information after restoration of the i-th point is Z(i)=(Z ０ , Z １ , Z ２ ) can be derived using the following (Equation 3).

[0137]

[0138] Note that X(i) = (X ０ , X １ , X ２ ) is the i-th attribute information of the first attribute component after decoding, and Y(i)=(Y ０ , Y １ , Y ２ ) is the i-th attribute information of the second attribute component after decoding, O is the offset value between attributes, and S is the scale value between attributes.

[0139] Furthermore, clipping processing may be performed to limit the range of values ​​depending on the bit precision of the restored data. The bit precision of the restored data may be specified to be the same as the bit precision of the reference component, or information indicating the bit precision of the restored data may be included in metadata and notified to the 3D data decoding device.

[0140] Although the example shown here uses reference_attr_id, which indicates the attribute component identifier of the referenced attribute component, a group_id, which indicates the referenced group, may also be used. Here, one group includes multiple attribute components, and multiple attribute components are assigned to one group_id. The three-dimensional data encoding device and the three-dimensional data decoding device may reference any of the multiple attribute components included in the referenced group. For example, of the multiple attribute components included in the referenced group, the attribute component with the smallest value of the attribute component identifier may be referenced.

[0141] Furthermore, in the above, an example of conversion using the difference between two pieces of color information, a scale, and an offset has been described as an example of conversion of a plurality of pieces of attribute information, but other methods may also be used.

[0142] For example, if a 3D point has three or more pieces of color information, one of the pieces of color information may be used as a reference component, and the other attribute components may reference the same reference component. In this case, the 3D data decoding device may first decode the reference component, thereby restoring the other attribute components using the data of the decoded reference component. This improves random accessibility when an arbitrary attribute component needs to be decoded.

[0143] Furthermore, multiple reference components may be set, which may improve random accessibility when there are a large number of attribute components.

[0144] Alternatively, the original attribute information may not be used as is as the reference component, but may be converted from the original attribute information. For example, quantized original attribute information may be used as the reference component. Alternatively, a decoded value of an encoded reference component may be used as the reference component.

[0145] Alternatively, an average of multiple pieces of attribute information may be calculated, and the calculated average value may be used as the reference component. In this case, a reference component indicating the average value and multiple attribute components of multiple pieces of difference information are generated. The multiple attribute components correspond to the multiple original pieces of attribute information and indicate the difference between the corresponding original attribute information and the reference component. Therefore, in this case, the number of attribute components is increased by one.

[0146] In either case, the three-dimensional data decoding device can realize the restoration process by including information such as dependent_type indicating each process in the bitstream. For example, the restoration process in the three-dimensional data decoding device is uniquely defined by dependent_type.

[0147] Furthermore, the fact that the data is difference information (or conversion information) may be indicated by a syntax different from that of dependent_type.

[0148] Additionally, an attribute component that cannot be independently decoded without a reference component may be referred to as a dependent component.

[0149] Furthermore, for example, when color information from multiple viewpoints exists, color information from a predetermined initial viewpoint may be used as the reference component.

[0150] Alternatively, the number of reference components may be limited to one, and the instance_id of the reference component may be set to the value 0. In this case, if the attribute component to be processed is a dependent component, the three-dimensional data encoding device may not add the attribute component identifier of the reference component to the bitstream. In this case, the three-dimensional data decoding device may refer to the instance_id and determine that the attribute component with instance_id = 0 is the reference component.

[0151] Also, in the above description, an example has been given in which the type (type) or number of dimensions of attribute information differs between the attribute component to be processed and the reference component. In this case, dependent_type may indicate the reference method. For example, dependent_type may indicate the referenced (dependent) element. For example, if the number of dimensions of the reference component is greater than the number of dimensions of the attribute component to be processed, dependent_type may indicate which element of the multiple elements of the reference component each element of the attribute component to be processed references. Furthermore, dependent_type may indicate that the type of attribute information differs between the attribute component to be processed and the reference component, or that the number of dimensions differs. Note that this information may be indicated by information other than dependent_type included in the bitstream.

[0152] 14 is a flowchart of the attribute information encoding process performed by a three-dimensional data encoding device. First, the three-dimensional data encoding device starts decoding a plurality of pieces of attribute information (S101). Next, the three-dimensional data encoding device converts at least one piece of attribute information using other piece of attribute information (S102). Next, the three-dimensional data encoding device encodes the reference component and the dependent component using a predetermined method. Furthermore, the three-dimensional data encoding device stores information for each attribute component in the metadata (S103).

[0153] Next, if the attribute component to be processed is a dependent component (Yes in S104), the three-dimensional data encoding device stores in the metadata the attribute component identifier (ID) of the reference component of the dependent component, information indicating the dependency, etc. (S105).

[0154] 15 is a flowchart of the attribute information decoding process performed by the three-dimensional data decoding device. First, the three-dimensional data decoding device decodes a plurality of attribute components (reference components and dependent components) and metadata from the bitstream (S201).

[0155] If the attribute component to be processed is a dependent component (Yes in S202), the three-dimensional data decoding device identifies the reference component based on the attribute component identifier (ID) of the reference component described in the metadata included in the bitstream, and restores the attribute information using a predetermined method using information on the reference component, dependent component, and metadata (S203).

[0156] The processes shown in FIGS. 14 and 15 may be performed in units of frames or in units of slices.

[0157] In the above description, an example is shown in which attribute_parameter is stored in SPS, but it may also be stored in APS or SEI. For example, if the transformation information (dependency) changes for each frame, the transformation information may be stored in SEI for each frame. For example, frame_attr_param_SEI is an example of SEI for each frame.

[0158] In addition, when transformation information is indicated in both the SPS and the SEI for each frame, the three-dimensional data decoding device uses the SEI for each frame preferentially.

[0159] 16 is a diagram showing an example of the syntax of frame_attr_param_SEI in this case. In this example, attribute_idx specifies the attribute component to which the conversion information (attribute_parameter) is applied. In other words, conversion information is specified for each attribute component. Note that sps_idx shown in FIG. 16 is the index of the SPS, and frame_idx is the index of the frame.

[0160] Alternatively, conversion information common to all attribute components may be specified. For example, the conversion information may be included in an SEI common to all attribute components, such as attribute_structure_SEI. In this case, the three-dimensional data decoding device can determine the dependency relationships between multiple attribute components by analyzing the attribute_structure_SEI.

[0161] Also, an SEI to be used for partial decoding may be defined. Fig. 17 is a diagram showing an example of the syntax of this SEI (angle_partial_access_SEI).

[0162] For example, when multiple pieces of color information that differ for each viewpoint are encoded, the SEI includes viewpoint (angle) information and an attribute component identifier (attr_id). Furthermore, for a dependent component (dependent_flag=1), the attribute component identifier (reference_attr_id) of the reference component is indicated.

[0163] 17, num_angle indicates the number of viewpoints, and angle indicates a viewpoint. Dependent_flag indicates whether the attribute component is a dependent component.

[0164] Although not shown, information indicating the mode of conversion (for example, information indicating that it is difference information with respect to a reference component, an offset value, a scale value, etc.) may be included in the SEI. Furthermore, it is sufficient that this information is included in at least one of the SPS and the SEI.

[0165] 18 is a flowchart of a decoding process when partial decoding is performed. For example, the process shown in FIG. 18 is performed in the system layer.

[0166] First, the 3D data decoding device decodes and analyzes the SEI for partial decoding (S211). Next, the 3D data decoding device analyzes viewpoint information and obtains an attribute component identifier (ID) of the attribute information of the viewpoint to be decoded (S212). Next, if the attribute component is a dependent component, the 3D data decoding device obtains an attribute component identifier of the reference component (S213). Next, the 3D data decoding device searches the attribute data units, extracts data units having the obtained attribute component identifier, and decodes the extracted data units (S214).

[0167] In this way, the 3D data decoding device obtains attribute component identifiers (IDs) of attribute components (reference components and dependent components) of a particular viewpoint from the bitstream, and extracts data units using the attribute component identifiers.

[0168] Alternatively, instead of decoding the extracted data units, a bitstream or a file may be reconstructed from the extracted data units. For example, the edge may request a bitstream or a file containing attribute information of a specific viewpoint from the server. The server may generate a bitstream or a file from which attribute information of a specific viewpoint is extracted through the above process and transmit the bitstream or file to the edge. This may reduce the amount of data to be transmitted, thereby potentially shortening communication time.

[0169] As described above, the decoding device (three-dimensional data decoding device) according to this embodiment performs the processing shown in Fig. 19 . The decoding device receives control information (e.g., conversion information) (S301), and decodes first attribute information related to a first attribute of a three-dimensional point according to the received control information (S302), where the received control information indicates that the first attribute depends on a second attribute of the three-dimensional point. For example, the first attribute and the second attribute are attribute information before conversion. Furthermore, the first attribute information related to the first attribute may be information before conversion or information after conversion.

[0170] By making the first attribute dependent on the second attribute, it is possible to reduce the amount of data in the first attribute information. The decoding device can use the control information to appropriately decode the coded data with the reduced amount of data. Furthermore, the amount of data handled by the decoding device can be reduced.

[0171] For example, the control information indicates that the first attribute information is the difference between the first attribute and the second attribute. By having the first attribute information be the difference between the first attribute and the second attribute, it is possible to reduce the amount of data for the first attribute information. For example, when multiple colors are associated with a three-dimensional point as attributes, the multiple colors will be approximately the same depending on the region in which the three-dimensional point exists. In this case, the first attribute information will be close to 0 depending on the region, reducing the amount of data. Therefore, control information may be provided for each region.

[0172] For example, the control information includes at least one of offset information (e.g., an offset value) and scale information (a scale value) that are applied to the difference. By performing conversion using at least one of the offset information and the scale information, it is possible to reduce the amount of data in the first attribute information. For example, when the first attribute and the second attribute satisfy a linear function relationship, the value of the first attribute information becomes close to 0.

[0173] For example, at least one of the offset information and the scale information is provided individually for each of the multiple attributes of the three-dimensional point. This allows at least one of the offset information and the scale information to be set for each attribute information, thereby reducing the data amount of the first attribute information.

[0174] For example, the first attribute information is stored in a first component that is different from a second component in which the second attribute information related to the second attribute is stored. The second attribute information related to the second attribute may be information before or after conversion.

[0175] For example, the number of first dimensions representing the first component is different from the number of second dimensions representing the second component, thereby reducing the amount of data in the first attribute information even when the number of dimensions of the first attribute and the second attribute are different.

[0176] Here, a component is a unit of storing information, and a dimension represents information. For example, multiple dimensions are R, G, and B components in an RGB format. However, multiple dimensions are not limited to these, and may also be multiple components in a YUV format or a CMYK format, or multiple components in other formats.

[0177] For example, the first and second attributes may represent first and second colors, respectively, of the three-dimensional point, and each of the first and second colors may be represented in multiple dimensions, thereby reducing the amount of data by exploiting dependencies between the multiple colors of the three-dimensional point, which may be correlated.

[0178] For example, the first attribute and the second attribute indicate the color and reflectance of the 3D point, respectively, with the first dimensional number representing the color being 3 and the second dimensional number representing the reflectance being 1. This allows the amount of data to be reduced by utilizing the dependency between color and reflectance, which may be correlated. For example, when the reflectance is substantially equal to the minimum value, the color of the 3D point having that reflectance is close to black, and when the reflectance is substantially equal to the maximum value, the color of the 3D point having that reflectance is close to white. Therefore, if the encoding device and the decoding device agree that the first attribute information is decoded into information indicating black or white based on the reflectance, it may be possible to reduce the amount of data.

[0179] Furthermore, since there may be consecutive positions with maximum or minimum reflectance, control information indicating that reflectance and color are dependent on each other may be provided for each region.

[0180] For example, the decoding device further decodes second attribute information related to the second attribute and converts the decoded first attribute information based on the decoded second attribute information. In this manner, the decoding device can restore the original first attribute information from the first attribute information whose data volume has been reduced by converting the first attribute information based on the second attribute information.

[0181] For example, the first attribute information is the difference between the first attribute and the second attribute, and the decoding device restores the first attribute from the first attribute information and the second attribute information.

[0182] For example, the decoding device includes a processor and a memory, and the processor performs the above processing using the memory.

[0183] Moreover, the encoding device (three-dimensional data encoding device) according to this embodiment performs the processing shown in FIG. 20 . The encoding device encodes first attribute information related to a first attribute of a three-dimensional point (S401), and transmits the encoded first attribute information and control information (S402), where the control information indicates that the first attribute depends on a second attribute of the three-dimensional point. For example, the first attribute and the second attribute are attribute information before conversion. Furthermore, the first attribute information related to the first attribute may be information before conversion or information after conversion. By having the first attribute depend on the second attribute, it is possible to reduce the amount of data of the first attribute information.

[0184] For example, the control information indicates that the first attribute information is a difference between the first attribute and the second attribute, which may reduce the amount of data of the first attribute information.

[0185] For example, the control information includes at least one of offset information (e.g., an offset value) and scale information (e.g., a scale value) that is applied to the difference. By performing conversion using at least one of the offset information and the scale information, it is possible to reduce the amount of data in the first attribute information.

[0186] For example, at least one of the offset information and the scale information is provided individually for each of the multiple attributes of the three-dimensional point. This allows at least one of the offset information and the scale information to be set for each attribute information, thereby reducing the data amount of the first attribute information.

[0187] For example, the first attribute information is stored in a first component that is different from a second component in which the second attribute information related to the second attribute is stored. The second attribute information related to the second attribute may be information before or after conversion.

[0188] For example, the number of first dimensions representing the first component is different from the number of second dimensions representing the second component, thereby reducing the amount of data in the first attribute information even when the number of dimensions of the first attribute and the second attribute are different.

[0189] For example, the first and second attributes may represent first and second colors, respectively, of the three-dimensional point, and each of the first and second colors may be represented in multiple dimensions, thereby reducing the amount of data by exploiting dependencies between the multiple colors of the three-dimensional point, which may be correlated.

[0190] For example, the first attribute and the second attribute respectively indicate the color and reflectance of the three-dimensional point, and the first dimension number representing the color is 3, and the second dimension number representing the reflectance is 1. This makes it possible to reduce the amount of data by utilizing the dependency between color and reflectance, which may be correlated.

[0191] For example, the encoding device includes a processor and a memory, and the processor performs the above processing using the memory.

[0192] For example, the decoding method decodes first attribute information of at least one three-dimensional point based on information indicating whether the first attribute information depends on second attribute information of the at least one three-dimensional point. The dependence of the first attribute information on the second attribute information may reduce the amount of data of the first attribute information. The decoding device can appropriately decode the coded data with reduced data amount using the control information. Furthermore, the amount of data handled by the decoding device can be reduced.

[0193] For example, the decoding method may decode information about a plurality of attributes of a plurality of three-dimensional points, the plurality of attributes including a first attribute, the information including first information about the first attribute, the first information indicating a difference between the reference attribute and another attribute, and the first information being the difference between the reference attribute and another attribute, which may reduce the amount of data of the first information.

[0194] For example, the reference attribute is a second attribute, and the other attribute is the first attribute before conversion. For example, the decoding method decodes first attribute information and second attribute information of at least one three-dimensional point, and each of the first attribute information and second attribute information is generated based on multiple attributes of the at least one three-dimensional point. By generating each of the first attribute information and second attribute information into multiple attributes, it is possible to reduce the data amount of the first attribute information and second attribute information.

[0195] For example, the decoding method decodes information about a plurality of attributes of at least one 3D point based on transformation information, and the transformation information is used to decode only a portion of the information about the plurality of attributes. This allows selective transformation of a plurality of attributes, thereby efficiently reducing the amount of encoded data.

[0196] For example, the transformation information is generated based on correlations between at least two of the plurality of attributes. Transformation using correlations between the plurality of attributes may potentially reduce the amount of coded data.

[0197] For example, the transformation information may include offset information that is applied to at least some of the attributes. Transformation using offset values ​​may potentially reduce the amount of coded data.

[0198] For example, the conversion information includes information indicating the converted attributes. This allows selective conversion of a plurality of attributes, thereby efficiently reducing the amount of coded data.

[0199] For example, the encoding method encodes first attribute information of at least one three-dimensional point based on information indicating whether the first attribute information depends on second attribute information of the at least one three-dimensional point. By making the first attribute information dependent on the second attribute information, it is possible to reduce the amount of data of the first attribute information.

[0200] For example, the encoding method encodes information about a plurality of attributes of a plurality of 3D points, the plurality of attributes including a first attribute, the information including first information about the first attribute, the first information indicating a difference between the reference attribute and another attribute, and the first information being the difference between the reference attribute and another attribute, which may reduce the amount of data of the first information.

[0201] For example, the reference attribute is a second attribute, and the other attribute is the first attribute before conversion. For example, the encoding method encodes first attribute information and second attribute information of at least one three-dimensional point, and each of the first attribute information and second attribute information is generated based on multiple attributes of the at least one three-dimensional point. By generating each of the first attribute information and second attribute information into multiple attributes, it is possible to reduce the data amount of the first attribute information and second attribute information.

[0202] For example, the encoding method encodes information about a plurality of attributes of at least one 3D point based on transformation information, and the transformation information is used to encode only a portion of the information about the plurality of attributes. This allows selective transformation of a plurality of attributes, thereby efficiently reducing the amount of encoded data.

[0203] For example, the transformation information is generated based on correlations between at least two of the plurality of attributes. Transformation using correlations between the plurality of attributes may potentially reduce the amount of coded data.

[0204] For example, the transformation information may include offset information that is applied to at least some of the attributes. Transformation using offset values ​​may potentially reduce the amount of coded data.

[0205] For example, the conversion information includes information indicating the converted attributes. This allows selective conversion of a plurality of attributes, thereby efficiently reducing the amount of coded data.

[0206] For example, the conversion method (encoding method) converts first attribute information of at least one three-dimensional point based on second attribute information of the at least one three-dimensional point, and outputs the converted first attribute information to a decoding device. By converting the first attribute information based on the second attribute information, the data amount of the converted first attribute information can be reduced.

[0207] For example, the conversion method further converts the second attribute information and outputs the converted second information to a decoding device, thereby reducing the data amount of the converted second attribute information.

[0208] For example, the conversion method (encoding method) generates information about the difference between a reference attribute of a 3D point and another attribute, and outputs the information and the information about the reference attribute to a decoding device. By generating the information about the difference between the reference attribute and another attribute, the data amount of the information can be reduced.

[0209] For example, the conversion method (encoding method) converts attribute information of a three-dimensional point based on dependency information indicating the dependency of the attribute information on other attribute information of the three-dimensional point, and outputs the dependency information and the converted attribute information to a decoding device. By converting the attribute information based on the dependency information, the data amount of the attribute information can be reduced.

[0210] For example, the inverse transformation method (decoding method) transforms (inversely transforms) first attribute information of at least one three-dimensional point based on second attribute information of the at least one three-dimensional point. In this way, the inverse transformation method can restore the original first attribute information from the first attribute information whose data amount has been reduced by transforming the first attribute information based on the second attribute information.

[0211] For example, the inverse transformation method further transforms (inversely transforms) the second attribute information, thereby making it possible to restore the original second attribute information from the second attribute information with reduced data volume.

[0212] For example, the inverse transformation method (decoding method) receives information on the difference between a reference attribute of a 3D point and another attribute and information on the reference attribute, and restores the other attribute from the information on the reference attribute and the information on the reference attribute. In this way, the original data can be restored from the information whose data volume has been reduced by generating information on the difference between the reference attribute and the other attribute.

[0213] For example, the inverse transformation method (decoding method) receives attribute information of a 3D point and dependency information indicating dependency of the attribute information on other attribute information of the 3D point, and transforms (inversely transforms) the attribute information based on the dependency information. This makes it possible to restore the original attribute information from attribute information whose data volume has been reduced by transforming the attribute information based on the dependency information.

[0214] The above describes a three-dimensional data encoding device (encoding device) and a three-dimensional data decoding device (decoding device) according to embodiments and modifications of the present disclosure, but the present disclosure is not limited to these embodiments.

[0215] Furthermore, each processing unit included in the three-dimensional data encoding device and three-dimensional data decoding device according to the above embodiments is typically realized as an LSI, which is an integrated circuit. These may be individually implemented as single chips, or some or all of them may be included in a single chip.

[0216] Furthermore, the integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI may also be used.

[0217] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0218] Furthermore, the present disclosure may be realized as a three-dimensional data encoding method (encoding method) or a three-dimensional data decoding method (decoding method) executed by a three-dimensional data encoding device (encoding device) and a three-dimensional data decoding device (decoding device), etc.

[0219] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0220] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Also, some of the steps may be executed simultaneously (in parallel) with other steps.

[0221] While the three-dimensional data encoding device and three-dimensional data decoding device according to one or more aspects have been described above based on the embodiments, the present disclosure is not limited to these embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by those skilled in the art to the present embodiments, and configurations constructed by combining components of different embodiments, may also be included within the scope of one or more aspects.

[0222] The present disclosure is applicable to a three-dimensional data encoding device and a three-dimensional data decoding device.

[0223] 100 Three-dimensional data encoding device 101, 101A Attribute information conversion unit 102 Attribute information encoding unit 111 LoD generation unit 112 Adjacent search unit 113 Prediction unit 114 Residual calculation unit 115 Quantization unit 116 Inverse quantization unit 117 Reconstruction unit 118 Memory 119 Arithmetic encoding unit 200 Three-dimensional data decoding device 201 Attribute information decoding unit 202 Attribute information inverse conversion unit 211 LoD generation unit 212 Adjacent search unit 213 Prediction unit 214 Arithmetic decoding unit 215 Inverse quantization unit 216 Reconstruction unit 217 Memory

Claims

1. Receive control information; Decoding first attribute information relating to a first attribute of the three-dimensional point according to the received control information; The received control information indicates that the first attribute is dependent on a second attribute of the three-dimensional point. Decryption method.

2. The control information indicates that the first attribute information is a difference between the first attribute and the second attribute. The method of decoding according to claim 1.

3. The control information includes at least one of offset information and scale information to be applied to the difference. The decoding method according to claim 2.

4. At least one of the offset information and the scale information is provided individually for each of the plurality of pieces of attribute information of the three-dimensional point. The decoding method according to claim 3.

5. The first attribute information is stored in a first component that is different from a second component in which second attribute information relating to the second attribute is stored. The method of decoding according to claim 1.

6. A first dimensional number representing the first component is different from a second dimensional number representing the second component. The decoding method according to claim 5.

7. The control information includes an identifier of the second component as information indicating the second attribute on which the first attribute depends. The decoding method according to claim 5.

8. the first attribute and the second attribute indicate a first color and a second color, respectively, of the three-dimensional point; Each of the first color and the second color is represented in multiple dimensions. The method of decoding according to claim 1.

9. the first attribute and the second attribute indicate a color and a reflectance, respectively, of the 3D point; The first dimension representing the color is 3, and the second dimension representing the reflectance is 1. The method of decoding according to claim 1.

10. The decoding method further comprises: Decoding second attribute information relating to the second attribute; The decoded first attribute information is converted based on the decoded second attribute information. The method of decoding according to claim 1.

11. the first attribute information is a difference between the first attribute and the second attribute, In the conversion, the first attribute is restored from the first attribute information and the second attribute information. The decoding method according to claim 10.

12. The first attribute and the second attribute are correlated. The method of decoding according to claim 1.

13. In the decoding of the first attribute information, the first attribute information is obtained by decoding encoded data; The encoded data is generated using the first attribute and the second attribute. The method of decoding according to claim 1.

14. Encoding first attribute information relating to a first attribute of the three-dimensional points; Transmitting the encoded first attribute information and control information; The control information indicates that the first attribute is dependent on a second attribute of the three-dimensional point. Encoding method.

15. The control information indicates that the first attribute information is a difference between the first attribute and the second attribute. The encoding method according to claim 14.

16. The control information includes at least one of offset information and scale information to be applied to the difference.

16. The encoding method of claim 15.

17. At least one of the offset information and the scale information is provided individually for each of a plurality of attributes of the three-dimensional point.

17. The encoding method of claim 16.

18. The first attribute information is stored in a first component that is different from a second component in which second attribute information relating to the second attribute is stored. The encoding method according to claim 14.

19. A first dimensional number representing the first component is different from a second dimensional number representing the second component.

20. The encoding method of claim 18.

20. the first attribute and the second attribute indicate a first color and a second color, respectively, of the three-dimensional point; Each of the first color and the second color is represented in multiple dimensions. The encoding method according to claim 14.

21. the first attribute and the second attribute indicate a color and a reflectance, respectively, of the 3D point; The first dimension representing the color is 3, and the second dimension representing the reflectance is 1. The encoding method according to claim 14.

22. A processor; A memory, The processor uses the memory to: Receive control information; Decoding first attribute information relating to a first attribute of the three-dimensional point according to the received control information; The received control information indicates that the first attribute is dependent on a second attribute of the three-dimensional point. Decryption device.

23. A processor; A memory, The processor uses the memory to: Encoding first attribute information relating to a first attribute of the three-dimensional points; Transmitting the encoded first attribute information and control information; The control information indicates that the first attribute is dependent on a second attribute of the three-dimensional point. Encoding device.