3D decoder and 3D encoder

Abstract

The present invention relates to a method and an apparatus for encoding and decoding 3D information, and more particularly, to a method and an apparatus for encoding and decoding 3D information.SOLUTION: In the 3D data decoding apparatus for decoding encoded data, the mesh decoding unit 3031 included in the base mesh coding unit includes a mesh prediction unit 30311 for deriving a predicted value of a base mesh vertex position and / or a base mesh attribute from encoded data, and an arithmetic decoding unit 3051 for arithmetically decoding a prediction residual, wherein the arithmetic decoding unit decodes the first b I n of the prefix of the coefficient of the prediction residual using a context, decodes the first b I n of the suffix of the coefficient of the prediction residual using a context, and adds the predicted value and the prediction residual to derive the base mesh vertex position and / or the base mesh attribute.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a 3D data encoding device and a 3D data decoding device. [Background technology]

[0002] To efficiently transmit or record 3D data, there are 3D data encoding devices that convert the 3D data into 2D images, encode them using a video encoding method, and generate encoded data, and 3D data decoding devices that decode the 2D images from the encoded data and reconstruct the 3D data.

[0003] Specific examples of 3D data encoding methods include MPEG-I's ISO / IEC 23090-5 V3C (Volumetric Video-based Coding) and V-PCC (Video-based Point Cloud Compression). V3C can encode and decode point clouds consisting of point positions and attribute information. Furthermore, it can also be used to encode and decode multi-viewpoint video and mesh video using ISO / IEC 23090-12 (MPEG Immersive Video, MIV) and ISO / IEC 23090-29 (Video-based Dynamic Mesh Coding, V-DMC), which is currently being standardized. The latest draft document of the V-DMC method is disclosed in Non-Patent Document 1.

[0004] In these 3D data coding methods, the geometry and attributes that make up the 3D data are encoded and decoded as images using video coding methods such as H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding).

[0005] In the case of a point cloud, the geometry image is the depth to the projection plane, and the attribute image is the image of the attributes projected onto the projection plane.

[0006] 3D data (mesh) such as that in Non-Patent Document 1 consists of a base mesh, mesh displacement, and texture mapping image. A vertex coding method such as Draco can be used to encode the base mesh. Mesh displacement can be encoded by encoding the mesh displacement image, which is a two-dimensional representation of the mesh displacement, using a video codec, or by directly encoding it using arithmetic coding. The texture mapping image is encoded as an attribute image using a video codec. The video codec can be the above-mentioned HEVC or VVC. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Text of ISO / IEC CD 23090-29 Video-based mesh coding, ISO / IEC JTC 1 / SC 29 / WG 7 N0885, April 2024 [Non-patent document 2] Basemesh vertex positions and texture coordinates entropy coding and contexts improvements, ISO / IEC JTC 1 / SC 29 / WG 7 m67456, April 2024 Summary of the Invention [Problem to be solved by the invention]

[0008] In the 3D data coding method in Non-Patent Document 1, mesh displacements (mesh displacement arrays, mesh displacement images), mesh motion information, and base meshes that make up 3D data (meshes) can be coded and decoded using arithmetic coding. Non-Patent Document 2 proposes an arithmetic coding method that shares contexts between syntax elements of base meshes. When arithmetically coding mesh displacements, mesh motion information, and base meshes, there is a challenge of improving coding efficiency without increasing processing complexity.

[0009] The present invention aims to improve the coding efficiency of base meshes without increasing the complexity of processing when encoding and decoding 3D data using arithmetic coding methods, thereby enabling high-quality encoding and decoding of 3D data. [Means for solving the problem]

[0010] In order to solve the above problem, a 3D data decoding device according to one embodiment of the present invention is a 3D data decoding device that decodes encoded data, and is equipped with a mesh prediction unit that derives predicted values ​​of base mesh vertex positions and / or base mesh attributes from the encoded data, and an arithmetic decoding unit that arithmetically decodes predicted residuals, wherein the arithmetic decoding unit decodes the first M bins of a prefix of a coefficient of the predicted residual using a context, decodes the first N bins of a suffix of a coefficient of the predicted residual using a context, and adds the predicted value and the predicted residual to derive the base mesh vertex positions and / or the base mesh attributes.

[0011] In order to solve the above problem, a 3D data encoding device according to one embodiment of the present invention is a 3D data encoding device for encoding 3D data, comprising: a mesh prediction unit that derives predicted values ​​of base mesh vertex positions and / or base mesh attributes; and an arithmetic coding unit that arithmetically codes a prediction residual, wherein the arithmetic coding unit encodes the first M bins of a prefix of a coefficient of the prediction residual using a context, and encodes the first N bins of a suffix of the coefficient of the prediction residual using a context. [Effects of the Invention]

[0012] According to one aspect of the present invention, the coding efficiency of base meshes can be improved, and 3D data can be coded and decoded with high quality. [Brief explanation of the drawings]

[0013] [Figure 1]1 is a schematic diagram showing the configuration of a 3D data transmission system according to the present embodiment. [Figure 2] FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. [Figure 3] FIG. 2 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31. [Figure 4] FIG. 2 is a functional block diagram showing the configuration of a base mesh decoding unit 303. [Figure 5] FIG. 10 is a functional block diagram showing the configuration of a mesh displacement decoding unit 305. [Figure 6] FIG. 2 is a functional block diagram showing the configuration of a mesh reconstruction unit 307. [Figure 7] 10 is an example of a syntax for transmitting coordinate transformation parameters and context initialization parameters at the sequence level (ASPS). [Figure 8] 10 is an example of a syntax for transmitting coordinate transformation parameters and context initialization parameters at the picture / frame level (AFPS). [Figure 9] FIG. 10 is a diagram for explaining the operation of the mesh reconstruction unit 307. [Figure 10] 1 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11. FIG. [Figure 11] FIG. 2 is a functional block diagram showing the configuration of a base mesh encoding unit 103. [Figure 12] FIG. 2 is a functional block diagram showing the configuration of a mesh displacement encoding unit 107. [Figure 13] FIG. 2 is a functional block diagram showing the configuration of a mesh separation unit 115. [Figure 14] 10 is a diagram for explaining the operation of the mesh separating unit 115. FIG. [Figure 15] 1 is an example of a syntax structure for mesh displacement. [Figure 16] FIG. 3 is a functional block diagram showing the configuration of a mesh decoding unit 3031. [Figure 17] 1 is an example of a syntax structure for a base mesh vertex position. [Figure 18]1 is an example of the syntax structure of a base mesh attribute. [Figure 19] FIG. 10 illustrates how the context of a base mesh vertex position syntax element is derived. [Figure 20] FIG. 10 illustrates how the context of a syntax element of a base mesh attribute is derived. [Figure 21] FIG. 10 is a functional block diagram showing the configuration of a mesh encoding unit 1031. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a schematic diagram showing the configuration of a 3D data transmission system 1 according to this embodiment.

[0016] The 3D data transmission system 1 is a system that transmits an encoded stream obtained by encoding 3D data to be encoded, decodes the transmitted encoded stream, and displays the 3D data. The 3D data transmission system 1 includes a 3D data encoding device 11, a network 21, a 3D data decoding device 31, and a 3D data display device 41.

[0017] The 3D data T is input to the 3D data encoding device 11.

[0018] The network 21 transmits the encoded stream Te generated by the 3D data encoding device 11 to the 3D data decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Furthermore, the network 21 may be replaced by a storage medium on which the encoded stream Te is recorded, such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blu-ray Disc: registered trademark).

[0019] The 3D data decoding device 31 decodes each of the coded streams Te transmitted by the network 21, and generates one or more decoded 3D data Td.

[0020] The 3D data display device 41 displays all or part of one or more pieces of decoded 3D data Td generated by the 3D data decoding device 31. The 3D data display device 41 is equipped with a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Display forms include stationary, mobile, and HMD. Furthermore, if the 3D data decoding device 31 has high processing power, it displays high-quality images, and if it has only lower processing power, it displays images that do not require high processing power or display power.

[0021] <operator> The operators used in this specification are listed below.

[0022] >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR, |= is the OR assignment operator, and || indicates logical sum.

[0023] x?y:z is a ternary operator that takes y if x is true (non-zero) and z if x is false (zero). y..z denotes the set of integers from y to z.

[0024] <Structure of the coded stream Te> Before proceeding to a detailed description of the 3D data encoding device 11 and the 3D data decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the 3D data encoding device 11 and decoded by the 3D data decoding device 31 will be described.

[0025] FIG. 2 shows the hierarchical structure of data in the coded stream Te. The coded stream Te has either a V3C sample stream or a V3C unit stream. The V3C sample stream includes a sample stream header and a V3C unit. The V3C unit stream includes a V3C unit. include.

[0026] A V3C unit includes a V3C unit header and a V3C unit payload. The V3C unit header is a Unit Type, which is an ID that indicates the type of V3C unit, and takes values ​​indicated by labels such as V3C_VPS, V3C_AD, V3C_AVD, V3C_GVD, and V3C_OVD.

[0027] If the Unit Type is V3C_VPS (Video Parameter Set), the V3C unit contains a V3C parameter set.

[0028] When the Unit Type is V3C_AD (Atlas Data), the V3C unit includes a VPS ID, an atlasID, a sample stream nal header, and multiple NAL units. The atlasID is an ID (Identification) and takes an integer value of 0 or greater.

[0029] A NAL unit includes a NALUnitType, a layerID, a TemporalID, and an RBSP (Raw byte sequence payload).

[0030] NAL units are identified by NALUnitType, and are classified as ASPS (Atlas Sequence Parameter Set), AAPS (Atlas Adaptation Parameter Set), ATL (Atlas Tile layer), SEI (Supplemental Enhancement Information), etc.

[0031] The ATL includes an ATL header and an ATL data unit, and the ATL data unit includes information such as the patch position and size, such as patch information data.

[0032] The SEI includes payloadType, which indicates the type of the SEI, payloadSize, which indicates the size (number of bytes) of the SEI, and sei_payload, which is the data of the SEI.

[0033] If the Unit Type is V3C_AVD (Attribute Video Data), The unit includes the VPS ID, atlasID, attribute image ID attrIdx, partition ID partIdx, map ID mapIdx, a flag auxFlag indicating whether it is auxiliary data or not, and a video stream. The video stream is data encoded using HEVC, VVC, etc. In V-DMC, attribute data corresponds to a texture image.

[0034] If NalUnitType is V3C_GVD (Geometry Video Data), the V3C unit contains VPS ID, atlasID, mapIdx, auxFlag, and video stream. Geometry data corresponds to mesh displacement in V-DMC.

[0035] If the Unit Type is V3C_OVD (Occupancy Video Data), the V3C unit includes a VPS ID, an atlas ID, and a video stream.

[0036] If the Unit Type is V3C_MD (Mesh data), the V3C unit contains the VPS ID, atlas ID, and mesh_payload. In V-DMC, it corresponds to the base mesh.

[0037] (Configuration of 3D data decoding device according to the first embodiment) 3 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31 according to the first embodiment. The 3D data decoding device 31 is composed of a demultiplexing unit 301, an atlas information decoding unit 302, a base mesh decoding unit 303, a mesh displacement decoding unit 305, a mesh reconstruction unit 307, an attribute decoding unit 306, and a color space conversion unit 308. The 3D data decoding device 31 inputs encoded 3D data and outputs atlas information, meshes, and attribute images.

[0038] The demultiplexer 301 demultiplexes the byte stream format ISOBMFF (ISO Base Media File For It inputs encoded data multiplexed with V3C_AD Atlas Data stream (NAL unit) and demultiplexes it to output an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a base mesh encoded stream (V3C_MD mesh_payload), a mesh displacement encoded stream (V3C_GVD video stream), and an attribute video stream (V3C_AVD video stream).

[0039] The atlas information decoding unit 302 receives the atlas information coded stream output from the demultiplexing unit 301 and decodes the atlas information.

[0040] 3 decodes, from the encoded data, coordinate system transformation information displacementCoordinateSystem (asps_vdmc_ext_displacement_coordinate_system, afps_vdmc_ext_displacement_coordinate_system) indicating the coordinate system. Note that a separate gating flag may be provided, and each piece of coordinate system transformation information may be decoded only when the gating flag is 1. The gating flag may be, for example, afps_vdmc_ext_displacement_coordinate_system_enable_flag.

[0041] The base mesh decoding unit 303 decodes the base mesh coded stream coded by vertex coding (3D data compression coding method, for example, Draco), and outputs a base mesh. The base mesh will be described later.

[0042] The mesh displacement decoding unit 305 decodes the mesh displacement coded stream and outputs the mesh displacement.

[0043] The mesh reconstructing unit 307 receives the base mesh and the mesh displacement and reconstructs the mesh in the 3D space.

[0044] The attribute decoding unit 306 decodes an attribute video stream encoded using VVC, HEVC, or the like, and outputs an attribute image. The attribute image may be a texture image unfolded on the UV axis (a texture-mapped image converted using the UV atlas method) in YCbCr format. The type of codec used for encoding is indicated by ptl_profile_codec_group_idc, which is obtained by decoding the V3C parameter set of the encoded data. It may also be indicated by the Four CC code indicated by ai_geometry_codec_id[atlasID] in the V3C parameter set. ai_geometry_codec_id[atlasID] indicates an index in the atlas ID that corresponds to the codec ID of the decoder used to decode the attribute video stream.

[0045] The color space conversion unit 308 converts the color space of the attribute image from YCbCr format to RGB format. Note that an attribute video stream coded in RGB format may be decoded and color space conversion may be omitted.

[0046] (Decoding the base mesh) FIG. 4 is a functional block diagram showing the configuration of the base mesh decoding unit 303. The base mesh decoding unit 303 is composed of a mesh decoding unit 3031, a motion information decoding unit 3032, a mesh motion compensation unit 3033, a reference mesh memory 3034, a switch 3035, and a switch 3036. The base mesh decoding unit 303 may also include a base mesh inverse quantization unit (not shown) before the output of the base mesh. When the base mesh to be decoded is coded (intra-coded) without reference to other base meshes (e.g., base meshes that have already been coded and decoded), the switches 3035 and 3036 are connected to the side that does not perform motion compensation. On the other hand, when the base mesh to be decoded is coded (inter-coded) with reference to other base meshes, the switches 3035 and 3036 are connected to the side that performs motion compensation. When motion compensation is performed, the target vertex coordinates are derived by referring to already decoded vertex coordinates and motion information.

[0047] The mesh decoding unit 3031 decodes the intra-coded base mesh coded stream and outputs the base mesh (base mesh vertex positions, base mesh vertex position vectors). As the coding method, Draco, Edge Breaker, etc. are used.

[0048] The motion information decoding unit 3032 decodes the inter-coded base mesh coded stream and outputs motion information (mesh motion information, mesh motion vectors) for each vertex of a reference mesh (described later). Entropy coding such as arithmetic coding is used as the coding method.

[0049] The mesh motion compensation unit 3033 performs motion compensation on each vertex of the reference mesh input from the reference mesh memory 3034 based on the motion information, and outputs a motion-compensated mesh.

[0050] The reference mesh memory 3034 is a memory that holds the decoded mesh for reference in subsequent decoding processes.

[0051] (Mesh displacement decoding) 5 is a functional block diagram showing the configuration of the mesh displacement decoding unit 305. The mesh displacement decoding unit 305 is made up of a CABAC decoding unit (arithmetic decoding unit 3051, multi-value conversion unit 3052, context selection unit 3056, context initialization unit 3057), an inverse quantization unit 3053, an inverse transformation unit 3054, and a coordinate system transformation unit 3055.

[0052] (Context-adaptive binary arithmetic coding) The arithmetic decoding unit 3051, multi-value conversion unit 3052, context selection unit 3056, and context initialization unit 3057 use a decoding method that uses contexts called Context-Adaptive Binary Arithmetic Coding (CABAC). CABAC encodes and decodes a binary string consisting of 0s and 1s bit by bit using state variables (CABAC states) called contexts. All CABAC states are initialized at the beginning of a segment. The CABAC decoding unit decodes each bit of a binary string (Bin String) corresponding to a syntax element. When a context is used, a context index ctxInc is derived for each bit of the syntax element, the bit is decoded using the context, and the CABAC state of the context is updated. Bits that do not use a context are decoded with equal probability (EP, bypass), and updating of the index ctxIdx that specifies the context and the identified context is omitted. A context is a variable (memory area) for storing the CABAC probability (state), and is identified by the value of ctxIdx (0, 1, 2, ...). When 0 and 1 are always equally probable, i.e., 0.5, 0.5, this is called EP (Equal Probability) or bypass. In this case, no context is used because there is no need to maintain a state for a specific syntax element. It is also possible to use a static context where the probability is fixed at 0.5 and does not need to be updated. In this sense, it can be called static rather than bypass. An integer value such as 128 can be used to indicate a probability of 0.5.

[0053] The process of decoding one bit without using a context (bypassing) may use the following pseudo code. rangeTimesProb = IvlRange >> 1 binVal = ( rangeTimesProb <= ( IvlCode - IvlLow ) ) if (binVal == 0) IvlRange = rangeTimesProb else { IvlLow += rangeTimesProb IvlRange -= rangeTimesProb } The process of decoding one bit using the context may be performed using the following pseudo code: Here, prob0 is a variable that indicates the probability of the context. rangeTimesProb = IvlRange * prob0 >> 16 binVal = ( rangeTimesProb <= ( IvlCode - IvlLow ) ) if (binVal == 0) IvlRange = rangeTimesProb else { IvlLow += rangeTimesProb IvlRange -= rangeTimesProb } (Coordinate system) The following two types of coordinate systems are used for mesh displacement (3D vector). Cartesian coordinate system (canonical): A rectangular coordinate system commonly defined throughout the entire 3D space. (X, Y, Z) coordinate system. A rectangular coordinate system whose direction does not change at the same time (within the same frame, within the same tile). Local coordinate system (local): A Cartesian coordinate system defined for each region or vertex in 3D space. A Cartesian coordinate system whose direction can change at the same time (within the same frame, the same tile). A coordinate system with normal (D), tangent (U), and bi-tangent (V) axes. In other words, it is a Cartesian coordinate system consisting of the first axis (D) indicated by the normal vector n_vec at a vertex (or the face containing the vertex), and the second axis (U) and third axis (V) indicated by two tangent vectors t_vec and b_vec that are perpendicular to the normal vector n_vec. n_vec, t_vec, and b_vec are three-dimensional vectors. The (D, U, V) coordinate system may also be called the (n, t, b) coordinate system.

[0054] (Decoding and derivation of sequence-level control parameters) Here, the control parameters used in the mesh displacement decoding unit 305 will be explained.

[0055] Figure 7 shows an example of syntax for transmitting coordinate system transformation parameters using sequence-level ASPS. ASPS (Atlas Sequence Parameter Set or Atlas sequence mesh information) is one of the NAL units of atlas information and contains syntax elements that are applied to the atlas information coded stream. In ASPS, coordinate system transformation parameters are transmitted using the asps_vdmc_extension() syntax. The semantics of each field are as follows:

[0056] asps_vdmc_ext_subdivision_iteration_count: A parameter indicating the number of times the mesh is divided and iterated.

[0057] asps_vdmc_ext_displacement_coordinate_system: Coordinate system transformation information indicating the coordinate system of the mesh displacement. If the value is equal to a certain first value (e.g., 0), it indicates a Cartesian coordinate system. If the value is equal to another second value (e.g., 1), it indicates a local coordinate system.

[0058] asps_vdmc_ext_1d_displacement_flag: A flag indicating whether the mesh displacement is one-dimensional. If the value is true, it indicates that the mesh displacement is one-dimensional. If the value is false, it indicates that the mesh displacement is three-dimensional.

[0059] (Decoding and derivation of picture / frame level control parameters) Figure 8 shows an example of the syntax for transmitting coordinate system transformation parameters at the picture / frame level in AFPS. AFPS (Atlas Frame Parameter Set or Atlas frame mesh information) is one of the NAL units of atlas information and contains syntax elements that apply to the atlas information coded stream. In AFPS, coordinate system transformation parameters are transmitted using the afps_vdmc_extension() syntax. The semantics of each field are as follows:

[0060] afps_vdmc_ext_overriden_flag: A flag indicating whether to update the coordinate system of mesh displacement. If this flag is set to true, the coordinate system of mesh displacement is updated based on the value of afps_vdmc_ext_displacement_coordinate_system, which will be described later. If this flag is set to false, the coordinate system of mesh displacement is not updated.

[0061] afps_vdmc_ext_subdivision_iteration_count: A parameter indicating the number of mesh division iterations.

[0062] afps_vdmc_ext_displacement_coordinate_system: Coordinate system transformation information indicating the coordinate system of the mesh displacement. If the value is equal to the first value (e.g., 0), it indicates a Cartesian coordinate system. If the value is equal to the second value (e.g., 1), it indicates a local coordinate system. If the syntax element is not present, the value is inferred as the value decoded by ASPS, and the default coordinate system is the coordinate system indicated by ASPS.

[0063] afps_vdmc_ext_1d_displacement_flag: A flag indicating whether the mesh displacement is one-dimensional. If the value is true, it indicates that the mesh displacement is one-dimensional. If the value is false, it indicates that the mesh displacement is three-dimensional.

[0064] (Mesh displacement syntax structure) Figure 15 shows an example of the syntax structure of mesh displacement. The semantics are as follows: Mesh displacement is a sequence of values ​​(coefficients) for the position pos and k-th component (k-component), and is represented by the array Qdisp[pos][k]. Displacement is a three-dimensional signal in the Cartesian coordinate system (xyz) or the local coordinate system (ntb), and each component of the three-dimensional displacement is called a component. The displacement Qdisp here is also called a coefficient because it is the value after transformation using discrete wavelet transform, lifting transform, DCT transform, etc. The component variable k takes the values ​​0, 1, or 2. The variable name is not limited to k, and dim can be any other variable name. The index order of QDisp can be reversed; that is, Qdisp[k][pos] can be used instead of Qdisp[pos][k].

[0065] diu_last_sig_coeff[k]: An index indicating the last position of a non-zero mesh displacement coefficient in the kth component.

[0066] diu_coded_block_flag[k][b]: indicates whether the block with index b in the kth component contains a non-zero mesh displacement coefficient: 1 if yes, 0 if not.

[0067] diu_coded_subblock_flag[k][b][s]: Indicates whether the subblock with index s of the block with index b in the kth component contains a non-zero mesh displacement coefficient. 1 if yes, 0 if not.

[0068] diu_coeff_abs_level_gt0[k][b][s][v]: Indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex with index v of the sub-block with index s of the block with index b in the kth component is greater than 0. If it is greater, it is 1; otherwise, it is 0.

[0069] diu_coeff_abs_level_gt1[k][b][s][v]: Indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex with index v of the sub-block with index s of the block with index b in the kth component is greater than 1. If it is greater, it is set to 1; otherwise, it is set to 0. If this syntax element is not present, it is assumed to be 0.

[0070] diu_coeff_abs_level_gt2[k][b][s][v]: Indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex with index v of the sub-block with index s of the block with index b in the kth component is greater than 2. If it is greater, it is set to 1; otherwise, it is set to 0. This syntax element If not present, it is assumed to be 0.

[0071] diu_coeff_abs_level_gt3[k][b][s][v]: Indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex with index v of the sub-block with index s of the block with index b in the kth component is greater than 3. If it is greater, it is set to 1; otherwise, it is set to 0. If this syntax element is not present, it is assumed to be 0.

[0072] diu_coeff_sign[k][b][s][v]: Indicates whether the non-zero mesh displacement coefficient of the vertex with index v of the sub-block with index s of the block with index b in the kth component is positive. For example, 1 if it is positive, 0 if it is not (negative). If this syntax element is not present, it is assumed to be 1.

[0073] diu_coeff_abs_level_rem[k][b][s][v]: The absolute value of the non-zero mesh displacement coefficient of the vertex with index v of the sub-block with index s of the block with index b in the kth component minus 4. If this syntax element is not present, it is assumed to be 0.

[0074] The mesh displacement decoding unit 305 decodes diu_last_sig_coeff for each component of the mesh displacement, and then derives the number of lods of the k component, lodCount, from diu_last_sig_coeff[k].

[0075] The mesh displacement decoding unit 305 decodes diu_coded_block_flag for each level of detail (lod) of mesh displacement, and derives the number of blocks b, vertexCount, from diu_coded_block_flag[k][b].

[0076] The mesh displacement decoding unit 305 decodes diu_coded_subblock_flag for each block of the mesh displacement, and derives the start position vStart of the subblock s from diu_coded_subblock_flag[k][b][s].

[0077] The mesh displacement decoding unit 305 decodes diu_coeff_abs_level_gt0 for each sub-block of the mesh displacement, and if diu_coeff_abs_level_gt0 is a predetermined value (for example, other than 0), decodes the subsequent diu_coeff_sign and diu_coeff_abs_level_gt1.

[0078] If diu_coeff_abs_level_gt1 is a predetermined value (for example, other than 0), the mesh displacement decoding unit 305 decodes the subsequent diu_coeff_abs_level_gt2.

[0079] If diu_coeff_abs_level_gt2 is a predetermined value (for example, other than 0), the mesh displacement decoding unit 305 decodes the subsequent diu_coeff_abs_level_gt3.

[0080] If diu_coeff_abs_level_gt3 is a predetermined value (for example, other than 0), the mesh displacement decoding unit 305 decodes the subsequent diu_coeff_abs_level_rem.

[0081] (Operation of mesh displacement decoding unit) The arithmetic decoding unit 3051 decodes the arithmetically coded mesh displacement coded stream according to a value (context) indicating a random variable, and outputs a binary signal. The binary signal may be an alpha code or a k-th order Exp-Golomb code. The Exp-Golomb code consists of prefix and suffix codes. The prefix is ​​an exponentially increasing value, and the suffix is ​​the remainder. When encoding and decoding the variable rem using the Exp-Golomb code, the prefix and suffix of the Exp-Golomb code are also called the prefix and suffix of rem.

[0082] The multi-value unit 3052 decodes the binary signal into a quantized mesh displacement Qdisp, which is a multi-value signal.

[0083] The context selector 3056 (context memory) has a memory for storing contexts, derives the context to be used for arithmetic decoding of the mesh displacement according to the state, and updates the value as necessary. For arithmetic decoding of each coefficient of the mesh displacement, the following different context arrays may be used depending on the frame type ft (e.g., 0: intraframe, 1: interframe), the mesh division level lod (level of detail), and the mesh displacement vector component dim. The context includes a variable indicating the occurrence probability of a binary signal. ctxCodedSubBlock[numFT][numLOD][numDim] ctxCoeffGtN[numFT][numLOD][MAX_GTN+1][numDim] ctxCoeffRemPrefix[numFT][numLOD][numDim][numPrefixBin] Note that a static context with a fixed probability that does not update the context is called ctxStatic. The syntax element indicated by ctxStatic may be decoded without using a context. decode(ctxStatic) may use a process dedicated to bypassing the context as decode_bypass().

[0084] Here, numFT is the number of frame types, and numFT = 2. numPrefixBin is the number of bins that use context in the prefix, and numPrefixBin = 2 may be used. numLOD is the maximum number of levels of detail for mesh division, and the value of the syntax element asps_vdmc_ext_subdivision_iteration_count or afps_vdmc_ext_subdivision_iteration_count decoded from the bitstream may be used, or numLOD=4 may be used. numLOD = asps_vdmc_ext_subdivision_iteration_count numLOD = afps_vdmc_ext_subdivision_iteration_count numDim is the number of dimensions of the mesh displacement vector, and may be the value of the syntax element asps_vdmc_ext_1d_displacement_flag or afps_vdmc_ext_1d_displacement_flag decoded from the bitstream, or numDim=3. numDim = asps_vdmc_ext_1d_displacement_flag ? 1 : 3 numDim = afps_vdmc_ext_1d_displacement_flag ? 1 : 3 The maximum value of the threshold for the coefficients, MAX_GTN, is 3.

[0085] ctxCodedSubBlock[numFT][numLOD][numDim] is a context array used for decoding the syntax element diu_coded_subblock_flag. The arithmetic decoding unit 3051 uses the values ​​of ctxCodedSubBlock[ft][lod][dim] to decode diu_coded_subblock_flag for the frame type ft, the level of detail lod, and the dimension dim of the mesh displacement vector.

[0086] ctxCoeffGtN[numFT][numLOD][MAX_GTN+1][numDim] is a context array used for decoding the syntax element diu_coeff_abs_level_gtN (N is replaced with 0, 1, 2, or MAX_GTN). The arithmetic decoding unit 3051 uses the values ​​of ctxCoeffGtN[ft][lod][N][dim] to decode diu_coeff_abs_level_gtN for the frame type ft, the level of detail lod, and the dimension dim of the mesh displacement vector.

[0087] The arithmetic decoding unit 3051 uses a bypass to decode diu_coeff_sign for the frame type ft, the level of detail lod, and the dimension dim of the mesh displacement vector.

[0088] ctxCoeffRemPrefix[numFT][numLOD][numDim][numPrefixBin] is an array of contexts used for decoding the syntax element diu_coeff_abs_level_rem. ctxCoeffRemPrefix[bin] indicates the context at the bin position in the binarization of the prefix of diu_coeff_abs_level_rem. The arithmetic decoding unit 3051 sets the values ​​of ctxCoeffRemPrefix[ft][lod][dim] as is used to decode diu_coeff_abs_level_rem for frame type ft, level of detail lod, and dimension dim of the mesh displacement vector.

[0089] The context initialization unit 3057 initializes the context (probability of occurrence of a binary signal). The context may be initialized for each frame, or for each set of one or more frames (Group of Frames, GoF). When the context is initialized for each frame, there is no context dependency between frames, making it easy to randomly access any frame. When the context is initialized for each GoF, the initialization is less frequent, and therefore coding efficiency can be improved compared to when the context is initialized for each frame.

[0090] (Mesh displacement derivation process) The mesh displacement decoding unit 305 decodes the syntax elements diu_last_sig_coeff, diu_coded_block_flag, diu_coded_subblock_flag, diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3, diu_coeff_abs_level_rem, and diu_coeff_sign through the following processing to derive the mesh displacement Qdisp. Here, the mesh displacement decoding unit 305 decodes diu_last_sig_coeff for each component. It decodes diu_coded_block_flag for each LOD (block), and decodes diu_coded_subblock_flag for each subblock of the subBlockSize size. If diu_coded_subblock_flag is a predetermined value, it decodes the mesh displacement coefficients within the subblock. for (k = 0; k < numDim; k++) { / / dimension (component) loop / / decode diu_last_sig_coeff diu_last_sig_coeff[k] = decodeExpGolomb(ctxStatic) dispOffset = 0 for (b = 0; b <numLOD; b++) { / / Level of Detail loop, block loop / / decode diu_coded_block_flag diu_coded_block_flag[k][b] = decode(ctxStatic) if (diu_coded_block_flag[k][b]) { numSubBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numSubBlocks; s++) { / / subblock loop / / decode diu_coded_subblock_flag diu_coded_subblock_flag[k][b][s] = decode(ctxCodedSubBlock[ft][b][k]) if (diu_coded_subblock_flag[k][b][s]) { for (v = 0; v < subBlockSize; v++) { / / coefficient loop within subblock value = 0 / / decode diu_coeff_abs_level_gt0 diu_coeff_abs_level_gt0[k][b][s][v] = decode(ctxCoeffGtN[ft][b][0][k]) if (diu_coeff_abs_level_gt0[k][b][s][v]) { value++ / / decode diu_coeff_sign diu_coeff_sign[k][b][s][v] = decode(ctxStatic) / / decode diu_coeff_abs_level_gt1 diu_coeff_abs_level_gt1[k][b][s][v] = decode(ctxCoeffGtN[ft][b][1][k]) if (diu_coeff_abs_level_gt1[k][b][s][v]) { value++ / / decode diu_coeff_abs_level_gt2 diu_coeff_abs_level_gt2[k][b][s][v] = decode(ctxCoeffGtN[ft][b][2][k]) if (diu_coeff_abs_level_gt2[k][b][s][v]) { value++ / / decode diu_coeff_abs_level_gt3 diu_coeff_abs_level_gt3[k][b][s][v] = decode(ctxCoeffGtN[ft][b][3][k]) if (diu_coeff_abs_level_gt3[k][b][s][v]) { / / decode diu_coeff_abs_level_rem diu_coeff_abs_level_rem[k][b][s][v] = decodeExpGolomb(ctxCoeffRemPrefix[ft][b][k]) value += (1 + diu_coeff_abs_level_rem) } } } if (diu_coeff_sign[k][b][s][v]) { value = -value } } Qdisp[dispOffset + s * subBlockSize + v][k] = value } } } } dispOffset += dispCount[b] } } Here, decode(ctx) is a function that decodes a 1-bit value using the corresponding context ctx as an argument, and decodeExpGolomb(ctxPrefix, ctxSuffix) is a function that decodes a value binarized with k-th order Exponential Golomb code (for example, k=0). ctxPrefix[n] is used as the context for bin position n of the prefix, and ctxSuffix[m] is used as the context for bin position m of the suffix. When no context is used for the suffix (when a bypass is used), simply write decodeExpGolomb(ctxPrefix). value++ is an operation that increments the variable value by 1, value += 1, value = value + 1. subBlockSize is the size of the subblock. for indicates a loop. subBlockSize may be a power of 2 between 16 and 4096, for example 128, 256. dispCount[b] is the number of mesh displacements at level of detail b.

[0091] The inverse quantization unit 3053 performs inverse quantization based on the quantization scale value iscale and derives the mesh displacement Tdisp after transformation (e.g., wavelet transform). Tdisp may be in a Cartesian coordinate system or a local coordinate system. iscale is a value derived from the quantization parameter of each component of the mesh displacement image. Tdisp[0][] = (Qdisp[0][] * iscale[0] + iscaleOffset) >> iscaleShift Tdisp[1][] = (Qdisp[1][] * iscale[1] + iscaleOffset) >> iscaleShift Tdisp[2][] = (Qdisp[2][] * iscale[2] + iscaleOffset) >> iscaleShift Here, iscaleOffset = 1<<(iscaleShift-1). iscaleShift may be a predetermined constant, or may be a value decoded from encoded data at the sequence level, picture / frame level, tile / patch level, etc.

[0092] The inverse transform unit 3054 performs an inverse transform g (for example, an inverse wavelet transform) to derive a mesh displacement d. d[0][] = g(Tdisp[0][]) d[1][] = g(Tdisp[1][]) d[2][] = g(Tdisp[2][]) The coordinate system conversion unit 3055 converts the mesh displacement (coordinate system of the mesh displacement) into a Cartesian coordinate system based on the value of the coordinate system conversion information displacementCoordinateSystem. Specifically, when displacementCoordinateSystem==1, the displacement in the local coordinate system is converted into a displacement in the Cartesian coordinate system. Here, d is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion. disp is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and is a Cartesian coordinate system. n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target region or target vertex. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } The derivation method shown above for vector multiplication can be expressed individually as scalars as follows: if (displacementCoordinateSystem == 0) { for (i = 0; i < 3; i++) {disp[i] = d[i]}} else if (displacementCoordinateSystem == 1){ for (i = 0; i < 3; i++) {disp[i] = d[0] * n_vec[i] + d[1] * t_vec[i] + d[2] * b_vec[i]} } Alternatively, the same variable name may be assigned before and after the transformation as disp=d, and the value of d may be updated by the coordinate transformation.

[0093] Alternatively, the following configuration may be used. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 2){ disp = d[0] * n_vec2 + d[1] * t_vec2 + d[2] * b_vec2 } Here, n_vec2, t_vec2, and b_vec2 are three-dimensional vectors (in the Cartesian coordinate system) corresponding to the axes of the local coordinate system of the adjacent region.

[0094] The following configuration may also be used. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } Here, n_vec3, t_vec3, and b_vec3 are 3D vectors (in a Cartesian coordinate system) corresponding to the axes of the local coordinate system of the target region with fluctuations suppressed. For example, the vectors of the coordinate system used for decoding are derived from the previous coordinate system and the current coordinate system as follows n_vec3 = (w*n_vec3 + (WT - w)*n_vec)>>wShift t_vec3 = (w*t_vec3 + (WT - w)*t_vec)>>wShift b_vec3 = (w*b_vec3 + (WT - w)*b_vec)>>wShift Here, for example, wShift = 2, 3, 4, WT = 1<<wShift, and w = 1..WT - 1. For example, when w = 3 and wShift = 3 n_vec3 = (3*n_vec3 + 5*n_vec)>>3 t_vec3 = (3*t_vec3 + 5*t_vec)>>3 b_vec3 = (3*b_vec3 + 5*b_vec)>>3 Also, a configuration may be adopted such that it can be selected according to the value of the coordinate system conversion information displacementCoordinateSystem decoded from the encoded data as in the following configuration if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 6){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } (Decoding of the base mesh) 16 is a functional block diagram showing the configuration of the mesh decoding unit 3031. The mesh decoding unit 3031 is made up of a mesh prediction unit 30311 and a CABAC decoding unit (an arithmetic decoding unit 3051, a multi-value conversion unit 3052, a context selection unit 3056, and a context initialization unit 3057).

[0095] (Base mesh syntax structure) 17 shows an example of the syntax structure of base mesh vertex position (vertex position vector prediction residual, base mesh vertex position vector). The semantics are as follows:

[0096] The mesh prediction unit 30311 classifies the predicted position vector of the current vertex as a first category, Fine, if the number of available predicted position vector candidates (position vectors of decoded vertices adjacent to the current vertex) is equal to or greater than a predetermined value, or as a second category, Coarse, if the number is less than the predetermined value. Depending on the classification, the position residual may be decoded using different syntax elements, mesh_position_fine_residual[i][j] and mesh_position_coarse_residual[i][j]. Alternatively, the context of the arithmetic code may be switched depending on the classification, as described below. Alternatively, the syntax elements may remain unchanged depending on the classification, but the variables obtained by decoding the syntax elements may be changed. Note that the classification method is not limited to this.

[0097] mesh_position_fine_residuals_count: indicates the size of the array mesh_position_fine_residual, which contains the prediction residuals of the fine predicted position 3D vectors.

[0098] mesh_position_coarse_residuals_count: Indicates the size of the array mesh_position_coarse_residual, which contains the prediction residuals of the coarse predicted position 3D vector.

[0099] mesh_coded_position_fine_residuals_size: Indicates the byte size of the arithmetic coding data of the prediction residuals of the fine prediction position 3D vector.

[0100] mesh_position_fine_residual[i][j]: indicates the value of the prediction residual for the j-th dimension (component) of the i-th fine predicted position 3D vector.

[0101] mesh_coded_position_coarse_residuals_size: Indicates the byte size of the arithmetic coding data of the prediction residuals of the coarse prediction position 3D vector.

[0102] mesh_position_coarse_residual[i][j]: Indicates the value of the prediction residual for the j-th dimension (component) of the i-th coarse predicted position 3D vector.

[0103] Figure 18 shows an example of the syntax structure of the base mesh attributes (texture coordinate prediction residual, base mesh texture coordinate, normal vector prediction residual, base mesh normal vector). The semantics are as follows:

[0104] The mesh prediction unit 30311 classifies the predicted attribute of the current vertex as fine if the number of available predicted attribute candidates (attributes of decoded vertices adjacent to the current vertex) is equal to or greater than a predetermined value, and as coarse if the number is less than the predetermined value. The classification method is not limited to this. Depending on the classification, the residual of the attribute (texture coordinates, normal vector, etc.) may be decoded using different syntax elements, mesh_attribute_fine_residual[i][j] and mesh_attribute_coarse_residual[i][j]. Alternatively, the context of the arithmetic code may be switched depending on the classification, as described below. Alternatively, depending on the classification, the syntax elements may remain unchanged, but the variables obtained by decoding the syntax elements may be changed. Note that the classification method is not limited to this.

[0105] mesh_attribute_fine_residuals_count[i]: indicates the size of the array mesh_attribute_fine_residual, which contains the prediction residuals of the fine N-dimensional vector of the i-th attribute.

[0106] mesh_attribute_coarse_residuals_count[i]: Indicates the size of the array mesh_attribute_coarse_residual, which contains the prediction residuals of the coarse N-dimensional vector of the i-th attribute.

[0107] mesh_coded_attribute_fine_residuals_size[i]: Indicates the byte size of the arithmetic coding data of the prediction residuals of the fine N-dimensional vector of the i-th attribute.

[0108] mesh_attribute_fine_residual[i][j][k]: Indicates the value of the prediction residual for the k-th dimension (component) of the j-th fine N-dimensional vector of the ith attribute.

[0109] mesh_coded_attribute_coarse_residuals_size[i]: Indicates the byte size of the arithmetic coding data of the prediction residuals of the coarse N-dimensional vector of the i-th attribute.

[0110] mesh_attribute_coarse_residual[i][j][k]: Indicates the value of the prediction residual for the kth dimension (component) of the jth coarse N-dimensional vector of the ith attribute.

[0111] Also, mesh_attribute_count indicates the number of attributes, and NumComponents[i] indicates the number of components (dimensions) of the i-th attribute.

[0112] (Operation of mesh decoding unit) The mesh prediction unit 30311 predicts the position vector of the current vertex based on the position vectors of the decoded vertices, and derives predicted position vectors BmVertexPosFinePred (fine predicted position vector) and BmVertexPosCoarsePred (coarse predicted position vector).

[0113] The mesh prediction unit 30311 predicts the attributes of the current vertex based on the attributes (texture coordinates, normal vectors) of the decoded vertex, and derives the predicted attributes BmVertexAttrFinePred (fine prediction attribute) and BmVertexAttrCoarsePred (coarse prediction attribute).

[0114] The basic operation of the CABAC decoding unit (arithmetic decoding unit 3051, multi-value conversion unit 3052, context selection unit 3056, context initialization unit 3057) is the same as that of the CABAC decoding unit of the mesh displacement decoding unit 305. In the arithmetic decoding of the position vector prediction residual (coefficient) of each vertex and the attribute prediction residual (coefficient), the prediction residual (coefficient) may be decoded by dividing it into a binarization (for example, Exponential Golomb code, Rice code) consisting of a prefix and a suffix. The prefix / suffix (for example, prefix part / suffix part of Exponential Golomb code), Fine / Coarse, vertex position / attribute, etc. may be decoded. Depending on the channel, etc., the following different context arrays (the number in [] is the number of elements in the array) may be used. The context includes a variable that indicates the occurrence probability of a binary signal. ctxCoeffRemPrefixPosFine[nbPfxCtxFine] ctxCoeffRemSuffixPosFine[nbSfxCtxFine] ctxCoeffRemPrefixPosCoarse[nbPfxCtxCoarse] ctxCoeffRemSuffixPosCoarse[nbSfxCtxCoarse] ctxCoeffRemPrefixAttrFine[mesh_attribute_count][nbPfxCtxFine] ctxCoeffRemSuffixAttrFine[mesh_attribute_count][nbSfxCtxFine] ctxCoeffRemPrefixAttrCoarse[mesh_attribute_count][nbPfxCtxCoarse] ctxCoeffRemSuffixAttrCoarse[mesh_attribute_count][nbSfxCtxCoarse] Here, mesh_attribute_count is the number of attributes (texture coordinates, normal vectors, etc.) for the vertex, decoded from the sequence-level parameter set (ASPS).

[0115] where nbPfxCtxFine, nbSfxCtxFine, nbPfxCtxCoarse, nbSfxCtxCoarse are the number of contexts, and are fixed values, e.g. nbPfxCtxFine = N1 nbSfxCtxFine = N1 nbPfxCtxCoarse = N2 nbSfxCtxCoarse = N2 Here, N1 >= N2 may be used, with the course being the smaller value. For example, {N1=6, N2=3}, {N1=5, N2=4}, or {N1=4, N2=4} may be used.

[0116] Alternatively, different values ​​may be set for the prefix and the suffix. nbPfxCtxFine = N1A nbSfxCtxFine = N1B nbPfxCtxCoarse = N2A nbSfxCtxCoarse = N2B Here, N1A >= N1B, N1A >= N1B may be used, with the suffix being the smaller value. For example, {N1A=6, N1B=5}, {N2A=6, N2B=5}, or {N1A=5, N1B=5}, {N2A=5, N2B=5} may be used.

[0117] Alternatively, the same value may be set for fine and coarse. nbPfxCtxFine = NA nbSfxCtxFine = NB nbPfxCtxCoarse = NA nbSfxCtxCoarse = NB Here, the suffix may be smaller than NA >= NB. For example, {NA=6, It is also possible to use {NB=3}, {NA=5, NB=4}, or {N1=4, N2=4}. Also, different values ​​may be set for the vertex position and the attribute.

[0118] ctxCoeffRemPrefixPosFine[nbPfxCtxFine] is a context array used to decode the prefix of the syntax element mesh_position_fine_residual[i][j]. The arithmetic decoding unit 3051 decodes the bin of the index BinIdxPfx in the binarization of the prefix of mesh_position_fine_residual[i][j] using the value of ctxCoeffRemPrefixPosFine[min(nbPfxCtxFine - 1, BinIdxPfx)]. Here, the index is a variable ranging from 0 to the bin length N-1 indicating the bit position of the bin string. If the bin string is bin0, bin1, bin2, ..., binN-1, the bins of index BinIdxPfx=0, 1, 2, ..., N-1 correspond to bin0, bin1, bin2, ..., binN-1.

[0119] ctxCoeffRemSuffixPosFine[nbSfxCtxFine] is the syntax element mesh_position_fine_r The arithmetic decoding unit 3051 decodes the bin of the index BinIdxSfx in the binarization of the suffix of mesh_position_fine_residual[i][j] by using the value of ctxCoeffRemSuffixPosFine[min(nbSfxCtxFine - 1, BinIdxSfx)].

[0120] ctxCoeffRemPrefixPosCoarse[nbPfxCtxCoarse] is a context array used to decode the prefix of the syntax element mesh_position_coarse_residual[i][j]. The arithmetic decoding unit 3051 decodes the bin of the index BinIdxPfx in the binarization of the prefix of mesh_position_coarse_residual[i][j] using the value of ctxCoeffRemPrefixPosCoarse[min(nbPfxCtxCoarse - 1, BinIdxPfx)].

[0121] ctxCoeffRemSuffixPosCoarse[nbSfxCtxCoarse] is a context array used to decode the suffix of the syntax element mesh_position_coarse_residual[i][j]. The arithmetic decoding unit 3051 decodes the bin of the index BinIdxSfx in the binarization of the suffix of mesh_position_coarse_residual[i][j] using the value of ctxCoeffRemSuffixPosCoarse[min(nbSfxCtxCoarse - 1, BinIdxSfx)].

[0122] ctxCoeffRemPrefixAttrFine[i][nbPfxCtxFine] is a context array used to decode the prefix of the syntax element mesh_attribute_fine_residual[i][j][k]. The arithmetic decoding unit 3051 decodes the bin of the index BinIdxPfx in the binarization of the prefix of mesh_attribute_fine_residual[i][j][k] using the value of ctxCoeffRemPrefixAttrFine[i][min(nbPfxCtxFine - 1, BinIdxPfx)].

[0123] ctxCoeffRemSuffixAttrFine[i][nbSfxCtxFine] is a context array used to decode the suffix of the syntax element mesh_attribute_fine_residual[i][j][k]. The arithmetic decoding unit 3051 decodes the bin of the index BinIdxSfx in the binarization of the suffix of mesh_attribute_fine_residual[i][j][k] using the value of ctxCoeffRemSuffixAttrFine[i][min(nbSfxCtxFine - 1, BinIdxSfx)].

[0124] ctxCoeffRemPrefixAttrCoarse[i][nbPfxCtxCoarse] is a context array used to decode the prefix of the syntax element mesh_attribute_coarse_residual[i][j][k]. The arithmetic decoding unit 3051 decodes the bin of the index BinIdxPfx in the binarization of the prefix of mesh_attribute_coarse_residual[i][j][k] using the value of ctxCoeffRemPrefixAttrCoarse[i][min(nbPfxCtxCoarse - 1, BinIdxPfx)].

[0125] ctxCoeffRemSuffixAttrCoarse[i][nbSfxCtxCoarse] is a context array used to decode the suffix of the syntax element mesh_attribute_coarse_residual[i][j][k]. The arithmetic decoding unit 3051 decodes the bin of the index BinIdxSfx in the binarization of the suffix of mesh_attribute_coarse_residual[i][j][k] using the value of ctxCoeffRemSuffixAttrCoarse[i][min(nbSfxCtxCoarse - 1, BinIdxSfx)].

[0126] (Configuration that uses context for some bins in prefix / suffix) 19 shows the settings of the context used by the mesh decoding unit 3031. The mesh decoding unit 3031 (context selection unit 3056) may decode a portion of the prefixes (e.g., M bins, where M = numPrefixCtxBinsFine) of mesh_position_fine_residual[i][j] using bins (BinIdxPfx <= numPrefixCtxBinsFine - 1) as the context, the other bins of the prefixes (BinIdxPfx > numPrefixCtxBinsFine - 1) as the bypass, the portion of the suffixes (e.g., N bins, where N = numSuffixCtxBinsFine) using bins (BinIdxSfx <= numSuffixCtxBinsFine - 1) as the context, and the other bins of the suffixes (BinIdxSfx > numSuffixCtxBinsFine - 1) as the bypass.

[0127] Here, numPrefixCtxBinsFine and numSuffixCtxBinsFine are the number of bins to be coded using context in the prefix and suffix (the maximum number of bins to be coded using context), and are determined using the fixed values ​​nbPfxCtxFine and nbSfxCtxFine already explained. numPrefixCtxBinsFine = nbPfxCtxFine numSuffixCtxBinsFine = nbSfxCtxFine Alternatively, they may be values ​​greater than nbPfxCtxFine and nbSfxCtxFine, respectively. for example, numPrefixCtxBinsFine = nbPfxCtxFine * 2 numSuffixCtxBinsFine = nbSfxCtxFine * 2 or numPrefixCtxBinsFine = nbPfxCtxFine + NN numSuffixCtxBinsFine = nbSfxCtxFine + NN (NN=1, 2, 3, etc.) If BinIdxPfx <= numPrefixCtxBinsFine - 1, the arithmetic decoding unit 3051 decodes the bin of the index BinIdxPfx in the binarization of the prefix of mesh_position_fine_residual[i][j] using the value of ctxCoeffRemPrefixPosFine[min(nbPfxCtxFine - 1, BinIdxPfx)].If BinIdxPfx > numPrefixCtxBinsFine - 1, the arithmetic decoding unit 3051 decodes the bin of the index BinIdxPfx in the binarization of the prefix of mesh_position_fine_residual[i][j] using a bypass.

[0128] If BinIdxSfx <= numSuffixCtxBinsFine - 1, the arithmetic decoding unit 3051 decodes the bin of the index BinIdxSfx in the binarization of the suffix of mesh_position_fine_residual[i][j] using the value of ctxCoeffRemSuffixPosFine[min(nbSfxCtxFine - 1, BinIdxSfx)].If BinIdxSfx > numSuffixCtxBinsFine - 1, the arithmetic decoding unit 3051 decodes the bin of the index BinIdxSfx in the binarization of the suffix of mesh_position_fine_residual[i][j] using a bypass.

[0129] The mesh decoding unit 3031 (context selection unit 3056) may decode a portion of the first bins (BinIdxPfx <= numPrefixCtxBinsCoarse - 1) of the prefix of mesh_position_coarse_residual[i][j] (e.g., M bins, M = numPrefixCtxBinsCoarse) as context, the other bins of the prefix (BinIdxPfx > numPrefixCtxBinsCoarse - 1) as bypass, the first portion of the suffix (e.g., N bins, N = numSuffixCtxBinsCoarse) (BinIdxSfx <= numSuffixCtxBinsCoarse - 1) as context, and the other bins of the suffix (BinIdxSfx > numSuffixCtxBinsCoarse - 1) as bypass.

[0130] Here, numPrefixCtxBinsCoarse and numSuffixCtxBinsCoarse are the number of bins for context coding in prefixes and suffixes, and are set using the fixed values ​​nbPfxCtxCoarse and nbSfxCtxCoarse already explained. numPrefixCtxBinsCoarse = nbPfxCtxCoarse numSuffixCtxBinsCoarse = nbSfxCtxCoarse Alternatively, they may be values ​​greater than nbPfxCtxCoarse and nbSfxCtxCoarse. For example, numPrefixCtxBinsCoarse = nbPfxCtxCoarse * 2 numSuffixCtxBinsCoarse = nbSfxCtxCoarse * 2 or numPrefixCtxBinsCoarse = nbPfxCtxCoarse + NN numSuffixCtxBinsCoarse = nbSfxCtxCoarse + NN (NN=1, 2, 3, etc.) may be.

[0131] If BinIdxPfx <= numPrefixCtxBinsCoarse - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxPfx in the binarization of the prefix of mesh_position_coarse_residual[i][j] using the value of ctxCoeffRemPrefixPosCoarse[min(nbPfxCtxCoarse - 1, BinIdxPfx)].If BinIdxPfx > numPrefixCtxBinsCoarse - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxPfx in the binarization of the prefix of mesh_position_coarse_residual[i][j] using bypass.

[0132] If BinIdxSfx <= numSuffixCtxBinsCoarse - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxSfx in the binarization of the suffix of mesh_position_coarse_residual[i][j] using the value of ctxCoeffRemSuffixPosCoarse[min(nbSfxCtxCoarse - 1, BinIdxSfx)].If BinIdxSfx > numSuffixCtxBinsCoarse - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxSfx in the binarization of the suffix of mesh_position_coarse_residual[i][j] using bypass.

[0133] 20 shows the settings of the context used by the mesh decoding unit 3031. The mesh decoding unit 3031 (context selection unit 3056) may decode some bins at the beginning of the prefix of mesh_attribute_fine_residual[i][j][k] using the context, and the other bins in the prefix using bypass, and some bins at the beginning of the suffix using the context, and the other bins in the suffix using bypass.

[0134] If BinIdxPfx <= numPrefixCtxBinsFine - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxPfx in the binarization of the prefix of mesh_attribute_fine_residual[i][j][k] using the value of ctxCoeffRemPrefixAttrFine[i][min(nbPfxCtxFine - 1, BinIdxPfx)].If BinIdxPfx > numPrefixCtxBinsFine - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxPfx in the binarization of the prefix of mesh_attribute_fine_residual[i][j][k] using bypass.

[0135] If BinIdxSfx <= numSuffixCtxBinsFine - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxSfx in the binarization of the suffix of mesh_attribute_fine_residual[i][j][k] using the value of ctxCoeffRemSuffixAttrFine[i][min(nbSfxCtxFine - 1, BinIdxSfx)].If BinIdxSfx > numSuffixCtxBinsFine - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxSfx in the binarization of the suffix of mesh_attribute_fine_residual[i][j][k] using bypass.

[0136] The mesh decoding unit 3031 (context selection unit 3056) may decode some of the bins at the beginning of the prefix of mesh_attribute_coarse_residual[i][j][k] using the context, the other bins in the prefix using bypass, and some of the bins at the beginning of the suffix using the context, and the other bins in the suffix using bypass.

[0137] If BinIdxPfx <= numPrefixCtxBinsCoarse - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxPfx in the binarization of the prefix of mesh_attribute_coarse_residual[i][j][k] using the value of ctxCoeffRemPrefixAttrCoarse[i][min(nbPfxCtxCoarse - 1, BinIdxPfx)].If BinIdxPfx > numPrefixCtxBinsCoarse - 1, the arithmetic decoding unit 3051 decodes the bin of index BinIdxPfx in the binarization of the prefix of mesh_attribute_coarse_residual[i][j][k] using bypass.

[0138] If BinIdxSfx <= numSuffixCtxBinsCoarse - 1, the arithmetic decoding unit 3051 performs the mesh_attribute The arithmetic decoding unit 3051 decodes the bin of the index BinIdxSfx in the binarization of the suffix of mesh_attribute_coarse_residual[i][j][k]. If BinIdxSfx > numSuffixCtxBinsCoarse - 1, the arithmetic decoding unit 3051 decodes the bin of the index BinIdxSfx in the binarization of the suffix of mesh_attribute_coarse_residual[i][j][k] using a bypass.

[0139] According to the above, even when the value to be ExpGolomb-encoded becomes large, only a maximum of numPrefixCtxBinsFine / numSuffixCtxBinsFine / numPrefixCtxBinsCoarse / numPrefixCtxBinsCoarse contexts are used, which reduces complexity compared to using contexts for all bins of the residual. Also, coding can be performed more efficiently than when bypassing is used for all bins.

[0140] (Configuration to limit the number of context coding bins to be decoded) To reduce the complexity of context encoding, the number of context-encoded bins may be limited. Specifically, the mesh decoding unit 3031 (context selection unit 3056) counts the number of bins decoded by context encoding in the prefix / suffix of each syntax element of mesh_position_fine_residual / mesh_position_coarse_residual / mesh_attribute_fine_residual / mesh_attribute_coarse_residual for each predetermined unit (e.g., for each predetermined number of vertices). Then, if the counted number is equal to or greater than a predetermined value, each bin of the prefix / suffix may be switched from decoding using a context to decoding without using a context (using a bypass or static context).

[0141] According to the above, the maximum value (worst case) of the number of bins to be context-coded can be reduced.

[0142] (Context sharing) In the above configuration, independent contexts are used for vertex position / attribute, fine / coarse, and prefix / suffix, respectively, but some contexts may be shared. Here, sharing the context of A and B means that the mesh decoding unit 3031 encodes and decodes syntax elements corresponding to A and B, such as vertices and attributes, using the same context (one context array). Instead of ctxCoeffRemA[] and ctxCoeffRemB[], ctxCoeffRemA[] or ctxCoeffRemB[], or ctxCoeffRem[] of the same size may be used.

[0143] For example, for vertex positions / attributes, some or all of the following may apply: 1) Share ctxCoeffRemPrefixPosFine[nbPfxCtxFine] and ctxCoeffRemPrefixAttrFine[mesh_attribute_count][nbPfxCtxFine] (for example, use ctxCoeffRemPrefixFine[nbPfxCtxFine]). 2) Share ctxCoeffRemSuffixPosFine[nbSfxCtxFine] and ctxCoeffRemSuffixAttrFine[mesh_attribute_count][nbSfxCtxFine]. 3) Share ctxCoeffRemPrefixPosCoarse[nbPfxCtxCoarse] and ctxCoeffRemPrefixAttrCoarse[mesh_attribute_count][nbPfxCtxCoarse]. 4) Share ctxCoeffRemSuffixPosCoarse[nbSfxCtxCoarse] and ctxCoeffRemSuffixAttrCoarse[mesh_attribute_count][nbSfxCtxCoarse].

[0144] Alternatively, for fine / course, some or all of the following may apply: 5) ctxCoeffRemPrefixPosFine[nbPfxCtxFine] and ctxCoeffRemPrefixPosCoarse[nbPfxCtxCoarse] are shared. 6) ctxCoeffRemSuffixPosFine[nbSfxCtxFine] and ctxCoeffRemSuffixPosCoarse[nbSfxCtxCoarse] are shared. 7) ctxCoeffRemPrefixAttrFine[mesh_attribute_count][nbPfxCtxFine] and ctxCoeffRemPrefixAttrCoarse[mesh_attribute_count][nbPfxCtxCoarse] are shared. 8) ctxCoeffRemSuffixAttrFine[mesh_attribute_count][nbSfxCtxFine] and ctxCoeffRemSuffixAttrCoarse[mesh_attribute_count][nbSfxCtxCoarse] are shared. Here, sharing the context of A and B means that in the mesh decoding unit 3031, for example, the same context (one context array) is used for encoding and decoding in fine and coarse.

[0145] Alternatively, some or all of the following may apply for prefixes / suffixes: 9) ctxCoeffRemPrefixPosFine[nbPfxCtxFine] and ctxCoeffRemSuffixPosFine[nbSfxCtxFine] are shared. 10) ctxCoeffRemPrefixPosCoarse[nbPfxCtxCoarse] and ctxCoeffRemSuffixPosCoarse[nbSfxCtxCoarse] are shared. 11) Share ctxCoeffRemPrefixAttrFine[mesh_attribute_count][nbPfxCtxFine] and ctxCoeffRemSuffixAttrFine[mesh_attribute_count][nbSfxCtxFine]. 12) Share ctxCoeffRemPrefixAttrCoarse[mesh_attribute_count][nbPfxCtxCoarse] and ctxCoeffRemSuffixAttrCoarse[mesh_attribute_count][nbSfxCtxCoarse]. Here, sharing the context of A and B means that in the mesh decoding unit 3031, for example, the prefix and suffix are encoded and decoded using the same context (one context array).

[0146] Alternatively, for suffixes, the following may apply: 13) ctxCoeffRemSuffixPosFine[nbSfxCtxFine], ctxCoeffRemSuffixPosCoarse[nbSfxCtxCoarse], ctxCoeffRemSuffixAttrFin Share e[mesh_attribute_count][nbSfxCtxFine], ctxCoeffRemSuffixAttrCoarse[mesh_attribute_count][nbSfxCtxCoarse].

[0147] According to the above, it is possible to significantly reduce the memory size of the context without reducing the coding efficiency.

[0148] (Mesh derivation process) The mesh decoding unit 3031 decodes the syntax element mesh_position_fine_residual from the coded data and derives the vertex position vector prediction residual BmVertexPosFinePredResidual of the base mesh through the following processing. if (mesh_position_fine_residuals_count > 0) { for (j = 0; j < 3; j++) { / / dimension (component) loop for (i = 0; i < mesh_position_fine_residuals_count; i++) { / / decode mesh_position_fine_residual[i][j] value = decodeTUExpGolombS(ctxCoeffRemPrefixPosFine, ctxCoeffRemSuffixPosFine) BmVertexPosFinePredResidual[i][j] = value } } } The mesh decoding unit 3031 decodes the syntax element mesh_position_coarse_residual from the coded data and derives the vertex position vector prediction residual BmVertexPosCoarsePredResidual of the base mesh through the following processing. if (mesh_position_coarse_residuals_count > 0) { for (j = 0; j < 3; j++) { / / dimension (component) loop for (i = 0; i < mesh_position_coarse_residuals_count; i++) { / / decode mesh_position_coarse_residual[i][j] value = decodeTUExpGolombS(ctxCoeffRemPrefixPosCoarse, ctxCoeffRemSuffixPosCoarse) BmVertexPosCoarsePredResidual[i][j] = value }} } The mesh decoding unit 3031 decodes the syntax element mesh_attribute_fine_residual from the encoded data and derives the attribute prediction residual (texture coordinate prediction residual, normal vector prediction residual) BmVertexAttrFinePredResidual of the base mesh by the following process: for (i = 0; i < mesh_attribute_count; i++) { if (mesh_attribute_fine_residuals_count[i] > 0) { for (j = 0; j < mesh_attribute_fine_residuals_count[i]; j++) { for (k = 0; k < NumComponents[i]; k++) { / / dimension (component) loop / / decode mesh_attribute_fine_residual[i][j][k] value = decodeTUExpGolombS(ctxCoeffRemPrefixAttrFine[i], ctxCoeffRemSuffixAttrFine[i]) BmVertexAttrFinePredResidual[i][j][k] = value } } } } The mesh decoding unit 3031 decodes the syntax element mesh_attribute_coarse_residual from the coded data and derives the attribute prediction residual (texture coordinate prediction residual, normal vector prediction residual) BmVertexAttrCoarsePredResidual of the base mesh through the following processing. for (i = 0; i < mesh_attribute_count; i++) { if (mesh_attribute_coarse_residuals_count[i] > 0) { for (j = 0; j < mesh_attribute_coarse_residuals_count[i]; j++) { for (k = 0; k < NumComponents[i]; k++) { / / dimension (component) loop / / decode mesh_attribute_coarse_residual[i][j][k] Value = decodeTUExpGolombS(ctxCoeffRemPrefixAttrCoarse[i], ctxCoeffRemSuffixAttrCoarse[i]) BmVertexAttrCoarsePredResidual[i][j][k] = value } } } } Here, decodeTUExpGolombS() is a process that performs arithmetic decoding using a context given the offset, prefix, suffix, and sign of the prediction residual value. It is also possible to decode a binarization that concatenates a truncated unary code (TU) using the maximum value maxOffset, a kth-order exponential Golomb code (ExpGolumb), and a code (S). First, parse (decode) the truncated unary encoded offset. offset = 0 for(BinIdxTu = 0; offset < maxOffset && dec_aebin() == 1; BinIdxTu++) offset++ Next, if offset==maxOffset, parse (decode) the unary encoded prefix. prefix=0 if(offset == maxOffset) { for (BinIdxPfx = ​​0; dec_aebin() != 0; BinIdxPfx++) prefix++ } Next, if offset==maxOffset, parse (decode) the suffix. suffix = 0 if(offset == maxOffset) { for(BinIdxSfx = 0; BinIdxSfx < k + prefix; BinIdxSfx++) suffix = (suffix << 1) + dec_aebin() } The output is a signed integer val, which is parsed by if(offset > 0) { sign = dec_aebin() absVal = offset + (1 << (prefix + k)) + suffix - (1 << k) val = sign ? - absVal : absVal } else { val = 0 } Here, dec_aebin() indicates decoding one bin.

[0149] The mesh decoding unit 3031 derives the base mesh vertex position vectors BmVertexPosFine and BmVertexPosCoarse by the following process. BmVertexPosFine = BmVertexPosFinePred + BmVertexPosFinePredResidual BmVertexPosCoarse = BmVertexPosCoarsePred + BmVertexPosCoarsePredResidual Alternatively, the following formula may be used: BmVertexPosFine[i][j] = BmVertexPosFinePred[i][j] + BmVertexPosFinePredResidual[i][j] i=0..mesh_position_fine_residuals_count-1, j=0..2 BmVertexPosCoarse[i][j] = BmVertexPosCoarsePred[i][j] + BmVertexPosCoarsePredResidual[i][j] i=0..mesh_position_coarse_residuals_count-1, j=0..2 The mesh decoding unit 3031 derives the base mesh attributes BmVertexAttrFine and BmVertexAttrCoarse through the following process. BmVertexAttrFine = BmVertexAttrFinePred + BmVertexAttrFinePredResidual BmVertexAttrCoarse = BmVertexAttrCoarsePred + BmVertexAttrCoarsePredResidual Alternatively, the following formula may be used: BmVertexAttrFine[i][j][k] = BmVertexAttrFinePred[i][j][k] + BmVertexAttrFinePredResidual[i][j][k] i=0.. mesh_attribute_count-1, j=0.. mesh_attribute_fine_residuals_count-1, k=0.. NumComponents[i]-1 BmVertexAttrCoarse[i][j][k] = BmVertexAttrCoarsePred[i][j][k] + BmVertexAttrCoarsePredResidual[i][j][k] i=0.. mesh_attribute_count-1, j=0.. mesh_attribute_coarse_residuals_count-1, k=0.. NumComponents[i]-1 (Mesh reconstruction) 6 is a functional block diagram showing the configuration of the mesh reconstruction unit 307. The mesh reconstruction unit 307 is made up of a mesh division unit 3071 and a mesh deformation unit 3072.

[0150] The mesh dividing unit 3071 divides the base mesh output from the base mesh decoding unit 303 to generate divided meshes.

[0151] Figure 9(a) shows a part (triangle) of the base mesh, and the triangle is composed of vertices v1, v2, and v3. v1, v2, and v3 are three-dimensional vectors. The mesh division unit 3071 generates and outputs divided meshes by adding new vertices v12, v13, and v23 to the middle of each side of the triangle (Figure 9(b)). v12 = (v1 + v2) / 2 v13 = (v1 + v3) / 2 v23 = (v2 + v3) / 2 The following is also possible: v12 = (v1 + v2 + 1) >> 1 v13 = (v1 + v3 + 1) >> 1 v23 = (v2 + v3 + 1) >> 1 The mesh deformation unit 3072 receives the division mesh and the mesh displacement, and outputs the mesh displacement d12, A deformed mesh is generated and output by adding d13 and d23 (FIG. 9(c)). The mesh displacement is the output of the mesh displacement decoding unit 305 (coordinate system conversion unit 3055). d12, d13, and d23 are mesh displacements corresponding to the vertices v12, v13, and v23 added by the mesh division unit 3071. v12' = v12 + d12 v13' = v13 + d13 v23' = v23 + d23 It should be noted that d12 = disp[0][], d23 = disp[1][], and d23 = disp[3][] may also be used.

[0152] (Configuration of 3D data encoding device according to the first embodiment) 10 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11 according to the first embodiment. The 3D data encoding device 11 includes an atlas information encoding unit 101, a base mesh encoding unit 103, a base mesh decoding unit 104, a mesh displacement updating unit 106, a mesh displacement encoding unit 107, a mesh displacement decoding unit 108, a mesh reconstruction unit 109, an attribute updating unit 110, a padding unit 111, a color space conversion unit 112, an attribute encoding unit 113, a multiplexing unit 114, and a mesh separation unit 115. The 3D data encoding device 11 receives as input atlas information, a base mesh, a mesh displacement, a mesh, and an attribute image as 3D data, and outputs encoded data.

[0153] The atlas information encoding unit 101 encodes the atlas information and outputs an atlas information encoded stream.

[0154] The base mesh encoding unit 103 encodes the base mesh and outputs a base mesh encoded stream using a coding method such as Draco.

[0155] The base mesh decoding unit 104 is similar to the base mesh decoding unit 303, and therefore a description thereof will be omitted.

[0156] The mesh displacement update unit 106 adjusts the mesh displacement based on the (original) base mesh and the decoded base mesh, and outputs the updated mesh displacement.

[0157] The mesh displacement encoding unit 107 encodes the updated mesh displacement and outputs a mesh displacement encoded stream.

[0158] The mesh displacement decoding unit 108 is similar to the mesh displacement decoding unit 305, and therefore a description thereof will be omitted.

[0159] The mesh reconstruction unit 109 is similar to the mesh reconstruction unit 307, and therefore a description thereof will be omitted.

[0160] The attribute update unit 110 inputs the (original) mesh, the reconstructed mesh output from the mesh reconstruction unit 109 (mesh deformation unit 3072), and the attribute image, updates the attribute image to match the position (coordinates) of the reconstructed mesh, and outputs the updated attribute image.

[0161] The padding unit 111 receives the attribute image and performs padding on areas where pixel values ​​are empty.

[0162] The color space conversion unit 112 performs color space conversion from the RGB format to the YCbCr format.

[0163] The attribute encoding unit 113 encodes the attribute image in YCbCr format output from the color space conversion unit 112, and outputs an attribute video stream. As the encoding method, VVC, HEVC, or the like is used.

[0164] The multiplexing unit 114 multiplexes the atlas information coded stream, base mesh coded stream, mesh displacement coded stream, and attribute video stream and outputs the multiplexed data as coded data. As a multiplexing method, a byte stream format, ISOBMFF, etc. is used.

[0165] (Mesh separation unit operation) The mesh separation unit 115 generates a base mesh and a mesh displacement from the mesh.

[0166] 13 is a functional block diagram showing the configuration of the mesh separation unit 115. The mesh separation unit 115 is made up of a mesh thinning unit 1151, a mesh division unit 1152, and a mesh displacement derivation unit 1153.

[0167] The mesh thinning unit 1151 generates a base mesh by thinning out some of the vertices from the mesh.

[0168] Figure 14(a) shows a part of a mesh, which has vertices v1, v2, v3, v4, v5, 14(b)).

[0169] The mesh dividing unit 1152 divides the base mesh to generate divided meshes, similar to the mesh dividing unit 3071 (FIG. 14(c)). v4' = (v1 + v2) / 2 v5' = (v1 + v3) / 2 v6' = (v2 + v3) / 2 The mesh displacement derivation unit derives and outputs the displacements d4, d5, d6 of vertices v4, v5, v6 relative to vertices v4', v5', v6' as mesh displacements based on the mesh and the divided meshes (FIG. 14(d)). d4 = v4 - v4' d5 = v5 - v5' d6 = v6 - v6' (Base mesh encoding) 11 is a functional block diagram showing the configuration of the base mesh encoding unit 103. The base mesh encoding unit 103 is composed of a mesh encoding unit 1031, a mesh decoding unit 1032, a motion information encoding unit 1033, a motion information decoding unit 1034, a mesh motion compensation unit 1035, a reference mesh memory 1036, a switch 1037, and a switch 1038. The base mesh encoding unit 103 may also include a base mesh quantization unit (not shown) after inputting the base mesh. When encoding a base mesh without referring to other base meshes (e.g., an already encoded base mesh) (intra-coding), the switches 1037 and 1038 are connected to the side that does not perform motion compensation. When encoding a base mesh with reference to other base meshes (inter-coding), the switches 1037 and 1038 are connected to the side that performs motion compensation.

[0170] The mesh encoding unit 1031 has an intra-encoding function, performs intra-encoding on the base mesh, and outputs a base mesh encoded stream. There are.

[0171] The mesh decoding unit 1032 is similar to the mesh decoding unit 3031, and therefore a description thereof will be omitted.

[0172] The motion information encoding unit 1033 has an inter-encoding function, performs inter-encoding on the base mesh, and outputs a base mesh encoded stream. The encoding method used is entropy encoding such as arithmetic encoding.

[0173] The motion information decoding unit 1034 is similar to the motion information decoding unit 3032, and therefore a description thereof will be omitted.

[0174] The mesh motion compensation unit 1035 is similar to the mesh motion compensation unit 3033, and therefore a description thereof will be omitted.

[0175] The reference mesh memory 1036 is similar to the reference mesh memory 3034, and therefore a description thereof will be omitted.

[0176] (Mesh displacement encoding) 12 is a functional block diagram showing the configuration of the mesh displacement encoding unit 107. The mesh displacement encoding unit 107 is made up of a coordinate system conversion unit 1071, a conversion unit 1072, a quantization unit 1073, a binarization unit 1074, an arithmetic encoding unit 1075, a context selection unit 1076, and a context initialization unit 1077.

[0177] The coordinate system conversion unit 1071 converts the coordinate system of the mesh displacement from a Cartesian coordinate system to a coordinate system that encodes the displacement (for example, a local coordinate system) based on the value of the coordinate system conversion information displacementCoordinateSystem. Here, disp is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion, d is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) indicating each axis of the local coordinate system. if (displacementCoordinateSystem == 0) { d = disp } else if (displacementCoordinateSystem == 1){ d = (disp * n_vec, disp * t_vec, disp * b_vec) } The mesh displacement coding unit 107 may update the value of displacementCoordinateSystem at the sequence level, or at the picture / frame level. The initial value is 0, which indicates a Cartesian coordinate system.

[0178] When updating the displacementCoordinateSystem at the sequence level, use the configuration syntax in Figure 7. Set asps_vdmc_ext_displacement_coordinate_system to 0 for a Cartesian coordinate system and 1 for a local coordinate system.

[0179] When changing the displacementCoordinateSystem at the picture / frame level, use the syntax in the configuration in Figure 8. For afps_vdmc_ext_displacement_coordinate_system_enable_flag, set 1 if you want to update the coordinate system, or 0 if you do not want to update the coordinate system. For afps_vdmc_ext_displacement_coordinate_system, set 0 if you want to use a Cartesian coordinate system, or 1 if you want to use a local coordinate system.

[0180] The transform unit 1072 performs a transform f (for example, a wavelet transform) and derives a mesh displacement Tdisp after the transform. Tdisp[0][] = f(d[0][]) Tdisp[1][] = f(d[1][]) Tdisp[2][] = f(d[2][]) The quantization unit 1073 performs quantization based on the quantization scale value "scale" derived from the quantization parameter of each component of the mesh displacement, and derives the mesh displacement Qdisp after quantization. Qdisp[0][] = Tdisp[0][] / scale[0] Qdisp[1][] = Tdisp[1][] / scale[1] Qdisp[2][] = Tdisp[2][] / scale[2] Alternatively, the scale value may be approximated by a power of 2 and Qdisp may be derived using the following formula: scale[i] = 1 << scale2[i] Qdisp[0][] = Tdisp[0][] >> scale2[0] Qdisp[1][] = Tdisp[1][] >> scale2[1] Qdisp[2][] = Tdisp[2][] >> scale2[2] The binarization unit 1074 encodes the quantized mesh displacement Qdisp, which is a multi-level signal, into a binary signal, which may be a k-th order exponential-Golomb code.

[0181] The arithmetic coding unit 1075 arithmetically codes the binary signal and outputs a mesh displacement coded stream.

[0182] The context selection unit 1076 is similar to the context selection unit 3056, and therefore a description thereof will be omitted.

[0183] Note that a static context with a fixed probability that does not update the context is called ctxStatic. The syntax element indicated by ctxStatic may be encoded without using a context. encode(ctxStatic) may be encoded_bypass(), using a process dedicated to bypassing.

[0184] The context initialization unit 1077 is similar to the context initialization unit 3057, and therefore a description thereof will be omitted. Although an example using a context will be described here, some syntax elements may be bypass coded without using a context. A bypass coding configuration has the effect of reducing the memory and processing load for the context. For example, the syntax elements diu_last_sig_coeff, diu_coded_block_flag, and diu_coeff_abs_level_rem may be bypass coded without using a context. By bypass coding these syntax elements, it is possible to reduce the memory and processing amount for the context while maintaining coding efficiency.

[0185] The mesh displacement encoding unit 107 encodes the mesh displacement Qdisp by the following process. for (k = 0; k < numDim; k++) { / / dimension (component) loop / / encode diu_last_sig_coeff encodeExpGolomb(diu_last_sig_coeff[k], ctxStatic) if (!lastSig) continue dispOffset = 0 for (b = 0; b <numLOD; b++) { / / Level of Detail loop, block loop / / encode diu_coded_block_flag encode(diu_coded_block_flag[k][b], ctxStatic) numBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numBlocks; s++) { / / subblock loop / / encode diu_coded_subblock_flag encode(diu_coded_subblock_flag[k][b][s], ctxCodedSubBlock[ft][b][k]) for (v = 0; v < subBlockSize; v++) { / / coefficient loop within subblock / / encode diu_coeff_abs_level_gt0 d = Qdisp[dispOffset + s * subBlockSize + v][k] encode(d != 0, ctxCoeffGtN[ft][b][0][k]) if (!d) continue / / encode diu_coeff_sign encode(d < 0, ctxStatic) d = abs(d) - 1 / / encode diu_coeff_abs_level_gt1 encode(d != 0, ctxCoeffGtN[ft][b][1][k]) if (!d) continue d = abs(d) - 1 / / encode diu_coeff_abs_level_gt2 encode(d != 0, ctxCoeffGtN[ft][b][2][k]) if (!d) continue d = abs(d) - 1 / / encode diu_coeff_abs_level_gt3 encode(d != 0, ctxCoeffGtN[ft][b][3][k]) if (!d) continue / / encode diu_coeff_abs_level_rem encodeExpGolomb(--d, ctxCoeffRemPrefix[ft][b][k]) } } dispOffset += dispCount[b] } } In pseudocode, continue means to skip the subsequent actions and jump to the beginning of the loop (next iteration). Here, encode() and encodeExpGolomb() are functions that take a value and corresponding context as arguments and arithmetically encode a 1-bit value and a k-th order Exponential Golomb code binary string, respectively. dispCount[b] is the number of mesh displacements at detail level b. lastSig is a flag indicating whether the current coefficient is the last non-zero coefficient in the sub-block in scan order. lastSig=0 indicates that the current coefficient is not the last non-zero coefficient in the sub-block in scan order. lastSig=1 indicates that the current coefficient is the last non-zero coefficient in the sub-block in scan order.

[0186] (Base mesh encoding) 21 is a functional block diagram showing the configuration of the mesh encoding unit 1031. The mesh encoding unit 1031 includes a mesh prediction unit 10311, a binarization unit 1074, an arithmetic encoding unit 1075, a context selection unit 1076, and a context initialization unit 1077.

[0187] The mesh prediction unit 10311 is similar to the mesh prediction unit 30311, and therefore a description thereof will be omitted.

[0188] The mesh encoding unit 1031 encodes the vertex position vector prediction residual BmVertexPosFinePredResidual (= BmVertexPosFine - BmVertexPosFinePred) of the base mesh through the following process. for (j = 0; j < 3; j++) { for (i = 0; i < mesh_position_fine_residuals_count; i++) { value = BmVertexPosFinePredResidual[i][j] encodeTUExpGolombS(value, ctxCoeffRemPrefixPosFine, ctxCoeffRemSuffixPosFine) } } The mesh encoding unit 1031 encodes the vertex position vector prediction residual BmVertexPosCoarsePredResidual (= BmVertexPosCoarse - BmVertexPosCoarsePred) of the base mesh through the following process. for (j = 0; j < 3; j++) { for (i = 0; i < mesh_position_coarse_residuals_count; i++) { value = BmVertexPosCoarsePredResidual[i][j] encodeTUExpGolombS(value, ctxCoeffRemPrefixPosCoarse, ctxCoeffRemSuffixPosCoarse) } } The mesh encoding unit 1031 encodes the attribute prediction residual BmVertexAttrFinePredResidual (= BmVertexAttrFine - BmVertexAttrFinePred) of the base mesh by the following process. for (i = 0; i < mesh_attribute_count; i++) { for (k = 0; k < NumComponents[i]; k++) { for (j = 0; j < mesh_attribute_fine_residuals_count; j++) { value = BmVertexAttrFinePredResidual[i][j][k] encodeTUExpGolombS(value, ctxCoeffRemPrefixAttrFine[i], ctxCoeffRemSuffixAttrFine[i]) } } } The mesh encoding unit 1031 encodes the attribute prediction residual BmVertexAttrCoarsePredResidual (= BmVertexAttrCoarse - BmVertexAttrCoarsePred) of the base mesh by the following process. for (i = 0; i < mesh_attribute_count; i++) { for (k = 0; k < NumComponents[i]; k++) { for (j = 0; j < mesh_attribute_coarse_residuals_count; j++) { value = BmVertexAttrCoarsePredResidual[i][j][k] encodeTUExpGolombS(value, ctxCoeffRemPrefixAttrCoarse[i], ctxCoeffRemSuffixAttrCoarse[i]) } } } Here, encodeTUExpGolombS() is a process of arithmetically encoding the offset, prefix, suffix, and sign of the prediction residual value using a given context.

[0189] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention.

[0190] [Application example] The above-described 3D data encoding device 11 and 3D data decoding device 31 can be mounted on various devices that transmit, receive, record, and play back 3D data. The 3D data may be natural 3D data captured by a camera or the like, or artificial 3D data (including CG and GUI) generated by a computer or the like.

[0191] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. In other words, embodiments obtained by combining technical means modified appropriately within the scope of the claims are also included in the technical scope of the present invention. [Industrial Applicability]

[0192] The embodiments of the present invention can be suitably applied to a 3D data decoding device that decodes coded data in which 3D data has been coded, and a 3D data coding device that generates coded data in which 3D data has been coded. The present invention can be suitably applied to the data structure of coded data referenced by a location. [Explanation of symbols]

[0193] 11 3D data encoding device 101 Atlas Information Encoding Unit 103 Base mesh coding unit 1031 Mesh coding unit 10311 Mesh prediction unit 1032 Mesh Decoding Unit 1033 Motion information encoding unit 1034 Motion information decoding unit 1035 Mesh motion compensation unit 1036 reference mesh memory 1037 Switch 1038 Switch 104 Base mesh decoding unit 106 Mesh displacement update section 107 Mesh displacement coding unit 1071 Coordinate system conversion unit 1072 Conversion Unit 1073 Quantization section 1074 Binarization section 1075 Arithmetic encoder 1076 Context Selection 1077 Context Initialization Unit 108 Mesh displacement decoding unit 109 Mesh reconstruction unit 110 Attribute Update Section 111 Padding section 112 Color space conversion unit 113 Attribute Encoding Unit 114 Multiplexer 115 mesh separation section 1151 Mesh thinning section 1152 Mesh division section 1153 Mesh displacement derivation part 21 Network 31 3D data decoding device 301 Demultiplexer 302 Atlas Information Decoding Unit 303 Base mesh decoding unit 3031 Mesh Decoding Unit 30311 Mesh prediction unit 3032 Motion information decoding unit 3033 Mesh Motion Compensation Unit 3034 Reference Mesh Memory 3035 Switch 3036 Switch 305 Mesh displacement decoding unit 3051 Arithmetic Decoding Unit 3052 Multilevel conversion section 3053 Inverse quantization section 3054 Inverse Conversion Unit 3055 Coordinate system conversion unit 3056 Context Selection Section 3057 Context Initialization Unit 307 Mesh Reconstruction Unit 306 Attribute Decoding Unit 3071 Mesh division section 3072 Mesh deformation part 308 Color Space Conversion Unit 41 3D data display device

Claims

1. A 3D data decoding device for decoding coded data, comprising: a mesh prediction unit for deriving predicted values ​​of base mesh vertex positions and / or base mesh attributes from the coded data; and an arithmetic decoding unit for arithmetically decoding prediction residuals; The arithmetic decoding unit decodes the first M bins of the prefix of the coefficient of the prediction residual using a context, decodes the first N bins of the suffix of the coefficient of the prediction residual using a context, and adds the prediction value and the prediction residual to derive the base mesh vertex position and / or the base mesh attribute.

2. The 3D data decoding device of claim 1, characterized in that the arithmetic decoding unit decodes more than M bins at the beginning of a prefix of the coefficient of the prediction residual using a bypass, decodes more than N bins at the beginning of a suffix of the coefficient of the prediction residual using a bypass, and adds the prediction value and the prediction residual to derive the base mesh vertex positions and / or the base mesh attributes.

3. 3. The 3D data decoding device according to claim 1, wherein the arithmetic decoding unit varies the values ​​of M and / or N between fine and coarse prediction residuals.

4. The 3D data decoding device according to any one of claims 1 to 3, wherein the arithmetic decoding unit shares a context of the prediction residuals of the base mesh vertex positions and a context of the prediction residuals of the base mesh attributes.

5. A 3D data encoding device for encoding 3D data, comprising: a mesh prediction unit for deriving predicted values ​​of base mesh vertex positions and / or base mesh attributes; and an arithmetic coding unit for arithmetically coding a prediction residual; The arithmetic coding unit encodes the first M bins of a prefix of the coefficients of the prediction residual using a context, and encodes the first N bins of a suffix of the coefficients of the prediction residual using a context. A 3D data coding device.

6. The 3D data encoding device according to claim 5, characterized in that the arithmetic coding unit encodes more than M first bins of a prefix of the coefficients of the prediction residual using a bypass, and encodes more than N first bins of a suffix of the coefficients of the prediction residual using a bypass.

7. 7. The 3D data encoding device according to claim 5, wherein the arithmetic encoding unit varies the values ​​of M and / or N depending on whether the prediction residual is fine or coarse.

8. The 3D data encoding device according to any one of claims 5 to 7, wherein the arithmetic coding unit shares a context of the prediction residual of the base mesh vertex position and a context of the prediction residual of the base mesh attribute.

Images