Three-dimensional data decoding apparatus and three-dimensional data encoding apparatus

Abstract

To encode / decode 3D data with high quality by improving encoding efficiency of mesh displacement in encoding / decoding 3D data using a moving image encoding system.SOLUTION: A 3D data decoding apparatus for decoding encoded data comprises: an arithmetic decoding unit for arithmetic decoding of mesh displacement from the encoded data; a context selection unit for selecting a context in the arithmetic decoding; and a context initialization unit for setting an initialization value of the context. The context initialization unit decodes a context initialization parameter for initializing the context, from the encoded data.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] An embodiment of the present invention relates to a 3D data encoding device and a 3D data decoding device. [Background technology]

[0002] In order to efficiently transmit or record 3D data, there are 3D data encoding devices that convert the 3D data into a 2D image, encode it using a video encoding method, and generate encoded data, and 3D data decoding devices that decode the encoded data into a 2D image and reconstruct the 3D data.

[0003] Specific examples of 3D data encoding methods include MPEG-I V3C (Volumetric Video-based Coding) and V-PCC (Video-based Point Cloud Compression) (Non-Patent Document 1). V3C can encode and decode multi-viewpoint video as well as point clouds consisting of point positions and attribute information. Existing video encoding methods include H.266 / VVC (Versatile Video Coding) and H.265 / HEVC (High Efficiency Video Coding). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] ISO / IEC 23090-5 [Non-Patent Document 2] [V-CG] Apple's Dynamic Mesh Coding CfP Response, ISO / IEC JTC 1 / SC 29 / WG 7 m59281, April 2022 [Non-Patent Document 3] Arithmetic Coding of Displacements for Subdivision-based Mesh Compression, ISO / IEC JTC 1 / SC 29 / WG 7 m60300, July 2022 Summary of the Invention [Problem to be solved by the invention]

[0005] In the 3D data coding method in Non-Patent Document 1, the geometry (depth image) and attributes (color image) constituting the 3D data (point cloud) are coded and decoded using a video coding method such as HEVC or VVC. In the 3D data coding method in Non-Patent Document 2, the geometry (base mesh, mesh displacement (mesh displacement array, mesh displacement image)) and attributes (texture mapping image) constituting the 3D data (mesh) are coded and decoded using a vertex coding method such as Draco or a video coding method such as HEVC or VVC. In addition, when coding and decoding 3D data (mesh) using the video coding method disclosed in Non-Patent Document 2, there are experimental results that show that performance is improved when the coding of the mesh displacement is replaced from the video coding method to an arithmetic coding method (Non-Patent Document 3). When the mesh displacement is arithmetically coded, there is a problem that the performance depends on the initial value of the context in the arithmetic coding.

[0006] An object of the present invention is to improve the coding efficiency of mesh displacement in coding and decoding of 3D data using a video coding method, and to code and decode 3D data with high quality. [Means for solving the problem]

[0007] In order to solve the above problem, a 3D data decoding device according to one embodiment of the present invention is a 3D data decoding device that decodes encoded data, and is characterized in that it comprises an arithmetic decoding unit that arithmetically decodes mesh displacement from the encoded data, a context selection unit that selects a context in the arithmetic decoding, and a context initialization unit that sets an initial value of the context, and in which the context initialization unit decodes a context initialization parameter that initializes the context from the encoded data.

[0008] In order to solve the above problems, a 3D data encoding device according to one embodiment of the present invention is a 3D data encoding device that encodes 3D data, and includes an arithmetic encoding unit that arithmetically encodes mesh displacement, a context selection unit that selects a context in the arithmetic encoding, and a context initialization unit that sets an initial value of the context, and is characterized in that the context initialization unit encodes a context initialization parameter that initializes the context into encoded data. Effect of the Invention

[0009] According to one aspect of the present invention, the encoding efficiency of mesh displacement is improved, and 3D data can be encoded and decoded with high quality. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing the configuration of a 3D data transmission system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. [Diagram 3] FIG. 2 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31. [Figure 4] FIG. 2 is a functional block diagram showing a configuration of a base mesh decoding unit 303. [Diagram 5] FIG. 2 is a functional block diagram showing the configuration of a mesh displacement decoding unit 305. [Figure 6] FIG. 2 is a functional block diagram showing the configuration of a mesh reconstruction unit 307. [Figure 7] This is an example of a syntax for transmitting coordinate transformation parameters and context initialization parameters at the sequence level (ASPS). [Figure 8] 13 is an example of a syntax for transmitting coordinate transformation parameters and context initialization parameters at the picture / frame level (AFPS). [Figure 9] FIG. 13 is a diagram for explaining the operation of the mesh reconstructing unit 307. [Figure 10] 1 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11. FIG. [Figure 11] FIG. 2 is a functional block diagram showing a configuration of a base mesh encoding unit 103. [Figure 12] FIG. 2 is a functional block diagram showing a configuration of a mesh displacement encoding unit 107. [Figure 13] FIG. 2 is a functional block diagram showing the configuration of a mesh separation unit 115. [Figure 14] 11 is a diagram for explaining the operation of the mesh separating unit 115. FIG. [Figure 15] 1 is an example of a syntax structure for mesh displacement. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] FIG. 1 is a schematic diagram showing the configuration of a 3D data transmission system 1 according to this embodiment.

[0013] The 3D data transmission system 1 is a system that transmits an encoded stream obtained by encoding 3D data to be encoded, decodes the transmitted encoded stream, and displays the 3D data. The 3D data transmission system 1 includes a 3D data encoding device 11, a network 21, a 3D data decoding device 31, and a 3D data display device 41.

[0014] 3D data T is input to the 3D data encoding device 11.

[0015] The network 21 transmits the encoded stream Te generated by the 3D data encoding device 11 to the 3D data decoding device 31. The network 21 is the Internet, a wide area network (WAN), a local area network (LAN), or a combination of these. The network 21 is not necessarily limited to a bidirectional communication network, and may be a unidirectional communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Furthermore, the network 21 may be replaced by a storage medium on which the encoded stream Te is recorded, such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blu-ray Disc: registered trademark).

[0016] The 3D data decoding device 31 decodes each of the encoded streams Te transmitted by the network 21, and generates one or more decoded 3D data Td.

[0017] The 3D data display device 41 displays all or part of one or more pieces of decoded 3D data Td generated by the 3D data decoding device 31. The 3D data display device 41 includes a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Display forms include stationary, mobile, HMD, and the like. Furthermore, when the 3D data decoding device 31 has high processing power, it displays high quality images, and when it has only lower processing power, it displays images that do not require high processing power or display power.

[0018] <Structure of the coding stream Te> Before going into a detailed description of the 3D data encoding device 11 and the 3D data decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the 3D data encoding device 11 and decoded by the 3D data decoding device 31 will be described.

[0019] Fig. 2 is a diagram showing a hierarchical structure of data in an encoded stream Te. The encoded stream Te illustratively includes a sequence and a plurality of pictures constituting the sequence. Fig. 2 shows diagrams showing an encoded video sequence that defines a sequence SEQ, an encoded picture that defines a picture PICT, an encoded slice that defines a slice S, encoded slice data that defines slice data, an encoding tree unit included in the encoded slice data, and an encoding unit included in the encoding tree unit.

[0020] (Coded Video Sequence) In the coded video sequence, a set of data to be referred to by the 3D data decoding device 31 in order to decode the sequence SEQ to be processed is specified. As shown in the coded video sequence of Fig. 2, the sequence SEQ includes a video parameter set, a sequence parameter set SPS (Sequence Parameter Set), a picture parameter set PPS (Picture Parameter Set), a picture PICT, and supplemental enhancement information SEI (Supplemental Enhancement Information).

[0021] The video parameter set VPS specifies a set of coding parameters common to multiple videos composed of multiple layers, as well as a set of coding parameters related to multiple layers and each individual layer included in the video.

[0022] The sequence parameter set SPS specifies a set of coding parameters that the 3D data decoding device 31 refers to in order to decode the target sequence. For example, the width and height of a picture are specified. Note that there may be multiple SPSs. In that case, one of the multiple SPSs is selected from the PPS.

[0023] The picture parameter set PPS specifies a set of coding parameters that the video decoding device 31 refers to in order to decode each picture in the target sequence. For example, the picture parameter set PPS includes a reference value of the quantization width (pic_init_qp_minus26) used in decoding the picture and a flag (weighted_pred_flag) indicating the application of weighted prediction. Note that there may be multiple PPSs. In that case, one of the multiple PPSs is selected for each picture in the target sequence.

[0024] (Encoded Picture) The coded picture defines a set of data to be referenced by the 3D data decoding device 31 in order to decode the picture PICT to be processed. As shown in the coded picture of Fig. 2, the picture PICT includes slices 0 to NS-1 (NS is the total number of slices included in the picture PICT).

[0025] (Coded Slice) An encoded slice specifies a set of data to be referenced by the 3D data decoding device 31 in order to decode a slice S to be processed. As shown in the encoded slice of Fig. 2, a slice includes a slice header and slice data.

[0026] The slice header includes a group of coding parameters to be referred to by the 3D data decoding device 31 in order to determine a decoding method for the target slice. Slice type designation information (slice_type) that designates the slice type is an example of a coding parameter included in the slice header.

[0027] (Encoded slice data) The coded slice data defines a set of data to be referenced by the 3D data decoding device 31 in order to decode the slice data to be processed. As shown in the coded slice header in Fig. 2, the slice data includes a CTU.

[0028] (coding tree unit) 2 specifies a set of data that the 3D data decoding device 31 refers to in order to decode a CTU to be processed. The CTU is divided into coding units CU, which are basic units of encoding processing, by recursive quad tree (QT) division, binary tree (BT) division, or ternary tree (TT) division.

[0029] (Encoding Unit) As shown in the coding unit of Fig. 2, a set of data to be referenced by the 3D data decoding device 31 in order to decode the coding unit to be processed is defined. Specifically, the CU is composed of a CU header CUH, prediction parameters, transformation parameters, quantization transformation coefficients, etc. The CU header defines a prediction mode, etc.

[0030] There are two types of prediction (prediction modes): intra prediction and inter prediction. Intra prediction is a prediction within the same picture, while inter prediction refers to a prediction process performed between different pictures (for example, between display times or between layer images).

[0031] The transform and quantization processes are performed in units of CUs, but the quantized transform coefficients may be entropy coded in units of sub-blocks such as 4x4.

[0032] (Configuration of 3D data decoding device according to the first embodiment) 3 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31 according to the first embodiment. The 3D data decoding device 31 is composed of a demultiplexing unit 301, an atlas information decoding unit 302, a base mesh decoding unit 303, a mesh displacement decoding unit 305, a mesh reconstruction unit 307, an attribute decoding unit 306, and a color space conversion unit 308. The 3D data decoding device 31 inputs encoded data of 3D data, and outputs atlas information, meshes, and attribute images.

[0033] The demultiplexing unit 301 inputs encoded data multiplexed in a byte stream format, ISOBMFF (ISO Base Media File Format), etc., demultiplexes it, and outputs an atlas information encoded stream, a base mesh encoded stream, a mesh displacement encoded stream, and an attribute image encoded stream.

[0034] The atlas information decoding unit 302 receives the atlas information encoded stream output from the demultiplexing unit 301 and decodes the atlas information.

[0035] The base mesh decoder 303 decodes the base mesh coded stream that has been coded using vertex coding (a 3D data compression coding method, for example, Draco), and outputs a base mesh. The base mesh will be described later.

[0036] The mesh displacement decoding unit 305 decodes the mesh displacement coded stream and outputs the mesh displacement.

[0037] A mesh reconstructing unit 307 receives the base mesh and the mesh displacement, and reconstructs the mesh in the 3D space.

[0038] The attribute decoding unit 306 decodes an attribute image coded stream coded by VVC, HEVC, or the like, and outputs an attribute image in the YCbCr format. The attribute image may be a texture image expanded on the UV axis (a texture mapping image converted by the UV atlas method).

[0039] The color space conversion unit 308 converts the color space of the attribute image from the YCbCr format to the RGB format. Note that the attribute image encoded stream encoded in the RGB format may be decoded and the color space conversion may be omitted.

[0040] (Decoding the base mesh) FIG. 4 is a functional block diagram showing the configuration of the base mesh decoding unit 303. The base mesh decoding unit 303 is composed of a mesh decoding unit 3031, a motion information decoding unit 3032, a mesh motion compensation unit 3033, a reference mesh memory 3034, a switch 3035, and a switch 3036. The base mesh decoding unit 303 may be configured to include a base mesh inverse quantization unit (not shown) before the output of the base mesh. When the base mesh to be decoded is coded (intra-coded) without referring to other base meshes (for example, base meshes that have already been coded and decoded), the switches 3035 and 3036 are connected to the side that does not perform motion compensation. On the other hand, when the base mesh to be decoded is coded (inter-coded) with reference to other base meshes, the switches 3035 and 3036 are connected to the side that performs motion compensation. When motion compensation is performed, the target vertex coordinates are derived by referring to the already decoded vertex coordinates and motion information.

[0041] The mesh decoding unit 3031 decodes the intra-coded base mesh coded stream and outputs the base mesh. Draco or the like is used as the coding method.

[0042] The motion information decoding unit 3032 decodes the inter-coded base mesh coded stream and outputs motion information for each vertex of a reference mesh, which will be described later. Entropy coding, such as arithmetic coding, is used as the coding method.

[0043] The mesh motion compensation unit 3033 performs motion compensation on each vertex of the reference mesh input from the reference mesh memory 3034 based on the motion information, and outputs a motion-compensated mesh.

[0044] The reference mesh memory 3034 is a memory that holds decoded meshes for reference in subsequent decoding processes.

[0045] (Mesh displacement decoding) 5 is a functional block diagram showing the configuration of the mesh displacement decoding unit 305. The mesh displacement decoding unit 305 is made up of an arithmetic decoding unit 3051, a multi-value conversion unit 3052, a context selection unit 3056, a context initialization unit 3057, an inverse quantization unit 3053, an inverse transformation unit 3054, and a coordinate system transformation unit 3055.

[0046] The atlas information decoding unit 302 in Fig. 3 decodes coordinate system conversion information (asps_vdmc_ext_displacement_coordinate_system, afps_vdmc_ext_displacement_coordinate_system) indicating a coordinate system from the encoded data, and notifies the mesh displacement decoding unit 305. The atlas information decoding unit 302 also decodes context initialization parameters (asps_vdmc_ext_displacement_context_init_type, asps_vdmc_ext_displacement_context_init_index, afps_vdmc_ext_displacement_context_init_type, afps_vdmc_ext_displacement_context_init_index) in arithmetic decoding, and notifies the mesh displacement decoding unit 305. The context initialization parameters are information indicating the initialization timing of the context in arithmetic decoding (for example, initialization in units of GoF (Group of Frames), initialization in units of frames) and the timing of switching the initial value of the context. Note that a gating flag may be provided separately, and each piece of coordinate system transformation information may be decoded only when the gating flag is 1. The gating flag may be, for example, afps_vdmc_ext_displacement_coordinate_system_enable_flag. Also, a gating flag may be provided for the context initialization parameter, and the context initialization parameter may be decoded only when the gating flag is 1. The gating flag may be, for example, afps_vdmc_ext_displacement_context_init_enable_flag.

[0047] (Context-Adaptive Binary Arithmetic Coding) The arithmetic decoding unit 3051, the multi-value conversion unit 3052, the context selection unit 3056, and the context initialization unit 3057 use a decoding method called Context-adaptive binary arithmetic coding (CABAC). These may be collectively called a CABAC decoding unit. In CABAC, all CABAC states are initialized at the beginning of a segment. The CABAC decoding unit decodes each bit of a binary string (Bin String) corresponding to a syntax element. When a context is used, a context index ctxInc is derived for each bit of the syntax element, the bit is decoded using the context, and the CABAC state of the used context is updated. Bits that do not use a context are decoded with equal probability (EP, bypass), and the derivation of ctxInc and the CABAC state are omitted. A context is a variable area for holding the probability (state) of CABAC, and is identified by the value of ctxIdx (0, 1, 2, ...). Also, when 0 and 1 always have equal probability, that is, 0.5, 0.5, it is called EP (Equal Probability) or bypass. In this case, there is no need to maintain state for a specific syntax element, so no context is used. ctxIdx is derived by referencing ctxInc.

[0048] (Coordinate System) The following two types of coordinate systems are used for mesh displacement (3D vector). Cartesian coordinate system: A Cartesian coordinate system that is commonly defined throughout the entire 3D space. (X,Y,Z) coordinate system. A Cartesian coordinate system whose direction does not change at the same time (within the same frame, within the same tile). Local coordinate system: A Cartesian coordinate system defined for each region or vertex in 3D space. A Cartesian coordinate system whose direction can change at the same time (within the same frame, the same tile). Normal (D), tangent (U), bi-tangent (V) coordinate system. In other words, it is a Cartesian coordinate system consisting of the first axis (D) indicated by the normal vector n_vec at a certain vertex (a surface that contains the certain vertex), and the second axis (U) and third axis (V) indicated by two tangent vectors t_vec and b_vec that are perpendicular to the normal vector n_vec. n_vec, t_vec, and b_vec are three-dimensional vectors. The (D, U, V) coordinate system may also be called the (n, t, b) coordinate system.

[0049] (Decoding and deriving sequence-level control parameters) Here, the control parameters used in the mesh displacement decoding unit 305 will be described.

[0050] Figure 7 shows an example of syntax for transmitting coordinate system transformation parameters and context initialization parameters at the sequence level (ASPS: Atlas Sequence Parameter Set). In ASPS, coordinate system transformation parameters and context initialization parameters are transmitted using the asps_vdmc_extension() syntax. The semantics of each field are as follows: asps_vdmc_ext_displacement_coordinate_system: Coordinate system transformation information indicating the coordinate system of the mesh displacement. If the value is equal to a certain first value (e.g. 0), it indicates a Cartesian coordinate system. If the value is equal to another second value (e.g. 1), it indicates a local coordinate system. asps_vdmc_ext_displacement_context_init_type: context initialization timing information. Indicates the context initialization timing in arithmetic decoding of mesh displacement. If the value is equal to a first value (e.g., 0), the context is initialized for each GoF. If the value is equal to a second value (e.g., 1), the context is initialized for each segment (e.g., a segment is a frame or a slice that constitutes a frame). Specifically, the mesh displacement decoding unit 305 initializes the context using a variable frameIdxInGoF indicating the frame position within the GoF when asps_vdmc_ext_displacement_context_init_type==1 or (asps_vdmc_ext_displacement_context_init_type==0 and frameIdxInGoF==0).

[0051] In yet another configuration, when asps_vdmc_ext_displacement_context_init_type is a first value, the contest is initialized only when the segment head is a random access point, and when asps_vdmc_ext_displacement_context_init_type is a second value, the contest is always initialized at the segment head. Whether or not a random access point is determined may be determined based on whether nal_unit_type is a specific type. For example, the determination may be made based on whether or not nal_unit_type is equal to NAL_GIDR_W_RADL, NAL_GBLA_N_LP, NAL_GCRA, or the range from NAL_GBLA_W_LP to NAL_GBLA_N_LP. Also, whether or not nal_unit_type is NAL_GBLA_W_LP to NAL_GBLA_N_LP, or NAL_GIDR_W_RADL to NAL_GIDR_N_LP may be determined as a random access point.

[0052] The above-mentioned configuration of initializing in units of GoFs, which are the timings for initializing in units of random access, can minimize the degradation of probability prediction accuracy due to initialization, thereby improving coding efficiency. Furthermore, the configuration of selecting frame-by-frame initialization by a flag also has the effect of improving error resistance. asps_vdmc_ext_displacement_context_init_index: indicates the index of the context initial value table in arithmetic decoding of mesh displacement.

[0053] (Decoding and derivation of picture / frame level control parameters) Figure 8 shows an example of a syntax for transmitting coordinate system transformation parameters and context initialization parameters at the picture / frame level (AFPS: Atlas Frame Parameter Set). In AFPS, coordinate system transformation parameters and context initialization parameters are transmitted using the afps_vdmc_extension() syntax. The semantics of each field are as follows: afps_vdmc_ext_displacement_coordinate_system_enable_flag: A flag indicating whether to update the coordinate system of the mesh displacement. If this flag is equal to true, the coordinate system of the mesh displacement is updated based on the value of afps_vdmc_ext_displacement_coordinate_system described below. If this flag is equal to false, the coordinate system of the mesh displacement is not updated. afps_vdmc_ext_displacement_coordinate_system: Coordinate system transformation information indicating the coordinate system of the mesh displacement. If the value is equal to the first value (e.g. 0), it indicates a Cartesian coordinate system. If the value is equal to the second value (e.g. 1), it indicates a local coordinate system. If the syntax element is not present, the value is inferred as the value decoded by ASPS and the default coordinate system is the coordinate system indicated by ASPS. afps_vdmc_ext_displacement_context_init_enable_flag: A flag indicating whether to update the context initialization method for mesh displacement. If this flag is true, the context initialization method is updated based on the value of afps_vdmc_ext_displacement_context_init_type and the value of afps_vdmc_ext_displacement_context_init_index described later. If this flag is false, the context initialization method is not updated. afps_vdmc_ext_displacement_context_init_type: Context initialization timing information. Indicates the context initialization timing for arithmetic decoding of mesh displacement. If the value is equal to the first value (e.g., 0), the context is initialized for each GoF. If the value is equal to the second value (e.g., 1), the context is initialized for each segment (e.g., a segment is a frame or a slice that constitutes a frame). afps_vdmc_ext_displacement_context_init_index: Context initialization index. Context initial value selection information. Indicates the index of the context initial value table in arithmetic decoding of mesh displacement.

[0054] (Derivation of coordinate system transformation parameters) The mesh displacement decoding unit 305 derives the coordinate system transformation parameter displacementCoordinateSystem as follows: if ( afps_vdmc_ext_displacement_coordinate_system_enable_flag ) { displacementCoordinateSystem = afps_vdmc_ext_displacement_coordinate_system } else { displacementCoordinateSystem = asps_vdmc_ext_displacement_coordinate_system } Alternatively, if the syntax element appears at multiple levels, the coordinate system transformation parameter displacementCoordinateSystem may be derived by overwriting it with the value at the lower level. displacementCoordinateSystem = asps_vdmc_ext_displacement_coordinate_system if ( afps_vdmc_ext_displacement_coordinate_system_enable_flag ) { displacementCoordinateSystem = afps_vdmc_ext_displacement_coordinate_system

[0055] (Deriving Context Initialization Parameters) The mesh displacement decoding unit 305 derives the context initialization parameters displacementContextInitType and displacementContextInitIndex as follows: if ( afps_vdmc_ext_displacement_context_init_enable_flag ) { displacementContextInitType = afps_vdmc_ext_displacement_context_init_type displacementContextInitIndex = afps_vdmc_ext_displacement_context_init_index } else { displacementContextInitType = asps_vdmc_ext_displacement_context_init_type displacementContextInitIndex = asps_vdmc_ext_displacement_context_init_index } Alternatively, if a syntax element appears at multiple levels, the context initialization parameters displacementContextInitType and displacementContextInitIndex may be derived by overriding values ​​from lower levels. displacementContextInitType = asps_vdmc_ext_displacement_context_init_type displacementContextInitIndex = asps_vdmc_ext_displacement_context_init_index if ( afps_vdmc_ext_displacement_context_init_enable_flag ) { displacementContextInitType = afps_vdmc_ext_displacement_context_init_type displacementContextInitIndex = afps_vdmc_ext_displacement_context_init_index } Also, the mesh displacement decoding unit 305 may derive the context initialization parameters displacementContextInitType and displacementContextInitIndex each time the mesh displacement decoding unit 305 decodes each of the context initialization parameters. The gating flag may be, for example, afps_vdmc_ext_displacement_context_init_enable_flag. The context initialization parameters may be, for example, asps_vdmc_ext_displacement_context_init_type, asps_vdmc_ext_displacement_context_init_index, afps_vdmc_ext_displacement_context_init_type, and afps_vdmc_ext_displacement_context_init_index. displacementContextInitType = asps_vdmc_ext_displacement_context_init_type displacementContextInitIndex = asps_vdmc_ext_displacement_context_init_index displacementContextInitType = afps_vdmc_ext_displacement_context_init_type displacementContextInitIndex = afps_vdmc_ext_displacement_context_init_inde

[0056] (Mesh displacement syntax structure) 15 is an example of a syntax structure of mesh displacement. The mesh displacement decoding unit 305 decodes coeff_num displacements from the encoded data. The mesh displacement decoding unit 305 decodes gt0_flag, and if gt0_flag is a predetermined value, decodes the following sign_flag and gt1_flag. If gt1_flag is a predetermined value, the mesh displacement decoding unit 305 decodes the following rem_prefix. Furthermore, if rem_prefix satisfies a predetermined condition (for example, if it exceeds a predetermined value), the mesh displacement decoding unit 305 decodes rem_suffix. gt0_flag is a flag indicating whether the absolute value of the mesh displacement coefficient is greater than 0. gt1_flag is a flag indicating whether the absolute value of the mesh displacement coefficient is greater than 1. rem_prefix is ​​the prefix part of the Golomb code of the mesh displacement coefficient. rem_suffix is ​​the suffix part of the Golomb code of the mesh displacement coefficient.

[0057] (Operation of Mesh Displacement Decoding Unit) The arithmetic decoding unit 3051 decodes the arithmetically coded mesh displacement coded stream and outputs a binary signal. The binary signal may be a k-th order Exp-Golomb code.

[0058] The multi-value unit 3052 decodes the binary signal into the quantized mesh displacement Qdisp, which is a multi-value signal.

[0059] The context selection unit 3056 has a memory for holding contexts, and updates various contexts used in arithmetic decoding of mesh displacements according to the state. In arithmetic decoding of each coefficient of mesh displacements, the following different context arrays may be used according to the frame type ft (e.g., 0: intraframe, 1: interframe), the mesh division level lod (level of detail), and the dimension dim of the mesh displacement vector. The context includes at least a variable indicating the occurrence probability of a binary signal. ctxSign[NUM_FT][NUM_LOD][NUM_DIM] ctxCoeffGtN[NUM_FT][NUM_LOD][2][NUM_DIM] ctxCoeffRemPrefix[NUM_FT][NUM_LOD][NUM_DIM][7] ctxCoeffRemSuffix[NUM_FT][NUM_LOD][NUM_DIM][7] Here, NUM_FT is the number of frame types, and NUM_FT = 2. NUM_LOD is the maximum number of levels in mesh division, and NUM_LOD = 4. NUM_DIM is the number of dimensions of the mesh displacement vector, and NUM_DIM = 3. ctxSign[NUM_FT][NUM_LOD][NUM_DIM] are an array of contexts used to decode the syntax element sign_flag. The arithmetic decoding unit 3051 decodes the sign_flag of the frame type ft, the level lod, and the displacement of the dimension dim of the mesh displacement vector using the values ​​of ctxSign[ft][lod][dim]. Here, an example using a context is described, but bypass may be used without using a context. The bypass configuration has the effect of reducing the memory and processing amount of the context. ctxCoeffGtN[NUM_FT][NUM_LOD][2][NUM_DIM] is a context array used for decoding the syntax elements gt0_flag and gt1_flag. The arithmetic decoding unit 3051 decodes gt0_flag and gt1_flag of the frame type ft, the level lod, and the displacement of the dimension dim of the mesh displacement vector, using the values ​​of ctxCoeffGtN[ft][lod][0][dim] and ctxCoeffGtN[ft][lod][1][dim]. ctxCoeffRemPrefix[NUM_FT][NUM_LOD][NUM_DIM][7] is a context array used for decoding the syntax element rem_prefix. The arithmetic decoding unit 3051 uses the value of ctxCoeffRemPrefix[ft][lod][dim][binIdx] to decode rem_prefix[ft][lod][dim] of the displacement of the frame type ft, the level lod, and the dimension dim of the mesh displacement vector. Here, binIdx indicates the bin position of the binary of rem_prefix. ctxCoeffRemSuffix[NUM_FT][NUM_LOD][NUM_DIM][7] is a context array used for decoding the syntax element rem_suffix. The arithmetic decoding unit 3051 decodes rem_suffix[ft][lod][dim] of the displacement of the frame type ft, the level lod, and the dimension dim of the mesh displacement vector, using the value of ctxCoeffRemPrefix[ft][lod][dim][binIdx].

[0060] The context initialization unit 3057 initializes the context (probability of occurrence of a binary signal) based on parameters (here, the context initialization parameters displacementContextInitType and displacementContextInitIndex) decoded from the encoded data. If the value of displacementContextInitType is equal to a first value (e.g., 0), the context is initialized every GoF. If the value of displacementContextInitType is equal to a second value (e.g., 1), the context is initialized every frame.

[0061] As described above, according to the context initialization parameters, the initialization timing and initial values of the context can be set. When initializing the context for each frame, since there is no context dependency between frames, random access to any frame can be easily performed, and the coding efficiency can be improved. When initializing the context for each GoF, the coding efficiency can be further improved compared to the case of initializing for each frame. An example of the context initial value table is shown below. Each value value in the initialization table is a value obtained by expressing the lower 16 bits of the occurrence probability p in hexadecimal, and p = value / (1 << 16). For example, value = 0x8000 indicates an occurrence probability p = 0.5.

[0062] <Example of the initialization table for sign_flag> const uint16_t LUT_ctxSign[NUM_FT][NUM_LOD][NUM_DIM] = { / / Intra { {0x6b22, 0x9d7d, 0x7e03}, {0xc733, 0x718c, 0x8d53}, {0xd1e1, 0x76a5, 0x7c78}, {0xb968, 0x7844, 0x7844} }, / / Inter { {0x5252, 0x9ba6, 0x853c}, {0x9d59, 0x7e6d, 0x80bf}, {0x8458, 0x685a, 0x91f7}, {0x88a0, 0x8870, 0x802d} } };

[0063] <Example of the initialization table for gt0_flag, gt1_flag> const uint16_t LUT_ctxCoeffGtN[NUM_FT][NUM_LOD][2][NUM_DIM] = { / / Intra { { {0x50f4, 0xf2fd, 0xf25a}, {0x5f4f, 0xf55d, 0xfc36} }, { {0x42d1, 0xf272, 0xfaf4}, {0x562d, 0xfb80, 0xfaa1} }, { {0xe6e7, 0xfe01, 0xfe02}, {0xe410, 0xff00, 0xff00} }, { {0xfd12, 0xfe01, 0xfe01}, {0xff01, 0x94a6, 0x94a6} } }, / / Inter { { {0xb029, 0xfb22, 0xf47c}, {0xed0f, 0xef09, 0xeefb} }, { {0xcfc7, 0xf752, 0xf85f}, {0xf04c, 0xf769, 0xfbb4} }, { {0xf48b, 0xfee4, 0xfe01}, {0xfd5f, 0xfd9c, 0xfb2d} }, { {0xfe02, 0xfe01, 0xfe02}, {0xff01, 0xb519, 0xa496} } }}

[0064] <Example of the initialization table for <rem_prefix>> const uint16_t LUT_ctxCoeffRemPrefix[NUM_FT][NUM_LOD][NUM_DIM][7] = { / / Intra { { {0xa512, 0xe6a5, 0xf888, 0xc5af, 0x8000, 0x8000, 0x8000}, {0xd0c3, 0x94a6, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0xcbcf, 0x8780, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0x8ab5, 0xbf66, 0xab08, 0xf98a, 0xf1f6, 0x8000, 0x8000}, {0xe4fe, 0x9a6b, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0xf6de, 0x94a6, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0xcc6b, 0xf94b, 0xf93d, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0xd48f, 0xa496, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} } }, / / Inter { { {0xc695, 0xa158, 0xca9d, 0xcde8, 0x8000, 0x8000, 0x8000}, {0x6338, 0xcde0, 0xb49d, 0x8000, 0x8000, 0x8000, 0x8000}, {0x9087, 0xb3f6, 0x9d0e, 0x8e60, 0x8000, 0x8000, 0x8000} }, { {0x9c98, 0xdd87, 0xcfaa, 0xb10f, 0x8000, 0x8000, 0x8000}, {0xca9c, 0xad38, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0xd91f, 0xa496, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0xd15f, 0xdb03, 0x8780, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x94a6, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0x8890, 0x8e60, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8780, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} } } };

[0065] <Example of the initialization table for rem_suffix> const uint16_t LUT_ctxCoeffRemSuffix[NUM_FT][NUM_LOD][NUM_DIM][7] = { / / Intra { { {0xd236, 0xbced, 0xbd22, 0x8000, 0x8000, 0x8000, 0x8000}, {0x94a6, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8780, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0x8e79, 0xcf96, 0xd6be, 0xed44, 0x8000, 0x8000, 0x8000}, {0x9a6b, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x94a6, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0xd88b, 0xea9d, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0xa496, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} } }, / / Inter { { {0xc3e1, 0xc53a, 0xb55e, 0x8000, 0x8000, 0x8000, 0x8000}, {0xbfc2, 0xb49d, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0xc348, 0x8e06, 0x8e60, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0xc29b, 0xc4f1, 0xa6d8, 0x8000, 0x8000, 0x8000, 0x8000}, {0xad38, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0xa496, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0xa848, 0x7871, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} }, { {0x8e60, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000}, {0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000, 0x8000} } } };

[0066] The context initialization unit 3057 initializes the initial values ​​of the context used for encoding and decoding the binaries of sign_flag, gt0_flag, gt1_flag, rem_prefix, and rem_suffix to the values ​​of LUT_ctxSign[ft][lod][dim], LUT_ctxCoeffGtN[ft][lod][][dim], LUT_ctxCoeffRemPrefix[ft][lod][dim][], and LUT_ctxCoeffRemSuffix[ft][lod][dim][], respectively, according to the frame type ft, the mesh division level lod, and the dimension dim of the mesh displacement vector. Note that if the mesh division level lod exceeds NUM_LOD-1, the initial values ​​are initialized to default values ​​(for example, value=0x8000, p=0.5).

[0067] Since the initial value of the context is switched according to the mesh displacement characteristics ft, lod and dim, the coding efficiency can be improved.

[0068] (Configuration to switch context initialization depending on displacementContextInitIndex) Furthermore, the initialization may be performed using displacementContextInitIndex decoded from the encoded data. The context initialization unit 3057 switches the initial value of the context used for encoding and decoding the binaries of sign_flag, gt0_flag, gt1_flag, rem_prefix, and rem_suffix according to the context initialization index displacementContextInitIndex, the frame type ft, the mesh division level lod, and the dimension dim of the mesh displacement vector. Specifically, the initial values ​​are initialized with the values ​​of LUT_ctxSign[displacementContextInitIndex][ft][lod][dim], LUT_ctxCoeffGtN[displacementContextInitIndex][ft][lod][][dim], LUT_ctxCoeffRemPrefix[displacementContextInitIndex][ft][lod][dim][], and LUT_ctxCoeffRemSuffix[displacementContextInitIndex][ft][lod][dim][]. Furthermore, by the encoder appropriately selecting the displacementContextInitIndex that uses the optimal context initial value and transmitting it as syntax, it is possible to derive a more appropriate context for the content, thereby improving the encoding efficiency.

[0069] (Configuration to switch context initialization depending on displacementCoordinateSystem) The context initialization unit 3057 may switch the context initialization method depending on whether the coordinate system is a Cartesian coordinate system or a local coordinate system.

[0070] For example, when sys=displacementCoordinateSystem, the context initialization unit 3057 sets the initial values ​​of the context to be used for encoding / decoding in binary sign_flag, gt0_flag, gt1_flag, rem_prefix, and rem_suffix, for example, by using the values ​​of LUT_ctxSign[sys][displacementContextInitIndex][ft][lod][dim], LUT_ctxCoeffGtN[sys][displacementContextInitIndex][ft][lod][][dim], LUT_ctxCoeffRemPrefix[sys][displacementContextInitIndex][ft][lod][dim][], and LUT_ctxCoeffRemSuffix[sys][displacementContextInitIndex][ft][lod][dim][], depending on the context initialization index displacementContextInitIndex, the coordinate system sys, the frame type ft, the mesh division level lod, and the dimension dim of the mesh displacement vector, respectively.

[0071] According to the above configuration, the context initial value is changed according to the syntax element displacementCoordinateSystem indicating the coordinate system in the encoded data, so that an appropriate initial value can be derived according to the coordinate system, and the encoding efficiency can be improved. In the local coordinate system (n, t, b), the change in displacement corresponding to a small dim value is larger than in the Cartesian coordinate system (x, y, z). Therefore, it is appropriate to set the initial value accordingly.

[0072] In addition, in the (x, y, z) coordinate system, a context initial value having the same value may be used regardless of the value of dim. Also, when the displacementCoordinateSystem indicates a Cartesian coordinate system, the amount of memory required for the context initial value table can be reduced by setting dim=0 and referencing the context initial value table.

[0073] The inverse quantization unit 3053 performs inverse quantization based on the quantization scale value iscale, and derives the mesh displacement Tdisp after transformation (for example, wavelet transformation). Tdisp may be in a Cartesian coordinate system or a local coordinate system. iscale is a value derived from the quantization parameter of each component of the mesh displacement image. Tdisp[0][] = (Qdisp[0][] * iscale[0] + iscaleOffset) >>iscaleShift Tdisp[1][] = (Qdisp[1][] * iscale[1] + iscaleOffset) >>iscaleShift Tdisp[2][] = (Qdisp[2][] * iscale[2] + iscaleOffset) >>iscaleShift Here, iscaleOffset = 1<<(iscaleShift-1). iscaleShift may be a predetermined constant, or may be a value encoded at the sequence level, picture / frame level, tile / patch level, etc., and decoded from the encoded data.

[0074] The inverse transform unit 3054 performs an inverse transform g (eg, an inverse wavelet transform) to derive the mesh displacement d. d[0][] = g(Tdisp[0][]) d[1][] = g(Tdisp[1][]) d[2][] = g(Tdisp[2][]) The coordinate system conversion unit 3055 converts the mesh displacement (coordinate system of the mesh displacement) into the Cartesian coordinate system based on the value of the coordinate system conversion parameter displacementCoordinateSystem. Specifically, when displacementCoordinateSystem = 1, the displacement in the local coordinate system is converted into the displacement in the Cartesian coordinate system. Here, d is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion. disp is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and is a Cartesian coordinate system. n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target area or target vertex. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } The derivation method shown above for vector multiplication can be expressed individually as scalars as follows. if (displacementCoordinateSystem == 0) { for (i = 0; i < 3; i++) {disp[i] = d[i]} } else if (displacementCoordinateSystem == 1){ for (i = 0; i < 3; i++) {disp[i] = d[0] * n_vec[i] + d[1] * t_vec[i] + d[2] * b_vec[i]} } Note that a configuration may be adopted in which disp=d is used, the same variable name is assigned before and after the transformation, and the value of d is updated by the coordinate transformation.

[0075] Alternatively, the following configuration may be used. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 2){ disp = d[0] * n_vec2 + d[1] * t_vec2 + d[2] * b_vec2 } Here, n_vec2, t_vec2, and b_vec2 are three-dimensional vectors (in the Cartesian coordinate system) corresponding to each axis of the local coordinate system of the adjacent region.

[0076] The following configuration may also be used. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } Here, n_vec3, t_vec3, and b_vec3 are three-dimensional vectors (in Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target area with reduced fluctuation. For example, derive the vector of the coordinate system used for decoding from the previous coordinate system and the current coordinate system as follows: n_vec3 = (w*n_vec3 + (WT-w)*n_vec)>>wShift t_vec3 = (w*t_vec3 + (WT-w)*t_vec)>>wShift b_vec3 = (w*b_vec3 + (WT-w)*b_vec)>>wShift Here, for example, wShift = 2, 3, 4, WT = 1 << wShift, and w = 1..WT - 1. For example, when w = 3 and wShift = 3, n_vec3 = (3 * n_vec3 + 5 * n_vec) >> 3 t_vec3 = (3 * t_vec3 + 5 * t_vec) >> 3 b_vec3 = (3 * b_vec3 + 5 * b_vec) >> 3 Also, a configuration may be adopted in which it is possible to select according to the value of the parameter displacementCoordinateSystem decoded from the encoded data as in the following configuration. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 6){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3

[0077] (Mesh Reconstruction) FIG. 6 is a functional block diagram showing the configuration of the mesh reconstruction unit 307. The mesh reconstruction unit 307 is composed of a mesh division unit 3071 and a mesh deformation unit 3072.

[0078] The mesh division unit 3071 divides the base mesh output from the base mesh decoding unit 303 to generate a divided mesh.

[0079] Fig. 9(a) shows a part (triangle) of the base mesh, and the triangle is composed of vertices v1, v2, and v3. v1, v2, and v3 are three-dimensional vectors. The mesh division unit 3071 generates and outputs a divided mesh by adding new vertices v12, v13, and v23 to the middle of each side of the triangle (Fig. 9(b)). v12 = (v1 + v2) / 2 v13 = (v1 + v3) / 2 v23 = (v2 + v3) / 2 The following is also acceptable. v12 = (v1 + v2 + 1) >>1 v13 = (v1 + v3 + 1) >>1 v23 = (v2 + v3 + 1) >>1 The mesh deformation unit 3072 receives the divided mesh and the mesh displacement, and generates and outputs a deformed mesh by adding the mesh displacements d12, d13, and d23 (FIG. 9(c)). The mesh displacements are the output of the mesh displacement decoding unit 305 (coordinate system conversion unit 3055). d12, d13, and d23 are mesh displacements corresponding to the vertices v12, v13, and v23 added by the mesh division unit 3071. v12' = v12 + d12 v13' = v13 + d13 v23' = v23 + d23 In addition, d12 = disp[0][], d23 = disp[1][], and d23 = disp[3][] may also be used.

[0080] (Configuration of 3D data encoding device according to the first embodiment) 10 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11 according to the first embodiment. The 3D data encoding device 11 includes an atlas information encoding unit 101, a base mesh encoding unit 103, a base mesh decoding unit 104, a mesh displacement updating unit 106, a mesh displacement encoding unit 107, a mesh displacement decoding unit 108, a mesh reconstruction unit 109, an attribute transfer unit 110, a padding unit 111, a color space conversion unit 112, an attribute encoding unit 113, a multiplexing unit 114, and a mesh separation unit 115. The 3D data encoding device 11 inputs atlas information, a base mesh, a mesh displacement, a mesh, and an attribute image as 3D data, and outputs encoded data.

[0081] The atlas information encoding unit 101 encodes the atlas information.

[0082] The base mesh encoding unit 103 encodes the base mesh and outputs a base mesh encoded stream. Draco or the like is used as the encoding method.

[0083] The base mesh decoding unit 104 is similar to the base mesh decoding unit 303, and therefore a description thereof will be omitted.

[0084] The mesh displacement update unit 106 adjusts the mesh displacement based on the (original) base mesh and the decoded base mesh, and outputs an updated mesh displacement.

[0085] The mesh displacement encoding unit 107 encodes the updated mesh displacement and outputs a mesh displacement encoded stream.

[0086] The mesh displacement decoding unit 108 is similar to the mesh displacement decoding unit 305, and therefore a description thereof will be omitted.

[0087] The mesh reconstructing unit 109 is similar to the mesh reconstructing unit 307, and therefore a description thereof will be omitted.

[0088] The attribute transfer unit 110 inputs the (original) mesh, the reconstructed mesh output from the mesh reconstruction unit 109 (mesh deformation unit 3072), and an attribute image, and outputs an attribute image optimized for the reconstructed mesh.

[0089] The padding unit 111 receives the optimized attribute image and performs padding processing on areas where pixel values ​​are empty.

[0090] A color space conversion unit 112 performs color space conversion from the RGB format to the YCbCr format.

[0091] The attribute encoding unit 113 encodes the attribute image in the YCbCr format output from the color space conversion unit 112, and outputs an encoded attribute image stream. As an encoding method, VVC, HEVC, or the like is used.

[0092] The multiplexing unit 114 multiplexes the atlas information coded stream, the base mesh coded stream, the mesh displacement coded stream, and the attribute image coded stream, and outputs the multiplexed stream as coded data. As a multiplexing method, a byte stream format, ISOBMFF, or the like is used.

[0093] (Mesh separation unit operation) The mesh separator 115 generates a base mesh and a mesh displacement from the mesh.

[0094] 13 is a functional block diagram showing the configuration of the mesh separation unit 115. The mesh separation unit 115 is made up of a mesh thinning unit 1151, a mesh division unit 1152, and a mesh displacement derivation unit 1153.

[0095] The mesh thinning unit 1151 generates a base mesh by thinning out some of the vertices from the mesh.

[0096] Fig. 14(a) shows a part of a mesh, which is composed of vertices v1, v2, v3, v4, v5, and v6. v1, v2, v3, v4, v5, and v6 are each three-dimensional vectors. The mesh thinning unit 1151 generates and outputs a base mesh by thinning out the vertices v4, v5, and v6 (Fig. 14(b)).

[0097] The mesh dividing unit 1152, like the mesh dividing unit 3071, divides the base mesh to generate divided meshes (FIG. 14(c)). v4' = (v1 + v2) / 2 v5' = (v1 + v3) / 2 v6' = (v2 + v3) / 2 The mesh displacement derivation unit derives and outputs the displacements d4, d5, d6 of vertices v4, v5, v6 relative to vertices v4', v5', v6' as mesh displacements based on the mesh and the divided meshes (FIG. 14(d)). d4 = v4 - v4' d5 = v5 - v5' d6 = v6 - v6

[0098] (Base mesh encoding) FIG. 11 is a functional block diagram showing the configuration of the base mesh encoding unit 103. The base mesh encoding unit 103 is composed of a mesh encoding unit 1031, a mesh decoding unit 1032, a motion information encoding unit 1033, a motion information decoding unit 1034, a mesh motion compensation unit 1035, a reference mesh memory 1036, a switch 1037, and a switch 1038. The base mesh encoding unit 103 may be configured to include a base mesh quantization unit (not shown) after the input of the base mesh. When the base mesh is encoded without referring to other base meshes (for example, an already encoded base mesh) (intra-encoding), the switch 1037 and the switch 3038 are connected to the side that does not perform motion compensation. When the base mesh is encoded with reference to other base meshes (inter-encoding), on the other hand, they are connected to the side that performs motion compensation.

[0099] The mesh encoding unit 1031 has a function of intra-encoding the base mesh, intra-encodes the base mesh, and outputs a base mesh encoded stream. Draco or the like is used as the encoding method.

[0100] The mesh decoding unit 1032 is similar to the mesh decoding unit 3031, and therefore a description thereof will be omitted.

[0101] The motion information encoding unit 1033 has a base mesh inter-encoding function, inter-encodes the base mesh, and outputs a base mesh encoded stream. Entropy encoding such as arithmetic encoding is used as the encoding method.

[0102] The motion information decoding unit 1034 is similar to the motion information decoding unit 3032, and therefore a description thereof will be omitted.

[0103] The mesh motion compensation unit 1035 is similar to the mesh motion compensation unit 3033, and therefore a description thereof will be omitted.

[0104] The reference mesh memory 1036 is similar to the reference mesh memory 3034, and therefore a description thereof will be omitted.

[0105] (Mesh displacement encoding) 12 is a functional block diagram showing the configuration of the mesh displacement encoding unit 107. The mesh displacement encoding unit 107 is made up of a coordinate system conversion unit 1071, a conversion unit 1072, a quantization unit 1073, a binarization unit 1074, an arithmetic encoding unit 1075, a context selection unit 1076, and a context initialization unit 1077.

[0106] The coordinate system conversion unit 1071 converts the coordinate system of the mesh displacement from a Cartesian coordinate system to a coordinate system that encodes the displacement (e.g., a local coordinate system) based on the value of the coordinate system conversion parameter displacementCoordinateSystem. Here, disp is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion, d is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) indicating each axis of the local coordinate system. if (displacementCoordinateSystem == 0) { d = disp } else if (displacementCoordinateSystem == 1){ d = (disp * n_vec, disp * t_vec, disp * b_vec) } The mesh displacement encoding unit 107 may update the value of displacementCoordinateSystem at the sequence level, or at the picture / frame level. The initial value is 0, which indicates a Cartesian coordinate system.

[0107] When updating the displacementCoordinateSystem at the sequence level, use the syntax of the configuration in Figure 7. Set asps_vdmc_ext_displacement_coordinate_system to 0 for a Cartesian coordinate system and 1 for a local coordinate system.

[0108] When changing the displacementCoordinateSystem at the picture / frame level, use the syntax in the configuration of Figure 8. For afps_vdmc_ext_displacement_coordinate_system_enable_flag, set 1 if you want to update the coordinate system, or 0 if you do not want to update the coordinate system. For afps_vdmc_ext_displacement_coordinate_system, set 0 if you want to use a Cartesian coordinate system, or 1 if you want to use a local coordinate system.

[0109] A transform unit 1072 performs a transform f (eg, wavelet transform) and derives a mesh displacement Tdisp after the transform. Tdisp[0][] = f(d[0][]) Tdisp[1][] = f(d[1][]) Tdisp[2][] = f(d[2][]) The quantization unit 1073 performs quantization based on a quantization scale value "scale" derived from the quantization parameter of each component of the mesh displacement, and derives a mesh displacement Qdisp after quantization. Qdisp[0][] = Tdisp[0][] / scale[0] Qdisp[1][] = Tdisp[1][] / scale[1] Qdisp[2][] = Tdisp[2][] / scale[2] Alternatively, the scale value may be approximated by a power of 2 and Qdisp may be derived using the following formula: scale[i] = 1 << scale2[i] Qdisp[0][] = Tdisp[0][] >> scale2[0] Qdisp[1][] = Tdisp[1][] >> scale2[1] Qdisp[2][] = Tdisp[2][] >> scale2[2] The binarization unit 1074 encodes the quantized mesh displacement Qdisp, which is a multi-value signal, into a binary signal. The binary signal may be a k-th order exponential-Golomb code.

[0110] The arithmetic coding unit 1075 arithmetically codes the binary signal and outputs a mesh displacement coded stream.

[0111] The context selection unit 1076 is similar to the context selection unit 3056, and therefore a description thereof will be omitted.

[0112] The context initialization unit 1077 is similar to the context initialization unit 3057, and therefore a description thereof will be omitted.

[0113] The mesh displacement encoding unit 107 may update the value of displacementContextInitType at the sequence level, or may update it at the picture / frame level.

[0114] When updating context initialization parameters at the sequence level, use the configuration syntax in Figure 7. For asps_vdmc_ext_displacement_context_init_type, set the value of displacementContextInitType (0 for initialization per GoF, 1 for initialization per segment). For asps_vdmc_ext_displacement_context_init_index, set the value of displacementContextInitIndex (index of the context initialization table).

[0115] When changing context initialization parameters at the picture / frame level, use the syntax configuration in Figure 8. For afps_vdmc_ext_displacement_context_init_enable_flag, set 1 if the context initialization parameters are to be updated, or 0 if the context initialization parameters are not to be updated. For afps_vdmc_ext_displacement_context_init_type, set the value of displacementContextInitType (0 for initialization per GoF, 1 for initialization per segment). For afps_vdmc_ext_displacement_context_init_index, set the value of displacementContextInitIndex (index of the context initialization table).

[0116] As described above, the initialization timing and the initial value of the context can be set according to the context initialization parameters. When the context is initialized for each frame, there is no context dependency between frames, so random access to any frame can be easily performed and the coding efficiency can be improved. When the context is initialized for each GoF, the coding efficiency can be improved more than when the context is initialized for each frame. In addition, the initial value of the context is switched according to the characteristics of the mesh displacement, so the coding efficiency can be improved.

[0117] Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention.

[0118] [Application example] The above-mentioned 3D data encoding device 11 and 3D data decoding device 31 can be mounted on various devices that transmit, receive, record, and play back 3D data for use. Note that the 3D data may be natural 3D data captured by a camera or the like, or artificial 3D data (including CG and GUI) generated by a computer or the like.

[0119] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. In other words, the technical scope of the present invention also includes embodiments obtained by combining technical means that are appropriately modified within the scope of the claims. [Industrial Applicability]

[0120] The embodiments of the present invention can be suitably applied to a 3D data decoding device that decodes encoded data in which 3D data is encoded, and a 3D data encoding device that generates encoded data in which 3D data is encoded. Also, the embodiments of the present invention can be suitably applied to the data structure of encoded data generated by the 3D data encoding device and referenced by the 3D data decoding device. [Explanation of symbols]

[0121] 11 3D data encoding device 101 Atlas Information Encoding Unit 103 Base mesh coding unit 1031 Mesh coding unit 1032 Mesh Decoding Unit 1033 Motion information encoding unit 1034 Motion information decoding unit 1035 Mesh motion compensation unit 1036 Reference Mesh Memory 1037 Switch 1038 Switch 104 Base mesh decoding unit 106 Mesh displacement update section 107 Mesh Displacement Encoding Unit 1071 Coordinate system conversion section 1072 Converter 1073 Quantization section 1074 Binarization section 1075 Arithmetic encoder 1076 Context Selection 1077 Context Initialization Section 108 Mesh Displacement Decoding Unit 109 Mesh reconstruction unit 110 Attribute Transfer Part 111 Padding section 112 Color space conversion section 113 Attribute Encoding Unit 114 Multiplexer 115 Mesh separation section 1151 Mesh thinning section 1152 Mesh division section 1153 Mesh displacement derivation part 21 Network 31 3D data decoding device 301 Demultiplexer 302 Atlas Information Decoding Unit 303 Base Mesh Decoding Unit 3031 Mesh Decoding Unit 3032 Motion information decoding unit 3033 Mesh Motion Compensation Unit 3034 Reference Mesh Memory 3035 Switch 3036 Switch 305 Mesh Displacement Decoding Unit 3051 Arithmetic Decoding Unit 3052 Multilevel conversion section 3053 Inverse quantization section 3054 Reverse conversion unit 3055 Coordinate system conversion section 3056 Context Selection Section 3057 Context Initialization Unit 307 Mesh reconstruction unit 306 Attribute Decoder 3071 Mesh division section 3072 Mesh transformation part 308 Color Space Conversion Unit 41 3D data display device

Claims

1. A 3D data decoding device that decodes encoded data includes an arithmetic decoding unit that arithmetically decodes mesh displacement from the encoded data, a context selection unit that selects a context in the arithmetic decoding, and a context initialization unit that sets an initial value of the context, wherein the context initialization unit decodes a context initialization parameter that initializes the context from the encoded data, the context initialization parameter includes initialization timing information indicating a timing for initializing the context; The context initialization unit initializes the context only when the initialization timing information is a first value and the context is at the beginning of a segment and a random access point, and when the initialization timing information is a second value, the context is always initialized at the beginning of the segment. A 3D data decoding device characterized by the above.

2. The 3D data decoding device according to claim 1, wherein the context initialization parameters further include context initial value selection information indicating a method for setting an initial value of the context.

3. A 3D data encoding device for encoding 3D data includes an arithmetic encoding unit that arithmetically encodes mesh displacement, a context selection unit that selects a context in the arithmetic encoding, and a context initialization unit that sets an initial value of the context, wherein the context initialization unit encodes a context initialization parameter that initializes the context into encoded data; the context initialization parameter includes initialization timing information indicating a timing for initializing the context; The context initialization unit initializes the context only when the initialization timing information is a first value and the context is at the beginning of a segment and a random access point, and when the initialization timing information is a second value, the context is always initialized at the beginning of the segment.

4. 4. The 3D data encoding device according to claim 3, wherein the context initialization parameters further include context initial value selection information indicating a method for setting an initial value of the context.

Images