3D data decoding device and 3D data encoding device

Abstract

The purpose of the present invention is to improve the encoding efficiency of a base mesh and encode / decode 3D data with high quality in encoding / decoding of 3D data using an arithmetic encoding scheme. A 3D data decoding device for decoding encoded data, the 3D data decoding device being characterized by being provided with: a mesh prediction unit for deriving, from the encoded data, a predicted position of a basic mesh vertex position; and an arithmetic decoding unit for arithmetically decoding a predicted residual, the arithmetic decoding unit decoding a portion of the beginning of the prefix of the absolute value of the coefficient of the predicted residual using a context, adding the predicted position to the predicted residual, and deriving the basic mesh vertex position.

Description

Technical Field

[0001] Embodiments of the present invention relate to a 3D data encoding device and a 3D data decoding device. Background Art

[0002] In order to efficiently transmit or record 3D data, there is a 3D data encoding device that converts 3D data into a two-dimensional image, encodes the image using a motion image encoding method, and generates encoded data, and a 3D data decoding device that decodes the two-dimensional image from the encoded data and reconstructs the 3D data.

[0003] As specific 3D data encoding methods, for example, MPEG-I's ISO / IEC 23090-5 V3C (Volumetric Video-based Coding) and V-PCC (Video-based Point Cloud Compression). V3C can encode and decode point clouds consisting of point positions and attribute information. In addition, it can also be used for encoding and decoding of multi-viewpoint images and mesh images through ISO / IEC 23090-12 (MPEG Immersive Video (MPEG Immersive Video: MIV)) and the ISO / IEC 23090-29 (Video-based Dynamic Mesh Coding (V-DMC)) that is being standardized. The latest draft file of the V-DMC method is disclosed in non-patent document 1.

[0004] In these 3D data encoding methods, geometry and attributes constituting 3D data are encoded / decoded as images using moving picture encoding methods such as H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding).

[0005] In the case of a point group, the geometric image is the depth to the projection plane, and the attribute image is the image obtained by projecting the attributes onto the projection plane.

[0006] 3D data (mesh) as described in non-patent document 1 is composed of a basic mesh, mesh displacement, and texture mapping image. The basic mesh can be encoded using a vertex encoding method such as Draco. In addition to the method of encoding the mesh displacement image by two-dimensionalizing the mesh displacement using an animation codec, there is also a method of directly encoding by arithmetic coding. The texture mapping image is encoded as an attribute image using an animation codec. The animation codec can use the above-mentioned HEVC and VVC. Prior art literature Non-patent literature Non-patent document 1

[0007] WD 4.0of V-DMC(MDS23075_WG07_N00680_clean), ISO / IEC JTC 1 / SC 29 / WG7N0680, July 2023 Summary of the invention Problem that the invention aims to solve

[0008] In the 3D data encoding method in non-patent document 1, an arithmetic coding method can be used to encode / decode the grid displacement (grid displacement arrangement, grid displacement image), grid motion information, and basic grid that constitute 3D data (grid). When the grid displacement, grid motion information, and basic grid are arithmetically encoded, the performance depends on the syntax elements and context, so there is a technical problem of poor coding efficiency.

[0009] An object of the present invention is to improve the coding efficiency of a basic grid in coding / decoding of 3D data using an arithmetic coding method, and to encode / decode 3D data with high quality. Technical Solution

[0010] In order to solve the above-mentioned problem, a 3D data decoding device of one scheme of the present invention is a 3D data decoding device for decoding encoded data, and the 3D data decoding device is characterized in that it comprises: a mesh prediction unit for deriving a predicted position of a basic mesh vertex position from the above-mentioned encoded data; and an arithmetic decoding unit for arithmetically decoding the prediction residual, the arithmetic decoding unit using a context to decode a part of the beginning of a prefix of the absolute value of a coefficient of the above-mentioned prediction residual, adding the above-mentioned prediction position to the above-mentioned prediction residual and deriving the above-mentioned basic mesh vertex position.

[0011] In order to solve the above-mentioned problem, a 3D data encoding device of one scheme of the present invention is a 3D data encoding device for encoding 3D data, and the 3D data encoding device is characterized in that it comprises: a mesh prediction unit for deriving a predicted position of a basic mesh vertex position; and an arithmetic coding unit for arithmetically coding the prediction residual, and the arithmetic coding unit uses a context to encode a part of the beginning of a prefix of the absolute value of the coefficient of the prediction residual. Beneficial Effects

[0012] According to an aspect of the present invention, the coding efficiency of the basic grid can be improved, and 3D data can be encoded / decoded with high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1This is a schematic diagram showing the configuration of a 3D data transmission system according to this embodiment. Figure 2 This is a diagram showing the hierarchical structure of the data of the coded stream. Figure 3 It is a functional block diagram showing a schematic configuration of the 3D data decoding device 31 . Figure 4 It is a functional block diagram showing the structure of the basic grid decoding unit 303. Figure 5 It is a functional block diagram showing the structure of the grid displacement decoding unit 305. Figure 6 It is a functional block diagram showing the configuration of the mesh reconfiguration unit 307 . Figure 7 This is an example of a syntax for transmitting coordinate conversion parameters and context initialization parameters at the sequence level (ASPS). Figure 8 This is an example of a syntax for transmitting coordinate conversion parameters and context initialization parameters at a picture / frame level (AFPS). Fig. 9 This is a diagram for explaining the operation of the mesh reconfiguration unit 307 . Fig.10 It is a functional block diagram showing a schematic configuration of the 3D data encoding device 11 . Fig.11 It is a functional block diagram showing the structure of the basic grid encoding unit 103. Fig.12 It is a functional block diagram showing the structure of the grid displacement encoding unit 107. Fig.13 It is a functional block diagram showing the configuration of the mesh separation unit 115 . Fig.14 It is a diagram for explaining the operation of the mesh separation unit 115 . Fig.15 is an example of a grammatical structure for a grid shift. Fig.16 It is a functional block diagram showing the structure of the grid decoding unit 3031. Fig.17 is an example of the syntax structure of the basic grid. Fig.18 This is a diagram showing a method of deriving the context of a syntax element of a basic grid. Fig.19 It is a functional block diagram showing the structure of the grid coding unit 1031. DETAILED DESCRIPTION

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0015] Figure 1 It is a schematic diagram showing the configuration of a 3D data transmission system 1 according to the present embodiment.

[0016] The 3D data transmission system 1 is a system that transmits a coded stream obtained by encoding the 3D data to be encoded, decodes the transmitted coded stream and displays the 3D data. The 3D data transmission system 1 includes a 3D data encoding device 11, a network 21, a 3D data decoding device 31 and a 3D data display device 41.

[0017] The 3D data T is input to the 3D data encoding device 11 .

[0018] The network 21 transmits the coded stream Te generated by the 3D data encoding device 11 to the 3D data decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small network (LAN) or a combination thereof. The network 21 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. In addition, the network 21 may also be replaced by a storage medium such as a DVD (Digital Versatile Disc: Digital Versatile Disc, registered trademark) or a BD (Blu-ray Disc: Blu-ray Disc, registered trademark) that records the coded stream Te.

[0019] The 3D data decoding device 31 decodes the encoded stream Te transmitted by the network 21 respectively, and generates one or more decoded 3D data Td.

[0020] The 3D data display device 41 displays all or part of one or more decoded 3D data Td generated by the 3D data decoding device 31. The 3D data display device 41 includes, for example, a display device such as a liquid crystal display, an organic EL (Electro-luminescence) display, etc. As the form of the display, a fixed type, a mobile type, an HMD (Head Mounted Display), etc. can be listed. In addition, when the 3D data decoding device 31 has a high processing capability, a high-quality image is displayed, and when it has only a low processing capability, an image that does not require a high processing capability or a high display capability is displayed. <operator>

[0021] The operators used in this specification are described below.

[0022] >> is a right shift, << is a left shift, & is a bitwise AND, | is a bitwise OR, |= is an OR substitution operator, and || represents a logical AND.

[0023] x? y:z is a ternary operator that takes y when x is true (other than 0) and takes z when x is false (0). y..z represents the set of integers from y to z. <Structure of coded stream Te>

[0024] Before describing in detail the 3D data encoding device 11 and the 3D data decoding device 31 of the present embodiment, the data structure of the encoded stream Te generated by the 3D data encoding device 11 and decoded by the 3D data decoding device 31 will be described.

[0025] Figure 2 The diagram shows the hierarchical structure of data in the coded stream Te. The coded stream Te has a data structure of either a V3C sample stream or a V3C unit stream. The V3C sample stream includes a sample stream header and a V3C unit. The V3C unit stream includes a V3C unit.

[0026] The V3C unit includes a V3C unit header and a V3C unit payload. The V3C unit header is a unit type (Unit Type) which is an ID indicating the type of the V3C unit and takes a value indicated by a tag such as V3C_VPS, V3C_AD, V3C_AVD, V3C_GVD, or V3C_OVD.

[0027] When the unit type is V3C_VPS (Video Parameter Set), the V3C unit includes a V3C parameter set.

[0028] When the unit type is V3C_AD (Atlas Data), the V3C unit includes a VPS ID, an atlasID, a sample stream nal header, and a plurality of NAL units. The atlasID is an ID (identification) and takes an integer value greater than 0.

[0029] The NAL unit includes the NAL unit type, layer ID, temporary ID, and RBSP (Raw byte sequence payload).

[0030] NAL units are identified by NAL unit types, including ASPS (Atlas Sequence Parameter Set), AAPS (Atlas Adaptation Parameter Set), ATL (Atlas Tile layer), SEI (Supplemental Enhancement Information), etc.

[0031] ATL includes an ATL header and an ATL data unit. The ATL data unit includes patch information data (patchinformation data) and other information such as the location and size of the patch.

[0032] SEI includes a payload type (payload Type) indicating the type of SEI, a payload size (payload Size) indicating the size (number of bytes) of SEI, and sei_payload indicating the data of SEI.

[0033] When the unit type is V3C_AVD (Attribute Video Data, Attribute Data), the V3C unit includes VPS ID, atlasID, attrIdx of the attribute image ID, partIdx of the partition ID, mapIdx of the map ID, auxFlag indicating whether it is auxiliary data (Auxiliary Date), and video stream (videostream). The video stream is data encoded by HEVC, VVC, etc. Attribute data corresponds to the texture image in V-DMC.

[0034] When the Nal unit type is V3C_GVD (Geometry Video Data, Geometry Data), the V3C unit includes VPS ID, atlasID, mapIdx, auxFlag, and video stream. Geometry data corresponds to grid displacement in V-DMC.

[0035] When the unit type is V3C_OVD (Occupancy Video Data, Occupancy Data), the V3C unit includes VPS ID, atlas ID, and video stream.

[0036] When the cell type is V3C_MD (Mesh data), the V3C cell includes VPS ID, atlas ID, and mesh_payload, which corresponds to the basic mesh in V-DMC. (Configuration of 3D Data Decoding Device According to First Embodiment)

[0037] Figure 3 3D data decoding device 31 of the first embodiment is a functional block diagram showing a schematic structure. 3D data decoding device 31 is composed of a demultiplexing unit 301, a graph information decoding unit 302, a basic grid decoding unit 303, a grid displacement decoding unit 305, a grid reconstruction unit 307, an attribute decoding unit 306, and a color space conversion unit 308. 3D data decoding device 31 inputs coded data of 3D data and outputs graph information, a grid, and an attribute image.

[0038] The demultiplexing unit 301 inputs the coded data multiplexed in the byte stream format, ISOBMFF (ISO Base Media File Format), etc., performs demultiplexing, and outputs the graph information coding stream (Atlas data stream, NAL unit of V3C_AD), the basic grid coding stream (grid_payload of V3C_MD), the grid displacement coding stream (video stream of V3C_GVD), and the attribute animation stream (video stream of V3C_AVD).

[0039] The spectral information decoding unit 302 inputs the spectral information encoded stream output from the demultiplexing unit 301, and decodes the spectral information.

[0040] Figure 3 The atlas information decoding unit 302 decodes the coordinate system conversion information displacementCoordinateSystem (asps_vdmc_ext_displacement_coordinate_system, afps_vdmc_ext_displacement_coordinate_system) representing the coordinate system from the encoded data. It should be noted that an enable flag may also be set separately, and each coordinate system conversion information is decoded only when the enable flag is 1. The enable flag is, for example, afps_vdmc_ext_displacement_coordinate_system_enable_flag.

[0041] The basic mesh decoding unit 303 decodes the basic mesh encoding stream encoded by vertex encoding (3D data compression encoding method, such as Draco), and outputs a basic mesh. The basic mesh will be described later.

[0042] The grid displacement decoding unit 305 decodes the grid displacement coding stream and outputs the grid displacement.

[0043] The mesh reconstruction unit 307 inputs the basic mesh and the mesh displacement, and reconstructs the mesh in the 3D space.

[0044] The attribute decoding unit 306 decodes the attribute animation stream encoded by VVC, HEVC, etc., and outputs an attribute image. The attribute image can be a texture image expanded on the UV axis (a texture mapping image converted in a UV atlas manner) or in YCbCr format. The type of codec used for encoding is represented by ptl_profile_codec_group_idc obtained by decoding the V3C parameter set of the encoded data. In addition, it can also be represented by four CC codes represented by ai_geometry_codec_id[atlasID] of the V3C parameter set. ai_geometry_codec_id[atlasID] represents the index corresponding to the codec ID of the decoder used to decode the attribute animation stream in the atlas ID.

[0045] The color space conversion unit 308 performs color space conversion on the attribute image from the YCbCr format to the RGB format. It should be noted that the attribute animation stream encoded in the RGB format may be decoded and the color space conversion may be omitted. (Decoding of the basic grid)

[0046] Figure 4 3031, a motion information decoding unit 3032, a mesh motion compensation unit 3033, a reference mesh memory 3034, a switch 3035, and a switch 3036. The basic mesh decoding unit 303 may include a basic mesh inverse quantization unit (not shown) before outputting the basic mesh. When the basic mesh of the decoding object is encoded (intra-frame encoding) without referring to other basic meshes (for example, an already encoded / decoded basic mesh), the switch 3035 and the switch 3036 are connected to the side where motion compensation is not performed. Otherwise, when the basic mesh of the decoding object is encoded (inter-frame encoding) with reference to other basic meshes, the switch 3035 and the switch 3036 are connected to the side where motion compensation is performed. When motion compensation is performed, the vertex coordinates of the object are derived with reference to the decoded vertex coordinates and motion information.

[0047] The mesh decoding unit 3031 decodes the base mesh coding stream after intra-frame coding, and outputs the base mesh (base mesh vertex position, base mesh vertex position vector). As the coding method, Draco, edge breaker, etc. are used.

[0048] The motion information decoding unit 3032 decodes the inter-coded basic mesh coded stream and outputs motion information (mesh motion information, mesh motion vector) for each vertex of a reference mesh described later. Entropy coding such as arithmetic coding is used as a coding method.

[0049] The mesh motion compensation unit 3033 performs motion compensation on each vertex of the reference mesh input from the reference mesh memory 3034 based on the motion information, and outputs a mesh after motion compensation.

[0050] The reference grid memory 3034 is a memory that stores the decoded grid for reference in subsequent decoding processing. (Decoding of grid displacement)

[0051] Figure 5 1 is a functional block diagram showing the structure of the grid displacement decoding unit 305. The grid displacement decoding unit 305 is composed of a CABAC decoding unit (arithmetic decoding unit 3051, multi-valued unit 3052, context selection unit 3056, context initialization unit 3057), an inverse quantization unit 3053, an inverse conversion unit 3054 and a coordinate system conversion unit 3055. (Context Adaptive Binary Arithmetic Coding)

[0052] The arithmetic decoding unit 3051, the multi-valued unit 3052, the context selection unit 3056, and the context initialization unit 3057 use a decoding method using a context called context-adaptive binary arithmetic coding (CABAC). In CABAC, a binary string consisting of 0 and 1 is encoded / decoded one bit at a time using a state variable (CABAC state) called a context. All CABAC states are initialized at the beginning of the segment. In the CABAC decoding unit, each bit of the binary string (Bin String) corresponding to the grammatical element is decoded. When using the context, a context index ctxInc is derived for each bit of the grammatical element, the bit is decoded using the context, and the CABAC state of the context is updated. Bits that do not use the context are decoded with equal probability (EP, bypass), and the update of the index ctxIdx of the specified context or the determined context is omitted. Context refers to a variable (memory area) used to maintain the probability (state) of CABAC, which is identified by the value of ctxIdx (0, 1, 2, ...). In addition, 0 and 1 are always of equal probability, that is, 0.5, and the case of 0.5 is called EP (Equal Probability) or bypass. In this case, there is no need to maintain the state for a specific grammatical element, so the context is not used. In addition, a static context with a fixed probability of 0.5 and no need to be updated can be used. In this sense, it can also be called static instead of bypass. An integer value such as 128 can be used to represent a value with a probability of 0.5.

[0053] It should be noted that the following pseudo code can be used for the process of performing 1-bit decoding without using context (through bypass). It should be noted that the following pseudo code can be used for the process of performing 1-bit decoding using context: Here, prob0 is a variable representing the probability of context. (Coordinate System)

[0054] The coordinate system of the mesh displacement (3D vector) uses the following two coordinate systems. Cartesian coordinate system (canonical): An orthogonal coordinate system commonly defined in the entire 3D space. (X, Y, Z) coordinate system. An orthogonal coordinate system whose direction remains unchanged at the same time (within the same frame, the same tile). Local coordinate system (local): An orthogonal coordinate system defined for each area or each vertex in 3D space. An orthogonal coordinate system whose direction can change at the same time (within the same frame, within the same tile). A coordinate system with axes of normal (D), tangent (U), and bi-tangent (V). That is, an orthogonal coordinate system including a first axis (D) represented by a normal vector n_vec at a vertex (a face containing a vertex) and a second axis (U) and a third axis (V) represented by two tangent vectors t_vec and b_vec orthogonal to the normal vector n_vec. n_vec, t_vec, and b_vec are three-dimensional vectors. The (D,U,V) coordinate system can also be called the (n,t,b) coordinate system. (Decoding / derivation of sequence level control parameters)

[0055] Here, the control parameters used in the grid displacement decoding unit 305 are described.

[0056] Figure 7 This is an example of the syntax for transmitting coordinate system conversion parameters in sequence-level ASPS. ASPS (Atlas Sequence Parameter Set or Atlas sequence mesh information) is one of the NAL units of atlas information, and includes syntax elements applied to the atlas information coding stream. In ASPS, coordinate system conversion parameters are transmitted through the asps_vdmc_extension() syntax. The semantics of each field are described below.

[0057] asps_vdmc_ext_subdivision_iteration_count: This is a parameter indicating the number of times the grid is segmented and repeated.

[0058] asps_vdmc_ext_displacement_coordinate_system: indicates the coordinate system conversion information of the coordinate system of the grid displacement. When the value is equal to a specified first value (for example, 0), it indicates the Cartesian coordinate system. When the value is equal to another second value (for example, 1), it indicates the local coordinate system.

[0059] asps_vdmc_ext_1d_displacement_flag: A flag indicating whether the grid displacement is one-dimensional. If the value is true, it indicates that the grid displacement is one-dimensional. If the value is false, it indicates that the grid displacement is three-dimensional. (Decoding / exporting of picture / frame level control parameters)

[0060] Figure 8 This is an example of the syntax for transmitting coordinate system conversion parameters by AFPS at the picture / frame level. AFPS (Atlas Frame Parameter Set or Atlas frame mesh information) is one of the NAL units of atlas information, including syntax elements applied to the atlas information coding stream. In AFPS, coordinate system conversion parameters are transmitted through the afps_vdmc_extension() syntax. The semantics of each field are described below.

[0061] afps_vdmc_ext_overriden_flag: A flag indicating whether to update the coordinate system of the mesh displacement. When this flag is true, the coordinate system of the mesh displacement is updated based on the value of afps_vdmc_ext_displacement_coordinate_system described later. When this flag is false, the coordinate system of the mesh displacement is not updated.

[0062] afps_vdmc_ext_subdivision_iteration_count: This is a parameter indicating the number of grid division repetitions.

[0063] afps_vdmc_ext_displacement_coordinate_system: Coordinate system conversion information of the coordinate system representing the grid displacement. When the value is equal to the first value (e.g., 0), the Cartesian coordinate system is represented. When the value is equal to the second value (e.g., 1), the local coordinate system is represented. When the syntax element is not present, the value is estimated as the value obtained by decoding according to ASPS, and the default coordinate system is set to the coordinate system represented by ASPS.

[0064] afps_vdmc_ext_1d_displacement_flag: A flag indicating whether the grid displacement is one-dimensional. If the value is true, it indicates that the grid displacement is one-dimensional. If the value is false, it indicates that the grid displacement is three-dimensional. (Syntax structure of grid displacement)

[0065] Fig.15This is an example of the grammatical structure of a grid displacement. The semantics are as follows. A grid displacement is a column of values ​​(coefficients) of a position pos and a k-component (k-component), as shown by the arrangement Qdisp[pos][k]. The displacement is a three-dimensional signal in a Cartesian coordinate system (xyz) or a local coordinate system (ntb), and each component of the three-dimensional displacement is called a component. The displacement Qdisp here is a value converted by discrete wavelet transform, lifting transform, DCT transform, etc., so it is also called a coefficient. The component variable k takes values ​​of 0, 1, or 2. The variable name is not limited to k, and dim can also be other variable names. The order of the index of QDisp can be reversed, that is, Qdisp[k][pos] can be used instead of Qdisp[pos][k].

[0066] diu_last_sig_coeff[k]: is the index representing the last position of the non-zero grid displacement coefficient in the k component.

[0067] diu_coded_block_flag[k][b]: Indicates whether the block with index b in component k includes non-zero grid displacement coefficients. If included, it is 1, otherwise it is 0.

[0068] diu_coded_subblock_flag[k][b][s]: indicates whether the subblock with index s of the block with index b in component k includes a non-zero grid displacement coefficient. If included, it is 1, otherwise it is 0.

[0069] diu_coeff_abs_level_gt0[k][b][s][v]: indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex of index v of the subblock of index s of the block of index b in component k is greater than 0. If greater than 0, it is 1, otherwise it is 0.

[0070] diu_coeff_abs_level_gt1[k][b][s][v]: indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex of index v of the subblock of index s of the block of index b in component k is greater than 1. If greater than 1, it is 1, otherwise it is 0. If this syntax element is not present, it is presumed to be 0.

[0071] diu_coeff_abs_level_gt2[k][b][s][v]: indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex of index v of the subblock of index s of the block of index b in component k is greater than 2. If greater than 2, it is 1, otherwise it is 0. If this syntax element is not present, it is presumed to be 0.

[0072] diu_coeff_abs_level_gt3[k][b][s][v]: indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex of index v of the subblock of index s of the block of index b in component k is greater than 3. If greater than 3, it is 1, otherwise it is 0. If this syntax element is not present, it is presumed to be 0.

[0073] diu_coeff_sign[k][b][s][v]: Indicates whether the non-zero mesh displacement coefficient of the vertex of index v of the sub-block of index s of the block of index b in component k is positive. For example, it is 1 if it is positive, and 0 otherwise (if it is negative). If this syntax element is not present, it is presumed to be 1.

[0074] diu_coeff_abs_level_rem[k][b][s][v]: is the value obtained by subtracting 4 from the absolute value of the non-zero mesh shift coefficient of the vertex of index v of the subblock of index s of the block of index b in the k component. If this syntax element is not present, it is presumed to be 0.

[0075] The trellis displacement decoding unit 305 decodes diu_last_sig_coeff for each trellis displacement component, and then derives the number lodCount of the k-component lod from diu_last_sig_coeff[k].

[0076] The lattice displacement decoding unit 305 decodes diu_coded_block_flag for each detail level (lod) of the lattice displacement, and then derives the number of blocks b, vertexCount, from diu_coded_block_flag[k][b].

[0077] The trellis shift decoding unit 305 decodes diu_coded_subblock_flag for each trellis shifted block, and then derives the start position vStart of the subblock s from diu_coded_subblock_flag[k][b][s].

[0078] The trellis shift decoding unit 305 decodes diu_coeff_abs_level_gt0 for each trellis shifted sub-block, and when diu_coeff_abs_level_gt0 is a predetermined value (eg, other than 0), decodes the subsequent diu_coeff_sign and diu_coeff_abs_level_gt1.

[0079] When diu_coeff_abs_level_gt1 is a predetermined value (for example, other than 0), the trellis displacement decoding unit 305 decodes the subsequent diu_coeff_abs_level_gt2.

[0080] When diu_coeff_abs_level_gt2 is a predetermined value (for example, other than 0), the trellis displacement decoding unit 305 decodes the subsequent diu_coeff_abs_level_gt3.

[0081] When diu_coeff_abs_level_gt3 is a predetermined value (for example, other than 0), the trellis displacement decoding unit 305 decodes the subsequent diu_coeff_abs_level_rem. (Operation of the Grid Shift Decoding Unit)

[0082] The arithmetic decoding unit 3051 decodes the grid displacement coding stream after arithmetic coding according to the value (context) representing the probability variable, and outputs a binary signal. The binary signal can be an alpha code or a k-th order Exp-Golomb-code. The exponential Golomb code consists of the symbols of the prefix and suffix. The prefix is ​​the value of the exponential increase, and the suffix is ​​the remainder. It should be noted that when the variable rem is encoded / decoded by the exponential Golomb code, the prefix and suffix of the exponential Golomb code are also referred to as the prefix and suffix of rem.

[0083] The multi-valued signal processing unit 3052 decodes the quantized grid displacement Qdisp as a multi-valued signal from the binary signal.

[0084] The context selection unit 3056 (context memory) has a memory for storing contexts, and derives contexts for arithmetic decoding of grid displacement according to the state, and updates the values ​​as needed. In the arithmetic decoding of each coefficient of grid displacement, the following different context arrangements can be used according to the frame type ft (for example, 0: intra frame, 1: inter frame), the level of detail lod (level of detail) of grid segmentation, and the component dim of the grid displacement vector. The context includes a variable indicating the probability of generation of a binary signal. ctxCodedSubBlock[numFT][numLOD][numDim] ctxCoeffGtN[numFT][numLOD][MAX_GTN+1][numDim] ctxCoeffRemPrefix[numFT][numLOD][numDim][numPrefixBin] It should be noted that a static context with a fixed probability of not updating the context is set as ctxStatic. The decoding of the syntax element indicated by ctxStatic can be performed without using the context. decode(ctxStatic) can use a bypass-specific process as decode_bypass().

[0085] Here, numFT is the number of types of frame types, and numFT = 2. numPrefixBin is the number of bins using the context in the prefix, and numPrefixBin = 2 may be used. numLOD is the maximum number of detail levels for mesh partitioning, and the value of the syntax element asps_vdmc_ext_subdivision_iteration_count or afps_vdmc_ext_subdivision_iteration_count obtained from bitstream decoding may be used, or numLOD=4 may be set. numLOD=asps_vdmc_ext_subdivision_iteration_count numLOD=afps_vdmc_ext_subdivision_iteration_count numDim is the dimension of the grid displacement vector, and the value of the syntax element asps_vdmc_ext_1d_displacement_flag or afps_vdmc_ext_1d_displacement_flag obtained by decoding the bitstream may be used, or numDim=3 may be set. numDim=asps_vdmc_ext_1d_displacement_flag? 1:3 numDim=afps_vdmc_ext_1d_displacement_flag? 1:3 The maximum value MAX_GTN of the threshold value for the coefficient is 3.

[0086] ctxCodedSubBlock[numFT][numLOD][numDim] is an array of contexts for decoding the syntax element diu_coded_subblock_flag. The arithmetic decoding unit 3051 decodes the diu_coded_subblock_flag of the frame type ft, the detail level lod, and the dimension dim of the grid displacement vector using the value of ctxCodedSubBlock[ft][lod][dim].

[0087] ctxCoeffGtN[numFT][numLOD][MAX_GTN+1][numDim] is an array of contexts for decoding the syntax element diu_coeff_abs_level_gtN (N is replaced with 0, 1, 2, MAX_GTN). The arithmetic decoding unit 3051 decodes diu_coeff_abs_level_gtN of the frame type ft, detail level lod, and dimension dim of the grid displacement vector using the value of ctxCoeffGtN[ft][lod][N][dim].

[0088] The arithmetic decoding unit 3051 uses a bypass to decode the frame type ft, the detail level lod, and the diu_coeff_sign of the dimension dim of the grid displacement vector.

[0089] ctxCoeffRemPrefix[numFT][numLOD][numDim][numPrefixBin] is an arrangement of contexts for decoding the syntax element diu_coeff_abs_level_rem. ctxCoeffRemPrefix[bin] represents the context at the bin position in the binarization of the prefix of diu_coeff_abs_level_rem. The arithmetic decoding unit 3051 decodes diu_coeff_abs_level_rem of the frame type ft, the detail level lod, and the dimension dim of the grid displacement vector using the value of ctxCoeffRemPrefix[ft][lod][dim].

[0090] The context initialization unit 3057 initializes the context (the probability of generating a binary signal). The context can be initialized for each frame or for a set of more than one frames (Group of Frames: GoF). When the context is initialized for each frame, there is no dependency of the context between frames, so random access to any frame can be easily performed. When the context is initialized for each GoF, the frequency of initialization is low, so the coding efficiency can be improved compared to the case of initialization for each frame. (Mesh displacement export processing)

[0091] The trellis displacement decoding unit 305 decodes the syntax elements diu_last_sig_coeff, diu_coded_block_flag, diu_coded_subblock_flag, diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3, diu_coeff_abs_level_rem, and diu_coeff_sign to derive the trellis displacement Qdisp. Here, the lattice displacement decoding unit 305 decodes diu_last_sig_coeff in units of components, decodes diu_coded_block_flag in units of LOD (in units of blocks), and decodes diu_coded_subblock_flag in units of subBlockSize size of subblocks. When diu_coded_subblock_flag is a predetermined value, the lattice displacement coefficient in the subblock is decoded. Here, decode(ctx) is a function that decodes a 1-bit value using the corresponding context ctx as an independent variable, and decodeExpGolomb(ctxPrefix, ctxSuffix) is a function that decodes a binarized value by k-times Golomb coding (for example, k=0). ctxPrefix[n] is used as the context of bin position n for the prefix, and ctxSuffix[m] is used as the context of bin position m for the suffix. When the context is not used in the suffix (by bypass), it is recorded as decodeExpGolomb(ctxPrefix). value++ is an operation that increments the variable value by 1, value+=1, value=value+1. subBlockSize is the size of the subblock. for indicates a loop. subBlockSize can use a power of 2 value between 16 and 4096. For example, it can be 128 or 256. dispCount[b] is the number of grid displacements at detail level b.

[0092] The inverse quantization unit 3053 performs inverse quantization based on the quantization level value iscale, and derives the grid displacement Tdisp after transformation (for example, wavelet transformation). Tdisp can be a Cartesian coordinate system or a local coordinate system. iscale is a value derived from the quantization parameters of each component of the grid displacement image. Tdisp[0][]=(Qdisp[0][]*iscale[0]+iscaleOffset)>>iscaleShift Tdisp[1][]=(Qdisp[1][]*iscale[1]+iscaleOffset)>>iscaleShift Tdisp[2][]=(Qdisp[2][]*iscale[2]+iscaleOffset)>>iscaleShift Here, iscaleOffset=1<<(iscaleShift-1). iscaleShift may be a preset constant, or a value obtained by decoding the encoded data after encoding at a sequence level, a picture / frame level, a tile / patch level, etc.

[0093] The inverse transformation unit 3054 performs inverse transformation g (for example, inverse wavelet transformation) to derive the mesh displacement d. d[0][]=g(Tdisp[0][]) d[1][]=g(Tdisp[1][]) d[2][]=g(Tdisp[2][]) The coordinate system conversion unit 3055 converts the mesh displacement (the coordinate system of the mesh displacement) into the Cartesian coordinate system based on the value of the coordinate system conversion information displacementCoordinateSystem. Specifically, when displacementCoordinateSystem == 1, the displacement from the local coordinate system is converted into the displacement in the Cartesian coordinate system. Here, d is a three-dimensional vector representing the mesh displacement before the coordinate system conversion. disp is a three-dimensional vector representing the mesh displacement after the coordinate system conversion, which is the Cartesian coordinate system. n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) corresponding to the respective axes of the local coordinate system of the object region or object vertex. When expressing the derivation method shown by the above vector multiplication individually by a scalar, it is as follows. It should be noted that a configuration can also be adopted where disp = d is set, the same variable name is assigned before and after the conversion, and the value of d is updated through coordinate conversion.

[0094] Alternatively, the following configuration can also be used. Here, n_vec2, t_vec2, and b_vec2 are three-dimensional vectors (in the Cartesian coordinate system) corresponding to the respective axes of the local coordinate system of the adjacent region.

[0095] In addition, the following configuration can also be used. Here, n_vec3, t_vec3, and b_vec3 are three-dimensional vectors (in the Cartesian coordinate system) corresponding to the respective axes of the local coordinate system of the object region where changes are suppressed. For example, the vectors for decoding the coordinate system are derived from the previous coordinate system and the current coordinate system as follows. n_vec3 = (w * n_vec3 + (WT - w) * n_vec) >> wShift t_vec3 = (w * t_vec3 + (WT - w) * t_vec) >> wShift b_vec3 = (w * b_vec3 + (WT - w) * b_vec) >> wShift Here, for example, wShift = 2, 3, 4, WT = 1 << wShift, and w = 1..WT - 1. For example, when w = 3 and wShift = 3, n_vec3 = (3 * n_vec3 + 5 * n_vec) >> 3 t_vec3=(3*t_vec3+5*t_vec)>>3 b_vec3=(3*b_vec3+5*b_vec)>>3 Alternatively, the following configuration may be used in which selection can be made based on the value of the coordinate system conversion information displacementCoordinateSystem obtained by decoding the encoded data. (Decoding of the basic grid)

[0096] Fig.16 This is a functional block diagram showing the configuration of the trellis decoding unit 3031. The trellis decoding unit 3031 is composed of a trellis prediction unit 30311 and a CABAC decoding unit (arithmetic decoding unit 3051, multi-valued unit 3052, context selection unit 3056, context initialization unit 3057). (Syntax structure of basic grid)

[0097] Fig.17 is an example of the syntax structure of a base mesh (vertex position vector prediction residual, base mesh vertex position vector). The semantics are as follows.

[0098] sismu_bm_zero[subMeshID][v][k]: Indicates whether the k component of the position vector prediction residual (coefficient) of the vertex with index v in the sub-mesh subMeshID is zero. If it is zero, it is 1, otherwise it is 0.

[0099] sismu_bm_sign[subMeshID][v][k]: Indicates whether the k component of the position vector prediction residual (coefficient) of the vertex with index v in the submesh subMeshID is a positive number. For example, it is 1 if it is a positive number, and 0 otherwise (if it is a negative number). If this syntax element does not exist, it is presumed to be 1.

[0100] sismu_bm_level_bias[subMeshID][v][k][b]: indicates whether there is a bias for the level b of the k component of the position vector prediction residual (coefficient) of the vertex with index v in the submesh subMeshID. It is 0 if there is no bias, otherwise it is 1. If this syntax element is not present, it is presumed to be 0.

[0101] sismu_bm_coeff_abs_rem[subMeshID][v][k]: represents the value obtained by subtracting 4 from the absolute value of the k component of the position vector prediction residual (coefficient) of the vertex of index v in the submesh subMeshID. If this syntax element does not exist, it is estimated to be 0.

[0102] The mesh decoding unit 3031 decodes sismu_bm_zero for each vertex of each sub-mesh and each component k, and when sismu_bm_zero is a predetermined value (for example, 0), decodes sismu_bm_sign and the subsequent sismu_bm_level_bias.

[0103] The mesh decoding unit 3031 decodes sismu_bm_coeff_abs_rem for each vertex of each sub-mesh and each component k when the absolute value of the coordinates (coefficient) of the vertex is greater than or equal to a predetermined value (for example, 4).

[0104] The semantics of sismu_bm_level_bias[subMeshID][v][k][b] can be defined as follows. sismu_bm_level_bias[subMeshID][v][k][b]: Indicates whether the absolute value of the k component of the position vector prediction residual of the vertex with index v in the sub-mesh subMeshID is greater than b+1. It should be noted that gt, which means greater than, can also be used instead of bias. If greater than, it is 1, otherwise it is 0. If this syntax element does not exist, it is presumed to be 0. (Operation of the Trellis Decoding Unit)

[0105] The mesh prediction unit 30311 predicts the position vector of the current vertex based on the position vector of the decoded vertex, and derives the predicted position vector BmVertexPosPred.

[0106] The basic operation of the CABAC decoding unit (arithmetic decoding unit 3051, multi-valued unit 3052, context selection unit 3056, context initialization unit 3057) is the same as that of the CABAC decoding unit of the grid displacement decoding unit 305. In the arithmetic decoding of the position vector prediction residual (coefficient) of each vertex, the following different context arrangements can be used according to the components (dimensions) of the vertex position vector, etc. The context includes a variable indicating the probability of generation of a binary signal. ctxBmZero ctxBmLevelBias[2][numLevel] ctxBmCoeffRemPrefix[numPrefixBin] Here, numLevel is the number of levels of bias using context, and is a fixed value, for example, numLevel = 2. numPrefixBin is the number of bins using context in the prefix, and can be numPrefixBin = 2.

[0107] ctxBmZero is a context used in decoding the syntax element sismu_bm_zero. The arithmetic decoding unit 3051 decodes sismu_bm_zero using the value of ctxBmZero. sismu_bm_zero may also be referred to as sismu_bm_sigcoeff.

[0108] The arithmetic decoding unit 3051 decodes sismu_bm_sign without using the context (through bypass).

[0109] ctxBmLevelBias[2][numLevel] is the sequence of contexts used in decoding the syntax element sismu_bm_level_bias. The arithmetic decoding unit 3051 decodes sismu_bm_level_bias in level b using the value of ctxBmLevelBias[0][min(b, numLevel-1)] or ctxBmLevelBias[1][min(b, numLevel-1)].

[0110] ctxBmCoeffRemPrefix[numPrefixBin] is the sequence of contexts used in decoding the syntax element sismu_bm_coeff_abs_rem. ctxBmCoeffRemPrefix[bin] indicates the context at the bin position in binarization of the prefix of sismu_bm_coeff_abs_rem. The arithmetic decoding unit 3051 decodes each bin of the prefix of sismu_bm_coeff_abs_rem of the vertex vector using the value of ctxBmCoeffRemPrefix[bin].

[0111] The arithmetic decoding unit 3051 decodes each bin of the suffix of sismu_bm_coeff_abs_rem of the vertex position vector without using the context (bypassing).

[0112] As another configuration, a configuration using context also for the suffix of sismu_bm_coeff_abs_rem may be used.

[0113] ctxBmCoeffRemSuffix[numSuffixBin] is the arrangement of contexts used in decoding of the syntax element sismu_bm_coeff_abs_rem. numSuffixBin is the number of bins in which the context is used in the suffix, and can be set to numSuffixBin = 2. ctxBmCoeffRemSuffix[bin] represents the context at the bin position in the binarization of the suffix of sismu_bm_coeff_abs_rem. The arithmetic decoding unit 3051 uses the value of ctxBmCoeffRemSuffix[bin] to decode each bin of the suffix of sismu_bm_coeff_abs_rem of the vertex position vector. (Equation-1) described later can also be expressed as (Equation-2). value+=decodeExpGolomb(ctxBmCoeffRemPrefix)(Equation-1) value+=decodeExpGolomb(ctxBmCoeffRemPrefix,ctxBmCoeffRemSuffix)(Equation-2) (Mesh export processing)

[0114] The mesh decoding unit 3031 decodes the syntax elements sismu_bm_zero, sismu_bm_sign, sismu_bm_level_bias, and sismu_bm_coeff_abs_rem from the coded data through the following processing, and derives the vertex position vector prediction residual BmVertexPosPredResidual of the basic mesh. The word break in the pseudocode means to skip subsequent actions and exit from the nearest loop.

[0115] It should be noted that the derivation process is performed in the order of dimension (component) loop and vertex loop, but it can also be in the order of vertex loop and dimension (component) loop. In addition, the method related to the sign of the coefficient can be the following arithmetic operation in addition to the method of reversing the value of sismu_bm_sign[subMeshID][v][k] in the above syntax (value=-value).

[0116] BmVertexPosPredResidual[subMeshID][v][k]= (1-2*sismu_bm_sign[subMeshID][v][k])*value Alternatively, the following may be obtained by reversing the positive and negative.

[0117] BmVertexPosPredResidual[subMeshID][v][k]= (2*sismu_bm_sign[subMeshID][v][k]–1)*value The mesh decoding unit 3031 derives the basic mesh vertex position vector BmVertexPos through the following processing. BmVertexPos=BmVertexPosPred+BmVertexPosPredResidual Alternatively, it may be the following formula. BmVertexPos[subMeshID][v][k]=BmVertexPosPred[subMeshID][v][k]+BmVertexPosPredResidual[subMeshID][v][k]k=0.. (Construction of using context for a part of the prefix / suffix bin)

[0118] Fig.18 Indicates the setting of the context used by the grid decoding unit 3031. The grid decoding unit 3031 (context selection unit 3056) uses the context to decode sismu_bm_zero and sismu_bm_level_bias. The grid decoding unit 3031 (context selection unit 3056) decodes sismu_bm_sign without using the context (through a bypass). The grid decoding unit 3031 (context selection unit 3056) uses the context to decode a part of the beginning of the prefix of sismu_bm_coeff_abs_rem, uses a bypass to decode the part other than the prefix, and uses a bypass to decode the suffix. In particular, different contexts can be used to decode the bin=0 and bin=1 of the prefix, and a bypass can be used to decode the bin>=2 of the prefix and later ( Fig.18 Alternatively, the context may be used to decode a portion of the beginning of the suffix. In particular, the context may be used to decode the prefix bin=0, the bypass may be used to decode the prefix bin>=1, the context may be used to decode the suffix bin=0, and the bypass may be used to decode the suffix bin>=1 ( Fig.18Alternatively, different contexts may be used to decode the prefix bin=0 and bin=1, and bypass may be used to decode the prefix bin>=2 and later, and different contexts may be used to decode the suffix bin=0 and bin=1, and bypass may be used to decode the suffix bin>=2 and later ( Fig.18 (c)). According to the above, even when the value of ExpGolomb coding becomes larger, only up to 4 contexts can be used, so the complexity can be reduced compared to the case where contexts are used in all bins of the remaining (sismu_bm_coeff_abs_rem). In addition, compared to the case where bypass is used in all bins of sismu_bm_coeff_abs_rem, coding can be performed efficiently.

[0119] exist Fig.18 In the decoding of sismu_bm_zero, the context ctxBmZero with CtxTbl=0 is used. In the decoding of sismu_bm_level_bias, the context ctxBmLevelBias[CtxInc][b] with CtxTbl=1 is used (CtxInc is replaced by islow). In the decoding of the prefix of sismu_bm_coeff_abs_rem, the context ctxBmCoeffRemPrefix[CtxInc] with CtxTbl=2 is used (CtxInc is replaced by the bin position bin of prefix). In the decoding of the suffix of sismu_bm_coeff_abs_rem, the context ctxBmCoeffRemSuffix[CtxInc] with CtxTbl=3 is used (CtxInc is replaced by suffix bin). (Construction that limits the number of context encoding bins to be decoded)

[0120] In order to reduce the complexity of context coding, the number of bins after context coding can be limited. Specifically, the grid decoding unit 3031 (context selection unit 3056) counts the number of bins decoded by context coding in the syntax element of sismu_bm_coeff_abs_rem in a specified unit for each specified number of vertices (vertexSize). Moreover, when this value is greater than the specified value maxContextInVertices, each bin of sismu_bm_coeff_abs_rem can be switched from decoding using context to decoding without using context (using bypass, static context). The pseudo code of this example is shown below. According to the above, the maximum value (worst case) of the number of bins for context coding can be reduced. (Reconstruction of the grid)

[0121] Figure 6 3 is a functional block diagram showing the configuration of the mesh reconfiguration unit 307. The mesh reconfiguration unit 307 is composed of a mesh division unit 3071 and a mesh deformation unit 3072.

[0122] The mesh dividing unit 3071 divides the basic mesh output from the basic mesh decoding unit 303 to generate divided meshes.

[0123] Fig. 9 (a) shows a part of the basic mesh (triangle), which is composed of vertices v1, v2, and v3. v1, v2, and v3 are three-dimensional vectors. The mesh segmentation unit 3071 generates a segmented mesh by adding new vertices v12, v13, and v23 in the middle of each side of the triangle and outputs ( Fig. 9 (b)). v12=(v1+v2) / 2 v13=(v1+v3) / 2 v23=(v2+v3) / 2 In addition, it may be the following. v12=(v1+v2+1)>>1 v13=(v1+v3+1)>>1 v23=(v2+v3+1)>>1 The mesh deformation unit 3072 generates a deformed mesh by inputting the segmented mesh and mesh displacement, adding the mesh displacements d12, d13, and d23, and outputs the deformed mesh ( Fig. 9 (c)). The mesh displacement is the output of the mesh displacement decoding unit 305 (coordinate system conversion unit 3055). d12, d13, d23 are the mesh displacements corresponding to the vertices v12, v13, v23 added by the mesh segmentation unit 3071. v12'=v12+d12 v13'=v13+d13 v23'=v23+d23 It should be noted that d12=disp[0][], d13=disp[1][], and d23=disp[3][]. (Configuration of 3D data encoding device according to first embodiment)

[0124] Fig.101 is a functional block diagram showing a schematic structure of a 3D data encoding device 11 according to the first embodiment. The 3D data encoding device 11 is composed of a graph information encoding unit 101, a basic grid encoding unit 103, a basic grid decoding unit 104, a grid displacement updating unit 106, a grid displacement encoding unit 107, a grid displacement decoding unit 108, a grid reconstruction unit 109, an attribute updating unit 110, a filling unit 111, a color space conversion unit 112, an attribute encoding unit 113, a multiplexing unit 114, and a grid separation unit 115. The 3D data encoding device 11 inputs graph information, a basic grid, a grid displacement, a grid, and an attribute image as 3D data, and outputs encoded data.

[0125] The graph information encoding unit 101 encodes the graph information and outputs a graph information encoding stream.

[0126] The basic grid coding unit 103 codes the basic grid and outputs a basic grid coding stream. As a coding method, Draco or the like is used.

[0127] The basic grid decoding unit 104 is the same as the basic grid decoding unit 303 , and therefore the description thereof is omitted.

[0128] The grid displacement updating unit 106 adjusts the grid displacement based on the (original) basic grid and the decoded basic grid, and outputs the updated grid displacement.

[0129] The grid displacement encoding unit 107 encodes the updated grid displacement and outputs a grid displacement encoding stream.

[0130] The grid displacement decoding unit 108 is the same as the grid displacement decoding unit 305, so the description is omitted.

[0131] The mesh reconfiguration unit 109 is the same as the mesh reconfiguration unit 307 , and thus the description thereof is omitted.

[0132] The attribute updating unit 110 inputs the (original) mesh, the reconstructed mesh output from the mesh reconstructing unit 109 (mesh deformation unit 3072), and the attribute image, updates the attribute image to match the position (coordinates) of the reconstructed mesh, and outputs the updated attribute image.

[0133] The filling unit 111 inputs an attribute image and performs filling processing on a region where pixel values ​​are empty.

[0134] The color space conversion unit 112 performs color space conversion from the RGB format to the YCbCr format.

[0135] The attribute encoding unit 113 encodes the attribute image in the YCbCr format output from the color space conversion unit 112, and outputs an attribute moving image stream. As the encoding method, VVC, HEVC, etc. are used.

[0136] The multiplexing unit 114 multiplexes the atlas information coded stream, the basic grid coded stream, the grid displacement coded stream, and the attribute animation stream, and outputs them as coded data. As a multiplexing method, a byte stream format, ISOBMFF, etc. are used. (Operation of the grid separation unit)

[0137] The mesh separation unit 115 generates a basic mesh and a mesh displacement from the mesh.

[0138] Fig.13 2 is a functional block diagram showing the structure of the mesh separation unit 115. The mesh separation unit 115 is composed of a mesh thinning unit 1151, a mesh dividing unit 1152, and a mesh displacement deriving unit 1153.

[0139] The mesh thinning unit 1151 generates a basic mesh by thinning out some vertices from the mesh.

[0140] Fig.14 (a) shows a part of a mesh, and the mesh is composed of vertices v1, v2, v3, v4, v5, and v6. v1, v2, v3, v4, v5, and v6 are three-dimensional vectors. The mesh thinning unit 1151 generates a basic mesh by thinning out vertices v4, v5, and v6 and outputs ( Fig.14 (b)).

[0141] The mesh division unit 1152 divides the basic mesh to generate divided meshes ( Fig.14 (c)). v4'=(v1+v2) / 2 v5'=(v1+v3) / 2 v6'=(v2+v3) / 2 The mesh displacement deriving unit derives the displacements d4, d5, d6 of vertices v4, v5, v6 relative to vertices v4', v5', v6' as mesh displacements based on the mesh and the segmented mesh, and outputs ( Fig.14 (d)). d4=v4-v4' d5=v5-v5' d6=v6–v6 (Code of the basic grid)

[0142] Fig.111 is a functional block diagram showing the structure of the basic grid coding unit 103. The basic grid coding unit 103 is composed of a grid coding unit 1031, a grid decoding unit 1032, a motion information coding unit 1033, a motion information decoding unit 1034, a grid motion compensation unit 1035, a reference grid memory 1036, a switch 1037, and a switch 1038. The basic grid coding unit 103 may also include a basic grid quantization unit (not shown) after the input of the basic grid. When the basic grid is encoded (intra-frame encoding) without referring to other basic grids (for example, an already encoded basic grid), the switch 1037 and the switch 1038 are connected to the side where motion compensation is not performed. Otherwise, when the basic grid is encoded (inter-frame encoding) with reference to other basic grids, they are connected to the side where motion compensation is performed.

[0143] The grid coding unit 1031 has an intra-frame coding function, performs intra-frame coding on the basic grid, and outputs a basic grid coding stream. As a coding method, Draco or the like is used.

[0144] The trellis decoding unit 1032 is the same as the trellis decoding unit 3031, and therefore the description thereof is omitted.

[0145] The motion information encoding unit 1033 has an inter-frame encoding function, performs inter-frame encoding on the basic grid, and outputs a basic grid encoding stream. Entropy encoding such as arithmetic coding is used as the encoding method.

[0146] The motion information decoding unit 1034 is the same as the motion information decoding unit 3032 , and thus the description thereof is omitted.

[0147] The mesh motion compensation unit 1035 is the same as the mesh motion compensation unit 3033 , and thus the description thereof is omitted.

[0148] The reference grid memory 1036 is the same as the reference grid memory 3034, so the description thereof is omitted. (Encoding of grid displacement)

[0149] Fig.12 107 is a functional block diagram showing the structure of the grid displacement coding unit 107. The grid displacement coding unit 107 is composed of a coordinate system conversion unit 1071, a conversion unit 1072, a quantization unit 1073, a binarization unit 1074, an arithmetic coding unit 1075, a context selection unit 1076, and a context initialization unit 1077.

[0150] The coordinate system conversion unit 1071 converts the coordinate system of the grid displacement from the Cartesian coordinate system to the coordinate system (e.g., the local coordinate system) encoding the displacement based on the value of the coordinate system conversion information displacementCoordinateSystem. Here, disp is a three-dimensional vector representing the grid displacement before the coordinate system conversion, d is a three-dimensional vector representing the grid displacement after the coordinate system conversion, and n_vec, t_vec, and b_vec are three-dimensional vectors (of the Cartesian coordinate system) representing the axes of the local coordinate system. The grid displacement encoder 107 may update the value of displacementCoordinateSystem at a sequence level, or at a picture / frame level. The initial value is 0, which represents a Cartesian coordinate system.

[0151] To update the displacementCoordinateSystem at the sequence level, use Figure 7 The syntax of the structure is: In the case of the Cartesian coordinate system, asps_vdmc_ext_displacement_coordinate_system is set to 0, and in the case of the local coordinate system, asps_vdmc_ext_displacement_coordinate_system is set to 1.

[0152] To change the displacementCoordinateSystem at the picture / frame level, use Figure 8 When updating the coordinate system, afps_vdmc_ext_displacement_coordinate_system_enable_flag is set to 1, and when not updating the coordinate system, afps_vdmc_ext_displacement_coordinate_system_enable_flag is set to 0. In the case of a Cartesian coordinate system, afps_vdmc_ext_displacement_coordinate_system is set to 0, and in the case of a local coordinate system, afps_vdmc_ext_displacement_coordinate_system is set to 1.

[0153] The conversion unit 1072 performs conversion f (for example, wavelet conversion) and derives the grid displacement Tdisp after the conversion. Tdisp[0][]=f(d[0][]) Tdisp[1][]=f(d[1][]) Tdisp[2][]=f(d[2][]) The quantization unit 1073 performs quantization based on the quantization scale value scale derived from the quantization parameter of each component of the grid displacement, and derives the quantized grid displacement Qdisp. Qdisp[0][]=Tdisp[0][] / scale[0] Qdisp[1][]=Tdisp[1][] / scale[1] Qdisp[2][]=Tdisp[2][] / scale[2] Alternatively, the scale value can be multiplied by a power of 2 to approximate Qdisp using the following formula. scale[i]=1< <scale2[i] Qdisp[0][]=Tdisp[0][]>>scale2[0] Qdisp[1][]=Tdisp[1][]>>scale2[1] Qdisp[2][]=Tdisp[2][]>>scale2[2] The binarization unit 1074 encodes the quantized grid displacement Qdisp, which is a multi-value signal, into a binary signal. The binary signal may be a k-order exponential Grunt code.

[0154] The arithmetic coding unit 1075 performs arithmetic coding on the binary signal and outputs a grid displacement code stream.

[0155] Context selection unit 1076 is the same as context selection unit 3056 , and therefore description thereof is omitted.

[0156] It should be noted that a static context with a fixed probability of not updating the context is set as ctxStatic. The encoding of the syntax element indicated by ctxStatic can be performed without using the context. encode(ctxStatic) can use a bypass-specific process as encode_bypass().

[0157] The context initialization unit 1077 is the same as the context initialization unit 3057, and therefore the description thereof is omitted. Here, an example using context is described, but it is also possible to bypass-code some syntax elements without using context. In the configuration of bypass coding, the effect of reducing the memory and processing amount of context is achieved. For example, the syntax elements diu_last_sig_coeff, diu_coded_block_flag, and diu_coeff_abs_level_rem may not be used but may be bypass-coded. By bypass-coding these syntax elements, the coding efficiency is maintained while reducing the memory and processing amount of the context.

[0158] The grid displacement encoding unit 107 encodes the grid displacement Qdisp through the following processing. The word continue in the pseudocode means to skip the subsequent actions and jump to the beginning of the loop (the next iteration). Here, encode() and encodeExpGolomb() are functions that take the value and the corresponding context as independent variables, and perform arithmetic encoding on the 1-bit value and the k-times Golomb-coded binary string respectively. dispCount[b] is the number of grid displacements at detail level b. lastSig is a flag indicating whether the current coefficient is the last non-zero coefficient in the sub-block in the scan order. lastSig=0 indicates that the current coefficient is not the last non-zero coefficient in the sub-block in the scan order. lastSig=1 indicates that the current coefficient is the last non-zero coefficient in the sub-block in the scan order. (Code of the basic grid)

[0159] Fig.19 1031 is a functional block diagram showing the structure of the trellis coding unit 1031. The trellis coding unit 1031 is composed of a trellis prediction unit 10311, a binarization unit 1074, an arithmetic coding unit 1075, a context selection unit 1076, and a context initialization unit 1077.

[0160] The grid prediction unit 10311 is the same as the grid prediction unit 30311, so the description is omitted.

[0161] The mesh encoding unit 1031 encodes the vertex position vector prediction residual BmVertexPosPredResidual (=BmVertexPos−BmVertexPosPred) of the basic mesh through the following processing. As mentioned above, one embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to the above embodiment, and various design changes and the like can be made within the scope not departing from the gist of the present invention. [Application Examples]

[0162] The 3D data encoding device 11 and the 3D data decoding device 31 can be mounted on various devices for sending, receiving, recording, and reproducing 3D data. It should be noted that the 3D data can be natural 3D data captured by a camera or the like, or artificial 3D data generated by a computer or the like (including CG (Computer Graphics: computer animation) and GUI (Graphical User Interface: graphical user interface)).

[0163] The embodiments of the present invention are not limited to the above-mentioned embodiments, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical solutions appropriately modified within the scope of the claims are also included in the technical scope of the present invention. Industrial Applicability

[0164] The embodiments of the present invention can be preferably applied to a 3D data decoding device that decodes coded data obtained by encoding 3D data and a 3D data encoding device that generates coded data obtained by encoding 3D data. In addition, the embodiments of the present invention can be preferably applied to the data structure of the coded data generated by the 3D data encoding device and referred to by the 3D data decoding device. Description of Reference Numerals 11: 3D data encoding device 101: Spectral Information Coding Department 103: Basic grid coding unit 1031: Grid Coding Department 10311: Grid Prediction Department 1032: Grid decoding unit 1033: Motion information encoding unit 1034: Motion information decoding unit 1035: Mesh Motion Compensation Unit 1036: Reference grid storage 1037: Switch 1038: Switch 104: Basic grid decoding unit 106: Grid displacement update section 107: Grid displacement coding unit 1071: Coordinate system conversion unit 1072: Conversion Department 1073: Quantitative Department 1074: Binarization 1075: Arithmetic Coding Unit 1076: Context selection unit 1077: Context initialization section 108: Grid displacement decoding unit 109: Grid Reconstruction Department 110: Attribute Update Department 111: Filling 112: Color space conversion unit 113: Attribute coding unit 114: Multiplexing Department 115: Grid separation unit 1151: Grid spacing elimination section 1152: Grid segmentation 1153: Grid displacement export section 21: Network 31: 3D data decoding device 301: Demultiplexing unit 302: Spectrum information decoding unit 303: Basic grid decoding unit 3031: Grid decoding unit 30311: Grid Prediction Department 3032: Motion information decoding unit 3033: Mesh Motion Compensation Department 3034: Reference grid storage 3035: Switch 3036: Switch 305: Grid displacement decoding unit 3051: Arithmetic decoding unit 3052: Multi-valued Department 3053: Inverse Quantization Department 3054: Reverse conversion unit 3055: Coordinate system conversion unit 3056: Context selection unit 3057: Context initialization section 307: Grid Reconstruction Department 306: Attribute decoding unit 3071: Mesh segmentation department 3072: Mesh Deformation Department 308: Color space conversion unit 41: 3D data display device

Claims

1. A 3D data decoding device for decoding coded data, wherein the 3D data decoding device comprises: a mesh prediction unit that derives predicted positions of basic mesh vertex positions from the coded data; and an arithmetic decoding unit that performs arithmetic decoding on the prediction residual. The arithmetic decoding unit decodes a portion of the beginning of a prefix of an absolute value of a coefficient of the prediction residual using a context, adds the prediction position to the prediction residual, and derives the basic mesh vertex position.

2. The 3D data decoding device according to claim 1, characterized in that: The arithmetic decoding unit counts the number of bins of the prefix decoded using the context for each specified number of vertices, and when the number of bins is greater than a specified value, decodes the bin of the prefix by switching from decoding using the context to decoding without using the context.

3. A 3D data encoding device for encoding 3D data, the 3D data encoding device comprising: a mesh prediction unit for deriving predicted positions of basic mesh vertex positions; and an arithmetic coding unit for performing arithmetic coding on the prediction residual. The arithmetic coding unit encodes a portion at the beginning of a prefix of an absolute value of a coefficient of the prediction residual using a context.

4. The 3D data encoding device according to claim 3, characterized in that: The arithmetic coding unit counts the number of bins of the prefix encoded using context for each specified number of vertices, and when the number of bins is greater than a specified value, switches the encoding of the prefix bin from encoding using context to encoding without context.

Images