3D data decoding device and 3D data coding device

Abstract

To improve a coding efficiency of a mesh deviation to code and decode a 3D data to a high quality in a coding and decoding of the 3D data using an arithmetic coding system.SOLUTION: In a 3D data decoding device that decodes a coding data, an arithmetic decoding part that performs an arithmetic decoding of a mesh deviation from the coding data. The arithmetic decoding part decodes a first flag indicating whether or not an absolute value of a coefficient of the mesh deviation is larger than 0, a second flag indicating whether or not the absolute value of the coefficient is larger than 1, a third flag indicating whether or not the absolute value of the coefficient is larger than 2 from the coding data.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a 3D data encoding device and a 3D data decoding device. [Background technology]

[0002] To efficiently transmit or record 3D data, the 3D data is converted into 2D images and video There are 3D data encoding devices that encode data using an image encoding method to generate encoded data, and 3D data decoding devices that decode the encoded data into a 2D image and reconstruct 3D data.

[0003] Specific examples of 3D data encoding methods include MPEG-I's ISO / IEC 23090-5 V3C (Volumetric Video-based Coding) and V-PCC (Video-based Point Cloud Compression). V3C encodes and decodes point clouds that consist of point positions and attribute information. Furthermore, ISO / IEC 23090-12 (MPEG Immersive Video, MIV) and ISO / IEC 23090-29 (Video-based Dynamic Mesh Coding, V-DMC), which are currently being standardized, are enabling the encoding of multi-viewpoint video, It is also used for encoding and decoding mesh images. A soft document has been disclosed.

[0004] In these 3D data coding methods, the geometry and attributes that make up the 3D data are encoded and decoded as images using video coding methods such as H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding).

[0005] In the case of a point cloud, the geometry image is the depth to the projection plane, and the attribute image is the image of the attributes projected onto the projection plane.

[0006] 3D data (mesh) as in Non-Patent Document 1 consists of a base mesh, mesh displacement, and texture mapping image. The encoding of the base mesh is done using vertex coding such as Draco. The mesh displacement is encoded by a two-dimensional mesh In addition to encoding the displacement image using a video codec, there is also a method of encoding it directly using arithmetic coding. The texture mapping image is encoded as an attribute image using a video codec. The video codec can be the above-mentioned HEVC or VVC. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] WD 3.0 of V-DMC (MDS22775_WG07_N00611_d9), ISO / IEC JTC 1 / SC 29 / WG 7 N0611, April 2023 Summary of the Invention [Problem to be solved by the invention]

[0008] In the 3D data coding method in Non-Patent Document 1, mesh displacements (mesh displacement arrays, mesh displacement images) that make up 3D data (meshes) can be coded and decoded using arithmetic coding. When mesh displacements are arithmetically coded, the performance depends on syntax and context, which results in poor coding efficiency.

[0009] The present invention aims to improve the coding efficiency of mesh displacement in the coding and decoding of 3D data using arithmetic coding, thereby enabling high-quality coding and decoding of 3D data. [Means for solving the problem]

[0010] In order to solve the above problem, a 3D data decoding device according to one aspect of the present invention is provided. a 3D data decoding device for decoding 3D data, the device comprising an arithmetic decoding unit for arithmetically decoding mesh displacements from the encoded data, the arithmetic decoding unit performing arithmetic decoding on mesh displacement coefficients whose absolute values ​​are greater than 0; a first flag indicating whether the absolute value of the coefficient is greater than 1; and a second flag indicating whether the absolute value of the coefficient is greater than 1. The method is characterized in that a lag and a third flag indicating whether the absolute value of the coefficient is greater than 2 are decoded from the coded data.

[0011] In order to solve the above problem, a 3D data encoding device according to one aspect of the present invention is a 3D data encoding device that encodes 3D data, and includes an arithmetic encoding unit that arithmetically encodes mesh displacements, and the arithmetic encoding unit determines whether an absolute value of a coefficient of the mesh displacement is greater than 0. a first flag indicating whether the absolute value of the coefficient is greater than 1; A third flag indicating whether the absolute value of the coefficient is greater than 2 is coded into the coded data. [Effects of the Invention]

[0012] According to one aspect of the present invention, the encoding efficiency of mesh displacement can be improved, and 3D data can be encoded and decoded with high quality. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic diagram showing the configuration of a 3D data transmission system according to the present embodiment. [Figure 2] FIG. 2 is a diagram showing a hierarchical structure of data in an encoded stream. [Figure 3] FIG. 2 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31. [Figure 4] FIG. 2 is a functional block diagram showing the configuration of a base mesh decoding unit 303. [Figure 5] FIG. 10 is a functional block diagram showing the configuration of a mesh displacement decoding unit 305. [Figure 6] FIG. 2 is a functional block diagram showing the configuration of a mesh reconstruction unit 307. [Figure 7] 10 is an example of a syntax for transmitting coordinate transformation parameters and context initialization parameters at the sequence level (ASPS). [Figure 8] 10 is an example of a syntax for transmitting coordinate transformation parameters and context initialization parameters at the picture / frame level (AFPS). [Figure 9] FIG. 10 is a diagram for explaining the operation of the mesh reconstruction unit 307. [Figure 10] 1 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11. FIG. [Figure 11] FIG. 2 is a functional block diagram showing the configuration of a base mesh encoding unit 103. [Figure 12] FIG. 2 is a functional block diagram showing the configuration of a mesh displacement encoding unit 107. [Figure 13] FIG. 2 is a functional block diagram showing the configuration of a mesh separation unit 115. [Figure 14] 10 is a diagram for explaining the operation of the mesh separating unit 115. FIG. [Figure 15] 1 is an example of a syntax structure for mesh displacement. [Figure 16] FIG. 10 is a diagram showing the syntax structure of mesh displacement and a method for deriving a context. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a schematic diagram showing the configuration of a 3D data transmission system 1 according to this embodiment.

[0016] The 3D data transmission system 1 is a system that transmits an encoded stream obtained by encoding 3D data to be encoded, decodes the transmitted encoded stream, and displays the 3D data. The 3D data transmission system 1 includes a 3D data encoding device 11, a network 21, a 3D data decoding device 31, and a 3D data display device 41.

[0017] The 3D data T is input to the 3D data encoding device 11.

[0018] The network 21 transmits the coded stream Te generated by the 3D data coding device 11 to the 3D data decoding device 31. The network 21 may be the Internet, a wide area network (WAN), a local area network (LAN), or The network 21 is not necessarily limited to a two-way communication network, but may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. The network 21 may also be a network that transmits DVD (Digital Versatile Disc: registered trademark), BD (Blu-ray Disc: registered trademark), etc. It may be replaced by a storage medium on which the coded stream Te, such as a standard (or a DVD-ROM drive), is recorded.

[0019] The 3D data decoding device 31 decodes each of the coded streams Te transmitted by the network 21, and generates one or more decoded 3D data Td.

[0020] The 3D data display device 41 displays all or part of one or more pieces of decoded 3D data Td generated by the 3D data decoding device 31. The 3D data display device 41 is equipped with a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. The display may be in the form of a stationary display, a mobile display, an HMD, or the like. When the device has a high processing power, it displays a high quality image, and when the device has a lower processing power, it displays an image that does not require a high processing power or display power.

[0021] <operator> The operators used in this specification are listed below.

[0022] >> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR , |= is the OR assignment operator, and || indicates logical sum.

[0023] x?y:z is a ternary operator that takes y if x is true (non-zero) and z if x is false (zero). y..z denotes the set of integers from y to z.

[0024] <Structure of the coded stream Te> Before proceeding to a detailed description of the 3D data encoding device 11 and the 3D data decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the 3D data encoding device 11 and decoded by the 3D data decoding device 31 will be described.

[0025] FIG. 2 shows the hierarchical structure of data in the coded stream Te. The frame Te has either a V3C sample stream or a V3C unit stream data structure. A V3C sample stream includes a sample stream header and a V3C unit. A V3C unit stream includes a V3C unit.

[0026] A V3C unit includes a V3C unit header and a V3C unit payload. The V3C unit header is a Unit Type, which is an ID that indicates the type of V3C unit, and can be V3C_VPS, V3C_AD, V3C_AVD, V3C_GVD, V3C_OVD, etc. It takes the value indicated by the label.

[0027] If the Unit Type is V3C_VPS (Video Parameter Set), the V3C unit contains a V3C parameter set.

[0028] If the Unit Type is V3C_AD (Atlas Data), the V3C unit includes a VPS ID, an atlasID, a sample stream nal header, and multiple NAL units. The atlasID is an ID (Identification). Yes, takes an integer value of 0 or greater.

[0029] A NAL unit includes a NALUnitType, a layerID, a TemporalID, and an RBSP (Raw byte sequence payload).

[0030] NAL units are identified by NALUnitType, and are classified as ASPS (Atlas Sequence Parameter Set), AAPS (Atlas Adaptation Parameter Set), ATL (Atlas Tile layer), SEI (Supplemental Enhancement Information), etc.

[0031] The ATL includes an ATL header and an ATL data unit, and the ATL data unit includes information such as the patch position and size, such as patch information data.

[0032] The SEI includes payloadType, which indicates the type of the SEI, payloadSize, which indicates the size (number of bytes) of the SEI, and sei_payload, which is the data of the SEI.

[0033] If the Unit Type is V3C_AVD (Attribute Video Data), The unit includes the VPS ID, atlasID, attribute image ID attrIdx, partition ID partIdx, map ID mapIdx, flag auxFlag indicating whether it is auxiliary data or not, and video stream. The video stream is data encoded using HEVC, VVC, etc. Attribute data corresponds to a texture image in V-DMC.

[0034] If the Unit Type is V3C_GVD (Geometry Video Data), the V3C unit includes the VPS ID, atlasID, mapIdx, auxFlag, and video stream. Geometry data corresponds to mesh displacement in V-DMC.

[0035] If the Unit Type is V3C_OVD (Occupancy Video Data), the V3C unit includes a VPS ID, an atlas ID, and a video stream.

[0036] If the Unit Type is V3C_MD (Mesh data), the V3C unit contains the VPS ID, atlas ID, and mesh_payload. In V-DMC, it corresponds to the base mesh.

[0037] (Configuration of 3D data decoding device according to the first embodiment) 3 is a functional block diagram showing a schematic configuration of a 3D data decoding device 31 according to the first embodiment. The 3D data decoding device 31 is composed of a demultiplexing unit 301, an atlas information decoding unit 302, a base mesh decoding unit 303, a mesh displacement decoding unit 305, a mesh reconstruction unit 307, an attribute decoding unit 306, and a color space conversion unit 308. The 3D data decoding device 31 receives encoded 3D data as input. Then, output the atlas information, mesh, and attribute images.

[0038] The demultiplexing unit 301 receives encoded data multiplexed in a byte stream format, ISOBMFF (ISO Base Media File Format), or the like, and demultiplexes it to generate an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a base mesh encoded stream (V3C_MD mesh_payload), a mesh displacement encoded stream (V3C_GVD video stream), an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a mesh displacement encoded stream (V3C_MD video stream), an atlas information encoded stream (V3C_AD Atlas Data stream, NAL unit), a mesh displacement encoded stream (V3C_MD video stream), an atlas information encoded stream (V3C_MD ... Outputs a live video stream (V3C_AVD video stream).

[0039] The atlas information decoding unit 302 receives the atlas information coded stream output from the demultiplexing unit 301 and decodes the atlas information.

[0040] 3 decodes, from the encoded data, coordinate system transformation information displacementCoordinateSystem (asps_vdmc_ext_displacement_coordinate_system, afps_vdmc_ext_displacement_coordinate_system) indicating the coordinate system. Note that a separate gating flag may be provided, and each piece of coordinate system transformation information may be decoded only when the gating flag is 1. The gating flag may be, for example, afps_vdmc_ext_displacement_coordinate_system_enable_flag.

[0041] The base mesh decoding unit 303 decodes the base mesh coded stream coded by vertex coding (3D data compression coding method, for example, Draco), and outputs a base mesh. The base mesh will be described later.

[0042] The mesh displacement decoding unit 305 decodes the mesh displacement coded stream and obtains the mesh displacement. Output.

[0043] The mesh reconstruction unit 307 receives the base mesh and the mesh displacement and reconstructs the mesh in 3D space. Reconstruct the cache.

[0044] The attribute decoding unit 306 decodes an attribute video stream coded by VVC, HEVC, or the like, and outputs an attribute image. The attribute image is a texture image expanded on the UV axis (a texture mapping image converted by the UV atlas method) in YCbCr frame. The type of codec used for encoding is indicated by ptl_profile_codec_group_idc, which is obtained by decoding the V3C parameter set of the encoded data. It may also be indicated by the Four CC code indicated by ai_geometry_codec_id[atlasID] in the V3C parameter set. ai_geometry_codec_id[atlasID] indicates the index corresponding to the codec ID of the decoder used to decode the attribute video stream in the atlas ID.

[0045] A color space conversion unit 308 converts the attribute image from the YCbCr format to the RGB format. Color space conversion is performed. Note that the attribute video stream is encoded as RGB format. It is also possible to decode the image data and omit the color space conversion.

[0046] (Decoding the base mesh) 4 is a functional block diagram showing the configuration of the base mesh decoding unit 303. The base mesh decoding unit 303 includes a mesh decoding unit 3031, a motion information decoding unit 3032, a mesh motion compensation unit 3033, a reference The base mesh decoding unit 303 includes a base mesh inverse quantization unit (not shown) before the output of the base mesh. The switches 3035 and 3036 may have a configuration as follows: When the base mesh to be decoded is coded (intra-coded) without reference to other base meshes (for example, base meshes that have already been coded and decoded), the switches 3035 and 3036 are connected to a side that does not perform motion compensation. On the other hand, when the base mesh to be decoded is coded (inter-coded) with reference to other base meshes, the switches 3035 and 3036 are connected to a side that performs motion compensation. When motion compensation is performed, the target vertex coordinates are derived by referring to the vertex coordinates and motion information that have already been decoded.

[0047] The mesh decoding unit 3031 decodes the intra-coded base mesh coded stream and outputs the base mesh. The coding method used is Draco, Edge Breaker, etc. There are.

[0048] The motion information decoding unit 3032 decodes the inter-coded base mesh coded stream and outputs motion information for each vertex of a reference mesh (to be described later). Entropy coding such as arithmetic coding is used as the coding method.

[0049] The mesh motion compensation unit 3033 performs motion compensation on each vertex of the reference mesh input from the reference mesh memory 3034 based on the motion information, and outputs a motion-compensated mesh.

[0050] The reference mesh memory 3034 is a memory that holds the decoded mesh for reference in subsequent decoding processes.

[0051] (Mesh displacement decoding) 5 is a functional block diagram showing the configuration of the mesh displacement decoding unit 305. The mesh displacement decoding unit 305 is made up of a CABAC decoding unit (arithmetic decoding unit 3051, multi-value conversion unit 3052, context selection unit 3056, context initialization unit 3057), an inverse quantization unit 3053, an inverse transformation unit 3054, and a coordinate system transformation unit 3055.

[0052] (Context-adaptive binary arithmetic coding) The arithmetic decoding unit 3051, the multi-value conversion unit 3052, the context selection unit 3056, and the context initialization unit 3057 use a context called Context-Adaptive Binary Arithmetic Coding (CABAC). CABAC uses a decoding method that was previously used. In CABAC, a binary string consisting of 0s and 1s is coded and decoded bit by bit using a state variable called a context (CABAC state). All CABAC states are initialized at the beginning of the segment. The CABAC decoding unit decodes each bit of the binary string (Bin String) corresponding to the syntax element. When a context is used, a context index ctxInc is derived for each bit of the syntax element, the bit is decoded using the context, and the CABAC state of the context is updated. Bits that do not use a context are coded with equal probability. The context is decoded at the rate (EP, bypass), and the index ctxIdx specifying the context and the update of the specified context are omitted. The context is the probability (state) of CABAC. It is a variable (memory area) to hold the data, and is identified by the value of ctxIdx (0, 1, 2, …). Also, the case where 0 and 1 are always equally probable, that is, 0.5, 0.5, is called EP (Equal Probability) or is called bypass. In this case, there is no need to maintain a state for a specific syntax element, so no context is used. Also, the probability is fixed at 0.5, so it is a static state that does not need to be updated. A context may be used. In this sense, it may be called static rather than bypass. An integer value such as 128 may be used to indicate a probability of 0.5.

[0053] The process of decoding one bit without using the context (bypassing) is as follows: A code may also be used. rangeTimesProb = IvlRange >> 1 binVal = ( rangeTimesProb <= ( IvlCode - IvlLow ) ) if (binVal == 0) IvlRange = rangeTimesProb else { IvlLow += rangeTimesProb IvlRange -= rangeTimesProb } The process of decoding one bit using the context may be performed using the following pseudo code: Here, prob0 is a variable that indicates the probability of the context. rangeTimesProb = IvlRange * prob0 >> 16 binVal = ( rangeTimesProb <= ( IvlCode - IvlLow ) ) if (binVal == 0) IvlRange = rangeTimesProb else { IvlLow += rangeTimesProb IvlRange -= rangeTimesProb } (Coordinate system) The following two types of coordinate systems are used for mesh displacement (3D vector). Cartesian coordinate system (canonical): A rectangular coordinate system commonly defined throughout the entire 3D space. (X, Y, Z) coordinate system. A rectangular coordinate system whose direction does not change at the same time (within the same frame, within the same tile). Local coordinate system: A Cartesian coordinate system defined per region or per vertex in 3D space. A Cartesian coordinate system whose direction can change at the same time (within the same frame, within the same tile). A coordinate system with normal (D), tangent (U), and bi-tangent (V) axes. In other words, a certain vertex (a certain vertex) The first axis (D) is indicated by the normal vector n_vec on the surface containing the normal vector n_vec, and the second axis (U) and third axis (V) are indicated by two tangent vectors t_vec and b_vec that are perpendicular to the normal vector n_vec. n_vec, t_vec, b_vec are three-dimensional vectors. (D, U, V) coordinates The reference system may also be called the (n, t, b) coordinate system.

[0054] (Decoding and derivation of sequence-level control parameters) Here, the control parameters used in the mesh displacement decoding unit 305 will be explained.

[0055] Figure 7 shows the syntax of the configuration for transmitting coordinate system transformation parameters in sequence-level ASPS. ASPS (Atlas Sequence Parameter Set or Atlas sequence mesh information) is one of the NAL units of atlas information, and is included in the atlas information coded stream. Contains syntax elements to apply. In ASPS, this is done using the asps_vdmc_extension() syntax. The semantics of each field are as follows:

[0056] asps_vdmc_ext_subdivision_iteration_count: Parameter indicating the number of mesh subdivision iterations It is data.

[0057] asps_vdmc_ext_displacement_coordinate_system: Coordinate system transformation information indicating the coordinate system of the mesh displacement. If the value is equal to a given first value (e.g., 0), it indicates a Cartesian coordinate system. If the value is equal to another second value (e.g., 1), it indicates a local coordinate system.

[0058] asps_vdmc_ext_1d_displacement_flag: Flag indicating whether the mesh displacement is 1-dimensional or not A value of true indicates that the mesh displacement is one-dimensional. A value of false indicates that the mesh displacement is three-dimensional.

[0059] (Decoding and derivation of picture / frame level control parameters) Figure 8 shows a system configuration in which coordinate system transformation parameters are transmitted using AFPS at the picture / frame level. AFPS (Atlas Frame Parameter Set or Atlas frame mesh information) is one of the NAL units of atlas information and is included in the atlas information coded stream. Contains syntax elements to apply. In AFPS, the afps_vdmc_extension() syntax The semantics of each field are as follows:

[0060] afps_vdmc_ext_overriden_flag: A flag indicating whether to update the mesh displacement coordinate system. If this flag is set to true, the mesh displacement coordinate system is updated based on the value of afps_vdmc_ext_displacement_coordinate_system (described later). If this flag is set to false, The coordinate system of the mesh displacement is not updated.

[0061] afps_vdmc_ext_subdivision_iteration_count: Parameter indicating the number of mesh subdivision iterations It is data.

[0062] afps_vdmc_ext_displacement_coordinate_system: Coordinate system transformation information indicating the coordinate system of the mesh displacement. If the value is equal to the first value (e.g., 0), it indicates the Cartesian coordinate system. If the value is equal to the second value If it is equal to (e.g. 1), it indicates a local coordinate system. If the syntax element is not present, The value is assumed to be the value decoded by ASPS, and the default coordinate system is the coordinate system indicated by ASPS.

[0063] afps_vdmc_ext_1d_displacement_flag: Flag indicating whether the mesh displacement is 1-dimensional A value of true indicates that the mesh displacement is one-dimensional. A value of false indicates that the mesh displacement is one-dimensional. Indicates that the mesh displacement is three-dimensional.

[0064] (Mesh displacement syntax structure) Figure 15 is an example of the syntax structure of mesh displacement. The semantics are as follows: Mesh displacement is a sequence of values ​​(coefficients) of position pos and k components (k components), and is represented by the array Qdisp[pos][k]. Displacement is a three-dimensional signal in the Cartesian coordinate system (xyz) or the local coordinate system (ntb), and each component of the three-dimensional displacement is called a component. The displacement Qdisp here is , which is the value after transformation by discrete wavelet transform, lifting transform, DCT transform, etc. Therefore, it is also called a coefficient. The component variable k takes on the values ​​0, 1, and 2. The variable name is not limited to k, and dim can be any other variable name. The order of the QDisp index can be reversed, i.e., Qdisp[k][pos] can be used instead of Qdisp[pos][k].

[0065] diu_last_sig_coeff[k]: An index indicating the last position of a non-zero mesh displacement coefficient in the kth component.

[0066] diu_coded_block_flag[k][b]: indicates whether the block with index b in the kth component contains a non-zero mesh displacement coefficient: 1 if yes, 0 if not.

[0067] diu_coded_subblock_flag[k][b][s]: indicates whether the subblock with index s of the block with index b in the kth component contains a non-zero mesh displacement coefficient. , otherwise 0.

[0068] diu_coeff_abs_level_gt0[k][b][s][v]: For the k component, the block with index b Indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex with index v in the subblock with index s is greater than 0. 1 if it is greater, 0 if it is not.

[0069] diu_coeff_abs_level_gt1[k][b][s][v]: For the k component, the block with index b Indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex with index v in the subblock with index s is greater than 1. If it is greater, it is set to 1; otherwise, it is set to 0. If not present, it is assumed to be 0.

[0070] diu_coeff_abs_level_gt2[k][b][s][v]: For the k component, the block with index b Indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex with index v in the subblock with index s is greater than 2. If it is greater, it is set to 1; otherwise, it is set to 0. If not present, it is assumed to be 0.

[0071] diu_coeff_abs_level_gt3[k][b][s][v]: For the k component, the block with index b Indicates whether the absolute value of the non-zero mesh displacement coefficient of the vertex with index v in the subblock with index s is greater than 3. If it is greater, it is set to 1; otherwise, it is set to 0. If not present, it is assumed to be 0.

[0072] diu_coeff_sign[k][b][s][v]: Indicates whether the non-zero mesh displacement coefficient of the vertex with index v of the sub-block with index s of the block with index b in the kth component is positive. If it is positive, it is set to 1, otherwise (if it is negative), it is set to 0. If this syntax does not exist, it is assumed to be 1.

[0073] diu_coeff_abs_level_rem[k][b][s][v]: For the k component, the value of the block with index b The absolute value of the non-zero mesh displacement coefficient of the vertex with index v in the subblock with index s minus 4. If this syntax is not present, it is assumed to be 0.

[0074] The mesh displacement decoding unit 305 calculates diu_last_sig_coeff for each component of the mesh displacement. Then, derive the number of lods of the k component, lodCount, from diu_last_sig_coeff[k]. .

[0075] The mesh displacement decoding unit 305 decodes diu_coded_block_flag for each level of detail (lod) of mesh displacement, and derives the number of blocks b, vertexCount, from diu_coded_block_flag[k][b].

[0076] The mesh displacement decoding unit 305 decodes diu_coded_subblock_flag for each block of the mesh displacement, and derives the start position vStart of the subblock s from diu_coded_subblock_flag[k][b][s].

[0077] The mesh displacement decoding unit 305 decodes diu_coeff_abs_level_gt0 for each sub-block of the mesh displacement, and if diu_coeff_abs_level_gt0 is a predetermined value (for example, other than 0), decodes the subsequent diu_coeff_sign and diu_coeff_abs_level_gt1.

[0078] If diu_coeff_abs_level_gt1 is a predetermined value (for example, other than 0), the mesh displacement decoding unit 305 If so, the following diu_coeff_abs_level_gt2 is decoded.

[0079] When diu_coeff_abs_level_gt2 is a predetermined value (for example, other than 0), the mesh displacement decoding unit 305 If so, the following diu_coeff_abs_level_gt3 is decoded.

[0080] If diu_coeff_abs_level_gt3 is a predetermined value (for example, other than 0), the mesh displacement decoding unit 305 If so, the following diu_coeff_abs_level_rem is decoded.

[0081] (Operation of mesh displacement decoding unit) The arithmetic decoding unit 3051 decodes the arithmetically coded mesh displacement coded stream according to a value (context) indicating a random variable, and outputs a binary signal. The binary signal may be an alpha code or a k-th order Exp-Golomb code. Golomb coding consists of prefix and suffix codes. The prefix is ​​an exponentially increasing value, and the suffix is ​​the remainder.

[0082] The multi-value conversion unit 3052 decodes the binary signal into a quantized mesh displacement Qdisp, which is a multi-value signal. do.

[0083] The context selector 3056 (context memory) has a memory for storing contexts, derives a context to be used for arithmetic decoding of mesh displacements according to the state, and updates the value as necessary. In arithmetic decoding of each coefficient of mesh displacements, the following different context arrays may be used depending on the frame type ft (e.g., 0: intraframe, 1: interframe), the mesh division level lod (level of detail), and the component dim of the mesh displacement vector. The context indicates at least the probability of occurrence of a binary signal. Contains variables. ctxLastSigCoeff[numFT][numDim] ctxCodedBlock[numFT][numLOD][numDim] ctxCodedSubBlock[numFT][numLOD][numDim] ctxCoeffGtN[numFT][numLOD][MAX_GTN+1][numDim] ctxCoeffRemPrefix[numFT][numLOD][numDim] A static context with a fixed probability that is not updated is denoted as ctxStatic. The syntax component indicated by ctxStatic may be decoded without using the context.

[0084] Here, numFT is the number of frame types, and numFT=2. numLOD is the maximum number of levels of detail for mesh division, and the value of the syntax element asps_vdmc_ext_subdivision_iteration_count or afps_vdmc_ext_subdivision_iteration_count decoded from the bitstream may be used, or numLOD=4 may be used. numLOD = asps_vdmc_ext_subdivision_iteration_count numLOD = afps_vdmc_ext_subdivision_iteration_count numDim is the number of dimensions of the mesh displacement vector, and may be the value of the syntax element asps_vdmc_ext_1d_displacement_flag or afps_vdmc_ext_1d_displacement_flag decoded from the bitstream, or numDim=3. numDim = asps_vdmc_ext_1d_displacement_flag ? 1 : 3 numDim = afps_vdmc_ext_1d_displacement_flag ? 1 : 3 The maximum value of the threshold for the coefficients, MAX_GTN, is 3.

[0085] ctxLastSigCoeff[numFT][numDim] is a context array used for decoding the syntax element diu_last_sig_coeff. The arithmetic decoding unit 3051 uses the value of ctxLastSigCoeff[ft][dim] to decode the diu_last_sig_coeff for the frame type ft and the dimension dim of the mesh displacement vector. Decrypt the

[0086] ctxCodedBlock[numFT][numLOD][numDim] is a context array used for decoding the syntax element diu_coded_block_flag. The arithmetic decoding unit 3051 uses the values ​​of ctxCodedBlock[ft][lod][dim] to decode the frame type ft, the level of detail lod, and the dimension dim of the mesh displacement vector. Decode the diu_coded_block_flag in

[0087] ctxCodedSubBlock[numFT][numLOD][numDim] is a context array used for decoding the syntax element diu_coded_subblock_flag. The arithmetic decoding unit 3051 uses the values ​​of ctxCodedSubBlock[ft][lod][dim] to decode the frame type ft, the level of detail lod, and the mesh displacement vector Decode the diu_coded_subblock_flag in the dimension dim.

[0088] ctxCoeffGtN[numFT][numLOD][MAX_GTN+1][numDim] is the array of contexts used to decode the syntax element diu_coeff_abs_level_gtN (N can be replaced with 0, 1, 2, or MAX_GTN). The arithmetic decoding unit 3051 uses the value of ctxCoeffGtN[ft][lod][N][dim] to decode diu_coeff_abs_level_gtN for the frame type ft, the level of detail lod, and the dimension dim of the mesh displacement vector.

[0089] The arithmetic decoding unit 3051 uses a bypass to decode the frame type ft, the detail level lod, and the mesh Decode diu_coeff_sign in dimension dim of the displacement vector.

[0090] ctxCoeffRemPrefix[numFT][numLOD][numDim] is a context array used for decoding the syntax element diu_coeff_abs_level_rem. The arithmetic decoding unit 3051 uses the values ​​of ctxCoeffRemPrefix[ft][lod][dim] to decode the frame type ft, the detail level lod, and the mesh displacement vector. Decode diu_coeff_abs_level_rem in dimension dim of the rule.

[0091] The context initialization unit 3057 initializes the context (probability of occurrence of binary signals). The context may be initialized for each frame, or may be initialized for each set of one or more frames (Group of Frames, GoF). When the context is initialized for each GoF, there is no dependency between frames, so random access to any frame can be easily performed. Since the degree of initialization is small, the coding efficiency can be improved more than when initialization is performed for each frame.

[0092] (Mesh displacement derivation process) The mesh displacement decoding unit 305 decodes the syntax elements diu_last_sig_coeff, diu_coded_block_flag, diu_coded_subblock_flag, diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3, diu_coeff_abs_level_rem, and diu_coeff_sign through the following processing to derive the mesh displacement Qdisp. Here, the mesh displacement decoding unit 305 decodes diu_last_sig_coeff for each component. Decode diu_coded_block_flag in LOD units (block units) and decode it in subBlockSize size. If diu_coded_subblock_flag is a predetermined value, the mesh displacement coefficients within the subblock are decoded. for (k = 0; k < numDim; k++) { / / Dimension (component) loop / / Decode diu_last_sig_coeff diu_last_sig_coeff[k] = decodeExpGolumn(ctxLastSigCoeff[ft][k]) dispOffset = 0 for (b = 0; b < numLOD; b++) { / / Detail level loop, block loop / / Decoding of diu_coded_block_flag diu_coded_block_flag[k][b] = decode(ctxCodedBlock[ft][b][k]) if (diu_coded_block_flag[k][b]) { numSubBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numSubBlocks; s++) { / / Sub-block loop / / Decoding of diu_coded_subblock_flag diu_coded_subblock_flag[k][b][s] = decode(ctxCodedSubBlock[ft][b][k]) if (diu_coded_subblock_flag[k][b][s]) { for (v = 0; v < subBlockSize; v++) { / / Coefficient loop within sub-block value = 0 / / Decoding of diu_coeff_abs_level_gt0 diu_coeff_abs_level_gt0[k][b][s][v] = decode(ctxCoeffGtN[ft][b][0][k]) if (diu_coeff_abs_level_gt0[k][b][s][v]) { value++ / / Decoding of diu_coeff_sign diu_coeff_sign[k][b][s][v] = decode(ctxStatic) / / Decoding of diu_coeff_abs_level_gt1 diu_coeff_abs_level_gt1[k][b][s][v] = decode(ctxCoeffGtN[ft][b][1][k]) if (diu_coeff_abs_level_gt1[k][b][s][v]) { value++ / / Decoding of diu_coeff_abs_level_gt2 diu_coeff_abs_level_gt2[k][b][s][v] = decode(ctxCoeffGtN[ft][b][2][k]) if (diu_coeff_abs_level_gt2[k][b][s][v]) { value++ / / Decoding of diu_coeff_abs_level_gt3 diu_coeff_abs_level_gt3[k][b][s][v] = decode(ctxCoeffGtN[ft][b][3][k]) if (diu_coeff_abs_level_gt3[k][b][s][v]) { / / Decoding of diu_coeff_abs_level_rem diu_coeff_abs_level_rem[k][b][s][v] = decodeExpGolumn(ctxCoeffRemPrefix[ft][b][k]) value += (1 + diu_coeff_abs_level_rem) } } if (diu_coeff_sign[k][b][s][v]) { value = -value } } Qdisp[dispOffset + s * subBlockSize + v][k] = value } }} } dispOffset += dispCount[b] } } Here, decode() and decodeExpGolumn() take the corresponding context as an argument and return 1 bit. value++ is an operation to increment the variable value by 1, value += 1, value = value + 1. subBlockSize is the number of subblocks. The subBlockSize is a value between 16 and 4096. For example, 100, 128, etc. dispCount[b] is the number of mesh displacements at level of detail b. decode(ctxStatic) may use a bypass-only process as decode_bypass().

[0093] The mesh displacement decoding unit 305 may derive the mesh displacement value from diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3, diu_coeff_abs_level_rem, and diu_coeff_level_sign as follows, instead of the method of the pseudo code above. The value is stored in QDisp.

[0094] absCoeff = diu_coeff_abs_level_gt0 + diu_coeff_abs_level_gt1 + diu_coeff_abs_level_gt2 + diu_coeff_abs_level_gt3 + diu_coeff_abs_level_rem value = absCoeff * (1 - 2 * diu_coeff_sign) Alternatively, the mesh displacement decoding unit 305 may decode the syntax elements diu_last_sig_coeff, diu_coded_block_flag, diu_coded_subblock_flag, diu_coeff_abs_level_gtN, diu_coeff_abs_level_rem, and diu_coeff_sign through the following process and derive the mesh displacement Qdisp. for (k = 0; k < numDim; k++) { / / Dimension (component) loop / / Decoding of diu_last_sig_coeff diu_last_sig_coeff[k] = decodeExpGolumn(ctxLastSigCoeff[ft][k]) dispOffset = 0 for (b = 0; b <numLOD; b++) { / / Detail level loop, block loop / / Decoding of diu_coded_block_flag diu_coded_block_flag[k][b] = decode(ctxCodedBlock[ft][b][k]) if (diu_coded_block_flag[k][b]) { numSubBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numSubBlocks; s++) { / / Sub-block loop / / Decoding of diu_coded_subblock_flag diu_coded_subblock_flag[k][b][s] = decode(ctxCodedSubBlock[ft][b][k]) if (diu_coded_subblock_flag[k][b][s]) { for (v = 0; v < subBlockSize; v++) { / / Coefficient loop within sub-block value = 0 / / Decoding of diu_coeff_abs_level_gt0 diu_coeff_abs_level_gt0[k][b][s][v] = decode(ctxCoeffGtN[ft][b][0][k]) if (diu_coeff_abs_level_gt0[k][b][s][v]) { / / Decoding of diu_coeff_sign diu_coeff_sign[k][b][s][v] = decode(ctxStatic) N = 1 maxGtN = 3 while (N <= maxGtN) { value++ / / Decoding of diu_coeff_abs_level_gtN (N = 1..maxGtN) diu_coeff_abs_level_gtN[k][b][s][v] = decode(ctxCoeffGtN[ft][b][N][k]) if (!diu_coeff_abs_level_gtN[k][b][s][v]) break N++ } if (diu_coeff_abs_level_gtN[k][b][s][v]) { / / Decoding of diu_coeff_abs_level_rem diu_coeff_abs_level_rem[k][b][s][v] = decodeExpGolumn(ctxCoeffRemPrefix[ft][b][k]) value += (1 + diu_coeff_abs_level_rem[k][b][s][v]) } if (diu_coeff_sign[k][b][s][v]) { value = -value } } Qdisp[dispOffset + s * subBlockSize + v][k] = value } } } } dispOffset += dispCount[b] } } Note that maxGtN is not limited to 3, and for example, maxGtN=2 may be used to encode / decode the syntax elements diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, and diu_coeff_abs_level_gt2, or maxGtN=4 may be used to encode / decode the syntax elements diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3, and diu_coeff_abs_level_gt4.

[0095] (Bypass configuration) Although the above describes an example in which a context is used, some syntax elements may be bypass-decoded without using a context. A bypass encoding / decoding configuration has the effect of reducing the memory and processing load for the context.

[0096] For example, the block-based syntax elements diu_last_sig_coeff and diu_coded_block_flag may be bypass coded without using a context. That is, static coding may be performed using ctxStatic instead of ctxLastSigCoeff[numFT][numDim] and ctxCodedBlock[numFT][numLOD][numDim]. Bypass coding and decoding of these syntax elements reduces the memory and processing amount of the context while maintaining coding efficiency.

[0097] Also, for the syntax element diu_coeff_abs_level_rem related to rem, bypass coding is used, that is, ctxStatic (static context) is used instead of ctxCoeffRemPrefix[ft][b][k]. This further reduces the update operations and memory required for the context. The syntax elements diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3 (diu_coeff_abs_level_gt4, diu_coeff_abs_level_gtN) that indicate whether the coeff value exceeds the threshold are decoded using the context that needs to be updated without using bypass coding.

[0098] (Configuration for limiting the number of context coding coefficients to be decoded) Note that, among the subBlockSize coefficients in a subblock, if context is used for coeff_abs_level_gt0, diu_coeff_abs_level_gt1, coeff_abs_level_gt2, and diu_coeff_abs_level_gt3, a maximum of 4*subBlockSize coefficients are context-coded. To reduce this complexity, the number of context-coded coefficients is counted in syntax elements other than coeff_abs_level_gt0 and diu_coeff_abs_level_gt1 (diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3). If this value is equal to or greater than a predetermined value maxContextInBlock, , diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3 may be switched from decoding using a context to decoding without using a context (bypass, using a static context).

[0099] Also, count the number of context-encoded coefficients other than coeff_abs_level_gt0 and diu_coeff_abs_level_gt1, and if the value is greater than or equal to a predetermined value maxContextInBlock, diu_coeff_level_rem may be decoded directly without decoding diu_coeff_abs_level_gt2 and diu_coeff_abs_level_gt3. In this case, initialize countGT = 0 before the loop of the sub-block and in the sub-block, increment countGT by 1 every time diu_coeff_abs_level_gt2 and diu_coeff_abs_level_gt3 are decoded. Further, only when countGT < maxContextInBlock decode diu_coeff_abs_level_gt2 and diu_coeff_abs_level_gt3. The syntax table (partial extract) is as follows.

[0100] if (diu_coded_subblock_flag[k][b][s]) { for (v = vStart; v < subBlockSize; v++) { countGT = 0 remainFlag = 0 diu_coeff_abs_level_gt0[k][b][s][v] if (diu_coeff_abs_level_gt0[k][b][s][v]) diu_coeff_sign[k][b][s][v] diu_coeff_abs_level_gt1[k][b][s][v] if (diu_coeff_abs_level_gt1[k][b][s][v]) { if (countGT < maxContextInBlock) { diu_coeff_abs_level_gt2[k][b][s][v] countGT += 1 if (diu_coeff_abs_level_gt2[k][b][s][v]) { if (countGT < maxContextInBlock) { diu_coeff_abs_level_gt3[k][b][s][v] countGT += 1 if (diu_coeff_abs_level_gt3[k][b][s][v]) remainFlag = 1 } else { remainFlag = 1 } } else { remainFlag = 1 } } else { remainFlag = 1 } } if (remainFlag) { diu_coeff_abs_rem[k][b][s][v] } value = (1 + diu_coeff_abs_level_gt1[k][b][s][v] + diu_coeff_abs_level_gt2[k][b][s][v] + diu_coeff_abs_level_gt3[k][b][s][v] + diu_coeff_abs_rem[k][b][s][v]) * (2 - diu_coeff_sign[k][b][s][v]) } } Also, diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2 Count the number of times decryption is performed, including diu_coeff_abs_level_gt3, and the value is If the value is less than the specified value maxContextInBlock, diu_coeff_abs_level_gt2 and diu_coeff_abs_level_gt3 may be decoded.

[0101] (Configuration that uses context for some bins in the prefix) Figure 16(a) is an example of the syntax structure of mesh displacement. The difference from Figure 15 is that it does not have the syntax of gt2(diu_coeff_abs_level_gt2) and gt3(diu_coeff_abs_level_gt3). If diu_coeff_abs_level_gt1 is true, the mesh displacement decoding unit 305 decodes diu_coeff_abs_rem. FIG. 16(b) shows the context settings used by the mesh displacement decoding unit 305. The mesh displacement decoding unit 305 uses diu_coeff_abs_level_gt0 and diu_coeff_abs_level_gt1. Context, bypass diu_coeff_sign, prefix diu_coeff_abs_level_rem The first part of the prefix is ​​decoded as a context, the rest of the prefix is ​​bypassed, and the suffix of diu_coeff_abs_level_rem is decoded as a bypass. In particular, different contexts can be used for bin==0 and bi===1 of the prefix, and bin>=2 and after of the prefix can be bypassed. The prefixes bin==0 and bin==1,bin==2 have different contexts. According to the above, the value to be ExpGolomb-encoded is large. Even if the number of contexts is less than 2 (or 3), the remaining (diu_coeff_abs_level_rem) is less complex than using contexts for all bins. It is also more efficient than using bypass for all bins.

[0102] For example, if maxContextInBlock=subBlockSize / 4, the number of bins to be context-encoded is The maximum number of blocks (worst case) can be reduced from 4*subBlockSize to subBlockSize*(2+1 / 4).

[0103] In addition, the number of times diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2, and diu_coeff_abs_level_gt3 are decoded is counted, and only if the counted value is less than the predetermined value maxContextInBlock, diu_coeff_abs_level_gt2 and diu_coeff_abs_level_gt3 are decoded. may be decrypted.

[0104] The inverse quantization unit 3053 performs inverse quantization based on the quantization scale value iscale and derives the mesh displacement Tdisp after transformation (e.g., wavelet transform). Tdisp may be in a Cartesian coordinate system or a local coordinate system. iscale is a value derived from the quantization parameter of each component of the mesh displacement image. Tdisp[0][] = (Qdisp[0][] * iscale[0] + iscaleOffset) >> iscaleShift Tdisp[1][] = (Qdisp[1][] * iscale[1] + iscaleOffset) >> iscaleShift Tdisp[2][] = (Qdisp[2][] * iscale[2] + iscaleOffset) >> iscaleShift Here, iscaleOffset = 1<<(iscaleShift-1). iscaleShift may be a predetermined constant, or may be a value decoded from encoded data at the sequence level, picture / frame level, tile / patch level, etc.

[0105] The inverse transform unit 3054 performs an inverse transform g (for example, an inverse wavelet transform) to derive a mesh displacement d. d[0][] = g(Tdisp[0][]) d[1][] = g(Tdisp[1][]) d[2][] = g(Tdisp[2][]) The coordinate system conversion unit 3055 converts the mesh displacement (coordinate system of the mesh displacement) into the Cartesian coordinate system based on the value of the coordinate system conversion information displacementCoordinateSystem. Specifically, when displacementCoordinateSystem==1, the displacement in the local coordinate system is converted into the displacement in the Cartesian coordinate system. Here, d is a 3D vector indicating the mesh displacement before the coordinate system transformation. disp is a 3D vector indicating the mesh displacement after the coordinate system transformation, and is in the Cartesian coordinate system. n_vec, t_vec, b_vec are 3D vectors (in the Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target region or target vertex. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } The derivation method shown above for vector multiplication can be expressed individually as scalars as follows: if (displacementCoordinateSystem == 0) { for (i = 0; i < 3; i++) {disp[i] = d[i]} } else if (displacementCoordinateSystem == 1){ for (i = 0; i < 3; i++) {disp[i] = d[0] * n_vec[i] + d[1] * t_vec[i] + d[2] * b_vec[i]}} Note that disp=d is used to assign the same variable name before and after the transformation, and the value of d is updated by the coordinate transformation. The configuration may be such that:

[0106] Alternatively, the following configuration may be used. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 2){ disp = d[0] * n_vec2 + d[1] * t_vec2 + d[2] * b_vec2 } Here, n_vec2, t_vec2, and b_vec2 are three-dimensional vectors (in the Cartesian coordinate system) corresponding to the axes of the local coordinate system of the adjacent region.

[0107] The following configuration may also be used. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } Here, n_vec3, t_vec3, b_vec3 are three-dimensional vectors (in Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target area where the fluctuation is suppressed. For example, the previous coordinates as follows: Derive the vector of the coordinate system used for decoding from the current coordinate system and the n_vec3 = (w*n_vec3 + (WT - w)*n_vec)>>wShift t_vec3 = (w*t_vec3 + (WT - w)*t_vec)>>wShift b_vec3 = (w*b_vec3 + (WT - w)*b_vec)>>wShift Here, for example, wShift = 2, 3, 4, WT = 1<<wShift, and w = 1..WT - 1. For example, when w = 3 and wShift = 3, n_vec3 = (3*n_vec3 + 5*n_vec)>>3 t_vec3 = (3*t_vec3 + 5*t_vec)>>3 b_vec3 = (3*b_vec3 + 5*b_vec)>>3 Also, a configuration may be adopted such that it can be selected according to the value of the coordinate system conversion information displacementCoordinateSystem decoded from the encoded data as follows. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 6){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } (Mesh reconstruction) FIG. 6 is a functional block diagram showing the configuration of the mesh reconstruction unit 307. The mesh reconstruction unit 307 is composed of a mesh division unit 3071 and a mesh deformation unit 3072.

[0108] The mesh division unit 3071 divides the base mesh output from the base mesh decoding unit 303 Divide the mesh into smaller pieces and generate a divided mesh.

[0109] FIG. 9(a) shows a part (triangle) of the base mesh, and the triangle is composed of vertices v1, v2, and v3. v1, v2, and v3 are three-dimensional vectors. The mesh dividing unit 3071 divides the triangle Generate a division mesh by adding new vertices v12, v13, and v23 in the middle of each edge of the shape. and output (Fig. 9(b)). v12 = (v1 + v2) / 2 v13 = (v1 + v3) / 2 v23 = (v2 + v3) / 2 The following is also possible: v12 = (v1 + v2 + 1) >> 1 v13 = (v1 + v3 + 1) >> 1 v23 = (v2 + v3 + 1) >> 1 The mesh deformation unit 3072 receives the division mesh and the mesh displacement, and outputs the mesh displacement d12, The deformed mesh is generated by adding d13 and d23 and output (Fig. 9(c)). The mesh displacements are the output of the mesh displacement decoding unit 305 (coordinate system conversion unit 3055). d12, d13, and d23 are mesh displacements corresponding to the vertices v12, v13, and v23 added by the mesh division unit 3071. v12' = v12 + d12 v13' = v13 + d13 v23' = v23 + d23 It should be noted that d12 = disp[0][], d23 = disp[1][], and d23 = disp[3][] may also be used.

[0110] (Configuration of 3D data encoding device according to the first embodiment) FIG. 10 is a functional block diagram showing a schematic configuration of a 3D data encoding device 11 according to the first embodiment. The 3D data encoding device 11 includes an atlas information encoding unit 101, a base mesh encoding unit 103, a base mesh decoding unit 104, a mesh displacement updating unit 106, a mesh displacement encoding unit 107, a mesh The 3D data encoding device 11 is composed of a mesh displacement decoding unit 108, a mesh reconstruction unit 109, an attribute update unit 110, a padding unit 111, a color space conversion unit 112, an attribute encoding unit 113, a multiplexing unit 114, and a mesh separation unit 115. The 3D data encoding device 11 inputs atlas information, a base mesh, a mesh displacement, a mesh, and an attribute image as 3D data, and outputs encoded data.

[0111] The atlas information encoding unit 101 encodes the atlas information and outputs the atlas information encoded stream. Output the program.

[0112] The base mesh encoding unit 103 encodes the base mesh and generates a base mesh encoding sequence. The stream is output using an encoding method such as Draco.

[0113] The base mesh decoding unit 104 is similar to the base mesh decoding unit 303, and therefore a description thereof will be omitted.

[0114] The mesh displacement update unit 106 updates the (original) base mesh and the decoded base mesh. Based on the mesh, adjust the mesh displacement and output the updated mesh displacement.

[0115] The mesh displacement encoding unit 107 encodes the updated mesh displacement and generates a mesh displacement code Outputs the encoded stream.

[0116] The mesh displacement decoding unit 108 is similar to the mesh displacement decoding unit 305, and therefore a description thereof will be omitted.

[0117] The mesh reconstruction unit 109 is similar to the mesh reconstruction unit 307, and therefore a description thereof will be omitted.

[0118] The attribute update unit 110 inputs the (original) mesh, the reconstructed mesh output from the mesh reconstruction unit 109 (mesh deformation unit 3072), and the attribute image, updates the attribute image to match the position (coordinates) of the reconstructed mesh, and outputs the updated attribute image.

[0119] The padding unit 111 receives the attribute image and pads the pixel values ​​in the empty area. Perform the processing.

[0120] The color space conversion unit 112 performs color space conversion from the RGB format to the YCbCr format.

[0121] The attribute encoding unit 113 encodes the YCbCr formatted image data output from the color space conversion unit 112. The attribute image is encoded and an attribute video stream is output. The encoding method is VVC or HEVC.

[0122] The multiplexing unit 114 multiplexes the atlas information coded stream, the base mesh coded stream, The mesh displacement coding stream and the attribute video stream are multiplexed and output as coded data. Multiplexing methods include byte stream format and ISOBMFF. is used.

[0123] (Mesh separation unit operation) The mesh separation unit 115 generates a base mesh and a mesh displacement from the mesh.

[0124] 13 is a functional block diagram showing the configuration of the mesh separation unit 115. The mesh separation unit 115 is made up of a mesh thinning unit 1151, a mesh division unit 1152, and a mesh displacement derivation unit 1153.

[0125] The mesh thinning unit 1151 generates a base mesh by thinning out some of the vertices from the mesh.

[0126] Figure 14(a) shows a part of a mesh, which has vertices v1, v2, v3, v4, v5, v1, v2, v3, v4, v5, and v6 are each three-dimensional vectors. The mesh thinning unit 1151 generates and outputs a base mesh by thinning out the vertices v4, v5, and v6. (Figure 14(b)).

[0127] The mesh dividing unit 1152 divides the base mesh to generate divided meshes, similar to the mesh dividing unit 3071 (FIG. 14(c)). v4' = (v1 + v2) / 2 v5' = (v1 + v3) / 2 v6' = (v2 + v3) / 2 The mesh displacement calculation part calculates the displacement for vertices v4', v5', and v6' based on the mesh and the divided mesh. The displacements d4, d5, and d6 of the vertices v4, v5, and v6 are calculated and output as mesh displacements (Fig. 14(d)). . d4 = v4 - v4' d5 = v5 - v5' d6 = v6 - v6' (Base mesh encoding) FIG. 11 is a functional block diagram showing the configuration of the base mesh encoding unit 103. The mesh encoding unit 103 is composed of a mesh encoding unit 1031, a mesh decoding unit 1032, a motion information encoding unit 1033, a motion information decoding unit 1034, a mesh motion compensation unit 1035, a reference mesh memory 1036, a switch 1037, and a switch 1038. The base mesh encoding unit 103 may also include a base mesh quantization unit (not shown) after the base mesh is input. When encoding a base mesh without referring to other base meshes (e.g., an already encoded base mesh) (intra-coding), the switches 1037 and 1038 are connected to the side that does not perform motion compensation. When encoding a base mesh with reference to other base meshes (inter-coding), the switches 1037 and 1038 are connected to the side that performs motion compensation.

[0128] The mesh encoding unit 1031 has an intra-encoding function, performs intra-encoding on the base mesh, and outputs a base mesh encoded stream. There are.

[0129] The mesh decoding unit 1032 is similar to the mesh decoding unit 3031, and therefore a description thereof will be omitted.

[0130] The motion information encoding unit 1033 has an inter-encoding function, performs inter-encoding on the base mesh, and outputs a base mesh encoded stream. The encoding method used is entropy encoding such as arithmetic encoding.

[0131] The motion information decoding unit 1034 is similar to the motion information decoding unit 3032, and therefore a description thereof will be omitted.

[0132] The mesh motion compensation unit 1035 is similar to the mesh motion compensation unit 3033, and therefore a description thereof will be omitted.

[0133] The reference mesh memory 1036 is similar to the reference mesh memory 3034, and therefore a description thereof will be omitted.

[0134] (Mesh displacement encoding) 12 is a functional block diagram showing the configuration of the mesh displacement encoding unit 107. The encoding unit 107 includes a coordinate system conversion unit 1071, a conversion unit 1072, a quantization unit 1073, a binarization unit 1074, and an arithmetic coder. It is composed of an encoding unit 1075, a context selection unit 1076, and a context initialization unit 1077.

[0135] The coordinate system conversion unit 1071 converts the coordinate system of the mesh displacement from a Cartesian coordinate system to a coordinate system that encodes the displacement (for example, a local coordinate system) based on the value of the coordinate system conversion information displacementCoordinateSystem. Here, disp is a three-dimensional vector indicating the mesh displacement before the coordinate system conversion, d is a three-dimensional vector indicating the mesh displacement after the coordinate system conversion, and n_vec, t_vec, and b_vec are three-dimensional vectors (in the Cartesian coordinate system) indicating each axis of the local coordinate system. if (displacementCoordinateSystem == 0) { d = disp } else if (displacementCoordinateSystem == 1){ d = (disp * n_vec, disp * t_vec, disp * b_vec) } The mesh displacement encoding unit 107 encodes the value of the displacementCoordinateSystem at the sequence level. It may be updated at the picture / frame level. The initial value is 0, which indicates a Cartesian coordinate system.

[0136] When updating the displacementCoordinateSystem at the sequence level, the synthesizer in the configuration of Figure 7 Set asps_vdmc_ext_displacement_coordinate_system to 0 for Cartesian coordinate system and 1 for local coordinate system.

[0137] When changing the displacementCoordinateSystem at the picture / frame level, the structure shown in Figure 8 is used. Use the following syntax for creating afps_vdmc_ext_displacement_coordinate_system. afps_vdmc_ext_displacement_coordinate_system_enable_flag: Set 1 to update the coordinate system, or 0 to not update the coordinate system. afps_vdmc_ext_displacement_coordinate_system: Set 0 to Cartesian coordinate system, or 1 to local coordinate system.

[0138] The transform unit 1072 performs a transform f (for example, a wavelet transform) and derives a mesh displacement Tdisp after the transform. Tdisp[0][] = f(d[0][]) Tdisp[1][] = f(d[1][]) Tdisp[2][] = f(d[2][]) The quantization unit 1073 performs quantization based on the quantization scale value "scale" derived from the quantization parameter of each component of the mesh displacement, and derives the mesh displacement Qdisp after quantization. Qdisp[0][] = Tdisp[0][] / scale[0] Qdisp[1][] = Tdisp[1][] / scale[1] Qdisp[2][] = Tdisp[2][] / scale[2] Alternatively, the scale value may be approximated by a power of 2 and Qdisp may be derived using the following formula: scale[i] = 1 << scale2[i] Qdisp[0][] = Tdisp[0][] >> scale2[0] Qdisp[1][] = Tdisp[1][] >> scale2[1] Qdisp[2][] = Tdisp[2][] >> scale2[2] The binarization unit 1074 encodes the quantized mesh displacement Qdisp, which is a multi-valued signal, into a binary signal. Do so. The binary signal may be a k-th order exponential Golomb code.

[0139] The arithmetic coding unit 1075 arithmetic-codes the binary signal and outputs a mesh displacement coded stream.

[0140] The context selection unit 1076 is the same as the context selection unit 3056, so the description is omitted.

[0141] The context initialization unit 1077 is the same as the context initialization unit 3057, so the description is omitted. Here, an example of using the context is described, but some syntax elements may be bypass-coded without using the context. In the configuration for bypass coding, the effect of reducing the context memory and processing amount is achieved. For example, the syntax elements diu_last_sig_coeff, diu_coded_block_flag, and diu_coeff_abs_level_rem may be bypass-coded without using the context. These syntax By bypass-coding these elements, the effect of reducing the context memory and processing amount is achieved while maintaining the coding efficiency.

[0142] The mesh displacement coding unit 107 codes the mesh displacement Qdisp by the following processing. for (k = 0; k < numDim; k++) { / / Dimension (component) loop / / Coding of diu_last_sig_coeff encodeExpGolumn(diu_last_sig_coeff[k], ctxLastSigCoeff[ft][k]) if (!lastSig) continue dispOffset = 0 for (b = 0; b <numLOD; b++) { / / Detail level loop, block loop / / Encoding diu_coded_block_flag encode(diu_coded_block_flag[k][b], ctxCodedBlock[ft][b][k]) numBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numBlocks; s++) { / / Sub-block loop / / Encoding diu_coded_subblock_flag encode(diu_coded_subblock_flag[k][b][s], ctxCodedSubBlock[ft][b][k]) for (v = 0; v < subBlockSize; v++) { / / Loop through coefficients in subblocks / / Encoding diu_coeff_abs_level_gt0 d = Qdisp[dispOffset + s * subBlockSize + v][k] encode(d != 0, ctxCoeffGtN[ft][b][0][k]) if (!d) continue / / encoding of diu_coeff_sign encode(d < 0, ctxSign[ft][b][k]) d = abs(d) - 1 / / Encoding diu_coeff_abs_level_gt1 encode(d != 0, ctxCoeffGtN[ft][b][1][k]) if (!d) continue d = abs(d) - 1 / / Encoding of diu_coeff_abs_level_gt2 encode(d != 0, ctxCoeffGtN[ft][b][2][k]) if (!d) continue d = abs(d) - 1 / / Encoding diu_coeff_abs_level_gt3 encode(d != 0, ctxCoeffGtN[ft][b][3][k]) if (!d) continue / / Encoding of diu_coeff_abs_level_rem encodeExpGolumn(--d, ctxCoeffRemPrefix[ft][b][k]) } } dispOffset += dispCount[b] } } Here, encode() and encodeExpGolumn() take a value and the corresponding context as arguments and This is a function that arithmetically encodes a binary string of k-th order Golomb code, each of which has a 1-bit value. dispCount[b] is , the number of mesh displacements at level b of detail. lastSig is a flag indicating whether the current coefficient is the last non-zero coefficient in the sub-block in scan order. lastSig=0 means the current coefficient is the last non-zero coefficient in the sub-block in scan order. lastSig=1 indicates that the current coefficient is not the last non-zero coefficient in the sub-block in scan order. Indicates that the number is the last non-zero coefficient in the sub-block in scan order.

[0143] Alternatively, the mesh displacement Qdisp may be encoded using the following process. for (k = 0; k < numDim; k++) { / / Dimension (component) loop / / Encoding of diu_last_sig_coeff encodeExpGolumn(diu_last_sig_coeff[k], ctxLastSigCoeff[ft][k]) if (!lastSig) continue dispOffset = 0 for (b = 0; b < numLOD; b++) { / / Detail level loop, block loop / / Encoding of diu_coded_block_flag encode(diu_coded_block_flag[k][b], ctxCodedBlock[ft][b][k]) numBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numBlocks; s++) { / / Sub-block loop / / Encoding of diu_coded_subblock_flag encode(diu_coded_subblock_flag[k][b][s], ctxCodedSubBlock[ft][b][k]) for (v = 0; v < subBlockSize; v++) { / / Coefficient loop within sub-block / / Encoding of diu_coeff_abs_level_gt0 d = Qdisp[dispOffset + s * subBlockSize + v][k] encode(d != 0, ctxCoeffGtN[ft][b][0][k]) if (!d) continue / / Encoding of diu_coeff_sign encode(d < 0, ctxSign[ft][b][k]) N = 1 maxGtN = 3 while (N <= maxGtN) { d = abs(d) - 1 / / Encoding of diu_coeff_abs_level_gtN (N = 1..maxGtN) encode(d != 0, ctxCoeffGtN[ft][b][N][k]) if (!d) break N++ } if (d) { / / Encoding of diu_coeff_abs_level_rem encodeExpGolumn(--d, ctxCoeffRemPrefix[ft][b][k]) } } } dispOffset += dispCount[b] } } Note that maxGtN is not limited to 3. For example, maxGtN=2 may be set and the syntax elements diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, and diu_coeff_abs_level_gt2 may be coded, or maxGtN=4 may be set and the syntax elements diu_coeff_abs_level_gt0, diu_coeff_abs_level_gt1, diu_coeff_abs_level_gt2, diu_coeff_abs_level_gt3, and diu_coeff_abs_level_gt4 may be coded. It may also be configured to encode the data.

[0144] One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention.

[0145] [Application example] The above-mentioned 3D data encoding device 11 and 3D data decoding device 31 can be mounted on various devices that transmit, receive, record, and play back 3D data. The 3D data may be natural 3D data captured by a camera or the like, or artificial 3D data generated by a computer or the like. It may be 3D data (including CG and GUI).

[0146] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. In other words, embodiments obtained by combining technical means modified appropriately within the scope of the claims are also included in the technical scope of the present invention. [Industrial Applicability]

[0147] The embodiments of the present invention can be suitably applied to a 3D data decoding device that decodes coded data in which 3D data has been coded, and a 3D data coding device that generates coded data in which 3D data has been coded, and can also be suitably applied to the data structure of coded data that is generated by the 3D data coding device and referenced by the 3D data decoding device. [Explanation of symbols]

[0148] 11 3D data encoding device 101 Atlas Information Encoding Unit 103 Base mesh coding unit 1031 Mesh coding unit 1032 Mesh Decoding Unit 1033 Motion information encoding unit 1034 Motion information decoding unit 1035 Mesh motion compensation unit 1036 reference mesh memory 1037 Switch 1038 Switch 104 Base mesh decoding unit 106 Mesh displacement update section 107 Mesh displacement coding unit 1071 Coordinate system conversion unit 1072 Conversion Unit 1073 Quantization section 1074 Binarization section 1075 Arithmetic encoder 1076 Context Selection 1077 Context Initialization Unit 108 Mesh displacement decoding unit 109 Mesh reconstruction unit 110 Attribute Update Section 111 Padding section 112 Color space conversion unit 113 Attribute Encoding Unit 114 Multiplexer 115 mesh separation section 1151 Mesh thinning section 1152 Mesh division section 1153 Mesh displacement derivation part 21 Network 31 3D data decoding device 301 Demultiplexer 302 Atlas Information Decoding Unit 303 Base mesh decoding unit 3031 Mesh Decoding Unit 3032 Motion information decoding unit 3033 Mesh Motion Compensation Unit 3034 Reference Mesh Memory 3035 Switch 3036 Switch 305 Mesh displacement decoding unit 3051 Arithmetic Decoding Unit 3052 Multi-level conversion section 3053 Inverse quantization section 3054 Inverse Conversion Unit 3055 Coordinate system conversion unit 3056 Context Selection Section 3057 Context Initialization Unit 307 Mesh Reconstruction Unit 306 Attribute Decoding Unit 3071 Mesh division section 3072 Mesh deformation part 308 Color Space Conversion Unit 41 3D data display device

Claims

1. A 3D data decoding device for decoding encoded data, comprising an arithmetic decoding unit for arithmetically decoding mesh displacement from the encoded data, The arithmetic decoding unit decodes from the encoded data a first flag indicating whether the absolute value of the coefficient of the mesh displacement is greater than 0, a second flag indicating whether the absolute value of the coefficient is greater than 1, and a third flag indicating whether the absolute value of the coefficient is greater than 2.

2. The 3D data decoding device according to claim 1, characterized in that the arithmetic decoding unit further decodes a fourth flag indicating whether or not an absolute value of the coefficient is greater than 3 from the encoded data.

3. The arithmetic decoding unit further includes a fifth flag indicating whether the block includes a non-zero coefficient of the mesh displacement and / or a third flag indicating a final position of the non-zero coefficient of the mesh displacement.

3. The 3D data decoding device according to claim 1, further comprising: a 3D data decoding unit for decoding an index of one from the encoded data.

4. The arithmetic decoding unit bypasses the fifth flag and / or the first index.

4. The 3D data decoding device according to claim 3, further comprising:

5. A 3D data encoding device for encoding 3D data includes an arithmetic encoding unit for arithmetically encoding mesh displacement, The arithmetic coding unit encodes a first flag indicating whether the absolute value of the coefficient of the mesh displacement is greater than 0, a second flag indicating whether the absolute value of the coefficient is greater than 1, and a third flag indicating whether the absolute value of the coefficient is greater than 2 into encoded data.

6. 6. The 3D data encoding device according to claim 5, wherein the arithmetic encoding unit further encodes a fourth flag indicating whether or not an absolute value of the coefficient is greater than 3 into the encoded data.

7. The arithmetic coding unit may further include a fifth flag indicating whether the block includes a non-zero coefficient of the mesh displacement and / or indicating a final position of the non-zero coefficient of the mesh displacement.

7. The 3D data encoding device according to claim 5, further comprising: encoding the first index into the encoded data.

8. The arithmetic coding unit bypass codes the fifth flag and the first index.

8. The 3D data encoding device according to claim 7,

Images