3D data decoder and 3D data encoder

Abstract

To improve encoding efficiency of a mesh displacement to encode / decode 3D data with high quality in encoding / decoding of the 3D data.SOLUTION: A 3D data decoder that decodes mesh data or point group data includes: a sub-mesh decoding unit that decodes sub-mesh information from encoded data obtained by encoding the mesh data or the point group data; a base mesh decoding unit that decodes a base mesh from the encoded data and the sub-mesh information; a mesh displacement decoding unit that decodes a mesh displacement from the encoded data and the sub-mesh information; and a mesh reconstitution unit that decodes a mesh from the decoded base mesh and mesh displacement. The mesh displacement decoding unit uses the sub-mesh information decoded by the sub-mesh decoding unit to decode the mesh displacement from the decoded data.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] Embodiments of the present invention relate to a 3D data encoding device and a 3D data decoding device.

Background Art

[0002] In order to efficiently transmit or record 3D data, a 3D data encoding device that converts 3D data into a two-dimensional image and encodes it using a video image encoding method to generate encoded data, and a 3D data decoding device that decodes a two-dimensional image from the encoded data to reconstruct 3D data exist.

[0003] Specific 3D data encoding methods include, for example, ISO / IEC 23090-5 V3C (Volumetric Video-based Coding) and V-PCC (Video-based Point Cloud Compression) of MPEG-I. V3C can encode and decode a point cloud composed of point positions and attribute information. Furthermore, ISO / IEC 23090-12 (MPEG Immersive Video, MIV) and ISO / IEC 23090-29 (Video-based Dynamic Mesh Coding, V-DMC) under standardization are also used for encoding and decoding multi-view video and mesh video. The V-DMC method is disclosed in the latest draft document of Non-Patent Document 1.

[0004] In these 3D data encoding methods, the geometry and attributes constituting the 3D data are encoded and decoded as an image using a moving image encoding method such as H.265 / HEVC (High Efficiency Video Coding) or H.266 / VVC (Versatile Video Coding).

[0005] ​​In the case of a point group, the geometry image is the depth to the projection plane, and the attribute image is the image obtained by projecting the attributes onto the projection plane.

[0006] 3D data (mesh) such as in Non-Patent Document 1 is composed of a base mesh, mesh displacement, and texture mapping image. The encoding of the base mesh can use a vertex encoding method such as Draco. For the encoding of mesh displacement, in addition to the method of encoding the mesh displacement image obtained by two-dimensioning the mesh displacement using a video codec, there is also a method of directly encoding by arithmetic coding. The texture mapping image is encoded as an attribute image using a video codec. As the video codec, the above-mentioned HEVC or VVC can be used.

Prior Art Documents

Non-Patent Documents

[0007]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0008] In the 3D data encoding method in Non-Patent Document 1, the mesh displacement (mesh displacement array, mesh displacement image) constituting the 3D data (mesh) can be encoded and decoded using an arithmetic coding method. Although the mesh displacement is arithmetic-coded to be encoded and decoded in frame units, there is a problem that it cannot be encoded and decoded in units smaller than a frame (sub-mesh units).

[0009] An object of the present invention is to improve the encoding granularity of mesh displacement in the encoding and decoding of 3D data using an arithmetic coding method, and to encode and decode 3D data with high efficiency.

Means for Solving the Problem

[0010] In order to solve the above problems, a 3D data decoding apparatus according to an aspect of the present invention is a 3D data decoding apparatus that decodes mesh data or point cloud data, and includes a sub-mesh decoding unit that decodes sub-mesh information from encoded data in which the mesh data or point cloud data is encoded, a base mesh decoding unit that decodes a base mesh from the encoded data and the sub-mesh information, a mesh displacement decoding unit that decodes a mesh displacement from the encoded data and the sub-mesh information, and a mesh reconstruction unit that decodes a mesh from the decoded base mesh and the mesh displacement. In the mesh displacement decoding unit, the mesh displacement is decoded from the encoded data using the sub-mesh information decoded by the sub-mesh decoding unit.

[0011] In order to solve the above problems, a 3D data encoding apparatus according to an aspect of the present invention is a 3D data encoding apparatus that encodes mesh data or point cloud data, and includes a sub-mesh encoding unit that encodes sub-mesh information, a base mesh encoding unit that encodes a base mesh using the sub-mesh information, and a mesh displacement encoding unit that encodes a mesh displacement using the sub-mesh information. In the mesh displacement encoding unit, the mesh displacement is encoded using the sub-mesh information encoded by the sub-mesh encoding unit.

Advantages of the Invention

[0012] According to an aspect of the present invention, the encoding efficiency of the mesh displacement can be improved, and 3D data can be encoded and decoded with high quality.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Best Mode for Carrying Out the Invention

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a schematic diagram showing the configuration of a 3D data transmission system 1 according to this embodiment.

[0016] The 3D data transmission system 1 is a system that transmits an encoded stream obtained by encoding 3D data to be encoded, decodes the transmitted encoded stream, and displays the 3D data. The 3D data transmission system 1 includes a 3D data encoding device 11, a network 21, a 3D data decoding device 31, and a 3D data display device 41.

[0017] 3D data T is input to the 3D data encoding device 11.

[0018] The network 21 transmits the encoded stream Te generated by the 3D data encoding device 11 to the 3D data decoding device 31. The network 21 may be the Internet, a wide area network (WAN), a local area network (LAN), or These are the combinations. The network 21 is not necessarily limited to a two-way communication network, and may be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. Also, the network 21 may be replaced by a storage medium that stores an encoded stream Te such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blu-ray Disc: registered trademark). It may be replaced by a storage medium that stores an encoded stream Te such as a DVD (Digital Versatile Disc: registered trademark) or a BD (Blu-ray Disc: registered trademark).

[0019] The 3D data decoder 31 decodes each of the encoded streams Te transmitted by the network 21 and generates one or more decoded 3D data Td that have been decoded.

[0020] The 3D data display device 41 displays all or part of the one or more decoded 3D data Td generated by the 3D data decoder 31. The 3D data display device 41 includes, for example, a display device such as a liquid crystal display or an organic EL (Electro-luminescence) display. Examples of the form of the display include stationary, mobile, and HMD. Also, when the 3D data decoder 31 has high processing power, it displays an image with high image quality, and when it has only low processing power, it displays an image that does not require high processing power or display ability. When the 3D data decoder 31 has high processing power, it displays an image with high image quality, and when it has only low processing power, it displays an image that does not require high processing power or display ability.

[0021] <Operator> The operators used in this specification are described below.

[0022] [[ID=22>> is a right bit shift, << is a left bit shift, & is a bitwise AND, | is a bitwise OR , |= is an OR assignment operator, and || indicates a logical OR.

[0023] x?y:z is a ternary operator that takes y when x is true (non-zero) and z when x is false (0). y..z indicates a set of integers from y to z.

[0024] <Structure of the encoded stream Te> Before going into a detailed description of the 3D data encoding device 11 and the 3D data decoding device 31 according to this embodiment, the data structure of the encoded stream Te generated by the 3D data encoding device 11 and decoded by the 3D data decoding device 31 will be described.

[0025] FIG. 2 is a diagram showing a hierarchical structure of data in the coded stream Te. Frame Te has either a V3C sample stream or a V3C unit stream data structure. A V3C sample stream includes a sample stream header and a V3C unit. A V3C unit stream includes a V3C unit.

[0026] A V3C unit contains a V3C unit header and a V3C unit payload. Unit Type is an ID that indicates the type, and can be V3C_VPS, V3C_AD, V3C_AVD, V3C_GVD, V3C_OVD, etc. Take the value indicated by the label.

[0027] If the Unit Type is V3C_VPS (Video Parameter Set), the V3C unit includes a V3C parameter set.

[0028] If the Unit Type is V3C_AD (Atlas Data), the V3C unit includes a VPS ID, an atlasID, a sample stream nal header, and multiple NAL units. The atlasID is an ID (Idenfication). Yes, takes an integer value of 0 or greater.

[0029] A NAL unit contains a NALUnitType, a layerID, a TemporalID, and a RBSP (Raw byte sequence payload).

[0030] The NAL unit is identified by the NALUnitType and includes ASPS (Atlas Sequence Parameter Set), AAPS (Atlas Adaptation Parameter Set), ATL (Atlas Tile layer), SEI (Supplemental Enhancement Information), etc.

[0031] The ATL includes an ATL header and an ATL data unit, and the ATL data unit includes information such as the position and size of the patch, such as patch information data.

[0032] The SEI includes a payloadType indicating the type of SEI, a payloadSize indicating the size (number of bytes) of the SEI, and a sei_payload of the SEI data.

[0033] When the Unit Type is V3C_AVD (Attribute Video Data), the V3C unit includes a VPS ID, an atlasID, an attrIdx of the ID of the attribute image, a partIdx of the partition ID, a mapIdx of the map ID, a flag auxFlag indicating whether it is auxiliary data, and a video stream. The video stream is data encoded by HEVC, VVC, etc. The attribute data corresponds to the texture image in V-DMC.

[0034] When the NalUnitType is V3C_GVD (Geometory Video Data), the V3C unit includes a VPS ID, an atlasID, a mapIdx, an auxFlag, and a video stream. The geometry data corresponds to the mesh displacement in V-DMC.

[0035] When the Unit Type is V3C_OVD (Occupancy Video Data), the V3C unit includes a VPS ID, an atlasID, and a video stream.

[0036] When the Unit Type is V3C_MD (Mesh data), the V3C unit includes a VPS ID, an atlasID, and a mesh_payload. In V-DMC, it corresponds to the base mesh.

[0037] (Configuration of the 3D data decoding device according to the first embodiment) FIG. 3 is a functional block diagram showing the schematic configuration of a 3D data decoding device 31 according to the first embodiment. The 3D data decoding device 31 includes a demultiplexing unit 301, a sub-mesh decoding unit 309, an atlas information decoding unit 302, a base mesh decoding unit 303, a mesh displacement decoding unit 305, a mesh reconstruction unit 307, an attribute decoding unit 306, and a color space conversion unit 308. The 3D data decoding device 31 inputs the encoded data of the 3D data and outputs the atlas information, the mesh, and the attribute image.

[0038] The demultiplexing unit 301 inputs the encoded data multiplexed in a byte stream format, ISOBMFF (ISO Base Media File Format), etc., demultiplexes it, and outputs an atlas information encoded stream (Atlas Data stream of V3C_AD, NAL unit), a base mesh encoded stream (mesh_payload of V3C_MD), a mesh displacement encoded stream (video stream of V3C_GVD), and an attribute video stream (video stream of V3C_AVD).

[0039] The sub-mesh decoding unit 309 inputs the atlas information sub-mesh encoded stream output from the demultiplexing unit 301 and decodes the sub-mesh information.

[0040] The atlas information decoding unit 302 inputs the atlas information encoded stream output from the sub-mesh decoding unit 309 and decodes the atlas information.

[0041] The atlas information decoding unit 302 in FIG. 3 decodes the coordinate system conversion information displacementCoordinateSystem (asps_vdmc_ext_displacement_coordinate_system, afps_vdmc_ext_displacement_coordinate_system) indicating the coordinate system from the encoded data. Note that a gating flag may be provided separately, and each coordinate system conversion information may be decoded only when the gating flag is 1. The gating flag is, for example, afve_displacement_coordinate_system_enable_flag.

[0042] The base mesh decoding unit 303 decodes the base mesh encoded stream encoded by vertex encoding (3D data compression encoding method, for example, Draco) and outputs the base mesh. The base mesh will be described later.

[0043] The mesh displacement decoding unit 305 decodes the mesh displacement encoded stream and outputs the mesh displacement.

[0044] The mesh reconstruction unit 307 inputs the base mesh and the mesh displacement and reconstructs the mesh in 3D space.

[0045] The attribute decoding unit 306 decodes the attribute video stream encoded by VVC, HEVC, etc. and outputs the attribute image. The attribute image is a texture image (texture mapping image converted by the UV atlas method) expanded on the UV axis and in YCbCr format It may also be in the format. The type of codec used for encoding is indicated by the ptl_profile_codec_group_idc obtained by decoding the V3C parameter set of the encoded data. It may also be indicated by the Four CC code indicated by ai_geometry_codec_id[atlasID] of the V3C parameter set. ai_geometry_codec_id[atlasID] indicates the index corresponding to the codec ID of the decoder used for decoding the attribute video stream at the atlas ID.

[0046] The color space conversion unit 308 converts the attribute image from the YCbCr format to the RGB format for color space conversion. Note that a configuration may be adopted in which the attribute video stream encoded in the RGB format is decoded and the color space conversion is omitted.

[0047] (Decoding of the base mesh) FIG. 4 is a functional block diagram showing the configuration of the base mesh decoding unit 303. The base mesh decoding unit 303 is composed of a mesh decoding unit 3031, a motion information decoding unit 3032, a mesh motion compensation unit 3033, a reference mesh memory 3034, a switch 3035, a switch 3036, and a skip decoding unit 3037. The base mesh decoding unit 303 is arranged before the output of the base mesh by a base mesh not shown in the figure ​It may also be configured to include an inverse quantization unit. When the base mesh to be decoded is intra-coded (encoded without referring to other base meshes (for example, base meshes that have already been encoded and decoded)), switches 3035 and 3036 are connected to the mesh decoding unit 3031 side. Otherwise, when the base mesh to be decoded is inter-coded (encoded with reference to other base meshes), it is connected to the side that performs motion compensation. When performing motion compensation, the target vertex coordinates are derived by referring to the already decoded vertex coordinates and motion information. Otherwise, when the base mesh to be decoded is skipped and other base meshes are encoded as the decoding target (skip encoding), it is connected to the skip decoding unit 3037 side.

[0048] Each base mesh is composed of one or more sub-meshes. When there are multiple sub-meshes, the tile header in the atlas data sub-bitstream requires an ID to search for the sub-mesh corresponding to the tile. Here, a sub-mesh is a subset of the mesh defined by specifying a part of the 3D model, and it is a mesh formed by dividing the mesh into multiple parts. By dividing the mesh into subsets, specific ranges of the mesh can be defined individually. Each sub-mesh has its own vertex coordinates, normal vectors, texture coordinates, etc., and can be operated and edited individually. The mesh of a certain frame is called a mesh frame. The mesh decoding unit 3031 decodes the intra-coded base mesh coding stream and outputs the base mesh (base mesh vertex position, base mesh vertex position vector). As the coding method, Draco, Edge Breaker, etc. are used.

[0049]

[0050] The motion information decoding unit 3032 decodes the inter-coded base mesh coded stream and outputs motion information (mesh motion information, mesh motion vector) for each vertex of a reference mesh described later. Entropy coding such as arithmetic coding is used as the coding method.

[0051] The mesh motion compensation unit 3033 performs motion compensation on each vertex of the reference mesh input from the reference mesh memory 3034 based on the motion information, and outputs a motion-compensated mesh.

[0052] The reference mesh memory 3034 is a memory that holds decoded meshes for reference in subsequent decoding processes.

[0053] (Mesh displacement decoding) 5 is a functional block diagram showing the configuration of the mesh displacement decoding unit 305. The mesh displacement decoding unit 305 is made up of a CABAC decoding unit (an arithmetic decoding unit 3051, a multi-value conversion unit 3052, a context selection unit 3056, and a context initialization unit 3057), an inverse quantization unit 3053, an inverse transformation unit 3054, and a coordinate system transformation unit 3055.

[0054] (Context-Adaptive Binary Arithmetic Coding) The arithmetic decoding unit 3051, the multi-value conversion unit 3052, the context selection unit 3056, and the context initialization unit 3057 use a context called Context-Adaptive Binary Arithmetic Coding (CABAC). CABAC uses a decoding method that has been developed for the 32-bit ... It is decoded at a rate (EP, bypass), and the index ctxIdx specifying the context and the update of the specified context are omitted. The context is a variable (memory area) for holding the probability (state) of CABAC, and is identified by the value of ctxIdx (0, 1, 2, …). Also, when 0 and 1 always have equal probabilities, that is, 0.5, 0.5, it is called EP (Equal Probability) or bypass. In this case, since there is no need to hold the state for a specific syntax element, the context is not used. Also, a static context with a fixed probability of 0.5 that does not need to be updated may be used. In this sense, it may be called static instead of bypass. A value indicating a probability of 0.5 may use an integer value such as 128.

[0055] Note that the process of decoding 1 bit without using the context (bypass) may use the following pseudo-code. rangeTimesProb = IvlRange >> 1 binVal = ( rangeTimesProb <= ( IvlCode - IvlLow ) ) if (binVal == 0) IvlRange = rangeTimesProb else { IvlLow += rangeTimesProb IvlRange -= rangeTimesProb } Note that the process of decoding 1 bit using the context may use the following pseudo-code. Here, prob0 is a variable indicating the probability of the context. rangeTimesProb = IvlRange * prob0 >> 16 binVal = ( rangeTimesProb <= ( IvlCode - IvlLow ) )​​​​ if (binVal == 0) IvlRange = rangeTimesProb else { IvlLow += rangeTimesProb IvlRange -= rangeTimesProb } (Coordinate System) The coordinate system of the mesh displacement (3D vector) uses the following two types of coordinate systems. Cartesian coordinate system (canonical): An orthogonal coordinate system commonly defined throughout the 3D space. (X, Y, Z) coordinate system. An orthogonal coordinate system whose direction does not change at the same time (within the same frame, within the same tile). Local coordinate system: An orthogonal coordinate system defined for each region or each vertex in the 3D space . An orthogonal coordinate system whose direction can change at the same time (within the same frame, within the same tile). A coordinate system with axes of normal (D), tangent (U), and bi - tangent (V). That is, the first axis (D) indicated by the normal vector n_vec at a certain vertex (a surface including a certain vertex), and the second axis (U) and the third axis (V) indicated by two tangent vectors t_vec and b_vec orthogonal to the normal vector n_vec . It consists of an orthogonal coordinate system. n_vec, t_vec, and b_vec are 3D vectors. The (D, U, V) coordinate system may also be referred to as the (n, t, b) coordinate system.

[0056] (Decoding and Derivation of Control Parameters at the Sequence Level) Here, the control parameters at the sequence level decoded from the encoded data by the mesh displacement decoding unit 305 will be described.

[0057] ​​Figure 7 shows an example of the syntax of an ASPS (Atlas Sequence Parameter Set), which is a sequence-level parameter set. The ASPS is one of the NAL units of the atlas information and contains syntax elements applied to the atlas information coding stream. The semantics of each field are as follows. It contains syntax elements applied to the atlas information coding stream. The semantics of each field are as follows.

[0058] asve_subdivision_iteration_count: Indicates the number of mesh subdivision iterations.

[0059] asve_displacement_coordinate_system: Coordinate system conversion information indicating the coordinate system of the mesh displacement. When the value is equal to a predetermined first value (e.g., 0), it indicates the Cartesian coordinate system. When the value is equal to another second value (e.g., 1), it indicates the local coordinate system.

[0060] asve_1d_displacement_flag: A flag indicating whether the mesh displacement is one-dimensional. When the value is true, it indicates that the mesh displacement is one-dimensional. When the value is false, it indicates that the mesh displacement is three-dimensional.

[0061] (Decoding / Derivation of Picture / Frame-Level Control Parameters) Figure 8 shows an example of the syntax of an AFPS (Atlas Frame Parameter Set), which is a picture / frame-level parameter set. The AFPS is one of the NAL units of the atlas information and contains syntax elements applied to the atlas information coding stream. The semantics of each field are as follows. The AFPS contains atlas_frame_mesh_information(). It is an example of the syntax of an AFPS (Atlas Frame Parameter Set), which is a picture / frame-level parameter set. The AFPS is one of the NAL units of the atlas information and contains syntax elements applied to the atlas information coding stream. It contains syntax elements applied to the atlas information coding stream. The semantics of each field are as follows. The AFPS contains atlas_frame_mesh_information().

[0062] afve_overriden_flag: A flag indicating whether to update the coordinate system of the mesh displacement. When this flag is equal to true, update the coordinate system of the mesh displacement based on the value of afve_displacement_coordinate_system described below. When this flag is equal to false, do not update the coordinate system of the mesh displacement. When this flag is equal to false, do not update the coordinate system of the mesh displacement. When this flag is equal to false, do not update the coordinate system of the mesh displacement.

[0063] afve_subdivision_iteration_count: Indicates the number of mesh subdivision iterations.

[0064] afve_displacement_coordinate_system: Coordinate system conversion information indicating the coordinate system of the mesh displacement. When the value is equal to the first value (for example, 0), it indicates the Cartesian coordinate system. When the value is equal to the second value (for example, 1), it indicates the local coordinate system. When no syntax element appears, assume the value decoded by ASPS, and the default coordinate system is the coordinate system indicated by ASPS.

[0065] (Decoding and Derivation of Mesh-Level Control Parameters) Figure 21 is an example of the syntax structure of atlas_frame_mesh_information() transmitted by AFPS. atlas_frame_mesh_information() may contain any of the following syntax elements. The semantics of each field are as follows.

[0066] afmi_use_single_mesh_flag: A flag indicating whether there is only one sub-mesh referenced by the mesh patch in each atlas frame referring to AFPS. When the value is true, it indicates that there is only one referenced sub-mesh. When the value is false, it indicates that there may be multiple referenced sub-meshes. it indicates that there is only one referenced sub-mesh. When the value is false, it indicates that there may be multiple referenced sub-meshes.

[0067] afmi_submesh_alignment_flag: The sub-mesh of the base mesh and the sub- A flag indicating whether the meshes correspond. If the value is true, it indicates that the base mesh and the sub-mesh of the mesh displacement correspond. If the value is false, it indicates that the base mesh and the sub-mesh of the mesh displacement may not correspond. Here, for the base mesh sub-mesh and the mesh displacement sub-mesh to correspond means that the vertices of the base mesh with the same ID and the corresponding vertices of the mesh displacement exist in the same region. From the base sub-mesh and the mesh displacement sub-mesh, a set of meshes (sub-meshes) can be decoded. Also, for the base mesh sub-mesh and the mesh displacement sub-mesh to correspond means that the number of sub-meshes is equal. Also, the vertices of the base mesh with a certain sub-mesh ID are referenced by the mesh displacement with the same sub-mesh ID. Or, the mesh displacement with a certain sub-mesh ID may decode the vertices by referring only to the vertices of the base mesh with the same sub-mesh ID. Also, there may be a condition of a bit stream where the mesh displacement with a certain sub-mesh ID refers only to the vertices of the base mesh with the same sub-mesh ID. It indicates that the sub-mesh of the mesh displacement may not correspond. Here, for the base mesh sub-mesh and the mesh displacement sub-mesh to correspond means that the vertices of the base mesh with the same ID and the corresponding vertices of the mesh displacement exist in the same region. From the base sub-mesh and the mesh displacement sub-mesh, a set of meshes (sub-meshes) can be decoded. Also, for the base mesh sub-mesh and the mesh displacement sub-mesh to correspond means that the number of sub-meshes is equal. Also, the vertices of the base mesh with a certain sub-mesh ID are referenced by the mesh displacement with the same sub-mesh ID. Or, the mesh displacement with a certain sub-mesh ID may decode the vertices by referring only to the vertices of the base mesh with the same sub-mesh ID. Also, there may be a condition of a bit stream where the mesh displacement with a certain sub-mesh ID refers only to the vertices of the base mesh with the same sub-mesh ID.

[0068] afmi_num_submeshes_minus1: A parameter indicating the number of sub-meshes referenced by the mesh patch.

[0069] afmi_signalled_submesh_id_flag: A flag indicating whether the sub-mesh ID referenced by the mesh patch is signaled. If the value is true, it indicates that the sub-mesh ID is signaled. If the value is false, it indicates that the sub-mesh ID is not signaled.

[0070] afmi_signalled_submesh_id_length_minus1: For the patch data with the index patchIdx ​​In the TA unit, when there exists a syntax element afmi_submesh_id[i] within the atlas style where the tile ID is equal to tileID, it is a parameter indicating the number of bits of the syntax element pdu_submesh_id / bmpdu_submesh_id[tileID][patchIdx].

[0071] afmi_submesh_id[i]: It is a parameter indicating the submesh ID of the i-th submesh. 。

[0072] The atlas information decoding unit 302 (submesh decoding unit 309) decodes the frame mesh information from the encoded data of the AFPS of the atlas information. For example, it decodes afmi_use_single_mesh_flag, afmi_num_submeshes_minus1, afmi_num_displ_submeshes_minus1, afmi_signalled_submesh_id_flag, afmi_signalled_displ_submesh_id_flag, afmi_signalled_submesh_id_length_minus1, afmi_signalled_displ_submesh_id_length_minus1, afmi_submesh_id, afmi_displ_submesh_id. Further, afmi_submesh_alignment_flag may be decoded. Also, when afmi_submesh_alignment_flag is false, it may be configured to decode and encode including the submesh information afmi_signalled_displ_submesh_id_length_minus1, afmi_submesh_id, afmi_displ_submesh_id of the mesh displacement. That is, when afmi_submesh_alignment_flag is true, it does not include the submesh information afmi_signalled_displ_submesh_id_length_minus1, afmi_submesh_id, afmi_displ_submesh_id of the mesh displacement and does not perform decoding and encoding. The atlas information encoding unit 101 (sub-mesh encoding unit 116) encodes the frame mesh information into the encoded data of the AFPS of the atlas information.

[0073] The submesh decoding unit 309 detects the submesh ID when afmi_signalled_submesh_id_flag is true. The first step is to decode afmi_submesh_id[ i ] in the range of i = 0.. afmi_num_submeshes_minus1 by the number of meshes (afmi_num_submeshes_minus1 + 1) and derive the arrays SubMeshIDToIndex and SubMeshIndextoID for i = 0.. afmi_num_submeshes_minus1 as follows:

[0074] SubMeshIDToIndex[ afmi_submesh_id[ i ] ] = i SubMeshIndextoID[ i ] = afmi_submesh_id[ i ] The submesh decoding unit 309 does not decode afmi_submesh_id[ i ] when afmi_signalled_submesh_id_flag is true, and arranges the following for i = 0..afmi_num_submeshes_minus1: Derive.

[0075] SubMeshIDToIndex[ i ] = i SubMeshIndextoID[ i ] = i The submesh decoding unit 309 may derive the array as follows for i = 0..afmi_num_displ_submeshes_minus1 when afmi_submesh_alignment_flag is true:

[0076] DisplSubMeshIDToIndex[ i ] = SubMeshIDToIndex[ i ] DisplSubMeshIndextoID[ i ] = SubMeshIndextoID[ i ] Furthermore, when afmi_submesh_alignment_flag is true, the SubMeshIDToIndex and SubMeshIndextoID common to the base mesh and mesh displacement may be used instead of the mesh mutation arrays DisplSubMeshIDToIndex and DisplSubMeshIndextoID.

[0077] Furthermore, if afmi_submesh_alignment_flag is false, or in a configuration in which afmi_submesh_alignment_flag is not decoded, it may be derived as follows.

[0078] The submesh decoding unit 309 performs the submesh decoding when afmi_signalled_displ_submesh_id_flag is true. afmi_num_displ_submeshes_minus1 + 1) for i = 0.. afmi_num_displ_ Decode afmi_displ_submesh_id[ i ] in the range of submeshes_minus1 and derive the arrays DisplSubMeshIDToIndex and DisplSubMeshIndextoID for i = 0..afmi_num_displ_submeshes_minus1 as follows:

[0079] DisplSubMeshIDToIndex[ afmi_displ_submesh_id[ i ] ] = i DisplSubMeshIndextoID[ i ] = afmi_displ_submesh_id[ i ] The submesh decoding unit 309 does not decode afmi_displ_submesh_id[ i ] when afmi_signalled_displ_submesh_id_flag is false, and decodes afmi_displ_submesh_id[ i ] for i = 0.. afmi_num_displ_submeshes_minus1. The array is derived as follows:

[0080] DisplSubMeshIDToIndex[ i ] = i DisplSubMeshIndextoID[ i ] = i This configuration has the effect of enabling decoding and display on a sub-mesh basis even when meshes are displaced. In addition, in the configuration where afmi_submesh_alignment_flag is decoded and encoded, the base mesh and Since it is possible to judge whether the sub-meshes of the mesh displacement are the same or different by checking this flag, it is possible to easily decode and display each sub-mesh. Also, since it is possible to omit encoding of the array, the amount of code can be reduced.

[0081] (Another configuration) The submesh decoding unit 309 may always derive an array as follows, without decoding and encoding afmi_submesh_alignment_flag, instead of the example syntax structure of FIG.

[0082] DisplSubMeshIDToIndex[ i ] = SubMeshIDToIndex[ i ] DisplSubMeshIndextoID[ i ] = SubMeshIndextoID[ i ] That is, the sub-meshes of the base mesh and the sub-meshes of the mesh displacement may always correspond to each other, in which case the sub-mesh information decoded by the base mesh is also used to decode the mesh displacement.

[0083] In this configuration, there is no freedom to make the base mesh and the sub-mesh of the mesh displacement different. However, since it is known in advance on the decoding side that the mesh division is always the same, this has the effect of facilitating decoding and display in units of sub-meshes.

[0084] (Syntax Structure of Mesh Displacement) In the 3D data encoding method in Non-Patent Document 1, mesh displacement (displacement data) was encoded and decoded at the frame level, but there was a problem that encoding and decoding could not be performed using sub-mesh information indicating the unit for dividing a frame into a plurality of meshes. That is, the base mesh encoding unit 103 and the base mesh decoding unit 303 that perform encoding and decoding independently for each sub-mesh, and the mesh displacement encoding unit 107 and the mesh displacement decoding unit 305 that perform encoding and decoding for each frame can perform mesh reconstruction only at the frame level, and there was a problem that they could not be processed at the sub-mesh level. As shown in the syntax structure described later, in this example, it has the NAL unit type of FIG. 20. That is, mesh displacement is decoded at the sub-mesh level. There was a problem that they could not be processed at the sub-mesh level.

[0085] As shown in the syntax structure described later, in this example, it has the NAL unit type of FIG. 20. That is, mesh displacement is decoded at the sub-mesh level. That is, mesh displacement is decoded at the sub-mesh level.

[0086] FIG. 15 is an example of the syntax of a configuration for transmitting mesh displacement parameters by a sequence-level DSPS. DSPS (Displacement Sequence Parameter Set) is one of the NAL units of mesh displacement and includes syntax elements applied to the mesh displacement encoding stream. The semantics of each field are as follows. That is, mesh displacement is decoded at the sub-mesh level.

[0087] dsps_sequence_parameter_set_id: Indicates the identifier of the mesh displacement sequence parameter set for other syntax elements to refer to.

[0088] dsps_single_dimension_flag: A flag indicating whether the mesh displacement is one-dimensional or not. A value of true indicates that the mesh displacement is one-dimensional, a value of false indicates that the mesh displacement is three-dimensional.

[0089] dsps_lod_count: Indicates the number of Level of Detail (LoD) of mesh displacement. Note that the number of LoDs minus 1 (dsps_lod_count_minus1) may be encoded and decoded. In that case, dsps_lod_count = dsps_lod_count_minus1 + 1 is used.

[0090] FIG. 16 shows an example of the syntax of a configuration for transmission in DFPS (Displacement Frame Parameter Set), which is a picture / frame level parameter set of mesh displacement parameters. DFPS is a mesh displacement NAL unit and is applicable to mesh displacement coding streams. The semantics of each field are as follows:

[0091] dfps_displ_sequence_parameter_set_id: indicates the value of dsps_sequence_parameter_set_id for the active mesh displacement sequence parameter set.

[0092] dfps_displ_frame_parameter_set_id: A mesh for other syntax elements to reference. This indicates the identifier of the displacement frame parameter set.

[0093] The mesh displacement decoding unit 305 and the mesh displacement encoding unit 107 decode dfps_displ_sequence_parameter_set_id and dfps_output_flag_present_flag from the encoded data of DFPS, and Encode the data into symbolic data.

[0094] Figure 17 shows an example of the syntax structure of displ_sub_mesh_information() transmitted by DFPS. The semantics of each field are as follows. displ_sub_mesh_information() is sub-mesh information indicating the unit that divides a frame into multiple meshes. The sub-mesh information may include the number of sub-meshes, the ID of the sub-mesh, and the coded length of the sub-mesh ID.

[0095] dsi_use_single_mesh_flag: When dsi_use_single_mesh_flag is equal to 1, it indicates that there is only one sub-mesh in each mesh frame referenced by DFPS. When dsi_use_single_mesh_flag is equal to 0, it indicates that there may be multiple sub-meshes in each mesh frame referenced by DFPS. dsi_use_single_mesh_flag: When dsi_use_single_mesh_flag is equal to 1, it indicates that there is only one sub-mesh in each mesh frame referenced by DFPS. When dsi_use_single_mesh_flag is equal to 0, it indicates that there may be multiple sub-meshes in each mesh frame referenced by DFPS.

[0096] dsi_num_submeshes_minus1: It indicates the number of sub-meshes in each mesh frame referenced by DFPS minus 1. When dsi_use_single_mesh_flag is equal to 1, the value of dsi_num_submeshes_minus1 is presumed to be equal to 0.

[0097] dsi_signalled_submesh_id_flag: When dsi_signalled_submesh_id_flag is equal to 1, it indicates that the sub-mesh ID of each mesh frame is signaled. When dsi_signalled_submesh_id_flag is equal to 0, it indicates that the sub-mesh ID is not signaled.

[0098] dsi_signalled_submesh_id_length_minus1: The syntax element dsi_submesh_id[i] exists When doing so, it indicates the number of bits of the syntax element submesh_id of the submesh header.

[0099] dsi_submesh_id: dsi_submesh_id[i] specifies the i-th submesh ID. The length of the dsi_submesh_id[i] syntax element is dsi_signalled_submesh_id_length_minus1 + 1 bits. If it does not exist, the value of dsi_submesh_id[i] is presumed to be equal to i for each i in the range from 0 to dsi_num_submeshes_minus1. The length of the syntax element of dsi_submesh_id[i] is dsi_signalled_submesh_id_length_minus1 + 1 bits. element is dsi_signalled_submesh_id_length_minus1 + 1 bits.

[0100] The mesh displacement decoding unit 305 decodes the mesh displacement submesh information from the encoded data of the DFPS. For example, it decodes dsi_use_single_mesh_flag, dsi_num_submeshes_minus1, dsi_signalled_submesh_id_flag, dsi_signalled_submesh_id_length_minus1, dsi_submesh_id. The mesh displacement encoding unit 107 encodes the mesh displacement submesh information into the encoded data of the DFPS.

[0101] When dsi_signalled_submesh_id_flag is true, the mesh displacement decoding unit 305 decodes dsi_submesh_id[i] in the range of i = 0.. dsi_num_submeshes_minus1 for the number of submeshes (dsi_num_submeshes_minus1 + 1), and the arrays DisplSubMeshIDToIndex and DisplSubMeshIndextoID ​​​​Derive it as follows for \(i = 0.. dsi\_num\_submeshes\_minus1\).

[0102] DisplSubMeshIDToIndex[ dsi_submesh_id[ i ] ] = i DisplSubMeshIndextoID[ i ] = dsi_submesh_id[ i ] When dsi_signalled_submesh_id_flag is false, the mesh displacement decoding unit 305 does not decode dsi_submesh_id[ i ], and derives an array as follows for \(i = 0.. dsi\_num\_submeshes\_minus1\).

[0103] DisplSubMeshIDToIndex[ i ] = i DisplSubMeshIndextoID[ i ] = i Although not shown, the mesh displacement decoding unit 305 and the mesh displacement encoding unit 107 may decode and encode a displacement layer displ_layer_rbsp( ) including a displacement header displ_header( ), displacement data displ_data_unit( displ_id ), and rbsp_trailing_bits( ) from the encoded data. The displacement data displ_data_unit( displ_id ) may have a syntax structure described later in FIG. 18. The mesh displacement decoding unit 305 and the mesh displacement encoding unit 107 may decode and encode the following syntax from the displacement header displ_header( ).

[0104] The mesh displacement decoding unit 305 and the mesh displacement encoding unit 107 may decode and encode the following syntax from the displacement header displ_header( ). dh_frame_parameter_set_id: Indicates the ID of the parameter set.

[0105] dh_frame_parameter_set_id: Indicates the ID of the parameter set.

[0106] displ_submesh_id: Indicates the ID of the submesh of the mesh displacement.

[0107] dh_type: It is the encoding type of mesh displacement. It also indicates the intra-encoding (I_DISPLACEMENT) or the inter-encoding (P_DISPLACEMENT) type. A variable st may be used as the variable indicating the sub-mesh type. Also, it may be used as st = dh_type. Or it may indicate the inter-encoding (P_DISPLACEMENT) type. A variable st may be used as the variable indicating the sub-mesh type. Also, it may be used as st = dh_type.

[0108] dh_output_flag: The flag indicating whether to output.

[0109] dh_frm_order_cnt_lsb: It is the value of the least significant bit (LSB) of Picture Order Cnd (POC).

[0110] Figures 18 and 19 are examples of the syntax structure of mesh displacement. The semantics are as follows. The mesh displacement is a column of sub-mesh ID (subMeshID), position level, and values (coefficients) of the k component (k component), and is represented by the array Qdisp[subMeshID][level][k]. The displacement is a three-dimensional signal in the Cartesian coordinate system (xyz) or the local coordinate system (ntb), and each component of the three-dimensional displacement is called a component. Here, the displacement Qdisp is also called a coefficient because it is the value after being transformed by discrete wavelet transform, lifting transform, DCT transform, etc. Since it is the value after being transformed by discrete wavelet transform, lifting transform, DCT transform, etc., it is also called a coefficient. The variable k takes values of 0, 1, and 2. The variable name is not limited to k, and it may also be dim or other variable names. The order of the indices of QDisp may be reversed, that is, instead of Qdisp[subMeshID][level][k], Qdisp[subMeshID][k][level] may be used. As in the example of the syntax structure in Figure 18(a), the mesh Or it may be reversed, that is, instead of Qdisp[subMeshID][level][k], Qdisp[subMeshID][k][level] may be used. As in the example of the syntax structure in Figure 18(a), the mesh Regardless of the coding type of displacement, the syntax structure ddu_intra_sub_mesh_unit( displSubmeshID ) in FIG. 18(b) may be used. Alternatively, depending on the coding type, ddu_intra_sub_mesh_unit( displSubmeshID ) and ddu_inter_sub_mesh_unit( displSubmeshID ) may be used as follows. displ_data_unit( displSubmeshID ) { if( dh_type == I_DISPLACEMENT) { ddu_intra_sub_mesh_unit( displSubmeshID ) } else if( df_type == P_DISPLACEMENT ) { ddu_inter_sub_mesh_unit( displSubmeshID ) } } Here, ddu_inter_sub_mesh_unit( displSubmeshID ) may use a syntax structure in which the mesh displacement to be decoded performs motion compensation with reference to other mesh displacements.

[0111] The mesh displacement decoding unit 305 decodes the syntax shown in FIGS. 18 and 19 in sub-mesh units indicated by displSubMeshID. For example, as shown below, the number of displacement coefficients, the absolute value of the displacement coefficients, the sign of the displacement coefficients, etc. may be decoded in units of displSubMeshID. Note that displSubMeshID may use the value displ_submesh_id of the displacement header displ_header( ). As another form displSubMeshID may use a value obtained by decoding the syntax element dsi_submesh_id[ i ] included in displ_sub_mesh_information( ).

[0112] ​​​ displSubMeshID = dsi_submesh_id[ i ] In addition, when decoding multiple submesh displacements indicated by index i, Alternatively, the ID (=displSubMeshID) may be derived from the index i using a previously decoded or derived array.

[0113] displSubMeshID = DisplSubMeshIndextoID[i] Furthermore, for index i, the mesh displacement decoding unit 305 and the mesh displacement encoding unit 107 may perform loop processing on i to decode and encode the displacement header displ_header( ) and the displacement data displ_data_unit( displ_id ).

[0114] Alternatively, the mesh displacement decoding unit 305 and the mesh displacement encoding unit 107 may decode and encode one displacement header displ_header( ) and then perform a loop process for i to decode and encode the displacement data displ_data_unit( displ_id ). In the displacement data, the values ​​decoded and encoded in one displ_header( ) are used in common.

[0115] The syntax elements in Figure 19 have the following meanings:

[0116] dismu_vertex_count_lod[displSubMeshID][i]: The coordinates included in the division (LoD) level i (variable vertCount[i] = dismu_vertex_count_lod[i], the number of vertices in block i. A number vertCount may be derived.

[0117] dismu_coeff_abs_level_gt0[displSubmeshID][k][v]: Submesh of displSubmeshID Indicates whether the absolute value of the non-zero mesh displacement coefficient at the vertex with index v of the component with index k (component, the same hereinafter) is greater than 0. If it is greater, it is 1; otherwise, it is 0.

[0118] dismu_coeff_abs_level_gt1[displSubmeshID][k][v]: For the submesh with displSubmeshID Indicates whether the absolute value of the non-zero mesh displacement coefficient at the vertex with index v of the component with index k is greater than 1. If it is greater, it is 1; otherwise, it is 0. If this syntax does not exist, it is presumed to be 0.

[0119] dismu_coeff_abs_level_gt2[displSubmeshID][k][v]: For the submesh with displSubmeshID Indicates whether the absolute value of the non-zero mesh displacement coefficient at the vertex with index v of the component with index k is greater than 2. If it is greater, it is 1; otherwise, it is 0. If this syntax does not exist, it is presumed to be 0.

[0120] dismu_coeff_abs_level_gt3[displSubmeshID][k][v]: For the submesh with displSubmeshID Indicates whether the absolute value of the non-zero mesh displacement coefficient at the vertex with index v of the component with index k is greater than 3. If it is greater, it is 1; otherwise, it is 0. If this syntax does not exist, it is presumed to be 0.

[0121] dismu_coeff_sign[displSubmeshID][k][v]: Indicates whether the non-zero mesh displacement coefficient at the vertex with index v of the component with index k in the submesh with displSubmeshID is a positive number. If it is a positive number, it is 1; otherwise (if it is a negative number), it is 0. If this syntax does not exist, it is presumed to be 1.

[0122] dismu_coeff_abs_level_rem[displSubmeshID][k][v]: The submesh of displSubmeshID In this case, the value obtained by subtracting 4 from the absolute value of the non-zero mesh displacement coefficient of the vertex with index v of the component with index k. If this syntax does not exist, it is assumed to be 0.

[0123] The mesh displacement decoding unit 305 decodes dismu_nz_subBlock for each sub-block of the mesh displacement. When dismu_nz_subBlock[displSubmeshID][k][block] is 1, the subsequent syntax elements are decoded at the component with index k and the sub-block level with index block. elements are decoded.

[0124] The mesh displacement decoding unit 305 decodes dismu_coeff_abs_level_gt0 for each sub-block of the mesh displacement, and when dismu_coeff_abs_level_gt0 is a predetermined value (for example, other than 0), it decodes the subsequent dismu_coeff_sign and dismu_coeff_abs_level_gt1.

[0125] The mesh displacement decoding unit 305 when dismu_coeff_abs_level_gt1 is a predetermined value (for example, other than 0), decodes the subsequent dismu_coeff_abs_level_gt2.

[0126] The mesh displacement decoding unit 305 when dismu_coeff_abs_level_gt2 is a predetermined value (for example, other than 0), decodes the subsequent dismu_coeff_abs_level_gt3.

[0127] The mesh displacement decoding unit 305 when dismu_coeff_abs_level_gt3 is a predetermined value (for example, other than 0) In this case, the subsequent dismu_coeff_abs_level_rem is decoded.

[0128] (Operation of Mesh Displacement Decoding Unit) The arithmetic decoding unit 3051 decodes a mesh displacement coded stream that has been arithmetic coded according to a value (context) indicating a random variable, and outputs a binary signal. The binary signal may be an alpha code or a k-th order Exp-Golomb-code. The Exp-Golomb code is composed of a prefix code and a suffix code. The prefix is a value that increases exponentially, and the suffix is the remainder. When encoding / decoding a variable rem using the Exp-Golomb code, the prefix and suffix of the Exp-Golomb code are also referred to as the prefix and suffix of rem.

[0129] The multi-valued conversion unit 3052 decodes a quantized mesh displacement Qdisp, which is a multi-valued signal, from the binary signal.

[0130] The context selection unit 3056 (context memory) has a memory for holding contexts, derives a context used for arithmetic decoding of mesh displacements according to the state, and updates values as necessary. In the arithmetic decoding of each coefficient of the mesh displacement, different context arrays may be used according to the sub-mesh type dh_type (e.g., 0: intra sub-mesh, 1: inter sub-mesh), the level of detail lod (level of detail) of the mesh division, and the component dim of the mesh displacement vector. The context includes a variable indicating the generation probability of the binary signal. depending on the level of detail lod (level of detail) of the mesh division and the component dim of the mesh displacement vector The following different context arrays may be used. The context includes a variable indicating the generation probability of the binary signal. is included. ctxCodedSubBlock[numST][numLOD][numDim] ctxCoeffGtN[numST][numLOD][MAX_GTN+1][numDim] ctxCoeffRemPrefix[numST][numLOD][numDim][numPrefixBin] Let ctxStatic be a static context with a fixed probability and no context update. The decoding of the syntax element indicated by ctxStatic may be performed without using a context. decode(ctxStatic) may be decode_bypass(), using a bypass-only process.

[0131] Here, numST is the number of sub-mesh type categories, and it may be numST = 2. numPrefixBin is the number of bins using a context in the prefix, and it may be numPrefixBin = 2. numLOD is the maximum number of detailed levels of mesh division, and it may be numLOD = 4. numDim is the number of dimensions of the mesh displacement vector, and it may be numDim = 3. The maximum value MAX_GTN of the threshold for the coefficient is 3.

[0132] ctxCodedSubBlock[numST][numLoD][numDim] is an array of contexts used for decoding the syntax element dismu_nz_subBlock. The arithmetic decoding unit 3051 uses the value of ctxCodedSubBlock[st][lod][dim] to decode dismu_nz_subBlock in the sub-mesh type st, detailed level lod, and dimension dim of the mesh displacement vector.

[0133] ctxCoeffGtN[numST][numLoD][MAX_GTN + 1][numDim] is an array of contexts used for decoding the syntax element dismu_coeff_abs_level_gtN (where N is replaced by 0, 1, 2, MAX_GTN). The arithmetic decoding unit 3051 uses the value of ctxCoeffGtN[st][lod][N][dim] to decode dismu_coeff_abs_level_gtN in the sub-mesh type st, detailed level lod, and dimension dim of the mesh displacement vector.

[0134] ​​The arithmetic decoding unit 3051 decodes dismu_coeff_sign in terms of the sub-mesh type st, the detailed level lod, and the dimension dim of the mesh displacement vector using bypass.

[0135] ctxCoeffRemPrefix[numST][numLoD][numDim] is an array of contexts used for decoding the syntax element dismu_coeff_abs_level_rem. The arithmetic decoding unit 3051 decodes dismu_coeff_abs_level_rem in terms of the sub-mesh type st, the detailed level lod, and the dimension dim of the mesh displacement vector using the value of ctxCoeffRemPrefix[st][lod][dim]. st may use dh_type decoded from the encoded data (the same applies hereinafter).

[0136] The context initialization unit 3057 initializes the context (the generation probability of a binary signal). The context may be initialized for each sub-mesh or for a plurality of sub-meshes. When initializing the context for each sub-mesh, since there is no context dependency between sub-meshes, random access to any sub-mesh can be easily performed. When initializing the context for a plurality of sub-meshes, since the frequency of initialization is low, the coding efficiency can be improved compared to the case of initializing for each sub-mesh.

[0137] (Mesh displacement derivation process) The mesh displacement decoding unit 305 decodes the syntax elements dismu_nz_subBlock, dismu_coeff_abs_level_gt0, dismu_coeff_abs_level_gt1, dismu_coeff_abs_level_gt2, dismu_coeff_abs_level_gt3, dismu_coeff_abs_level_rem, and dismu_coeff_sign by the following process. Then, the mesh displacement Qdisp is derived. Here, for the sub-meshes of the base mesh decoding unit 303 and the mesh displacement decoding unit 305, constraints may be imposed so as to have a corresponding relationship using the syntax element afmi_submesh_alignment_flag decoded from atlas_frame_mesh_information() shown in FIG. 21. Here, the mesh displacement decoding unit 305 decodes dismu_nz_subBlock in sub-block units of SubBlockSize. When dismu_nz_subBlock has a predetermined value, the mesh displacement coefficients within the sub-block are decoded. The decoding may be performed in sub-mesh units indicated by subMeshID (= displSubMeshID). st may use dh_type decoded from the encoded data. for (k = 0; k < numDim; k++) { / / dimension (component) loop for (b = 0; b<numLOD; b++) { / / Level of Detail loop, block loop numSubBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numSubBlocks; s++) { / / subblock loop / / decode dismu_nz_subBlock dismu_nz_subBlock [k][b][s] = decode(ctxCodedSubBlock[st][b][k]) if (dismu_nz_subBlock [k][b][s]) { for (v = 0; v < subBlockSize; v++) { / / coefficient loop within subblock value = 0 / / decode dismu_coeff_abs_level_gt0 dismu_coeff_abs_level_gt0[k][b][s][v] = decode(ctxCoeffGtN[st][b][0][k]) if (dismu_coeff_abs_level_gt0[k][b][s][v]) { value++ / / decode dismu_coeff_sign dismu_coeff_sign[k][b][s][v] = decode(ctxStatic) / / decode dismu_coeff_abs_level_gt1 dismu_coeff_abs_level_gt1[k][b][s][v] = decode(ctxCoeffGtN[st][b][1][k]) if (dismu_coeff_abs_level_gt1[k][b][s][v]) { value++ / / decode dismu_coeff_abs_level_gt2 dismu_coeff_abs_level_gt2[k][b][s][v] = decode(ctxCoeffGtN[st][b][2][k]) if (dismu_coeff_abs_level_gt2[k][b][s][v]) { value++ / / decode dismu_coeff_abs_level_gt3 dismu_coeff_abs_level_gt3[k][b][s][v] = decode(ctxCoeffGtN[st][b][3][k]) if (dismu_coeff_abs_level_gt3[k][b][s][v]) { / / decode dismu_coeff_abs_level_rem dismu_coeff_abs_level_rem[k][b][s][v] = decodeExpGolomb(ctxCoeffRemPrefix[st][b][k]) value += (1 + dismu_coeff_abs_level_rem) } } } if (dismu_coeff_sign[k][b][s][v]) { value = -value } } Qdisp[displSubMeshID][dispOffset + s * subBlockSize + v][k] = value } } } dispOffset += dispCount[b] } } Here, decode(ctx) is a function that decodes a 1-bit value with the corresponding context ctx as an argument, and decodeExpGolomb(ctxPrefix, ctxSuffix) is a function that decodes a value binary-coded with a k-th Golomb code (e.g., k = 0) using ctxPrefix[n] as the context at the bin position n of the prefix and ctxSuffix[m] as the context at the bin position m of the suffix. When not using a context for the suffix (using bypass), it is simply described as decodeExpGolomb(ctxPrefix). value++ is an operation that increments the variable value by 1, and value += 1, value = value + 1. subBlockSize is the size of the sub-block. for indicates a loop. subBlockSize Values that are powers of 2 from 16 to 4096 may be used. For example, 128 or 256 may be used. dispCount[b] is the number of mesh displacements at detail level b.

[0138] Instead of the method of the pseudo-code described above, the mesh displacement decoding unit 305 may derive the value of the mesh displacement from dismu_coeff_abs_level_gt0, dismu_coeff_abs_level_gt1, dismu_coeff_abs_level_gt2, dismu_coeff_abs_level_gt3, dismu_coeff_abs_level_rem, and dismu_coeff_level_sign as follows. The value is stored in QDisp. The decoding may be performed in units of sub-meshes indicated by subMeshID (= displSubMeshID).

[0139] absCoeff = dismu_coeff_abs_level_gt0 + dismu_coeff_abs_level_gt1 + dismu_coeff_abs_level_gt2 + dismu_coeff_abs_level_gt3 + dismu_coeff_abs_level_rem value = absCoeff * (1 - 2 * dismu_coeff_sign) Alternatively, the mesh displacement decoding unit 305 may decode the syntax elements dismu_nz_subBlock, dismu_coeff_abs_level_gtN, dismu_coeff_abs_level_rem, and dismu_coeff_sign by the following process and derive the mesh displacement Qdisp. for (k = 0; k < numDim; k++) { / / dimension (component) loop dispOffset = 0 for (b = 0; b <numLOD; b++) { / / Level of Detail loop, block loop numSubBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numSubBlocks; s++) { / / subblock loop / / decode dismu_nz_subBlock dismu_nz_subBlock [k][b][s] = decode(ctxCodedSubBlock[st][b][k]) if (dismu_nz_subBlock [k][b][s]) { for (v = 0; v < subBlockSize; v++) { / / coefficient loop within subblock value = 0 / / decode dismu_coeff_abs_level_gt0 dismu_coeff_abs_level_gt0[k][b][s][v] = decode(ctxCoeffGtN[st][b][0][k]) if (dismu_coeff_abs_level_gt0[k][b][s][v]) { / / decode dismu_coeff_sign dismu_coeff_sign[k][b][s][v] = decode(ctxStatic) N = 1 maxGtN = 3 while (N <= maxGtN) { value++ / / decode dismu_coeff_abs_level_gtN (N=1..maxGtN) dismu_coeff_abs_level_gtN[k][b][s][v] = decode(ctxCoeffGtN[st][b][N][k]) if (!dismu_coeff_abs_level_gtN[k][b][s][v]) break N++ } if (dismu_coeff_abs_level_gtN[k][b][s][v]) { / / decode dismu_coeff_abs_level_rem dismu_coeff_abs_level_rem[k][b][s][v] = decodeExpGolomb(ctxCoeffRemPrefix[st][b][k]) value += (1 + dismu_coeff_abs_level_rem[k][b][s][v]) } if (dismu_coeff_sign[k][b][s][v]) { value = -value } } Qdisp[displSubMeshID][dispOffset + s * subBlockSize + v][k] = value } } } dispOffset += dispCount[b] } } In the suspected code, break means skipping the subsequent operations and exiting from the nearest loop. That is the meaning.

[0140] Note that maxGtN is not limited to 3. For example, when maxGtN = 2, a configuration for encoding / decoding the syntax elements dismu_coeff_abs_level_gt0, dismu_coeff_abs_level_gt1, dismu_coeff_abs_level_gt2 may be used. Or when maxGtN = 4, the syntax elements dismu_coeff_abs_level_gt0, It may be configured to encode / decode dismu_coeff_abs_level_gt1, dismu_coeff_abs_level_gt2, dismu_coeff_abs_level_gt3, and dismu_coeff_abs_level_gt4.

[0141] The inverse quantization unit 3053 performs inverse quantization based on the quantization scale value iscale, and derives the mesh displacement Tdisp after conversion (for example, wavelet conversion). Tdisp may be in the Cartesian coordinate system or in the local coordinate system. iscale is a value derived from the quantization parameters of each component of the mesh displacement image. The inverse quantization is performed in units of sub-meshes indicated by subMeshID (= displSubMeshID). It may be performed in units of sub-meshes. Tdisp[subMeshID][0][] = (Qdisp[subMeshID][0][] * iscale[0] + iscaleOffset) >> iscaleShift Tdisp[subMeshID][1][] = (Qdisp[subMeshID][1][] * iscale[1] + iscaleOffset) >> iscaleShift Tdisp[subMeshID][2][] = (Qdisp[subMeshID][2][] * iscale[2] + iscaleOffset) >> iscaleShift Here, iscaleOffset = 1<<(iscaleShift-1). iscaleShift may be a predetermined constant, or it may be encoded at the sequence level, picture / frame level, sub-mesh level indicated by subMeshID (= displSubMeshID), tile / patch level, etc., and the decoded value from the encoded data may be used. It may be used.

[0142] The inverse conversion unit 3054 performs an inverse conversion g (for example, inverse wavelet conversion) to derive the mesh displacement d. d[0][] = g(Tdisp[subMeshID][0][]) d[1][] = g(Tdisp[subMeshID][1][]) d[2][] = g(Tdisp[subMeshID][2][]) The coordinate system conversion unit 3055 converts the mesh displacement (coordinate system of the mesh displacement) to the Cartesian coordinate system based on the value of the coordinate system conversion information displacementCoordinateSystem. Specifically, when displacementCoordinateSystem == 1, it converts the displacement in the local coordinate system to the displacement in the Cartesian coordinate system. Here, d is a 3D vector indicating the mesh displacement before coordinate system conversion. disp is a 3D vector indicating the mesh displacement after coordinate system conversion, which is in the Cartesian coordinate system. n_vec, t_vec, and b_vec are 3D vectors (in the Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target region or target vertex. Here, d is a 3D vector indicating the mesh displacement before coordinate system conversion. disp is a 3D vector indicating the mesh displacement after coordinate system conversion, which is in the Cartesian coordinate system. n_vec, t_vec, and b_vec are 3D vectors (in the Cartesian coordinate system) corresponding to each axis of the local coordinate system of the target region or target vertex. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } When the derivation method shown by the above vector multiplication is individually expressed by a scalar, it is as follows. if (displacementCoordinateSystem == 0) { for (i = 0; i < 3; i++) {disp[i] = d[i]} } else if (displacementCoordinateSystem == 1){ for (i = 0; i < 3; i++) {disp[i] = d[0] * n_vec[i] + d[1] * t_vec[i] + d[2] * b_vec[i]} } Note that the same variable name is assigned before and after conversion with disp = d, and the value of d is updated by coordinate transformation. This may also be configured as such.

[0143] Alternatively, the following configuration may also be used. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 2){ disp = d[0] * n_vec2 + d[1] * t_vec2 + d[2] * b_vec2 } Here, n_vec2, t_vec2, and b_vec2 are 3D vectors (in the Cartesian coordinate system) corresponding to the axes of the local coordinate system of the adjacent region.

[0144] Also, the following configuration may be used. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } Here, n_vec3, t_vec3, and b_vec3 are 3D vectors (in the Cartesian coordinate system) corresponding to the axes of the local coordinate system of the target region with fluctuations suppressed. For example, the vectors of the coordinate system used for decoding are derived from the previous coordinate system and the current coordinate system as follows. system and the current coordinate system. n_vec3 = (w*n_vec3 + (WT - w)*n_vec)>>wShift t_vec3 = (w * t_vec3 + (WT - w) * t_vec) >> wShift b_vec3 = (w * b_vec3 + (WT - w) * b_vec) >> wShift Here, for example, wShift = 2, 3, 4, WT = 1 << wShift, w = 1..WT - 1. For example, when w = 3 and wShift = 3, n_vec3 = (3 * n_vec3 + 5 * n_vec) >> 3 t_vec3 = (3 * t_vec3 + 5 * t_vec) >> 3 b_vec3 = (3 * b_vec3 + 5 * b_vec) >> 3 Also, a configuration may be adopted in which selection is possible according to the value of the coordinate system conversion information displacementCoordinateSystem decoded from the encoded data as in the following configuration. if (displacementCoordinateSystem == 0) { disp = d } else if (displacementCoordinateSystem == 1){ disp = d[0] * n_vec + d[1] * t_vec + d[2] * b_vec } else if (displacementCoordinateSystem == 6){ disp = d[0] * n_vec3 + d[1] * t_vec3 + d[2] * b_vec3 } (Mesh Reconstruction) FIG. 6 is a functional block diagram showing the configuration of the mesh reconstruction unit 307. The mesh reconstruction unit 307 is composed of a mesh division unit 3071 and a mesh deformation unit 3072.

[0145] The mesh division unit 3071 divides the base mesh output from the base mesh decoding unit 303 and generates divided meshes.

[0146] Fig. 9(a) shows a part of the base mesh (triangle), and the triangle is composed of vertices v1, v2, and v3. v1, v2, and v3 are three-dimensional vectors. The mesh division unit 3071 generates a divided mesh by adding new vertices v12, v13, and v23 to the middle of each side of the triangle and outputs it (Fig. 9(b)). v12 = (v1 + v2) / 2 v13 = (v1 + v3) / 2 v23 = (v2 + v3) / 2 The following may also be used. v12 = (v1 + v2 + 1) >> 1 v13 = (v1 + v3 + 1) >> 1 v23 = (v2 + v3 + 1) >> 1 The mesh deformation unit 3072 inputs the divided mesh and the mesh displacement, generates a deformed mesh by adding the mesh displacements d12, d13, and d23, and outputs it (Fig. 9(c)). The mesh displacement is the output of the mesh displacement decoding unit 305 (coordinate system conversion unit 3055). d12, d13, and d23 are the mesh displacements corresponding to the vertices v12, v13, and v23 added by the mesh division unit 3071. v12' = v12 + d12 v13' = v13 + d13 v23' = v23 + d23 Note that d12 = disp[0][], d23 = disp[1][], and d23 = disp[3][] may also be used.

[0147] (Configuration of the 3D data encoding device according to the first embodiment) Fig. 10 is a functional block diagram showing the schematic configuration of the 3D data encoding device 11 according to the first embodiment ​​It is as follows. The 3D data encoding device 11 is composed of an atlas information encoding unit 101, a base mesh encoding unit 103, a base mesh decoding unit 104, a mesh displacement update unit 106, a mesh displacement encoding unit 107, a mesh displacement decoding unit 108, a mesh reconstruction unit 109, an attribute update unit 110, a padding unit 111, a color space conversion unit 112, an attribute encoding unit 113, a sub-mesh encoding unit 116, a multiplexing unit 114, and a mesh separation unit 115. The 3D data encoding device 11 inputs atlas information, a base mesh, a mesh displacement, a mesh, and an attribute image as 3D data, and outputs encoded data.

[0148] The atlas information encoding unit 101 encodes the atlas information and outputs an atlas information encoding stream .

[0149] The base mesh encoding unit 103 encodes the base mesh and outputs a base mesh encoding stream . As the encoding method, Draco or the like is used.

[0150] Since the base mesh decoding unit 104 is the same as the base mesh decoding unit 303, the description is omitted.

[0151] The mesh displacement update unit 106 adjusts the mesh displacement based on the (original) base mesh and the decoded base mesh, and outputs the updated mesh displacement.

[0152] The mesh displacement encoding unit 107 encodes the updated mesh displacement and outputs a mesh displacement encoding stream.

[0153] Since the mesh displacement decoding unit 108 is the same as the mesh displacement decoding unit 305, the description is omitted.

[0154] The mesh reconstruction unit 109 is the same as the mesh reconstruction unit 307, and thus the description thereof is omitted.

[0155] The attribute update unit 110 inputs the (original) mesh, the reconstructed mesh output from the mesh reconstruction unit 109 (mesh deformation unit 3072), and the attribute image, updates the attribute image to match the position (coordinates) of the reconstructed mesh, and outputs the updated attribute image.

[0156] The padding unit 111 inputs the attribute image and performs padding processing on the area where the pixel value is empty. ring processing.

[0157] The color space conversion unit 112 performs color space conversion from the RGB format to the YCbCr format.

[0158] The attribute encoding unit 113 encodes the attribute image in the YCbCr format output from the color space conversion unit 112 and outputs an attribute video stream. As the encoding method, VVC, HEVC, etc. are used. The attribute image in the YCbCr format output from the color space conversion unit 112 is encoded, and an attribute video stream is output. As the encoding method, VVC, HEVC, etc. are used.

[0159] The sub-mesh encoding unit 116 encodes the sub-mesh information of the atlas information encoding stream. encoding.

[0160] The multiplexing unit 114 multiplexes the atlas information sub-mesh encoding stream, the base mesh encoding stream, the mesh displacement encoding stream, and the attribute video stream and outputs them as encoded data. As the multiplexing method, the byte stream format, ISOBMFF, etc. are used. The multiplexing unit 114 multiplexes the atlas information sub-mesh encoding stream, the base mesh encoding stream, the mesh displacement encoding stream, and the attribute video stream and outputs them as encoded data. As the multiplexing method, the byte stream format, ISOBMFF, etc. are used.

[0161] (Operation of the mesh separation unit) The mesh separation unit 115 generates a base mesh and a mesh displacement from the mesh.

[0162] FIG. 13 is a functional block diagram showing the configuration of the mesh separation unit 115. The mesh separation unit 115 is composed of a mesh decimation unit 1151, a mesh division unit 1152, and a mesh displacement derivation unit 1153.

[0163] The mesh decimation unit 1151 generates a base mesh by decimating some vertices from the mesh.

[0164] FIG. 14(a) shows a part of the mesh, and the mesh is composed of vertices v1, v2, v3, v4, v5, v6. v1, v2, v3, v4, v5, v6 are three-dimensional vectors respectively. The mesh decimation unit 1151 generates and outputs a base mesh by decimating vertices v4, v5, v6 (FIG. 14(b)).

[0165] Similar to the mesh division unit 3071, the mesh division unit 1152 divides the base mesh to generate divided meshes (FIG. 14(c)). v4' = (v1 + v2) / 2 v5' = (v1 + v3) / 2 v6' = (v2 + v3) / 2 Based on the mesh and the divided meshes, the mesh displacement derivation unit derives and outputs displacements d4, d5, d6 of vertices v4, v5, v6 with respect to vertices v4', v5', v6' as mesh displacements (FIG. 14(d)). . d4 = v4 - v4' d5 = v5 - v5' d6 = v6 - v6' (Encoding of the base mesh) FIG. 11 is a functional block diagram showing the configuration of the base mesh encoding unit 103. The base mesh ​​The SH encoding unit 103 is composed of a mesh encoding unit 1031, a mesh decoding unit 1032, a motion information encoding unit 1033, a motion information decoding unit 1034, a mesh motion compensation unit 1035, a reference mesh memory 1036, a switch 1037, and a switch 1038. The base mesh encoding unit 103 may have a configuration including a base mesh quantization unit (not shown) after the input of the base mesh. The switches 1037 and 1038 are connected to the side where no motion compensation is performed when encoding (intra encoding) the base mesh without referring to another base mesh (for example, a base mesh that has already been encoded). Otherwise, when encoding (inter encoding) the base mesh by referring to another base mesh, it is connected to the side where motion compensation is performed.

[0166] The mesh encoding unit 1031 has an intra encoding function, intra - encodes the base mesh, and outputs a base mesh encoding stream. As the encoding method, Draco or the like is used. is used.

[0167] The mesh decoding unit 1032 is the same as the mesh decoding unit 3031, so the description is omitted.

[0168] The motion information encoding unit 1033 has an inter encoding function, inter - encodes the base mesh, and outputs a base mesh encoding stream. As the encoding method, entropy encoding such as arithmetic encoding is used.

[0169] The motion information decoding unit 1034 is the same as the motion information decoding unit 3032, so the description is omitted.

[0170] The mesh motion compensation unit 1035 is the same as the mesh motion compensation unit 3033, so the description is omitted.

[0171] The reference mesh memory 1036 is the same as the reference mesh memory 3034, so the description is omitted.

[0172] (Encoding of Mesh Displacement) FIG. 12 is a functional block diagram showing the configuration of the mesh displacement encoding unit 107. The mesh displacement encoding unit 107 is composed of a coordinate system conversion unit 1071, a conversion unit 1072, a quantization unit 1073, a binarization unit 1074, an arithmetic encoding unit 1075, a context selection unit 1076, and a context initialization unit 1077.

[0173] The coordinate system conversion unit 1071 converts the coordinate system of the mesh displacement from the Cartesian coordinate system to a coordinate system (for example, a local coordinate system) for encoding the displacement based on the value of the coordinate system conversion information displacementCoordinateSystem. Here, disp is a three-dimensional vector indicating the mesh displacement before coordinate system conversion, d is a three-dimensional vector indicating the mesh displacement after coordinate system conversion, and n_vec, t_vec, b_vec are three-dimensional vectors (in the Cartesian coordinate system) indicating the respective axes of the local coordinate system. if (displacementCoordinateSystem == 0) { d = disp } else if (displacementCoordinateSystem == 1){ d = (disp * n_vec, disp * t_vec, disp * b_vec) } The mesh displacement encoding unit 107 may update the value of displacementCoordinateSystem at the sequence level or at the picture / frame level. The initial value is 0 indicating the Cartesian coordinate system.

[0174] When updating displacementCoordinateSystem at the sequence level, the syntax of the configuration in FIG. 7 is used. For asps_vdmc_ext_displacement_coordinate_system, 0 is set for the Cartesian coordinate system and 1 is set for the local coordinate system.

[0175] ​When changing the displacement coordinate system at the picture / frame level, use the syntax of the configuration in Fig. 8. For afps_vdmc_ext_displacement_coordinate_system_enable_flag, set 1 when updating the coordinate system and 0 when not updating the coordinate system. For afps_vdmc_ext_displacement_coordinate_system, set 0 for the Cartesian coordinate system and 1 for the local coordinate system.

[0176] The conversion unit 1072 performs a conversion f (e.g., wavelet conversion) and derives the mesh displacement Tdisp after conversion. Tdisp[displSubMeshID][0][] = f(d[displSubMeshID][0][]) Tdisp[displSubMeshID][1][] = f(d[displSubMeshID][1][]) Tdisp[displSubMeshID][2][] = f(d[displSubMeshID][2][]) The quantization unit 1073 performs quantization based on the quantization scale value scale derived from the quantization parameters of each component of the mesh displacement, and derives the quantized mesh displacement Qdisp. Qdisp[displSubMeshID][0][] = Tdisp[displSubMeshID][0][] / scale[0] Qdisp[displSubMeshID][1][] = Tdisp[displSubMeshID][1][] / scale[1] Qdisp[displSubMeshID][2][] = Tdisp[displSubMeshID][2][] / scale[2] Alternatively, the Qdisp may be derived by approximating the scale value with a power of 2 as shown in the following formula. scale[i] = 1 << scale2[i] Qdisp[displSubMeshID][0][] = Tdisp[displSubMeshID][0][] >> scale2[0] Qdisp[displSubMeshID][1][] = Tdisp[displSubMeshID][1][] >> scale2[1] Qdisp[displSubMeshID][2][] = Tdisp[displSubMeshID][2][] >> scale2[2] The binary quantization unit 1074 encodes the quantized mesh displacement Qdisp, which is a multi-valued signal, into a binary signal. The binary signal may be a k-th order exponential Golomb code.

[0177] The arithmetic coding unit 1075 arithmetically encodes the binary signal and outputs a mesh displacement encoded stream.

[0178] The context selection unit 1076 is the same as the context selection unit 3056, so the description is omitted.

[0179] Note that a static context with a fixed probability without context update is set as ctxStatic. The encoding of the syntax element indicated by ctxStatic may be encoded without using a context. encode(ctxStatic) may be encode_bypass(), and dedicated bypass processing may be used.

[0180] The context initialization unit 1077 is the same as the context initialization unit 3057, so the description is omitted. Here, an example of using a context is described, but some syntax elements may be bypass-encoded without using a context. In a configuration for bypass encoding, the effect of reducing the context memory and processing amount is achieved. For example, the syntax element dismu_coeff_abs_level_rem is bypassed without using a context. It may be S-encoded. By bypass-encoding these syntax elements, it is possible to reduce the context memory and processing volume while maintaining the encoding efficiency.

[0181] The mesh displacement encoding unit 107 encodes the mesh displacement Qdisp by the following process. for (k = 0; k < numDim; k++) { / / dimension (component) loop if (!lastSig) continue dispOffset = 0 for (b = 0; b <numLOD; b++) { / / Level of Detail loop, block loop numBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numBlocks; s++) { / / subblock loop / / encode dismu_nz_subBlock encode(dismu_nz_subBlock[k][b][s], ctxCodedSubBlock[st][b][k]) for (v = 0; v < subBlockSize; v++) { / / coefficient loop within subblock / / encode dismu_coeff_abs_level_gt0 d = Qdisp[dispOffset + s * subBlockSize + v][k] encode(d != 0, ctxCoeffGtN[st][b][0][k]) if (!d) continue / / encode dismu_coeff_sign encode(d < 0, ctxStatic) d = abs(d) - 1 / / encode dismu_coeff_abs_level_gt1 encode(d != 0, ctxCoeffGtN[st][b][1][k]) if (!d) continue d = abs(d) - 1 / / encode dismu_coeff_abs_level_gt2 encode(d != 0, ctxCoeffGtN[st][b][2][k]) if (!d) continue d = abs(d) - 1 / / encode dismu_coeff_abs_level_gt3 encode(d != 0, ctxCoeffGtN[st][b][3][k]) if (!d) continue / / encode dismu_coeff_abs_level_rem encodeExpGolomb(--d, ctxCoeffRemPrefix[st][b][k]) } } dispOffset += dispCount[b] } } In the suspect code, continue means skipping the subsequent operations and jumping to the beginning of the loop (the next iteration). Here, encode() and encodeExpGolomb() are functions that take a value and the corresponding context as arguments and arithmetically encode each 1-bit value and the binary sequence of the k-th Golomb code. dispCount[b] is the number of mesh displacements at the detailed level b. lastSig is a flag indicating whether the current coefficient is the last non-zero coefficient within the subblock in the scan order. lastSig = 0 means the current coefficient is Indicates that it is not the last non-zero coefficient in the sub-block in scan order. lastSig = 1 indicates that the current coefficient is the last non-zero coefficient in the sub-block in scan order. Indicates that the current coefficient is the last non-zero coefficient in the sub-block in scan order.

[0182] Alternatively, the mesh displacement Qdisp may be encoded by the following process. for (k = 0; k < numDim; k++) { / / dimension (component) loop if (!lastSig) continue dispOffset = 0 for (b = 0; b <numLOD; b++) { / / Level of Detail loop, block loop numBlocks = dispCount[b] / subBlockSize + 1 for (s = 0; s < numBlocks; s++) { / / subblock loop / / encode dismu_nz_subBlock encode(dismu_nz_subBlock[k][b][s], ctxCodedSubBlock[st][b][k]) for (v = 0; v < subBlockSize; v++) { / / coefficient loop within subblock / / encode dismu_coeff_abs_level_gt0 d = Qdisp[dispOffset + s * subBlockSize + v][k] encode(d != 0, ctxCoeffGtN[st][b][0][k]) if (!d) continue / / encode dismu_coeff_sign encode(d < 0, ctxStatic) N = 1 maxGtN = 3 while (N <= maxGtN) { d = abs(d) - 1 / / encode dismu_coeff_abs_level_gtN (N=1..maxGtN) encode(d != 0, ctxCoeffGtN[st][b][N][k]) if (!d) break N++ } if (d) { / / encode dismu_coeff_abs_level_rem encodeExpGolomb(--d, ctxCoeffRemPrefix[st][b][k]) } } } dispOffset += dispCount[b] } } Note that maxGtN is not limited to 3. For example, when maxGtN = 2, it may be configured to encode the syntax elements dismu_coeff_abs_level_gt0, dismu_coeff_abs_level_gt1, dismu_coeff_abs_level_gt2. Or when maxGtN = 4, it may be configured to encode the syntax elements dismu_coeff_abs_level_gt0, dismu_coeff_abs_level_gt1, dismu_coeff_abs_level_gt2, dismu_coeff_abs_level_gt3, dismu_coeff_abs_level_gt4.

[0183] As described above, an embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist of the present invention.

[0184] 〔Application Example〕 The above-mentioned 3D data encoding device 11 and 3D data decoding device 31 transmit, receive, record, and transmit 3D data. The 3D data can be installed in various devices that record and play back the 3D data. The 3D data may be natural 3D data captured by a camera or the like, or artificial 3D data (including CG and GUI) generated by a computer or the like.

[0185] The present invention is not limited to the above-described embodiment, and various modifications are possible within the scope of the claims. In other words, the technical scope of the present invention also includes embodiments obtained by combining technical means that are appropriately modified within the scope of the claims. [Industrial Applicability]

[0186] The embodiments of the present invention can be suitably applied to a 3D data decoding device that decodes encoded data in which 3D data is encoded, and a 3D data encoding device that generates encoded data in which 3D data is encoded. Also, the embodiments of the present invention can be suitably applied to the data structure of encoded data generated by the 3D data encoding device and referenced by the 3D data decoding device. [Explanation of symbols]

[0187] 11 3D data encoding device 101 Atlas Information Encoding Unit 103 Base mesh coding unit 1031 Mesh coding unit 1032 Mesh Decoding Unit 1033 Motion information encoding unit 1034 Motion information decoding unit 1035 Mesh motion compensation unit 1036 Reference Mesh Memory 1037 Switch 1038 Switch 1039 Skip Coding 104 Base mesh decoding unit 106 Mesh displacement update section 107 Mesh Displacement Encoding Unit 1071 Coordinate transformation unit 1072 Transformation unit 1073 Quantization unit 1074 Binarization unit 1075 Arithmetic symbolization unit 1076 Context selection unit 1077 Context initialization unit 108 Mesh displacement decoding unit 109 Mesh reconstruction unit 110 Attribute update unit 111 Padding unit 112 Color space conversion unit 113 Attribute encoding unit 114 Multiplexing unit 115 Mesh separation unit 1151 Mesh thinning unit 1152 Mesh division unit 1153 Mesh displacement derivation unit 116 Sub-mesh encoding unit 21 Network 31 3D data decoding device 301 Demultiplexing unit 302 Atlas information decoding unit 303 Base mesh decoding unit 3031 Mesh decoding unit 3032 Motion information decoding unit 3033 Mesh motion compensation unit 3034 Reference mesh memory 3035 Switch 3036 Switch 3037 Skip decoding unit 305 Mesh displacement decoding unit 3051 Arithmetic decoding unit 3052 Multi-value conversion unit 3053 Inverse quantization unit 3054 Inverse transformation unit 3055 Coordinate transformation unit 3056 Context selection unit 3057 Context initialization unit 307 Mesh reconstruction unit 306 Attribute Decoder 3071 Mesh Division Unit 3072 Mesh Deformation Unit 308 Color Space Conversion Unit 309 Sub-Mesh Decoder 41 3D Data Display Device

Claims

1. In a 3D data decoding device that decodes mesh data or point cloud data, a sub-mesh decoding unit that decodes sub-mesh information from encoded data in which the mesh data or point cloud data is encoded, a base-mesh decoding unit that decodes a base mesh from the encoded data and the sub-mesh information, a mesh displacement decoding unit that decodes a mesh displacement from the encoded data and the sub-mesh information, and a mesh reconstruction unit that decodes a mesh from the decoded base mesh and the mesh displacement. In the mesh displacement decoding unit, the mesh displacement is decoded from the encoded data using the sub-mesh information decoded by the sub-mesh decoding unit. A 3D data decoding device characterized by this.

2. The 3D data decoding device according to claim 1, wherein the sub-mesh information includes a flag indicating whether the sub-mesh of the base mesh and the sub-mesh of the mesh displacement correspond to each other.

3. In a 3D data encoding device that encodes mesh data or point cloud data, a sub-mesh encoding unit that encodes sub-mesh information, a base-mesh encoding unit that encodes a base mesh using the sub-mesh information, and a mesh displacement encoding unit that encodes a mesh displacement using the sub-mesh information. In the mesh displacement encoding unit, the mesh displacement is encoded using the sub-mesh information encoded by the sub-mesh encoding unit. A 3D data encoding device characterized by this.

4. The 3D data encoding device according to claim 3, wherein the sub-mesh information includes a flag indicating whether the sub-mesh of the base mesh and the sub-mesh of the mesh displacement correspond to each other.

Images