3D data decoding device and 3D data encoding device

Abstract

The purpose of the present invention is to encode / decode a mesh shift image into an image in a 4: 2: 0 format in encoding / decoding of 3D data using a moving image encoding scheme, reduce distortion caused by encoding, and encode / decode 3D data with high quality. A 3D data decoding device is provided with: a video decoding unit for decoding a mesh shift image decoded from a geometric video stream in which the unit type of encoded data is V3CGVD; and a displacement demapping unit that derives a mesh displacement for each position pos and component compIdx from the mesh displacement image, the displacement demapping unit deriving the mesh displacement by deriving a Y coordinate of the geometric image by a product of a height and a variable from 0 to a dimensionality value-1 representing geometry when the geometric image is 4: 2: 0.

Description

Technical Field

[0001] Embodiments of the present invention relate to a 3D data encoding device and a 3D data decoding device. Background Art

[0002] In order to efficiently transmit or record 3D data, there is a 3D data encoding device that converts 3D data into a two-dimensional image, encodes the image using a motion image encoding method, and generates encoded data, and a 3D data decoding device that decodes the two-dimensional image from the encoded data and reconstructs the 3D data.

[0003] As specific 3D data encoding methods, for example, MPEG-I's ISO / IEC 23090-5 V3C (Visual Volumetric Video-based Coding) and V-PCC (Video-based Point Cloud Compression). V3C can encode and decode point clouds consisting of point positions and attribute information. In addition, it can also be used for encoding and decoding of multi-viewpoint images and mesh images through ISO / IEC 23090-12 (MPEG Immersive Video (MPEG Immersive Video: MIV)) and the ISO / IEC 23090-29 (Video-based Dynamic Mesh Coding (V-DMC)) that is being standardized. The latest draft file of the V-DMC method is disclosed in non-patent document 1.

[0004] In these 3D data encoding methods, geometry and attributes constituting 3D data are encoded / decoded as images using moving picture encoding methods such as H.265 / HEVC (High Efficiency Video Coding) and H.266 / VVC (Versatile Video Coding).

[0005] In the case of a point group, the geometric image is the depth to the projection plane, and the attribute image is the image obtained by projecting the attributes onto the projection plane.

[0006] 3D data (mesh data) as described in Non-Patent Document 1 is composed of a basic mesh, mesh displacement, mesh displacement arrangement, and texture mapping image. The basic mesh can use a vertex encoding method such as Draco, the geometric image is a mesh displacement image that is two-dimensionalized by mesh displacement, and the attribute image is a texture mapping image. As described above, they can be encoded / decoded using a moving image encoding method such as HEVC and VVC. Prior art literature Non-patent literature Non-patent document 1

[0007] WD 3.0ofV-DMC(MDS22775_WG07_N00611_d9), ISO / IEC JTC 1 / SC 29 / WG 7N0611, April 2023 Summary of the invention Problem that the invention aims to solve

[0008] The method of encoding / decoding a grid-displaced image into an image in a 4:2:0 format in Non-Patent Document 1 has a problem that the scalability function and the error-resistance function corresponding to the capability of the decoding device are insufficient.

[0009] The purpose of the present invention is to encode / decode a grid displacement image into an image in YCbCr4:2:0 format in encoding / decoding of 3D data using a motion image encoding method, reduce distortion caused by encoding, and encode / decode 3D data with high quality. Technical Solution

[0010] In order to solve the above-mentioned problems, a 3D data decoding device of one embodiment of the present invention decodes encoded data, and the 3D data decoding device is characterized in that it comprises: an animation decoding unit, which decodes a grid displacement image decoded from a geometric animation stream whose unit type is V3C_GVD of the above-mentioned encoded data; and a displacement demapping unit, which derives a grid displacement QDisp[compIdx][pos] of each position pos and component compIdx from a grid displacement image gFrame[compIdx][y][x] having an x, y position and a component compIdx, and the above-mentioned displacement demapping unit derives the above-mentioned grid displacement by deriving the Y coordinate of the geometric image by multiplying a variable compIdx whose value ranges from 0 to -1 representing the geometric dimension DisplacementDim and a height dispCompHeight. Beneficial Effects

[0011] According to an aspect of the present invention, distortion caused by encoding of a grid displacement image can be reduced, and 3D data can be encoded / decoded with high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a schematic diagram showing the configuration of a 3D data transmission system according to this embodiment. Figure 2 This is a diagram showing the hierarchical structure of the data of the coded stream. Figure 3It is a functional block diagram showing a schematic configuration of the 3D data decoding device 31 . Figure 4 It is a functional block diagram showing the structure of the basic grid decoding unit 303. Figure 5 It is a functional block diagram showing the structure of the grid displacement decoding unit 305. Figure 6 It is a functional block diagram showing the configuration of the mesh reconfiguration unit 307 . Figure 7 This is an example of a syntax for transmitting the structure of coordinate transformation parameters and displacement mapping parameters at the sequence level (ASPS). Figure 8 This is an example of a syntax for transmitting coordinate conversion parameters and displacement mapping parameters at the picture / frame level (AFPS). Fig. 9 is an example of the grid displacement image gFrame being divided into segments (slices) of geometric dimensions. Fig.10 is an example of a grid displacement image gFrame being split into segments (slices) of geometric dimensions and LODs. Fig.11 This is a diagram for explaining the operation of the mesh reconfiguration unit 307 . Fig.12 It is a functional block diagram showing a schematic configuration of the 3D data encoding device 11 . Fig.13 It is a functional block diagram showing the structure of the basic grid encoding unit 103. Fig.14 It is a functional block diagram showing the structure of the grid displacement encoding unit 107. Fig.15 It is a functional block diagram showing the configuration of the mesh separation unit 115 . Fig.16 It is a diagram for explaining the operation of the mesh separation unit 115 . Fig.17 This is an example of a syntax for transmitting the structure of the displacement slice segmentation parameters for each detail level at the sequence level (ASPS). Fig.18 This is an example of a syntax for transmitting displacement slice segmentation parameters for each detail level at the picture / frame level (AFPS). Fig.19 is an example of the syntax for the displacement slice split parameter. Fig. 20 is an example of the syntax for the displacement slice split parameter. Fig.21 is an example of the syntax for the displacement slice split parameter. Fig. 22is an example of the syntax for the displacement slice split parameter. Fig.23 is an example of the syntax for the displacement slice split parameter. DETAILED DESCRIPTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0014] Figure 1 It is a schematic diagram showing the configuration of a 3D data transmission system 1 according to the present embodiment.

[0015] The 3D data transmission system 1 is a system that transmits a coded stream obtained by encoding the 3D data to be encoded, decodes the transmitted coded stream and displays the 3D data. The 3D data transmission system 1 includes a 3D data encoding device 11, a network 21, a 3D data decoding device 31 and a 3D data display device 41.

[0016] The 3D data T is input to the 3D data encoding device 11 .

[0017] The network 21 transmits the coded stream Te generated by the 3D data encoding device 11 to the 3D data decoding device 31. The network 21 is the Internet, a wide area network (WAN), a small network (LAN) or a combination thereof. The network 21 is not necessarily limited to a two-way communication network, but may also be a one-way communication network that transmits broadcast waves such as terrestrial digital broadcasting and satellite broadcasting. In addition, the network 21 may also be replaced by a storage medium such as a DVD (Digital Versatile Disc: Digital Versatile Disc, registered trademark) or a BD (Blu-ray Disc: Blu-ray Disc, registered trademark) that records the coded stream Te.

[0018] The 3D data decoding device 31 decodes the encoded stream Te transmitted by the network 21 respectively, and generates one or more decoded 3D data Td.

[0019] The 3D data display device 41 displays all or part of one or more decoded 3D data Td generated by the 3D data decoding device 31. The 3D data display device 41 includes, for example, a display device such as a liquid crystal display, an organic EL (Electro-luminescence) display, etc. As the form of the display, a fixed type, a mobile type, an HMD (Head Mounted Display), etc. can be listed. In addition, when the 3D data decoding device 31 has a high processing capability, a high-quality image is displayed, and when it has only a low processing capability, an image that does not require a high processing capability or a high display capability is displayed. <operator>

[0020] The operators used in this specification are described below. >> is a right shift, << is a left shift, & is a bitwise AND, | is a bitwise OR, |= is an OR substitution operator, and || represents a logical AND. x?y:z is a three-term operator that takes y when x is true (other than 0) and takes z when x is false (0). y..z represents the set of integers from y to z. <Structure of coded stream Te>

[0021] Before describing in detail the 3D data encoding device 11 and the 3D data decoding device 31 of the present embodiment, the data structure of the encoded stream Te generated by the 3D data encoding device 11 and decoded by the 3D data decoding device 31 will be described.

[0022] Figure 2 The diagram shows the hierarchical structure of data in the coded stream Te. The coded stream Te has a data structure of either a V3C sample stream or a V3C unit stream. The V3C sample stream includes a sample stream header and a V3C unit. The V3C unit stream includes a V3C unit.

[0023] The V3C unit includes a V3C unit header and a V3C unit payload. The starting point of the V3C unit header is the unit type (Unit Type) which is an ID indicating the type of the V3C unit and takes the value indicated by a label such as V3C_VPS, V3C_AD, V3C_AVD, V3C_GVD, or V3C_OVD.

[0024] When the unit type is V3C_VPS (Video Parameter Set), the V3C unit includes a V3C parameter set.

[0025] When the unit type is V3C_AD (Atlas Data), the V3C unit includes a VPS ID, an atlas ID, a sample stream nal header, and a plurality of NAL units.

[0026] The NAL unit includes a NAL unit type, a layer ID (layerID), a temporary ID (TemporalID), and RBSP (Raw byte sequence payload).

[0027] NAL units are identified by NAL unit types, including ASPS (Atlas Sequence Parameter Set), AAPS (Atlas Adaptation Parameter Set), ATL (Atlas Tile layer), SEI (Supplemental Enhancement Information), etc.

[0028] ATL includes an ATL header and an ATL data unit. The ATL data unit includes patch information data (patchinformation data) and other information such as the location and size of the patch.

[0029] SEI includes a payload type (payload Type) indicating the type of SEI, a payload size (payload Size) indicating the size (number of bytes) of SEI, and sei_payload indicating the data of SEI.

[0030] When the unit type is V3C_AVD (Attribute Video Data, Attribute Data), the V3C unit includes VPS ID, atlasID, attrIdx of the attribute image ID, partIdx of the partition ID, mapIdx of the map ID, auxFlag indicating whether it is auxiliary data (Auxiliary Date), and video stream (videostream). Video stream represents data encoded by HEVC, VVC, etc. It corresponds to the texture image in V-DMC.

[0031] When the unit type is V3C_GVD (Geometory Video Data, Geometry Data), the V3C unit includes VPS ID, atlasID, mapIdx, auxFlag, and video stream. In V-DMC, it corresponds to the grid displacement.

[0032] When the unit type is V3C_OVD (Occumancy Video Data, occupancy data), the V3C unit includes VPS ID, atlas ID, and video stream.

[0033] When the cell type is V3C_MD (Mesh data), the V3C cell includes VPS ID, atlas ID, and mesh_payload, which corresponds to the basic mesh in V-DMC. (Configuration of 3D Data Decoding Device According to First Embodiment)

[0034] Figure 3 3D data decoding device 31 of the first embodiment is a functional block diagram showing a schematic structure. 3D data decoding device 31 is composed of a demultiplexing unit 301, a graph information decoding unit 302, a basic grid decoding unit 303, a grid displacement decoding unit 305, a grid reconstruction unit 307, an attribute decoding unit 306, and a color space conversion unit 308. 3D data decoding device 31 inputs coded data of 3D data and outputs graph information, a grid, and an attribute image.

[0035] The demultiplexing unit 301 inputs the coded data multiplexed in the byte stream format, ISOBMFF (ISO Base Media File Format), etc., performs demultiplexing, and outputs the graph information coding stream (Atlas data stream, NAL unit of V3C_AD), the basic grid coding stream (grid_payload of V3C_MD), the grid displacement coding stream (video stream of V3C_GVD), and the attribute animation stream (video stream of V3C_AVD).

[0036] The spectral information decoding unit 302 inputs the spectral information encoded stream output from the demultiplexing unit 301, and decodes the spectral information.

[0037] The basic mesh decoding unit 303 decodes the basic mesh encoding stream encoded by vertex encoding (3D data compression encoding method, such as Draco), and outputs a basic mesh. The basic mesh will be described later.

[0038] The grid displacement decoding unit 305 decodes the geometric animation stream (grid displacement coding stream) encoded by VVC, HEVC, etc., and outputs the grid displacement. The type of codec used for encoding is represented by ptl_profile_codec_group_idc obtained by decoding the V3C parameter set of the encoded data. In addition, it can also be represented by the Four CC code (4 character codes, 4 CC codes) shown in gi_geometry_codec_id[atlasID] of the V3C parameter set. gi_geometry_codec_id[atlasID] represents the index corresponding to the codec ID of the decoder used for decoding the geometric animation stream in the atlas ID. The set representing the correspondence between the codec ID (ccm_codec_id) and its 4CC code (ccm_codec_4cc[ccm_codec_id]) can also be transmitted through other codec mapping SEI (component_codec_mapping SEI). The codec can decode the grid displacement in units of segments (slices) obtained by further dividing the frame. HEVC and VVC can divide frames into slices. Slices are encoded in Coded Tree Units (CTU). It should be noted that instead of using slices, sub-pictures or tiles can be used as segmentation. These sub-pictures, tiles, and slices can be decoded independently, so only part of the frame can be decoded instead of the entire frame. When using sub-pictures and tiles, the slices are replaced with sub-pictures or tiles.

[0039] The mesh reconstruction unit 307 inputs the basic mesh and the mesh displacement, and reconstructs the mesh in the 3D space.

[0040] The attribute decoding unit 306 decodes the attribute animation stream encoded by VVC, HEVC, etc., and outputs an attribute image in YCbCr format. The attribute image can be a texture image expanded on the UV axis (a texture mapping image converted in a UV atlas manner). The type of codec used for encoding is represented by ptl_profile_codec_group_idc obtained by decoding the V3C parameter set of the encoded data. In addition, it can also be represented by the four CC codes represented by ai_geometry_codec_id[atlasID] of the V3C parameter set. ai_geometry_codec_id[atlasID] represents the index corresponding to the codec ID of the decoder used to decode the attribute animation stream in the atlas ID.

[0041] The color space conversion unit 308 performs color space conversion on the attribute image from the YCbCr format to the RGB format. It should be noted that the attribute animation stream encoded in the RGB format may be decoded and the color space conversion may be omitted. (Decoding of the basic grid)

[0042] Figure 4 3031, a motion information decoding unit 3032, a mesh motion compensation unit 3033, a reference mesh memory 3034, a switch 3035, and a switch 3036. The basic mesh decoding unit 303 may include a basic mesh inverse quantization unit (not shown) before outputting the basic mesh. When the basic mesh of the decoding object is encoded (intra-frame encoding) without referring to other basic meshes (for example, an already encoded / decoded basic mesh), the switch 3035 and the switch 3036 are connected to the side where motion compensation is not performed. Otherwise, when the basic mesh of the decoding object is encoded (inter-frame encoding) with reference to other basic meshes, the switch 3035 and the switch 3036 are connected to the side where motion compensation is performed. When motion compensation is performed, the vertex coordinates of the object are derived with reference to the decoded vertex coordinates and motion information.

[0043] The trellis decoding unit 3031 decodes the basic trellis coding stream after intra-frame coding, and outputs the basic trellis. As the coding method, Draco, edge breaker, etc. are used.

[0044] The motion information decoding unit 3032 decodes the inter-coded basic mesh coded stream, and outputs motion information for each vertex of a reference mesh described later. Entropy coding such as arithmetic coding is used as a coding method.

[0045] The mesh motion compensation unit 3033 performs motion compensation on each vertex of the reference mesh input from the reference mesh memory 3034 based on the motion information, and outputs a mesh after motion compensation.

[0046] The reference grid memory 3034 is a memory that stores the decoded grid for reference in subsequent decoding processing. (Decoding of grid displacement)

[0047] Figure 5 305 is a functional block diagram showing the structure of the grid displacement decoding unit 305. The grid displacement decoding unit 305 is composed of a displacement demapping unit 3052 (image splitting unit, displacement decoding unit), an inverse quantization unit 3053, an inverse conversion unit 3054, and a coordinate system conversion unit 3055. In addition, the displacement demapping unit can be referred to as a displacement mapping unit. Figure 5As shown, the grid displacement decoding unit 305 may be configured to further include the animation decoding unit 3051, or the grid displacement decoding unit 305 may be configured to not include the animation decoding unit 3051, and the decoding of the displacement image (displacement arrangement) may be performed using the 3D data decoding device 31. In addition, the grid displacement decoding unit 305 may be configured to not include the inverse quantization unit 3053, and the image quality control may be performed only by the 3D data decoding device 31.

[0048] The atlas information decoding unit 302 decodes the coordinate system conversion information displacementCoordinateSystem (asps_vdmc_ext_displacement_coordinate_system, afps_vdmc_ext_displacement_coordinate_system) representing the coordinate system from the encoded data. In addition, the slice segmentation information (displacement slice segmentation parameter, displacement segmentation parameter, displacement slice segmentation flag, displacement segmentation flag) of the grid displacement can also be decoded. The slice segmentation information can be displacementSliceFlag indicating whether to divide into segments, the slice segmentation type indicating the segmentation method, and displacementSliceType (asps_vdmc_ext_displacement_slice_type, afps_vdmc_ext_displacement_slice_type). Moreover, the slice segmentation information may include the component height dispCompHeight. In addition, the slice segmentation information may include the syntax element dispPos[lodIdx] representing the starting position of each LOD of the grid displacement or the number of grid displacements of each LOD dispCount[lodIdx]. In addition, the index dispCountIdx[lodIdx] indicating the number of grid displacements may also be included. In addition, the slice segmentation information may include the size ctuSize of the aligned block of the slice or the index ctuSizeIdx indicating ctuSize. The slice segmentation type is a parameter indicating the category of the slice segmentation. The component height is a parameter indicating the height of the image corresponding to each component (e.g., n, t, b) of the grid displacement three-dimensional vector. In addition, the components of n (normal), t (tangent), and b (bitangent) are also referred to as components.

[0049] The slice segmentation information may be a syntax element displacementSliceFlag (displacementSliceEnabledFlag, displacementSliceUsedFlag) indicating that the displacement is segmented into component units (Componentunit) by slices. Here, the displacement is segmented into component units by slices, which means that when the displacement is encoded using AVC, HEVC, VVC, etc., at least the first component (normal) of normal, tangent, and bitangent is encoded by a slice different from the second component (tangent) and the third component (bitangent). For example, displacementSliceFlag==1 may be a case where the picture is divided into two, slice 0 is normal, and slice 1 is tangent and bitangent. displacementSliceFlag==0 is a case where the picture is not segmented or the segmentation is not explicitly indicated. In addition, for example, displacementSliceFlag==1 may be a case where the picture is segmented into three, slice 0 is normal, slice 1 is tangent, and slice 2 is bitangent. In addition, it can also be defined as slicing at the boundary of the displacement component. In addition, as described later, it is also possible to indicate the case where a picture is divided into three slices when asps_vdmc_ext_displacement_slice_type is used to divide the picture into two slices. In this way, by using explicit syntax elements to indicate the pre-division into slices, the 3D data decoding device only decodes specific slices, thereby enabling scalable decoding corresponding to the decoding capability and power consumption. It should be noted that slice segmentation is listed here, but when supporting segments (decoding units) other than slices such as HEVC and VVC, slices can be replaced with tiles or sub-pictures. The same applies below.

[0050] The slice segmentation information may also be a flag indicating that rectangular segmentation of the same size is performed in multiple frames consecutive in the time direction, and inter-picture prediction is performed only in the corresponding rectangular area. For example, it may be a tile segmentation that utilizes the motion constrained tile set (MCTS) of HEVC. Alternatively, a segmentation other than limiting the prediction from sub-pictures consecutive in the time direction of VVC may be used. The reference sub-picture is a sub-picture with a configuration position that is restricted for temporal prediction. Generally, slice segmentation is characterized in that it has independence from prediction and filtering of slices other than the slice within the picture, but the sub-picture is independent in a manner that has no dependency not only in the spatial direction but also in the temporal direction. The information related to the above-mentioned HEVC tile or VVC sub-picture is called spatiotemporally independent slice segmentation information.

[0051] It should be noted that a gating flag may be set separately, and each coordinate system conversion information is decoded only when the gating flag is 1. The gating flag is, for example, afps_vdmc_ext_overriden_flag. In addition, a gating flag may be set for slice segmentation information, and the slice segmentation information is decoded only when the gating flag is 1. The gating flag is, for example, afps_vdmc_ext_displacement_slice_alignment_flag. (Coordinate System)

[0052] The coordinate system of the mesh displacement (3D vector) uses the following two coordinate systems.

[0053] Cartesian coordinate system (canonical): An orthogonal coordinate system commonly defined in the entire 3D space. (X, Y, Z) coordinate system. An orthogonal coordinate system whose direction remains unchanged at the same time (within the same frame, the same tile).

[0054] Local coordinate system (local): An orthogonal coordinate system defined for each area or each vertex in 3D space. An orthogonal coordinate system whose direction can change at the same time (within the same frame, within the same tile). Normal (D), tangent (U), bi-tangent (V) coordinate systems. That is, an orthogonal coordinate system including a first axis (D) represented by a normal vector n_vec at a vertex (a face containing a vertex) and a second axis (U) and a third axis (V) represented by two tangent vectors t_vec and b_vec orthogonal to the normal vector n_vec. n_vec, t_vec, and b_vec are three-dimensional vectors. The (D, U, V) coordinate system can also be called the (n, t, b) coordinate system. (Decoding / derivation of sequence level control parameters)

[0055] Here, the control parameters used in the grid displacement decoding unit 305 are described.

[0056] Figure 7 This is an example of the syntax for transmitting coordinate system conversion parameters and displacement slice segmentation parameters in sequence-level ASPS.

[0057] ASPS (Atlas Sequence Parameter Set or Atlas sequence mesh information) is one of the NAL units of atlas information, including syntax elements used to encode atlas sequences. In ASPS, coordinate system conversion parameters and displacement slice segmentation parameters are transmitted through asps_vdmc_extension() syntax. The semantics of each field are described as follows.

[0058] asps_vdmc_ext_displacement_coordinate_system: indicates the coordinate system conversion information of the coordinate system of the grid displacement. When the value is equal to a specified first value (for example, 0), it indicates the Cartesian coordinate system. When the value is equal to another second value (for example, 1), it indicates the local coordinate system.

[0059] asps_vdmc_ext_displacement_slice_type: indicates the type of slicing of each component of the mesh displacement. The meaning of the value is as follows. 0: Make an appointment. 1: Assign the first component of the mesh displacement (e.g., normal, n) to the first slice of the geometry animation stream, the second component (e.g., tangent, t) to the second slice of the geometry animation stream, and the third component of the geometry animation stream (e.g., bi-tangent, b) to the third slice. 2: Assign the above first component of the grid displacement (eg, n) to the first slice, and assign the above second and third components (eg, t, b) to the second slice. 3: Make an appointment.

[0060] As described above, it may also be as follows. 0: Do not specify that the mesh displacement is not divided by slices or is divided by slices. 1: Assign the first component of the mesh displacement (e.g., normal) to the first slice of the geometry animation stream, the second component (e.g., tangent) to the second slice of the geometry animation stream, and the third component (e.g., bitangent) to the third slice of the geometry animation stream.

[0061] As described above, it may also be as follows. 0: Do not specify that the mesh displacement is not divided by slices or is divided by slices. 1: Assign the first component of the grid displacement (eg, normal) to the first slice, and assign the second and third components (eg, tangent, bitangent) to the second slice.

[0062] asps_vdmc_ext_displacement_component_height: indicates the height of the image corresponding to each component (eg, Y, U, V) of the grid displacement. (Decoding / exporting of picture / frame level control parameters)

[0063] Figure 8 This is an example of the syntax for transmitting coordinate system transformation parameters and displacement slice segmentation parameters by AFPS at picture / frame level. AFPS (Atlas Frame Parameter Set or Atlas frame mesh information) is one of the NAL units of atlas information, including syntax elements applied to coded atlas frames. In AFPS, coordinate system transformation parameters and displacement slice segmentation parameters are transmitted through the afps_vdmc_extension() syntax. The semantics of each field is described below.

[0064] afps_vdmc_ext_overriden_flag: A flag indicating whether to update the coordinate system of the mesh displacement. When this flag is true, the coordinate system of the mesh displacement is updated based on the value of afps_vdmc_ext_displacement_coordinate_system described later. When this flag is false, the coordinate system of the mesh displacement is not updated.

[0065] afps_vdmc_ext_displacement_coordinate_system: Coordinate system conversion information of the coordinate system representing the grid displacement. When the value is equal to the first value (e.g., 0), the Cartesian coordinate system is represented. When the value is equal to the second value (e.g., 1), the local coordinate system is represented. When the syntax element is not present, the value is estimated as the value obtained by decoding according to ASPS, and the default coordinate system is set to the coordinate system represented by ASPS.

[0066] afps_vdmc_ext_displacement_slice_alignment_update_flag: This is a flag indicating whether to update the slice segmentation information of the grid displacement. When this flag is equal to true (e.g., 1), the slice segmentation information of the grid displacement is updated based on the values ​​of asps_vdmc_ext_displacement_slice_type and asps_vdmc_ext_displacement_component_height described later. When this flag is equal to false (e.g., 0), the slice segmentation information of the grid displacement is not updated.

[0067] afps_vdmc_ext_displacement_slice_type: indicates the type of slice segmentation for each component of the grid displacement. The meaning of the value is as described by the semantics of asps_vdmc_ext_displacement_slice_type. In the absence of afps_vdmc_ext_displacement_slice_type, afps_vdmc_ext_displacement_slice_type is set equal to asps_vdmc_ext_displacement_slice_type.

[0068] afps_vdmc_ext_displacement_component_height: represents the height of the image corresponding to each component (e.g., n, t, b) of the grid displacement 3D vector. If afps_vdmc_ext_displacement_component_height is not present, afps_vdmc_ext_displacement_component_height is set equal to asps_vdmc_ext_displacement_component_height.

[0069] The grid displacement decoding unit 305 derives the coordinate system conversion information displacementCoordinateSystem as follows. displacementCoordinateSystem=afps_vdmc_ext_displacement_coordinate_system

[0070] It should be noted that, when afps_vdmc_ext_displacement_coordinate_system does not appear, afps_vdmc_ext_displacement_coordinate_system is set equal to asps_vdmc_ext_displacement_coordinate_system. (Derivation of displacement slice segmentation parameters)

[0071] The grid displacement decoding unit 305 derives the displacement slice segmentation parameters displacementSliceType and dispCompHeight as follows. displacementSliceType=afps_vdmc_ext_displacement_slice_typedispCompHeight=afps_vdmc_ext_displacement_component_heihht

[0072] That is, when the syntax element of the displacement slice partitioning parameter appears in AFPS, the value of the corresponding syntax element of AFPS is used, and when it does not appear, the value of the corresponding syntax element of ASPS is used.

[0073] It should be noted that, instead of displacementSliceType, a slice segmentation flag displacementSliceFlag indicating whether to use segmentation may be decoded from Atlas frame mesh information. <Syntax example of displacement slice segmentation parameter>

[0074] Fig.19 This is an example of the syntax of the displacement slice segmentation parameter. Here, the variable name encoded / decoded in ASPS (variables starting with the prefix asps) is used for explanation, but it can also be a variable name encoded / decoded in AFPS units (variables starting with the prefix afps).

[0075] asps_vdmc_ext_subdivision_method (afps_vdmc_ext_subdivision_method) indicates the displacement division method. When the value is 0, displacement division is not performed. When the value is 1, displacement division is performed. When afps_vdmc_ext_subdivision_method is not present, it can be set as follows. afps_vdmc_ext_subdivision_method=asps_vdmc_ext_subdivision_metod

[0076] asps_vdmc_ext_subdivision_iteration_count (afps_vdmc_ext_subdivision_iteration_count) indicates the number of displacement divisions. Decoding is performed when asps_vdmc_ext_subdivision_method is other than 0. The number of displacement divisions corresponds to the number of LODs lodCount. When asps_vdmc_ext_subdivision_iteration_count=0, 1, 2, the number of LODs is 1, 2, 3, respectively.

[0077] asps_vdmc_ext_displacement_coordinate_system indicates the coordinate system as described above.

[0078] asps_vdmc_ext_packing_method indicates the packing method when the displacement is packed into a rectangular image. When asps_vdmc_ext_packing_method = 0, scan from the front and set the displacement value to a two-dimensional pixel value. When asps_vdmc_ext_packing_method = 1, scan from the back and set the displacement value to a two-dimensional pixel value. The case where the value is 0 may be referred to as forward, and the case where the value is 1 may be referred to as reverse. Alternatively, the case where the value is 0 may be referred to as ascending order, and the case where the value is 1 may be referred to as descending order. When afps_vdmc_ext_packing_method does not appear, it may be set as follows. afps_vdmv_ext_subdivion_method=asps_vdmc_ext_packing_method

[0079] asps_vdmc_ext_displacement_video_block_size_idc (afps_vdmc_ext_displacement_video_block_size_idc) indicates the size of a basic block (CTU size) when the displacement is encoded by the video codec. If afps_vdmc_ext_video_block_size_idc is not present, it can be set as follows. afps_vdmc_ext_video_block_size_idc=asps_vdmc_ext_video_block_size_idc

[0080] In addition, asps_vdmc_ext_displacement_video_block_size_idc indicates the size of a basic block when rearranging the displacement into an image in units of blocks. Block sizes of 16, 32, 64, 128, and 256 may be assigned to asps_vdmc_ext_displacement_video_block_size_idc=0, 1, 2, 3, and 4, respectively. blockSize=ctuSize=16<<asps_vdmc_ext_displacment_video_block_size_ide

[0081] In addition, block sizes of 32, 64, 128, 256, and 512 may also be allocated respectively. blockSize=ctuSize=32<<asps_vdmc_ext_displacement_video_block_size_idc

[0082] asps_vdmc_ext_1d_displacement_flag is a flag indicating whether to use one-dimensional displacement. When the value is 1, one-dimensional displacement is used, including only normal. In this case, the number of displacement components DisplacementDim is 1. When the value is 0, three-dimensional displacement is used, and the number of displacement components DisplacementDim is 3.

[0083] asps_vdmc_ext_displacement_video_segment_enabled_flag is the slice segmentation information that has been described, and is a flag indicating whether the displacement is divided into component units by slices (whether the displacement is sliced ​​at the component boundary). It should be noted that asps_vdmc_ext_displacement_video_segment_enabled_flag can represent the MCTS of HEVC as an independent segment in the spatiotemporal direction or a sub-picture with restrictions on the temporal prediction of VVC. The sub-picture with restrictions on the temporal prediction of VVC is a sub-picture with sps_subpic_treated_as_pic_flag=1. Moreover, it can also indicate that loop filtering between slices (segments) is prohibited (sps_loop_filter_across_subpic_enabled_flag==0). In order to make it clear that slices (segments) are independent, the syntax name can also be asps_vdmc_ext_displacement_video_independent_segment_enabled_flag, etc. It should be noted that it may also be asps_vdmc_ext_displacement_video_scalable_enabled_flag indicating that segmentation decoding is possible.

[0084] In the case of independent segments in the spatiotemporal direction, sps_subpic_treated_as_pic_flag=1 and sps_loop_filter_across_subpic_enabled_flag==0 may be used.

[0085] It should be noted that sps_subpic_treated_as_pic_flag[i] is 1, indicating that the i-th sub-picture of each coded picture of CLVS is processed into an image in the decoding process other than loop filtering. 0 indicates that the i-th sub-picture of each coded picture of CLVS is not processed into an image in the decoding process other than loop filtering. sps_loop_filter_across_subpic_enabled_flag[i] is 1, indicating that loop filtering may be applied between the i-th sub-pictures of each coded picture of CLVS. 0 indicates that loop filtering is not applied between the i-th sub-pictures of each coded picture of CLVS.

[0086] It should be noted that when sps_subpic_treated_as_pic_flag=1, there is a restriction that the picture width, picture height, and CTU size are equal between the target picture and the reference picture list 0 (RefPicList[0]) or the reference picture that is an active entry in the reference picture list 1 (RefPicList[1]). In addition, in the time direction prediction, when sps_subpic_treated_as_pic_flag=1, not only the width and height of the picture are derived according to the following formula, but also the right range and bottom range of the reference area are derived based on the range of the sub-picture. That is, the x-coordinate of the reference position is clipped to be below rightBoundaryPos, and the y-coordinate of the reference position is clipped to be below botBoundaryPos. rightBoundaryPos=sps_subpic_tredted_as_pic_flag?SubpicRightBoundaryPos: ppspic_width_in_luma_samples-1 botoundaryPos=sps_subpic_treated_as_pic_flag? SubpicBotBoundaryPos:pps_pic_height_in_luma_samples-1

[0087] It should be noted that SubpicRightBoundaryPos indicates the right end of the object sub-picture, SubpicBotBoundaryPos indicates the bottom end of the object sub-picture, and pps_pic_width_in_luma_samples and pps_pic_height_in_luma_samples indicate the width and height of the picture.

[0088] Fig. 20 Example of syntax indicating displacement slice segmentation parameters: Here, when asps_vdmc_ext_1d_displacement_flag is a predetermined value (a value other than one-dimensional, a value indicating three-dimensional), slice segmentation information (eg, asps_vdmc_ext_displacement_video_segment_enabled_flag, asps_vdmc_ext_displacement_slice_type) is decoded from the encoded data.

[0089] Fig.21This is an example of the syntax of the displacement slice segmentation parameter. Here, when asps_vdmc_ext_1d_displacement_flag is a specified value (a value other than one-dimensional, a value indicating three-dimensionality) or the number of LOD divisions is 1 or more (asps_vdmc_ext_subdivision_iteration_count>0), the slice segmentation information (e.g., asps_vdmc_ext_displacement_video_segment_enabled_flag, asps_vdmc_ext_displacement_slice_type) is decoded from the encoded data.

[0090] Fig. 22 This is an example of the syntax of displacement slice segmentation parameters. Here, when asps_vdmc_ext_subdivision_method is a specified value indicating that the segmentation is to be performed, the syntax elements related to the characteristics of the displacement and the encoding method are decoded. For example, the syntax elements related to the characteristics of the displacement and the encoding method are asps_vdmc_ext_subdivision_iteration_count, asps_vdmc_ext_displacement_coordinate_system, asps_vdmc_ext_1d_displacement_flag, asps_vdmc_ext_displacement_video_block_size_idc, asps_vdmc_ext_packing_method, and asps_vdmc_ext_displacement_video_segment_enabled_flag. As a result, when the displacement is not sliced, unnecessary syntax elements are not transmitted, thereby achieving the effect of reducing the amount of coding.

[0091] Fig.23This is an example of the syntax of the displacement slice segmentation parameter. Here, a certain syntax element asps_vdmc_ext_displacement_video_flag is decoded. asps_vdmc_ext_displacement_video_flag is a syntax element indicating whether the displacement is transmitted through the image codec. When asps_vdmc_ext_displacement_video_flag indicates that the displacement is transmitted through the image codec, the parameters used when the displacement is encapsulated (configured) in the image are decoded. The above-mentioned parameters related to the encapsulation of the displacement are, for example, the block size asps_vdmc_ext_displacement_video_block_size_idc, the encapsulation method asps_vdmc_ext_packing_method, and the slice segmentation information asps_vdmc_ext_displacement_video_segment_enabled_flag. As a result, when the displacement is transmitted by arithmetic coding (when the image codec is not used), unnecessary syntax elements are not transmitted, thereby achieving the effect of reducing the amount of coding. It should be noted that, instead of asps_vdmc_ext_displacement_arithmetric_coding_flag, a flag indicating whether displacement is transmitted by arithmetic coding may be used. In this case, if(!asps_vdmc_ext_displacement_arithmetric_coding_flag) is used instead of the judgment of if(asps_vdmc_ext_displacement_video_flag). That is, when asps_vdmc_ext_displacement_arithmetric_coding_flag does not indicate the use of arithmetic coding, the parameters related to the encapsulation of displacement are encoded / decoded. (Operation of the Grid Shift Decoding Unit)

[0092] The animation decoding unit 3051 decodes the geometric animation stream (V3C_GVD video stream) encoded by VVC, HEVC, etc., and outputs a decoded image (grid displacement image, grid displacement arrangement) with (quantized) grid displacement as pixel value. The color component of the geometry is represented by DecGeoChromaFormat. The image can be in YCbCr4:2:0 format. In addition, the grid displacement image can be a converted grid displacement image. In addition, it can also be a residual of the grid displacement image.

[0093] The displacement demapping unit 3052 generates a grid displacement based on the grid displacement image. Specifically, based on the correspondence of the coordinate positions, the grid displacement Qdisp[pos][compIdx] of the one-dimensional signal of the component compIdx unit (=cIdx) is derived from the gFrame[compIdx][y][x] of the two-dimensional grid displacement image. It should be noted that gFrame can be an image arrangement DecGeoFrames[mapIdx][frameIdx] or GeoFramesNF[mapIdx][compTimeIdx] decoded from a geometric animation stream (video stream of V3C_GVD). Here, the correspondence of the coordinate positions can be a Z-order scan in units of blocks. NF is the abbreviation of nominal format, which is an image after the image size, color sampling, etc. are adjusted. frameIdx and compTimeIdx are composition time indexes. It should be noted that the name of the arrangement of the grid displacement image gFrame[compIdx][y][x] can also be the quantized displacement wavelet coefficient dispQuantCoeffFrame, etc., and the order of the arrangement index is not limited to gFrame[compIdx][y][x], but can also be dispQuantCoeffFrame[x][y][compIdx] (the same below).

[0094] The displacement demapping unit 3052 derives DisplacementDim based on the value of the flag asps_vdmc_ext_1d_displacement_flag indicating whether to use the one-dimensional displacement decoded from the encoded data. DisplacementDim=(asps_vdmc_ext_1d_displacement_flag)? 1:3

[0095] Here, asps_vdmc_ext_1d_displacement_flag = 1 indicates that only one dimension of the three-dimensional displacement is transmitted. The normal component or x component (first component) representing the displacement exists in the (compressed) geometric image. When the one-dimensional flag is true, the displacement demapping unit 3052 infers that the remaining two components are 0. asps_vdmc_ext_1d_displacement_flag = 0 indicates that all three components of the displacement exist in the (compressed) geometric image.

[0096] The displacement demapping unit 3052 derives the number of blocks blockCount from the number of grid displacements (number of points) verCoordCount, and derives the displacement height dispCompHeight from blockCount. Block is a displacement coefficient block, and blockSize is a variable representing the size of the displacement coefficient block. width and height are variables representing the width and height of the grid displacement image. pixelsPerBlock=blockSize*blockSize widthInBlocks=width / blockSize shift=(1<<bitDepth)>>1 blockCount=(verCoordCount+pixelsPerBlock-1) / pixelsPerBlock heightInBlocks=(blockCount+widthInBlocks-1) / widthInBlocks

[0097] In a certain configuration, the displacement demapping unit 3052 may use dispCompHeight decoded from the syntax element (eg, asps_vdmc_ext_displacement_component_height) as described above.

[0098] In another configuration, when the grid displacement is sliced ​​(when displacementSliceType is a predetermined value and displacementSliceFlag is true), the displacement demapping unit 3052 may derive dispCompHeight using 1 / 3 of the height of the grid-divided image. dispCompHeight = height / 3

[0099] In another configuration, the displacement demapping unit 3052 may derive the height dispCompHeight of the image corresponding to each component of the grid displacement three-dimensional vector by aligning the size of the coding tree unit (CTU Size, ctuSize, videoBlockSize) of the codec used for encoding the displacement with the prescribed size when the grid displacement is sliced ​​(when displacementSliceType is a prescribed value and displacementSliceFlag is true). It should be noted that it is not limited to the size of the CTU, but may be a prescribed block size of the grid image. In this case, it is not called ctuSize, but videoBlockSize. dispCompHeight=heightInBlock*blockSize dispCmpHeight=(dispCompHeight+ctuSize-1) / ctuSize*ctuSize dispCompHeight=(dispCompHeight+ctuSize-1)&(ctuSize-1)

[0100] Here, ~ inverts each bit using a bitwise negation operator.

[0101] That is, it can also be derived in a way that becomes a constant multiple of the specified value ctuSize.

[0102] Alternatively, blockSize = ctuSize. Alternatively, blockSize = ctuSize may be set regardless of the value of displacementSliceFlag.

[0103] Here, the displacement demapping unit 3052 may derive ctuSize from the parameter SPS (Sequence Parameter Set) of the geometry animation stream of the codec indicated by gi_geometry_codec_id[DecAtlasID] of V3C.

[0104] The displacement demapping unit 3052 may decode ctuSize from the encoded data of the NAL unit of the map, for example, the syntax element of ASPS. Alternatively, the value of ctuSizeIdx (videoBlockSizeIdx) may be decoded to derive ctuSize from 16<<ctuSizeIdx or 32<<ctuSizeIdx, 64<<ctuSizeIdx.

[0105] As described below, the displacement demapping unit 3052 can use 64 if gi_geometry_codec_id[DecAtlasID] is HEVC, for example, and can use 128 if it is VVC. ctuSize=ptl_profile_codec_group_idc==3(VVC)? 128:64

[0106] Here, the value of ptl_profile_codec_group_idc indicates 0: AVC Progressive High, 1: HEVC Main 10, 2: HEVC Main 444, 3: VVC Main 10.

[0107] ctuSize=4 CC codes of gi_geometry_codec_id[DecAtlasID] indicate HEVC? 64:128

[0108] Here, the character strings representing the four CC codes of HEVC and VVC are "hev1" and "vvi1", respectively.

[0109] Alternatively, the displacement demapping unit 3052 may fixedly use the larger one of the maximum value of the CTU size of 64 of HEVC and the maximum value of the CTU size of 128 of VVC.

[0110] Fig. 9 This is an example of dividing the grid displacement image gFrame into segments (slices) of geometric dimensions. Here, it is divided into three slices, and the values ​​of the first, second, and third components of QDisp are assigned to the first component of gFrame. Here, W=width, H=height.

[0111] In one configuration, the displacement demapping unit 3052 derives a (quantized) grid displacement arrangement Qdisp from a 3×height×width grid displacement image gFrame. In the following configuration, in a 3D data decoding device, there is provided: an animation decoding unit that decodes a grid displacement image decoded from a geometric animation stream whose unit type is V3C_GVD of the above-mentioned coded data; and a displacement demapping unit that derives a grid displacement QDisp[compIdx][pos] of each position pos and component compIdx from the grid displacement image gFrame[compIdx][y][x] having the above-mentioned component compIdx of the x, y position, wherein the above-mentioned displacement demapping unit derives the above-mentioned grid displacement by deriving the Y coordinate of the geometric image from the product of a variable compIdx representing a value of -1 of the geometric dimension DisplacementDim from 0 to -1 and a height dispCompHeight.

[0112] Here, in the case of a 4:2:0 image (DecGeoChromaFormat==1), Qdisp can be derived from gFrame[0][y][x] which is the first component (luminance (Y) image component) of gFrame according to dispCompHeight.

[0113] Here, asps_vdmc_ext_packing_method = 0 indicates that the displacement component samples are packed in ascending order. asps_vdmc_ext_packing_method = 1 indicates that the displacement component samples are packed in descending order. computeMorton2D is a function for implementing Z-order scanning, which is defined as follows.

[0114] It should be noted that, in the following, only the DecGeoChromaFormat == 4:2:0 portion is illustrated with loop processing, variable derivation processing, and branching omitted, but the above processing may be directly applied to the case of branching based on DecGeoChromaFormat and 4:4:4.

[0115] In one configuration, the grid displacement arrangement Qdisp may be derived based on the value of the displacement slice segmentation parameter dispCompHeight. For example, the displacement demapping unit 3052 may decode the value of the height dispCompHeight from the encoded data.

[0116] In one configuration, the grid displacement arrangement Qdisp may be derived based on the value of the displacement slice segmentation flag displacementSliceFlag. For example, the displacement demapping unit 3052 may also derive the value of the height dispCompHeight based on a syntax element indicating whether a slice is used.

[0117] Furthermore, as already described, the displacement demapping unit 3052 may derive the value of the height dispCompHeight so as to be a constant multiple of a predetermined value.

[0118] Moreover, the displacement demapping unit 3052 can also derive the value of the above-mentioned height dispCompHeight according to the height of the CTU of the geometric animation stream.

[0119] The grid displacement decoding unit 305 may be configured to decode only the first slice including the first component of the grid displacement when the displacementSliceType is 1 or 2 or when specified by an external unit. In this case, Qdisp may be derived through the following process. or Qdisp[v0][compIdx]=gFrame[0][y1][x1]-shift

[0120] According to the above, only the necessary part can be decoded according to the communication path and the capability of the decoding side, thus achieving scalability. The same applies to the following.

[0121] The grid displacement decoding unit 305 may also be configured to decode only the first slice including the first component of grid displacement and the second slice including the second component when displacementSliceType is 1 or when specified by an external unit. In this case, Qdisp can be derived by the following process. or

[0122] The grid displacement decoding unit 305 may also be configured to decode only the first slice including the first component of grid displacement and the third slice including the third component when displacementSliceType is 1 or when specified by an external unit. In this case, Qdisp can be derived by the following process.

[0123] As described above, by decoding only a part of the components of the grid displacement, the processing can be simplified. In addition, by decoding the slices including the second and third components of the grid displacement as needed (for example, only the first component is decoded in high-speed reproduction, and all components are decoded in normal reproduction), a scalable function can be realized. In addition, even if an error is mixed in the encoded data, only the slices (components) without the error are decoded, thereby enhancing the error tolerance.

[0124] To ensure that segmentation is used for encoding, constraints can be placed on the bitstream.

[0125] For example, when the coordinate system conversion information displacementCoordinateSystem is a predetermined value, slicing the geometry animation stream may be requested as a standard stream (consistent stream). In particular, slicing the component units by mesh displacement may be requested as a consistent stream.

[0126] Therefore, when the syntax element indicating whether encoding is performed in segments uses segments, the 3D data decoding device can decode the encoded geometry animation stream using segments through the compIdx unit of the mesh image component.

[0127] For example, when asps_vdmc_ext_1d_displacement_flag is a predetermined value (eg, 1), slicing the geometry animation stream may be requested as a standard-compliant stream (conformant stream). In particular, slicing by grid-displaced component units may be requested as a conformant stream.

[0128] Therefore, when one-dimensional encoding is performed, the 3D data decoding device always decodes the encoded geometric animation stream using segments in units of components compIdx of the mesh image. (Decoding and exporting at each detail level)

[0129] Alternatively, a configuration may be adopted in which individual displacement slice segmentation parameters are decoded and derived according to the level of details (LOD) of each mesh. Fig.17 , Fig.18 In the case of slice_type being 1 or 2, Figure 7 , Figure 8 Similarly, the same displacement segmentation parameters are used regardless of the detail level of the mesh, and when slice_type is 3, different displacement segmentation parameters are used for each mesh detail level. slice_type=3 is a type that allocates different slices for each mesh displacement detail level.

[0130] Fig.17 is an example of a syntax for transmitting the displacement slice segmentation parameter asps_vdmc_ext_displacement_component_height_lod[i] for each detail level at the sequence level (ASPS). asps_vdmc_ext_displacement_component_height_lod[i] represents the height of the image corresponding to each component of the grid displacement.

[0131] Fig.18 is an example of a syntax for transmitting the displacement slice segmentation parameter afps_vdmc_ext_displacement_component_height_lod[i] for each detail level at the picture / frame level (AFPS). afps_vdmc_ext_displacement_component_height_lod[i] represents the height of the slice in the picture corresponding to each component of the grid displacement.

[0132] Fig.17 , Fig.18 The grammatical elements that are not specified are Figure 7 , Figure 8 Same definition.

[0133] The displacement demapping unit 3052 can derive Qdisp through the following processing.

[0134] Here, width and height are the width and height of the grid displacement image gFrame[compIdx][y][x] (quantized displacement wavelet coefficients dispQuantCoeffFrame) (the same applies below). It should be noted that dispCompHeight=height / 3 can be used. In particular, in the case of spatiotemporally independent slices (eg, sub-pictures of VVC), the same screen height is used in multiple frames, so it is appropriate to use dispCompHeight=height / 3.

[0135] When a slice is not a rectangular slice (CTU row unit) but a slice continuous in a CTU unit is used, the starting position of the slice decoded from the encoded data, dispCompPos[lodIdx], may be used instead of the slice height compIdx*dispCompHeight[lodIdx].

[0136] The displacement demapping unit 3052 can derive Qdisp through the following processing. <Example 1 of processing in LOD units>

[0137] Here, asps_vdmc_ext_subdivision_iteration_count and asps_vdmc_ext_displacement_component_height_lod[i] may be replaced with afps_vdmc_ext_subdivision_iteration_count and afps_vdmc_ext_displacement_component_height_lod[i] respectively.

[0138] The grid displacement decoding unit 305 may be configured to decode the displacement slice segmentation parameters for only the slices including the components below the specified detail level of the grid displacement (e.g., only levelOfDetails=0, only levelOfDetails=0 and 1, etc.) when displacementSliceType is 3 (a type that assigns different slices to each detail level of the grid displacement). It should be noted that dispCompHeight=height / 3 may be used. In particular, in the case of spatiotemporally independent slices (e.g., sub-pictures of VVC), the same screen height is used in multiple frames, so it is appropriate to use dispCompHeight=height / 3. <Other structures for slicing in LOD units>

[0139] Fig.10 This is an example of dividing the grid displacement image gFrame into segments (slices) of geometric dimensions. Here, it is divided into three slices, and the values ​​of the first, second, and third components of QDisp are assigned to the first component of gFrame. Here, W=width, H=height.

[0140] The displacement demapping unit 3052 can derive Qdisp from the mesh displacement image sliced ​​in units of LODs through the following processing: Here, levelOfDetailAttributeCounts[i] is a variable indicating the start position pos of the mesh displacement of the LOD indicated by index i+1.

[0141] The displacement demapping unit 3052 aligns the size of LOD numBlocksInLod with a constant multiple of the block size pixelsPerBlock when the displacement slice segmentation parameter decoded from the encoded data indicates LOD packing. Qdisp can be derived by looping the block in units of LOD using the block size. The displacement slice segmentation parameter indicates LOD packing, for example, when displacementSliceType is a specified value, and is represented by lodBlockPacking=1. lodBlockPacking=0 is a case where LOD packing is not performed. It can be set to lodBlockPacking=displacementSliceFlag.

[0142] The formula for deriving the number of blocks of the LOD unit aligned with a constant multiple in units of LOD is represented by, for example, the following. numBlocksInLod[0]= (level0fDetailAttributeCounts[0]+pixelsPerBlock-1) / pixelsPerBlock

[0143] It should be noted that, in the case where the grid displacement image (geometric animation stream) is sliced, pixelsPerBlock can be derived from ctuSize (blockSize = ctuSize). In addition, ctuSize can be included in the displacement slice segmentation parameters for decoding, or it can be determined based on the type of codec. In addition, constants such as 128 can be used in ctuSize. In addition, in the case where the displacement slice segmentation parameters represent LOD encapsulation, pixelsPerBlock can be derived from ctuSize.

[0144] It should be noted that in another configuration, the number of blocks of the LOD unit aligned with the constant multiple may be directly transmitted as the displacement slice segmentation parameter. In this case, it can be derived as follows. numBlocksInLod[i]=dispCount[i], i=0..lodCount

[0145] Alternatively, you can export as follows. numBlockssInLod[i]=32<<dispCountIdx[i], i=0..lodCount

[0146] Here, 32 can be 64, 128, ctuSize. <Example 2 of processing in LOD units>

[0147] It should be noted that dispCompHeight = height / 3 may be used. In particular, in the case of temporally and spatially independent slices (eg, sub-pictures of VVC), the same screen height is used in multiple frames, so it is appropriate to use dispCompHeight = height / 3. <Example 3 of processing in LOD units>

[0148] In addition, as described below, the displacement demapping unit 30 can use ctuSize decoded from the encoded data to derive the block size blockSize of the scanning unit of the displacement of the grid displacement image. As a result, the encoder can freely set the block size used in the geometric animation stream of the grid displacement image, and can match the scanning unit of the grid displacement with the block size of the animation stream. In addition, the displacement demapping unit 3052 derives the starting point and length of the block, vStart and vEnd outside the loop in the loop of the LOD unit from 0 to lodCount, and makes the updated vStart equal to the vEnd so far just before the end of the loop, and derives vStart by adding the specified block size to vEnd to obtain the value vEnd = startBlock + numBlocksInLod[lodIdx+1]. lodCount=asps_vdmc_ext_subdivision_iteration_count+1 blockSize=ctuSize or

[0149] It should be noted that the lodBlockPacking flag may not be used and lodBlockPacking = 1 may always be used (the same applies below). In addition, in the above, dispCompHeight = height / 3 may be used. In particular, in the case of spatiotemporally independent slices (eg, sub-pictures of VVC), the same screen height is used in multiple frames, so it is appropriate to use dispCompHeight = height / 3. <Example 4 of processing in LOD units>

[0150] Moreover, as described below, the displacement demapping unit 3052 can also set the LOD number when processing the LOD unit (lodBlockPacking) to lodCount, set the other LODs to 0, calculate the start position and end position of the LOD unit and the block unit in advance, and derive QDisp through the loop of the LOD unit and the point unit. lodCount=asps_vdmc_ext_subdivision_iteration_count+1 blockSize=ctuSize / / or if(lbdBlockPacking)bllockSize=ctuSize pixelsPerBlockk=blockSize*blockSize

[0151] According to the above, regardless of whether the processing is performed in units of LOD, the loop is performed in units of lodCount, so the processing becomes simple. In addition, in the above, dispCompHeight can be set to height / 3. In particular, in the case of spatiotemporally independent slices (eg, sub-pictures of VVC), the same screen height is used in multiple frames, so it is appropriate to use dispCompHeight=height / 3.

[0152] For example, when displacementSliceType is a specified value (when displacementSliceFlag is true), a request may be made to slice the mesh displacement image (geometry animation stream) as a standard stream (consistent stream). In particular, a request may be made to slice the mesh displacement image (geometry animation stream) in units of LOD as a consistent stream. Furthermore, when lodBlockPacking is a specified value (e.g., 1), a request may be made to slice the mesh displacement image (geometry animation stream) in units of LOD as a consistent stream.

[0153] Therefore, when the syntax element indicating whether the encoding in units of slices (segments) uses slices (segments), the 3D data decoding device can decode the encoded geometric animation stream using slices (segments) in units of LOD of the grid image.

[0154] For example, when asps_vdmc_ext_1d_displacement_flag is a predetermined value (eg, 1), the geometry animation stream may be sliced ​​to be a standard-compliant stream (consistent stream). In particular, the geometry animation stream may be sliced ​​in units of LOD of mesh displacement to be a consistent stream.

[0155] Therefore, when one-dimensional encoding is performed, the 3D data decoding device always uses segments to decode the encoded geometric animation stream in units of LOD of the mesh image.

[0156] asps_vdmc_ext_subdivision_iteration_count and afps_vdmc_ext_subdivision_iteration_count are parameters indicating the number of repetitions of mesh division notified at each ASPS level and AFPS level, and are parameters used for deriving displacement slice division parameters at each detail level.

[0157] The inverse quantization unit 3053 performs inverse quantization based on the quantization level value iscale, and derives the grid displacement Tdisp after transformation (for example, wavelet transformation). Tdisp can be a Cartesian coordinate system or a local coordinate system. iscale is a value derived from the quantization parameters of each component of the grid displacement image. Tdisp[0][pos]=(Qdisp[0][pos]*iscale[0]+iscaleOffset)>>iscaleShift Tdisp[1][pos]=(Qdisp[1][pos]*iscale[1]+iscaleOffset)>>iscaleShift Tdisp[2][pos]=(Qdisp[2][pos]*iscale[2]+iscaleOffset)>>iscaleShift

[0158] Here, iscaleOffset=1<<(iscaleShift-1). iscaleShift may be a preset constant, or a value obtained by decoding the encoded data after encoding at a sequence level, a picture / frame level, a tile / patch level, etc. may be used.

[0159] The inverse transformation unit 3054 performs inverse transformation g (for example, inverse wavelet transformation) to derive the mesh displacement d. d[0][pos] = g(Tdisp[0][pos]) d[1][pos]=g(Tdisp[1][pos]) d[2][pos]=g(Tdisp[2][pos])

[0160] The coordinate system conversion unit 3055 converts the grid displacement (the coordinate system of the grid displacement) into a Cartesian coordinate system based on the value of the coordinate system conversion information displacementCoordinateSystem. Specifically, when displacementCoordinateSystem=1, the displacement of the local coordinate system is converted to the displacement of the Cartesian coordinate system. Here, d is a three-dimensional vector representing the grid displacement before the coordinate system conversion. disp is a three-dimensional vector representing the grid displacement after the coordinate system conversion, which is a Cartesian coordinate system. n_vec, t_vec, b_vec are three-dimensional vectors (of the Cartesian coordinate system) corresponding to the axes of the local coordinate system of the object area or object vertex. if(displacementCoordinateSystem==0){ disp=d }else if(displacementCoordinateSystem===1){ disp=d[0]*n_vec+d[1]*t_vec+d[2]*b_vec}

[0161] The derivation method shown by the above-mentioned vector multiplication is expressed as follows when expressed by scalars alone.

[0162] It should be noted that it is also possible to adopt a configuration in which disp=d is set to assign the same variable name before and after the conversion, and the value of d is updated by coordinate conversion.

[0163] Alternatively, the following configuration may be adopted.

[0164] Here, n_vec2, t_vec2, and b_vec2 are three-dimensional vectors (of a Cartesian coordinate system) corresponding to the axes of the local coordinate system of the adjacent region.

[0165] In addition, the following configuration is also possible.

[0166] Here, n_vec3, t_vec3, and b_vec3 are three-dimensional vectors (of the Cartesian coordinate system) corresponding to the axes of the local coordinate system of the target region for suppressing fluctuations. For example, a vector of the coordinate system used for decoding is derived from the previous coordinate system and the current coordinate system as shown below. n_vec3=(w*n_vec3+(WT-w)*n_vec)>>wShift t_vec3=(w*t_vec3+(WT-w)*t_vec)>>wShift b_vec3=(w*b_vec3+(WT-w)*b_vec)>>wShift

[0167] Here, for example, wShift=2, 3, 4, WT=1<<wShift, w=1..WT-1.

[0168] For example, when w=3, wShift=3, it can be as follows. n_vec3=(3*n_vec3+5*n_vec)>>3 t_vec3=(3*t_vec3+5*t_vec)>>3 b_vec3=(3*b_vec3+5*b_vec)>>3

[0169] Alternatively, the following configuration may be used in which selection can be made based on the value of the coordinate system conversion information displacementCoordinateSystem obtained by decoding the encoded data. (Reconstruction of the grid)

[0170] Figure 6 3 is a functional block diagram showing the configuration of the mesh reconfiguration unit 307. The mesh reconfiguration unit 307 is composed of a mesh division unit 3071 and a mesh deformation unit 3072.

[0171] The mesh dividing unit 3071 divides the basic mesh output from the basic mesh decoding unit 303 to generate a subdivided mesh.

[0172] Fig.11 (a) represents a part of the basic mesh (triangle), which is composed of vertices v1, v2, and v3. v1, v2, and v3 are three-dimensional vectors. The mesh segmentation unit 3071 generates a subdivided mesh by adding new vertices v12, v13, and v23 in the middle of each side of the triangle and outputs ( Fig.11 (b)). v12=(v1+v2) / 2 v13=(v1+v3) / 2 v23=(v2+v3) / 2

[0173] In addition, it may be the following. v12=(v1+v2+1)>>1 v13=(v1+v3+1)>>1 v23=(v2+v3+1)>>1

[0174] The mesh deformation unit 3072 generates a deformed mesh by inputting the subdivided mesh and the mesh displacement, and adding the mesh displacements d12, d13, and d23, and outputs the deformed mesh ( Fig.11 (c)) The mesh displacement is the output of the mesh displacement decoding unit 305 (coordinate system conversion unit 3055). d12, d13, d23 are the mesh displacements corresponding to the vertices v12, v13, v23 added by the mesh dividing unit 3071. v12'=v12+d12 v13'=v13+d13 v23'=v23+d23

[0175] It should be noted that d12=disp[0][], d13=disp[1][], d23=disp[3][] may also be true. (Configuration of 3D Data Encoding Device of First Embodiment)

[0176] Fig.12 1 is a functional block diagram showing a schematic structure of a 3D data encoding device 11 according to the first embodiment. The 3D data encoding device 11 is composed of a graph information encoding unit 101, a basic grid encoding unit 103, a basic grid decoding unit 104, a grid displacement updating unit 106, a grid displacement encoding unit 107, a grid displacement decoding unit 108, a grid reconstruction unit 109, an attribute updating unit 110, a filling unit 111, a color space conversion unit 112, an attribute encoding unit 113, a multiplexing unit 114, and a grid separation unit 115. The 3D data encoding device 11 inputs graph information, a basic grid, a grid displacement, a grid, and an attribute image as 3D data, and outputs encoded data.

[0177] The graph information encoding unit 101 encodes the graph information and outputs a graph information encoding stream.

[0178] The basic grid coding unit 103 codes the basic grid and outputs a basic grid coding stream. As a coding method, Draco or the like is used.

[0179] The basic grid decoding unit 104 is the same as the basic grid decoding unit 303 , and therefore the description thereof is omitted.

[0180] The grid displacement updating unit 106 adjusts the grid displacement based on the (original) basic grid and the decoded basic grid, and outputs the updated grid displacement.

[0181] The grid displacement encoding unit 107 encodes the updated grid displacement and outputs a grid displacement encoding stream. As the encoding method, VVC, HEVC, etc. are used.

[0182] The grid displacement decoding unit 108 is the same as the grid displacement decoding unit 305, so the description is omitted.

[0183] The mesh reconfiguration unit 109 is the same as the mesh reconfiguration unit 307 , and thus the description thereof is omitted.

[0184] The attribute updating unit 110 inputs the (original) mesh, the reconstructed mesh output from the mesh reconstructing unit 109 (mesh deformation unit 3072), and the attribute image, updates the attribute image to match the position (coordinates) of the reconstructed mesh, and outputs the updated attribute image.

[0185] The filling unit 111 inputs an attribute image and performs filling processing on a region where pixel values ​​are empty.

[0186] The color space conversion unit 112 performs color space conversion from the RGB format to the YCbCr format.

[0187] The attribute encoding unit 113 encodes the attribute image in the YCbCr format output from the color space conversion unit 112, and outputs an attribute moving image stream. As the encoding method, VVC, HEVC, etc. are used.

[0188] The multiplexing unit 114 multiplexes the atlas information coded stream, the basic grid coded stream, the grid displacement coded stream, and the attribute animation stream, and outputs them as coded data. As a multiplexing method, a byte stream format, ISOBMFF, etc. are used. (Operation of the grid separation unit)

[0189] The mesh separation unit 115 generates a basic mesh and a mesh displacement from the mesh.

[0190] Fig.15 2 is a functional block diagram showing the structure of the mesh separation unit 115. The mesh separation unit 115 is composed of a mesh thinning unit 1151, a mesh dividing unit 1152, and a mesh displacement deriving unit 1153.

[0191] The mesh thinning unit 1151 generates a basic mesh by thinning out some vertices from the mesh.

[0192] Fig.16(a) represents a part of a mesh, and the mesh is composed of vertices v1, v2, v3, v4, v5, and v6. v1, v2, v3, v4, v5, and v6 are three-dimensional vectors. The mesh thinning unit 1151 generates a basic mesh by thinning out vertices v4, v5, and v6 and outputs ( Fig.16 (b)).

[0193] The mesh division unit 1152 divides the basic mesh to generate a subdivided mesh ( Fig.16 (c)). v4'=(v1+v2) / 2 v5'=(v1+v3) / 2 v6'=(v2+v3) / 2

[0194] The mesh displacement derivation unit derives and outputs the displacements d4, d5, d6 of vertices v4, v5, v6 relative to vertices v4′, v5′, v6′ as mesh displacements based on the mesh and the subdivided mesh ( Fig.16 (d)). d4=v4-v4' d5=v5-v5' d6=v6-v6' (Code of the basic grid)

[0195] Fig.13 1 is a functional block diagram showing the structure of the basic grid coding unit 103. The basic grid coding unit 103 is composed of a grid coding unit 1031, a grid decoding unit 1032, a motion information coding unit 1033, a motion information decoding unit 1034, a grid motion compensation unit 1035, a reference grid memory 1036, a switch 1037, and a switch 1038. The basic grid coding unit 103 may also include a basic grid quantization unit (not shown) after the input of the basic grid. When the basic grid is encoded (intra-frame encoding) without referring to other basic grids (for example, an already encoded basic grid), the switch 1037 and the switch 1038 are connected to the side where motion compensation is not performed. Otherwise, when the basic grid is encoded (inter-frame encoding) with reference to other basic grids, they are connected to the side where motion compensation is performed.

[0196] The grid coding unit 1031 has an intra-frame coding function, performs intra-frame coding on the basic grid, and outputs a basic grid coding stream. As a coding method, Draco or the like is used.

[0197] The trellis decoding unit 1032 is the same as the trellis decoding unit 3031, and therefore the description thereof is omitted.

[0198] The motion information encoding unit 1033 has an inter-frame encoding function, performs inter-frame encoding on the basic grid, and outputs a basic grid encoding stream. Entropy encoding such as arithmetic coding is used as the encoding method.

[0199] The motion information decoding unit 1034 is the same as the motion information decoding unit 3032 , and thus the description thereof is omitted.

[0200] The mesh motion compensation unit 1035 is the same as the mesh motion compensation unit 3033 , and thus the description thereof is omitted.

[0201] The reference grid memory 1036 is the same as the reference grid memory 3034, so the description thereof is omitted. (Encoding of grid displacement)

[0202] Fig.14 1 is a functional block diagram showing the structure of the grid displacement coding unit 107. The grid displacement coding unit 107 is composed of a coordinate system conversion unit 1071, a conversion unit 1072, a quantization unit 1073, and a displacement mapping unit 1074 (image packing unit, displacement coding unit). Fig.14 As shown, the grid displacement coding unit 107 may also include a moving picture coding unit 1075. Alternatively, the moving picture coding unit 1075 may not be included in the grid displacement coding unit 107, and the coding of the displacement image may be performed using an external image coding device.

[0203] The coordinate system conversion unit 1071 converts the coordinate system of the grid displacement from the Cartesian coordinate system to the coordinate system (e.g., the local coordinate system) encoding the displacement based on the value of the coordinate conversion information displacementCoordinateSystem. Here, disp is a three-dimensional vector representing the grid displacement before the coordinate system conversion, d is a three-dimensional vector representing the grid displacement after the coordinate system conversion, and n_vec, t_vec, and b_vec are three-dimensional vectors (of the Cartesian coordinate system) representing the axes of the local coordinate system.

[0204] The grid displacement encoding unit 107 may update the value of displacementCoordinateSystem at a picture / frame level.

[0205] When encoding the displacementCoordinateSystem at the sequence level, use Figure 7The syntax of the structure is: In the case of the Cartesian coordinate system, asps_vdmc_ext_displacement_coordinate_system is set to 0, and in the case of the local coordinate system, asps_vdmc_ext_displacement_coordinate_system is set to 1.

[0206] To change the displacementCoordinateSystem at the picture / frame level, use Figure 8 When updating the coordinate system, afps_vdmc_ext_overriden_flag is set to 1, and when not updating the coordinate system, afps_vdmc_ext_overriden_flag is set to 0. In the case of a Cartesian coordinate system, afps_vdmc_ext_displacement_coordinate_system is set to 0, and in the case of a local coordinate system, afps_vdmc_ext_displacement_coordinate_system is set to 1.

[0207] The conversion unit 1072 performs conversion f (for example, wavelet conversion) to derive the mesh displacement Tdisp after conversion. The following is performed for pos=0..NumDisp-1. Here, NumDisp is the number of mesh vertices. Tdisp[0][pos]=f(d[0][pos]) Tdisp[1][pos]=f(d[1][pos]) Tdisp[2][pos]=f(d[2][pos])

[0208] The quantization unit 1073 performs quantization based on the quantization scale value scale derived from the quantization parameter of each component of the grid displacement, and derives the quantized grid displacement Qdisp. Qdisp[0][pos]=Tdisp[0][pos] / soale[0] Qdisp[1][pos]=Tdisp[1][pos] / scale[1] Qdisp[2][pos]=Tdisp[2][pos] / scale[2]

[0209] Alternatively, the scale value can be multiplied by a power of 2 to approximate Qdisp using the following formula. scalc[i]=1<<scale2[i] Qdisp[0][pos]=Tdisp[0][pos]>>scale2[0] Qdisp[1][pos]=Tdisp[1][pos]>>scale2[1] Qdisp[2][pos]=Tdisp[2][pos]>>scale2[2]

[0210] The displacement mapping unit 1074 generates an image gFrame according to the quantized grid displacement Qdisp based on the value of the displacement mapping parameter displacementChromaLocationType.

[0211] The displacement mapping unit 1074 packs the first component Qdisp[0] of the (quantized) grid displacement arrangement into a luminance (Y) image component.

[0212] Perform the following for x=0..W-1, y=0..H-1.

[0213] In the image width W and height H, the following applies to (y=0..H-1, x=0..W-1). Here, yc=y / 2, xc=x / 2. H = dispCompHeight shift=(1<<bitDepth)>>1 gFrame[0][y][x]=Qdisp[0][pos]+shift gFrame[0][H+y][x]=Qdisp[1][pos]+shift gFrame[0][2*H+y][x]=Qdisp[2][pos]+shift pos++

[0214] Perform the following for xc=0..W / 2-1, yc=0..H / 2-1. gFrame[1][yc][xc]=shift gFrame[1][H / 2*yc][xc]=shift gFrame[1][H+yc][xc]=shift gFrame[2][yc][xc]=shift gFrame[2][H / 2+yc][xc]=shift gFrame[2][H+yc][xc]=shift

[0215] Alternatively, the following can be done.

[0216] Alternatively, the following can be done. Alternatively, the following can be done.

[0217] It should be noted that the processing may be switched according to DecGeoChromaFormat. That is, when DecGeoChromaFormat=1 (4:2:0), the above processing is performed, and when DecGeoChromaFormat=3 (4:4:4), the following processing is performed. gFrame[0][y][x]=Qdisp[0][pos] gFrame[1][y][x]=Qdisp[1][pos] gFrame[2][y][x]=Qdisp[2][pos] pos=pos+1

[0218] The grid displacement encoding unit 107 may update the values ​​of displacementSliceType and dispCompHeight at a picture / frame level.

[0219] The meanings of the values ​​of displacementSliceType and dispCompHeight are as described by asps_vdmc_ext_displacement_slice_type and asps_vdmc_ext_displacement_component_height.

[0220] In case of updating displacementSliceType at picture / frame level, use Figure 8 The syntax of the structure is as follows. The value of displacementSliceType is set to afps_vdmc_ext_displacement_slice_type. The value of dispCompHeight is set to afps_vdmc_ext_displacement_component_height.

[0221] The moving image encoding unit 1075 encodes the image in the YCbCr 4:2:0 format including the (quantized) grid displacement image, and outputs a grid displacement encoded stream. As the encoding method, VVC, HEVC, etc. are used.

[0222] The video encoding unit 1075 may divide the grid-displaced image into slices for each dispCompHeight and encode the slices. Alternatively, the dispCompHeight may be aligned with a predetermined size according to the CTU size.

[0223] The animation encoding unit 1075 may also allocate the first component (eg, D) of the grid displacement to the first slice for encoding, allocate the second component (eg, U) to the second slice for encoding, and allocate the third component (eg, V) to the third slice for encoding (displacementSliceType=1).

[0224] Alternatively, the animation encoding unit 1075 may allocate the first component of the grid displacement to the first slice for encoding, and allocate the second and third components to the second slice for encoding (displacementSliceType=2).

[0225] As described above, by allocating different slices to each grid-shifted component, the decoding device can decode only a portion of the components, thereby simplifying the processing. In addition, the decoding device can decode the slices including the second and third components of the grid shift as needed, thereby realizing a scalable function. In addition, even if an error is mixed in the encoded data, the decoding device can decode only the slices (components) without errors, thereby enhancing the error tolerance.

[0226] As mentioned above, one embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to the above embodiment, and various design changes and the like can be made within the scope not departing from the gist of the present invention. [Application Examples]

[0227] The 3D data encoding device 11 and the 3D data decoding device 31 can be mounted on various devices for sending, receiving, recording, and reproducing 3D data. It should be noted that the 3D data can be natural 3D data captured by a camera or the like, or artificial 3D data generated by a computer or the like (including CG (Computer Graphics: computer animation) and GUI (Graphical User Interface: graphical user interface)).

[0228] The embodiments of the present invention are not limited to the above-mentioned embodiments, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical solutions appropriately modified within the scope of the claims are also included in the technical scope of the present invention. Industrial Applicability The embodiments of the present invention can be preferably applied to a 3D data decoding device that decodes coded data obtained by encoding 3D data and a 3D data encoding device that generates coded data obtained by encoding 3D data. In addition, the embodiments of the present invention can be preferably applied to the data structure of the coded data generated by the 3D data encoding device and referred to by the 3D data decoding device. Description of Reference Numerals 11: 3D data encoding device 101: Spectral Information Coding Department 103: Basic grid coding unit 1031: Grid Coding Department 1032: Grid decoding unit 1033: Motion information encoding unit 1034: Motion information decoding unit 1035: Mesh Motion Compensation Unit 1036: Reference grid storage 1037: Switch 1038: Switch 104: Basic grid decoding unit 106: Grid displacement update section 107: Grid displacement coding unit 1071: Coordinate system conversion unit 1072: Conversion Department 1073: Quantitative Department 1074: Displacement Mapping Department 1075: Animation Coding Department 108: Grid displacement decoding unit 109: Grid Reconstruction Department 110: Attribute Update Department 111: Filling 112: Color space conversion unit 113: Attribute coding unit 114: Multiplexing Department 115: Grid separation unit 1151: Grid spacing elimination section 1152: Grid segmentation 1153: Grid displacement export section 21: Network 31: 3D data decoding device 301: Demultiplexing unit 302: Spectrum information decoding unit 303: Basic grid decoding unit 3031: Grid decoding unit 3032: Motion information decoding unit 3033: Mesh Motion Compensation Department 3034: Reference grid storage 3035: Switch 3036: Switch 305: Grid displacement decoding unit 3051: Animation decoding department 3052: Displacement demapping unit 3053: Inverse Quantization Department 3054: Reverse conversion unit 3055: Coordinate system conversion unit 307: Grid Reconstruction Department 306: Attribute decoding unit 3071: Mesh segmentation department 3072: Mesh Deformation Department 308: Color space conversion unit 41: 3D data display device

Claims

1. A 3D data decoding device for decoding coded data, wherein the 3D data decoding device comprises: An animation decoding unit that decodes a grid displacement image decoded from a geometric animation stream whose unit type is V3C_GVD of the encoded data; and The displacement demapping unit derives the grid displacement QDisp[compIdx][pos] of each position pos and component compIdx from the grid displacement image gFrame[compIdx][y][x] with x, y positions and components compIdx. When the geometric image is 4:2:0, the displacement demapping unit derives the grid displacement by deriving the Y coordinate of the geometric image by multiplying the variable compIdx from 0 to -1 representing the value of the geometric dimension DisplacementDim and the height dispCompHeight.

2. The 3D data decoding device according to claim 1, characterized in that: The displacement demapping unit decodes a syntax element indicating whether a slice is used from the encoded data.

3. The 3D data decoding device according to claim 1, characterized in that: The displacement demapping unit derives a value of the height dispCompHeight according to a syntax element indicating whether a slice is used.

4. The 3D data decoding device according to claim 1, characterized in that: The displacement demapping unit derives the value of the height dispCompHeight so as to be a constant multiple of a predetermined value.

5. The 3D data decoding device according to claim 4, characterized in that: The displacement demapping unit derives the value of the height dispCompHeight according to the height of the CTU of the geometric animation stream.

6. The 3D data decoding device according to claim 1, characterized in that: The 3D data decoding device decodes the encoded geometric animation stream using segments in units of mesh images, using segments, when the syntax element indicating whether to use segment units such as slices uses segments.

7. The 3D data decoding device according to claim 1, characterized in that: When one-dimensional encoding is performed, the 3D data decoding device always uses segments to decode the encoded geometric animation stream using the component compIdx of the mesh image as a unit.

8. The 3D data decoding device according to claim 1, characterized in that: When the geometric animation stream is one-dimensionally encoded using the 4:2:0 format, the 3D data decoding device always uses segments to decode the encoded geometric animation stream in units of components compIdx of the grid image.

9. A 3D data encoding device for encoding 3D data, wherein the 3D data encoding device comprises: An animation encoding unit encodes a grid displacement image encoded by a geometric animation stream whose unit type of the encoded data is V3C_GVD; and The displacement mapping part derives the grid displacement image gFrame[compIdx][y][x] with x, y position and component compIdx from the grid displacement QDisp[compIdx][pos], When the geometric image is 4:2:0, the displacement mapping unit derives the geometric image by deriving the Y coordinate of the geometric image by multiplying the variable compIdx from 0 to the value -1 representing the geometric dimension DisplacementDim and the height dispCompHeight.

Images