Information processing apparatus and method

Abstract

The present disclosure pertains to an information processing device and method that make it possible to suppress an increase in processing load associated with reconstruction of 3D data including a displacement vector. According to the method, base grids and attributes are coded; coding the displacement vector according to each level of fineness; generating a content file; storing the encoded data of the base grid in a base grid track of the content file; storing the encoded data of the attribute in an attribute track of the content file; storing the encoded data of the displacement vector of each level in different displacement tracks of the content file; and storing association information for associating the displacement track with the hierarchy in the content file. The present disclosure can be applied to, for example, an information processing device or an information processing method.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and method, and more particularly, to an information processing apparatus and method that enable suppression of an increase in a processing load associated with reconstruction of 3D data including a displacement vector. Background Art

[0002] Conventionally, there is V-DMC (Video-based Dynamic Mesh Coding) as a method for encoding a mesh, which is 3D data that represents the three-dimensional structure of an object by means of vertices and connections (for example, see Non-Patent Document 1). In V-DMC, the mesh of the encoding target (original mesh) is represented by a coarser (i.e., lower level of detail) base mesh, displacement vectors of segmentation points obtained by subdividing the base mesh, texture, and atlas information for reconstruction, and each of them is encoded. It should be noted that the displacement vector can be hierarchical according to the level of detail (subdivision iteration count of the base mesh).

[0003] In recent years, it is expected to establish a method for storing and distributing the V-DMC encoded bitstream (also referred to as V-DMC bitstream) in ISOBMFF (International Organization for Standardization Base Media File Format) or other formats. ISOBMFF is a file container specification of "MPEG-4 (Moving Picture Experts Group-4), which is an international standard technology for video compression (for example, see non-patent documents 2 and 3). For example, it can be assumed that the V-PCC distribution technology standard ISO / IEC 23090-10 (for example, see non-patent document 4) is extended and standardized. In this case, the V-DMC bitstream is stored in the ISOBMFF track.

[0004] Citation List

[0005] Patent Literature

[0006] Non-patent literature

[0007] Non-patent literature 1: Khaled Mammou, Jungsun Kim, Alexis Tourapis, DimitriPodborski, Krasimir Kolarov, “[V-CG] Apple's Dynamic Mesh Coding CfP Response”, ISO / IEC JTC 1 / SC 29 / WG 7m59281, April 2022.

[0008] Non-patent document 2: “Information technology-Coding of audio-visual objects-Part 12:ISO base media file format, TECHNICAL CORRIGENDUM 1”, ISO / IEC 14496-12:2015 / Cor.1, ISO / IEC JTC 1 / SC 29 / WG 11, 2016 / 6 / 3, 7th edition, 2022-01.

[0009] Non-patent document 3: "Information technology-Coding of audio-visual objects-Part 15: Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format", ISO / IEC FDIS14496-15:2014(E), ISO / IEC JTC 1 / SC29 / WG 11, 2014 / 1 / 13, ISO / IEC 14496-15:2019, 5th edition, 2019-09.

[0010] Non-patent document 4: “Text of ISO / IEC FDIS23090-10 Carriage of Visual Volumetric Video-based Coding Data”, ISO / IEC JTC 1 / SC 29 / WG 03N00241, 2021 / 8 / 20, ISO / IEC 23090-10:2022, 1st edition, 2022-05. Summary of the invention

[0011] Technical issues

[0012] However, if the entire displacement vector stream (displacement video) is stored in a single track, it is necessary to obtain and decode the displacement vector streams of all levels from the track when decoding the displacement vectors. This may unnecessarily increase the processing load associated with reconstruction.

[0013] The present disclosure has been made in view of such circumstances to enable suppression of an increase in a processing load associated with reconstruction of 3D data including displacement vectors.

[0014] Solution to the problem

[0015] An information processing device according to one aspect of the present technology is an information processing device including: an encoding unit that encodes a base mesh and attributes, and encodes a displacement vector at each level in a hierarchy of fineness; and a content file generating unit that generates a content file, stores the encoded data of the base mesh in a base mesh track of the content file, stores the encoded data of the attributes in an attribute track of the content file, stores the encoded data of the displacement vector of each level in the hierarchy in mutually different displacement tracks of the content file, and stores first association information for associating the displacement track with the hierarchy in the content file, wherein the base mesh is a mesh having a fineness lower than that of the original mesh generated by thinning out vertices from an original mesh of an encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of an object, the attributes including a texture applied to a surface of the original mesh, and the displacement vector being vector information indicating the displacement of vertices obtained by subdividing the base mesh.

[0016] An information processing method according to one aspect of the present technology is an information processing method comprising: encoding a base mesh and attributes, and encoding a displacement vector at each level in a hierarchy of fineness; and generating a content file, storing the encoded data of the base mesh in a base mesh track of the content file, storing the encoded data of the attributes in an attribute track of the content file, storing the encoded data of the displacement vector of each level in the hierarchy in mutually different displacement tracks of the content file, and storing association information for associating the displacement tracks with the hierarchy in the content file, wherein the base mesh is a mesh having a fineness lower than that of an original mesh generated by thinning vertices from an original mesh of an encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of an object, the attributes including a texture applied to a surface of the original mesh, and the displacement vector being vector information indicating the displacement of vertices obtained by subdividing the base mesh.

[0017] An information processing device according to another aspect of the present technology is an information processing device including: an acquisition unit that acquires encoded data of a base mesh and encoded data of attributes from a content file, and based on first association information for associating a displacement track with a level of fineness of a mesh, acquires encoded data of a displacement vector from a displacement track of the content file corresponding to a desired level in the level; a decoding unit that decodes each of the acquired encoded data of the base mesh, the acquired encoded data of the attributes, and the acquired encoded data of the displacement vector; and a reconstruction unit that reconstructs a mesh of a desired level using the base mesh, the attributes, and the displacement vectors obtained by decoding, wherein the base mesh is a mesh having a fineness lower than that of an original mesh generated by thinning out vertices from an original mesh of an encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of an object, the attributes including a texture applied to a surface of the original mesh, the displacement vector being vector information indicating displacement of vertices obtained by subdividing the base mesh, and the content file including a base mesh track storing encoded data of the base mesh, an attribute track storing encoded data of the attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different levels.

[0018] An information processing method according to another aspect of the present technology is an information processing method comprising: acquiring encoded data of a base mesh and encoded data of attributes from a content file, and acquiring encoded data of displacement vectors from a displacement track of the content file corresponding to a desired level in the hierarchy based on association information for associating the displacement track with a level of fineness of the mesh; decoding each of the acquired encoded data of the base mesh, the acquired encoded data of the attributes, and the acquired encoded data of the displacement vectors; and reconstructing a mesh of the desired level using the base mesh, the attributes, and the displacement vectors obtained by decoding, wherein the base mesh is a mesh having a fineness lower than that of the original mesh generated by thinning out vertices from an original mesh of an encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of an object, the attributes including a texture applied to a surface of the original mesh, the displacement vector being vector information indicating displacement of vertices obtained by subdividing the base mesh, and the content file including a base mesh track storing the encoded data of the base mesh, an attribute track storing the encoded data of the attributes, and a plurality of displacement tracks storing the encoded data of displacement vectors of mutually different levels.

[0019] According to another aspect of the present technology, an information processing device is an information processing device including: an encoding unit that encodes a base mesh and attributes, and encodes a displacement vector at each level in a hierarchy of fineness; a content file generation unit that generates a content file, stores the encoded data of the base mesh in a base mesh track of the content file, stores the encoded data of the attributes in an attribute track of the content file, and stores the encoded data of the displacement vector of each level in the hierarchy in mutually different displacement tracks of the content file; and a control file generation unit that generates a control file including control information for controlling distribution of the content file, and stores first association information for associating a displacement adaptation set for managing the displacement track with the hierarchy in the control file, wherein the base mesh is a mesh having a fineness lower than that of the original mesh generated by thinning out vertices from an original mesh of an encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of an object, the attributes including a texture applied to a surface of the original mesh, and the displacement vector is vector information indicating the displacement of vertices obtained by subdividing the base mesh.

[0020] According to another aspect of the present technology, an information processing method is such an information processing method, comprising: encoding a base mesh and attributes, and encoding a displacement vector at each level in a hierarchy of fineness; generating a content file, storing the encoded data of the base mesh in a base mesh track of the content file, storing the encoded data of the attributes in an attribute track of the content file, and storing the encoded data of the displacement vector of each level in the hierarchy in mutually different displacement tracks of the content file; and generating a control file including control information for controlling the distribution of the content file, and storing association information for associating a displacement adaptation set for managing the displacement track with the hierarchy in the control file, wherein the base mesh is a mesh with a fineness lower than that of the original mesh generated by thinning vertices from an original mesh of an encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of an object, the attributes including a texture applied to a surface of the original mesh, and the displacement vector being vector information indicating the displacement of vertices obtained by subdividing the base mesh.

[0021] An information processing device according to still another aspect of the present technology is an information processing device including: a selection unit that selects a displacement adaptation set corresponding to a desired level in the levels of fineness of a grid, based on first association information for associating the displacement adaptation set with the levels of fineness of the grid, the first association information being stored in a control file including control information for controlling distribution of a content file; an acquisition unit that acquires encoded data of a base grid and encoded data of an attribute stored in the content file, and acquires encoded data of a displacement vector stored in a displacement track of the content file managed by the selected displacement adaptation set; a decoding unit that decodes each of the acquired encoded data of the base grid, the acquired encoded data of the attribute, and the acquired encoded data of the displacement vector; and a reconstruction unit that reconstructs a mesh of a desired level using a base mesh obtained by decoding, attributes, and displacement vectors, wherein the base mesh is a mesh having a lower fineness than the original mesh generated by thinning vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of an object, the attributes including a texture applied to a surface of the original mesh, the displacement vector being vector information indicating displacement of vertices obtained by subdividing the base mesh, the content file including a base mesh track storing encoded data of the base mesh, an attribute track storing encoded data of the attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different levels, and the control file including a base mesh adaptation set managing the base mesh track, an attribute adaptation set managing the attribute track, and a displacement adaptation set managing the displacement track.

[0022] An information processing method according to another aspect of the present technology is such an information processing method, comprising: selecting a displacement adaptation set corresponding to a desired level in the hierarchy among the displacement adaptation sets based on first association information for associating the displacement adaptation set with a level of fineness of a grid, the first association information being stored in a control file including control information for controlling distribution of a content file; acquiring encoded data of a base grid and encoded data of attributes stored in the content file, and acquiring encoded data of a displacement vector stored in a displacement track of the content file managed by the selected displacement adaptation set; decoding each of the acquired encoded data of the base grid, the acquired encoded data of the attributes, and the acquired encoded data of the displacement vector; and utilizing the decoded The invention relates to a method for reconstructing a mesh of a desired level by using a base mesh obtained by coding, attributes, and displacement vectors, wherein the base mesh is a mesh having a lower fineness than the original mesh generated by thinning vertices from the original mesh of the coding target, the original mesh is composed of vertices and connections representing the three-dimensional structure of the object, the attributes include a texture applied to the surface of the original mesh, the displacement vector is vector information indicating the displacement of the vertices obtained by subdividing the base mesh, the content file includes a base mesh track storing encoded data of the base mesh, an attribute track storing encoded data of the attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of different levels, and the control file includes a base mesh adaptation set managing the base mesh track, an attribute adaptation set managing the attribute track, and a displacement adaptation set managing the displacement track.

[0023] In an information processing device and method according to one aspect of the present technology, a base grid and attributes are encoded, a displacement vector is encoded at each level in a hierarchy of fineness, a content file is generated, the encoded data of the base grid is stored in a base grid track of the content file, the encoded data of the attributes is stored in an attribute track of the content file, the encoded data of the displacement vectors of each level in the hierarchy is stored in mutually different displacement tracks of the content file, and association information for associating the displacement tracks with the levels is stored in the content file.

[0024] In an information processing device and method according to another aspect of the present technology, encoded data of a base grid and encoded data of attributes are obtained from a content file, and based on association information for associating a displacement track with a level of fineness of a grid, encoded data of a displacement vector is obtained from a displacement track of the content file corresponding to a desired level in the level, each of the acquired encoded data of the base grid, the acquired encoded data of the attributes, and the acquired encoded data of the displacement vector is encoded, and a grid of the desired level is reconstructed using the base grid, attributes, and displacement vectors obtained by decoding.

[0025] In an information processing device and method according to another aspect of the present technology, a base grid and attributes are encoded, a displacement vector is encoded at each level in a hierarchy of fineness, a content file is generated, the encoded data of the base grid is stored in a base grid track of the content file, the encoded data of the attributes is stored in an attribute track of the content file, the encoded data of the displacement vectors of each level in the hierarchy are stored in mutually different displacement tracks of the content file, a control file including control information for controlling the distribution of the content file is generated, and association information for associating a displacement adaptation set for managing the displacement track with the hierarchy is stored in the control file.

[0026] In an information processing device and method according to another aspect of the present technology, based on first association information for associating a displacement adaptation set with a level of fineness of a grid, a displacement adaptation set corresponding to a desired level in the level is selected from the displacement adaptation set, the first association information being stored in a control file including control information for controlling the distribution of a content file, encoded data of a base grid and encoded data of attributes stored in the content file are acquired, encoded data of a displacement vector stored in a displacement track of the content file managed by the selected displacement adaptation set is acquired, each of the acquired encoded data of the base grid, the acquired encoded data of the attributes, and the acquired encoded data of the displacement vector is decoded, and the base grid, attributes, and displacement vectors obtained by decoding are used to reconstruct the grid of the desired level. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] [ Figure 1 ] A diagram used to describe a grid.

[0028] [ Figure 2 ] is used to describe the diagram of V-DMC.

[0029] [ Figure 3 ]A diagram showing a configuration example of a V-DMC bit stream.

[0030] [ Figure 4 ] is used to describe a diagram that layers displacement vectors.

[0031] [ Figure 5 ]A diagram showing an example of a V-DMC bitstream distribution method.

[0032] [ Figure 6 ] A diagram showing an example of a main track configuration of a content file including relative displacement vectors.

[0033] [ Figure 7 ] A diagram showing an example of a main track configuration of a content file in the case of an absolute displacement vector.

[0034] [ Figure 8 ]A diagram showing examples of relative displacement vectors and absolute displacement vectors.

[0035] [ Fig. 9 ]A diagram showing an example of associated information.

[0036] [ Fig.10 ]A diagram showing examples of type information and quality information.

[0037] [ Fig.11 ]A diagram showing examples of association information and type information.

[0038] [ Fig.12 ]A diagram showing an example of an MPD description.

[0039] [ Fig.13 ]A diagram showing an example of quality information.

[0040] [ Fig.14 ]A diagram showing an example of an MPD description.

[0041] [ Fig.15 ]A diagram showing an example of associated information.

[0042] [ Fig.16 ]A diagram showing an example of associated information.

[0043] [ Fig.17 ]A diagram showing an example of grouping information.

[0044] [ Fig.18 ]A diagram showing an example of grouping information.

[0045] [ Fig.19 ]A diagram showing an example of grouping information.

[0046] [ Fig. 20 ]A diagram showing an example of grouping information.

[0047] [ Fig.21 ]A diagram showing an example of grouping information.

[0048] [ Fig. 22 ]A diagram showing an example of an MPD description.

[0049] [ Fig.23 ]A diagram showing an example of an MPD description.

[0050] [ Fig.24 ]A diagram showing an example of an MPD description.

[0051] [ Fig.25 ]A diagram showing an example of associated information.

[0052] [ Fig.26 ]A diagram showing an example of associated information.

[0053] [ Fig. 27 ]A diagram showing an example of an MPD description.

[0054] [ Fig.28 ]A diagram showing an example of an MPD description.

[0055] [ Fig.29 ]A diagram showing a configuration example of a Matroska media container.

[0056] [ Fig.30 ]A block diagram showing a main configuration example of a file generating device.

[0057] [ Fig.31 ]A flowchart describing an example of the process of file generation processing.

[0058] [ Fig.32 ]A flowchart describing an example of the process of file generation processing.

[0059] [ Fig.33 ]A block diagram showing an example of the main configuration of a playback device.

[0060] [ Fig.34 ]A flowchart describing an example of the flow of playback processing.

[0061] [ Fig.35 ]A flowchart describing an example of the flow of playback processing.

[0062] [ Fig.36 ]A flowchart describing an example of the flow of playback processing.

[0063] [ Fig.37 ]A flowchart describing an example of the flow of playback processing.

[0064] [ Fig.38 ]A block diagram showing an example of the main configuration of a computer. DETAILED DESCRIPTION

[0065] Hereinafter, a mode for implementing the present disclosure (hereinafter, referred to as an embodiment) will be described. It should be noted that the description will be given in the following order.

[0066] 1. Documents supporting technical content and technical terminology, etc.

[0067] 2. V-DMC and its distribution

[0068] 3. Level-by-level playback control of displacement vectors

[0069] 4. Level-by-level playback control of attributes

[0070] 5. Matroska Media Container

[0071] 6. First Embodiment (File Generating Device)

[0072] 7. Second Embodiment (Playback Device)

[0073] 8. Appendix

[0074] <1. Documents supporting technical content and technical terms, etc.>

[0075] The scope disclosed in the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents and the like known at the time of filing the present application and the contents of other documents and the like cited in the following non-patent documents and the like.

[0076] Non-patent document 1: (as mentioned above)

[0077] Non-patent document 2: (as mentioned above)

[0078] Non-patent document 3: (as mentioned above)

[0079] Non-patent document 4: (as mentioned above)

[0080] That is to say, the contents described in the above non-patent documents, the contents of other documents cited in the above non-patent documents, etc. are also the basis for determining the support requirements.

[0081] <2. V-DMC and its distribution>

[0082] <Grid>

[0083] Conventionally, as 3D data representing the three-dimensional structure of a three-dimensional structure object (object having a three-dimensional shape), there is a mesh representing the three-dimensional shape of the surface of the object by forming polygons having vertices and connections (also called edges).

[0084] like Figure 1 As shown in the upper left part of , in the mesh, polygonal planes (polygons) are formed by vertices 11 and connections 12 connecting the vertices 11 to each other. In the following, polygons will be described as triangles. Polygons (also called faces) represent the surface of an object with a three-dimensional structure, that is, the three-dimensional shape of the object. It should be noted that a texture 13 can be attached (also called applied to) each face of the mesh.

[0085] For example, the data for the grid is given by Figure 1 The information shown at the bottom of the Figure 1 The vertex information 14 shown first from the left in the bottom row is information indicating the three-dimensional position information (three-dimensional coordinates (X, Y, Z)) of each vertex 11 constituting the mesh. Figure 1 The connection information 15 shown second from the left in the bottom row is information indicating each connection (edge) 12 constituting the mesh. Figure 1 The texture image 16 shown third from the left in the bottom row is mapping information of the texture 13 attached to each surface. Figure 1 The UV sticker shown fourth from the left in the bottom row Fig.17 It is information indicating the correspondence between the vertex 11 and the texture 13. Fig.17 , the coordinates (UV coordinates) of each vertex 11 in the texture image 16 are shown.

[0086] <v-dmc>

[0087] For example, as disclosed in Non-Patent Document 1, there is V-DMC (Video-based Dynamic Mesh Coding) as a coding method for such a mesh.

[0088] In V-DMC, a mesh of a coding target (also referred to herein as an original mesh) is represented by a base mesh having a lower degree of fineness (i.e., coarse) than the original mesh and displacement vectors of segmentation points obtained by subdividing the base mesh, and the base mesh and the displacement vectors are coded. For example, a dynamic mesh stream is set as a coding target, which is non-registered mesh data generated by capturing a camera and whose structure changes every frame.

[0089] For example, suppose there is a Figure 2 The original mesh is shown in the top row. Figure 2 In FIG. 1 , black dots show vertices, and lines connecting the black dots show connections (edges). As described above, a mesh originally forms a plane (polygon) in space through vertices and edges, but here, for ease of description, it is assumed that it is a set of vertices connected by lines (series) in a plane.

[0090] By thinning out some of the vertices of the original mesh, such as Figure 2 A coarser (lower fineness) grid is formed as shown in the second row from the top in . This will be referred to as the base grid.

[0091] By subdividing each polygon of the base mesh, such as Figure 2 vertices and edges are added as shown in the third row from the top in . For example, through this subdivision, the same number of vertices as the vertices that were thinned out in the original mesh can be added. That is, by subdividing the base mesh, a mesh with the same number of vertices as the original mesh is obtained. In this article, the vertices added by such a subdivision will also be referred to as split points.

[0092] However, since the connections are updated when thinning the vertices of the original mesh and the split points are formed on these updated connections (edges), the subdivided base mesh ( Figure 2 The shape of the third row from the top is similar to the original grid ( Figure 2 More specifically, as Figure 2 As shown in the bottom row of , the positions of the segmentation points (on the dotted lines) are different from their positions in the original mesh. In other words, ideally, the original mesh can be restored by moving (shifting) the positions of the segmentation points of the subdivided base mesh to the vertex positions of the original mesh. In this article, such a displacement (shift) of the segmentation points (shown as a vector) will be referred to as a displacement vector.

[0093] That is, in an ideal situation, the original mesh can be represented as a base mesh and a displacement vector. By representing the original mesh as a base mesh and a displacement vector in this way, the number of polygons (i.e., the number of vertices and edges) is reduced. Therefore, by encoding the base mesh and the displacement vector instead of the original mesh, a decrease in decoding efficiency (an increase in the amount of code) can be suppressed.

[0094] That is, in V-DMC, according to Figure 1 the mesh data shown, V-DMC data including a base mesh, a displacement vector, texture, and atlas information is generated, and the V-DMC data is encoded. The mesh data is converted into V-DMC data, and each frame (mesh data representing the three-dimensional structure of a three-dimensional structure object at a certain moment) is independently encoded. For example, in the case where the encoding target is a dynamic mesh stream, the mesh data in each frame is converted into V-DMC data and encoded. With such a configuration, an increase in the amount of code for each frame can be suppressed, and a decrease in decoding efficiency can be suppressed.

[0095] <V-DMC bitstream>

[0096] During encoding, the base mesh, the displacement vector, the texture, and the atlas information are each encoded to generate their respective bitstreams (encoded data). As Figure 3 shown, a single bitstream (V-DMC bitstream) is generated with the base mesh stream, the displacement vector stream, the texture stream, and the atlas information stream as sub-streams.

[0097] The atlas information is necessary for reconstructing the mesh and includes, for example, information indicating the correspondence relationships between the base mesh, the displacement vector, and the texture with each other. The atlas information may include information for associating tiles of the mesh with tiles of attributes, such as Figure 1 the UV map in Fig.17 . The atlas sub-stream, which is a sub-stream of the atlas information, is stored using the extended V-PCC (Video-based Point Cloud Compression) syntax and semantics.

[0098] In the intra-frame case, the base mesh is composed of a static mesh (vertices and connections). In the inter-frame case, the base mesh is composed of motion vectors. That is, the base mesh can include a static mesh and motion vectors. The base mesh sub-stream, which is a sub-stream of the base mesh, is stored using a new data unit added to V-PCC (vuh_unit_type == V3C_MD).

[0099] The displacement vectors are packed into the frame image and encoded into a moving image (also called displacement video) using a coding scheme for moving images. The displacement substream, which is a substream of displacement vectors (displacement video), is stored in the geometry data of the V-PCC.

[0100] The attribute (texture) is packed into the frame image (texture image) and encoded into a moving image (also called attribute video) using a coding scheme for moving images. The attribute substream as the attribute substream (attribute video) is stored in the attribute data of the V-PCC.

[0101] During decoding, the atlas information, base mesh, displacement vector, and texture are restored (generated) by decoding the bitstream using a decoding method corresponding to the encoding method. As described above, the mesh is reconstructed by subdividing the base mesh based on the atlas information, applying the displacement vector to the corresponding segmentation point (i.e., shifting the displacement point), and applying the texture to each face.

[0102] <Hierarchical segmentation>

[0103] Incidentally, the subdivision of the base mesh may be an iterative process of subdividing by a predetermined percentage. Figure 4 As shown in , one subdivision can split an edge into two parts (adding a vertex to each edge), and the subdivision can be repeated. In this case, repeated subdivisions increase the number of split points and improve the fineness of the mesh. Figure 4 In the example of , the base mesh 41-1 is subdivided once to generate segmentation points v1-1 to v1-4, thereby obtaining mesh 41-2 (iteration count = 1). In addition, the mesh 41-2 is subdivided once to generate segmentation points v2-1 to v2-8, thereby obtaining mesh 41-3 (iteration count = 2). In this way, with each repeated subdivision, the number of vertices and connections increases, and the fineness is improved.

[0104] In other words, the number of iterations of subdivision can control the fineness of the resulting mesh. It should be noted that in order to reconstruct the mesh of each fineness, the displacement vector of the segmentation point of each fineness is required. That is, by layering the displacement vector in the V-DMC data in association with the fineness of the mesh, the repeated processing of subdivision as described above becomes feasible, and the fineness of the mesh to be reconstructed can be controlled. That is, the displacement vector of each level should be stored in each frame image in the displacement video.

[0105] exist Figure 4 In the case of the example, each rectangular box in the displacement video 42 represents the displacement vector of each segmentation point. The symbol "vn-m" in the rectangular box represents the segmentation point corresponding to the displacement vector. In the displacement video 42, the displacement vectors are layered. Among them, the displacement vectors in the first row from the top (above the thick line) are the displacement vectors applied to the segmentation points after the first subdivision, and the displacement vectors in the second to fourth rows from the top (below the thick line) are the displacement vectors applied to the segmentation points after the second subdivision. It should be noted that the displacement vector shown in gray in the second row from the top is the relative displacement with respect to the position of the grid 41-2 (iteration count = 1) after applying this displacement vector.

[0106] That is to say, in the case of performing one subdivision on the base grid during reconstruction, the displacement vectors in the first row from the top of the displacement video 42 are respectively applied to the segmentation points v1-1 to v1-4. In addition, in the case of performing two subdivisions on the base grid during reconstruction, the displacement vectors in the first and second rows from the top of the displacement video 42 are respectively applied to the segmentation points v1-1 to v1-4. Additionally, the displacement vectors in the third and fourth rows from the top of the displacement video 42 are respectively applied to the segmentation points v2-1 to v2-8.

[0107] That is to say, the displacement vectors of the segmentation points after the first subdivision, the displacement vectors of the segmentation points after the second subdivision, and so on are all generated after each subdivision, and the iteration count of this subdivision is used as the level. With such a configuration, by applying the displacement vectors of the level corresponding to the iteration count of the subdivision performed on the base grid during reconstruction, a grid with a fineness corresponding to the iteration count can be reconstructed.

[0108] <Distribution of the V-DMC Bitstream>

[0109] In recent years, it is desired to establish a method for storing and distributing the V-DMC encoded bit stream (also referred to as the V-DMC bit stream) in ISOBMFF (International Organization for Standardization Base Media File Format) or the like. For example, as described in non-patent document 2 and non-patent document 3, ISOBMFF is a file container specification of "MPEG-4 (Moving Picture Experts Group-4)", an international standard technology for video compression. In this case, for example, it can be assumed that the V-PCC distribution technology standard ISO / IEC 23090-10 is extended and standardized as described in non-patent document 4. In this case, the V-DMC bit stream is stored in the ISOBMFF track. For example, it can be imagined that sub-streams of atlas information, base grid, displacement vector, and attributes are respectively stored in the ISOBMFF track.

[0110] However, if the entire displacement vector stream (displacement video) is stored on a single track, it is necessary to obtain and decode the displacement vector streams of all levels from the track when decoding the displacement vectors. Therefore, for example, even in the case where a grid with a lower fineness than the original grid is to be reconstructed (i.e., the reconstructed grid does not have the highest fineness), it is necessary to obtain and decode the displacement vector streams of all levels from the track. That is, even unnecessary information must be decoded, which may unnecessarily increase the processing load associated with the reconstruction. That is, the processing may become inefficient, for example, the performance requirements of the hardware required to perform the processing may increase or the processing time may increase. In addition, low processing efficiency may reduce the stability of the processing against external factors such as fluctuations in processing power.

[0111] <3. Level-by-level playback control of displacement vectors>

[0112] <Method 1>

[0113] In view of this, if Figure 5 As shown in the top row of the table in , the displacement video is stored in the track of the content file according to each level (method 1). For example, the configuration of the track is set to Figure 6 In the content file 110. Figure 6 As shown, the content file 110 includes an atlas track 111 , a base grid track 112 , a displacement track 113 , a displacement track 114 , and an attribute track 115 .

[0114] The atlas track 111 is a track storing the encoded data (substream) of the atlas information. The base grid track 112 is a track storing the encoded data (substream) of the base grid. The attribute track 115 is a track storing the encoded data (substream) of the attribute. The displacement track 113 and the displacement track 114 are tracks storing the encoded data (substream) of the displacement vector. The displacement track 113 stores the displacement vector of the level whose subdivision iteration count is 1 (iteration count = 1). The displacement track 114 stores the displacement vector of the level whose subdivision iteration count is 2 (iteration count = 2). It should be noted that in this example, the level number of the displacement vector is 2, and two displacement tracks are used, but the level number of the displacement vector can be any level number in the case where there are multiple displacement vectors. Since the displacement vectors of each level are stored in mutually different tracks of the content file 110, the content file 110 has as many tracks as the level number of the displacement vector. That is, the number of displacement tracks that the content file 110 has can be any number of displacement tracks as long as it is two or more.

[0115] In this way, by storing the displacement vectors of each level in different tracks of the content file 110, the acquisition (and decoding) of the encoded data of the displacement vectors can be controlled on a track-by-track basis during reconstruction. Thus, for example, the encoded data of the displacement vectors can be acquired and decoded from certain displacement tracks. For example, the track storing the displacement vectors required for reconstruction can be selected according to the fineness of the grid to be reconstructed, and the encoded data of the displacement vectors can be acquired and decoded only from the selected track. Such control can suppress unnecessary acquisition and decoding of displacement vectors, such as acquiring and decoding displacement vectors of a level with a fineness higher than that of the grid to be reconstructed, thereby suppressing an increase in the processing load associated with reconstruction. Therefore, an increase in the performance requirements of the hardware required to perform the processing can be suppressed, and an increase in the processing time can be suppressed. That is, the processing associated with the reconstruction can be performed more efficiently. Therefore, a reduction in processing stability can also be suppressed.

[0116] <Relative displacement vector and absolute displacement vector>

[0117] exist Figure 6 In the case of the example in , the displacement track 113 stores a displacement vector group 121-1 of displacement vectors in a level in which the subdivision iteration count is 1 (iteration count=1) in the displacement video 120. That is, the displacement vectors of the segmentation points v1-1 to v1-4 generated in the first subdivision (iteration count=1) are stored in the displacement track 113. In contrast, the displacement track 114 stores a displacement vector group 121-2 of displacement vectors in a level in which the subdivision iteration count is 2 (iteration count=2) in the displacement video 120. That is, the displacement vectors of the segmentation points v2-1 to v2-8 generated in the second subdivision (iteration count=1) and the displacement vectors of the segmentation points v1-1 to v1-4 are stored in the displacement track 114. The displacement vectors of the segmentation points v1-1 to v1-4 shown in gray are relative displacements relative to the position after the displacement vector group 121-1 is applied to the mesh after the first subdivision.

[0118] For example, suppose the surface of the base grid 131 is as follows Figure 8 The upper portion is formed as a solid line, the surface of the mesh 132 after the first subdivision is formed as a dotted line, and the surface of the mesh 133 after the second subdivision is formed as a dashed line. In addition, it is assumed that the first subdivision generates a division point v1-1 at the position of the white circle 141 on the surface of the base mesh 131. Then, it is assumed that as a result of making the shape of the subdivided mesh approximate the original mesh, the division point v1-1 is shifted to the position of the white circle 142 on the surface of the mesh 132 and is shifted to the position of the white circle 143 on the surface of the mesh 133. Figure 6 In the case of the example in , the displacement vector corresponding to the first subdivision (iteration count = 1) of the segmentation point v1-1 is the displacement vector 151 showing the displacement from the white circle 141 to the white circle 142, and the displacement vector corresponding to the second subdivision (iteration count = 2) is the displacement vector 152 showing the displacement from the white circle 142 to the white circle 143. The same applies to the displacement vectors of the segmentation points v1-2 to v1-4. Figure 6 The displacement vectors of the division points v1 - 1 to v1 - 4 shown in gray in FIG. 1 are displacement vectors indicating displacements relative to positions in a hierarchy one level lower, such as the displacement vector 152 .

[0119] Therefore, in order to obtain the position of the segmentation point v1-1 on the surface of the grid 133 (the position of the white circle 143), the displacement vector 151 and the displacement vector 152 are required. That is, in this case, the encoded data of the displacement vector is acquired and decoded not only from the track of the level corresponding to the fineness corresponding to the grid to be reconstructed, but also from all tracks corresponding to the levels with fineness lower than that level. For example, in Figure 6 In the case of , when a mesh having a subdivision iteration count of 1 (iteration count=1) is reconstructed with respect to the geometry, the encoded data is read and decoded from the base grid track 112 and the displacement track 113. In contrast, when a mesh having a subdivision iteration count of 2 (iteration count=2) is reconstructed, the encoded data is read and decoded from the base grid track 112, the displacement track 113, and the displacement track 114.

[0120] Herein, a displacement vector that indicates the displacement of a vertex relative to its position in a lower level of the hierarchy, such as displacement vector 152, will also be referred to as a relative displacement vector. In addition, a displacement vector that indicates the displacement of a vertex relative to its position on the base mesh, such as displacement vector 151, will also be referred to as an absolute displacement vector. For example, in Figure 6 In the above, the displacement vectors constituting the displacement vector group 121-1 (the displacement vectors of the division points v1-1 to v1-4) are absolute displacement vectors. In addition, the displacement vectors of the division points v2-1 to v2-8 in the displacement vector group 121-2 are absolute displacement vectors.

[0121] Although the relative displacement vector can be Figure 6 is stored in the displacement track as in the example of Figure 7 As shown, only the absolute displacement vector is stored in each displacement track. Figure 7 In the example, Figure 6 The displacement track 113 stores the displacement vector group 121-1 of the displacement video 120. In contrast, the displacement track 114 stores the displacement vector group 121-3 of the displacement video 120. In the case of the displacement vector group 121-3, the displacement vectors of the division points v1-1 to v1-4 are also absolute displacement vectors, just like the displacement vectors of the division points v2-1 to v2-8. That is, Figure 8 As shown in the lower portion of , the displacement vector of segmentation point v1-1 corresponding to the second subdivision (iteration count = 2) is displacement vector 153, which shows the displacement from white circle 141 to white circle 143. The same applies to the displacement vectors of segmentation points v1-2 to v1-4.

[0122] Therefore, in order to determine the position of the segmentation point v1-1 on the surface of the grid 133 (the position of the white circle 143), only the displacement vector 153 is required. That is, in this case, only the encoded data of the displacement vector is acquired and decoded from the track corresponding to the level of fineness corresponding to the grid to be reconstructed. For example, Figure 7 In the case of , when the mesh with subdivision iteration count 1 (iteration count = 1) is rebuilt with respect to the geometry, Figure 6 Likewise, the encoded data is read and decoded from the base grid track 112 and the displacement track 113. In contrast, when a grid having a subdivision iteration count of 2 (iteration count=2) is reconstructed, the encoded data is read and decoded from the base grid track 112 and the displacement track 114.

[0123] Each of the methods described below can be applied to Figure 6 The storage method in the example of (wherein the relative displacement vector can be stored in the track) or as Figure 7 The storage method in the example of (where only the absolute displacement vector is stored in each track).

[0124] <Method 1-1>

[0125] In the case where method 1 is applied and the displacement video is stored in the content file track for each level, it is difficult to know which level's displacement vector is stored in which track simply by storing the displacement vector, which may increase unnecessary processing.

[0126] In view of this, in the case of applying method 1, the association information between the level of fineness (subdivision iteration count) and the displacement track can be stored in the content file, such as Figure 5 Then, during reconstruction, the encoded data of the displacement vector can be obtained from the displacement track corresponding to the desired level based on the association information.

[0127] For example, a file generating device (first information processing device) that generates a content file storing 3D data including displacement vectors may include: an encoding unit that encodes a base grid and attributes, and encodes the displacement vectors for each level in a hierarchy of fineness; and a content file generating unit that generates a content file, stores the encoded data of the base grid in a base grid track of the content file, stores the encoded data of the attributes in an attribute track of the content file, stores the encoded data of the displacement vectors of each level in the hierarchy in mutually different displacement tracks of the content file, and stores first association information for associating the displacement tracks with the levels in the content file.

[0128] In addition, in the file generating device (first information processing device), the base grid and the attributes can be encoded, the displacement vector can be encoded for each level in the hierarchy of fineness, a content file can be generated, the encoded data of the base grid can be stored in a base grid track of the content file, the encoded data of the attributes can be stored in the attribute track of the content file, the encoded data of the displacement vector of each level in the hierarchy can be stored in mutually different displacement tracks of the content file, and association information for associating the displacement track with the hierarchy can be stored in the content file.

[0129] It should be noted that the base mesh is a mesh with a lower degree of fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, and the original mesh is composed of vertices and connections representing the three-dimensional structure of the object. Attributes include textures applied to the surface of the original mesh. Displacement vectors are vector information indicating the displacement of vertices obtained by subdividing the base mesh.

[0130] By storing the first association information for associating the displacement track with the hierarchy in the content file in this manner, the first information processing device can enable the playback device that plays back the 3D data to select the displacement track based on the first association information to obtain the encoded data of the displacement vector. Therefore, the playback device can more easily select the displacement track corresponding to the desired hierarchy, and the increase of unnecessary processing can be suppressed. That is, the first information processing device can suppress the increase of the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0131] For example, a playback device (second information processing device) that plays back 3D data including displacement vectors may include: an acquisition unit that acquires encoded data of a base grid and encoded data of attributes from a content file, and based on first association information for associating the displacement track with a level of fineness of the grid, acquires encoded data of the displacement vector from a displacement track of the content file corresponding to a desired level in the level; a decoding unit that decodes each of the acquired encoded data of the base grid, the acquired encoded data of the attributes, and the acquired encoded data of the displacement vector; and a reconstruction unit that reconstructs a mesh of the desired level using the base grid, attributes, and displacement vectors obtained by decoding.

[0132] In addition, in a playback device (second information processing device) that plays back 3D data including displacement vectors, encoded data of a base mesh and encoded data of attributes can be obtained from a content file, and based on association information for associating the displacement track with the level of fineness of the mesh, encoded data of the displacement vector can be obtained from a displacement track of the content file corresponding to a desired level in the level, and each of the obtained encoded data of the base mesh, the obtained encoded data of the attributes, and the obtained encoded data of the displacement vector can be decoded, and the base mesh, attributes, and displacement vectors obtained by decoding can be used to reconstruct a mesh of the desired level.

[0133] It should be noted that the base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing the three-dimensional structure of the object. The attribute includes a texture applied to the surface of the original mesh. The displacement vector is vector information indicating the displacement of the vertex obtained by subdividing the base mesh. The content file includes a base mesh track storing the encoded data of the base mesh, an attribute track storing the encoded data of the attribute, and a plurality of displacement tracks storing the encoded data of the displacement vectors of different hierarchies.

[0134] By selecting the displacement track for obtaining the encoded data of the displacement vector based on the first association information stored in the content file for associating the displacement track with the hierarchy in this manner, the second information processing device can more easily select the displacement track corresponding to the desired hierarchy. That is, the second information processing device can more easily obtain the encoded data of the displacement vector required for reconstruction, and can suppress the increase of unnecessary processing. That is, the second information processing device can suppress the increase of the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0135] <Method 1-1-1>

[0136] In the case of applying method 1-1, subdivision iteration count information indicating the level of the displacement vector stored in the displacement track may be stored as the first association information, such as Figure 5 Then, during reconstruction, the encoded data of the displacement vector can be obtained from the displacement track corresponding to the desired level based on the subdivision iteration count information.

[0137] For example, in the first information processing device, the content file generation unit may store information indicating the hierarchy of the displacement vector stored in the displacement track (subdivision iteration count information) as the first associated information in the displacement track. Furthermore, in the second information processing device, the acquisition unit may acquire the encoded data of the displacement vector based on the information indicating the hierarchy of the displacement vector (subdivision iteration count information), wherein the information (subdivision iteration count information) is stored in the displacement track as the first associated information.

[0138] By referring to the subdivision iteration count information stored in the displacement track, the acquisition unit of the second information processing device can easily determine the level (subdivision iteration count) of the displacement vector stored in the displacement track. Therefore, the second information processing device can more easily select the displacement track corresponding to the desired level and acquire the encoded data of the displacement vector required for reconstruction. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load for reconstructing the 3D data including the displacement vector.

[0139] For example, in the first information processing device, the content file generation unit may store information indicating the hierarchy of the displacement vector (subdivision iteration count information) in the SubdivIterationInfoBox provided in the sample entry of the displacement track. That is, in the content file, the information indicating the hierarchy of the displacement vector (subdivision iteration count information) may be stored in the SubdivIterationInfoBox provided in the sample entry of the displacement track. In the second information processing device, the acquisition unit may acquire the information indicating the hierarchy of the displacement vector (subdivision iteration count information) from the SubdivIterationInfoBox provided in the sample entry of the displacement track.

[0140] For example, Fig. 9 As shown, under schi (Scheme Information), siti (Subdivision iteration information) is set, a new SubdivIterationInfoBox is defined in the sample entry of the displacement track, and the subdivision iteration count information is stored in the SubdivIterationInfoBox. A description example of the SubdivIterationInfoBox is shown in a rectangular frame 161. As shown in this description example, a variable subdiv_iteration_count_minus1 indicating the level of the displacement vector is stored in the SubdivIterationInfoBox as the subdivision iteration count information.

[0141] By referring to the subdivision iteration count information (variable subdiv_iteration_count_minus1) stored in the SubdivIterationInfoBox, the acquisition unit of the second information processing device can easily grasp the level (subdivision iteration count) of the displacement vector stored in the displacement track. Therefore, the second information processing device can more easily select the displacement track corresponding to the desired level, and can obtain the encoded data of the displacement vector required for reconstruction. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. It should be noted that the base grid track can be regarded as corresponding to iteration_count=0.

[0142] It should be noted that, as described above, the displacement vector stored in the displacement track may be a relative displacement vector or may be an absolute displacement vector. For example, in the first information processing device, the content file generation unit may store the encoded data of the absolute displacement vector of the vertex obtained in the corresponding level and the relative displacement vector of the vertex obtained in the level lower than the corresponding level in the displacement track. In addition, in the content file provided to the second information processing device, the absolute displacement vector of the vertex obtained in the corresponding level and the encoded data of the relative displacement vector of the vertex obtained in the level lower than the corresponding level may be stored in the displacement track. That is, in the second information processing device, the acquisition unit may acquire the encoded data of the absolute displacement vector of the vertex obtained in the corresponding level and the relative displacement vector of the vertex obtained in the level lower than the corresponding level from the displacement track of the content file. It should be noted that the absolute displacement vector is a displacement vector indicating the displacement of the vertex obtained by subdividing the mesh relative to the base mesh. In addition, the relative displacement vector is a displacement vector indicating the displacement of the vertex obtained by subdividing the mesh relative to the position in the level of the lower level.

[0143] In addition, in the first information processing device, the control file generation unit may store the encoded data of the absolute displacement vectors of the vertices obtained in the corresponding level and the absolute displacement vectors of the vertices obtained in the level lower than the corresponding level in the displacement track. In addition, in the content file provided to the second information processing device, the encoded data of the absolute displacement vectors of the vertices obtained in the corresponding level and the absolute displacement vectors of the vertices obtained in the level lower than the corresponding level may be stored in the displacement track. That is, in the second information processing device, the acquisition unit may acquire the encoded data of the absolute displacement vectors of the vertices obtained in the corresponding level and the absolute displacement vectors of the vertices obtained in the level lower than the corresponding level from the displacement track of the content file. It should be noted that the absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a base mesh.

[0144] It should be noted that the subdivision iteration count information may be stored in a location other than the displacement track. For example, a SubdivIterationTrackGroupBox may be provided, and the subdivision iteration count information may be stored therein. In this case, a track group is formed by a base grid track and a displacement track. In particular, in the case where a relative displacement vector is stored in a displacement track, encoded data of the displacement vector may be obtained from a plurality of displacement tracks. By grouping such a plurality of displacement tracks, the acquisition unit of the second information processing device may more easily identify the displacement track from which the track group should obtain encoded data of the displacement vector.

[0145] <Method 1-1-1-1>

[0146] In the case of applying method 1-1-1, for a vertex obtained in a level lower than the corresponding level, a relative displacement vector or an absolute displacement vector may be selected as the displacement vector stored in the displacement track. In this case, type information specifying which of the relative displacement vector or the absolute displacement vector is stored in the displacement track may be stored in the content file, such as Figure 5 Then, during reconstruction, the encoded data of the displacement vector can be obtained from the displacement track corresponding to the desired level based on the type information.

[0147] For example, in the first information processing device, the control file generation unit may further store type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the displacement track. Furthermore, in the second information processing device, the acquisition unit may further acquire the encoded data of the displacement vector based on the type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, wherein the type information is stored in the displacement track. It should be noted that the absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a base mesh, and the relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a position at a lower level of hierarchy.

[0148] By referring to the type information stored in the displacement track, the acquisition unit of the second information processing device can easily identify whether the displacement vector stored in the displacement track is an absolute displacement vector or a relative displacement vector. Therefore, the second information processing device can more easily appropriately select the displacement track corresponding to the desired level and acquire the encoded data of the displacement vector required for reconstruction. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0149] For example, in the first information processing device, the control file generation unit may store the type information in the SubdivIterationInfoBox provided in the sample entry of the displacement track. That is, in the content file, the type information may be stored in the SubdivIterationInfoBox provided in the sample entry of the displacement track. In the second information processing device, the acquisition unit may acquire the type information from the SubdivIterationInfoBox provided in the sample entry of the displacement track.

[0150] exist Fig.10 An example description of the SubdivIterationInfoBox in this case is shown in the rectangular frame 162. In this example, the variable absolute_displacement is stored in the SubdivIterationInfoBox as type information together with the variable subdiv_iteration_count_minus1 (subdivision iteration count information) indicating the level of the displacement vector. For example, when the value of the variable absolute_displacement is "0", this indicates that the relative displacement vector is stored as the displacement vector of the vertices obtained in a level lower than the corresponding level. In addition, when the value of the variable absolute_displacement is "1", this indicates that the absolute displacement vector is stored as the displacement vector of these vertices. It should be noted that this value example is an example. Without being limited to this example, the variable absolute_displacement can have any value, and the value can indicate any type of displacement vector.

[0151] By referring to the type information (variable absolute_displacement) stored in the SubdivIterationInfoBox, the acquisition unit of the second information processing device can easily identify whether the displacement vector stored in the displacement track is an absolute displacement vector or a relative displacement vector. Therefore, the second information processing device can more easily and appropriately select the displacement track corresponding to the desired level and obtain the encoded data of the displacement vector required for reconstruction. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0152] <Method 1-1-1-2>

[0153] In the case of applying method 1-1-1, quality information indicating the quality of the displacement vector stored in the displacement track or the base grid stored in the base grid track may be stored in the content file, such as Figure 5 Then, during reconstruction, the encoded data of the displacement vectors can be obtained from the displacement track corresponding to the desired level based on the quality information.

[0154] For example, in the first information processing device, the control file generation unit may further store quality information indicating the quality of the base grid or the displacement vector in the base grid track or the displacement track. In addition, in the second information processing device, the acquisition unit may further acquire the encoded data of the displacement vector based on the quality information indicating the quality of the base grid or the displacement vector, wherein the quality information is stored in the base grid track or the displacement track.

[0155] In V-DMC, there is no restriction on the fineness (e.g., the number of vertices or polygons) of the original mesh or base mesh, and any information about the fineness can be stored in the base mesh track and each displacement track in the content file. Therefore, in the case of reconstructing a mesh with a desired number of vertices and polygons, it is sometimes difficult to determine which level corresponds to the fineness. In view of this, by storing the quality information in the track as described above, the acquisition unit of the second information processing device can easily grasp the quality of the base mesh stored in the base mesh track or the displacement vector stored in the displacement track by referring to the quality information stored in the base mesh track or the displacement track. That is, the second information processing device can more easily identify the level corresponding to the mesh with the desired fineness (the number of vertices and the number of polygons). That is, the second information processing device can more easily select the displacement track corresponding to the desired fineness and acquire the encoded data of the displacement vector required to reconstruct the mesh with the fineness. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load for reconstructing the 3D data including the displacement vector.

[0156] The quality information may be any type of information. For example, in the first information processing device, the control file generation unit may store the average value of the number of vertices of the mesh of the level corresponding to the base grid track or the displacement track in the time axis direction as quality information in the base grid track or the displacement track. That is, the quality information stored in the content file may include the average value of the number of vertices of the mesh of the level corresponding to the base grid track or the displacement track in the time axis direction. In the second information processing device, the acquisition unit may acquire the encoded data of the displacement vector based on the average value of the number of vertices of the mesh of the level corresponding to the base grid track or the displacement track in the time axis direction, wherein the average value is stored in the base grid track or the displacement track as quality information.

[0157] In addition, in the first information processing device, the control file generation unit may store the average value of the number of triangles of the mesh of the level corresponding to the base grid track or the displacement track in the time axis direction as quality information in the base grid track or the displacement track. That is, the quality information stored in the content file may include the average value of the number of triangles of the mesh of the level corresponding to the base grid track or the displacement track in the time axis direction. In the second information processing device, the acquisition unit may acquire the encoded data of the displacement vector based on the average value of the number of triangles of the mesh of the level corresponding to the base grid track or the displacement track in the time axis direction, wherein the average value is stored in the base grid track or the displacement track as quality information.

[0158] It should be noted that the quality information is not limited to such an average value, but may be any statistical value as long as it is a statistical value of multiple frames with respect to any parameter. For example, the quality information may include the median value of the number of vertices or the number of triangles of the mesh of the level corresponding to the base mesh track or the displacement track in the time axis direction, may include the upper limit of the number of vertices or the number of triangles, or may include the lower limit of the number of vertices or the number of triangles.

[0159] For example, in the first information processing device, the control file generation unit may store the quality information in a MeshInfoBox provided in a sample entry of a base grid track or a displacement track. That is, in the content file, the quality information may be stored in a MeshInfoBox provided in a sample entry of a base grid track or a displacement track. In the second information processing device, the acquisition unit may acquire the quality information from the MeshInfoBox provided in the sample entry of the displacement track.

[0160] exist Fig.10 An example description of the MeshInfoBox provided under the schi of the sample entry of the base mesh track or the displacement track is shown in the rectangular box 163 of . In this example, variables ave_vertex_count_minus1 and ave_triangle_count_minus1 are stored in the MeshInfoBox as quality information. The variable ave_vertex_count_minus1 stored in the base mesh track indicates the average value of the number of vertices of the base mesh in the time axis direction. The variable ave_vertex_count_minus1 stored in the displacement track indicates the average value of the number of vertices of the mesh (shifted mesh) to which the subdivision iterations and shifts corresponding to the base mesh have been applied (i.e., the number of vertices of the mesh of the level corresponding to the displacement track) in the time axis direction. The variable ave_triangle_count_minus1 stored in the base mesh track indicates the average value of the number of triangles of the base mesh (the number of mesh triangles) in the time axis direction. The variable ave_triangle_count_minus1 stored in the displacement track indicates the average value of the number of triangles (the number of mesh triangles) of the mesh (shifted mesh) to which the subdivision iterations and shifts corresponding to the base mesh have been applied in the time axis direction (i.e., the average value of the number of triangles of the meshes of the level corresponding to the displacement track in the time axis direction).

[0161] It should be noted that the substream of atlas information (atlas substream) stores the number of vertices and triangles of the mesh subdivided according to the subdivision iteration count per tile, frame by frame. At the system level, the average value of the entire stream is stored per atlas frame so that it can be used to select tracks.

[0162] By referring to the quality information stored in the MeshInfoBox (for example, the variable ave_vertex_count_minus1 and the variable ave_triangle_count_minus1), the acquisition unit of the second information processing device can easily grasp the quality of the base mesh stored in the base mesh track or the displacement vector stored in the displacement track. Therefore, the second information processing device can more easily identify the level corresponding to the mesh with the desired fineness (the number of vertices and the number of polygons). That is, the second information processing device can more easily select the displacement track corresponding to the desired fineness and obtain the encoded data of the displacement vector required to reconstruct the mesh with the fineness. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0163] For example, in the first information processing device, the control file generation unit may store the quality information in a SubdivIterationInfoBox provided in a sample entry of a base grid track or a displacement track. That is, in the content file, the quality information may be stored in a SubdivIterationInfoBox provided in a sample entry of a base grid track or a displacement track. In the second information processing device, the acquisition unit may acquire the quality information from the SubdivIterationInfoBox provided in the sample entry of the displacement track.

[0164] By referring to the quality information stored in the SubdivIterationInfoBox (for example, the variable ave_vertex_count_minus1 and the variable ave_triangle_count_minus1), the acquisition unit of the second information processing device can easily grasp the quality of the base mesh stored in the base mesh track or the displacement vector stored in the displacement track. Therefore, the second information processing device can more easily identify the level corresponding to the mesh with the desired fineness (the number of vertices and the number of polygons). That is, the second information processing device can more easily select the displacement track corresponding to the desired fineness and obtain the encoded data of the displacement vector required to reconstruct the mesh with the fineness. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0165] <Method 1-2>

[0166] The content file may be distributed through MPEG-DASH (Moving Picture Experts Group Dynamic Adaptive Streaming over HTTP (Hypertext Transfer Protocol)). That is, the distribution of the content file may be controlled by using MPD (Media Presentation Description) of MPEG-DASH.

[0167] In this case, if the entire displacement vector stream (displacement video) is stored in a single track, it is necessary to distribute the displacement vector streams of all levels when distributing the displacement vectors because the distribution is controlled track by track. Therefore, for example, even in the case of reconstructing a grid with a lower fineness than the original grid (i.e., the reconstructed grid does not have the highest fineness), it is necessary to distribute the displacement vector streams of all levels from the track. That is, unnecessary information must also be distributed, which may result in low distribution efficiency, for example, increasing the transmission bandwidth required for distribution or increasing the transmission time. In addition, low distribution efficiency may reduce the stability of the distribution against external factors such as bandwidth fluctuations.

[0168] Furthermore, when method 1 is applied and the displacement video is stored in the track of the content file at each level as described above, it is difficult to know which level's displacement vector is stored in which track by only storing the displacement vector, which may increase unnecessary processing.

[0169] Therefore, in the case of applying method 1, the association information between the level of fineness (subdivision iteration count) and the displacement track can be stored in the MPD, such as Figure 5 As shown in the sixth row from the top of the table (method 1-2), the encoded data (stream) of the displacement vector stored in the displacement track corresponding to the desired level can then be distributed based on the association information.

[0170] For example, a file generating device (first information processing device) that generates a content file storing 3D data including displacement vectors may include: an encoding unit that encodes a base grid and attributes, and encodes the displacement vectors at each level in a hierarchy of fineness; a content file generating unit that generates a content file, stores the encoded data of the base grid in a base grid track of the content file, stores the encoded data of the attributes in an attribute track of the content file, and stores the encoded data of the displacement vectors of each level in the hierarchy in mutually different displacement tracks of the content file; and a control file generating unit that generates a control file including control information for controlling distribution of the content file, and stores first association information for associating a displacement adaptation set for managing the displacement track with the hierarchy in the control file.

[0171] In addition, in a file generating device (first information processing device) that generates a content file storing 3D data including displacement vectors, a base grid and attributes can be encoded, a displacement vector can be encoded for each level in a hierarchy of fineness, a content file can be generated, the encoded data of the base grid can be stored in a base grid track of the content file, the encoded data of the attributes can be stored in an attribute track of the content file, the encoded data of the displacement vectors of each level in the hierarchy can be stored in mutually different displacement tracks of the content file, a control file including control information for controlling distribution of the content file can be generated, and association information for associating a displacement adaptation set for managing the displacement track with the hierarchy can be stored in the control file.

[0172] It should be noted that the base mesh is a mesh with a lower degree of fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, and the original mesh is composed of vertices and connections representing the three-dimensional structure of the object. Attributes include textures applied to the surface of the original mesh. Displacement vectors are vector information indicating the displacement of vertices obtained by subdividing the base mesh.

[0173] By storing the first association information for associating the displacement track with the hierarchy in the control file (e.g., MPD) in this manner, the first information processing device can cause the playback device that plays back the 3D data to select the displacement track based on the first association information stored in the control file, and distribute the encoded data of the displacement vector stored in the displacement track. Therefore, the first information processing device can cause the encoded data of the displacement vector to be distributed more efficiently. For example, the first information processing device can suppress the increase in the transmission bandwidth required for distribution, or can suppress the increase in transmission time. In addition, the first information processing device can suppress the reduction in the stability of distribution against external factors such as bandwidth fluctuations. In addition, the first information processing device can suppress the playback device from acquiring or decoding the displacement vector of an unnecessary hierarchy. That is, the first information processing device can also suppress the increase in the processing load associated with reconstruction in the playback device. Therefore, the first information processing device can suppress the increase in the performance requirements of the hardware required to perform the processing, or can suppress the increase in the processing time. That is, the first information processing device can cause the playback device to perform the processing associated with reconstruction more efficiently. Therefore, the first information processing device can also suppress the reduction in the stability of the processing.

[0174] For example, a playback device (second information processing device) that plays back 3D data including displacement vectors may include: a selection unit that selects a displacement adaptation set corresponding to a desired level in the hierarchy among the displacement adaptation sets based on first association information for associating the displacement adaptation set with a level of fineness of a mesh, wherein the first association information is stored in a control file including control information for controlling distribution of a content file; an acquisition unit that acquires encoded data of a base grid and encoded data of attributes stored in the content file, and acquires encoded data of displacement vectors stored in a displacement track of the content file managed by the selected displacement adaptation set; a decoding unit that decodes each of the acquired encoded data of the base grid, the acquired encoded data of the attributes, and the acquired encoded data of the displacement vectors; and a reconstruction unit that reconstructs the mesh of the desired level using the base grid, attributes, and displacement vectors obtained by decoding.

[0175] In addition, in a playback device (second information processing device) that plays back 3D data including displacement vectors, a displacement adaptive set corresponding to a desired level in the hierarchy can be selected from the displacement adaptive sets based on first association information used to associate the displacement adaptive set with a level of fineness of the mesh, wherein the first association information is stored in a control file including control information for controlling the distribution of the content file, encoded data of a base grid and encoded data of attributes stored in the content file can be obtained, encoded data of displacement vectors stored in a displacement track of the content file managed by the selected displacement adaptive set can be obtained, each of the obtained encoded data of the base grid, the obtained encoded data of the attributes, and the obtained encoded data of the displacement vectors can be decoded, and the mesh of the desired level can be reconstructed using the base grid, attributes, and displacement vectors obtained by decoding.

[0176] It should be noted that a base mesh is a mesh having a lower fineness than an original mesh generated by thinning out vertices from an original mesh of an encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of an object. Attributes include a texture applied to a surface of the original mesh. Displacement vectors are vector information indicating displacements of vertices obtained by subdividing the base mesh. The content file includes a base mesh track storing encoded data of a base mesh, an attribute track storing encoded data of attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies. The control file includes a base mesh adaptation set managing a base mesh track, an attribute adaptation set managing an attribute track, and a displacement adaptation set managing a displacement track.

[0177] By selecting the displacement adaptation set based on the first association information for associating the displacement track with the hierarchy stored in the control file in this manner, the second information processing device can more easily select the displacement adaptation set corresponding to the desired hierarchy. In addition, by causing the encoded data of the displacement vector stored in the displacement track corresponding to the selected displacement adaptation set to be distributed, the second information processing device can more efficiently distribute the encoded data of the displacement vector required for reconstruction. For example, the second information processing device can suppress the distribution of unnecessary information, and can suppress the increase in transmission bandwidth information and the increase in transmission time required for distribution. In addition, by suppressing the distribution of unnecessary information, the second information processing device can suppress the reduction in the stability of distribution against external factors such as bandwidth fluctuations. In addition, by suppressing the distribution of unnecessary information, the second information processing device can suppress the increase in unnecessary processing such as obtaining and decoding the encoded data of the displacement vector. That is, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including the displacement vector. Therefore, the second information processing device can suppress the increase in the performance requirements of the hardware required to perform the processing, or can suppress the increase in the processing time. That is, the second information processing device can perform the processing associated with the reconstruction more efficiently. Therefore, the second information processing apparatus can also suppress a decrease in processing stability.

[0178] <Method 1-2-1>

[0179] In the case of applying method 1-2, the subdivision iteration count information indicating the corresponding level of the displacement vector can be stored as the first associated information in the displacement adaptation set of the control file (eg, MPD), such as in Figure 5 Then, during reconstruction, a displacement adaptation set corresponding to the desired level may be selected based on the subdivision iteration count information, and the encoded data of the displacement vectors stored in the displacement track managed by the displacement adaptation set may be distributed.

[0180] For example, in the first information processing device, the control file generation unit may store information indicating the hierarchy of the displacement vectors stored in the displacement track managed by the displacement adaptation set (subdivision iteration count information) as the first associated information in the displacement adaptation set of the control file. Furthermore, in the second information processing device, the selection unit may select one of the displacement adaptation sets based on the information indicating the hierarchy of the displacement vectors stored in the displacement track managed by the displacement adaptation set (subdivision iteration count information), the information (subdivision iteration count information) being stored as the first associated information in the displacement adaptation set of the control file.

[0181] By referring to the subdivision iteration count information stored in the displacement adaptation set of the control file, the selection unit of the second information processing device can more easily grasp the level (subdivision iteration count) of the displacement vector stored in the displacement track managed by the displacement adaptation set. Therefore, the second information processing device can more easily select the displacement adaptation set corresponding to the desired level. In addition, by causing the encoded data of the displacement vector stored in the displacement track corresponding to the selected displacement adaptation set to be distributed, the second information processing device can more efficiently distribute the encoded data of the displacement vector required for reconstruction. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load for reconstructing the 3D data including the displacement vector.

[0182] For example, by extending the V3C video component descriptor, information indicating the level of the displacement vector (subdivision iteration count information) may be defined and stored. Then, during reconstruction, a displacement adaptation set corresponding to a desired level may be selected based on the subdivision iteration count information stored in the basic property or the supplementary property of the displacement adaptation set by using the extended V3C video component descriptor, and the encoded data of the displacement vector stored in the displacement track managed by the displacement adaptation set may be distributed.

[0183] For example, in the first information processing device, the control file generation unit may store a V3C video component descriptor indicating a level as information indicating a level in a basic property or a supplementary property of a displacement adaptation set. In addition, in the second information processing device, the selection unit may select one of the displacement adaptation sets based on the V3C video component descriptor indicating a level, wherein the V3C video component descriptor is stored as information indicating a level in a basic property or a supplementary property of a displacement adaptation set.

[0184] For example, Fig.11 As shown, videoComponent@subdivIterationCount is defined as an extended V3C video component descriptor. The videoComponent@subdivIterationCount is a descriptor indicating a subdivision iteration count corresponding to a displacement video (displacement vector). Using the videoComponent@subdivIterationCount, the subdivision iteration count information is stored in the basic properties or supplementary properties of the displacement adaptation set. Fig.12 An example of the description of the MPD in this case is shown. Fig.12 In the example description in , the basic properties of the displacement adaptation set (AdaptationSet id = "3") (<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), the subdivision iteration count is shown as "1" (subdivisionIterationCount = '1'). In addition, in the basic properties of the displacement adaptation set (AdaptationSet id = "4") (<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), it is shown that the subdivision iteration count is "2" (subdivision iteration count="2").

[0185] Since the information indicating the hierarchy of the displacement vector (subdivision iteration count information) is stored in the basic property or supplementary property of the displacement adaptation set in the MPD in this manner, the selection unit of the second information processing device can more easily grasp the hierarchy (subdivision iteration count) of the displacement vector stored in the displacement track managed by the displacement adaptation set by referring to the information. Therefore, the second information processing device can more easily select the displacement adaptation set corresponding to the desired hierarchy. In addition, by causing the encoded data of the displacement vector stored in the displacement track corresponding to the selected displacement adaptation set to be distributed, the second information processing device can more efficiently distribute the encoded data of the displacement vector required for reconstruction. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0186] It should be noted that instead of using the V3C video component descriptor, a new descriptor indicating information (subdivision iteration count information) representing the level of a displacement vector may be defined.

[0187] <Method 1-2-1-1>

[0188] In addition, in the case of applying method 1-2-1, for a vertex obtained in a level lower than the corresponding level, a relative displacement vector or an absolute displacement vector may be selected as the displacement vector stored in the displacement track. In this case, type information specifying which of the relative displacement vector or the absolute displacement vector is stored in the displacement track may be stored in the MPD, such as Figure 5 Then, during reconstruction, the encoded data of the displacement vector can be obtained from the displacement track corresponding to the desired level based on the type information.

[0189] For example, in the first information processing device, the control file generation unit may further store type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the displacement adaptation set of the control file. Furthermore, in the second information processing device, the selection unit may further select one of the displacement adaptation sets based on the type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, wherein the type information is stored in the displacement adaptation set of the control file. It should be noted that the absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a base mesh, and the relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a position in a hierarchy one level lower.

[0190] By referring to the type information stored in the displacement adaptation set, the selection unit of the second information processing device can easily identify whether the displacement vector stored in the displacement track corresponding to the displacement adaptation set is an absolute displacement vector or a relative displacement vector. Therefore, the second information processing device can more easily select a suitable displacement track corresponding to the desired level and distribute the encoded data of the displacement vector required for reconstruction. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load for reconstructing the 3D data including the displacement vector.

[0191] The type information may be stored, for example, in a basic property or a supplementary property of a displacement adaptation set. For example, by extending a V3C video component descriptor, the type information may be defined and stored. Then, during reconstruction, a displacement adaptation set corresponding to a desired level may be selected based on the type information stored in the basic property or the supplementary property of the displacement adaptation set by using the extended V3C video component descriptor, and the encoded data of the displacement vectors stored in the displacement track managed by the displacement adaptation set may be distributed.

[0192] For example, in the first information processing device, the control file generation unit stores a V3C video component descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector as type information in a basic property or a supplementary property of the displacement adaptation set. In addition, in the second information processing device, the selection unit may select one of the displacement adaptation sets based on the V3C video component descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, wherein the V3C video component descriptor is stored as type information in a basic property or a supplementary property of the displacement adaptation set.

[0193] For example, Fig.11 As shown, videoComponent@absoluteDisplacement is defined as an extended V3C video component descriptor. In the case where the value of videoComponent@absoluteDisplacement is "0", this indicates that relative displacement vectors are stored as displacement vectors of vertices obtained in a hierarchy lower than the corresponding hierarchy. In addition, in the case where the value of videoComponent@absoluteDisplacement is "1", this indicates that absolute displacement vectors are stored as displacement vectors of these vertices. It should be noted that this value example is an example. Without being limited to this example, videoComponent@absoluteDisplacement can have any value, and the value can indicate any type of displacement vector.

[0194] exist Fig.12 In the example description of , the basic properties of the displacement adaptation set (AdaptationSet id = "3") (<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), it is shown that the relative displacement vector is stored (absoluteDisplacement = '0') as the displacement vector of the vertex obtained in the level lower than the corresponding level. In addition, in the basic properties (<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), it is shown that the relative displacement vector is stored (absoluteDisplacement='0') as the displacement vector of the vertex obtained in a hierarchy lower in level than the corresponding hierarchy.

[0195] Since the type information is stored in the basic property or the supplementary property of the displacement adaptation set in the MPD in this way, the selection unit of the second information processing device can easily identify whether the displacement vector stored in the displacement track managed by the displacement adaptation set is an absolute displacement vector or a relative displacement vector by referring to the type information. Therefore, the second information processing device can more easily appropriately select the displacement adaptation set corresponding to the desired level and cause the encoded data of the displacement vector required for reconstruction to be distributed. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0196] It should be noted that instead of using the V3C video component descriptor, a new descriptor indicating type information may be defined.

[0197] <Method 1-2-1-2>

[0198] In the case of applying method 1-2-1, quality information indicating the quality of the displacement vectors stored in the displacement track or the base grid stored in the base grid track may be stored in a control file (e.g., MPD), such as Figure 5 Then, during reconstruction, the encoded data of the displacement vector corresponding to the desired level can be distributed based on the quality information.

[0199] For example, in the first information processing device, the control file generation unit may further store quality information indicating the quality of the base grid or the displacement vector in a base grid adaptation set that manages the base grid track of the control file or in a displacement adaptation set of the control file. In addition, in the second information processing device, the selection unit may further select one of the displacement adaptation sets based on the quality information indicating the quality of the base grid or the displacement vector, wherein the quality information is stored in the base grid adaptation set or the displacement adaptation set.

[0200] In V-DMC, there is no restriction on the fineness (e.g., the number of vertices or polygons) of the original mesh or base mesh, and any information about the fineness can be stored in the base mesh track and each displacement track in the content file. Therefore, in the case of a reconstructed mesh with a desired number of vertices and polygons, it is sometimes difficult to identify which level corresponds to the fineness. In view of this, by storing the quality information in the adaptation set in the control file as described above, the acquisition unit of the second information processing device can easily grasp the quality of the base mesh or displacement vector stored in the base mesh track or displacement track managed by the adaptation set by referring to the quality information stored in the base mesh adaptation set or the displacement adaptation set. That is, the second information processing device can more easily identify the level corresponding to the mesh with the desired fineness (the number of vertices and the number of polygons). That is, the second information processing device can more easily select the displacement adaptation set corresponding to the desired fineness and distribute the encoded data of the displacement vector required to reconstruct the mesh with the fineness. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load for reconstructing the 3D data including the displacement vector.

[0201] The quality information may be stored, for example, in the basic properties or supplementary properties of the displacement adaptation set. For example, a new mesh information descriptor may be defined, and the quality information may be stored by using the mesh information descriptor. Then, during reconstruction, a displacement adaptation set corresponding to the desired fineness (number of vertices and number of polygons) may be selected based on the quality information stored in the basic properties or supplementary properties of the displacement adaptation set by using the mesh information descriptor, and the encoded data of the displacement vectors stored in the displacement track managed by the displacement adaptation set may be distributed.

[0202] For example, in the first information processing device, the control file generation unit may store a mesh information descriptor indicating the quality of the base mesh or the displacement vector as quality information in the basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set. In the second information processing device, the selection unit may select one of the displacement adaptation sets based on the mesh information descriptor indicating the quality of the base mesh or the displacement vector, the mesh information descriptor being stored as quality information in the basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set.

[0203] The quality information may be any type of information. For example, in the first information processing device, the control file generation unit may store a mesh information descriptor indicating the average value of the number of vertices of the mesh of the level corresponding to the base mesh adaptation set or the displacement adaptation set in the time axis direction in the basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set. That is, the quality information stored in the control file may include the average value of the number of vertices of the mesh of the level corresponding to the base mesh adaptation set or the displacement adaptation set in the time axis direction. In the second information processing device, the selection unit may select one of the displacement adaptation sets based on the mesh information descriptor indicating the average value of the number of vertices of the mesh of the level corresponding to the base mesh adaptation set or the displacement adaptation set in the time axis direction, the mesh information descriptor being stored in the basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set.

[0204] In addition, in the first information processing device, the control file generation unit may store a mesh information descriptor indicating an average value of the number of triangles of the mesh of the level corresponding to the base mesh adaptation set or the displacement adaptation set in the time axis direction in the basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set. That is, the quality information stored in the control file may include the average value of the number of triangles of the mesh of the level corresponding to the base mesh track or the displacement track in the time axis direction. In the second information processing device, the selection unit may select one of the displacement adaptation sets based on the mesh information descriptor indicating an average value of the number of triangles of the mesh of the level corresponding to the base mesh adaptation set or the displacement adaptation set in the time axis direction, the mesh information descriptor being stored in the basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set.

[0205] For example, Fig.13 As shown, @aveVertexCount and @aveTriangleCount are defined as mesh information descriptors (MeshInfo descriptors). @aveVertexCount stored in the base mesh adaptation set indicates the average value of the number of vertices of the base mesh in the time axis direction. @aveVertexCount stored in the displacement adaptation set indicates the average value of the number of vertices of the mesh (displaced mesh) to which the subdivision iteration and displacement corresponding to the base mesh are applied (i.e., the number of vertices of the mesh of the level corresponding to the displacement track) in the time axis direction. @aveTriangleCount stored in the base mesh adaptation set indicates the average value of the number of triangles of the base mesh (the number of mesh triangles) in the time axis direction. @aveTriangleCount stored in the displacement adaptation set indicates the average value of the number of triangles of the mesh (displaced mesh) to which the subdivision iteration and displacement corresponding to the base mesh are applied (i.e., the average value of the number of triangles of the mesh of the level corresponding to the displacement track (the number of mesh triangles) in the time axis direction).

[0206] Fig.14 An example of the description of the MPD in this case is shown. Fig.14 In the example description in , in the supplementary properties of the base mesh adaptation set (AdaptationSet id="2"), it is shown as quality information that the average value of the number of vertices of the base mesh in the time axis direction (aveVertexCount) is 3000, and the average value of the number of triangles of the base mesh in the time axis direction (aveTriangleCount) is 1000 ( <SupplementalProperty schemeIdUri="urn:mpeg:mpegI:v3c:2022:meshInfo"aveVertexCount='3000'aveTriangleCount='1000' / > ).

[0207] In addition, in the supplementary properties of the displacement adaptation set (AdaptationSet id="3"), it is shown as quality information that the average value of the number of vertices of the meshes of this level in the time axis direction (aveVertexCount) is 30000, and the average value of the number of triangles of the meshes of this level in the time axis direction (aveTriangleCount) is 10000 ( <SupplementalProperty schemeIdUri="urn:mpeg:mpegI:v3c:2022:meshInfo"aveVertexCount='30000'aveTriangleCount='10000' / > ).

[0208] In addition, in the supplementary properties of the displacement adaptation set (AdaptationSet id="4"), it is shown as quality information that the average value of the number of vertices of the meshes of this level in the time axis direction (aveVertexCount) is 60000, and the average value of the number of triangles of the meshes of this level in the time axis direction (aveTriangleCount) is 20000 ( <SupplementalProperty schemeIdUri="urn:mpeg:mpegI:v3c:2022:meshInfo"aveVertexCount='60000'aveTriangleCount='20000' / > ).

[0209] Since the quality information is stored in the basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set in the MPD in this manner, the selection unit of the second information processing device can easily grasp the quality of the base mesh stored in the base mesh track corresponding to the base mesh adaptation set or the quality of the displacement vector stored in the displacement track corresponding to the displacement adaptation set by referring to the quality information. Therefore, the second information processing device can more easily identify the hierarchy corresponding to the mesh with the desired fineness (number of vertices and number of polygons). That is, the second information processing device can more easily select the base mesh adaptation set or the displacement adaptation set corresponding to the desired fineness, and distribute the encoded data of the displacement vector required to reconstruct the mesh of the fineness. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0210] It should be noted that the quality information may be stored by using a V3C video component descriptor instead of a grid information descriptor. For example, in the first information processing device, the control file generation unit may store a V3C video component descriptor indicating the quality of a base grid or a displacement vector as quality information in a basic property or a supplementary property of a base grid adaptation set or a displacement adaptation set. In the second information processing device, the selection unit may select one of the displacement adaptation sets based on a V3C video component descriptor indicating the quality of a base grid or a displacement vector, the V3C video component descriptor being stored as quality information in a basic property or a supplementary property of a base grid adaptation set or a displacement adaptation set.

[0211] It should be noted that the quality information is not limited to such an average value, but any statistical value as long as it is a statistical value of multiple frames about any parameter. For example, the quality information may include the median value of the number of vertices or the number of triangles of the mesh of the level corresponding to the base mesh track or the displacement track in the time axis direction, may include the upper limit of the number of vertices or the number of triangles, or may include the lower limit of the number of vertices or the number of triangles.

[0212] <4. Level-by-level playback control of attributes>

[0213] <Method 2>

[0214] In the method of storing a V-DMC bitstream in a file container and distributing it, it is conceivable that a substream of an attribute is also stored in a track as described above. In the V-DMC data, an attribute corresponding to a grid having the highest fineness is provided. Therefore, generally, in the case where a substream of an attribute is stored in a track as described above, it is assumed that the substream is stored in a single track.

[0215] However, in this case, it is necessary to obtain all substreams of the attribute from the track and decode it when decoding. That is, the attribute corresponding to the grid with the highest fineness is restored. Therefore, for example, in the case of a grid with a reconstruction fineness lower than the original grid (that is, the reconstructed grid does not have the highest fineness), texture mapping may not be easily and correctly performed due to the mismatch with the fineness of the attribute (that is, it may be difficult to correctly apply the attribute to the face of the grid). This may reduce the quality of the reconstructed grid (for example, the image quality of the reconstructed grid in the displayed image). In addition, if the texture mapping is adjusted to suppress the reduction in grid quality, the amount of processing may increase. In addition, since the processing related to obtaining and decoding unnecessary attributes is also performed, the processing load related to reconstruction may be unnecessarily increased. That is, these may lead to low processing efficiency, for example, increasing the performance requirements of the hardware required to perform the processing or increasing the processing time. In addition, low processing efficiency may reduce the stability of the processing against external factors such as fluctuations in processing power.

[0216] In view of this, if Figure 5 As shown in the 10th row from the top of the table, the attributes are hierarchical by granularity and stored in tracks for each level in the content file (method 2). For example, the configuration of the track is set as follows Fig.15 The configuration shown. Fig.15 In the case of the example in , in addition to the atlas track and the base grid track, the content file also includes multiple (N (N is an arbitrary natural number)) displacement tracks and multiple (N (N is an arbitrary natural number)) attribute tracks.

[0217] That is, the attributes of the V-DMC data are hierarchical according to the fineness of the grid, and the attributes of each hierarchy are encoded independently of each other and stored in mutually different attribute tracks of the content file having such a track configuration.

[0218] With such a configuration, for example, the encoded data of the attribute can be obtained and decoded from certain attribute tracks. For example, control can be performed to select a track storing an attribute having the same fineness as the mesh to be reconstructed, and the encoded data of the attribute can be obtained and decoded only from the selected track. With such control, an attribute having a fineness corresponding to the mesh to be reconstructed can be obtained. Therefore, texture mapping can be performed easily and correctly, and the reduction in the quality of the reconstructed mesh (for example, the image quality of the reconstructed mesh in the displayed image) can be suppressed. In addition, the increase in the processing load associated with the reconstruction can also be suppressed. Therefore, the increase in the performance requirements of the hardware required to perform the processing can be suppressed, and the increase in the processing time can be suppressed. That is, the processing related to the reconstruction can be performed more efficiently. Therefore, the reduction in processing stability can also be suppressed.

[0219] It should also be noted that in the case of method 2, the method of storing the displacement vector can be as follows Figure 6 The relative displacement vector can be stored in the track as in the example of Figure 7 In the example of FIG. 1 , only the absolute displacement vector is stored in each track. In addition, both storage methods can be selected.

[0220] In addition, this method 2 can be applied in combination with the above method 1. For example, Fig.15 In the track configuration shown, each of the displacement vectors and attributes can be layered and stored in mutually different displacement tracks or attribute tracks at each level.

[0221] <Method 2-1>

[0222] In the case where method 2 is applied and attributes are stored in content file tracks per hierarchy, it is difficult to know which hierarchy's attributes are stored in which attribute track by storing only the attributes, which may increase unnecessary processing.

[0223] In view of this, when applying method 2, if Figure 5 As shown in the 11th row from the top of the table, the association information between the attribute track and the displacement track can be stored in the content file (method 2-1). Then, during reconstruction, the encoded data of the attribute can be obtained from the attribute track corresponding to the displacement track corresponding to the desired level based on the association information.

[0224] For example, in a file generating device (first information processing device) that generates a content file storing 3D data including displacement vectors, the encoding unit may encode the attribute in each of the hierarchical levels of fineness. Furthermore, the content file generating unit may store the encoded data of the attribute in each of the hierarchical levels in mutually different attribute tracks of the content file, and store second association information for associating the attribute track with the displacement track in the content file.

[0225] By storing the second association information for associating the attribute track with the displacement track in the content file by the first information processing device in this way, the playback device that plays back the 3D data can more easily select the appropriate attribute track based on the second association information. That is, the playback device can more easily select the appropriate attribute track based on the second association information. That is, the playback device can more easily obtain the attribute with the fineness corresponding to the mesh to be reconstructed. Therefore, the playback device can more easily and correctly perform texture mapping. That is, the first information processing device can suppress the reduction of the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, by selecting the attribute track based on the second association information, the playback device can suppress the increase of unnecessary processing such as obtaining and decoding attributes with unnecessary fineness. That is, the first information processing device can suppress the increase of the processing load associated with the reconstruction of 3D data including the displacement vector. Therefore, the first information processing device can suppress the increase of the performance requirements of the hardware required to perform the processing of the playback device and the increase of the processing time of the playback device. That is, the first information processing device can make the processing related to the reconstruction be performed more efficiently. Therefore, the first information processing device can also suppress the reduction of the stability of the processing related to the reconstruction.

[0226] Furthermore, the content file may include a base grid track, a plurality of attribute tracks storing encoded data of attributes of mutually different hierarchies, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies. Then, in a playback device (second information processing device) that plays back 3D data including the displacement vectors, an acquisition unit may acquire the encoded data of the attributes from the attribute track corresponding to a desired hierarchy in the hierarchy of the content file based on second association information for associating the attribute track with the displacement track.

[0227] By selecting the attribute track of the encoded data for acquiring the attribute based on the second association information stored in the content file for associating the attribute track with the displacement track in this way, the second information processing device can more easily select the attribute track corresponding to the displacement track of the desired level. That is, the second information processing device can more easily select the attribute track corresponding to the level of the mesh to be reconstructed. Therefore, the second information processing device can more easily and correctly perform texture mapping. That is, the second information processing device can suppress the reduction of the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, by selecting the attribute track based on the second association information, the second information processing device can suppress the increase of unnecessary processing such as acquiring and decoding attributes with unnecessary fineness. That is, the second information processing device can suppress the increase of the processing load associated with the reconstruction of 3D data including displacement vectors. Therefore, the second information processing device can suppress the increase of the performance requirements of the hardware required to perform the processing, or can suppress the increase of the processing time. That is, the second information processing device can perform the processing related to the reconstruction more efficiently. Therefore, the second information processing device can also suppress the reduction of the stability of the processing related to the reconstruction.

[0228] <Method 2-1-1>

[0229] In the case of applying method 2-1, subdivision iteration count information indicating the level of the attribute stored in the attribute track may be stored as the second association information, such as Figure 5 As shown in the 12th row from the top of the table (method 2-1-1). Then, during reconstruction, the encoded data of the attribute can be obtained from the attribute track corresponding to the desired level based on the subdivision iteration count information.

[0230] For example, in the first information processing device, the content file generation unit may store information indicating the hierarchy of the attribute stored in the attribute track (subdivision iteration count information) as the second associated information in the attribute track. Furthermore, in the second information processing device, the acquisition unit may acquire the encoded data of the attribute based on the information indicating the hierarchy of the attribute (subdivision iteration count information) stored in the attribute track as the second associated information.

[0231] By referring to the subdivision iteration count information stored in the attribute track, the acquisition unit of the second information processing device can easily grasp the hierarchy (subdivision iteration count) of the attribute stored in the attribute track. Therefore, the second information processing device can more easily select the attribute track corresponding to the desired hierarchy and obtain the encoded data of the attribute required for reconstruction. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0232] For example, in the first information processing device, the content file generation unit may store information indicating the hierarchy of the attribute (subdivision iteration count information) in the SubdivIterationInfoBox provided in the sample entry of the attribute track. That is, in the content file, the information indicating the hierarchy of the attribute (subdivision iteration count information) may be stored in the SubdivIterationInfoBox provided in the sample entry of the attribute track. In the second information processing device, the acquisition unit may acquire the information indicating the hierarchy of the attribute (subdivision iteration count information) from the SubdivIterationInfoBox provided in the sample entry of the attribute track.

[0233] For example, Fig. 9 As shown, under schi (Scheme Information), set siti (Subdivision iteration information) is set, a new SubdivIterationInfoBox is defined in the sample entry of the attribute track, and the subdivision iteration count information is stored in the SubdivIterationInfoBox. Fig.16 A description example of SubdivIterationInfoBox is shown in a rectangular box 161 of FIG. As shown in the description example, a variable subdiv_iteration_count_minus1 indicating the level of an attribute is stored in SubdivIterationInfoBox as subdivision iteration count information. Using the value of the variable subdiv_iteration_count_minus1 (i.e., subdivision iteration count), attribute tracks and displacement tracks corresponding to the same level are grouped.

[0234] By referring to the subdivision iteration count information (variable subdiv_iteration_count_minus1) stored in the SubdivIterationInfoBox, the acquisition unit of the second information processing device can easily grasp the hierarchy (subdivision iteration count) of the attributes stored in the attribute track. Therefore, the second information processing device can more easily select the attribute track corresponding to the desired hierarchy and obtain the encoded data of the attribute required for reconstruction. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0235] It should be noted that in this case, too, the displacement vector stored in the displacement track may be a relative displacement vector or may be an absolute displacement vector.

[0236] <Method 2-1-2>

[0237] In the case of applying method 2-1, if Figure 5 As shown in the 13th row from the top of the table, track group information for grouping the attribute track and the displacement track may be stored as the second association information (method 2-1-2). Then, during reconstruction, the encoded data of the attribute may be obtained from the attribute track corresponding to the displacement track of the desired level based on the track group information. It should be noted that the track group information will also be referred to as grouping information.

[0238] For example, in the first information processing device, the content file generation unit may store grouping information for grouping the attribute track and the displacement track required for reconstructing the grid according to each level in the hierarchy as the second association information in the content file. In addition, in the second information processing device, the acquisition unit may acquire the encoded data of the attribute based on the grouping information for grouping the attribute track and the displacement track required for reconstructing the grid according to each level in the hierarchy, the grouping information being stored in the content file as the second association information.

[0239] Fig.17 is a diagram showing how to group the displacement track and the attribute track by grouping information. Fig.17 In the case of the example in , as shown by the solid line, the topmost attribute track is grouped with the displacement track of iteration_count=1. Therefore, the topmost attribute track is the attribute track of iteration_count=1. That is, in the case where the grid to be reconstructed is a level of iteration_count=1, the encoded data is obtained from the displacement track of iteration_count=1 and the topmost attribute track associated with each other by the solid line and decoded (the description of the base grid and atlas information is omitted). With such a configuration, the displacement vectors and attributes required to reconstruct the grid of the level of iteration_count=1 can be obtained (there is no need to obtain the displacement vectors and attributes of unnecessary levels).

[0240] In addition, as shown by the dotted line, the bottom attribute track is grouped with the displacement tracks of all levels (iteration_count=1 to iteration_count=N). Therefore, the bottom attribute track is the attribute track of iteration_count=N (i.e., the highest fineness). That is, in the case where the grid to be reconstructed is a level of iteration_count=N (i.e., the highest fineness), the encoded data is obtained from the displacement tracks of iteration_count=1 to iteration_count=N and the bottom attribute track associated with each other by the dotted line and decoded (the description of the base grid and atlas information is omitted). With such a configuration, the displacement vectors and attributes required to reconstruct the grid of the level of iteration_count=N can be obtained (there is no need to obtain the displacement vectors and attributes of unnecessary levels).

[0241] By referring to the grouping information (track group information), the acquisition unit of the second information processing device can grasp the combination of the displacement track and the attribute track corresponding to the same level. Therefore, the second information processing device can more easily select the attribute track corresponding to the desired level and obtain the encoded data of the attribute required for reconstruction. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0242] For example, in the first information processing device, the content file generation unit may store the grouping information in SubdivIterationTrackGroupBox. That is, in the content file, the grouping information may be stored in SubdivIterationTrackGroupBox. In the second information processing device, the acquisition unit may acquire the grouping information from SubdivIterationTrackGroupBox.

[0243] For example, Fig.18 As shown in FIG. 1 , under trgr (tack grouping indication), sitg (Subdivision iteration track group box) is set, a new SubdivIterationTrackGroupBox is defined, and the grouping information is stored in the SubdivIterationTrackGroupBox. Fig.18 A description example of SubdivIterationTrackGroupBox is shown in a rectangular box 182 in FIG. As shown in the description example, track_group_id is inherited from TrackGroupTypeBox indicates that group identification information is stored in SubdivIterationTrackGroupBox as grouping information. TrackGroupTypeBox is stored. Attribute tracks and displacement tracks corresponding to the same level are grouped according to the identification information (track_group_id) indicated by track_group_id is inherited from TrackGroupTypeBox.

[0244] By referring to the grouping information (track_group_id is inherited from TrackGroupTypeBox) stored in the SubdivIterationTrackGroupBox, the acquisition unit of the second information processing device can easily identify which tracks belong to the same group. Therefore, the second information processing device can more easily select the displacement track and attribute track corresponding to the desired level, and obtain the encoded data of the displacement vector and attribute required for reconstruction. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0245] It should be noted that in this case, too, the displacement vector stored in the displacement track may be a relative displacement vector or may be an absolute displacement vector. In the case of storing a relative displacement vector in the displacement track, such as Fig.17 As in the example with the dashed line, all displacement tracks required for reconstruction are associated via the grouping information.

[0246] In addition, a track reference may be used instead of the track group identification information (track_group_id) as the above grouping information. A track reference is information indicating a reference relationship between tracks. The displacement track and the attribute track corresponding to the same level may be associated with each other using the track reference. It should be noted that, for example, 4CC may be 'disp'.

[0247] For example, in the first information processing device, the content file generation unit may store, as grouping information, a track reference that refers to other tracks to group attribute tracks and displacement tracks required for reconstructing a grid in each of the hierarchies in the content file. Furthermore, in the second information processing device, the acquisition unit may acquire the encoded data of the attribute based on the track reference that refers to other tracks to group attribute tracks and displacement tracks required for reconstructing a grid in each of the hierarchies, the track reference being stored as grouping information in the content file.

[0248] For example, in Fig.19 In the case where, as indicated by the solid arrow, the track reference (tref) referring to the topmost attribute track is stored in the displacement track of iteration_count=1. That is, the displacement track of iteration_count=1 is associated with the topmost attribute track. Therefore, the topmost attribute track is the attribute track of iteration_count=1. That is, in the case where the grid to be reconstructed is a grid of a level with iteration_count=1, the encoded data is obtained from the displacement track of iteration_count=1 and the topmost attribute track associated through the track reference and decoded (the description of the base grid and atlas information is omitted). With such a configuration, the displacement vectors and attributes required to reconstruct the grid of the level with iteration_count=1 can be obtained (there is no need to obtain the displacement vectors and attributes of unnecessary levels).

[0249] In addition, if Fig.19 As shown by the dotted arrows in , the track reference (tref) referring to the bottom attribute track and the track reference (tref) referring to all other displacement tracks are stored in the displacement track of iteration_count=N. The track reference (tref) referring to the bottom attribute track is stored. That is, the displacement track of iteration_count=N is associated with all other displacement tracks and the bottom attribute track. Therefore, the bottom attribute track is the attribute track of iteration_count=N (i.e., the highest fineness). That is, in the case where the grid to be reconstructed is a grid of a level of iteration_count=N (i.e., the highest fineness), the encoded data is obtained from the displacement tracks and the bottom attribute track of iteration_count=1 to iteration_count=N associated through the track reference and decoded (the description of the base grid and atlas information is omitted). With such a configuration, the displacement vectors and attributes required to reconstruct the grid of the level of iteration_count=N can be obtained (there is no need to obtain the displacement vectors and attributes of unnecessary levels).

[0250] <Method 2-1-2-1>

[0251] In addition, when the method 2-1-2 is applied, if Figure 5 As shown in the 14th row from the top of the table, the subdivision iteration count information can also be stored in SubdivIterationTrackGroupBox (method 2-1-2-1). Then, during reconstruction, the level of the displacement track or attribute track can be identified based on the subdivision iteration count information.

[0252] For example, in the first information processing device, the content file generation unit may further store information indicating the hierarchy (subdivision iteration count information) in SubdivIterationTrackGroupBox. In addition, in the second information processing device, the acquisition unit may further acquire the encoded data of the attribute based on the information indicating the hierarchy (subdivision iteration count information) stored in SubdivIterationTrackGroupBox.

[0253] For example, Fig. 20 As shown, together with the grouping information for grouping the topmost attribute track and the displacement track with iteration_count=1 into groups, information indicating the level corresponding to the topmost attribute track (i.e., iteration_count=1) is stored in SubdivIterationTrackGroupBox. Therefore, based on the information indicating the level, it is easier to grasp that the topmost attribute track is the attribute track with iteration_count=1.

[0254] In addition, together with the grouping information for grouping the bottom attribute track and the displacement tracks of iteration_count=1 to iteration_count=N into groups, information indicating the level corresponding to the bottom attribute track (i.e., iteration_count=N) is stored in SubdivIterationTrackGroupBox. Therefore, based on the information indicating the level, it is possible to more easily grasp that the bottom attribute track is an attribute track of iteration_count=N.

[0255] exist Fig.21 A description example of SubdivIterationTrackGroupBox is shown in a rectangular frame 183 in . As shown in this description example, together with grouping information (track_group_id is inherited from TrackGroupTypeBox), a variable subdiv_iteration_count_minus1 indicating the hierarchy of the attribute is stored in SubdivIterationTrackGroupBox as subdivision iteration count information.

[0256] The acquisition unit of the second information processing device can more easily identify the level corresponding to the attribute track through the subdivision iteration count information stored in the SubdivIterationTrackGroupBox. Therefore, the second information processing device can more easily select the displacement track and the attribute track corresponding to the desired level, and obtain the encoded data of the displacement vector and the attribute required for reconstruction. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including the displacement vector. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including the displacement vector.

[0257] It should be noted that in this case, too, the displacement vector stored in the displacement track may be a relative displacement vector or may be an absolute displacement vector. In the case of storing a relative displacement vector in the displacement track, such as Fig. 20 As in the example with dashed lines in FIG. 2 , all displacement tracks required for reconstruction are associated by grouping information. Also in this case, SubdivIterationInfoBox can be omitted.

[0258] <Method 2-2>

[0259] In the case of method 2, the content file can be distributed through MPEG-DASH as in the case of method 1. That is, the distribution of the content file can be controlled by using MPD of MPEG-DASH.

[0260] For example, in the case of applying method 2, if Figure 5 As shown in the 15th row from the top of the table, the association information between the attribute adaptation set and the displacement adaptation set may be stored in the MPD (method 2-2). Then, the encoded data (stream) of the attribute stored in the attribute track corresponding to the attribute adaptation set corresponding to the desired level (displacement adaptation set) may be distributed based on the association information.

[0261] For example, in a file generating device (first information processing device) that generates a content file storing 3D data including a displacement vector, an encoding unit may encode attributes according to each level in a hierarchy of fineness, the content file generating unit may store the encoded data of the attributes of each level in the hierarchy in mutually different attribute tracks of the content file, and a control file generating unit may store second association information for associating an attribute adaptation set that manages the attribute track with a displacement adaptation set in the control file.

[0262] By storing second association information for associating an attribute track with a displacement track in a control file (e.g., MPD), a playback device that plays back 3D data can more easily select an appropriate attribute adaptation set based on the second association information. That is, the playback device can make it easier to distribute the attributes of the fineness corresponding to the mesh to be reconstructed. Therefore, the playback device can perform texture mapping more easily and correctly. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (e.g., the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can cause the encoded data of the attribute to be distributed more efficiently. For example, the first information processing device can suppress the increase in the transmission bandwidth required for distribution, or can suppress the increase in transmission time. In addition, the first information processing device can suppress the reduction in the stability of distribution against external factors such as bandwidth fluctuations.

[0263] In addition, by distributing attributes based on the second associated information, the playback device can suppress the acquisition or decoding of attributes of unnecessary levels. That is, the first information processing device can also suppress the increase in processing load associated with reconstruction in the playback device. Therefore, the first information processing device can suppress the increase in performance requirements of hardware required to perform processing, or can suppress the increase in processing time. That is, the first information processing device can enable the playback device to perform processing related to reconstruction more efficiently. Therefore, the first information processing device can also suppress the reduction in processing stability.

[0264] In addition, the content file may include a base grid track, a plurality of attribute tracks storing encoded data of attributes of different hierarchies, and a plurality of displacement tracks storing encoded data of displacement vectors of different hierarchies, and the control file may include second association information for associating the attribute adaptation set with the displacement adaptation set, and, in a playback device (second information processing device) that plays back 3D data including the displacement vectors, a selection unit may select an attribute adaptation set corresponding to a desired level in the attribute adaptation set based on the second association information, and an acquisition unit may acquire the encoded data of the attributes stored in the attribute track of the content file managed by the selected attribute adaptation set.

[0265] By storing the second association information for associating the attribute track with the displacement track in the control file (e.g., MPD) in this manner, the second information processing device can more easily select an appropriate attribute adaptation set based on the second association information. That is, the second information processing device can make the attributes of the fineness corresponding to the mesh to be reconstructed easier to distribute. Therefore, the second information processing device can perform texture mapping more easily and correctly. That is, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (e.g., the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can cause the encoded data of the attribute to be distributed more efficiently. For example, the second information processing device can suppress the increase in the transmission bandwidth required for distribution, or can suppress the increase in transmission time. In addition, the second information processing device can suppress the reduction in the stability of distribution against external factors such as bandwidth fluctuations.

[0266] In addition, by distributing attributes based on the second associated information, the second information processing device can suppress the acquisition or decoding of attributes of unnecessary levels. That is, the second information processing device can also suppress the increase in processing load related to reconstruction. Therefore, the second information processing device can suppress the increase in performance requirements of hardware required to perform processing, or can suppress the increase in processing time. That is, the second information processing device can perform processing related to reconstruction more efficiently. Therefore, the second information processing device can also suppress the reduction in processing stability.

[0267] <Method 2-2-1>

[0268] In the case of applying method 2-2, if Figure 5 As shown in the 16th row from the top of the table, the segmentation iteration count information indicating the corresponding level of the attribute may be stored as the second associated information in the attribute adaptation set of the control file (e.g., MPD) (method 2-2-1). Then, the attribute adaptation set corresponding to the desired level may be selected based on the segmentation iteration count information, and the encoded data of the attribute stored in the attribute track managed by the attribute adaptation set may be distributed.

[0269] For example, in the first information processing device, the control file generation unit may store information indicating the hierarchy of the attributes stored in the attribute track managed by the attribute adaptation set (subdivision iteration count information) in the attribute adaptation set of the control file as the second association information. Furthermore, in the second information processing device, the selection unit may select one of the attribute adaptation sets based on the information indicating the hierarchy of the attributes stored in the attribute track managed by the attribute adaptation set (subdivision iteration count information), the information (subdivision iteration count information) being stored in the attribute adaptation set of the control file as the second association information.

[0270] By referring to the subdivision iteration count information stored in the attribute adaptation set of the control file, the selection unit of the second information processing device can more easily grasp the hierarchy (subdivision iteration count) of the attribute stored in the attribute track managed by the attribute adaptation set. Therefore, the second information processing device can more easily select the attribute adaptation set corresponding to the desired hierarchy and cause the encoded data of the attribute stored in the attribute track corresponding to the selected attribute adaptation set to be distributed. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can cause the encoded data of the attribute required for the playback device to be reconstructed to be distributed more efficiently. In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can cause the encoded data of the attribute required for reconstruction to be distributed more efficiently. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0271] For example, in the first information processing device, the control file generation unit may store the V3C video component descriptor indicating the hierarchy as information indicating the hierarchy in the basic property or the supplementary property of the attribute adaptation set. In addition, in the second information processing device, the selection unit may select one of the attribute adaptation sets based on the V3C video component descriptor indicating the hierarchy, the V3C video component descriptor being stored as information indicating the hierarchy in the basic property or the supplementary property of the attribute adaptation set.

[0272] For example, with reference Fig.11 Similar to the described example, information indicating the hierarchy may be stored in the basic property or the supplementary property of the attribute adaptation set by using the V3C video component descriptor (subdivIterationCount).

[0273] Fig. 22 and Fig.23 A description example of MPD is shown. Fig. 22 The description example shown in the rectangular frame 201 in FIG. 1 shows a description example of the first half of the MPD. Fig. 22 The dashed rectangular box 202 in FIG. 1 shows an example of a description of a displacement adaptation set (AdaptationSetid="3"). In the basic properties of the adaptation set (<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), it is shown that the subdivision iteration count is "1" (subdivIterationCount = '1'). In addition, Fig. 22 The dashed rectangle 203 in FIG. 1 shows an example of a description of a displacement adaptation set (AdaptationSet id="4").<EssentialPropertyschemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), it is shown that the subdivision iteration count is "2" (subdivIterationCount='2').

[0274] Fig.23 The description example shown in the rectangular frame 211 of shows a description example of the second half of the MPD. That is, Fig.23 The description example shown in the rectangular frame 211 is Fig. 22 The rectangular box 201 shows a continuation of the description example. Fig.23 The dashed rectangle 212 in FIG. 1 shows an example description of an attribute adaptation set (AdaptationSet id="5").<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), it is shown that the subdivision iteration count is "1" (subdivIterationCount = '1'). In addition, Fig.23 The dashed rectangle 213 in FIG. 1 shows an example of a description of an attribute adaptation set (AdaptationSet id="6").<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), it is shown that the subdivision iteration count is "2" (subdivIterationCount='2').

[0275] That is, the subdivision iteration count information indicates that the displacement adaptation set (AdaptationSet id="3") and the attribute adaptation set (AdaptationSet id="5") correspond to the same level (subdivIterationCount="1"), that is, these adaptation sets are associated. Similarly, the subdivision iteration count information indicates that the displacement adaptation set (AdaptationSet id="4") and the attribute adaptation set (AdaptationSet id="6") correspond to the same level (subdivIterationCount="2"), that is, these adaptation sets are associated.

[0276] Therefore, as described above, the second information processing device can more easily select the attribute adaptation set corresponding to the desired level and cause the encoded data of the attributes stored in the attribute track corresponding to the selected attribute adaptation set to be distributed. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can cause the encoded data of the attributes required for the playback device to be reconstructed to be distributed more efficiently. In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can cause the encoded data of the attributes required for reconstruction to be distributed more efficiently. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0277] It should be noted that instead of using the V3C video component descriptor, a new descriptor indicating information (subdivision iteration count information) representing the level of a displacement vector may be defined.

[0278] <Method 2-2-1-1>

[0279] In addition, in the case of applying method 2-2-1, for vertices obtained in a level lower than the corresponding level, a relative displacement vector or an absolute displacement vector can be selected as the displacement vector stored in the displacement track. In this case, as Figure 5 As shown in the 17th row from the top of the table in , type information specifying which of the relative displacement vector or the absolute displacement vector is stored in the displacement track may be stored in the MPD (method 2-2-1-1). Then, during reconstruction, the encoded data of the attribute stored in the attribute track managed by the attribute adaptation set corresponding to the desired level may be distributed based on the type information.

[0280] For example, in the first information processing device, the control file generation unit may further store type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the attribute adaptation set of the control file. Furthermore, in the second information processing device, the selection unit may further select one of the attribute adaptation sets based on the type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, wherein the type information is stored in the attribute adaptation set of the control file. It should be noted that the absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a base mesh, and the relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a position in a hierarchy one level lower.

[0281] The type information may be stored, for example, in a basic property or a supplementary property of an attribute adaptation set. For example, by extending a V3C video component descriptor, the type information may be defined and stored. Then, during reconstruction, an attribute adaptation set corresponding to a desired level may be selected based on the type information stored in a basic property or a supplementary property of the attribute adaptation set using the extended V3C video component descriptor.

[0282] For example, in the first information processing device, the control file generation unit stores a V3C video component descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector as type information in a basic property or a supplementary property of the displacement adaptation set. In addition, in the second information processing device, the selection unit may select one of the attribute adaptation sets based on the V3C video component descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, the V3C video component descriptor being stored as type information in a basic property or a supplementary property of the attribute adaptation set.

[0283] For example, as referenced Fig.11 As described, videoComponent@absoluteDisplacement is defined as an extended V3C video component descriptor. In the case where the value of videoComponent@absoluteDisplacement is "0", this indicates that relative displacement vectors are stored as displacement vectors of vertices obtained in a level lower than the corresponding level. In addition, in the case where the value of videoComponent@absoluteDisplacement is "1", this indicates that absolute displacement vectors are stored as displacement vectors of these vertices. It should be noted that this value example is an example. Without being limited to this example, videoComponent@absoluteDisplacement can have any value, and the value can indicate any type of displacement vector.

[0284] like Fig.23 As shown in the dashed rectangular box 212 in FIG. 1 , the basic properties (<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), indicating that the relative displacement vector is stored in the displacement track as the displacement vector of the vertex obtained in the level lower than the corresponding level (absoluteDisplacement = '0'). In addition, in the basic properties (<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), indicating that the relative displacement vector is stored in the displacement track as the displacement vector of the vertex obtained in a level lower than the corresponding level (absoluteDisplacement='0').

[0285] Since the type information is stored in the basic property or the supplementary property of the attribute adaptation set in the MPD in this way, the selection unit of the second information processing device can easily identify whether the displacement vector stored in the displacement track managed by the displacement adaptation set corresponding to the attribute adaptation set is an absolute displacement vector or a relative displacement vector by referring to the type information. Therefore, the second information processing device can more easily appropriately select the displacement adaptation set corresponding to the desired level and cause the encoded data of the displacement vector required for reconstruction to be distributed. That is, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including the displacement vector. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of the 3D data including the displacement vector.

[0286] It should be noted that instead of using the V3C video component descriptor, a new descriptor indicating type information may be defined.

[0287] <Method 2-2-2>

[0288] In the case of applying method 2-2, if Figure 5 As shown in the 18th row from the top of the table, attribute adaptation sets and displacement adaptation sets can be grouped by using preselection of MPD (method 2-2-2). Then, during reconstruction, an attribute adaptation set corresponding to a desired level can be selected based on the preselection, and the encoded data of the attributes stored in the attribute track managed by the attribute adaptation set can be distributed.

[0289] For example, in the first information processing device, the control file generation unit may store a preselection for grouping the attribute adaptation set and the displacement adaptation set required for managing the track reconstruction according to each level in the hierarchy as the second association information in the control file. In addition, in the second information processing device, the selection unit may select one of the attribute adaptation sets based on the preselection for grouping the attribute adaptation set and the displacement adaptation set required for managing the track reconstruction according to each level in the hierarchy, wherein the preselection is stored in the control file as the second association information.

[0290] Fig.24 The rectangular box 221 in the figure shows an example description of the MPD in this case. As shown in the description example, a preselection is provided in the MPD, and information for grouping the attribute adaptation set and the displacement adaptation set is stored in the preselection. For example, in the description shown in the dotted rectangular box 222, it is specified that the atlas adaptation set (AdaptationSet id="1") that manages the atlas track, the base grid adaptation set (AdaptationSet id="2") that manages the base grid track, the displacement adaptation set (AdaptationSet id="3") that manages the displacement track, and the attribute adaptation set (AdaptationSet id="5") that manages the attribute track are divided into groups (<Preselection id="1"tag="1"preselectionComponents="1 2 3 5"codecs="v3c1"> ). In addition, in the description shown in the dotted rectangular frame 223, it is specified that the atlas adaptation set (AdaptationSet id="1"), the base grid adaptation set (AdaptationSet id="2"), the displacement adaptation set (AdaptationSet id="3"), the displacement adaptation set (AdaptationSet id="4"), the attribute adaptation set (AdaptationSet id="5"), and the attribute adaptation set (AdaptationSet id="6") are divided into groups (<Preselection id="1"tag="1"preselectionComponents="1 2 3 4 56"codecs="v3c1"> ).

[0291] Therefore, the second information processing device can more easily select the attribute adaptation set corresponding to the desired level based on the pre-selection, and cause the encoded data of the attributes stored in the attribute track corresponding to the selected attribute adaptation set to be distributed. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can cause the encoded data of the attributes required for the playback device to be reconstructed to be distributed more efficiently. In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can cause the encoded data of the attributes required for reconstruction to be distributed more efficiently. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0292] <Method 2-2-2-1>

[0293] It should be noted that, in addition, information indicating the level of the displacement vector (subdivision iteration count information) may be stored in the control file. For example, by a method similar to the case of method 1-2-1, information indicating the level of the displacement vector (subdivision iteration count information) may be stored in the control file.

[0294] For example, in the case of applying method 2-2-2, if Figure 5 As shown in the 19th row from the top of the table in , the subdivision iteration count information can be stored in the displacement adaptation set of the MPD (method 2-2-2-1). Then, during reconstruction, the attribute adaptation set corresponding to the desired level can be selected based on the subdivision iteration count information, and the encoded data of the attributes stored in the attribute track managed by the attribute adaptation set can be distributed.

[0295] For example, the subdivision iteration count information may be stored in the basic properties or supplementary properties of the displacement adaptation set. In addition, the V3C video component descriptor indicating the level may be stored as the subdivision iteration count information. Fig.11 As described, you can store videoComponent@subdivIterationCount. For example, in Fig.24 In the description example in , in the basic properties of the displacement adaptation set (AdaptationSet id = "3")<EssentialPropertyschemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), the subdivision iteration count is shown as "1" (subdivIterationCount = '1'). In addition, in the basic properties of the displacement adaptation set (AdaptationSet id = "4") (<EssentialProperty schemeIdUri="urn:mpeg:mpegI:v3c:2020:component"> ), it is shown that the subdivision iteration count is "1" (subdivIterationCount = '2').

[0296] By storing the subdivision iteration count information in the MPD in this way, it is possible to more easily grasp the level corresponding to the displacement adaptation set based on the subdivision iteration count information.

[0297] <Method 2-2-2-2>

[0298] In addition, information indicating the level of the displacement vector (subdivision iteration count information) may be stored in the preselection of the control file. That is, in the case of applying method 2-2-2, as Figure 5 As shown in the 20th row from the top of the table in , the segmentation iteration count information can be stored in the pre-selection of the MPD (method 2-2-2-2). Then, during reconstruction, the attribute adaptation set corresponding to the desired level can be selected based on the segmentation iteration count information, and the encoded data of the attributes stored in the attribute track managed by the attribute adaptation set can be distributed.

[0299] For example, in the first information processing device, the control file generation unit stores information indicating the hierarchy of the attributes stored in the attribute track managed by the attribute adaptation set in the preselection. In addition, in the second information processing device, the selection unit may select one of the attribute adaptation sets based on the information indicating the hierarchy of the attributes stored in the attribute track managed by the attribute adaptation set, the information being stored in the preselection.

[0300] For example, in the first information processing device, the control file generation unit may store the subdivision iteration information descriptor indicating the hierarchy as information indicating the hierarchy in a pre-selected basic property or supplementary property. In addition, in the second information processing device, the selection unit may select one of the attribute adaptation sets based on the subdivision iteration information descriptor indicating the hierarchy, the subdivision iteration information descriptor being stored as information indicating the hierarchy in a pre-selected basic property or supplementary property.

[0301] For example, Fig.25 As shown, @subdivIterationCount may be provided as a subdivision iteration information descriptor indicating a level, and the @subdivIterationCount may be stored in a pre-selected basic property or a supplementary property. Fig.26 An example of the description of the MPD in this case is shown. Fig.26 In the case of the example in , in the supplementary properties pre-selected in the upper part, subdivIterationCount='1' is shown. In addition, in the supplementary properties pre-selected in the lower part, subdivIterationCount='2' is shown.

[0302] By storing information indicating the hierarchy in the preselection in this way, the hierarchy corresponding to the displacement adaptation set or the attribute adaptation set can be easily grasped based on the information indicating the hierarchy.

[0303] It should be noted that in this case, storing the information indicating the level in the displacement adaptation set or the attribute adaptation set may be omitted.

[0304] <Method 2-2-2-2-1>

[0305] In the case of applying method 2-2-2-2, if Figure 5 As shown in the 21st row from the top of the table, the type information can be stored in the pre-selection of the MPD (method 2-2-2-2-1). Then, during reconstruction, the encoded data of the attribute stored in the attribute track managed by the attribute adaptation set corresponding to the desired level can be distributed based on the type information.

[0306] For example, in the first information processing device, the control file generation unit may store type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the preselection. Furthermore, in the second information processing device, the selection unit may also select one of the attribute adaptation sets based on the type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, the type information being stored in the preselection. It should be noted that the absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a base mesh. The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to a position in a hierarchy one level lower.

[0307] For example, in the first information processing device, the control file generation unit may store a descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector as type information in a pre-selected basic property or a supplementary property. In addition, in the second information processing device, the selection unit may select one of the attribute adaptation sets based on a descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, the descriptor being stored as type information in a pre-selected basic property or a supplementary property.

[0308] For example, Fig.25 As shown, @absoluteDisplacement may be provided as a subdivision iteration information descriptor indicating whether a displacement vector is an absolute displacement vector or a relative displacement vector, and the @absoluteDisplacement may be stored in a pre-selected basic property or supplementary property. Fig.26 In the case of the example in , in the pre-selected supplementary properties, they indicate that the displacement vector is a relative displacement vector (absoluteDisplacement='0').

[0309] By storing the type information in the preselection in this way, it is possible to easily identify whether the displacement vector stored in the displacement track managed by the displacement adaptation set is an absolute displacement vector or a relative displacement vector.

[0310] It should be noted that instead of using the V3C video component descriptor, a new descriptor indicating type information may be defined. In addition, in this case, storing the type information in the displacement adaptation set or the attribute adaptation set may be omitted.

[0311] <Method 2-2-3>

[0312] In the case of applying method 2-2, if Figure 5 As shown in the bottom row of the table in , the displacement adaptation set and the attribute adaptation set can be grouped by the association identification information (association id) or the dependency identification information (dependency id) of the MPD (method 2-2-3). Then, during reconstruction, the encoded data of the attribute stored in the attribute track managed by the attribute adaptation set corresponding to the desired level can be distributed based on the association identification information or the dependency identification information. The association identification information and the dependency identification information are information indicating other adaptation sets associated with the adaptation set.

[0313] For example, in the first information processing device, the control file generation unit may store, as the second association information, in the control file, association identification information for associating the attribute adaptation set and the displacement adaptation set for managing the tracks required for reconstructing the grid in each level of the hierarchy with each other. Furthermore, in the second information processing device, the selection unit may select one of the attribute adaptation sets based on the association identification information for associating the attribute adaptation set and the displacement adaptation set for managing the tracks required for reconstructing the grid in each level of the hierarchy with each other, the association identification information being stored as the second association information in the control file.

[0314] Fig. 27 and Fig.28 A description example of the MPD in this case is shown. Fig. 27 The description example shown in the rectangular frame 231 shows a description example of the first half of the MPD. Fig. 27 The dotted rectangular frame 232 in the figure shows a description example of a representation of a displacement adaptation set (AdaptationSet id="3"). In this representation, identification information of the representation (Representation id="disp1") is shown. In addition, the dotted rectangular frame 233 shows a description example of a representation of a displacement adaptation set (AdaptationSet id="4"). In this representation, identification information of the representation (Representation id="disp2") is shown.

[0315] Fig.28 The description example shown in the rectangular frame 241 of shows a description example of the second half of the MPD. That is, Fig.28 The description example shown in the rectangular frame 241 is Fig. 27 A continuation of the description example shown in rectangular box 231 . Fig.28 The dashed rectangular frame 242 in the figure shows a description example of the representation of the attribute adaptation set (AdaptationSet id="5"). In this representation, the identification information of the representation (Representation id="attr1") is shown. In addition, the dashed rectangular frame 243 shows a description example of the representation of the attribute adaptation set (AdaptationSet id="6"). In this representation, the identification information of the representation (Representation id="attr2") is shown.

[0316] In addition, Fig. 27 The dotted rectangular box 232 in FIG. 1 shows the associated identification information (association id="attr1") of the displacement adaptation set (AdaptationSet id="3"), that is, the displacement adaptation set (AdaptationSet id="3") and the attribute adaptation set (AdaptationSet id="5") are associated with each other. Fig. 27 The dotted rectangular box 233 in FIG. 2 shows the associated identification information (association id="disp2attr1 attr2") of the displacement adaptation set (AdaptationSet id="4"), that is, the displacement adaptation set (AdaptationSet id="4") is associated with the attribute adaptation set (AdaptationSet id="5") and the attribute adaptation set (AdaptationSet id="6").

[0317] Regarding the dependency identification information, association can be performed in a similar manner to the association identification information.

[0318] Therefore, the second information processing device can more easily select the attribute adaptation set corresponding to the desired level based on the association identification information or the dependency identification information, and cause the encoded data of the attribute stored in the attribute track corresponding to the selected attribute adaptation set to be distributed. That is, the first information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the first information processing device can cause the encoded data of the attribute required for the playback device to be reconstructed to be distributed more efficiently. In addition, the first information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors. In addition, the second information processing device can suppress the reduction in the quality of the mesh to be reconstructed (for example, the image quality of the reconstructed mesh in the displayed image). In addition, the second information processing device can cause the encoded data of the attribute required for reconstruction to be distributed more efficiently. In addition, the second information processing device can suppress the increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0319] <Combination>

[0320] Each of the methods mentioned above in <3. Layer-by-layer playback control of displacement vectors> and <4. Layer-by-layer playback control of attributes> can be applied in combination with any other method. That is, two or more of the methods mentioned above can be applied in combination.

[0321] <5. Matroska media container>

[0322] In the above, ISOBMFF is described as an example of a file (file container) for distribution, but any file format or specification for distribution that extends the definition of media associated with 3D data may be used without being limited to ISOBMFF. Fig.29 The Matroska media container is shown in . Of course, any other format may be used.

[0323] <6. First embodiment>

[0324] <File creation device>

[0325] The present technology described above can be applied to any device. Fig.30 : is a block diagram showing a configuration example of a file generating device as an embodiment of an information processing device to which the present technology is applied. Fig.30 The file generating device 300 shown is a device that generates a content file by converting mesh data into V-DMC data, encoding it and storing it in a file container (e.g., ISOBMFF). In addition, the file generating device 300 generates a control file (e.g., MPD of MPEG-DASH) storing control information for controlling the distribution of the content file.

[0326] It should be noted that Fig.30 The main processing units and data flows are shown in FIG. 1 . The present technology is not limited to Fig.30 That is, in the file generating device 300, there may be Fig.30 A processing unit is not shown as a block, or may exist Fig.30 Not shown are processes or data flows, etc., as arrows.

[0327] like Fig.30 As shown, the file generating apparatus 300 includes a grid sparse unit 311 , a V-DMC encoding unit 312 , a bit stream generating unit 313 , a content file generating unit 314 , and an MPD generating unit 315 .

[0328] The mesh sparse unit 311 performs processing related to the sparseness of the mesh. Sparseness is a process of reducing the number of faces (polygons) by sparsely processing some vertices (and connections) from the mesh. For example, the mesh sparse unit 311 obtains mesh data of the encoding target (distribution target) provided to the file generation device 300. The mesh sparse unit 311 generates a base mesh by sparsely processing the acquired mesh data (original mesh). In addition, the mesh sparse unit 311 subdivides the base mesh to generate a displacement vector corresponding to the base mesh. In addition, the mesh sparse unit 311 generates atlas information corresponding to the base mesh, the displacement vector, and the attribute (texture). That is, the mesh sparse unit 311 converts the mesh data into V-DMC data. At this time, the mesh sparse unit 311 performs the subdivision as an iterative process and hierarchizes the displacement vector according to the fineness. The mesh sparse unit 311 provides the generated V-DMC data to the V-DMC encoding unit 312.

[0329] The V-DMC encoding unit 312 performs processing related to encoding of V-DMC data. For example, the V-DMC encoding unit 312 acquires V-DMC data provided by the grid sparse unit 311. In addition, the V-DMC encoding unit 312 encodes the acquired V-DMC data. That is, the V-DMC encoding unit 312 encodes each of the atlas information, the base grid, the displacement vector, and the attribute included in the V-DMC data to generate the encoded data of the atlas information, the encoded data of the base grid, the encoded data of the displacement vector, and the encoded data of the attribute. The V-DMC encoding unit 312 provides the generated encoded data of the atlas information, the generated encoded data of the base grid, the generated encoded data of the displacement vector, and the generated encoded data of the attribute to the bitstream generation unit 313. It can also be said that the V-DMC encoding unit 312 is an encoding unit.

[0330] The bitstream generation unit 313 performs processing related to the generation of a bitstream. For example, the bitstream generation unit 313 acquires the encoded data of the atlas information, the encoded data of the base grid, the encoded data of the displacement vector, and the encoded data of the attribute provided by the V-DMC encoding unit 312. In addition, the bitstream generation unit 313 combines the acquired encoded data of various types into substreams to generate a V-DMC bitstream. The bitstream generation unit 313 provides the generated V-DMC bitstream to the content file generation unit 314 and the MPD generation unit 315.

[0331] The content file generation unit 314 performs processing related to the generation of the content file. For example, the content file generation unit 314 acquires the V-DMC bitstream provided by the bitstream generation unit 313. In addition, the content file generation unit 314 generates a content file such as ISOBMFF and stores the acquired V-DMC bitstream in the content file. The content file generation unit 314 outputs the generated content file to an external device (e.g., a distribution server or a playback device) of the file generation device 300. In addition, the content file generation unit 314 also provides the content file to the MPD generation unit 315.

[0332] The MPD generation unit 315 performs processing related to the generation of a control file (MPD) for controlling the distribution of content files. For example, the MPD generation unit 315 obtains a V-DMC bitstream provided by the bitstream generation unit 313. In addition, the MPD generation unit 315 obtains a content file provided by the content file generation unit 314. The MPD generation unit 315 generates an MPD corresponding to the content file based on the obtained information. The MPD generation unit 315 outputs the generated MPD to an external device (e.g., a distribution server or a playback device) of the file generation device 300. It can also be said that the MPD generation unit 315 is a control file generation unit.

[0333] In the file generation device 300 having the above-mentioned configuration, it can be the first information processing device as described above, and the present technology described in <3. Layer-by-layer playback control of displacement vectors> to <5. Matroska media container> can be applied. That is, the file generation device 300 can be applied in Figure 5 One or more of the various methods shown in the table in to generate a content file and an MPD. With such a configuration, the file generating device 300 can obtain the effects mentioned above in <3. Layer-by-layer playback control of displacement vectors> and <4. Layer-by-layer playback control of attributes>. For example, the file generating device 300 can suppress an increase in a processing load associated with reconstruction of 3D data including displacement vectors.

[0334] For example, the above-mentioned method 1 can be applied to the file generating device 300. In addition, the above-mentioned method 1-1 can be applied to the file generating device 300. That is, the V-DMC encoding unit 312 can encode the base grid and the attribute, and encode the displacement vector according to each level in the hierarchy of fineness. In addition, the content file generating unit 314 can generate a content file, store the encoded data of the base grid in the base grid track of the content file, store the encoded data of the attribute in the attribute track of the content file, store the encoded data of the displacement vector of each level in the mutually different displacement tracks of the content file, and store the first association information for associating the displacement track with the level in the content file. With such a configuration, the file generating device 300 can obtain the effect mentioned above in <method 1-1>. Of course, other methods belonging to this method 1-1 (for example, method 1-1-1, method 1-1-1-1 and method 1-1-1-2) can also be applied to the file generating device 300. In addition, methods other than these methods can be applied.

[0335] In addition, the above-mentioned method 1-2 can be applied to the file generating device 300. That is, the V-DMC encoding unit 312 can encode the base grid and the attribute, and encode the displacement vector according to each level in the hierarchy of fineness. In addition, the content file generating unit 314 can generate a content file, store the encoded data of the base grid in the base grid track of the content file, store the encoded data of the attribute in the attribute track of the content file, and store the encoded data of the displacement vector of each level in mutually different displacement tracks of the content file. In addition, the MPD generating unit 315 can generate a control file (MPD) including control information for controlling the distribution of the content file, and store the first association information for associating the displacement adaptation set for managing the displacement track with the level in the control file (MPD). With such a configuration, the file generating device 300 can obtain the effect mentioned above in <method 1-2>. Of course, other methods belonging to this method 1-2 (for example, method 1-2-1, method 1-2-1-1 and method 1-2-1-2) can be applied to the file generating device 300. Furthermore, methods other than these methods may be applied.

[0336] In addition, the above-mentioned method 2 can be applied to the file generating device 300. In addition, the above-mentioned method 2-1 can be applied to the file generating device 300. That is, the V-DMC encoding unit 312 can encode the attributes according to each level in the hierarchy of fineness. In addition, the content file generating unit 314 can store the encoded data of the attributes of each level in the hierarchy in mutually different attribute tracks of the content file, and store the second association information for associating the attribute track with the displacement track in the content file. With such a configuration, the file generating device 300 can obtain the effect mentioned above in <Method 2-1>. Of course, other methods belonging to this method 2-1 (for example, method 2-1-1, method 2-1-2, and method 2-1-2-1) can be applied to the file generating device 300. In addition, methods other than these methods can be applied.

[0337] In addition, the above-mentioned method 2-2 can be applied to the file generation device 300. That is, the V-DMC encoding unit 312 can encode the attributes according to each level in the hierarchy of fineness. In addition, the content file generation unit 314 can store the encoded data of the attributes of each level in the hierarchy in mutually different attribute tracks of the content file. In addition, the MPD generation unit 315 can store the second association information for associating the attribute adaptation set of the management attribute track with the displacement adaptation set in the control file (MPD). With such a configuration, the file generation device 300 can obtain the effect mentioned above in <Method 2-2>. Of course, other methods belonging to this method 2-2 (for example, method 2-2-1, method 2-2-1-1, method 2-2-2, method 2-2-2-1, method 2-2-2-2, method 2-2-2-2-1 and method 2-2-3) can be applied to the file generation device 300. In addition, methods other than these methods can be applied.

[0338] <File Generation Process Flow 1>

[0339] Next, we will refer to Fig.31 An example of the flow of the file generation processing executed by the file generation device 300 is described with reference to the flowchart in FIG. Fig.31 The flowchart in shows an example of the flow of file generation processing in the case where the above-mentioned method 1 (and other methods belonging to method 1) are applied.

[0340] When the file generation process starts, in step S301 , the mesh thinning unit 311 of the file generation device 300 generates V-DMC data by, for example, thinning mesh data.

[0341] In step S302, the V-DMC encoding unit 312 encodes atlas information, base grid, and attribute data of the V-DMC data. In step S303, the V-DMC encoding unit 312 encodes a displacement vector in each of the levels of detail.

[0342] In step S304, the bitstream generation unit 313 combines the encoded data of the atlas information, the encoded data of the base grid and the encoded data of the attributes generated in step S302, and the encoded data of the displacement vector of each level generated in step S303 into a substream to generate a V-DMC stream.

[0343] In step S305, the content file generation unit 314 generates a content file and stores the substream of the displacement vector of each layer stored in the V-DMC bitstream in different tracks for each layer. In addition, the content file generation unit 314 stores other substreams (the substream of atlas information, the substream of base grid, and the substream of attribute) stored in the V-DMC bitstream in tracks respectively.

[0344] Furthermore, in step S306 , the content file generation unit 314 stores association information for associating the displacement track with the hierarchy (subdivision iteration count) in the content file generated in step S305 .

[0345] In step S307 , the MPD generation unit 315 generates a control file (MPD) that controls the distribution of the content file generated in step S305 .

[0346] In step S308 , the MPD generation unit 315 stores association information for associating the displacement track with the hierarchy (subdivision iteration count) in the control file (MPD) generated in step S307 .

[0347] When the processing in step S307 is completed, the file generation processing ends.

[0348] By performing the corresponding processing as described above, the file generation device 300 can obtain the effects mentioned above in <3. Layer-by-layer playback control of displacement vectors> For example, the file generation device 300 can suppress an increase in the processing load associated with reconstruction of 3D data including displacement vectors.

[0349] <File Generation Process Flow 2>

[0350] Next, we will refer to Fig.32 An example of the flow of the file generation processing executed by the file generation device 300 is described with reference to the flowchart in FIG. Fig.32 The flowchart in shows an example of the flow of file generation processing in the case where the above-mentioned method 2 (and other methods belonging to method 2) are applied.

[0351] When the file generation process starts, in step S321 , the mesh thinning unit 311 of the file generation device 300 generates V-DMC data by, for example, thinning mesh data.

[0352] In step S322, the V-DMC encoding unit 312 encodes the atlas information and the base grid of the V-DMC data. In step S323, the V-DMC encoding unit 312 encodes the displacement vector at each level of the hierarchy of fineness. In step S324, the V-DMC encoding unit 312 encodes the attribute at each level of the hierarchy of fineness.

[0353] In step S325, the bitstream generation unit 313 combines the encoded data of the atlas information and the base grid generated in step S322, the encoded data of the displacement vector of each level generated in step S323, and the encoded data of the attributes of each level generated in step S324 into a sub-stream to generate a V-DMC stream.

[0354] In step S326, the content file generation unit 314 generates a content file, and stores the substream of the displacement vector of each level stored in the V-DMC bitstream in a mutually different track for each level. In addition, the content file generation unit 314 stores the substream of the attribute of each level stored in the V-DMC bitstream in a mutually different track for each level of the content file. In addition, the content file generation unit 314 stores other substreams (the substream of the atlas information and the substream of the base grid) stored in the V-DMC bitstream in tracks respectively.

[0355] In step S327, the content file generation unit 314 stores the first association information for associating the displacement track with the hierarchy (subdivision iteration count) in the content file generated in step S326. In addition, in step S328, the content file generation unit 314 stores the second association information for associating the attribute track with the displacement track in the content file generated in step S326.

[0356] In step S329 , the MPD generation unit 315 generates a control file (MPD) that controls the distribution of the content file generated in step S326 .

[0357] In step S330, the MPD generation unit 315 stores the first association information for associating the displacement track with the level (subdivision iteration count) in the control file (MPD) generated in step S329. In addition, in step S331, the MPD generation unit 315 stores the second association information for associating the attribute track with the displacement track in the control file (MPD) generated in step S329.

[0358] When the processing in step S331 is completed, the file generation processing ends.

[0359] By performing the corresponding processing as described above, the file generation device 300 can obtain the effects mentioned above in <4. Layer-by-layer playback control of attributes> For example, the file generation device 300 can suppress an increase in the processing load associated with reconstruction of 3D data including displacement vectors.

[0360] <7. Second Embodiment>

[0361] <Playback Device>

[0362] The above-mentioned present technology can be applied to any device. Fig.33 : is a block diagram showing a configuration example of a playback device as an embodiment of an information processing device to which the present technology is applied. Fig.33 The playback device 400 shown is a playback device that performs playback processing of mesh data (V-DMC data). For example, the playback device 400 acquires the content file generated by the file generating device 300, and plays back the V-DMC data stored in the content file.

[0363] It should be noted that Fig.33 In the figure, the main processing units and data flows are shown, and the present technology is not limited to Fig.33 That is, in the playback device 400, there may be Fig.33 A processing unit is not shown as a block, or may exist Fig.33 Not shown are processes or data flows, etc., as arrows.

[0364] like Fig.33 As shown, the playback device 400 includes a file acquisition unit 411, an MPD analysis unit 412, a file processing unit 413, a decoding unit 414, a reconstruction unit 415, and a display processing unit 416.

[0365] The file acquisition unit 411 performs processing related to the acquisition of files such as content files and control files (MPD). For example, the file acquisition unit 411 acquires the control file (MPD) generated by the file generation device 300 to which the present technology is applied. The file acquisition unit 411 provides the acquired control file (MPD) to the MPD analysis unit 412.

[0366] In addition, the file acquisition unit 411 acquires the content file generated by the file generation device 300 to which the present technology is applied according to the control of the MPD analysis unit 412. For example, the file acquisition unit 411 acquires a bit stream (substream) stored in a content file distributed from a distribution server or the like according to the control of the MPD analysis unit 412. The file acquisition unit 411 supplies the acquired bit stream (substream) to the file processing unit 413. It can also be said that the file acquisition unit 411 is an acquisition unit.

[0367] The MPD analysis unit 412 performs processing related to the analysis of the control file (MPD). For example, the MPD analysis unit 412 obtains the control file (MPD) provided by the file acquisition unit 411. The MPD analysis unit 412 analyzes the obtained control file (MPD) and controls the file acquisition unit 411 based on the analysis result to obtain the bitstream (substream) stored in the content file. For example, in the case of obtaining some substreams of the substreams stored in the content file, the MPD analysis unit 412 selects some substreams based on the control file (MPD). It can also be said that the MPD analysis unit 412 is a selection unit.

[0368] It should be noted that the distribution of content files is not necessarily controlled by the control file (MPD). For example, the file acquisition unit 411 can acquire a content file provided by an external device of the playback apparatus 400, and provide the acquired content file to the file processing unit 413. In this case, the MPD analysis unit 412 can be omitted.

[0369] The file processing unit 413 performs processing related to the content file. For example, the file processing unit 413 obtains the content file provided by the file acquisition unit 411. The file processing unit 413 obtains the bit stream (substream) required for reconstructing the grid from the acquired content file. For example, the file processing unit 413 obtains the substream of atlas information, the substream of the base grid, the substream of the displacement vector, and the substream of the attribute from the content file. The file processing unit 413 provides the acquired bit stream to the decoding unit 414. It can also be said that the file processing unit 413 is an acquisition unit.

[0370] It should be noted that, in the case where a bit stream (substream) is provided from the file acquisition unit 411 to the file processing unit 413, the file processing unit 413 provides the provided bit stream to the decoding unit 414. In this case, the file processing unit 413 may be omitted, and the bit stream may be provided from the file acquisition unit 411 to the decoding unit 414.

[0371] The decoding unit 414 performs processing related to decoding the bitstream. For example, the decoding unit 414 obtains the bitstream provided by the file processing unit 413 (for example, a substream of atlas information, a substream of base grid, a substream of displacement vector, a substream of attributes). The decoding unit 414 decodes the obtained bitstream using a decoding method according to the encoding method of the bitstream. For example, the decoding unit 414 generates (restores) information such as atlas information, base grid, displacement vector, and attribute by decoding. That is, the decoding unit 414 generates (restores) V-DMC data. The decoding unit 414 provides the generated V-DMC data to the reconstruction unit 415.

[0372] The reconstruction unit 415 performs processing related to the reconstruction of the mesh (3D data). For example, the reconstruction unit 415 converts the V-DMC data provided by the decoding unit 414 into mesh data, and reconstructs the mesh (3D data) using the mesh data. The reconstruction unit 415 provides the reconstructed mesh (3D data) to the display processing unit 416.

[0373] The display processing unit 416 performs processing related to displaying the mesh. For example, the display processing unit 416 acquires the mesh (3D data) provided by the reconstruction unit 415. The display processing unit 416 generates a display image for displaying the mesh (3D data) and provides the display image to a display unit (not shown) such as a monitor for display.

[0374] In the playback device 400 having the above-mentioned configuration, it can be the second information processing device as described above, and the present technology described above in <3. Hierarchical playback control of displacement vectors> to <5. Matroska media container> can be applied to the playback device. That is, the playback device 400 can apply Figure 5 The mesh is reconstructed using one or more of the various methods shown in the table in . With such a configuration, the playback device 400 can obtain the effects mentioned above in <3. Layer-by-layer playback control of displacement vectors> and <4. Layer-by-layer playback control of attributes>. For example, the playback device 400 can suppress an increase in the processing load associated with the reconstruction of 3D data including displacement vectors.

[0375] For example, the above method 1 can be applied to the playback device 400. In addition, the above method 1-1 can be applied to the playback device 400. That is, the file processing unit 413 can obtain the encoded data of the base grid and the encoded data of the attribute from the content file, and can obtain the encoded data of the displacement vector from the encoded data of the displacement track corresponding to the desired level in the level of the grid based on the first association information for associating the displacement track with the level of fineness of the grid. In addition, the decoding unit 414 can decode each of the encoded data of the acquired base grid, the encoded data of the acquired attribute, and the encoded data of the acquired displacement vector. In addition, the reconstruction unit 415 can reconstruct the grid of the desired level using the base grid, the attribute, and the displacement vector obtained by decoding. With such a configuration, the playback device 400 can obtain the effect mentioned above in <Method 1-1>. Of course, other methods belonging to this method 1-1 (for example, method 1-1-1, method 1-1-1-1, and method 1-1-1-2) can be applied to the playback device 400. Furthermore, methods other than these methods may be applied.

[0376] In addition, the above-mentioned method 1-2 can be applied to the playback device 400. That is, the MPD analysis unit 412 can select a displacement adaptation set corresponding to a desired level in the level of the displacement adaptation set based on the first association information for associating the displacement adaptation set with the level of fineness of the grid, and the first association information is stored in a control file including control information for controlling the distribution of the content file. In addition, the file acquisition unit 411 can acquire the encoded data of the base grid and the encoded data of the attribute stored in the content file, and acquire the encoded data of the displacement vector stored in the displacement track of the content file managed by the selected displacement adaptation set. The decoding unit 414 can decode each of the acquired encoded data of the base grid, the acquired encoded data of the attribute, and the acquired encoded data of the displacement vector. In addition, the reconstruction unit 415 can reconstruct the grid of the desired level using the base grid, the attribute, and the displacement vector obtained by decoding. With such a configuration, the playback device 400 can obtain the effects mentioned above in <Method 1-2>. Of course, other methods belonging to the method 1-2 (for example, the method 1-2-1, the method 1-2-1-1, and the method 1-2-1-2) can be applied to the playback apparatus 400. Furthermore, methods other than these methods can be applied.

[0377] In addition, the above method 2 can be applied to the playback device 400. In addition, the above method 2-1 can be applied to the playback device 400. That is, the content file may include a base grid track, a plurality of attribute tracks storing encoded data of attributes of mutually different hierarchies, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies. In addition, the file processing unit 413 may obtain the encoded data of the attribute from the attribute track corresponding to the desired hierarchy in the content file based on the second association information for associating the attribute track with the displacement track.

[0378] In addition, the above-mentioned method 2-2 can be applied to the playback device 400. That is, the content file may include a base grid track, multiple attribute tracks storing encoded data of attributes of different levels, and multiple displacement tracks storing encoded data of displacement vectors of different levels. In addition, the control file may include second association information for associating the attribute adaptation set with the displacement adaptation set. In addition, the MPD analysis unit 412 can select the attribute adaptation set corresponding to the desired level in the attribute adaptation set based on the second association information, and the file acquisition unit 411 can obtain the encoded data of the attribute stored in the attribute track managed by the selected attribute adaptation set in the content file. With such a configuration, the playback device 400 can obtain the effect mentioned above in <Method 2-2>. Of course, other methods belonging to this method 2-2 (for example, method 2-2-1, method 2-2-1-1, method 2-2-2, method 2-2-2-1, method 2-2-2-2, method 2-2-2-2-1 and method 2-2-3) can be applied to the playback device 400. Furthermore, methods other than these methods may be applied.

[0379] <Playback Processing Flow 1>

[0380] Next, we will refer to Fig.34 The flowchart in describes an example of the flow of playback processing performed by the playback device 400. Fig.34 The flowchart in shows an example of the flow of playback processing in the case where the above-mentioned method 1-1 (and other methods belonging to method 1-1) are applied.

[0381] When the playback process starts, in step S401 , the file acquisition unit 411 of the playback device 400 acquires a content file.

[0382] In step S402, the file processing unit 413 selects a displacement track corresponding to a desired level from the acquired content file based on association information for associating the displacement track with the level of fineness of the grid (subdivision iteration count), and extracts a substream of the displacement vectors stored in the displacement track. In step S403, the file processing unit 413 extracts other substreams from the content file.

[0383] In step S404, the decoding unit 414 decodes each of the extracted substreams and generates V-DMC data. The reconstruction unit 415 generates mesh data using the V-DMC data.

[0384] In step S405 , the reconstruction unit 415 reconstructs the mesh (3D data) using the mesh data.

[0385] In step S406 , the display processing unit 416 generates display information including a grid, and causes the display information to be displayed.

[0386] When the processing in step S406 is completed, the playback processing ends.

[0387] By performing the corresponding processing as described above, the playback device 400 can obtain the effects mentioned above in <3. Layer-by-layer playback control of displacement vectors> For example, the playback device 400 can suppress an increase in the processing load associated with reconstruction of 3D data including displacement vectors.

[0388] <Playback Processing Flow 2>

[0389] Next, we will refer to Fig.35 The flowchart in describes an example of the flow of playback processing performed by the playback device 400. Fig.35 The flowchart in shows an example of the flow of playback processing in the case where the above-mentioned method 1-2 (and other methods belonging to method 1-2) are applied.

[0390] When the playback process starts, in step S421 , the file acquisition unit 411 of the playback device 400 acquires a control file (MPD).

[0391] In step S422, the MPD analysis unit 412 analyzes the control file (MPD) and selects a displacement adaptation set corresponding to the desired level based on association information for associating displacement tracks with levels (subdivision iteration counts), the association information being stored in the control file (MPD).

[0392] In step S423, the file acquisition unit 411 acquires the substream of the displacement vectors stored in the displacement track corresponding to the selected displacement adaptation set according to the control of the MPD analysis unit 412. In step S424, the file acquisition unit 411 acquires other necessary substreams (e.g., the substream of atlas information, the substream of base grids, the substream of displacement vectors, and the substream of attributes) according to the control of the MPD analysis unit 412.

[0393] In step S425, the decoding unit 414 decodes those sub-streams and generates V-DMC data. The reconstruction unit 415 converts the V-DMC data into mesh data.

[0394] In step S427 , the reconstruction unit 415 reconstructs the mesh (3D data) using the mesh data.

[0395] In step S427, the display processing unit 416 generates display information including a grid, and causes the display information to be displayed.

[0396] After the processing in step S427 is completed, the playback processing ends.

[0397] By performing the corresponding processing as described above, the playback device 400 can obtain the effects mentioned above in <3. Layer-by-layer playback control of displacement vectors> For example, the playback device 400 can suppress an increase in the processing load associated with reconstruction of 3D data including displacement vectors.

[0398] <Playback Processing Flow 3>

[0399] Next, we will refer to Fig.36 The flowchart in describes an example of the flow of playback processing performed by the playback device 400. Fig.36 The flowchart in shows an example of the flow of playback processing in the case where the above-mentioned method 2-1 (and other methods belonging to method 2-1) are applied.

[0400] When the playback process starts, in step S441 , the file acquisition unit 411 of the playback device 400 acquires a content file.

[0401] In step S442, the file processing unit 413 selects a displacement track corresponding to a desired level from the content file based on first association information for associating the displacement track with the level of fineness of the grid (subdivision iteration count), and extracts a substream of displacement vectors stored in the displacement track. In step S443, the file processing unit 413 selects an attribute track corresponding to the desired level from the content file based on second association information for associating the attribute track with the displacement track, and extracts a substream of attributes stored in the attribute track. In step S444, the file processing unit 413 extracts other necessary substreams (e.g., a substream of atlas information, a substream of base grid, and a substream of attributes) from the content file.

[0402] In step S445, the decoding unit 414 decodes each of the extracted substreams and generates V-DMC data. The reconstruction unit 415 generates mesh data using the V-DMC data.

[0403] In step S446 , the reconstruction unit 415 reconstructs the mesh (3D data) using the mesh data.

[0404] In step S447, the display processing unit 416 generates display information including a grid, and causes the display information to be displayed.

[0405] When the processing in step S447 is completed, the playback processing ends.

[0406] By performing the corresponding processing as described above, the playback device 400 can obtain the effects mentioned above in <4. Layer-by-layer playback control of attributes> For example, the playback device 400 can suppress an increase in the processing load associated with reconstruction of 3D data including displacement vectors.

[0407] <Playback Processing Flow 4>

[0408] Next, we will refer to Fig.37 The flowchart in describes an example of the flow of playback processing performed by the playback device 400. Fig.37 The flowchart in shows an example of the flow of playback processing in the case where the above-mentioned method 2-2 (and other methods belonging to method 2-2) are applied.

[0409] When the playback process starts, in step S461 , the file acquisition unit 411 of the playback device 400 acquires a control file (MPD).

[0410] In step S462, the MPD analysis unit 412 analyzes the control file (MPD) and selects a displacement adaptation set corresponding to the desired level based on first association information used to associate the displacement track with the level of fineness of the grid (subdivision iteration count), and the first association information is stored in the control file (MPD).

[0411] In step S463 , the file acquisition unit 411 acquires the sub-stream of the displacement vectors stored in the displacement track corresponding to the selected displacement adaptation set according to the control of the MPD analysis unit 412 .

[0412] In step S464, the MPD analysis unit 412 analyzes the control file (MPD) and selects a property adaptation set corresponding to the desired level based on second association information for associating the property track with the displacement track, the second association information being stored in the control file (MPD).

[0413] In step S465 , the file acquisition unit 411 acquires the substream of the attribute stored in the attribute track corresponding to the selected attribute adaptation set according to the control of the MPD analysis unit 412 .

[0414] In step S466 , the file acquisition unit 411 acquires other necessary substreams (eg, a substream of atlas information, a substream of a base grid, a substream of attributes) according to the control of the MPD analysis unit 412 .

[0415] In step S467, the decoding unit 414 decodes those sub-streams and generates V-DMC data. The reconstruction unit 415 converts the V-DMC data into mesh data.

[0416] In step S468 , the reconstruction unit 415 reconstructs the mesh (3D data) using the mesh data.

[0417] In step S469, the display processing unit 416 generates display information including a grid, and causes the display information to be displayed.

[0418] When the processing in step S469 is completed, the playback processing ends.

[0419] By performing the corresponding processing as described above, the playback device 400 can obtain the effects mentioned above in <4. Layer-by-layer playback control of attributes> For example, the playback device 400 can suppress an increase in the processing load associated with reconstruction of 3D data including displacement vectors.

[0420] <8. Appendix>

[0421] <Combination>

[0422] As long as no contradiction occurs, each example (each method) of the present technology can be appropriately applied in combination with other examples (other methods). In addition, each example of the present technology can be applied in combination with other technologies other than the above-mentioned technologies.

[0423] In addition, in the above, the encoding target is described by taking V-DMC data as an example. However, the encoding object is not limited to V-DMC data, as long as the encoding target is 3D data including a displacement vector. That is, the encoding method to which the present technology is applied may be an encoding method other than V-DMC. In addition, although the control file is described above by taking the MPD of MPEG-DASH as an example, the control method used for the content file is arbitrary and may be a method other than MPEG-DASH. That is, the control file for controlling distribution may be a file other than MPD.

[0424] <Computer>

[0425] The above series of processes can be performed by hardware or by software. In the case where the series of processes are performed by software, a program configuring the software is installed in a computer. Here, the computer includes a computer built in dedicated hardware, a general-purpose personal computer capable of performing various functions, for example, by installing various programs, etc.

[0426] Fig.38 : is a block diagram showing a configuration example of hardware of a computer that executes the above-mentioned series of processes according to a program.

[0427] exist Fig.38 In the computer 900 shown, a central processing unit (CPU) 901 , a read only memory (ROM) 902 , and a random access memory (RAM) 903 are interconnected via a bus 904 .

[0428] An input / output interface 910 is also connected to the bus 904. An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input / output interface 910.

[0429] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and an input terminal. The output unit 912 includes, for example, a display, a speaker, and an output terminal. The storage unit 913 includes, for example, a hard disk, a RAM disk, and a nonvolatile memory. The communication unit 914 includes, for example, a network interface. The drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

[0430] In the computer configured in this way, the CPU 901 loads a program stored in, for example, the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executes the program. In this way, the above-mentioned series of processing is performed. In the RAM 903, data and the like required for the CPU 901 to perform various types of processing are also appropriately stored.

[0431] The program executed by the computer can be provided by being recorded on the removable medium 921 as a package medium, for example. In this case, the program can be installed into the storage unit 913 via the input / output interface 910 by mounting the removable medium 921 on the drive 915 .

[0432] In addition, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, and digital satellite broadcasting. In this case, the program can be received through the communication unit 914 and can be installed in the storage unit 913.

[0433] Furthermore, the program may be installed in the ROM 902 or the storage unit 913 in advance.

[0434] <Target audience of this technology>

[0435] The present technique can be applied to any encoding and decoding method.

[0436] In addition, the present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0437] In addition, for example, the present technology can also be implemented as some configurations of the device, such as a processor used as a system large-scale integration (LSI) (e.g., a video processor), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a complete set of equipment obtained by further adding other additional functions to the unit (e.g., a video complete set of equipment).

[0438] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology can be implemented as cloud computing, and multiple devices share and jointly process the cloud computing via a network. For example, for any terminal such as a computer, an audio-visual (AV) device, a portable information processing terminal, or an Internet of Things (IoT) device, the present technology can be implemented in a cloud service that provides services related to images (moving images).

[0439] It should be noted that in this specification, a system means a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network and a single device with multiple modules housed in a housing are both systems.

[0440] <Fields and applications where this technology can be applied>

[0441] Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, for example, transportation, medical treatment, crime prevention, agriculture, animal husbandry, mining, beauty care, factories, household appliances, weather and nature monitoring. In addition, they are used for any application.

[0442] For example, the present technology can be applied to systems and devices provided for providing content for viewing, etc. In addition, for example, the present technology can also be applied to systems and devices provided for transportation, such as monitoring traffic conditions and automatic driving control. In addition, for example, the present technology can also be applied to systems and devices provided for safety. In addition, for example, the present technology can be applied to systems and devices provided for automatic control of machines, etc. In addition, for example, the present technology can also be applied to systems and devices provided for agriculture and animal husbandry. In addition, the present technology can also be applied to systems and devices for monitoring the conditions of nature, such as volcanoes, forests and oceans, wild animals, etc. In addition, for example, the present technology can also be applied to systems and devices provided for sports.

[0443] <Others>

[0444] It should be noted that in the present specification, "flag" refers to information used to identify multiple states, and includes not only information used when identifying two states of true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that the "flag" can take can be provided as a binary value of 1 / 0, for example, or can be three or more values. That is, the number of bits constituting the "flag" is arbitrary, and it can be 1 bit or can be multiple bits. In addition, with respect to identification information (also including flags), it is assumed that not only the identification information is included in the bit stream, but also the difference information of the identification information relative to some reference information is included in the bit stream. Therefore, in the present specification, with respect to "flag" and "identification information", not only the information but also the difference information relative to the reference information is covered.

[0445] In addition, various types of information (metadata, etc.) about the encoded data (bitstream) can be transmitted or recorded in any form, as long as the information is associated with the encoded data. Here, the term "association" means that, for example, another data can be used (associated) when processing one piece of data. That is, data associated with each other can be combined into a single data, or can be discrete multiple pieces of data. For example, information associated with the encoded data (image) can be transmitted on a communication channel different from the encoded data (image). In addition, for example, information associated with the encoded data (image) can be recorded on a recording medium different from the recording medium of the encoded data (image) (or another recording area of ​​the same recording medium). It should be noted that the "association" can be not only the entire data, but also a part of the data. For example, an image and information corresponding to the image can be associated with each other in any unit such as multiple frames, a frame, or a part of a frame.

[0446] It should be noted that in this specification, the terms "synthesis", "multiplexing", "addition", "integration", "inclusion", "storage", "input", "put into", "insertion" and the like mean combining multiple objects into one, such as combining encoded data and metadata into one piece of data, and mean a method of the above-mentioned "association".

[0447] The embodiments of the present technology are not limited to the above-mentioned embodiments and various modifications may be made without departing from the gist of the present technology.

[0448] For example, a configuration described as a single device (or processing unit) may be divided to make it a plurality of devices (or processing units). Conversely, the configurations described above as a plurality of devices (or processing units) may be combined and configured as a single device (or processing unit). Furthermore, configurations other than those described above may of course be added to the configuration of each device (or each processing unit). In addition, some configurations of a certain device (or processing unit) may be included in the configuration of another device (or another processing unit) as long as the configuration and operation are substantially the same as the entire system.

[0449] In addition, for example, the above-mentioned program can be executed in any device. In this case, it is sufficient to enable the device to have necessary functions (functional blocks, etc.) and to be able to obtain necessary information.

[0450] In addition, for example, a single device may be enabled to perform the various steps of a single flowchart so that multiple devices share and execute them. In addition, in the case where a single step includes multiple processes, the multiple processes may be performed by a single device and may be shared and executed by multiple devices. In other words, the multiple processes included in a single step may be executed as processes of multiple steps. Conversely, processes described as multiple steps may be combined and executed as a single step.

[0451] In addition, for example, with respect to a program executed by a computer, the processing of the steps describing the program may be performed in chronological order according to the order described herein, or in parallel, or individually at necessary timings such as when a call is made. That is, the processing in each step may be performed in an order different from the order mentioned above as long as no contradiction occurs. In addition, the processing of the steps describing the program may be performed in parallel with the processing of other programs or may be performed in combination with the processing of other programs.

[0452] In addition, for example, as long as no contradiction occurs, multiple technologies related to the present technology can be implemented separately and independently. Of course, any multiple technologies can also be implemented in combination. For example, some or all of the technologies described in any embodiment can also be implemented in combination with some or all of the technologies described in other embodiments. In addition, part or all of any of the above-mentioned technologies can also be implemented together with other technologies not described above.

[0453] It should be noted that the present technology can also take the following configurations.

[0454] (1) An information processing device comprising:

[0455] an encoding unit that encodes the base grid and the attributes and encodes the displacement vectors at each level of the levels of refinement; and

[0456] a content file generating unit that generates a content file, stores the encoded data of the base grid in a base grid track of the content file, stores the encoded data of the attribute in an attribute track of the content file, stores the encoded data of the displacement vector of each of the levels in mutually different displacement tracks of the content file, and stores first association information for associating the displacement track with the level in the content file, wherein

[0457] The base mesh is a mesh with a lower fineness than the original mesh generated by thinning out the vertices from the original mesh of the encoded object. The original mesh consists of vertices and connections representing the three-dimensional structure of the object.

[0458] Attributes include the texture applied to the surface of the original mesh, and

[0459] The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

[0460] (2) The information processing device according to (1), wherein:

[0461] The content file generation unit stores information indicating the hierarchy of the displacement vectors stored in the displacement track as first association information in the displacement track.

[0462] (3) The information processing device according to (2), wherein:

[0463] The content file generation unit stores information indicating the hierarchy of the displacement vector in the SubdivIterationInfoBox provided in the sample entry of the displacement track.

[0464] (4) The information processing device according to (2) or (3), wherein:

[0465] The content file generation unit stores the encoded data of the absolute displacement vectors of the vertices obtained in the corresponding level and the relative displacement vectors of the vertices obtained in the level lower than the corresponding level in the displacement track,

[0466] The absolute displacement vector is a displacement vector indicating the displacement of the vertex relative to the base mesh, and

[0467] The relative displacement vector is a displacement vector indicating the displacement of a vertex relative to a position in a hierarchy one level lower.

[0468] (5) The information processing device according to (2) or (3), wherein:

[0469] The content file generation unit stores the encoded data of the absolute displacement vectors of the vertices obtained in the corresponding hierarchy and the absolute displacement vectors of the vertices obtained in the hierarchy lower than the corresponding hierarchy in the displacement track, and

[0470] An absolute displacement vector is a displacement vector that indicates the displacement of a vertex relative to the base mesh.

[0471] (6) The information processing device according to any one of (2) to (5), wherein:

[0472] The content file generation unit further stores type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the displacement track,

[0473] The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the base mesh, and

[0474] The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the position in a hierarchy one level lower.

[0475] (7) The information processing device according to (6), wherein:

[0476] The content file generation unit stores the type information in the SubdivIterationInfoBox provided in the sample entry of the displacement track.

[0477] (8) The information processing device according to any one of (2) to (7), wherein:

[0478] The content file generation unit also stores quality information indicating the quality of the base grid or the displacement vector in the base grid track or the displacement track.

[0479] (9) The information processing device according to (8), wherein

[0480] The content file generation unit stores the quality information in the MeshInfoBox provided in the sample entry of the base mesh track or the displacement track.

[0481] (10) The information processing device according to (8) or (9), wherein:

[0482] The content file generation unit stores the quality information in the SubdivIterationInfoBox provided in the sample entry of the base grid track or the displacement track.

[0483] (11) The information processing device according to any one of (8) to (10), wherein:

[0484] The content file generation unit stores an average value of the number of vertices of the mesh of the level corresponding to the base grid track or the displacement track in the time axis direction in the base grid track or the displacement track as quality information.

[0485] (12) The information processing device according to any one of (8) to (11), wherein:

[0486] The content file generation unit stores an average value of the number of triangles of the mesh of the level corresponding to the base grid track or the displacement track in the time axis direction in the base grid track or the displacement track as quality information.

[0487] (13) The information processing device according to any one of (1) to (12), wherein:

[0488] The coding unit encodes the attributes at each level in the hierarchy of fineness, and

[0489] The content file generation unit stores the encoded data of the attribute of each of the hierarchies in mutually different attribute tracks of the content file, and stores second association information for associating the attribute track with the displacement track in the content file.

[0490] (14) The information processing device according to (13), wherein:

[0491] The content file generation unit stores information indicating the hierarchy of the attributes stored in the attribute track as second association information in the attribute track.

[0492] (15) The information processing device according to (14), wherein:

[0493] The content file generation unit stores information indicating the hierarchy of the attribute in the SubdivIterationInfoBox provided in the sample entry of the attribute track.

[0494] (16) The information processing device according to any one of (13) to (15), wherein:

[0495] The content file generation unit stores grouping information for grouping the attribute track and the displacement track required to reconstruct the mesh in each of the hierarchies in the content file as second association information.

[0496] (17) The information processing device according to (16), wherein:

[0497] The content file generation unit stores the grouping information in SubdivIterationTrackGroupBox.

[0498] (18) The information processing device according to (17), wherein:

[0499] The content file generation unit also stores information indicating the hierarchy in SubdivIterationTrackGroupBox.

[0500] (19) The information processing device according to any one of (16) to (18), wherein

[0501] The content file generation unit stores, as grouping information, in the content file a track reference that refers to other tracks to group the attribute track and the displacement track required to reconstruct the mesh in each of the hierarchies.

[0502] (20) An information processing method comprising:

[0503] encoding a base grid and attributes, and encoding a displacement vector at each level of levels of refinement; and

[0504] generating a content file, storing the encoded data of the base grid in a base grid track of the content file, storing the encoded data of the attributes in an attribute track of the content file, storing the encoded data of the displacement vectors of each level in the hierarchy in mutually different displacement tracks of the content file, and storing association information for associating the displacement tracks with the levels in the content file, wherein

[0505] The base mesh is a mesh with a lower fineness than the original mesh generated by thinning out the vertices from the original mesh of the encoded object. The original mesh consists of vertices and connections representing the three-dimensional structure of the object.

[0506] Attributes include the texture applied to the surface of the original mesh, and

[0507] The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

[0508] (21) An information processing device comprising:

[0509] an acquisition unit that acquires the encoded data of the base grid and the encoded data of the attribute from the content file, and acquires the encoded data of the displacement vector from the displacement track of the content file corresponding to a desired level in the levels based on first association information for associating the displacement track with the levels of refinement of the grid;

[0510] a decoding unit that decodes each of the acquired encoded data of the base grid, the acquired encoded data of the attribute, and the acquired encoded data of the displacement vector; and

[0511] A reconstruction unit, which reconstructs a mesh of a desired level using the base mesh, attributes, and displacement vectors obtained by decoding, wherein:

[0512] The base mesh is a mesh with a lower fineness than the original mesh generated by thinning out the vertices from the original mesh of the encoded object. The original mesh consists of vertices and connections representing the three-dimensional structure of the object.

[0513] Properties include the texture applied to the surface of the original mesh,

[0514] The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh, and

[0515] The content file includes a base grid track storing encoded data of a base grid, an attribute track storing encoded data of attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies.

[0516] (22) The information processing device according to (21), wherein:

[0517] The acquisition unit acquires the encoded data of the displacement vector based on information indicating the hierarchy of the displacement vector, the information being stored in the displacement track as first associated information.

[0518] (23) The information processing device according to (22), wherein:

[0519] Information indicating the level of the displacement vector is stored in the SubdivIterationInfoBox provided in the sample entry of the displacement track.

[0520] (24) The information processing device according to (22) or (23), wherein:

[0521] In the displacement track, the encoded data of the absolute displacement vectors of the vertices obtained in the corresponding level and the relative displacement vectors of the vertices obtained in the level lower than the corresponding level are stored, and

[0522] An absolute displacement vector is a displacement vector that indicates the displacement of a vertex relative to the base mesh.

[0523] The relative displacement vector is a displacement vector indicating the displacement of a vertex relative to a position in a hierarchy one level lower.

[0524] (25) The information processing device according to (22) or (23), wherein:

[0525] In the displacement track, the encoded data of the absolute displacement vectors of the vertices obtained in the corresponding level and the absolute displacement vectors of the vertices obtained in the level lower than the corresponding level are stored, and

[0526] An absolute displacement vector is a displacement vector that indicates the displacement of a vertex relative to the base mesh.

[0527] (26) The information processing device according to any one of (22) to (25), wherein:

[0528] The acquisition unit also acquires the encoded data of the displacement vector based on type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, the type information being stored in the displacement track,

[0529] The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the base mesh, and

[0530] The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the position in a hierarchy one level lower.

[0531] (27) The information processing device according to (26), wherein:

[0532] The type information is stored in the SubdivIterationInfoBox provided in the sample entry of the displacement track.

[0533] (28) The information processing device according to any one of (22) to (27), wherein:

[0534] The acquisition unit also acquires the encoded data of the displacement vector based on quality information indicating the quality of the base grid or the displacement vector, the quality information being stored in the base grid track or the displacement track.

[0535] (29) The information processing device according to (28), wherein:

[0536] The quality information is stored in the MeshInfoBox provided in the sample entry of the base mesh track or displacement track.

[0537] (30) The information processing device according to (28), wherein:

[0538] The quality information is stored in the SubdivIterationInfoBox provided in the sample entry of the base grid track or displacement track.

[0539] (31) The information processing device according to any one of (28) to (30), wherein:

[0540] The quality information includes an average value of the number of vertices of the mesh of the level corresponding to the base mesh track or the displacement track in the time axis direction.

[0541] (32) The information processing device according to any one of (28) to (31), wherein:

[0542] The quality information includes an average value of the number of triangles of a mesh of a level corresponding to a base mesh track or a displacement track in the time axis direction.

[0543] (33) The information processing device according to any one of (21) to (32), wherein:

[0544] The content file includes a base grid track, a plurality of attribute tracks storing encoded data of attributes of different hierarchies, and a plurality of displacement tracks storing encoded data of displacement vectors of different hierarchies, and

[0545] The acquisition unit acquires the encoded data of the attribute from the attribute track corresponding to the desired hierarchy among the hierarchies of the content file based on the second association information for associating the attribute track with the displacement track.

[0546] (34) The information processing device according to (33), wherein:

[0547] The acquisition unit acquires the encoded data of the attribute based on information indicating the hierarchy of the attribute, the information being stored in the attribute track as second association information.

[0548] (35) The information processing device according to (34), wherein:

[0549] Information indicating the level of the attribute is stored in the SubdivIterationInfoBox provided in the sample entry of the attribute track.

[0550] (36) The information processing device according to any one of (33) to (35), wherein:

[0551] The acquisition unit acquires the encoded data of the attribute based on grouping information for grouping the attribute track and the displacement track required to reconstruct the mesh in each of the hierarchies, the grouping information being stored in the content file as the second association information.

[0552] (37) The information processing device according to (36), wherein:

[0553] The grouping information is stored in SubdivIterationTrackGroupBox.

[0554] (38) The information processing device according to (37), wherein:

[0555] The acquisition unit also acquires the encoded data of the attribute based on the information indicating the hierarchy, the information being stored in the SubdivIterationTrackGroupBox.

[0556] (39) The information processing device according to any one of (36) to (38), wherein:

[0557] The acquisition unit acquires the encoded data of the attribute based on a track reference that refers to other tracks to group the attribute track and the displacement track required to reconstruct the grid in each of the levels, the track reference being stored as grouping information in the content file.

[0558] (40) An information processing method comprising:

[0559] Retrieving encoded data of a base grid and encoded data of attributes from a content file, and retrieving encoded data of displacement vectors from a displacement track of the content file corresponding to a desired level in the hierarchy based on association information for associating the displacement track with the hierarchy of granularity of the grid;

[0560] decoding each of the acquired encoded data of the base grid, the acquired encoded data of the attribute, and the acquired encoded data of the displacement vector; and

[0561] The mesh of the desired level is reconstructed using the base mesh, attributes and displacement vectors obtained by decoding, wherein:

[0562] The base mesh is a mesh with a lower fineness than the original mesh generated by thinning out the vertices from the original mesh of the encoded object. The original mesh consists of vertices and connections representing the three-dimensional structure of the object.

[0563] Properties include the texture applied to the surface of the original mesh,

[0564] The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh, and

[0565] The content file includes a base grid track storing encoded data of a base grid, an attribute track storing encoded data of attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies.

[0566] (41) An information processing device comprising:

[0567] an encoding unit that encodes the base grid and the attributes and encodes the displacement vectors at each level of the levels of refinement;

[0568] a content file generating unit that generates a content file, stores the encoded data of the base grid in a base grid track of the content file, stores the encoded data of the attribute in an attribute track of the content file, and stores the encoded data of the displacement vector of each of the hierarchies in mutually different displacement tracks of the content file; and

[0569] A control file generating unit generates a control file including control information for controlling distribution of content files, and stores first association information for associating a displacement adaptation set managing a displacement track with a hierarchy in the control file, wherein

[0570] The base mesh is a mesh of lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoded object, the original mesh consisting of vertices and connections representing the three-dimensional structure of the object, and

[0571] Properties include the texture applied to the surface of the original mesh,

[0572] The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

[0573] (42) The information processing device according to (41), wherein:

[0574] The control file generation unit stores information indicating the hierarchy of the displacement vectors stored in the displacement track managed by the displacement adaptation set in the control file as first association information.

[0575] (43) The information processing device according to (42), wherein:

[0576] The control file generation unit stores the V3C video component descriptor indicating the hierarchy as information indicating the hierarchy in the basic property or the supplementary property of the displacement adaptation set.

[0577] (44) The information processing device according to (42) or (43), wherein:

[0578] The control file generation unit also stores type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the displacement adaptation set of the control file.

[0579] The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the base mesh, and

[0580] The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the position in a hierarchy one level lower.

[0581] (45) The information processing device according to (44), wherein:

[0582] The control file generation unit stores a V3C video component descriptor indicating whether a displacement vector is an absolute displacement vector or a relative displacement vector as type information in a basic property or a supplementary property of a displacement adaptation set.

[0583] (46) The information processing device according to any one of (42) to (45), wherein:

[0584] The control file generation unit further stores quality information indicating the quality of the base grid or the displacement vector in a base grid adaptation set managing the base grid track of the control file or in a displacement adaptation set of the control file.

[0585] (47) The information processing device according to (46), wherein:

[0586] The control file generation unit stores a mesh information descriptor indicating the quality of the base mesh or the displacement vector as quality information in the basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set.

[0587] (48) The information processing device according to (47), wherein:

[0588] The control file generation unit stores a mesh information descriptor indicating an average value of the number of vertices of the mesh of the level corresponding to the base mesh adaptation set or the displacement adaptation set in the time axis direction in the basic property or the supplementary property of the base mesh adaptation set or the displacement adaptation set.

[0589] (49) The information processing device according to (47) or (48), wherein:

[0590] The control file generation unit stores a mesh information descriptor indicating an average value of the number of triangles of a mesh of a level corresponding to a base mesh adaptation set or a displacement adaptation set in a time axis direction in a basic property or a supplementary property of the base mesh adaptation set or the displacement adaptation set.

[0591] (50) The information processing device according to any one of (46) to (49), wherein:

[0592] The control file generation unit stores the V3C video component descriptor indicating the quality of the base grid or the displacement vector as quality information in the basic properties or supplementary properties of the base grid adaptation set or the displacement adaptation set.

[0593] (51) The information processing device according to any one of (41) to (50), wherein:

[0594] The encoding unit encodes the attributes at each level in the hierarchy of granularity.

[0595] The content file generation unit stores the encoded data of the attributes of each of the hierarchies in mutually different attribute tracks of the content file, and

[0596] The control file generation unit stores second association information for associating the attribute adaptation set managing the attribute track with the displacement adaptation set in the control file.

[0597] (52) The information processing device according to (51), wherein:

[0598] The control file generation unit stores information indicating the hierarchy of the attribute stored in the attribute track managed by the attribute adaptation set in the control file as second association information.

[0599] (53) The information processing device according to (52), wherein:

[0600] The control file generation unit stores the V3C video component descriptor indicating the hierarchy as information indicating the hierarchy in the basic property or the supplementary property of the attribute adaptation set.

[0601] (54) The information processing device according to (52) or (53), wherein:

[0602] The control file generation unit further stores type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the attribute adaptation set of the control file.

[0603] The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the base mesh, and

[0604] The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the position in a hierarchy one level lower.

[0605] (55) The information processing device according to (54), wherein:

[0606] The control file generation unit stores a V3C video component descriptor indicating whether a displacement vector is an absolute displacement vector or a relative displacement vector as type information in a basic property or a supplementary property of a displacement adaptation set.

[0607] (56) The information processing device according to any one of (51) to (55), wherein:

[0608] The control file generation unit stores, as second association information, in the control file a preselection for grouping attribute adaptation sets and displacement adaptation sets for managing tracks required for reconstructing the mesh at each of the levels.

[0609] (57) The information processing device according to (56), wherein:

[0610] The control file generation unit stores information indicating the hierarchy of the attributes stored in the attribute track managed by the attribute adaptation set in the preselection.

[0611] (58) The information processing device according to (57), wherein:

[0612] The control file generation unit stores a subdivision iteration information descriptor indicating a hierarchy as information indicating a hierarchy in a preselected basic property or a supplementary property.

[0613] (59) The information processing device according to (57) or (58), wherein:

[0614] The control file generation unit further stores type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the preselection,

[0615] The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the base mesh, and

[0616] The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the position in a hierarchy one level lower.

[0617] (60) The information processing device according to (59), wherein:

[0618] The control file generation unit stores a descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector as type information in a pre-selected basic property or a supplementary property.

[0619] (61) The information processing device according to any one of (51) to (60), wherein:

[0620] The control file generation unit stores, as second association information, in the control file, association identification information for associating the attribute adaptation set and the displacement adaptation set for managing tracks required for reconstructing the mesh in each of the hierarchies.

[0621] (62) An information processing method comprising:

[0622] encoding a base mesh and attributes, and encoding a displacement vector at each level in the hierarchy of fineness;

[0623] generating a content file, storing the encoded data of the base grid in a base grid track of the content file, storing the encoded data of the attributes in an attribute track of the content file, and storing the encoded data of the displacement vectors of each of the levels in mutually different displacement tracks of the content file; and

[0624] A control file including control information for controlling distribution of the content file is generated, and association information for associating a displacement adaptation set managing the displacement track with a hierarchy is stored in the control file, wherein

[0625] The base mesh is a mesh with a lower fineness than the original mesh generated by thinning out the vertices from the original mesh of the encoded object. The original mesh consists of vertices and connections representing the three-dimensional structure of the object.

[0626] Attributes include the texture applied to the surface of the original mesh, and

[0627] The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

[0628] (71) An information processing device comprising:

[0629] a selection unit for selecting a displacement adaptation set corresponding to a desired level among the displacement adaptation sets based on first association information for associating the displacement adaptation sets with levels of fineness of the mesh, the first association information being stored in a control file including control information for controlling distribution of the content file;

[0630] an acquisition unit that acquires the encoded data of the base grid and the encoded data of the attributes stored in the content file, and acquires the encoded data of the displacement vectors stored in the displacement track of the content file managed by the selected displacement adaptation set;

[0631] a decoding unit that decodes each of the acquired encoded data of the base grid, the acquired encoded data of the attribute, and the acquired encoded data of the displacement vector; and

[0632] A reconstruction unit, which reconstructs a mesh of a desired level using the base mesh, attributes, and displacement vectors obtained by decoding, wherein:

[0633] The base mesh is a mesh with a lower fineness than the original mesh generated by thinning out the vertices from the original mesh of the encoded object. The original mesh consists of vertices and connections representing the three-dimensional structure of the object.

[0634] Properties include the texture applied to the surface of the original mesh,

[0635] The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

[0636] The content file includes a base grid track storing encoded data of a base grid, an attribute track storing encoded data of attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies, and

[0637] The control file includes a base grid adaptation set that manages the base grid track, an attribute adaptation set that manages the attribute track, and a displacement adaptation set that manages the displacement track.

[0638] (72) The information processing device according to (71), wherein:

[0639] The selection unit selects one of the displacement adaptation sets based on information indicating the hierarchy of the displacement vectors stored in the displacement track managed by the displacement adaptation set, the information being stored as first association information in the displacement adaptation set of the control file.

[0640] (73) The information processing device according to (72), wherein:

[0641] The selection unit selects one of the displacement adaptation sets based on a V3C video component descriptor indicating a level, the V3C video component descriptor being stored in a basic property or a supplementary property of the displacement adaptation set as information indicating a level.

[0642] (74) The information processing device according to (72) or (73), wherein:

[0643] The selection unit further selects one of the displacement adaptation sets based on type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, the type information being stored in the displacement adaptation set of the control file,

[0644] The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the base mesh, and

[0645] The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the position in a hierarchy one level lower.

[0646] (75) The information processing device according to (74), wherein:

[0647] The selection unit selects one of the displacement adaptation sets based on a V3C video component descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, wherein the V3C video component descriptor is stored as type information in a basic property or a supplementary property of the displacement adaptation set.

[0648] (76) The information processing device according to any one of (72) to (75), wherein:

[0649] The selection unit further selects one of the displacement adaptation sets based on quality information indicating a quality of the base grid or the displacement vector, the quality information being stored in the base grid adaptation set or the displacement adaptation set.

[0650] (77) The information processing device according to (76), wherein:

[0651] The selection unit selects one of the displacement adaptation sets based on a mesh information descriptor indicating the quality of the base mesh or the displacement vector, the mesh information descriptor being stored as quality information in basic properties or supplementary properties of the base mesh adaptation set or the displacement adaptation set.

[0652] (78) The information processing device according to (77), wherein:

[0653] The selection unit selects one of the displacement adaptation sets based on a mesh information descriptor indicating an average value of the number of vertices of the mesh of the level corresponding to the base mesh adaptation set or the displacement adaptation set in the time axis direction, the mesh information descriptor being stored in the basic property or the supplementary property of the base mesh adaptation set or the displacement adaptation set.

[0654] (79) The information processing device according to (77) or (78), wherein:

[0655] The selection unit selects one of the displacement adaptation sets based on a mesh information descriptor indicating an average value of the number of triangles of a mesh of a level corresponding to the base mesh adaptation set or the displacement adaptation set in a time axis direction, the mesh information descriptor being stored in a basic property or a supplementary property of the base mesh adaptation set or the displacement adaptation set.

[0656] (80) The information processing device according to any one of (76) to (79), wherein:

[0657] The selection unit selects one of the displacement adaptation sets based on a V3C video component descriptor indicating the quality of the base grid or the displacement vector, the V3C video component descriptor being stored as quality information in basic properties or supplementary properties of the base grid adaptation set or the displacement adaptation set.

[0658] (81) The information processing device according to any one of (71) to (80), wherein:

[0659] The content file includes a base grid track, a plurality of attribute tracks storing encoded data of attributes of different hierarchies, and a plurality of displacement tracks storing encoded data of displacement vectors of different hierarchies.

[0660] The control file includes second association information for associating the attribute adaptation set with the displacement adaptation set,

[0661] The selection unit selects one of the attribute adaptation sets corresponding to the desired level based on the second association information, and

[0662] The acquisition unit acquires the encoded data of the attribute stored in the attribute track of the content file managed by the selected attribute adaptation set.

[0663] (82) The information processing device according to (81), wherein:

[0664] The selection unit selects one of the attribute adaptation sets based on information indicating the hierarchy of the attributes stored in the attribute track managed by the attribute adaptation set, the information being stored as second association information in the attribute adaptation set of the control file.

[0665] (83) The information processing device according to (82), wherein:

[0666] The selection unit selects one of the attribute adaptation sets based on a V3C video component descriptor indicating a level, the V3C video component descriptor being stored in a basic property or a supplementary property of the attribute adaptation set as information indicating a level.

[0667] (84) The information processing device according to (82) or (83), wherein:

[0668] The selection unit further selects one of the attribute adaptation sets based on type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, the type information being stored in the attribute adaptation set of the control file,

[0669] The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the base mesh, and

[0670] The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the position in a hierarchy one level lower.

[0671] (85) The information processing device according to (84), wherein:

[0672] The selection unit selects one of the attribute adaptation sets based on a V3C video component descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, wherein the V3C video component descriptor is stored as type information in a basic property or a supplementary property of the attribute adaptation set.

[0673] (86) The information processing device according to any one of (81) to (85), wherein:

[0674] The selection unit selects one of the attribute adaptation sets based on a preselection for grouping attribute adaptation sets and displacement adaptation sets managing tracks required to reconstruct a mesh at each of the levels, the preselection being stored as second association information in the control file.

[0675] (87) The information processing device according to (86), wherein:

[0676] The selection unit selects one of the attribute adaptation sets based on information indicating the hierarchy of the attributes stored in the attribute track managed by the attribute adaptation set, the information being stored in the preselection.

[0677] (88) The information processing device according to (87), wherein:

[0678] The selection unit selects one of the attribute adaptation sets based on a subdivision iteration information descriptor indicating a level, the subdivision iteration information descriptor being stored in a preselected basic property or a supplementary property as information indicating a level.

[0679] (89) The information processing device according to (87) or (88), wherein:

[0680] The selection unit further selects one of the attribute adaptation sets based on type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, the type information being stored in a preselected

[0681] The absolute displacement vector is a displacement vector indicating the displacement of the vertex obtained by subdividing the mesh relative to the base mesh, and

[0682] The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to the position in a hierarchy one level lower.

[0683] (90) The information processing device according to (89), wherein:

[0684] The selection unit selects one of the attribute adaptation sets based on a descriptor indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector, the descriptor being stored as type information in a preselected basic property or a supplementary property.

[0685] (91) The information processing device according to any one of (81) to (90), wherein:

[0686] The selection unit selects one of the attribute adaptation sets based on association identification information for associating the attribute adaptation set and the displacement adaptation set managing tracks required to reconstruct the mesh at each of the levels with each other, the association identification information being stored as second association information in the control file.

[0687] (92) An information processing method comprising:

[0688] selecting one of the displacement adaptation sets corresponding to a desired level in the levels based on first association information for associating the displacement adaptation sets with levels of fineness of the mesh, the first association information being stored in a control file including control information for controlling distribution of the content file;

[0689] Retrieve encoded data of a base grid and encoded data of attributes stored in a content file, and retrieve encoded data of displacement vectors stored in a displacement track of the content file managed by the selected displacement adaptation set;

[0690] decoding each of the acquired encoded data of the base grid, the acquired encoded data of the attribute, and the acquired encoded data of the displacement vector; and

[0691] The mesh of the desired level is reconstructed using the base mesh, attributes and displacement vectors obtained by decoding, wherein:

[0692] The base mesh is a mesh with a lower fineness than the original mesh generated by thinning out the vertices from the original mesh of the encoded object. The original mesh consists of vertices and connections representing the three-dimensional structure of the object.

[0693] Properties include the texture applied to the surface of the original mesh,

[0694] The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

[0695] The content file includes a base grid track storing encoded data of a base grid, an attribute track storing encoded data of attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies, and

[0696] The control file includes a base grid adaptation set that manages the base grid track, an attribute adaptation set that manages the attribute track, and a displacement adaptation set that manages the displacement track.

[0697] Reference numerals list

[0698] 300 file generating device, 311 grid sparse unit, 312 V-DMC encoding unit, 313 bit stream generating unit, 314 content file generating unit, 315 MPD generating unit, 400 playback device, 411 file acquiring unit, 412 MPD analyzing unit, 413 file processing unit, 414 decoding unit, 415 reconstruction unit, 416 display processing unit, 900 computer

Claims

1. An information processing device, comprising: an encoding unit that encodes the base grid and the attributes and encodes the displacement vectors at each level of the levels of refinement; as well as a content file generating unit, which generates a content file, stores the encoded data of the base grid in a base grid track of the content file, stores the encoded data of the attribute in an attribute track of the content file, stores the encoded data of the displacement vector of each level in the hierarchy in mutually different displacement tracks of the content file, and stores first association information for associating the displacement track with the hierarchy in the content file, wherein The base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of the object, The attributes include a texture applied to a surface of the original mesh, and The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

2. The information processing device according to claim 1, wherein: The content file generation unit stores information indicating a hierarchy of the displacement vectors stored in the displacement track as the first association information in the displacement track.

3. The information processing device according to claim 2, wherein: The content file generation unit further stores type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the displacement track, The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to the base mesh, and The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to a position in a hierarchy one level lower.

4. The information processing device according to claim 2, wherein: The content file generation unit further stores quality information indicating the quality of the base grid or the displacement vector in the base grid track or the displacement track.

5. The information processing device according to claim 1, wherein: The encoding unit encodes the attribute at each level of the levels of detail, and The content file generation unit stores the encoded data of the attribute of each of the hierarchies in mutually different attribute tracks of the content file, and stores second association information for associating the attribute track with the displacement track in the content file.

6. The information processing device according to claim 5, wherein: The content file generation unit stores, as the second association information, in the content file, grouping information for grouping the attribute track and the displacement track required to reconstruct the mesh in each of the hierarchies.

7. The information processing device according to claim 6, wherein: The content file generation unit further stores information indicating the hierarchy in the content file.

8. An information processing method, comprising: encoding a base mesh and attributes, and encoding a displacement vector at each level in the hierarchy of fineness; as well as generating a content file, storing the encoded data of the base grid in a base grid track of the content file, storing the encoded data of the attribute in an attribute track of the content file, storing the encoded data of the displacement vector of each level in the hierarchy in mutually different displacement tracks of the content file, and storing association information for associating the displacement tracks with the hierarchy in the content file, wherein The base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of the object, The attributes include a texture applied to a surface of the original mesh, and The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

9. An information processing device, comprising: an acquisition unit that acquires the encoded data of the base grid and the encoded data of the attribute from the content file, and acquires the encoded data of the displacement vector from the displacement track of the content file corresponding to the desired level in the levels based on first association information for associating the displacement track with the levels of refinement of the grid; a decoding unit that decodes each of the acquired encoded data of the base grid, the acquired encoded data of the attribute, and the acquired encoded data of the displacement vector; and A reconstruction unit, which reconstructs the grid of the desired level using the base grid, attributes and displacement vectors obtained by the decoding, wherein: The base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of the object, The attributes include a texture applied to the surface of the original mesh, The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh, and The content file includes a base grid track storing encoded data of the base grid, an attribute track storing encoded data of the attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies.

10. An information processing method, comprising: Retrieving encoded data of a base grid and encoded data of attributes from a content file, and retrieving encoded data of displacement vectors from a displacement track of the content file corresponding to a desired level in the hierarchy based on association information for associating the displacement track with the hierarchy of fineness of the grid; decoding each of the obtained encoded data of the base grid, the obtained encoded data of the attribute, and the obtained encoded data of the displacement vector; as well as The grid of the desired level is reconstructed using the base grid, attributes and displacement vectors obtained by the decoding, wherein: The base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of the object, The attributes include a texture applied to the surface of the original mesh, The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh, and The content file includes a base grid track storing encoded data of the base grid, an attribute track storing encoded data of the attributes, and a plurality of displacement tracks storing encoded data of displacement vectors of mutually different hierarchies.

11. An information processing device, comprising: an encoding unit that encodes the base grid and the attributes and encodes the displacement vectors at each level of the levels of refinement; a content file generating unit that generates a content file, stores the encoded data of the base grid in a base grid track of the content file, stores the encoded data of the attribute in an attribute track of the content file, and stores the encoded data of the displacement vector of each of the hierarchies in mutually different displacement tracks of the content file; as well as a control file generating unit, which generates a control file including control information for controlling distribution of the content file, and stores first association information for associating a displacement adaptation set managing the displacement track with the hierarchy in the control file, wherein: The base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of the object, The attributes include a texture applied to a surface of the original mesh, and The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

12. The information processing device according to claim 11, wherein: The control file generation unit further stores type information indicating whether the displacement vector is an absolute displacement vector or a relative displacement vector in the displacement adaptation set of the control file, The absolute displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing a mesh relative to the base mesh, and The relative displacement vector is a displacement vector indicating the displacement of a vertex obtained by subdividing the mesh relative to a position in a hierarchy one level lower.

13. The information processing device according to claim 11, wherein: The control file generation unit further stores quality information indicating the quality of the base grid or the displacement vector in a base grid adaptation set managing the base grid track of the control file or in the displacement adaptation set of the control file.

14. The information processing device according to claim 11, wherein: The control file generation unit stores information indicating the hierarchy of attributes stored in the attribute track managed by the attribute adaptation set managing the attribute track in the attribute adaptation set of the control file as second association information for associating the attribute adaptation set with the displacement adaptation set.

15. The information processing device according to claim 11, wherein: The control file generation unit stores in the control file a pre-selection for grouping an attribute adaptation set and a displacement adaptation set for managing tracks required to reconstruct the mesh according to each of the levels as second association information for associating the attribute adaptation set with the displacement adaptation set, wherein the attribute adaptation set manages the attribute tracks.

16. The information processing device according to claim 15, wherein: The control file generation unit stores information indicating a hierarchy of attributes stored in an attribute track managed by the attribute adaptation set in the preselection.

17. The information processing device according to claim 11, wherein: The control file generation unit stores in the control file association identification information for associating an attribute adaptation set for managing tracks required to reconstruct the grid according to each level in the levels with the displacement adaptation set as second association information for associating the attribute adaptation set with the displacement adaptation set, wherein the attribute adaptation set manages the attribute track.

18. An information processing method, comprising: encoding a base mesh and attributes, and encoding a displacement vector at each level in the hierarchy of fineness; generating a content file, storing the encoded data of the base grid in a base grid track of the content file, storing the encoded data of the attributes in an attribute track of the content file, and storing the encoded data of the displacement vectors of each of the levels in mutually different displacement tracks of the content file; as well as generating a control file including control information for controlling distribution of the content file, and storing association information for associating a displacement adaptation set managing the displacement track with the hierarchy in the control file, wherein The base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of the object, The attributes include a texture applied to a surface of the original mesh, and The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh.

19. An information processing device, comprising: a selection unit for selecting a displacement adaptation set corresponding to a desired level in the levels among the displacement adaptation sets based on first association information for associating the displacement adaptation sets with the levels of fineness of the mesh, the first association information being stored in a control file including control information for controlling distribution of the content file; an acquisition unit that acquires the encoded data of the base grid and the encoded data of the attributes stored in the content file, and acquires the encoded data of the displacement vectors stored in the displacement track managed by the selected displacement adaptation set in the content file; a decoding unit that decodes each of the acquired encoded data of the base grid, the acquired encoded data of the attribute, and the acquired encoded data of the displacement vector; and A reconstruction unit, which reconstructs the grid of the desired level using the base grid, attributes and displacement vectors obtained by the decoding, wherein: The base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of the object, The attributes include a texture applied to the surface of the original mesh, The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh, The content file includes a base grid track storing the encoded data of the base grid, an attribute track storing the encoded data of the attribute, and a plurality of displacement tracks storing the encoded data of displacement vectors of mutually different hierarchies, and The control file includes a base grid adaptation set managing the base grid track, an attribute adaptation set managing the attribute track, and the displacement adaptation set managing the displacement track.

20. An information processing method, comprising: selecting a displacement adaptation set corresponding to a desired level in the levels among the displacement adaptation sets based on first association information for associating the displacement adaptation sets with levels of fineness of the grid, the first association information being stored in a control file including control information for controlling distribution of the content file; Retrieve the encoded data of the base grid and the encoded data of the attributes stored in the content file, and retrieve the encoded data of the displacement vectors stored in the displacement track of the content file managed by the selected displacement adaptation set; decoding each of the obtained encoded data of the base grid, the obtained encoded data of the attribute, and the obtained encoded data of the displacement vector; as well as The grid of the desired level is reconstructed using the base grid, attributes and displacement vectors obtained by the decoding, wherein: The base mesh is a mesh having a lower fineness than the original mesh generated by thinning out vertices from the original mesh of the encoding target, the original mesh being composed of vertices and connections representing a three-dimensional structure of the object, The attributes include a texture applied to the surface of the original mesh, The displacement vector is vector information indicating the displacement of a vertex obtained by subdividing the base mesh, The content file includes a base grid track storing the encoded data of the base grid, an attribute track storing the encoded data of the attribute, and a plurality of displacement tracks storing the encoded data of displacement vectors of mutually different hierarchies, and The control file includes a base grid adaptation set managing the base grid track, an attribute adaptation set managing the attribute track, and the displacement adaptation set managing the displacement track.

Images