Information processing device and method

Abstract

The present disclosure pertains to an information processing device and a method for making it possible to curb any increase in the data volume of a scene description. The invention generates a scene description file including track information designating, as a referent, some tracks that have information for accessing other tracks, from among a plurality of tracks of a file container that manages information pertaining to data of 3D object content. The invention also acquires data of 3D object content managed in all of the tracks on the basis of the scene description file including track information designating, as a referent, some tracks that have information for accessing other tracks, from among the plurality of tracks of the file container, and on the basis of inter-track reference relationships. The present disclosure can be applied to, for example, an information processing device, an information processing method, and the like.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and method, and more particularly to an information processing device and method that can suppress an increase in the amount of scene description data. [Background technology]

[0002] Conventionally, there is the GL Transmission Format (glTF) (registered trademark) 2.0, which is a scene description format for arranging 3D (three-dimensional) objects in a three-dimensional space (see, for example, Non-Patent Document 1).

[0003] In recent years, a method has been proposed in MPEG (Moving Picture Experts Group)-I Scene Description to extend glTF2.0 and handle timed media that has been coded and stored in ISOBMFF or the like as texture data (see, for example, Non-Patent Document 2).

[0004] Incidentally, as an encoding method for point clouds, which are collections of points that simultaneously have position information and attribute information (color, reflection, etc.) in three-dimensional space, V-PCC (Video-based Point Cloud Compression) has been proposed, which segments the point cloud to form regions, projects each region onto a plane, and encodes it using a video codec (see, for example, Non-Patent Document 3).

[0005] Also, a method of storing a V3C bitstream composed of coded data of point clouds coded with this V-PCC in ISOBMFF has been studied (see, for example, Non-Patent Document 4). For example, when ISOBMFF has a multi-track structure composed of multiple tracks, all tracks are specified when specifying V-PCC data in MPEG_media of the scene description. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Saurabh Bhatia, Patrick Cozzi, Alexey Knyazev, Tony Parisi, "Khronos glTF2.0", https: / / github.com / KhronosGroup / glTF / tree / master / specification / 2.0, June 9, 2017 [Non-patent document 2] "Text of ISO / IEC CD 23090-14 Scene Description for MPEG Media", ISO / IEC JTC 1 / SC 29 / WG 3 N00026, 2020-12-07 [Non-patent document 3] "ISO / IEC FDIS 23090-5 Visual Volumetric Video-based Coding and Video-based Point Cloud Compression", ISO / IEC JTC 1 / SC 29 / WG 11 N19579, 2020-09-21 [Non-patent document 4] "Text of ISO / IEC DIS 23090-10 Carriage of Visual Volumetric Video-based Coding Data", ISO / IEC JTC 1 / SC 29 / WG 11 N19285, 2020-06-1 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the decoder can obtain the reference relationship between tracks from ISOBMFF, etc. Therefore, in the conventional method described above, specifying all tracks by the scene description is redundant, and there is a risk that the amount of data in the scene description will increase unnecessarily.

[0008] The present disclosure has been made in light of such circumstances, and makes it possible to suppress an increase in the amount of data in scene descriptions. [Means for solving the problem]

[0009] An information processing device according to one aspect of the present technology is an information processing device that includes a file generation unit that generates a scene description file that describes a scene of the 3D object content, the scene description file including track information that specifies, as a reference, some tracks that have information for accessing other tracks out of multiple tracks of a file container that manages information related to data of the 3D object content.

[0010] An information processing method according to one aspect of the present technology is an information processing method for generating a scene description file describing a scene of 3D object content, the scene description file including track information that specifies, as a reference, some tracks among a plurality of tracks of a file container that manages information about data of the 3D object content, the some tracks having information for accessing other tracks.

[0011] An information processing device according to another aspect of the present technology is an information processing device that includes a scene description file that describes a scene of the 3D object content, including track information that specifies, as a reference destination, some tracks that have information for accessing other tracks out of multiple tracks of a file container that manages information related to data of the 3D object content, and an acquisition unit that acquires the data of the 3D object content managed in all of the tracks based on the reference relationships between the tracks.

[0012] Another aspect of the information processing method of the present technology is an information processing method for acquiring a scene description file describing a scene of the 3D object content, including track information that specifies, as a reference destination, some tracks that have information for accessing other tracks out of multiple tracks of a file container that manages information related to data of the 3D object content, and the data of the 3D object content managed in all of the tracks based on the reference relationships between the tracks.

[0013] In an information processing device and method according to one aspect of the present technology, a scene description file is generated that describes a scene of 3D object content, including track information that specifies, as a reference, some tracks that have information for accessing other tracks out of multiple tracks in a file container that manages information related to data of 3D object content.

[0014] In an information processing device and method according to another aspect of the present technology, data of the 3D object content managed in all tracks is acquired based on a scene description file describing a scene of the 3D object content, including track information that specifies, as a reference destination, some tracks that have information for accessing other tracks among a plurality of tracks of a file container that manages information related to data of the 3D object content, and the reference relationships between the tracks. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating an example of the main configuration of glTF2.0. [Figure 2] FIG. 1 is a diagram illustrating an example of glTF objects and reference relationships. [Figure 3] FIG. 10 is a diagram showing an example of a scene description. [Figure 4] FIG. 10 is a diagram illustrating a method for accessing binary data. [Figure 5] FIG. 10 is a diagram showing an example of a scene description. [Figure 6] FIG. 10 is a diagram illustrating the relationship between a buffer object, a buffer view object, and an accessor object. [Figure 7] 10A and 10B are diagrams illustrating examples of descriptions of a buffer object, a buffer view object, and an accessor object. [Figure 8] FIG. 10 is a diagram illustrating an example of the configuration of a scene description object. [Figure 9] FIG. 10 is a diagram showing an example of a scene description. [Figure 10] FIG. 10 is a diagram illustrating a method for expanding an object. [Figure 11] FIG. 10 is a diagram illustrating an example of the configuration of a scene description object. [Figure 12] This is a diagram explaining the signal contents of KHR_draco_mesh_compression. [Figure 13] FIG. 10 is a diagram showing an example of a scene description. [Figure 14] FIG. 10 is a diagram illustrating a configuration of a client process. [Figure 15] FIG. 10 is a diagram illustrating an example of the configuration of an extension for handling timed metadata. [Figure 16] FIG. 10 is a diagram showing an example of a scene description. [Figure 17] FIG. 10 is a diagram showing an example of a scene description. [Figure 18] FIG. 10 is a diagram illustrating an example of the configuration of an extension for handling timed metadata. [Figure 19] FIG. 2 illustrates an example of the main configuration of a client. [Figure 20] 10 is a flowchart illustrating an example of the flow of a client process. [Figure 21] FIG. 1 is a diagram illustrating an overview of a V-PCC. [Figure 22] A diagram showing an example of the structure of an ISOBMFF that stores a V3C bitstream. [Figure 23] FIG. 10 is a diagram showing an example of track specification using an MPD. [Figure 24] FIG. 10 is a diagram showing an example of a description regarding track designation in the case of a single track structure. [Figure 25] FIG. 10 is a diagram showing an example of a description regarding track designation in the case of a multi-track structure. [Figure 26] FIG. 10 is a diagram illustrating a method for specifying a track. [Figure 27] FIG. 10 is a diagram showing an example of a description regarding track designation in the case of a multi-track structure. [Figure 28] FIG. 2 is a block diagram illustrating an example of the main configuration of a file generation device. [Figure 29] 10 is a flowchart illustrating an example of the flow of a file generation process. [Figure 30] FIG. 2 is a block diagram illustrating an example of the main configuration of a client device. [Figure 31] 10 is a flowchart illustrating an example of the flow of a client process. [Figure 32] 10 is a flowchart illustrating an example of the flow of a client process. [Figure 33] FIG. 10 is a diagram illustrating a configuration of a client process. [Figure 34] FIG. 10 is a diagram illustrating an example of the configuration of a scene description object. [Figure 35] 10 is a flowchart illustrating an example of the flow of a client process. [Figure 36] FIG. 10 is a diagram illustrating an example of the configuration of a scene description object. [Figure 37] 10 is a flowchart illustrating an example of the flow of a client process. [Figure 38] FIG. 10 is a diagram showing an example of how attributes other than color are specified. [Figure 39] FIG. 10 is a diagram illustrating a method for specifying an attribute. [Figure 40] FIG. 10 is a diagram illustrating an example of the configuration of a scene description object. [Figure 41] FIG. 10 is a diagram showing an example of a scene description. [Figure 42] FIG. 10 is a diagram illustrating an example of the configuration of a scene description object. [Figure 43] FIG. 10 is a diagram showing an example of a scene description. [Figure 44] FIG. 10 is a diagram illustrating an example of the configuration of a scene description object. [Figure 45] FIG. 10 is a diagram showing an example of a scene description. [Figure 46] 10 is a flowchart illustrating an example of the flow of a file generation process. [Figure 47] 10 is a flowchart illustrating an example of the flow of a client process. [Figure 48] 10 is a flowchart illustrating an example of the flow of a client process. [Figure 49] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1.MPEG-I Scene Description 2. Redundancy of track information is reduced 3. Specifying attributes 4. Notes

[0017] <1.MPEG-I Scene Description> <References supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that were publicly known at the time of filing, as well as the contents of other documents referenced in the following non-patent documents.

[0018] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: (mentioned above)

[0019] In other words, the contents of the above-mentioned non-patent documents and the contents of other documents referenced in the above-mentioned non-patent documents are also used as the basis for determining the support requirements. For example, even if syntax and terminology such as glTF2.0 and its extensions described in Non-Patent Documents 1 to 3 are not directly defined in this disclosure, they are considered to be within the scope of this disclosure and meet the support requirements of the claims. Similarly, even if technical terms such as parsing, syntax, and semantics are not directly defined in this disclosure, they are considered to be within the scope of this disclosure and meet the support requirements of the claims.

[0020] <gltf2.0> Conventionally, as described in Non-Patent Document 1, for example, there is glTF (The GL Transmission Format) (registered trademark) 2.0, which is a format for placing 3D (three-dimensional) objects in a three-dimensional space. As shown in FIG. 1, for example, glTF 2.0 is composed of a JSON format file (.glTF), a binary file (.bin), and an image file (.png, .jpg, etc.). The binary file stores binary data such as geometry and animation. The image file stores data such as texture.

[0021] A JSON format file is a scene description file written in JSON (JavaScript (registered trademark) Object Notation). A scene description is metadata that describes (a description of) a scene of 3D content. The description of this scene description defines what kind of scene it is. A scene description file is a file that stores such a scene description. In this disclosure, a scene description file is also referred to as a scene description file.

[0022] The description of the JSON format file consists of a list of key and value pairs. An example of the format is shown below. “KEY”:”VALUE”

[0023] Keys consist of strings, and values ​​consist of numbers, strings, booleans, arrays, objects, or null.

[0024] Additionally, multiple key-value pairs ("KEY":"VALUE") can be grouped together using {} (curly braces). This grouping is also called a JSON object. An example of the format is shown below. “user”:{"id":1, "name":"tanaka"}

[0025] In this example, a JSON object that combines the pair "id":1 and the pair "name":"tanaka" is defined as the value corresponding to the key (user).

[0026] You can also create an array of zero or more values ​​using square brackets ([]). This array is also called a JSON array. For example, a JSON object can be applied as an element of this JSON array. An example of the format is shown below. test":["hoge", "fuga", "bar"] "users":[{"id":1, "name":"tanaka"},{"id":2,"name":"yamada"},{"id":3, "name":"sato"}]

[0027] Figure 2 shows the glTF objects that can be written at the top level of a JSON format file and the reference relationships they can have. The long circles in the tree structure shown in Figure 2 represent objects, and the arrows between those objects show the reference relationships. As shown in Figure 2, objects such as "scene", "node", "mesh", "camera", "skin", "material", and "texture" are written at the top level of a JSON format file.

[0028] An example of such a JSON format file (scene description) is shown in FIG. 3. The JSON format file 20 in FIG. 3 shows an example of a portion of the top-level description. In this JSON format file 20, all top-level objects 21 used are described at the top level. These top-level objects 21 are the glTF objects shown in FIG. 2. Furthermore, in the JSON format file 20, reference relationships between objects are shown as indicated by arrows 22. More specifically, the reference relationships are shown by specifying the index of an element in the array of the referencing object in the property of the higher-level object.

[0029] Fig. 4 is a diagram illustrating a method for accessing binary data. As shown in Fig. 4, binary data is stored in a buffer object. That is, information for accessing the binary data (for example, a uniform resource identifier (URI)) is indicated in the buffer object. In a JSON format file, as shown in Fig. 4, objects such as a mesh, camera, and skin can access the buffer object via an accessor object and a bufferView object.

[0030] That is, for objects such as mesh, camera, and skin, the accessor object to be referenced is specified. An example of a mesh object description in a JSON format file is shown in Figure 5. For example, as shown in Figure 5, in a mesh object, vertex attributes such as NORMAL, POSITION, TANGENT, and TEXCORD_0 are defined as keys, and for each attribute, the accessor object to be referenced is specified as a value.

[0031] The relationship between buffer objects, buffer view objects, and accessor objects is shown in Figure 6. An example of how these objects are written in a JSON format file is shown in Figure 7.

[0032] 6, buffer object 41 is an object that stores information (such as a URI) for accessing binary data, which is actual data, and information indicating the data length (for example, byte length) of that binary data. A in FIG. 7 shows an example of the description of buffer object 41. "bytelength":102040" shown in A in FIG. 7 indicates that the byte length of buffer object 41 is 102040 bytes, as shown in FIG. 6. Furthermore, "uri":"duck.bin" shown in A in FIG. 7 indicates that the URI of buffer object 41 is "duck.bin", as shown in FIG. 6.

[0033] 6, the buffer view object 42 is an object that stores information about a subset area of ​​binary data specified in the buffer object 41 (i.e., information about a partial area of ​​the buffer object 41). B of Fig. 7 shows an example of description of the buffer view object 42. As shown in Fig. 6 and B of Fig. 7, the buffer view object 42 stores information such as identification information of the buffer object 41 to which the buffer view object 42 belongs, an offset (e.g., a byte offset) indicating the position of the buffer view object 42 within the buffer object 41, and a length (e.g., a byte length) indicating the data length (e.g., a byte length) of the buffer view object 42.

[0034] As shown in B of Fig. 7, when there are multiple buffer view objects, information is written for each buffer view object (i.e., for each subset area). For example, information such as "buffer":0, "bytelength":25272, and "byteOffset":0 shown at the top of B of Fig. 7 is information for the first buffer view object 42 (bufferView[0]) shown in the buffer object 41 in Fig. 6. Furthermore, information such as "buffer":0, "bytelength":76768, and "byteOffset":25272 shown at the bottom of B of Fig. 7 is information for the second buffer view object 42 (bufferView[1]) shown in the buffer object 41 in Fig. 6.

[0035] "buffer":0" of the first buffer view object 42 (bufferView[0]) shown in B of FIG. 7 indicates that the identification information of the buffer object 41 to which the buffer view object 42 (bufferView[0]) belongs is "0" (Buffer[0]), as shown in FIG. 6. Also, "bytelength":25272" indicates that the byte length of the buffer view object 42 (bufferView[0]) is 25272 bytes. Furthermore, "byteOffset":0" indicates that the byte offset of the buffer view object 42 (bufferView[0]) is 0 bytes.

[0036] "buffer":0" of the second buffer view object 42 (bufferView[1]) shown in B of FIG. 7 indicates that the identification information of the buffer object 41 to which the buffer view object 42 (bufferView[0]) belongs is "0" (Buffer[0]), as shown in FIG. 6. Also, "bytelength":76768" indicates that the byte length of the buffer view object 42 (bufferView[0]) is 76768 bytes. Furthermore, "byteOffset":25272" indicates that the byte offset of the buffer view object 42 (bufferView[0]) is 25272 bytes.

[0037] 6, the accessor object 43 is an object that stores information about how to interpret data in the buffer view object 42. C in Fig. 7 shows an example of the description of the accessor object 43. As shown in Figs. 6 and 7C, the accessor object 43 stores information such as the identification information of the buffer view object 42 to which the accessor object 43 belongs, the offset (e.g., byte offset) indicating the position of the buffer view object 42 within the buffer object 41, the component type of the buffer view object 42, the number of data items stored in the buffer view object 42, and the type of data items stored in the buffer view object 42. This information is described for each buffer view object.

[0038] In the example of C in FIG. 7, information such as "bufferView":0, "byteOffset":0, "componentType":5126, "count":2106," and "type":"VEC3" is shown. "bufferView":0" indicates that the identification information of the buffer view object 42 to which the accessor object 43 belongs is "0" (bufferView[0]), as shown in FIG. 6. Furthermore, "byteOffset":0" indicates that the byte offset of the buffer view object 42 (bufferView[0]) is 0 bytes. Furthermore, "componentType":5126" indicates that the component type is FLOAT type (OpenGL macro constant). Furthermore, "count":2106" indicates that 2106 pieces of data are stored in the buffer view object 42 (bufferView[0]). Furthermore, "type":"VEC3" indicates that the data (type) stored in the buffer view object 42 (bufferView[0]) is a three-dimensional vector.

[0039] All accesses to data other than images are defined by reference to this accessor object 43 (by specifying the index of the accessor).

[0040] Next, we will explain how to specify a 3D object in a point cloud in such a glTF2.0-compliant scene description (JSON format file). A point cloud is 3D content that represents a three-dimensional structure (an object with a three-dimensional shape) as a collection of many points. Point cloud data consists of position information (also called geometry) and attribute information (also called attributes) for each point. Attributes can contain any information. For example, attributes may include color information, reflectance information, normal information, etc. for each point. As such, point clouds have a relatively simple data structure, and by using a sufficient number of points, they can represent any three-dimensional structure with sufficient accuracy.

[0041] When a point cloud does not change in the time direction (also called static), 3D objects are specified using the mesh.primitives object of glTF2.0. Figure 8 shows an example of the configuration of objects in a scene description when the point cloud is static. Figure 9 shows an example of how the scene description is written.

[0042] As shown in Figure 9, the mode of the primitives object is set to 0, which indicates that the data is treated as points in a point cloud. As shown in Figures 8 and 9, the POSITION property of the attributes object in mesh.primitives specifies an accessor to a buffer that stores the position information of the points. Similarly, the COLOR property of the attributes object specifies an accessor to a buffer that stores the color information of the points. The buffer and bufferView may be one (the data may be stored in a single file).

[0043] Next, we will explain how to extend such scene description objects. Each object in glTF 2.0 can store newly defined objects within an extension object. Figure 10 shows an example of how to specify a newly defined object (ExtensionExample). As shown in Figure 10, when using a newly defined extension, the extension object name (ExtensionExample in the example in Figure 10) is written in "extensionUsed" and "extensionRequired." This indicates that this extension is an extension that will be used, or an extension that needs to be loaded.

[0044] An example of an extension of this object will be described below. Figure 11 is a diagram showing an example of the configuration of objects in a scene description when an extension is defined in mesh.primitives. As shown in Figure 11, in this example, an HKR_draco_mesh_compression extension object is defined in mesh.primitives. This HKR_draco_mesh_compression extension is an extension object for using draco mesh-compressed mesh data as data for a glTF mesh object. Draco is one of the mesh encoding methods.

[0045] The HKR_draco_mesh_compression extension stores a property (bufferView) that specifies the compressed data. As shown in Figure 12, the decoder (Draco mesh decoder) can obtain the compressed data based on this information. The HKR_draco_mesh_compression extension also stores an object (attributes) that contains linking information for using the multiple attribute data (Raw data A, B) obtained by decompressing the compressed data as attribute data indicated by glTF's primitives.attributes. The property names of these attributes are the same as the property names in primitives.attributes. The property values ​​are set to the attribute ID values ​​of the mesh data handled by the decoder (draco mesh decoder). Figure 13 shows an example of this mesh object description.

[0046] <Client processing> Next, we will explain the processing of a client device in MPEG-I Scene Description. The client device acquires a scene description, acquires 3D object data based on the scene description, and generates a display image using the scene description and 3D object data.

[0047] As described in Non-Patent Document 2, in a client device, a presentation engine, a media access function, and the like perform processing. For example, as shown in FIG. 14, a presentation engine 51 of a client device 50 acquires the initial value of a scene description and information for updating the scene description (hereinafter also referred to as update information), and generates a scene description for the processing time. The presentation engine 51 then analyzes the scene description and identifies the media to be played (video, audio, etc.). The presentation engine 51 then requests a media access function 52 to acquire the media via a media access API (Application Program Interface). The presentation engine 51 also sets up pipeline processing, specifies buffers, and the like.

[0048] The media access function 52 acquires various media data requested by the presentation engine 51 from the cloud, local storage, etc. The media access function 52 supplies the acquired various media data (encoded data) to a pipeline 53.

[0049] The pipeline 53 decodes various data (encoded data) of the supplied media through pipeline processing and supplies the decoding result to a buffer 54. The buffer 54 holds various data of the supplied media.

[0050] The presentation engine 51 performs rendering and the like using various data of the media held in the buffer 54.

[0051] <Application of Timed media> In recent years, for example, as shown in Non-Patent Document 2, in MPEG-I Scene Description, it has been studied to extend glTF2.0 and apply timed media as 3D object content. Timed media is media data that changes in the time axis direction, like moving images in 2D images.

[0052] glTF was only applicable to still image data as media data (3D object content). That is, glTF did not support media data of moving images. When moving a three-dimensional object, animation (a method of switching still images along the time axis) was applied.

[0053] In MPEG-I Scene Description, it has been studied to apply its glTF2.0, apply a JSON format file as a scene description, and further extend glTF so that timed media (for example, video data) can be handled as media data. To handle timed media, for example, the following extensions are made.

[0054] Fig. 15 is a diagram illustrating an extension for handling timed media. In the example of Fig. 15, an MPEG media object (MPEG_media) is a glTF extension, and is an object that specifies attributes of MPEG media such as video data, for example, uri, track, renderingRate, startTime, etc.

[0055] 15, an MPEG texture video object (MPEG_texture_video) is provided as an extension object of the texture object (texture). The MPEG texture video object stores information of an accessor corresponding to a buffer object to be accessed. That is, the MPEG texture video object is an object that specifies the index of an accessor corresponding to a buffer where the texture media specified by the MPEG media object (MPEG_media) is decoded and stored.

[0056] 16 is a diagram showing an example of description of an MPEG media object (MPEG_media) and an MPEG texture video object (MPEG_texture_video) in a scene description to explain extensions for handling timed media. In the example of FIG. 16, in the second line from the top, an MPEG texture video object (MPEG_texture_video) is set as an extension object (extensions) of a texture object (texture), as shown below. Then, the accessor index ("2" in this example) is specified as the value of that MPEG video texture object.

[0057] "texture":[{"sampler":0, "source":1, "extensions":{"MPEG_texture_video ":"accessor":2}}],

[0058] 16, an MPEG media object (MPEG_media) is set as a glTF extension object (extensions) on lines 7 to 16 from the top, as shown below: The value of the MPEG media object stores various information about the MPEG media object, such as the encoding and URI of the MPEG media object.

[0059] "MPEG_media":{ "media":[ {"name":"source_1", "renderingRate":30.0, "startTime":9.0, "timeOffset":0.0, "loop":"true", "controls":"false", "alternatives":[{"mimeType":"video / mp4;codecs=\"avc1.42E01E\"", "uri":"video1.mp4", "tracks":[{"track":""#track_ID=1"}] }] } ] }

[0060] Furthermore, each frame of data is decoded and stored sequentially in a buffer. However, since the position and other information fluctuate, a mechanism is provided in the scene description to store this fluctuating information and enable a renderer to read the data. For example, as shown in FIG. 15, an MPEG buffer circular object (MPEG_buffer_circular) is provided as an extension object of a buffer object. This MPEG buffer circular object stores information for dynamically storing data in the buffer object. For example, information indicating the data length of the buffer header and information indicating the number of frames are stored in this MPEG buffer circular object. The buffer header stores information such as an index, a timestamp and data length of the frame data to be stored, etc.

[0061] 15, an MPEG accessor timed object (MPEG_timed_accessor) is provided as an extension object of the accessor object. In this case, since the media data is a moving image, the buffer view object (bufferView) referenced in the time direction may change (its position may fluctuate). Therefore, information indicating the buffer view object to be referenced is stored in this MPEG accessor timed object. For example, the MPEG accessor timed object stores information indicating a reference to the buffer view object (bufferView) in which the timed accessor information header (timedAccessor information header) is written. The timed accessor information header is header information that stores, for example, dynamically changing accessor objects and information in the buffer view object.

[0062] 17 is a diagram showing an example of the description of an MPEG buffer circular object (MPEG_buffer_circular) and an MPEG accessor timed object (MPEG_accessor_timed) in a scene description to explain extensions for handling timed media. In the example of Fig. 17, on the fifth line from the top, an MPEG accessor timed object (MPEG_accessor_timed) is set as an extension object (extensions) of the accessor object (accessors) as shown below. Then, as the value of the MPEG accessor timed object, parameters such as the index of the buffer view object ("1" in this example), update rate (updataRate), and immutable information (immutable) and their values ​​are specified.

[0063] "MPEG_accessor_timed":{"bufferView":1, "updateRate":25.0, "immutable":1,"}

[0064] 17, on the 13th line from the top, an MPEG buffer circular object (MPEG_buffer_circular) is set as an extension object (extensions) of the buffer object (buffer), as shown below: Then, parameters such as a buffer frame count (count), a header length (headerLength), and an update rate (updataRate) and their values ​​are specified as values ​​of the MPEG buffer circular object.

[0065] "MPEG_buffer_circular":{"count":5, "headerLength":12, "updateRate":25.0}

[0066] FIG. 18 is a diagram for explaining an extension for handling time media. In FIG. 18, an example of the relationship between an MPEG accessor time object or an MPEG buffer circular object and an accessor object, a buffer view object, and a buffer object is shown.

[0067] In the MPEG buffer circular object of the buffer object, as described above, information necessary for storing time-varying data in the buffer area indicated by the buffer object, such as buffer frame count (count), header length (headerLength), update rate (updataRate), etc., is stored. Also, parameters such as index (idex), timestamp (timestamp), data length (length), etc. are stored in the buffer header (bufferHeader) which is the header of that buffer area.

[0068] In the MPEG accessor time object of the accessor object, as described above, information regarding the buffer view object to be referenced, such as the index of the buffer view object (bufferView), update rate (updataRate), immutable information (immutable), etc., is stored. Also, information regarding the buffer view object in which the time-domain accessor information header to be referenced is stored is stored in this MPEG accessor time object. The time-domain accessor information header may store a timestamp delta (timestamp_delta), updated data of the accessor object, updated data of the buffer view object, etc.

[0069] ​A scene description is spatial layout information for arranging one or more 3D objects in a 3D space. The contents of this scene description can be updated along the time axis. In other words, the layout of 3D objects can be updated over time. This section describes the client processing performed on the client device in this case.

[0070] Fig. 19 shows an example of the main configuration of a client device related to client processing, and Fig. 20 is a flowchart showing an example of the flow of the client processing. As shown in Fig. 19, the client device has a presentation engine (hereinafter also referred to as PE) 51, a media access function (hereinafter also referred to as MAF) 52, a pipeline 53, and a buffer 54.

[0071] When the client processing is started, the glTF analysis unit 63 of the presentation engine (PE) 51 starts the PE processing, and in step S21, acquires the SD(glTF) file 62, which is a scene description file, and parses the scene description.

[0072] In step S22, the glTF analyzer 63 checks the media associated with the 3D object (texture), the buffer in which the media will be stored after processing, and the accessor. In step S23, the glTF analyzer 63 notifies the media access function 52 of this information as a file acquisition request.

[0073] The media access function (MAF) 52 starts MAF processing and receives the notification in step S11. In step S12, the media access function 52 receives the media (3D object file (mp4)) based on the notification.

[0074] In step S13, the media access function 52 decodes the acquired media (3D object file (mp4)). In step S14, the media access function 52 stores the data of the decoded media in the buffer 54 based on the notification from the presentation engine (PE51).

[0075] In step S24, the rendering processing unit 64 of the presentation engine 51 reads (acquires) the data from the buffer 54 at an appropriate timing. In step S25, the rendering processing unit 64 performs rendering using the acquired data to generate a display image.

[0076] The media access function 52 executes these processes for each time (each frame) by repeating the processes of step S13 and step S14. Also, the rendering processing unit 64 of the presentation engine 51 executes these processes for each time (each frame) by repeating the processes of step S24 and step S25. When the processing for all frames is completed, the media access function 52 ends the MAF process, and the presentation engine 51 ends the PE process. That is, the client process ends.

[0077] <Overview of V-PCC> By the way, as described in, for example, Non-Patent Document 3, as an encoding method for a point cloud, which is a set of points having position information and attribute information (such as color and reflection) simultaneously in a three-dimensional space, V-PCC (Video based Point Cloud Compression) has been proposed, in which the point cloud is segmented to form regions, and each region is projected onto a plane and encoded by a video codec.

[0078] In V-PCC, the geometry and attributes of a point cloud are projected onto a two-dimensional plane for each small region. In the present disclosure, these small regions may be referred to as partial regions. An image in which the geometry and attributes are projected onto a two-dimensional plane is also referred to as a projected image. Furthermore, the projected image for each small region (partial region) is referred to as a patch. For example, an object 71 (3D data) in A of FIG. 21 is decomposed into patches 72 (2D data) as shown in B of FIG. 21. In the case of a geometry patch, each pixel value indicates the position information of a point. However, in this case, the position information of the point is expressed as position information (depth value) in the direction perpendicular to the projection plane (depth direction).

[0079] Then, each patch generated in this manner is arranged in a frame image (also referred to as a video frame) of a video sequence. A frame image in which geometry patches are arranged is also referred to as a geometry video frame. A frame image in which attribute patches are arranged is also referred to as an attribute video frame. For example, from an object 71 in A of FIG. 21, a geometry video frame 81 in which geometry patches 73 are arranged as shown in C of FIG. 21, and an attribute video frame 82 in which attribute patches 74 are arranged as shown in D of FIG. 21 are generated. For example, each pixel value of the geometry video frame 81 indicates the above-mentioned depth value.

[0080] These video frames are then encoded using a coding method for two-dimensional images, such as AVC (Advanced Video Coding) or HEVC (High Efficiency Video Coding). In other words, point cloud data, which is 3D data representing a three-dimensional structure, can be encoded using a codec for two-dimensional images.

[0081] An occupancy map (also called an occupancy image) can also be used. The occupancy map is map information that indicates the presence or absence of a projected image (patch) for each NxN pixel of a geometry video frame or an attribute video frame. For example, the occupancy map indicates areas (NxN pixels) in the geometry video frame or the attribute video frame where a patch exists with a value of "1" and areas (NxN pixels) in which a patch does not exist with a value of "0."

[0082] By referencing this occupancy map, the decoder can determine whether an area contains a patch, thereby suppressing the effects of noise caused by encoding and decoding and restoring 3D data more accurately. For example, even if depth values ​​change due to encoding and decoding, the decoder can ignore depth values ​​in areas where no patches exist by referencing the occupancy map. In other words, by referencing the occupancy map, the decoder can avoid processing the depth values ​​as position information for 3D data.

[0083] For example, an occupancy map 83 as shown in Fig. 21E may be generated for the geometry video frame 11 and the attribute video frame 12. In the occupancy map 83, white areas indicate a value of "1" and black areas indicate a value of "0."

[0084] Such an occupancy map can be encoded as data (video frame) separate from the geometry video frame and the attribute video frame and transmitted to the decoding side. That is, the occupancy map can also be encoded using a coding method for two-dimensional images such as AVC or HEVC, just like the geometry video frame and the attribute video frame.

[0085] The coded data (bitstream) generated by coding a geometry video frame is also called a geometry video sub-bitstream. The coded data (bitstream) generated by coding an attribute video frame is also called an attribute video sub-bitstream. The coded data (bitstream) generated by coding an occupancy map is also called an occupancy map video sub-bitstream. Note that when there is no need to distinguish between the geometry video sub-bitstream, attribute video sub-bitstream, and occupancy map video sub-bitstream, they are all called video sub-bitstreams.

[0086] Furthermore, atlas information (atlas), which is information for reconstructing a point cloud (3D data) from the patches (2D data), is encoded and transmitted to the decoding side. The encoding method (and decoding method) of the atlas information is arbitrary. The encoded data (bitstream) generated by encoding the atlas information is also called the atlas sub-bitstream.

[0087] In the following, the point cloud (objects) is assumed to be capable of changing in the time direction (also referred to as dynamic), like a moving image of a 2D image. That is, the geometry data and the attribute data have a time direction concept and are data sampled at predetermined time intervals, like a moving image of a 2D image. Note that the data at each sampling time is referred to as a frame, like a video frame of a 2D image. That is, the point cloud data (geometry data and attribute data) is assumed to be composed of a plurality of frames, like a moving image of a 2D image. In the present disclosure, this frame of the point cloud is also referred to as a point cloud frame. In the case of V-PCC, even for such a point cloud of a moving image (a plurality of frames), by video-framing each point cloud frame to form a video sequence, it can be encoded with high efficiency using a moving image encoding method.

[0088] <Method of storing in ISOBMFF> Also, for example, as described in Non-Patent Document 4, a method of storing a V3C bitstream composed of the encoded data of the point cloud encoded by this V-PCC in ISOBMFF has been studied. Non-Patent Document 4 defines two types of methods for storing a V3C bitstream in ISOBMFF: a single track structure and a multi-track structure.

[0089] The single track structure is a method of storing a V3C bitstream in one track. That is, in this case, the geometry video sub-bitstream, the attribute video sub-bitstream, the occupancy map video sub-bitstream, and the atlas sub-bitstream are stored in the same track as each other.

[0090] The multi-track structure is a method of storing the geometry video sub-bitstream, attribute video sub-bitstream, occupancy video sub-bitstream, and atlas sub-bitstream in individual tracks. Since each video sub-bitstream is a conventional 2D video stream, it can be stored (managed) using the same method as for 2D. An example of a file configuration when the multi-track structure is applied is shown in FIG. 22. As shown in FIG. 22, in the case of the multi-track structure, one track (V3C track) stores a track reference, which is information for accessing other tracks (also called component tracks) that store V3C bitstreams. In other words, each component track is linked to the V3C track by this track reference.

[0091] When 3D object content is distributed by applying MPEG-DASH (Moving Picture Experts Group Dynamic Adaptive Streaming over HTTP (Hypertext Transfer Protocol)), a preselection element and a preselection descriptor may be stored in an MPD (Media Presentation Description), which is a control file for controlling the distribution, as information for compiling the Adaptation Sets that make up the V-PCC. An example of such a description is shown in Fig. 23. In other words, in this case, these pieces of information in the MPD associate the individual bitstreams that make up the V3C bitstream with each other.

[0092] <Track information> The MPEG media object (MPEG_media) of the scene description stores information for acquiring 3D object content data (V3C bitstream). For example, information specifying, by uri, a file (e.g., an MP4 file) that stores the V3C bitstream or a control file (e.g., an MPD) that controls the distribution of the V3C bitstream is stored. Also stored is track information (tracks array) that specifies a track that stores the V3C bitstream as a reference destination. This track information specifies the reference destination track using track identification information (track ID) (AdaptationSet ID in the case of an MPD) in the track property.

[0093] Figure 24 is a diagram showing an example of a description of an MPEG media object (MPEG_media) when the ISOBMFF has a single track structure. In the example of Figure 24, only one track (or adaptation set) is specified as a reference destination in the track information (tracks array) as follows:

[0094] track":"#track=1" "codecs":"v3e1"

[0095] 25 is a diagram showing an example of a description of an MPEG media object (MPEG_media) when the ISOBMFF has a multi-track structure. In the example of FIG. 25, all tracks (or adaptation sets) that store V3C bitstreams are specified as reference destinations in the track information (tracks array) as follows:

[0096] "tracks": [ {“track”:“#track=1” “codecs”: “v3c1”}, {“track”:“#track=2” “codecs”: “resv.vvvc.hvc1”}, {“track”:“#track=3” “codecs”: “resv.vvvc.hvc1”}, {“track”:“#track=4” “codecs”: “resv.vvvc.hvc1”} ]

[0097] <Redundancy of track information> However, when ISOBMFF has a multi-track structure, information on other component tracks linked to the V3C track is stored in the information on the DASH layer or ISOBMFF layer, and the decoder can obtain that information from there. Therefore, specifying all tracks as references in the scene description (track information) described above is redundant, and there is a risk that the amount of data in the scene description will increase unnecessarily.

[0098] 2. Reduction of redundancy in track information Therefore, as shown in the top row of the table in Figure 26, in the scene description, some tracks that contain information about other tracks in the file container of the multi-track structure may be specified as reference destinations (Method 1).

[0099] For example, an information processing device may include a file generation unit that generates a scene description file that describes a scene of 3D object content, including track information that specifies, as a reference, some tracks that have information for accessing other tracks out of multiple tracks of a file container that manages information related to data of 3D object content.

[0100] For example, in an information processing method, a scene description file describing a scene of 3D object content may be generated, which includes track information that specifies, as a reference, some tracks that have information for accessing other tracks out of multiple tracks of a file container that manages information about 3D object content data.

[0101] For example, an information processing device may be provided with a scene description file describing a scene of the 3D object content, including track information that specifies, as a reference, some tracks that have information for accessing other tracks out of multiple tracks of a file container that manages information related to data of the 3D object content, and an acquisition unit that acquires data of the 3D object content managed in all tracks based on the reference relationships between the tracks.

[0102] For example, in an information processing method, a scene description file describing a scene of the 3D object content, including track information that specifies as a reference some tracks having information for accessing other tracks among multiple tracks of a file container that manages information regarding data of the 3D object content, and data of the 3D object content managed in all tracks may be acquired based on the reference relationships between the tracks.

[0103] By doing so, it is possible to prevent an increase in redundancy of track information in the scene description, and therefore to prevent an increase in the data amount of the scene description.

[0104] In other words, in the track information of the scene description file, the designation of reference destinations for some of the multiple tracks constituting the file container is omitted. As described above, the reference relationship between tracks is obtained in a file container such as ISOBMFF or a control file such as MPD, so the acquisition unit can obtain the data of the 3D object content even if the designation of reference destinations for some of the tracks in the track information is omitted. Therefore, by doing so, it is possible to suppress an increase in redundancy of the track information and an increase in the data amount of the scene description.

[0105] The track information may be information that specifies a single track that has information for accessing all other tracks as a reference. By referencing a track that has information for accessing all other tracks, the acquisition unit can acquire the data of the 3D object content managed in all tracks from that information. Therefore, by doing so, it is possible to further suppress an increase in redundancy of track information and further suppress an increase in the amount of data in the scene description.

[0106] The file container may also be an ISOBMFF (International Organization for Standardization Base Media File Format) that applies a multi-track structure. This makes it possible to suppress an increase in redundancy of track information for an ISOBMFF that applies a multi-track structure, and to suppress an increase in the data amount of the scene description.

[0107] Furthermore, the data of the 3D object content may be a V3C (Visual Volumetric Video-based Coding) bitstream in which the point cloud is encoded using a method that complies with V-PCC (Video-based Point Cloud Compression). This can prevent an increase in redundancy of track information for a file container that manages the V3C bitstream, and can also prevent an increase in the data volume of the scene description.

[0108] The scene description file may also be written in a format compliant with glTF (The GL Transmission Format) 2.0, which can prevent an increase in redundancy of track information written in a format compliant with glTF (The GL Transmission Format) 2.0, and can also prevent an increase in the data volume of the scene description.

[0109] <When referring to a track with a track reference> When the above-mentioned method 1 is applied, as shown in the second row from the top of the table in FIG. 26, a track that stores the track reference of the file container may be specified as the reference destination (method 1-1).

[0110] For example, the file generator may generate a scene description file containing track information that specifies a single track among multiple tracks in a file container as a reference destination, the track having track references for accessing all other tracks.

[0111] In addition, the track information may be information that specifies a single track among multiple tracks of a file container as a reference destination, the single track having a track reference for accessing all other tracks, and the acquisition unit may refer to the single track based on the track information and acquire data of the 3D object content managed in all tracks based on the track reference.

[0112] In this way, by specifying a track having a track reference as the reference destination, the acquisition unit can acquire the data of the 3D object content more easily than when other tracks are specified as the reference destination.

[0113] <When referring to an adaptation set in a control file> When the above-mentioned method 1 is applied, an adaptation set that manages tracks that store track references of a file container in an MPD (Media Presentation Description) may be specified as the reference destination (method 1-2), as shown in the bottom row of the table in Fig. 26. The MPD is a control file for controlling the distribution of 3D object content data that is distributed in a manner that complies with MPEG-DASH (Moving Picture Experts Group Dynamic Adaptive Streaming over Hypertext Transfer Protocol).

[0114] For example, the file generation unit may generate a scene description file including track information that specifies as a reference an adaptation set that stores information for a single track that has information for accessing all other tracks in a control file for controlling the distribution of 3D object content data.

[0115] In addition, the track information may be information that specifies as a reference an adaptation set that stores information about a single track that has information for accessing all other tracks in a control file for controlling the distribution of 3D object content data, and the acquisition unit may acquire the control file, refer to the single track based on the information about the adaptation set specified by the track information in the control file, and acquire the data of the 3D object content managed in all tracks based on the reference relationship between the tracks.

[0116] By doing this, even when 3D object content data is distributed using a control file, the increase in redundancy of track information can be suppressed, and the increase in the amount of data in the scene description can be suppressed.

[0117] The 3D object content may be distributed in a format conforming to MPEG-DASH (Moving Picture Experts Group Dynamic Adaptive Streaming over Hypertext Transfer Protocol). The control file may be MPD (Media Presentation Description). That is, the present technology may also be applied to the distribution of 3D object content to which MPEG-DASH is applied. Therefore, even when 3D object content is distributed using MPEG-DASH, it is possible to suppress an increase in redundancy of track information and an increase in the amount of data for the scene description.

[0118] <Example> Fig. 27 shows a description example of a scene description (MPEG media object (MPEG_media)) when method 1 described above is applied. Fig. 27 shows a description example when method 1-2 is applied, that is, a description example when an adaptation set of an MPD is specified as the reference destination. For example, on the eighth line from the bottom of this description example, an MPD for controlling distribution of a V3C bitstream is specified by a uri as follows, as information for acquiring a V3C bitstream:

[0119] "uri": "manifest.mpd",

[0120] The portion enclosed by the dotted line in this example contains the following track information that specifies the adaptation set that stores the information of the V3C track to be referenced:

[0121] "tracks": [{ “tracks”:“#track=1” “codecs”: “v3c1” }]

[0122] In other words, even if the ISOBMFF has a multi-track structure, the track information specifies a single track as the reference destination. This prevents the track information from becoming redundant, and also prevents the data volume of the scene description from increasing.

[0123] <File generation device> The above-described method 1 (including method 1-1 and method 1-2) may be applied to any device. FIG. 28 is a block diagram showing an example of the configuration of a file generation device, which is one aspect of an information processing device to which the present technology is applied. The file generation device 100 shown in FIG. 28 is a device that generates information for distributing 3D object content. For example, the file generation device 100 generates a 3D object content file to be distributed and a scene description file (scene description) for the 3D object content.

[0124] Note that Fig. 28 shows the main processing units, data flows, etc., and does not necessarily include everything shown in Fig. 28. In other words, file generation device 100 may have processing units that are not shown as blocks in Fig. 28, or processes or data flows that are not shown as arrows or the like in Fig. 28.

[0125] As shown in FIG. 28 , the file generation device 100 has a control unit 101 and a file generation processing unit 102. The control unit 101 controls the file generation processing unit 102. The file generation processing unit 102 is controlled by the control unit 101 to perform processing related to file generation. For example, the file generation processing unit 102 generates a 3D object content file, which is a content file that stores data of 3D object content to be distributed. The file generation processing unit 102 also generates a scene description file that stores a scene description corresponding to the 3D object content. The file generation processing unit 102 outputs the generated file to the outside of the file generation device 100. For example, the file generation processing unit 102 uploads the generated file to a distribution server or the like.

[0126] The file generation processing unit 102 includes an input unit 111, a preprocessing unit 112, an encoding unit 113, a file generation unit 114, a recording unit 115, and an output unit 116.

[0127] The input unit 111 acquires point cloud data as 3D object content data and supplies it to the pre-processing unit 112. The pre-processing unit 112 acquires information necessary for generating a scene description from the point cloud data, etc. The pre-processing unit 112 supplies the acquired information to the file generation unit 114. The pre-processing unit 112 also supplies the point cloud data to the encoding unit 113.

[0128] The encoding unit 113 encodes the point cloud data supplied from the preprocessing unit 112 and generates the encoded data. The encoding unit 113 supplies the generated encoded data of the point cloud to the file generation unit 114 as a V3C bitstream.

[0129] The file generation unit 114 performs processing related to the generation of files, etc. For example, the file generation unit 114 acquires a V3C bitstream supplied from the encoding unit 113. The file generation unit 114 also acquires information supplied from the pre-processing unit 112. The file generation unit 114 also generates an ISOBMFF, which is a file container that stores the V3C bitstream supplied from the encoding unit 113. The file generation unit 114 also generates a scene description file using the information supplied from the pre-processing unit 112. In this case, the file generation unit 114 may generate track information, etc. using information such as the V3C bitstream and ISOBMFF, and store the information in the scene description file. Furthermore, if the V3C bitstream is distributed in a format compliant with MPEG-DASH, the file generation unit 114 generates an MPD. The file generation unit 114 supplies the generated files, etc. (ISOBMFF, scene description file, MPD, etc.) to the recording unit 115.

[0130] Recording unit 115 has an arbitrary recording medium such as a hard disk or semiconductor memory, and records the files etc. supplied from file generation unit 114 on the recording medium. Furthermore, recording unit 115 reads out the files etc. recorded on the recording medium in accordance with a request from control unit 101 or output unit 116, or at a predetermined timing, and supplies them to output unit 116.

[0131] Output unit 116 acquires the files etc. supplied from recording unit 115, and outputs the files etc. to an external device outside file generation device 100 (for example, a distribution server, a playback device, etc.).

[0132] In the file generation device 100 configured as above, the file generation unit 114 generates files and the like by applying the present technology.

[0133] For example, the file generation unit 114 may apply the above-mentioned method 1 to generate a scene description file including track information that specifies, as a reference, some tracks among multiple tracks of a file container that manages information about 3D object content data, which tracks have information for accessing other tracks.

[0134] By doing so, the file generating device 100 can suppress an increase in redundancy of the track information of the scene description, and can suppress an increase in the data amount of the scene description.

[0135] The track information may also be information that designates as a reference a single track that has information for accessing all other tracks.

[0136] The file container may be an ISOBMFF that applies a multi-track structure. The 3D object content data may be a V3C bitstream in which the point cloud is encoded in a format that complies with V-PCC. The scene description file may be written in a format that complies with glTF2.0.

[0137] The file generation unit 114 may also apply the above-mentioned method 1-1 to generate a scene description file including track information that specifies as a reference a single track among multiple tracks in a file container that has track references for accessing all other tracks.

[0138] In addition, the file generation unit 114 may apply the above-mentioned method 1-2 to generate a scene description file including track information that specifies as a reference an adaptation set that stores information on a single track that has information for accessing all other tracks in a control file for controlling the distribution of 3D object content data.

[0139] In this case, the 3D object content may be distributed in a format conforming to MPEG-DASH, and the control file may be an MPD.

[0140] By applying any one or more of these present techniques, the file generation device 100 can obtain the same effects as those described in <2. Suppression of redundancy in track information>.

[0141] <File generation process flow> An example of the flow of file generation processing executed by file generation device 100 configured as above will be described with reference to the flowchart of FIG.

[0142] When the file generation process starts, in step S101, input unit 111 of file generation device 100 acquires data of a 3D object (3D data). For example, input unit 111 acquires point cloud data as this 3D data.

[0143] In step S102, the preprocessing unit 112 acquires information used to generate a scene description, which is spatial arrangement information for arranging one or more 3D objects in a 3D space, from the 3D object data acquired in step S101. The file generation unit 114 then uses the acquired information to generate a scene description file.

[0144] In step S103, the encoding unit 113 encodes the point cloud data (3D data) acquired in step S101, and generates the encoded data (V3C bitstream).

[0145] In step S104, the file generation unit 114 generates a file container (ISOBMFF) that stores the V3C bitstream generated in step S103. The file generation unit 114 can also generate a control file such as an MPD.

[0146] In step S105, the file generation unit 114 generates track information that specifies, as a reference, some of the tracks that have information on other tracks among the multiple tracks that make up the ISOBMFF, based on the ISOBMFF (or MPD) generated in step S104, and stores the track information in the scene description file generated in step S102.

[0147] In step S106, the recording unit 115 records the generated file container (ISOBMFF), (MPD,) scene description file, etc. on a recording medium.

[0148] In step S107, output unit 116 reads the file or the like recorded in step S106 from the recording medium and outputs the read file to the outside of file generation device 100 at a predetermined timing. For example, output unit 116 may transmit (upload) the file or the like read from the recording medium to another device such as a distribution server or a playback device via a communication medium such as a network. Also, output unit 116 may record the file or the like read from the recording medium on an external recording medium such as removable media. In this case, the output file or the like may be supplied to another device (such as a distribution server or a playback device) via the external recording medium, for example.

[0149] When the process of step S107 ends, the file generation process ends.

[0150] In step S105 of this file generation process, the file generation unit 114 can apply various methods of the present technology as described above in <File Generation Device> of <2. Suppression of Redundancy in Track Information> to generate track information and store it in a scene description file. Therefore, by performing the file generation process in this manner, the file generation device 100 can suppress an increase in redundancy of the track information in the scene description, and can suppress an increase in the data amount of the scene description. Furthermore, the file generation device 100 can obtain the same effects as those described in <2. Suppression of Redundancy in Track Information>.

[0151] <Client device> Fig. 30 is a block diagram showing an example of the configuration of a client device, which is one aspect of an information processing device to which the present technology is applied. The client device 200 shown in Fig. 30 is a playback device that performs playback processing of 3D object content based on a scene description file. For example, the client device 200 plays back 3D object data stored in a 3D object file generated by the file generation device 100. In doing so, the client device 200 performs processing related to the playback based on the scene description.

[0152] Note that Fig. 30 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in client device 200, there may be processing units that are not shown as blocks in Fig. 30, and there may be processing or data flows that are not shown as arrows, etc. in Fig. 30.

[0153] 30, the client device 200 has a control unit 201 and a playback processing unit 202. The control unit 201 performs processing related to control of the playback processing unit 202. The playback processing unit 202 performs processing related to playback of 3D object data.

[0154] The playback processing unit 202 includes a file acquisition unit 211 , a file processing unit 212 , a decoding unit 213 , a display information generation unit 214 , a display unit 215 , and a display control unit 216 .

[0155] The file acquisition unit 211 acquires files and the like supplied from outside the client device 200, such as a distribution server or the file generation device 100. For example, the file acquisition unit 211 acquires a scene description file from outside the client device 200 and supplies it to the file processing unit 212. Furthermore, the file acquisition unit 211 acquires a file container (ISOBMFF) that stores a bitstream of 3D object content in accordance with a request from the file processing unit 212, and supplies it to the file processing unit 212. Furthermore, the file acquisition unit 211 can also acquire a control file (for example, MPD) that controls the distribution of data of the 3D object content in accordance with a request from the file processing unit 212, and supply it to the file processing unit 212.

[0156] The file processing unit 212 acquires files, etc. supplied from the file acquisition unit 211, and performs processing related to the acquired files, etc. For example, the file processing unit 212 acquires a scene description file, etc. supplied from the file acquisition unit 211. Then, the file processing unit 212 performs processing such as controlling the file acquisition unit 211 based on the scene description file. Furthermore, the file processing unit 212 acquires an MPD supplied from the file acquisition unit 211. Then, the file processing unit 212 performs processing such as controlling the file acquisition unit 211 based on the MPD. Furthermore, the file processing unit 212 acquires information on a file container (ISOBMFF) supplied from the file acquisition unit 211. Then, the file processing unit 212 can extract arbitrary information from the ISOBMFF. Then, the file processing unit 212 performs processing such as controlling the file acquisition unit 211 based on the extracted information. Furthermore, the file processing unit 212 acquires a bitstream of 3D object content (e.g., a V3C bitstream) supplied from the file acquisition unit 211.

[0157] The file processing unit 212 supplies the acquired bitstream and the like to the decoding unit 213. The file processing unit 212 also supplies the display control unit 216 with information useful for generating display information contained in the scene description and the like.

[0158] The decoding unit 213 decodes the bitstream supplied from the file processing unit 212. The decoding unit 213 supplies the 3D object content data (for example, point cloud data) obtained by the decoding to the display information generation unit 214.

[0159] The display information generation unit 214 acquires the 3D object data supplied from the decoding unit 213. Furthermore, the display information generation unit 214 renders the data of the 3D object content and generates a display image, etc., under the control of the display control unit 216. The display information generation unit 214 supplies the generated display image, etc. to the display unit 215.

[0160] The display unit 215 has a display device and displays the display image supplied from the display information generation unit 214 using the display device.

[0161] The display control unit 216 acquires information such as the scene description supplied from the file processing unit 212. The display control unit 216 controls the display information generation unit 214 based on this information. In the client device 200 configured as above, the file processing unit 212 acquires a scene description file, (MPD,) 3D object content data, and the like by applying the present technology.

[0162] For example, the file processing unit 212 may apply the above-described method 1 to acquire the 3D object content data managed in all tracks, based on a scene description file including track information that specifies, as reference destinations, some tracks having information for accessing other tracks among multiple tracks of a file container that manages information about 3D object content data, and the reference relationships between the tracks. The file processing unit 212 realizes such processing by controlling the file acquisition unit 211. In other words, the file processing unit 212 can also be called an acquisition unit.

[0163] By doing so, the client device 200 can suppress an increase in redundancy of the track information of the scene description, and can suppress an increase in the data amount of the scene description.

[0164] The track information may also be information that designates as a reference a single track that has information for accessing all other tracks.

[0165] The file container may be an ISOBMFF that applies a multi-track structure. The 3D object content data may be a V3C bitstream in which the point cloud is encoded in a format that complies with V-PCC. The scene description file may be written in a format that complies with glTF2.0.

[0166] Alternatively, the file processing unit 212 may apply the above-described method 1-1, referencing a single track among multiple tracks of a file container based on track information that specifies the single track as a reference destination, the single track having a track reference for accessing all other tracks, and acquiring data of 3D object content managed in all tracks based on the track reference. The file processing unit 212 realizes such processing by controlling the file acquisition unit 211.

[0167] The file processing unit 212 may also apply the above-mentioned method 1-2 to obtain a control file for controlling the distribution of 3D object content data, and based on the information of the adaptation set in the control file that is specified as the reference destination by the track information, refer to a single track that has information for accessing all other tracks, and obtain the 3D object content data managed in all tracks based on the reference relationships between the tracks.

[0168] In this case, the 3D object content may be distributed in a format conforming to MPEG-DASH, and the control file may be an MPD.

[0169] By applying any one or more of these present techniques, the client device 200 can obtain the same effects as those described in <2. Suppression of redundancy of track information>.

[0170] <Client processing flow 1> An example of the flow of client processing executed by client device 200 configured as above will be described with reference to the flowchart of Fig. 31. The flowchart of Fig. 31 is an example of the flow of client processing when method 1-1 is applied.

[0171] When the client process is started, the file processing unit 212 of the client device 200 controls the file acquisition unit 211 in step S201 to acquire a scene description file.

[0172] In step S202, the file processing unit 212 analyzes the scene description file acquired in step S201.

[0173] In step S203, the file processing unit 212 controls the file acquisition unit 211 based on the analysis result of the scene description file obtained by the processing in step S202, and refers to the track specified as the reference destination by the track information of the scene description file.

[0174] In step S204, the file processing unit 212 acquires data of 3D object content managed in another track linked by the track reference stored in the track referenced in step S203.

[0175] In step S205, the decoding unit 213 decodes the data of the 3D object content (the bit stream of encoded data) acquired in step S204.

[0176] In step S206, the display information generation unit 214 arranges the 3D objects in the 3D space based on the scene description file, and performs rendering to generate a display image.

[0177] In step S207, display unit 215 displays the display image generated in step S206. When the process of step S207 ends, the client process ends.

[0178] In steps S201 to S204 of this client processing, the file processing unit 212 applies various methods of the present technology as described above in <Client Device> of <2. Suppression of Redundancy of Track Information> to reference a single track having a track reference for accessing all other tracks based on the track information, and acquires 3D object content data managed in all tracks based on the track reference. Therefore, by performing the client processing in this manner, the client device 200 can suppress an increase in redundancy of the track information of the scene description, thereby suppressing an increase in the data volume of the scene description. Furthermore, the client device 200 can obtain the same effects as those described in <2. Suppression of Redundancy of Track Information>.

[0179] <Client processing flow 2> Next, an example of the flow of client processing when Method 1-2 is applied will be described with reference to the flowchart in FIG.

[0180] When the client process starts, in step S221, the file processing unit 212 of the client device 200 controls the file acquisition unit 211 to acquire a scene description file. In step S222, the file processing unit 212 analyzes the scene description file.

[0181] In step S223, the file processing unit 212 controls the file acquisition unit 211 based on the analysis result of the scene description file obtained by the processing of step S222, and acquires a control file that controls the distribution of 3D object content data, which is specified in the scene description file. In step S224, the file processing unit 212 analyzes the control file.

[0182] In step S225, the file processing unit 212 controls the file acquisition unit 211 based on the analysis result of the control file obtained by the processing of step S224, and references tracks stored in the adaptation set specified as the reference destination by the track information of the scene description file. Then, the file processing unit 212 acquires data of 3D object content managed in all tracks based on the reference relationships between the tracks. For example, the file processing unit 212 acquires information stored in other tracks linked by track references stored in the referenced track. Then, the file processing unit 212 acquires data of 3D object content managed in each track.

[0183] In step S226, the decoding unit 213 decodes the data of the 3D object content (the bit stream of encoded data) acquired in step S225.

[0184] In step S227, the display information generation unit 214 arranges the 3D objects in the 3D space based on the scene description file, and performs rendering to generate a display image.

[0185] In step S228, display unit 215 displays the display image generated in step S227. When the process of step S228 ends, the client process ends.

[0186] In each of steps S221 to S225 of this client processing, the file processing unit 212 applies various methods of the present technology as described above in <Client Device> of <2. Suppression of Redundancy of Track Information> to acquire a control file based on track information, references tracks stored in an adaptation set specified by the track information in the control file, and acquires 3D object content data managed in all tracks based on the reference relationships between the tracks. Therefore, by performing the client processing in this manner, the client device 200 can suppress an increase in redundancy of track information in the scene description, thereby suppressing an increase in the data volume of the scene description. Furthermore, the client device 200 can achieve the same effects as those described in <2. Suppression of Redundancy of Track Information>.

[0187] <3. Specifying attributes> <Data stored in the buffer> Now, let us consider a case where, for example, in a system that provides 3D object content of point clouds using glTF2.0, the V3C bitstream is managed by applying the provisions of MPEG-I Part 14 Scene Description for MPEG Media.

[0188] As shown in FIG. 33, the client device decodes the V3C bitstream (V-PCC data) (V-PCC decode) and reconstructs the point cloud (Point cloud reconstruct).

[0189] As described in Non-Patent Document 2, the presentation engine (PE) and media access function (MAF) of the client device exchange data via a buffer. Therefore, the following two types of data can be stored in the buffer (circular buffer).

[0190] Case 1: Point cloud data after reconstruction (i.e. point cloud data) Case 2: Data after V-PCC decoding (i.e., decoded component video data)

[0191] <Case 1> An example of the configuration of objects in a scene description for Case 1 is shown in Figure 34. In this Case 1, the reconstructed point cloud data (geometry and attributes) is stored in the buffer, so the attribute object (attribute) in mesh.primitives has properties for position information (POSITION) and color information (COLOR_0) set, and each property specifies an accessor to the buffer where the respective information is stored.

[0192] Because the data stored in the buffer is dynamic (time-varying) point cloud data, the object structure used when using glTF2.0 point cloud data is used. Furthermore, functions extended in Non-Patent Document 2 (such as MPEG_accessor_timed and MPEG_buffer_circular) are applied to utilize timed media. MPEG_media specifies V-PCC data (V3C video stream), which is decoded and reconstructed by the Media Access Function (MAF) and stored in the buffer. The Presentation Engine (PE) retrieves the data in the buffer and renders the 3D object content in the same way as it processes glTF2.0 point cloud data.

[0193] The client device has the configuration described with reference to Fig. 19, and plays back 3D object content by executing client processing. Fig. 35 is a flowchart showing an example of the flow of client processing in Case 1. When client processing starts, glTF analysis unit 63 of presentation engine (PE) 51 starts PE processing, and in step S321 obtains SD(glTF) file 62, which is a scene description file, and parses the scene description.

[0194] In step S322, the glTF parser 63 checks the media associated with the 3D object (texture), the buffer in which the media will be stored after processing, and the accessor. In step S323, the glTF parser 63 notifies the media access function (MAF) 52 of this information as a file acquisition request.

[0195] The media access function (MAF) 52 starts MAF processing and receives the notification in step S311. In step S312, the media access function (MAF) 52 receives the media (3D object file (mp4)) based on the notification.

[0196] In step S313, the media access function (MAF) 52 decodes the acquired media (3D object file (mp4)) using a 2D video decoding method. In step S314, the media access function (MAF) 52 reconstructs a point cloud using the video data obtained by decoding. In step S315, the media access function (MAF) 52 stores the reconstructed point cloud data in the buffer 54 based on a notification from the presentation engine (PE) 51.

[0197] In step S324, the rendering processing unit 64 of the presentation engine (PE) 51 reads (acquires) the point cloud data at an appropriate timing from the buffer 54. In step S325, the rendering processing unit 64 performs rendering using the acquired data to generate an image for display.

[0198] The media access function (MAF) 52 repeats the processes of steps S313 to S315, thereby executing these processes for each time (each frame). The presentation engine (PE) 51 repeats the processes of steps S324 and S325, thereby executing these processes for each time (each frame). When the processes for all frames have been completed, the media access function (MAF) 52 ends the MAF process, and the presentation engine (PE) 51 ends the PE process. In other words, the client process ends.

[0199] <Case 2> FIG. 36 shows an example of the configuration of objects in a scene description for Case 2. In this case, the buffer stores decoded component video data, etc. In other words, geometry video frames, attribute video frames, occupancy maps, atlas information, etc. However, existing glTF 2.0 does not provide a way to signal such data. Therefore, in the scene description, as shown in FIG. 36, an extension (MPEG_vpcc) is defined in mesh.primitives, and identification information for accessors to each piece of decoded component video data is stored. That is, an object (vpccattr) is defined in MPEG_vpcc, and properties such as atlas, geometry, attributes, and occupancy are set in the vpccattr, with each property specifying an accessor to the buffer where the corresponding bitstream is stored.

[0200] In MPEG media (MPEG_media), V-PCC data is specified, which is decoded in the media access function (MAF), and the resulting video data (decoded component video data) is stored in a buffer. The presentation engine (PE) retrieves the data in the buffer, uses them to reconstruct a point cloud, and renders it in the same way as in Case 1.

[0201] In case 2, the client device also has the configuration described with reference to Fig. 19, and plays back 3D object content by executing client processing. Fig. 37 is a flowchart showing an example of the flow of client processing in case 2. When the client processing starts, the glTF analysis unit 63 of the presentation engine (PE) 51 starts the PE processing, and in step S341, obtains the SD(glTF) file 62, which is a scene description file, and parses the scene description.

[0202] In step S342, the glTF parser 63 checks the media associated with the 3D object (texture), the buffer in which the media will be stored after processing, and the accessor. In step S343, the glTF parser 63 notifies the media access function (MAF) 52 of this information as a file acquisition request.

[0203] The media access function (MAF) 52 starts MAF processing and receives the notification in step S331. In step S332, the media access function (MAF) 52 receives the media (3D object file (mp4)) based on the notification.

[0204] In step S333, the media access function (MAF) 52 decodes the acquired media (3D object file (mp4)) using a 2D video decoding method. In step S334, the media access function (MAF) 52 stores the decoded video data (decoded component video data) and the like in the buffer 54 based on a notification from the presentation engine (PE) 51.

[0205] In step S344, the rendering processing unit 64 of the presentation engine (PE) 51 reads (acquires) the video data, etc. from the buffer 54 at an appropriate timing. In step S345, the rendering processing unit 64 reconstructs a point cloud using the acquired video data, etc. In step S346, the rendering processing unit 64 performs rendering using the reconstructed point cloud data to generate an image for display.

[0206] The media access function (MAF) 52 repeats the processes of steps S333 and S334 to execute these processes for each time (each frame). The presentation engine (PE) 51 repeats the processes of steps S344 to S346 to execute these processes for each time (each frame). When the processes for all frames are completed, the media access function (MAF) 52 ends the MAF process, and the presentation engine (PE) 51 ends the PE process. In other words, the client process ends.

[0207] <Management of attributes other than color information> As shown in Figure 38, point cloud attributes can include information other than color information. However, the glTF2.0 primitives.attributes property could only handle position information (POSITION) and color information (COLOR). Therefore, if the data obtained by decoding a V3C bitstream contains attribute data other than color information (data of a type that cannot be expressed by the glTF2.0 primitives.attributes property), it can be difficult to correctly reconstruct a point cloud. This can occur in both Case 1 and Case 2 described above.

[0208] Therefore, as shown in the top row of the table in FIG. 39, attribute information other than the color information of the point cloud may be stored in mesh.primitives of the scene description (method 2).

[0209] For example, an information processing device may include a file generating unit that generates a scene description file that describes a scene of 3D object content having an object that stores information about attributes other than the color of the 3D object content.

[0210] For example, a method of processing may generate a scene description file describing a scene of 3D object content having objects that store information about attributes other than color of the 3D object content.

[0211] For example, an information processing device may include an acquisition unit that acquires data on attributes other than the color of 3D object content based on information about the attributes stored in an object defined in a scene description file that describes a scene of the 3D object content.

[0212] For example, in the information processing method, data of an attribute other than the color of the 3D object content may be obtained based on information about the attribute stored in an object defined in a scene description file that describes a scene of the 3D object content.

[0213] By doing this, even if the point cloud data has attributes other than color information, it is possible to correctly reconstruct a 3D object (point cloud). In other words, it is possible to handle attributes other than color information. This attribute other than color information is arbitrary. For example, it may be reflectance, normal vector, or other attributes.

[0214] The 3D object content may be a point cloud, the scene description file may be described in a format compliant with glTF2.0, and an object storing information about an attribute of the 3D object content other than color may be specified in mesh.primitives of the scene description file. The file generator may generate such a scene description file. The acquirer may acquire data about the attribute based on the information about the attribute. This allows the 3D object (point cloud) to be correctly reconstructed in the 3D object content compliant with glTF2.0 (capable of handling attributes other than color information).

[0215] The file generator may also define an extension in the mesh.primitives file and define the object in the extension. The acquirer may acquire the attribute data based on information about the attribute stored in the object defined in the extension in the mesh.primitives file. Applying the extension can minimize degradation of compatibility with existing glTF 2.0.

[0216] <Application example 1 of Case 1> The following describes the above-mentioned case 1, that is, the case where the V3C bitstream is decoded, a point cloud is reconstructed, and the obtained point cloud data is stored in a buffer. When applying the above-mentioned method 1 to this case 1, an object that stores attribute information other than color information may be specified using an extension (method 2-1), as shown in the second row from the top in Fig. 39.

[0217] For example, the object may store identification information of attribute data other than color in the 3D object content data and identification information of an accessor associated with a buffer in which the attribute data is stored. The file generation unit may generate a scene description file in which such an object is defined in an extension in mesh.primitives. The acquisition unit may acquire the attribute data from the buffer based on the information of such an object.

[0218] FIG. 40 is a diagram showing an example of the object configuration in this case. As shown in FIG. 40, a new extension (MPEG_vpcc) is provided in mesh.primitives. An attribute object (attribute) is provided in MPEG_vpcc. An object that stores information on "attributes that cannot be represented in glTF2.0" is defined in the attribute object. In the example of FIG. 40, reflectance is applied as the attribute, and an object "_REFLECTANCE" for that reflectance is defined. In such an attribute-specific object, a property indicating the data to be stored and a property indicating the buffer that stores that data are set. The data to be stored can be specified using any information. For example, the identification information (attributeId(attrId)) of the attribute data in the 3D object content data can be used. In addition, the buffer that stores the attribute data can be specified using any information. For example, the identification information (accessorId (acceId)) of the accessor associated with the buffer may be used.

[0219] Fig. 41 shows an example of a scene description (mesh) in this case. As shown in the dotted line box in Fig. 41, a "_REFLECTANCE" object is defined in MPEG_vpcc.attribute, and values ​​are set in the accessorId and attributeId properties.

[0220] The media access function of the client device stores the data of the attribute specified by this information in a specified buffer, and the presentation engine reads the data of the attribute stored in the specified buffer.

[0221] By doing this, it is possible to correctly reconstruct a 3D object (point cloud) having attributes other than color information even in Case 1. In other words, it is possible to handle attributes other than color information.

[0222] In the example of Fig. 40, the "POSITION" and "COLOR" properties of MPEG_vpcc may be omitted, or may be provided only when necessary.

[0223] When applying the above-mentioned method 1 to case 1, attribute information other than color information may be added to primitives.attributes instead of using extension, as shown in the third row from the top in Figure 39 (method 2-2). For example, the attribute other than color may be reflectance, and a property "_REFLECTANCE" that stores information about that reflectance may be defined in primitives.attributes. This "_REFLECTANCE" property has identification information for the accessor.

[0224] That is, an object that stores information about an attribute other than the color of a 3D object content may further store information about the position of the 3D object content and information about the color of the 3D object content. The file generator may define such an object in mesh.primitives. The acquirer may acquire the attribute data based on the object defined in mesh.primitives.

[0225] <Example of application of Case 2> The following describes the above-mentioned case 2, i.e., when a V3C bitstream is decoded and the resulting video data, etc. is stored in a buffer. When applying the above-mentioned method 1 to this case 2, as shown in the fourth row from the top in Fig. 39, an extension may be used to define an object that stores atlas information, geometry information, attribute information including attributes other than point cloud color, and occupancy information (method 2-3).

[0226] For example, the object may store identification information of an accessor associated with a buffer storing atlas information of the 3D object content, identification information of an accessor associated with a buffer storing the geometry of the 3D object content, identification information of an accessor associated with a buffer storing all attributes of the 3D object content, and identification information of an accessor associated with a buffer storing occupancy information (e.g., an occupancy map) of the 3D object content. The file generation unit may generate a scene description file in which such objects are defined in an extension within mesh.primitives. The acquisition unit may acquire video data, etc. from each buffer based on the information of such objects.

[0227] FIG. 42 is a diagram showing an example of the configuration of objects in this case. As shown in FIG. 42, a new extension (MPEG_vpcc) is provided in mesh.primitives. An attribute object (attribute) is provided in MPEG_vpcc. In the attribute object, a property (_ATLAS) specifying an accessor associated with the buffer in which atlas information is stored is set as a property indicating the buffer in which each piece of data is stored. Similarly, a property (_GEOMETRY) specifying an accessor associated with the buffer in which geometry is stored is set. Similarly, a property (_ATTRIBUTE_0) specifying an accessor associated with the buffer in which all attributes are stored is set. Similarly, a property (_OCCUPANCY) specifying an accessor associated with the buffer in which occupancy information is stored is set. The accessor may be specified using identification information (accessorId(acceId)) of the accessor.

[0228] Fig. 43 is a diagram showing an example of a scene description (mesh) in this case. As shown in the dotted line frame in Fig. 41, in MPEG_vpcc.attribute, the respective values ​​(identification information of the accessor) are set in the properties "_ATLAS", "_GEOMETRY", "_ATTRIBUTE_0", and "_OCCUPANCY".

[0229] As in Method 2-2, instead of using an extension, you can add the properties "_ATLAS", "_GEOMETRY", "_ATTRIBUTE_0", and "_OCCUPANCY" to primitives.attributes.

[0230] The media access function of the client device stores the video data etc. in the buffer specified by this information, and the presentation engine reads the video data etc. stored in the specified buffer.

[0231] By doing this, it is possible to correctly reconstruct a 3D object (point cloud) having attributes other than color information even in Case 2. In other words, it is possible to handle attributes other than color information.

[0232] <Application example 2 of case 1> When applying the above-mentioned method 1 to the above-mentioned case 1, an object that stores general-purpose attribute information may be defined using an extension, as shown in the bottom row of Figure 39 (method 2-4).

[0233] For example, an object may store information identifying data of an attribute other than color in the data of the 3D object content, information identifying an accessor associated with a buffer in which the data of the attribute is stored, and information indicating the type of the attribute. The file generation unit may generate a scene description file in which such an object is defined in an extension in mesh.primitives. The acquisition unit may acquire the data of the attribute from the buffer based on the information of the object.

[0234] FIG. 44 is a diagram showing an example of the configuration of objects in this case. As shown in FIG. 44, a new extension (MPEG_vpcc) is provided in mesh.primitives. An attribute object (attribute) is provided in MPEG_vpcc. A general-purpose object (vpccattributes) that stores information about any attribute is defined in the attribute object. In this general-purpose object (vpccattributes), a property (acceId) indicating the data to be stored, a property (attrId) indicating the buffer that stores the data, and a property (attrType) indicating the type of the data are set. The data to be stored can be specified using any information. For example, identification information (attributeId(attrId)) of the attribute data in the 3D object content data can be used. Furthermore, the buffer that stores the attribute data can be specified using any information. For example, identification information (accessorId(acceId)) of the accessor associated with the buffer can be used. The type of the attribute can be specified using any information, for example, identification information (attributeType) for identifying the type of the attribute.

[0235] Figure 45 is a diagram showing an example of a scene description (mesh) in this case. As shown in the dotted line frame in Figure 45, a general-purpose object "vpccattributes" is defined in MPEG_vpcc.attribute, and this object has properties "accessorId" and "attributeId" as well as a property "attributeType" for specifying the type of attribute. In the example of Figure 45, identification information "ATTR_REFLECTANCE" indicating reflectance is set as the value of this "attributeType".

[0236] The media access function of the client device stores the data of the attribute specified by this information in a specified buffer, and the presentation engine reads the data of the attribute stored in the specified buffer.

[0237] By doing so, it is possible to specify attribute information other than color information using a general-purpose object in the scene description, without the need to provide an object dedicated to attributes.

[0238] In the example of Fig. 44, the "POSITION" and "COLOR" properties of MPEG_vpcc may be omitted, or may be provided only when necessary.

[0239] <File generation device> The above-described method 2 (including methods 2-1 to 2-4) may be applied to any device. For example, it may be applied to a file generation device that generates information for distribution of 3D object content. In this case, the configuration of the file generation device is the same as that described with reference to FIG. 28.

[0240] In the file generation device 100 shown in FIG. 28, the file generation unit 114 generates a file or the like by applying the present technology.

[0241] For example, the file generator 114 may apply the above-described method 2 to generate a scene description file having objects that store information about attributes other than the color of the 3D object content.

[0242] In this way, file generation device 100 enables the client device to correctly reconstruct a 3D object (point cloud) having attributes other than color information, i.e., it can handle attributes other than color information.

[0243] The 3D object content may be a point cloud. The file generation unit 114 may generate a scene description file written in a format compliant with glTF2.0, and define an object that stores information about attributes of the 3D object content other than color in mesh.primitives of the scene description file.

[0244] Furthermore, the file generation unit 114 may define an extension in mesh.primitives, and define an object in the extension that stores information about attributes other than the color of the 3D object content.

[0245] In addition, by applying the above-mentioned method 2-1, the object may store identification information of attribute data in the 3D object content data and identification information of an accessor associated with the buffer in which the attribute data is stored.

[0246] Furthermore, by applying the above-mentioned method 2-4, the object may further store information indicating the type of the attribute in addition to the identification information of the attribute data and the identification information of the accessor.

[0247] Furthermore, by applying the above-mentioned method 2-3, the object may store identification information of an accessor associated with a buffer in which atlas information of the 3D object content is stored, identification information of an accessor associated with a buffer in which the geometry of the 3D object content is stored, identification information of an accessor associated with a buffer in which all attributes of the 3D object content are stored, and identification information of an accessor associated with a buffer in which occupancy information of the 3D object content is stored.

[0248] Alternatively, by applying the above-described method 2-2, the file generation unit 114 may define an object in mesh.primitives that stores information about the position, color, and attributes of the 3D object content.

[0249] By applying any one or more of these techniques of the present invention, the file generation device 100 can obtain the same effects as those described in <3. Attribute specification>.

[0250] <File generation process flow> An example of the flow of file generation processing executed by file generation device 100 configured as above will be described with reference to the flowchart of FIG.

[0251] When the file generation process starts, in step S401, input unit 111 of file generation device 100 acquires data of a 3D object (3D data). For example, input unit 111 acquires point cloud data as this 3D data.

[0252] In step S402, the preprocessing unit 112 acquires information used to generate a scene description, which is spatial arrangement information for arranging one or more 3D objects in 3D space, from the 3D object data acquired in step S401. The file generation unit 114 then uses the acquired information to generate a scene description file.

[0253] In step S403, the encoding unit 113 encodes the point cloud data (3D data) acquired in step S401, and generates the encoded data (V3C bitstream).

[0254] In step S404, the file generation unit 114 generates a file container (ISOBMFF) for storing the V3C bitstream generated in step S403. The file generation unit 114 may also generate a control file such as an MPD.

[0255] In step S405, the file generation unit 114 stores attribute information other than the color information of the point cloud in mesh.primitives of the scene description based on the ISOBMFF (or MPD) generated in step S404.

[0256] In step S406, the recording unit 115 records the generated file container (ISOBMFF), (MPD,) scene description file, etc. on a recording medium.

[0257] In step S407, output unit 116 reads the file or the like recorded in step S406 from the recording medium and outputs the read file to the outside of file generation device 100 at a predetermined timing. For example, output unit 116 may transmit (upload) the file or the like read from the recording medium to another device such as a distribution server or a playback device via a communication medium such as a network. Furthermore, output unit 116 may record the file or the like read from the recording medium on an external recording medium such as removable media. In this case, the output file or the like may be supplied to another device (such as a distribution server or a playback device) via the external recording medium, for example.

[0258] When the process of step S407 ends, the file generation process ends.

[0259] In step S405 of this file generation process, the file generation unit 114 can store attribute information other than the color information of the point cloud in mesh.primitives of the scene description by applying various methods of the present technology as described above in <File Generation Device> of <3. Attribute Designation>. Therefore, by performing the file generation process in this manner, the file generation device 100 can enable the client device to correctly reconstruct a 3D object (point cloud) having attributes other than color information. In other words, it can handle attributes other than color information. Furthermore, the file generation device 100 can achieve the same effects as those described in <3. Attribute Designation>.

[0260] <Client device> Furthermore, the present technology may be applied to a client device that is a playback device that performs playback processing of 3D object content based on a scene description file (scene description). In this case, the configuration of the client device is the same as that described with reference to FIG.

[0261] In the client device 200 shown in FIG. 30, the file processing unit 212 applies the present technology to acquire a scene description file, (MPD,) 3D object content data, and the like.

[0262] For example, the file processing unit 212 may apply the above-described method 2 to acquire data of an attribute other than the color of the 3D object content based on information about the attribute stored in an object defined in a scene description file that describes a scene of the 3D object content. The file processing unit 212 realizes such processing by controlling the file acquisition unit 211. In other words, the file processing unit 212 can also be called an acquisition unit.

[0263] In this way, the client device 200 can correctly reconstruct a 3D object (point cloud) having attributes other than color information, that is, can handle attributes other than color information.

[0264] The 3D object content may be a point cloud, the scene description file may be written in a format compliant with glTF2.0, and the object may be specified in mesh.primitives of the scene description file.

[0265] The object may also be defined within an extension defined within mesh.primitives.

[0266] In addition, by applying the above-mentioned method 2-1, the acquisition unit may acquire the attribute data from the buffer based on the identification information of the attribute data in the 3D object content data stored in the object and the identification information of the accessor associated with the buffer in which the attribute data is stored.

[0267] Alternatively, the above-described method 2-4 may be applied, and the acquisition unit may further acquire the data of the attribute from the buffer based on information indicating the type of the attribute stored in the object.

[0268] Furthermore, by applying the above-mentioned method 2-3, the acquisition unit may acquire the data of the attribute from the buffer based on the identification information of the accessor associated with the buffer in which all attributes of the 3D object content are stored, which are stored in the object together with the identification information of the accessor associated with the buffer in which the atlas information of the 3D object content is stored, the identification information of the accessor associated with the buffer in which the geometry of the 3D object content is stored, and the identification information of the accessor associated with the buffer in which the occupancy information of the 3D object content is stored.

[0269] Alternatively, by applying the above-described method 2-2, the object may further store information about the position of the 3D object content and information about the color of the 3D object content.

[0270] By applying any one or more of these present techniques, the client device 200 can obtain the same effects as those described in <3. Attribute Designation>.

[0271] <Client processing flow 1> An example of the flow of client processing executed by client device 200 configured as above will be described with reference to the flowchart in Fig. 47. The flowchart in Fig. 47 is an example of the flow of client processing in case 1.

[0272] When the client process is started, the file processing unit 212 of the client device 200 controls the file acquisition unit 211 in step S501 to acquire a scene description file.

[0273] In step S502, the file processing unit 212 analyzes the scene description file acquired in step S501.

[0274] In step S503, the file processing unit 212 controls the file acquisition unit 211 based on the analysis result of the scene description file obtained by the processing in step S502, and acquires coded data (V3C bitstream) of the 3D object content.

[0275] In step S504, the decoding unit 213 decodes the coded data (V3C bitstream) acquired in step S503 to generate video data, etc. That is, the decoding unit 213 acquires atlas information, geometry frames, attribute frames, occupancy maps, etc. The decoding unit 213 further reconstructs a point cloud using the acquired data, and generates point cloud data (geometry, attributes, etc.). The decoding unit 213 then stores the acquired point cloud data in a buffer specified by the scene description file.

[0276] In step S505, the display information generation unit 214 acquires point cloud data (geometry, attributes, etc.) from the buffer based on the scene description. That is, the display information generation unit 214 acquires position information, color information, and other attribute information of the point cloud. The display information generation unit 214 places the point cloud (3D object) in three-dimensional space according to the scene description, performs rendering, and generates a display image.

[0277] In step S506, the display unit 215 displays the display image generated in step S505. When the process of step S506 ends, the client process ends.

[0278] In each process of steps S501 to S504 of this client processing, the file processing unit 212 can apply various methods of the present technology as described above in <Client Device> of <3. Attribute Designation> to acquire attribute data based on attribute information other than color information of the point cloud stored in mesh.primitives of the scene description. Therefore, by performing the client processing in this manner, the client device 200 can correctly reconstruct a 3D object (point cloud) having attributes other than color information. In other words, it can handle attributes other than color information. Furthermore, the client device 200 can obtain the same effects as those described in <3. Attribute Designation>.

[0279] <Client processing flow 2> Next, an example of the flow of client processing in case 2 will be described with reference to the flowchart in FIG.

[0280] When the client process starts, the file processing unit 212 of the client device 200 controls the file acquisition unit 211 in step S521 to acquire a scene description file.

[0281] In step S522, the file processing unit 212 analyzes the scene description file acquired in step S521.

[0282] In step S523, the file processing unit 212 controls the file acquisition unit 211 based on the analysis result of the scene description file obtained by the processing in step S522, and acquires the coded data (V3C bitstream) of the 3D object content.

[0283] In step S524, the decoding unit 213 decodes the coded data (V3C bitstream) acquired in step S523 to generate video data, etc. That is, the decoding unit 213 obtains atlas information, geometry frames, attribute frames, occupancy maps, etc. The decoding unit 213 stores the obtained data in a buffer specified by the scene description file.

[0284] In step S525, the decoding unit 213 obtains video data and the like (atlas information, geometry frames, attribute frames, occupancy maps, etc.) from the buffer based on the scene description. The decoding unit 213 reconstructs a point cloud using the video data and generates point cloud data (geometry, attributes, etc.). The decoding unit 213 then supplies the obtained point cloud data to the display information generation unit 214.

[0285] In step S526, the display information generation unit 214 places the point cloud (3D object) in three-dimensional space in accordance with the scene description, and performs rendering to generate a display image.

[0286] In step S527, the display unit 215 displays the display image generated in step S526. When the process of step S527 ends, the client process ends.

[0287] In each process of steps S521 to S525 of this client processing, the file processing unit 212 can apply various methods of the present technology as described above in <Client Device> of <3. Attribute Designation> to acquire attribute data based on attribute information other than color information of the point cloud stored in mesh.primitives of the scene description. Therefore, by performing the client processing in this manner, the client device 200 can correctly reconstruct a 3D object (point cloud) having attributes other than color information. In other words, it can handle attributes other than color information. Furthermore, the client device 200 can obtain the same effects as those described in <3. Attribute Designation>.

[0288] <4. Notes> <Combination> As long as no contradiction occurs, any two or more of the above-described methods may be applied in combination. For example, Method 1 and Method 2 may be applied in combination. Furthermore, any of the above-described methods may be applied in combination with any other method not described above.

[0289] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0290] FIG. 49 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0291] In a computer 900 shown in FIG. 49, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0292] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0293] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0294] In a computer configured as above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0295] The program executed by the computer can be applied by recording it on removable media 921 such as package media, for example. In this case, the program can be installed in storage unit 913 via input / output interface 910 by inserting removable media 921 into drive 915.

[0296] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0297] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0298] <Applicable targets of this technology> This technology can be applied to any encoding / decoding method.

[0299] Furthermore, the present technology can be applied to any configuration, for example, various electronic devices.

[0300] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0301] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, AV (Audio Visual) equipment, a portable information processing terminal, or an IoT (Internet of Things) device.

[0302] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0303] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications thereof are also arbitrary.

[0304] For example, the present technology can be applied to systems and devices used to provide viewing content, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for transportation, such as monitoring traffic conditions and controlling automatic driving. Furthermore, for example, the present technology can also be applied to systems and devices used for security. Furthermore, for example, the present technology can also be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can also be applied to systems and devices used to monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife. Furthermore, for example, the present technology can also be applied to systems and devices used for sports.

[0305] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be assumed not only to include the identification information in the bit stream, but also to include difference information of the identification information relative to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information relative to the reference information.

[0306] Furthermore, various types of information (metadata, etc.) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. In other words, data associated with each other may be combined into one piece of data or may be individual pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer to only a portion of the data, rather than the entire data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0307] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0308] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0309] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0310] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and is able to obtain the necessary information.

[0311] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0312] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0313] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0314] The present technology can also be configured as follows. (1) A file generating unit that generates a scene description file that describes a scene of the 3D object content, the scene description file including track information that specifies, as a reference destination, some tracks that have information for accessing other tracks among a plurality of tracks of a file container that manages information about data of the 3D object content. An information processing device comprising: (2) The track information is information that designates a single track that has information for accessing all of the other tracks as the reference destination. The information processing device described in (1). (3) The track information is information that designates, as the reference destination, the single track among the plurality of tracks of the file container, which has track references for accessing all of the other tracks. (2) An information processing device according to the present invention. (4) The track information is information that specifies, as the reference destination, an adaptation set that stores information of the single track in a control file for controlling distribution of data of the 3D object content. (2) An information processing device according to the present invention. (5) The 3D object content is distributed in a manner compliant with MPEG-DASH (Moving Picture Experts Group Dynamic Adaptive Streaming over Hypertext Transfer Protocol), The control file is an MPD (Media Presentation Description). (4) An information processing device according to the present invention. (6) The file container is an ISOBMFF (International Organization for Standardization Base Media File Format) that applies a multi-track structure. An information processing device according to any one of (1) to (5). (7) The data of the 3D object content is a V3C (Visual Volumetric Video-based Coding) bitstream in which the point cloud is encoded in a format conforming to V-PCC (Video-based Point Cloud Compression). An information processing device according to any one of (1) to (6). (8) The scene description file is written in a format that complies with glTF (The GL Transmission Format) 2.0. An information processing device according to any one of (1) to (7). (9) A scene description file is generated that describes a scene of the 3D object content, the scene description file including track information that specifies, as a reference destination, some tracks that have information for accessing other tracks among a plurality of tracks of a file container that manages information about data of the 3D object content. Information processing methods.

[0315] (11) An acquisition unit that acquires a scene description file that describes a scene of the 3D object content, including track information that specifies, as a reference destination, some tracks having information for accessing other tracks among a plurality of tracks of a file container that manages information about data of the 3D object content, and acquires data of the 3D object content managed in all the tracks based on the reference relationships between the tracks. An information processing device comprising: (12) The track information is information that designates a single track having information for accessing all of the other tracks as the reference destination. (11) An information processing device according to (11). (13) The track information is information that designates, as the reference destination, the single track among the plurality of tracks of the file container, which has track references for accessing all of the other tracks, The acquisition unit refers to the single track based on the track information, and acquires data of the 3D object content managed in all the tracks based on the track reference. (12) An information processing device according to (12). (14) The track information is information that specifies, as the reference destination, an adaptation set that stores information of the single track in a control file for controlling distribution of data of the 3D object content, The acquisition unit acquires the control file, references the single track based on information of the adaptation set specified by the track information of the control file, and acquires data of the 3D object content managed in all of the tracks based on the reference relationship between the tracks. (12) An information processing device according to (12). (15) The 3D object content is distributed in a manner compliant with MPEG-DASH (Moving Picture Experts Group Dynamic Adaptive Streaming over Hypertext Transfer Protocol), The control file is an MPD (Media Presentation Description). (14) An information processing device according to (14). (16) The file container is an ISOBMFF (International Organization for Standardization Base Media File Format) that applies a multi-track structure. An information processing device according to any one of (11) to (15). (17) The data of the 3D object content is a V3C (Visual Volumetric Video-based Coding) bitstream in which the point cloud is encoded in a format conforming to V-PCC (Video-based Point Cloud Compression). An information processing device according to any one of (11) to (16). (18) The scene description file is written in a format that complies with glTF (The GL Transmission Format) 2.0. An information processing device according to any one of (11) to (17). (19) Acquire a scene description file describing a scene of the 3D object content, including track information specifying, as a reference destination, some tracks having information for accessing other tracks among a plurality of tracks of a file container that manage information about data of the 3D object content, and the data of the 3D object content managed in all the tracks based on the reference relationships between the tracks. Information processing methods.

[0316] (21) A file generating unit that generates a scene description file that describes a scene of the 3D object content, the scene description file having an object that stores information about an attribute other than the color of the 3D object content. An information processing device comprising: (22) The 3D object content is a point cloud; The file generation unit generating the scene description file described in a format compliant with glTF (The GL Transmission Format) 2.0; The object is defined in mesh.primitives of the scene description file. (21) An information processing device according to (21). (23) The file generation unit defines an extension in the mesh.primitives and defines the object in the extension. (22) An information processing device according to (22). (24) The object stores identification information of the attribute data in the 3D object content data and identification information of an accessor associated with a buffer in which the attribute data is stored. (23) An information processing device according to (23). (25) The object further stores information indicating the type of the attribute. (24) An information processing device according to (24). (26) The object stores identification information of an accessor associated with a buffer in which atlas information of the 3D object content is stored, identification information of an accessor associated with a buffer in which geometry of the 3D object content is stored, identification information of an accessor associated with a buffer in which all attributes of the 3D object content are stored, and identification information of an accessor associated with a buffer in which occupancy information of the 3D object content is stored. (23) An information processing device according to (23). (27) The file generation unit defines the object in the mesh.primitives, the object storing information about the position of the 3D object content, information about the color, and information about the attributes. (22) An information processing device according to (22). (28) generating a scene description file describing a scene of the 3D object content, the scene description file having objects that store information about attributes other than the color of the 3D object content; Information processing methods.

[0317] (31) An acquisition unit that acquires data of an attribute other than a color of a 3D object content based on information about the attribute stored in an object defined in a scene description file that describes a scene of the 3D object content. An information processing device comprising: (32) The 3D object content is a point cloud; The scene description file is written in a format that complies with glTF (The GL Transmission Format) 2.0, The object is defined in mesh.primitives of the scene description file. (31) An information processing device according to (31). (33) The object is defined within an extension defined within the mesh.primitives. (32) An information processing device according to (32). (34) The acquisition unit acquires the attribute data from the buffer based on identification information of the attribute data in the data of the 3D object content stored in the object and identification information of an accessor associated with the buffer in which the attribute data is stored. (33) An information processing device according to (33). (35) The acquisition unit further acquires data of the attribute from the buffer based on information indicating a type of the attribute stored in the object. (34) An information processing device according to (34). (36) The acquisition unit acquires data from the buffer based on identification information of an accessor associated with a buffer in which atlas information of the 3D object content is stored, identification information of an accessor associated with a buffer in which geometry of the 3D object content is stored, and identification information of an accessor associated with a buffer in which all attributes of the 3D object content are stored in the object together with identification information of an accessor associated with a buffer in which occupancy information of the 3D object content is stored. (33) An information processing device according to (33). (37) The object further stores information about the position of the 3D object content and information about the color of the 3D object content. (32) An information processing device according to (32). (38) Obtaining data of an attribute other than the color of a 3D object content based on information about the attribute stored in an object defined in a scene description file that describes a scene of the 3D object content. Information processing methods. [Explanation of symbols]

[0318] 100 file generation device, 101 control unit, 102 file generation processing unit, 111 input unit, 112 preprocessing unit, 113 encoding unit, 114 file generation unit, 115 recording unit, 116 output unit, 200 client device, 201 control unit, 202 client processing unit, 211 file acquisition unit, 212 file processing unit, 213 decoding unit, 214 display information generation unit, 215 display unit, 216 display control unit

Claims

1. a file generating unit that generates a scene description file that describes a scene of the 3D object content, the scene description file including track information that specifies, as a reference destination, some tracks having information for accessing other tracks among a plurality of tracks of a file container that manages information related to data of the 3D object content; An information processing device comprising:

2. The track information is information that designates a single track that has information for accessing all of the other tracks as the reference destination. The information processing device according to claim 1 .

3. The track information is information that specifies, as the reference destination, the single track that has track references for accessing all the other tracks among the plurality of tracks of the file container. The information processing device according to claim 2 .

4. The track information is information that specifies, as the reference destination, an adaptation set that stores information of the single track in a control file for controlling distribution of data of the 3D object content. The information processing device according to claim 2 .

5. The 3D object content is distributed in a manner compliant with MPEG-DASH (Moving Picture Experts Group Dynamic Adaptive Streaming over Hypertext Transfer Protocol), The control file is an MPD (Media Presentation Description) The information processing device according to claim 4 .

6. The file container is the ISOBMFF (International Organization for Standardization Base Media File Format) that applies a multi-track structure. The information processing device according to claim 1 .

7. The 3D object content data is a V3C (Visual Volumetric Video-based Coding) bitstream in which the point cloud is encoded in a format conforming to V-PCC (Video-based Point Cloud Compression). The information processing device according to claim 1 .

8. The scene description file is written in a format that complies with glTF (The GL Transmission Format) 2.

0. The information processing device according to claim 1 .

9. A scene description file is generated that describes a scene of the 3D object content, the scene description file including track information that specifies, as a reference destination, some tracks having information for accessing other tracks among a plurality of tracks of a file container that manages information related to data of the 3D object content. Information processing methods.

10. a scene description file describing a scene of the 3D object content, the file including track information specifying, as a reference destination, some tracks having information for accessing other tracks among a plurality of tracks of a file container that manages information relating to data of the 3D object content; and an acquisition unit that acquires data of the 3D object content managed in all of the tracks based on the reference relationships between the tracks. An information processing device comprising:

11. The track information is information that designates a single track that has information for accessing all of the other tracks as the reference destination. The information processing device according to claim 10.

12. the track information is information that specifies, as the reference destination, the single track among the plurality of tracks of the file container, which has track references for accessing all of the other tracks; The acquisition unit refers to the single track based on the track information, and acquires data of the 3D object content managed in all the tracks based on the track reference. The information processing device according to claim 11.

13. the track information is information that specifies, as the reference destination, an adaptation set that stores information of the single track in a control file for controlling distribution of data of the 3D object content; The acquisition unit acquires the control file, references the single track based on information of the adaptation set specified by the track information of the control file, and acquires data of the 3D object content managed in all of the tracks based on the reference relationship between the tracks. The information processing device according to claim 11.

14. The 3D object content is distributed in a manner compliant with MPEG-DASH (Moving Picture Experts Group Dynamic Adaptive Streaming over Hypertext Transfer Protocol), The control file is an MPD (Media Presentation Description) The information processing device according to claim 13.

15. The file container is the ISOBMFF (International Organization for Standardization Base Media File Format) that applies a multi-track structure. The information processing device according to claim 10.

16. The 3D object content data is a V3C (Visual Volumetric Video-based Coding) bitstream in which the point cloud is encoded in a format conforming to V-PCC (Video-based Point Cloud Compression). The information processing device according to claim 10.

17. The scene description file is written in a format that complies with glTF (The GL Transmission Format) 2.

0. The information processing device according to claim 10.

18. A scene description file describing a scene of the 3D object content, including track information specifying, as a reference destination, some tracks having information for accessing other tracks among a plurality of tracks of a file container that manages information relating to data of the 3D object content, and data of the 3D object content managed in all the tracks based on the reference relationships between the tracks. Information processing methods.

Images