Information processing apparatus and method

Abstract

The present disclosure relates to an information processing device and a method, with each of which it is possible to suppress an increase in processing load when random access is made to a scene description. Generated as update information for updating space arrangement information for arranging at least one 3D object in a 3D space is initial value reference difference information, in which the initial value of the space arrangement information serves as a reference, or post-update information, in which an update to initial value reference difference information is reflected in the initial value of the space arrangement information. An update file for storing the update information is generated, and the initial value reference difference information or the post-update information is stored, as a random access point, in the update file. The present disclosure can be applied to, for example, an information processing device or an information processing method.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and method, and more particularly, to an information processing apparatus and method capable of suppressing an increase in processing load when randomly accessing a scene description.

Background Art

[0002] Conventionally, there has been glTF (The GL Transmission Format) (registered trademark) 2.0, which is a format for arranging 3D (three-dimensional) objects in a three-dimensional space (see, for example, Non-Patent Document 1).

[0003] In recent years, in MPEG (Moving Picture Experts Group)-I Scene Description, in order to extend glTF2.0 and update the scene description, it has been considered to send difference information from the previous state as a JSON patch (see, for example, Non-Patent Documents 2 to 4).

[0004] Also, in MPEG-I Scene Description, it has been considered to store a JSON patch, which is update information for the scene description, in ISOBMFF (International Organization for Standardization Base Media File Format). As a method therefor, studies have been advanced to use an extension based on the Web Resource in ISOBMFF standard (see, for example, Non-Patent Documents 3, 5, and 6).

[0005] Furthermore, the transmission of the updated scene description has also been considered, and it has been considered to use the scene description for random access (see, for example, Non-Patent Documents 2 and 3).

Prior Art Documents

Non-Patent Literature

[0006]

Non-Patent Literature 1

Non-Patent Literature 2

Non-Patent Literature 3

Non-Patent Literature 4

Non-Patent Literature 5

[0007] However, in the conventional method, there is a risk that the processing load of the client device that randomly accesses the scene description may increase.

[0008] The present disclosure has been made in view of such a situation, and aims to suppress an increase in the processing load when randomly accessing the scene description. [Means for Solving the Problems]

[0009] An information processing apparatus according to one aspect of the present technology includes an update information generation unit that generates, as update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, initial value reference difference information that is difference information based on an initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, and a file generation unit that generates an update file for storing the update information and stores the initial value reference difference information or the updated information in the update file as a random access point.

[0010] An information processing method according to one aspect of the present technology is initial value reference difference information, which is difference information based on an initial value of the spatial arrangement information, as update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, generates an update file that stores the update information, and stores the initial value reference difference information or the updated information as a random access point in the update file.

[0011] An information processing apparatus according to another aspect of the present technology is an update information acquisition unit that acquires, as the update information, initial value reference difference information, which is difference information based on an initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, stored as a random access point in an update file that stores update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, and a spatial arrangement information generation unit that generates the spatial arrangement information at the update application time of the initial value reference difference information by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information or applying the updated information.

[0012] An information processing method according to another aspect of the present technology acquires, as the update information, initial value reference difference information, which is difference information based on an initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, stored as a random access point in an update file that stores update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, and generates the spatial arrangement information at the update application time of the initial value reference difference information by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information or applying the updated information.

[0013] In an information processing apparatus and method according to one aspect of the present technology, as update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, initial value reference difference information that is difference information based on the initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information is generated, an update file for storing the update information is generated, and the initial value reference difference information or the updated information is stored in the update file as a random access point.

[0014] In an information processing apparatus and method according to another aspect of the present technology, the initial value reference difference information that is difference information based on the initial value of the spatial arrangement information, or the updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, which is stored as a random access point in an update file for storing update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space, is acquired as update information, and the spatial arrangement information at the time of applying the update of the initial value reference difference information is generated by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information or by applying the updated information.

Brief Description of the Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Embodiments for Carrying Out the Invention

[0016] Hereinafter, embodiments for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The description will be made in the following order. 1. MPEG-I Scene Description 2. Transmission of Initial Value Reference Difference Information 3. Transmission of Initial Value Reference Difference Information or Updated Information 4. First Embodiment (File Generation Device) 5. Second Embodiment (Client Device) 6. Supplementary Note

[0017] <1.MPEG-I Scene Description> <Literature etc. Supporting Technical Content and Technical Terms> The scope disclosed by this technology includes not only the content described in the embodiments, but also the content described in the following non-patent documents etc. that were publicly known at the time of filing, and the content of other documents referred to in the following non-patent documents etc.

[0018] Non-Patent Document 1: (Above-mentioned) Non-Patent Document 2: (Above-mentioned) Non-Patent Document 3: (Above-mentioned) Non-Patent Document 4: (Above-mentioned) Non-Patent Document 5: (Above-mentioned) Non-Patent Document 6: (Above-mentioned)

[0019] That is, the content described in the above non-patent documents and the content of other documents referred to in the above non-patent documents etc. also serve as the basis for judging the support requirements. For example, even if the syntax and terms such as glTF2.0 and its extensions described in Non-Patent Documents 1 to 3 are not directly defined in this disclosure, they are within the scope of this disclosure and are considered to meet the support requirements of the claims. Also, for example, for technical terms such as Parsing, Syntax, Semantics, etc., even if they are not directly defined in this disclosure, they are within the scope of this disclosure and are considered to meet the support requirements of the claims.

[0020] <gltf2.0> Conventionally, for example, as described in Non-Patent Document 1, there was glTF (The GL Transmission Format) (registered trademark) 2.0, which is a format for arranging 3D (three-dimensional) objects in a three-dimensional space. In glTF 2.0, for example, as shown in FIG. 1, it is composed of a JSON format file (.glTF), a binary file (.bin), and an image file (.png,.jpg, etc.). The binary file stores binary data such as geometry and animation. The image file stores data such as textures.

[0021] The JSON format file is a scene description file described in JSON (JavaScript (registered trademark) Object Notation). The scene description is metadata that describes the scene (description) of 3D content. By describing this scene description, what kind of scene is defined. The scene description file is a file that stores such a scene description. In the present disclosure, the scene description file is also referred to as a scene description file.

[0022] The description of the JSON format file is composed of a list of key-value pairs. An example of its format is shown below. “KEY”:”VALUE”

[0023] The key is composed of a string. The value is composed of a numerical value, a string, a boolean value, an array, an object, or null, etc.

[0024] Also, a plurality of key-value pairs (“KEY”:”VALUE”) can be grouped using {}(curly brackets). What is grouped by these curly brackets is also referred to as a JSON object. An example of its format is shown below. “user”:{"id":1, "name":"tanaka”}

[0025] In this example, as the values corresponding to the key (user), a JSON object combining the pair of "id": 1 and the pair of "name": "tanaka" is defined.

[0026] Also, zero or more values can be arrayed using [ ] (square brackets). This array is also referred to as a JSON array. As an element of this JSON array, for example, a JSON object can also be applied. An example of its format is shown below. test: ["hoge", "fuga", "bar"] "users": [{"id": 1, "name": "tanaka"}, {"id": 2, "name": "yamada"}, {"id": 3, "name": "sato"}]

[0027] The glTF object that can be described at the top level of the JSON format file (glTF object) and the reference relationships they can have are shown in Figure 2. The ovals in the tree structure shown in Figure 2 indicate objects, and the arrows between those objects indicate reference relationships. As shown in Figure 2, objects such as "scene", "node", "mesh", "camera", "skin", "material", "texture", etc. are described at the top level of the JSON format file.

[0028] An example of the description of such a JSON format file (scene description) is shown in FIG. 3. The JSON format file 20 in FIG. 3 shows some of the top-level description examples. In this JSON format file 20, the top-level objects 21 used are all described at the top level. This top-level object 21 is the glTF object shown in FIG. 2. Also, in the JSON format file 20, as shown by the arrow 22, the reference relationship between objects is shown. More specifically, the reference relationship is shown by specifying the index of an element of an array of the object to be referenced in the property of the upper-level object.

[0029] FIG. 4 is a diagram for explaining a method of accessing binary data. As shown in FIG. 4, the binary data is stored in a buffer object. That is, information (such as a URI (Uniform Resource Identifier), etc.) for accessing the binary data in the buffer object is shown. In the JSON format file, as shown in FIG. 4, for example, from objects such as a mesh, a camera, and a skin, it is possible to access the buffer object via an accessor object and a bufferView object.

[0030] That is, for objects such as mesh, camera, and skin, the referenced accessor object is specified. An example of the description of the mesh object in the JSON format file is shown in FIG. 5. For example, as shown in FIG. 5, in the mesh object, vertex attributes such as NORMAL, POSITION, TANGENT, and TEXCORD_0 are defined as keys, and for each attribute, the referenced accessor object is specified as the value.

[0031] The relationship between the buffer object, buffer view object, and accessor object is shown in FIG. 6. Also, examples of the description of these objects in the JSON format file are shown in FIG. 7.

[0032] In FIG. 6, the buffer object 41 is an object that stores information (such as URI) for accessing the binary data that is the actual data and information indicating the data length (for example, byte length) of the binary data. A in FIG. 7 shows an example of the description of the buffer object 41. The "bytelength": 102040 shown in A of FIG. 7 indicates that the byte length of the buffer object 41 is 102040 bytes, as shown in FIG. 6. Also, the "uri": "duck.bin" shown in A of FIG. 7 indicates that the URI of the buffer object 41 is "duck.bin", as shown in FIG. 6.

[0033] In FIG. 6, the buffer view object 42 is an object that stores information about a subset area of the binary data specified in the buffer object 41 (that is, information about a part of the buffer object 41). B in FIG. 7 shows a description example of the buffer view object 42. As shown in FIG. 6 and B in FIG. 7, the buffer view object 42 stores, for example, identification information of the buffer object 41 to which the buffer view object 42 belongs, an offset (for example, a byte offset) indicating the position of the buffer view object 42 within the buffer object 41, a length (for example, a byte length) indicating the data length (for example, a byte length) of the buffer view object 42, and other information.

[0034] As shown in B of FIG. 7, when there are multiple buffer view objects, information is described for each of the buffer view objects (that is, for each subset area). For example, the information such as ""buffer":0", ""bytelength":25272", ""byteOffset":0" shown on the upper side in B of FIG. 7 is the information of the first buffer view object 42 (bufferView[0]) shown in the buffer object 41 in FIG. 6. Also, the information such as ""buffer":0", ""bytelength":76768", ""byteOffset":25272" shown on the lower side in B of FIG. 7 is the information of the second buffer view object 42 (bufferView[1]) shown in the buffer object 41 in FIG. 6.

[0035] The "buffer":0 of the first buffer view object 42 (bufferView[0]) shown at B in FIG. 7 indicates that, as shown in FIG. 6, the identification information of the buffer object 41 to which the buffer view object 42 (bufferView[0]) belongs is "0" (Buffer[0]). Also, the "bytelength":25272 indicates that the byte length of the buffer view object 42 (bufferView[0]) is 25272 bytes. Further, the "byteOffset":0 indicates that the byte offset of the buffer view object 42 (bufferView[0]) is 0 bytes.

[0036] The "buffer":0 of the second buffer view object 42 (bufferView[1]) shown at B in FIG. 7 indicates that, as shown in FIG. 6, the identification information of the buffer object 41 to which the buffer view object 42 (bufferView[0]) belongs is "0" (Buffer[0]). Also, the "bytelength":76768 indicates that the byte length of the buffer view object 42 (bufferView[0]) is 76768 bytes. Further, the "byteOffset":25272 indicates that the byte offset of the buffer view object 42 (bufferView[0]) is 25272 bytes.

[0037] In FIG. 6, the accessor object 43 is an object that stores information regarding the method of interpreting the data of the buffer view object 42. C in FIG. 7 shows an example description of the accessor object 43. As shown in FIG. 6 and C in FIG. 7, the accessor object 43 stores information such as, for example, the identification information of the buffer view object 42 to which the accessor object 43 belongs, the offset (e.g., byte offset) indicating the position of the buffer view object 42 within the buffer object 41, the component type of the buffer view object 42, the number of data stored in the buffer view object 42, and the type of the data stored in the buffer view object 42. Such information is described for each buffer view object.

[0038] In the example of C in FIG. 7, information such as "bufferView":0, "byteOffset":0, "componentType":5126, "count":2106, "type":"VEC3" is shown. As shown in FIG. 6, "bufferView":0 indicates that the identification information of the buffer view object 42 to which the accessor object 43 belongs is "0" (bufferView[0]). Also, "byteOffset":0 indicates that the byte offset of the buffer view object 42 (bufferView[0]) is 0 bytes. Furthermore, "componentType":5126 indicates that the component type is of the FLOAT type (OpenGL macro constant). Also, "count":2106 indicates that there are 2106 pieces of data stored in the buffer view object 42 (bufferView[0]). Furthermore, "type":"VEC3" indicates that the data (type) stored in the buffer view object 42 (bufferView[0]) is a three-dimensional vector.

[0039] Access to data other than an image is all defined by reference to this accessor object 43 (by specifying the index of the accessor).

[0040] <Client Processing> Next, the processing of the client device in the MPEG-I Scene Description will be described. The client device acquires a scene description, acquires 3D object data based on the scene description, and generates a display image using the scene description and the 3D object data.

[0041] As described in Non-Patent Document 2, in the client device, a presentation engine, a media access function, etc. perform processing. For example, as shown in FIG. 8, the presentation engine 51 of the client device 50 acquires the initial value of the scene description and information for updating the scene description (hereinafter also referred to as update information), and generates a scene description at the processing target time. Then, the presentation engine 51 analyzes the scene description and identifies the media (video, audio, etc.) to be played. Then, the presentation engine 51 requests the media access function 52 to acquire the media via the media access API (Application Program Interface). The presentation engine 51 also performs settings for pipeline processing, specification of buffers, etc.

[0042] The media access function 52 acquires various data of the media requested by the presentation engine 51 from the cloud, local storage, etc. The media access function 52 supplies the acquired various data (encoded data) of the media to the pipeline 53.

[0043] Pipeline 53 decodes various data (encoded data) of the supplied media through pipeline processing and supplies the decoding result to buffer 54. Buffer 54 holds various data of the supplied media.

[0044] Presentation engine 51 performs rendering and the like using various data of the media held in buffer 54.

[0045] <Update of Scene Description> A scene description is spatial arrangement information for arranging one or more 3D objects in a 3D space. This scene description can update its content along the time axis. That is, the arrangement of 3D objects can be updated as time passes.

[0046] As described in Non-Patent Document 2 and Non-Patent Document 3, in MPEG-I Scene Description, such an update of the scene description is performed using a JSON patch. That is, as shown in FIG. 9, update information 61 of the scene description at each time is described as a JSON patch and provided to the client device. For example, the hash value of the scene description to be updated, timing information at which the update is executed, and the like are described in this JSON patch. The JSON patch is sent using Web Resouce in ISOBMFF (International Organization for Standardization Base Media File Format) described in Non-Patent Document 3 and Non-Patent Document 6.

[0047] In the SD (glTF) file 62 where the initial value of the scene description is stored, link information to such update information 61 is stored. For example, in the MPEG_dynamic_scene extension within glTF, a URL (Uniform Resource Locator) where an ISOBMFF containing update information 61 (JSON patch) is distributed is described.

[0048] In FIG. 9, the glTF analysis unit 63 of the presentation engine 51 acquires the update information 61 corresponding to the processing target time via the link information of this SD (glTF file) 62. Then, the glTF analysis unit 63 updates the scene description using the update information 61 and generates a scene description corresponding to the processing target time.

[0049] The glTF analysis unit 63 analyzes the generated scene description corresponding to the processing target time and requests the media access function 52 to acquire the necessary files. Then, as described above, the media access function 52 acquires the files, the pipeline 53 decodes them, and the buffer 54 holds the decoding results.

[0050] The rendering processing unit 64 of the presentation engine 51 performs rendering using the media data held in the buffer 54 to generate a display image. In this way, the client device 50 can generate a display image based on the scene description corresponding to the processing target time.

[0051] As described in Non-Patent Document 5, a JSON patch represents operations such as addition, update, and deletion to a JSON document, and the JSON patch itself is also represented as a JSON document. For example, as shown in FIG. 10, when a JSON patch 72 is applied to the original JSON document 71, the JSON document after application becomes like the JSON document 73.

[0052] As described in Non-Patent Document 2 and Non-Patent Document 3, in MPEG-I Scene Description, an ISOBMFF including an original SD file (Original SD file), which is an initial scene description before update, and a JSON patch storing difference information from the previous update as a sample is distributed. The client device calculates a unique value such as a hash code (also referred to as a transaction ID) from the scene description currently expanded in the memory. The JSON patch document supplies the client device with the transaction ID of the scene description to which the update is applied and the difference information from that scene description. When the transaction ID calculated from the currently expanded scene description matches the transaction ID of the JSON patch, the client device applies the update of the JSON patch.

[0053] For example, assume that Scene Description 81 is played. This Scene Description 81 can change in the time direction. Scene Description 81-0 shown in FIG. 11 is the initial value of Scene Description 81. Also, assume that the transaction ID of Scene Description 81-0 is V.1. And Scene Description 81-0 arranges 3D objects a, b, and c in the 3D space.

[0054] The update information for this scene description 81 is supplied to the client device as samples of JSON patches, such as JSON patch 82-1 to JSON patch 82-9. The update information for the scene description 81, such as JSON patch 82-1 to JSON patch 82-9, is referred to as JSON patch 82. JSON patch 82-1 to JSON patch 82-9 are each one of the samples of JSON patch 82. The samples of JSON patch 82 are stored in a track (for example, Track #1) that stores the update information of ISOBMFF and supplied to the client device.

[0055] JSON patch 82-1 is update information (Add A) to add object A to the scene description with a transaction ID of V.1. Therefore, when the client device applies the sample of JSON patch 82-1 to scene description 81-0, scene description 81-1 is obtained. That is, scene description 81-1 arranges objects a, b, c, and A in the 3D space. Also, assume that the transaction ID of scene description 81-1 is V.11.

[0056] JSON patch 82-2 is update information (Add B) to add object B to the scene description with a transaction ID of V.11. Therefore, when the client device applies the sample of JSON patch 82-2 to scene description 81-1, scene description 81-2 is obtained. That is, scene description 81-2 arranges objects a, b, c, A, and B in the 3D space. Also, assume that the transaction ID of scene description 81-2 is V.12.

[0057] JSON patch 82-3 is update information (Add C) that adds object C to the scene description with a transaction ID of V.12. Therefore, when the client device applies a sample of JSON patch 82-3 to scene description 81-2, scene description 81-3 is obtained. That is, scene description 81-3 arranges objects a, b, c, A, B, and C in 3D space. Also, assume that the transaction ID of scene description 81-3 is V.13.

[0058] JSON patch 82-4 is update information (Del B) that deletes object B from the scene description with a transaction ID of V.13. Therefore, when the client device applies a sample of JSON patch 82-4 to scene description 81-3, scene description 81-4 is obtained. That is, scene description 81-4 arranges objects a, b, c, A, and C in 3D space. Also, assume that the transaction ID of scene description 81-4 is V.14.

[0059] JSON patch 82-5 is update information (Del a) that deletes object a from the scene description with a transaction ID of V.14. Therefore, when the client device applies a sample of JSON patch 82-5 to scene description 81-4, scene description 81-5 is obtained. That is, scene description 81-5 arranges objects b, c, A, and C in 3D space. Also, assume that the transaction ID of scene description 81-5 is V.15.

[0060] JSON patch 82-6 is update information (Del b) that deletes object b for the scene description with transaction ID V.15. Therefore, when the client device applies a sample of JSON patch 82-6 to scene description 81-5, scene description 81-6 is obtained. That is, scene description 81-6 arranges objects c, A, and C in 3D space. Also, assume that the transaction ID of scene description 81-6 is V.16.

[0061] Similarly, samples of JSON patches 82-7 to 82-9 are sequentially applied, and scene description 81 is updated. In the case of normal playback where the sequence is sequentially played from the beginning along the time axis, each sample of JSON patch 82 of track #1 is applied at its respective timing in order from the sample of JSON patch 82-1. That is, scene description 81 is updated at each timing, and its content changes along the time axis as scene description 81-0, scene description 81-1, scene description 81-2, ···.

[0062] Also, the update of this scene description 81 can be accessed in a so-called random access manner. That is, the client device can start from an update at an intermediate timing without performing the update of scene description 81 along the time series. For this purpose, a random access point that can be accessed without following the time series is provided, and as data of the random access point, an updated scene description (also referred to as post-update information in this specification) is prepared. As shown in FIG. 12, this post-update information is stored in a track (Track #2) different from the track (Track #1) where the above-mentioned JSON patch 82 is stored.

[0063] In FIG. 12, scene description 83-1 and scene description 83-2 are updated information prepared as random access points. Updated information prepared as a random access point, such as scene description 83-1 and scene description 83-2, is referred to as scene description 83. That is, scene description 83-1 and scene description 83-2 are one of the samples of scene description 83.

[0064] The sample of scene description 83-1 corresponds to the sample of JSON patch 82-6. That is, the content of scene description 83-1 is equivalent to the result of applying the updates of JSON patches 82-1 to 82-6 to scene description 81-0 (scene description 81-6 in FIG. 11). That is, when the client device randomly accesses the sample of scene description 83-1, scene description 81-6 in FIG. 11 can be obtained without applying the updates of JSON patches 82-1 to 82-6. That is, random access becomes possible. When performing normal playback thereafter, the sample of JSON patch 82-7 is applied to scene description 83-1.

[0065] The sample of scene description 83-2 corresponds to the sample of JSON patch 82-9. That is, the content of scene description 83-2 is equivalent to the result of applying the updates of JSON patches 82-1 to 82-9 to scene description 81-0. That is, when the client device randomly accesses the sample of scene description 83-2, the results of those updates can be obtained without applying the updates of JSON patches 82-1 to 82-9. That is, random access becomes possible. When performing normal playback thereafter, the next sample of JSON patch 82 is applied to scene description 83-2.

[0066] However, with such a method, there was a risk of increasing the processing load on the client device that randomly accesses the scene description. That is, since the above-described scene description 83 (updated information) is the scene description itself (the entire scene description), the data volume is large. Also, among the contents of the scene description 83, the information that has not been updated from the scene description 81-0 becomes redundant. Therefore, there was a risk of increasing the processing amount, processing time, buffer amount, etc. of the client device during random access.

[0067] <2. Transmission of Initial-Value Based Difference Information> <Method 1> Therefore, as a random access point, update information (also referred to as initial-value based difference information) indicating the difference from the initial value of the scene description up to that timing is prepared, and when the client device performs random access, the initial-value based difference information is provided. That is, as shown in the uppermost row of the table shown in FIG. 13, the initial-value based difference information is stored in the file as a random access point (Method 1).

[0068] For example, in an information processing device that generates a file for storing update information of a scene description, as update information for updating spatial arrangement information (that is, a scene description) for arranging one or more 3D objects in a 3D space, an update information generation unit that generates initial-value based difference information, which is difference information based on the initial value of the spatial arrangement information, and a file generation unit that generates an update file for storing the update information and stores the initial-value based difference information in the update file as a random access point are provided.

[0069] Also, for example, in an information processing method for generating a file for storing update information of a scene description, as update information for updating spatial arrangement information (i.e., scene description) for arranging one or more 3D objects in a 3D space, initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information, is generated, an update file for storing the update information is generated, and the initial value reference difference information is stored in the update file as a random access point.

[0070] For example, in an information processing apparatus that acquires a file for storing update information of a scene description and updates the scene description, a difference information acquisition unit that acquires initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information (i.e., scene description) stored as a random access point in an update file for storing update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space, and a spatial arrangement information generation unit that generates spatial arrangement information at the update application time of the initial value reference difference information by reflecting the update of the initial value reference difference information on the initial value of the spatial arrangement information are provided.

[0071] Also, for example, in an information processing method that acquires a file for storing update information of a scene description and updates the scene description, initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information (i.e., scene description) stored as a random access point in an update file for storing update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space, is acquired, and spatial arrangement information at the update application time of the initial value reference difference information is generated by reflecting the update of the initial value reference difference information on the initial value of the spatial arrangement information.

[0072] The initial value reference difference information shows the difference between the initial value of the spatial arrangement information (scene description) and the spatial arrangement information (updated information) at the timing of its random access point. That is, the initial value reference difference information can be provided as a JSON patch. For example, in FIG. 12, when a sample of the scene description 83-1 is used as the random access point and the initial value reference difference information is stored as the data of that random access point, the initial value reference difference information shows the difference (Del a, Del b, Add A, Add C) between the scene description 81-0 and the scene description 83-1.

[0073] Therefore, the amount of data provided to the client device can be reduced compared to the case of providing a sample of the scene description 83-1. Also, if the client device applies the sample of the initial value reference difference information to the scene description 81-0, that is, by one update, the scene description 83 (that is, the scene description 83-1) at the timing of that sample can be obtained. Therefore, an increase in the processing amount, processing time, buffer amount, etc. of the client device during random access can be suppressed. That is, an increase in the processing load of the client device that randomly accesses the scene description can be suppressed.

[0074] Note that during random access, samples of all JSON patches 82 up to that random access point are provided to the client device, and the client device updates the scene description 81 using those JSON patches 82, whereby the scene description 83 (updated information) at the timing of that random access point can be obtained. For example, in the case of FIG. 12, samples of each of the JSON patches 82-1 to JSON patches 82-6 are provided to the client device that has randomly accessed them, and the client device updates the scene description 81-0 using them, whereby the scene description 83-1 (equivalent scene description 81-6) is obtained.

[0075] However, in the case of this method, a plurality of JSON patches 82 need to be obtained, and the scene description 81 needs to be updated multiple times. This makes the process complicated and there is a risk that the processing load on the client device will increase. Also, if redundant processing is included in those multiple updates, there is a risk that the processing load on the client device will increase unnecessarily. For example, in the case of FIG. 12, updates using the JSON patch 82-2 (Add B) and the JSON patch 82-4 (Del B) are unnecessary. As described above, when updating the scene description 81-0 using the initial value-based difference information, the client device can obtain the scene description at the timing of the random access point more efficiently.

[0076] <Method 1-1> When applying such Method 1, as shown in the second row from the top of the table shown in FIG. 13, the initial value-based difference information may be stored in a track different from the immediately preceding value-based difference information (Method 1-1). Note that the immediately preceding value-based difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. That is, the immediately preceding value-based difference information is update information (JSON patch) indicating the difference from the scene description in which the updates up to the immediately preceding JSON patch are reflected. In the example of FIG. 12, the JSON patch 82 corresponds to the immediately preceding value-based difference information. Samples of this immediately preceding value-based difference information (JSON patch 82) are applied in the normal playback of the scene description 81 as described above.

[0077] For example, the file generation unit may store the initial value-based difference information in a track different from the immediately preceding value-based difference information of the update file. Also, the difference information acquisition unit may acquire the initial value-based difference information stored in a track different from the immediately preceding value-based difference information of the update file.

[0078] Fig. 14 shows an example in that case. In the example of Fig. 14, initial value reference difference information is prepared as data of the random access point. In Fig. 14, JSON Patch 101-1 and JSON Patch 101-2 are the initial value reference difference information prepared as the random access point. The initial value reference difference information prepared as the random access point like JSON Patch 101-1 and JSON Patch 101-2 is referred to as JSON Patch 101. That is, JSON Patch 101-1 and JSON Patch 101-2 are each one of the samples of JSON Patch 101.

[0079] In the case of this Method 1-1, as shown in Fig. 14, the sample of JSON Patch 101 (initial value reference difference information) is stored in a track (Track #2) different from the track (Track #1) where the above-mentioned JSON Patch 82 is stored. Therefore, the client device can more easily obtain the sample of JSON Patch 101 (initial value reference difference information) independently of the sample of JSON Patch 82 by specifying the track.

[0080] Note that in the example of Fig. 14, the sample of JSON Patch 101 (initial value reference difference information) may be difference information based on a scene description other than the scene description 81-0 (that is, the initial value of the scene description 81). In other words, for the sample of JSON Patch 101 (initial value reference difference information), as long as the scene description at the timing of the sample can be obtained by the update using the JSON Patch 101, the scene description used as the reference for the difference is arbitrary.

[0081] For example, separately from the scene description 81-0, a scene description for random access may be prepared and stored, for example, in track #2. Then, each JSON patch 101 (initial value-based difference information) may indicate the difference from the scene description for random access. For example, the scene description 81-5 (FIG. 11) may be used as the scene description for random access. In this case, the content of the JSON patch 101-1 (FIG. 14) is equivalent to the JSON patch 82-6. That is, by doing so, the data amount of the JSON patch 101-1 can be reduced. The overall data amount depends on the number of random access points, update content, etc. However, for example, after considering these, the scene description for random access points may be set so that the overall data amount is reduced.

[0082] <Method 1-1-1> When applying such a method 1-1, as shown in the third row from the top of the table shown in FIG. 13, the link information to the track storing the sample of the initial value-based difference information may be stored as the link information for random access in the initial value of the scene description (SD) (Method 1-1-1).

[0083] For example, the file generation unit may generate an initial value file storing the initial value of the spatial arrangement information, and store, as the link information for random access, the link information indicating the link to the track storing the initial value-based difference information in the initial value file. Further, the difference information acquisition unit may acquire the initial value-based difference information using the link information indicating the link to the track storing the initial value-based difference information, which is stored as the link information for random access in the initial value file storing the initial value of the spatial arrangement information.

[0084] Fig. 15 shows a partial description example of the scene description 81-0 in that case. As shown in Fig. 15, in the scene description 81-0 (Scene.gltf), apart from the link information ("uri": scen_updata.json-patch) to the track storing the samples of the previous value-based difference information, the link information ("uri_for_randomaccess": scene_updata_type2.json-patch}) to the track storing the samples of the initial value-based difference information is described. The link information to the track storing the samples of the previous value-based difference information is the link information used during normal playback. In contrast, the link information to the track storing the samples of the initial value-based difference information is the link information used during random access. That is, in the example of Fig. 15, the link information is described in a state where these two can be distinguished. That is, the link information to the track storing the samples of the initial value-based difference information is stored in the SD initial value as the link information for random access. By doing so, the client device can more easily identify the track in which the appropriate update information is stored. For example, the client device can more easily select the link information for obtaining the samples of the previous value-based difference information during normal playback. Also, the client device can more easily select the link information for obtaining the samples of the initial value-based difference information during random access.

[0085] <Method 1-2> Also, when applying Method 1, as shown in the fourth row from the top of the table shown in Fig. 13, the initial value-based difference information may be stored in the same sample of the same track as the previous value-based difference information (Method 1-2). Note that the previous value-based difference information is the difference information based on the spatial arrangement information reflecting the updates up to the previous update information, as described above in <Method 1-1>.

[0086] For example, the file generation unit may store the initial value-based difference information in the same sample of the same track as the immediately preceding value-based difference information of the updated file. Further, the difference information acquisition unit may acquire the initial value-based difference information stored in the same sample of the same track as the immediately preceding value-based difference information of the updated file.

[0087] An example in that case is shown in FIG. 16. In the example of FIG. 16, similar to the case of FIG. 14, the initial value-based difference information (JSON Patch 101-1 and JSON Patch 101-2) is prepared as the data of the random access point.

[0088] However, in the case of this Method 1-2, the JSON Patch 101 (initial value-based difference information) is stored in the same sample of the same track (Track #1) as the above-described JSON Patch 82. For example, JSON Patch 101-1 is stored in the same sample 111-1 as JSON Patch 82-6. The timings at which JSON Patch 101-1 and JSON Patch 82-6 are applied for update are the same as each other, and the update results obtained by applying each of them are also the same as each other. Further, JSON Patch 101-2 is stored in the same sample 111-2 as JSON Patch 82-9. The timings at which JSON Patch 101-2 and JSON Patch 82-9 are applied for update are the same as each other, and the update results obtained by applying each of them are also the same as each other.

[0089] By doing so, the number of tracks of the updated file can be made one. Therefore, the client device can obtain update information (immediately preceding value-based difference information or initial value-based difference information) using the same link information both during normal playback and during random access.

[0090] <Method 1-2-1> When applying such a method 1-2, the samples of the random access points are grouped with the immediately preceding value reference difference information and the initial value reference difference information as one sample. Therefore, the immediately preceding value reference difference information and the initial value reference difference information may be stored so that the client device can use them separately. That is, as shown in the fifth row from the top of the table shown in FIG. 13, they may be stored as a list with each of the immediately preceding value reference difference information and the initial value reference difference information as an element (Method 1-2-1).

[0091] For example, the file generation unit may store the immediately preceding value reference difference information and the initial value reference difference information as lists with each as an element. Also, the immediately preceding value reference difference information and the initial value reference difference information may be stored in the sample as lists with each as an element.

[0092] For example, in the case of FIG. 17, the immediately preceding value reference difference information and the initial value reference difference information are each delimited by [] and described as elements of the list. That is, the first element ([{"op": “remove", "path": "”, "value": ””}]) described in the third row from the top is the immediately preceding value reference difference information (in the case of sample 111-1 in FIG. 16, JSON patch 82-6). In contrast, the second element (the following description) described in the fourth to seventh rows from the top is the initial value reference difference information (in the case of sample 111-1 in FIG. 16, JSON patch 101-1).

[0093] [{"op": “remove", "path": "”, "value": ””}, {"op": “remove", "path": "”, "value": ””}, {"op": “add", "path": "”, "value": ””}, {"op": “add", "path": "”, "value": ””}]

[0094] For example, the client device acquires a sample (the entire list). In the case of normal playback, it uses the first element of the list as update information (difference information based on the previous value). In the case of random access, it uses the second element of the list as update information (difference information based on the initial value). In this way, the client device can more easily acquire difference information based on the previous value and difference information based on the initial value.

[0095] <Method 1-2-2> Also, when applying Method 1-2, as shown in the sixth row from the top of the table shown in FIG. 13, in the management area of the file, difference information based on the previous value and difference information based on the initial value may be managed as sub-samples respectively (Method 1-2-2).

[0096] For example, the file generation unit may store in the management area of the update file management information that manages difference information based on the previous value and difference information based on the initial value as sub-samples respectively. Also, the difference information acquisition unit may acquire difference information based on the initial value based on the management information that manages difference information based on the previous value and difference information based on the initial value as sub-samples respectively, which is stored in the management area of the update file.

[0097] For example, as described in Non-Patent Document 5, in the sub-sample information box of ISOBMFF (Sub sample information box), the data within the sample can be divided into sub-samples, and the data size (subsample_size) of each sub-sample can be stored. Thereby, the client device can acquire the data of the sub-sample, which is the data in the middle, without interpreting the data within the sample from the beginning.

[0098] That is, by storing management information that manages each of the immediately preceding value reference difference information and the initial value reference difference information as sub-samples in the management area of the update file, the client device can obtain the immediately preceding value reference difference information or the initial value reference difference information from the sample without parsing inside the sample based on the management information. That is, the client device can more easily obtain the immediately preceding value reference difference information and the initial value reference difference information.

[0099] <Method 1-2-2-1> When applying this Method 1-2-2, as shown in the seventh row from the top of the table shown in FIG. 13, identification information for identifying the sub-sample in which the immediately preceding value reference difference information is stored and the sub-sample in which the initial value reference difference information is stored may be stored in the management area of the file (Method 1-2-2-1).

[0100] For example, the management information that manages each of the immediately preceding value reference difference information and the initial value reference difference information as sub-samples may include identification information for identifying the sub-sample in which the immediately preceding value reference difference information is stored and the sub-sample in which the initial value reference difference information is stored.

[0101] For example, as described in Non-Patent Document 5, the Sub sample information box of ISOBMFF can store codec_specific_parameters. That codec_specific_parameters may be used as identification information for identifying the sub-sample in which the immediately preceding value reference difference information is stored and the sub-sample in which the initial value reference difference information is stored.

[0102] An example of the semantics of codec-specific parameters in that case is shown in FIG. 19. In the example of FIG. 19, when the codec-specific parameter is false (codec_specific_parameters = 0), it indicates that the sub-picture is previous-value-based difference information, and when the codec-specific parameter is true (codec_specific_parameters = 1), it indicates that the sub-picture is initial-value-based difference information. By referring to the value of this codec-specific parameter, the client device can easily identify (without parsing inside the sample) whether the sub-sample stores previous-value-based difference information or initial-value-based difference information.

[0103] <Method 1-2-3> Also, when applying Method 1-2, as shown in the eighth row from the top of the table shown in FIG. 13, in the management area of the file, the samples storing the initial-value-based difference information may be managed as randomly accessible samples (Method 1-2-3).

[0104] For example, the file generation unit may store in the management area of the update file the management information for managing the samples storing the initial-value-based difference information as randomly accessible samples. Also, the difference information acquisition unit may acquire the initial-value-based difference information based on the management information stored in the management area of the update file for managing the samples storing the initial-value-based difference information as randomly accessible samples.

[0105] For example, in the SyncSampleBox of ISOBMFF, a sample in which initial value reference difference information is stored is described as a SyncSample. This SyncSample is a randomly accessible sample. FIG. 18 is a diagram showing a description example of the SyncSampleBox in that case. In this way, by describing a sample in which initial value reference difference information is stored in the SyncSampleBox (SyncSampleBox), the client device can easily identify a randomly accessible sample (without parsing the inside of the sample) by referring to the SyncSampleBox. That is, the client device can more easily identify a sample in which initial value reference difference information is stored based on this information.

[0106] <Method 1-3> Also, when applying Method 1, as shown in the 9th row from the top of the table shown in FIG. 13, the initial value reference difference information may be stored in a different sample on the same track as the immediately preceding value reference difference information (Method 1-3). Note that the immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected, as described above in <Method 1-1>.

[0107] For example, the file generation unit may store the initial value reference difference information in a different sample on the same track as the immediately preceding value reference difference information of the update file. Also, the difference information acquisition unit may acquire the initial value reference difference information stored in a different sample on the same track as the immediately preceding value reference difference information of the update file.

[0108] An example in that case is shown in FIG. 20. In the example of FIG. 20, similar to the case of FIG. 14, the initial value reference difference information (JSON Patch 101-1 and JSON Patch 101-2) is prepared as the data of the random access point.

[0109] However, in the case of this Method 1-3, the JSON patch 101 (initial value-based difference information) is stored in different samples of the same track (Track #1) as the JSON patch 82 with the same update application timing as that JSON patch 101. For example, the JSON patch 101-1 is stored in the sample next to the sample in which the JSON patch 82-6 is stored in the same track (Track #1) as the JSON patch 82-6. The JSON patch 101-1 and the JSON patch 82-6 have the same update application timing, and the update results applied to each are also the same. Also, the JSON patch 101-2 is stored in the sample next to the sample in which the JSON patch 82-9 is stored in the same track (Track #1) as the JSON patch 82-9. The JSON patch 101-2 and the JSON patch 82-9 have the same update application timing, and the update results applied to each are also the same.

[0110] By doing so, the track of the update file can be made into one. Therefore, the client device can obtain update information (immediate previous value-based difference information or initial value-based difference information) using the same link information both during normal playback and during random access. Also, since either immediate previous value-based difference information or initial value-based difference information is stored in the sample (since immediate previous value-based difference information and initial value-based difference information are not stored in the same sample), the client device can select either immediate previous value-based difference information or initial value-based difference information by selecting a sample.

[0111] <Method 1-3-1> When applying such Method 1-3, as shown in the 10th row from the top of the table shown in FIG. 13, the update application time information that is the same as that of the sample in which the immediate previous value-based difference information is stored may be stored in the sample in which the initial value-based difference information to which the update is applied at the same timing as that immediate previous value-based difference information is stored (Method 1-3-1).

[0112] For example, the file generation unit may store the same update application time information as the sample in which the immediately preceding value-based difference information is stored in the sample in which the initial value-based difference information is stored. Also, the sample in which the initial value-based difference information is stored may store the same update application time information as the sample in which the immediately preceding value-based difference information is stored.

[0113] As described in Non-Patent Document 3, by extending the JSON patch and storing, as time information specifying the time when an update is executed, a time other than the presentation time included in the sample of the JSON patch, it is possible to store, for example, a version ID (version_id), an event ID (event_id), an absolute time in Coordinated Universal Time (absolute_time_UTC(Coordinated Universal Time)), an absolute time in International Atomic Time (absolute_time_TAI(International Atomic Time)), etc. in the JSON patch as this time information. Of course, the time information is arbitrary and may be time information other than these examples.

[0114] By storing such time information specifying the time when such an update is executed in the sample, the sample in which the initial value-based difference information is stored is made to store the same time information as the sample in which the immediately preceding value-based difference information with the same update application timing as that initial value-based difference information is stored. For example, in the case of FIG. 20, the sample in which the initial value-based difference information is stored is made to store the same time information as the immediately preceding sample. For example, the same time information is stored in the sample of JSON patch 82-6 and the sample of JSON patch 101-1. Also, the same time information is stored in the sample of JSON patch 82-9 and the sample of JSON patch 101-2.

[0115] By doing so, the client device can easily identify the immediately preceding value-based difference information corresponding to the initial value-based difference information (with the same update application timing as the initial value-based difference information) without parsing the inside of the sample.

[0116] <Method 1-3-2> Also, when applying Method 1-3, as shown in the 11th row from the top of the table shown in FIG. 13, in the management area of the file, the samples in which the initial value reference difference information is stored may be managed as samples not used during normal playback (Method 1-3-2).

[0117] For example, the file generation unit may store, in the management area of the update file, management information for managing the samples in which the initial value reference difference information is stored as samples not used during normal playback. Also, the difference information acquisition unit may acquire the initial value reference difference information based on the management information stored in the management area of the update file for managing the samples in which the initial value reference difference information is stored as samples not used during normal playback.

[0118] By managing, in the management area of the update file for storing the update information, the samples in which the initial value reference difference information is stored as samples not used during normal playback, the client device can easily omit the acquisition of the samples in which the initial value reference difference information is stored without parsing the inside of the samples during normal playback. That is, when sequentially acquiring the samples of the update file as normal playback, the client device can easily refrain from acquiring the samples in which the initial value reference difference information is stored based on this management information and acquire only the samples in which the immediately preceding value reference difference information is stored.

[0119] In other words, the client device can easily acquire the samples in which the initial value reference difference information is stored without parsing the inside of the samples during random access. That is, when performing random access, the client device can easily acquire the samples in which the initial value reference difference information is stored based on this management information.

[0120] <Method 1-3-2-1> When applying such a method 1-3-2, as shown in the 12th row from the top of the table shown in FIG. 13, identification information indicating whether a sample is a sample used during normal playback may be stored in the management area of the file for each sample (Method 1-3-2-1).

[0121] For example, management information that manages samples in which initial value reference difference information is stored as samples not used during normal playback may include identification information indicating whether it is a sample used during normal playback.

[0122] For example, as described in Non-Patent Document 5, a sample_has_redundancy flag indicating the redundancy of a sample can be stored in a SampleDependencyTypeBox defined in ISOBMFF. When the value of the sample_has_redundancy flag is "1" (sample_has_redundancy flag = 1), it indicates that the sample contains redundant information with other samples. Also, when the value of the sample_has_redundancy flag is "2" (sample_has_redundancy flag = 2), it indicates that the sample does not contain redundant information with other samples. Further, when the value of the sample_has_redundancy flag is "0" (sample_has_redundancy flag = 0), it indicates that it is unknown whether the sample contains redundant information with other samples.

[0123] A description example of the SampleDependencyTypeBox is shown in FIG. 21. As shown in FIG. 21, in this SampleDependencyTypeBox, a sample_has_redundancy flag can be stored for each sample.

[0124] Such a sample_has_redundancy flag may be used as identification information indicating whether a sample is a sample to be used during normal playback. That is, in the sample dependency type box, for a sample in which initial value-based difference information is stored, a sample_has_redundancy flag with a value of "1" (sample_has_redundancy flag = 1) is stored. That is, it is shown that this sample contains redundant information with other samples. The client device omits obtaining a sample with a value of "1" for this sample_has_redundancy flag during normal playback. By doing so, the client device can easily omit obtaining a sample in which initial value-based difference information is stored without parsing the inside of the sample during normal playback. That is, when sequentially obtaining samples of the update file as normal playback, the client device can easily omit obtaining a sample in which initial value-based difference information is stored based on this sample_has_redundancy flag, and obtain only a sample in which previous value-based difference information is stored.

[0125] <Method 1-3-3> Also, when applying Method 1-3, as shown in the 13th row from the top of the table shown in FIG. 13, in the management area of the file, a sample in which initial value-based difference information is stored may be managed as a randomly accessible sample (Method 1-3-3).

[0126] For example, the file generation unit may store management information for managing a sample in which initial value-based difference information is stored as a randomly accessible sample in the management area of the update file. Also, the difference information acquisition unit may acquire initial value-based difference information based on management information stored in the management area of the update file for managing a sample in which initial value-based difference information is stored as a randomly accessible sample.

[0127] For example, similar to the case of <Method 1-2-3>, the sample in which the initial value reference difference information is stored is described as a sync sample in the sync sample box (SyncSampleBox). By doing so, the client device can easily identify a randomly accessible sample (without parsing the inside of the sample) by referring to the sync sample box. That is, the client device can more easily identify the sample in which the initial value reference difference information is stored based on this information.

[0128] <Method 1-4> When applying Method 1, as shown in the 14th row from the top of the table shown in FIG. 13, the initial value reference difference information may be replaced with the immediately preceding value reference difference information (Method 1-4). Note that the immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected, as described above in <Method 1-1>.

[0129] For example, the file generation unit may replace the immediately preceding value reference difference information whose update application time is the same as the initial value reference difference information and is stored in the update file with the initial value reference difference information. Also, the difference information acquisition unit may acquire the initial value reference difference information in which the immediately preceding value reference difference information whose update application time is the same as the initial value reference difference information and is stored in the update file is replaced.

[0130] An example of that case is shown in FIG. 22. In the example of FIG. 22, the JSON patch 101 (JSON patch 101-1 and JSON patch 101-2) is the initial value reference difference information and is the data of the random access point. In the case of this Method 1-4, similar to the cases of Method 1-2 and Method 1-3, the JSON patch 101 is stored in the same track (Track #1) as the sample of the JSON patch 82 which is the immediately preceding value reference difference information. However, in the case of this Method 1-4, the sample of the JSON patch 101 is replaced with the sample of the JSON patch 82 corresponding to the JSON patch 101 (with the same update application timing).

[0131] For example, in the example of FIG. 20, JSON patch 101-1 has the same update application timing as JSON patch 82-6. In the example of FIG. 22, the sample of JSON patch 101-1 has been replaced with the sample of that JSON patch 82-6. That is, the sample of JSON patch 82-6 has been deleted, and instead, the sample of JSON patch 101-1 has been placed.

[0132] Similarly, in the example of FIG. 20, JSON patch 101-2 has the same update application timing as JSON patch 82-9. In the example of FIG. 22, the sample of JSON patch 101-2 has been replaced with the sample of that JSON patch 82-9. That is, the sample of JSON patch 82-9 has been deleted, and instead, the sample of JSON patch 101-2 has been placed.

[0133] Therefore, in the case of normal playback, the scene description of the update application timing of samples other than the samples at the random access points is generated using the immediately preceding value-based difference information (JSON patch 82) and the scene description at the timing immediately before that. On the other hand, the scene description of the update application timing of the samples at the random access points is generated using the initial value-based difference information (JSON patch 101) and the initial value of the scene description (scene description 81-0) both in the case of normal playback and in the case of random access.

[0134] By doing so, the data amount of the update file can be reduced compared to the cases of Method 1-2 and Method 1-3.

[0135] Note that in the case of this Method 1-4, since there is no immediately preceding value-based difference information with the same initial value-based difference information and update application timing as in the case of Method 1-3, the sample dropout redundancy flag is not necessary (can be omitted).

[0136] <Method 1-4-1> Also, when applying Method 1-4, as shown in the bottom row of the table shown in FIG. 13, in the file management area, a sample in which initial value reference difference information is stored may be managed as a randomly accessible sample (Method 1-3-3).

[0137] For example, the file generation unit may store in the management area of the update file management information for managing a sample in which initial value reference difference information is stored as a randomly accessible sample. Also, the difference information acquisition unit may acquire the initial value reference difference information based on the management information stored in the management area of the update file for managing a sample in which initial value reference difference information is stored as the randomly accessible sample.

[0138] For example, similar to the case of <Method 1-2-3>, a sample in which initial value reference difference information is stored is described as a sync sample in a sync sample box (SyncSampleBox). By doing so, the client device can easily identify a randomly accessible sample (without parsing the inside of the sample) by referring to the sync sample box. That is, the client device can more easily identify a sample in which initial value reference difference information is stored based on this information.

[0139] <3. Transmission of Initial Value Reference Difference Information or Updated Information> <Method 2> Also, prepare the initial value reference difference information or the updated information as a random access point, and provide the initial value reference difference information or the updated information when the client device performs random access. That is, as shown in the top row of the table shown in FIG. 23, store the initial value reference difference information or the updated information in the file as a random access point (Method 2).

[0140] The initial - value - based difference information is difference information based on the initial value of the scene description, as described above in <2. Transmission of Initial - value - based Difference Information>. The updated information is a scene description in which the update of each immediately - preceding - value - based difference information from the initial value to the sample to be processed is reflected. In other words, the updated information is a scene description in which the update of the initial - value - based difference information is reflected with respect to the initial value of the scene description.

[0141] For example, in an information - processing apparatus that generates a file for storing update information of a scene description, as update information for updating spatial arrangement information (i.e., scene description) for arranging one or more 3D objects in a 3D space, there is provided an update - information generation unit that generates initial - value - based difference information, which is difference information based on the initial value of the spatial arrangement information, or updated information in which the update of the initial - value - based difference information is reflected in the initial value of the spatial arrangement information, and a file - generation unit that generates an update file for storing the update information and stores the initial - value - based difference information or the updated information as a random - access point in the update file.

[0142] Also, for example, in an information - processing method for generating a file for storing update information of a scene description, as update information for updating spatial arrangement information (i.e., scene description) for arranging one or more 3D objects in a 3D space, initial - value - based difference information, which is difference information based on the initial value of the spatial arrangement information, or updated information in which the update of the initial - value - based difference information is reflected in the initial value of the spatial arrangement information is generated, an update file for storing the update information is generated, and the initial - value - based difference information or the updated information is stored in the update file as a random - access point.

[0143] For example, in an information processing apparatus that acquires a file storing update information for a scene description and updates the scene description, initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information (i.e., the scene description) for arranging one or more 3D objects in a 3D space and is stored as a random access point in an update file storing update information for updating the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information is acquired as update information, and a spatial arrangement information generation unit that generates spatial arrangement information at the time of applying the update of the initial value reference difference information by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information or by applying the updated information is provided.

[0144] Also, for example, in an information processing method that acquires a file storing update information for a scene description and updates the scene description, initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information (i.e., the scene description) for arranging one or more 3D objects in a 3D space and is stored as a random access point in an update file storing update information for updating the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information is acquired as update information, and the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information or the updated information is applied to generate spatial arrangement information at the time of applying the update of the initial value reference difference information.

[0145] When the initial value reference difference information is stored in the file as a random access point, it is the same as in the case of Method 1. That is, an increase in the processing load of the client device that randomly accesses the scene description can be suppressed.

[0146] By the way, in this case, for example, in a case where almost all of the scene description is updated, the amount of information of the initial value-based difference information becomes almost the same as that of the scene description. That is, in this case, compared with the example of FIG. 12, the amount of information transmitted to the client device during random access remains almost the same, but the update process performed in the client device increases.

[0147] On the other hand, when the updated information is stored in the file as a random access point, the update process of the scene description is not performed in the client device during random access. Therefore, in this case, as described above, in a case where almost all of the scene description is updated, compared with the example of FIG. 14, the amount of information transmitted to the client device during random access remains almost the same, and the update process performed in the client device can be reduced.

[0148] That is, by storing the initial value-based difference information or the updated information in the file as a random access point, it is possible to suppress an increase in the processing load when randomly accessing the scene description in a wider variety of cases. For example, when the amount of updated information of the scene description is large as described above, the updated information is stored in the file as a random access point, and when the amount of updated information of the scene description is small, the initial value-based difference information is stored in the file as a random access point, so that regardless of the amount of updated information of the scene description, an increase in the processing load of the client device that randomly accesses the scene description can be suppressed.

[0149] Note that the selection method of whether to store the initial value-based difference information or the updated information as a random access point is arbitrary. You may select which one to store based on arbitrary information, or you may select according to external requests or instructions such as user instructions or application requests.

[0150] <Method 2-1> When applying such Method 2, as shown in the second row from the top of the table shown in FIG. 23, the initial value reference difference information or the updated information may be stored in a track different from the immediately preceding value reference difference information (Method 2-1). Note that the immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected, as described above in <Method 1-1> and the like.

[0151] For example, the file generation unit may store the initial value reference difference information or the updated information in a track different from the immediately preceding value reference difference information of the update file. Also, the update information acquisition unit may acquire the initial value reference difference information or the updated information stored in a track different from the immediately preceding value reference difference information of the update file.

[0152] When the initial value reference difference information is stored in a track different from the immediately preceding value reference difference information as a random access point, the initial value reference difference information is stored as in the example of FIG. 14. Therefore, similar to the case of Method 1-1, the client device can more easily acquire a sample of the initial value reference difference information (JSON Patch 101) independently of the sample of the immediately preceding value reference difference information (JSON Patch 82) by specifying the track.

[0153] Also, when the updated information is stored in a track different from the immediately preceding value reference difference information as a random access point, the updated information is stored as in the example of FIG. 12. That is, the client device can obtain the latest scene description without updating the scene description during random access.

[0154] Note that the selection of whether to store the initial value reference difference information or the updated information as the random access point may be made in any data unit. For example, as in the examples of FIGS. 12 and 14, this selection may be made in sequence units. That is, in this case, the initial value reference difference information and the updated information do not coexist on the same track (only one of them is stored).

[0155] On the other hand, the selection of whether to store the initial value reference difference information or the updated information may be made for each sample (for each random access point). An example of this case is shown in FIG. 24. In the example of FIG. 24, as data of the random access point, the JSON patch 101-1 and the scene description 83-2 are stored on the same track (Track #2). The JSON patch 101-1 is the initial value reference difference information, and the scene description 83-2 is the updated information. That is, in this case, the initial value reference difference information and the updated information may coexist on the same track.

[0156] Note that also in this case, similar to the case of Method 1-1, the difference information stored as the random access point may be based on other than the initial value.

[0157] <Method 2-1-1> When applying such Method 2-1, as shown in the third row from the top of the table shown in FIG. 23, the link information to the track for storing the initial value reference difference information or the updated information may be stored as the link information for random access in the initial value of the scene description (SD) (Method 2-1-1).

[0158] For example, the file generation unit may generate an initial value file for storing the initial value of the spatial arrangement information, and store, as link information for random access, link information indicating a link to a track for storing the initial value reference difference information or the updated information. Further, the update information acquisition unit may use the link information indicating a link to a track for storing the initial value reference difference information or the updated information, which is stored as link information for random access in an initial value file for storing the initial value of the spatial arrangement information, to acquire the initial value reference difference information or the updated information.

[0159] FIG. 25 shows a part of a description example of the scene description 81-0 in that case. As shown in FIG. 25, in the scene description 81-0 (Scene.gltf), in addition to the link information ("uri": scen_updata.json-patch) to the track for storing the sample of the immediately preceding value reference difference information, the link information ("uri_for_random_access": random_access.json) to the track for storing the sample of the initial value reference difference information or the updated information is described. The link information to the track for storing the sample of the immediately preceding value reference difference information is the link information used during normal playback. In contrast, the link information to the track for storing the sample of the initial value reference difference information or the updated information is the link information used during random access. That is, in the example of FIG. 15, the link information is described in a state where these two can be distinguished. That is, the link information to the track for storing the sample of the initial value reference difference information or the updated information is stored as the link information for random access in the initial value of the scene description. By doing so, the client device can more easily identify the track in which the appropriate update information is stored. For example, the client device can more easily select the link information for obtaining the sample of the immediately preceding value reference difference information during normal playback. Also, the client device can more easily select the link information for obtaining the sample of the initial value reference difference information or the updated information during random access.

[0160] <Method 2-2> Also, when applying Method 2, as shown in the fourth row from the top of the table shown in FIG. 23, the initial value reference difference information or the updated information may be stored in the same sample of the same track as the immediately preceding value reference difference information (Method 2-2). Note that the immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected, as described above in <Method 1-1>.

[0161] For example, the file generation unit may store the initial value reference difference information or the updated information in the same sample of the same track as the immediately preceding value reference difference information of the update file. Also, the update information acquisition unit may acquire the initial value reference difference information or the updated information stored in the same sample of the same track as the immediately preceding value reference difference information of the update file.

[0162] When the initial value reference difference information is stored in the same sample of the same track as the immediately preceding value reference difference information as a random access point, the initial value reference difference information is stored as in the example of FIG. 16. Therefore, similar to the case of Method 1-2, the client device can make the track of the update file one. Therefore, the client device can obtain update information (immediately preceding value reference difference information or initial value reference difference information) using the same link information both during normal playback and during random access.

[0163] Also, as a random access point, when the updated information is stored in the same sample of the same track as the immediately preceding value-based difference information, the updated information is stored as in the example of FIG. 26. That is, the scene description 83-1 (updated information) is stored in the same sample 131-1 as the JSON patch 82-6. The update result to which the JSON patch 82-6 is applied is equivalent to the scene description 83-1. Also, the scene description 83-2 (updated information) is stored in the same sample 131-2 as the JSON patch 82-9. The update result to which the JSON patch 82-9 is applied is equivalent to the scene description 83-2. By doing so, the track of the update file can be made into one. Therefore, the client device can obtain the update information (immediately preceding value-based difference information or updated information) using the same link information both during normal playback and during random access.

[0164] Note that the selection of whether to store the initial value-based difference information or the updated information as a random access point may be made in any data unit. For example, as in the examples of FIGS. 16 and 26, this selection may be made in sequence units. That is, in this case, the initial value-based difference information and the updated information do not coexist in the same track (only one of them is stored).

[0165] On the other hand, for each sample (for each random access point), a selection may be made as to whether to store the initial value-based difference information or the updated information. An example in that case is shown in FIG. 27. In the example of FIG. 27, the JSON patch 101-1 is stored as the data of the random access point in the same sample 132-1 of the same track as the JSON patch 82-6. Also, the scene description 83-2 is stored as the data of the random access point in the same sample 132-2 of the same track as the JSON patch 82-9. That is, the JSON patch 101-1 and the scene description 83-2 are stored in the same track (Track #1). The JSON patch 101-1 is the initial value-based difference information, and the scene description 83-2 is the updated information. That is, in this case, the initial value-based difference information and the updated information can be mixed in the same track.

[0166] <Method 2-2-1> When applying such a method 2-2, the samples of the random access points are grouped as one sample with the immediately preceding value-based difference information and the initial value-based difference information or the updated information. Therefore, the immediately preceding value-based difference information and the initial value-based difference information or the updated information may be stored so that the client device can use them separately. That is, as shown in the fifth row from the top in the table shown in FIG. 23, they may be stored as a list having the immediately preceding value-based difference information and the initial value-based difference information or the updated information as elements respectively (Method 2-2-1).

[0167] For example, the file generation unit may store the immediately preceding value-based difference information and the initial value-based difference information or the updated information as a list having them as elements respectively. Also, the immediately preceding value-based difference information and the initial value-based difference information or the updated information may be stored in the sample as a list having them as elements respectively.

[0168] That is, when the immediately preceding value-based difference information and the initial value-based difference information are stored in the same sample, similar to the case of Method 1-2-1 described with reference to FIG. 17, the immediately preceding value-based difference information and the initial value-based difference information are each delimited by [] and described as elements of a list. Similarly, when the immediately preceding value-based difference information and the updated information are stored in the same sample, the immediately preceding value-based difference information and the updated information are each delimited by [] and described as elements of a list.

[0169] In this way, the client device can more easily acquire the immediately preceding value-based difference information and the initial value-based difference information or the updated information.

[0170] <Method 2-2-2> Also, when applying Method 2-2, as shown in the sixth row from the top of the table shown in FIG. 23, in the management area of the file, the immediately preceding value-based difference information and the initial value-based difference information or the updated information may each be managed as a subsample (Method 2-2-2).

[0171] For example, the file generation unit may store management information for managing each of the immediately preceding value-based difference information and the initial value-based difference information or the updated information as a subsample in the management area of the update file. Also, the update information acquisition unit may acquire the initial value-based difference information or the updated information based on the management information for managing each of the immediately preceding value-based difference information and the initial value-based difference information or the updated information stored in the management area of the update file as a subsample.

[0172] For example, in the Sub sample information box of ISOBMFF, the data within the sample is divided into subsamples, and the data size (subsample_size) of each subsample is stored. Thereby, the client device can acquire the data of the subsample, which is the data in the middle, without interpreting the data within the sample from the beginning.

[0173] That is, similar to the case of Method 1-2-2, management information for managing each of the immediately preceding value-based difference information and the initial value-based difference information as sub-samples is stored in the management area of the update file. As a result, the client device can obtain the immediately preceding value-based difference information or the initial value-based difference information from the sample without parsing the inside of the sample based on the management information. Similarly, management information for managing each of the immediately preceding value-based difference information and the updated information as sub-samples is stored in the management area of the update file. As a result, the client device can obtain the immediately preceding value-based difference information or the updated information from the sample without parsing the inside of the sample based on the management information. That is, the client device can more easily obtain the immediately preceding value-based difference information, the initial value-based difference information, and the updated information.

[0174] <Method 2-2-2-1> When applying this Method 2-2-2, as shown in the seventh row from the top of the table shown in FIG. 23, identification information for identifying the sub-sample in which the immediately preceding value-based difference information is stored and the sub-sample in which the initial value-based difference information or the updated information is stored may be stored in the management area of the file (Method 2-2-2-1).

[0175] For example, the management information for managing each of the immediately preceding value-based difference information, the initial value-based difference information, or the updated information as sub-samples may include identification information for identifying the sub-sample in which the immediately preceding value-based difference information is stored and the sub-sample in which the initial value-based difference information or the updated information is stored.

[0176] For example, the codec-specific parameters stored in the sub-sample information box of ISOBMFF (Sub sample information box) may be used as identification information for identifying the sub-sample in which the immediately preceding value-based difference information is stored and the sub-sample in which the initial value-based difference information or the updated information is stored.

[0177] For example, when the codec specific parameter is false (codec_specific_parameters = 0), it indicates that the sub-picture is previous-value-based differential information, and when the codec specific parameter is true (codec_specific_parameters = 1), it indicates that the sub-picture is initial-value-based differential information or updated information. By referring to the value of this codec specific parameter, the client device can easily identify (without parsing inside the sample) whether the subsample stores previous-value-based differential information or the subsample stores initial-value-based differential information or updated information.

[0178] <Method 2-2-3> Also, when applying Method 2-2, as shown in the eighth row from the top of the table shown in FIG. 23, in the management area of the file, the sample storing the initial-value-based differential information or the updated information may be managed as a randomly accessible sample (Method 2-2-3).

[0179] For example, the file generation unit may store in the management area of the update file the management information for managing the sample storing the initial-value-based differential information or the updated information as a randomly accessible sample. Also, the update information acquisition unit may acquire the initial-value-based differential information or the updated information based on the management information stored in the management area of the update file for managing the sample storing the initial-value-based differential information or the updated information as a randomly accessible sample.

[0180] For example, in the SyncSampleBox of ISOBMFF, a sample in which initial value-based difference information is stored is described as a SyncSample. This SyncSample is a randomly accessible sample. By describing a sample in which initial value-based difference information or updated information is stored within the SyncSampleBox, a client device can easily identify a randomly accessible sample (without parsing the inside of the sample) by referring to the SyncSampleBox. That is, based on this information, the client device can more easily identify a sample in which initial value-based difference information or updated information is stored.

[0181] <Method 2-3> Also, when applying Method 2, as shown in the ninth row from the top of the table shown in FIG. 23, the initial value-based difference information or the updated information may be stored in a different sample on the same track as the immediately preceding value-based difference information (Method 2-3). Note that the immediately preceding value-based difference information is difference information based on the spatial arrangement information that reflects the updates up to the immediately preceding update information, as described above in <Method 1-1>.

[0182] For example, the file generation unit may store the initial value-based difference information or the updated information in a different sample on the same track as the immediately preceding value-based difference information of the updated file. Also, the update information acquisition unit may acquire the initial value-based difference information or the updated information stored in a different sample on the same track as the immediately preceding value-based difference information of the updated file.

[0183] As a random access point, when the initial value reference difference information is stored in different samples of the same track where the immediate value reference difference information is the same, the initial value reference difference information is stored as in the example of FIG. 20. Therefore, similar to the case of Method 1-3, the tracks of the update file can be made into one. Therefore, the client device can obtain update information (immediate value reference difference information or initial value reference difference information) using the same link information both during normal playback and during random access. Also, since either the immediate value reference difference information or the initial value reference difference information is stored in the sample (since the immediate value reference difference information and the initial value reference difference information are not stored in the same sample), the client device can select either the immediate value reference difference information or the initial value reference difference information by selecting the sample.

[0184] On the other hand, as a random access point, when the updated information is stored in different samples of the same track where the immediate value reference difference information is the same, the updated information is stored, for example, as in the example of FIG. 28. In the example of FIG. 28, the scene description 83-1 (updated information) is stored in the sample next to the sample in which the JSON patch 82-6 is stored in the same track (Track #1) as the JSON patch 82-6. The update result to which the JSON patch 82-6 is applied is equivalent to the scene description 83-1. Also, the scene description 83-2 (updated information) is stored in the sample next to the sample in which the JSON patch 82-9 is stored in the same track (Track #1) as the JSON patch 82-9. The update result to which the JSON patch 82-9 is applied is equivalent to the scene description 83-2.

[0185] By doing so, the track of the update file can be made single. Therefore, the client device can obtain update information (immediately preceding value-based difference information or updated information) using the same link information both during normal playback and during random access. Also, since either immediately preceding value-based difference information or updated information is stored in a sample (since immediately preceding value-based difference information and updated information are not stored in the same sample), the client device can select either immediately preceding value-based difference information or updated information by selecting a sample.

[0186] Note that the selection of whether to store initial value-based difference information or updated information as a random access point may be made in any data unit. For example, as in the examples of FIGS. 20 and 28, this selection may be made in sequence units. That is, in this case, the initial value-based difference information and the updated information do not coexist in the same track (only one of them is stored).

[0187] On the other hand, the selection of whether to store initial value-based difference information or updated information may be made for each sample (for each random access point). An example of that case is shown in FIG. 29. In the example of FIG. 29, JSON Patch 101-1 is stored as data of a random access point in the sample next to JSON Patch 82-6. Also, Scene Description 83-2 is stored as data of a random access point in the sample next to JSON Patch 82-9. That is, JSON Patch 101-1 and Scene Description 83-2 are stored in the same track (Track #1). JSON Patch 101-1 is initial value-based difference information, and Scene Description 83-2 is updated information. That is, in this case, the initial value-based difference information and the updated information can coexist in the same track.

[0188] <Method 2-3-1> When applying such a method 2-3, as shown in the 10th row from the top of the table shown in FIG. 23, the same update application time information as that of the sample storing the immediately preceding value-based difference information may be stored in the sample storing the initial value-based difference information or the updated information to which the update is applied at the same timing as that of the immediately preceding value-based difference information (Method 2-3-1).

[0189] For example, the file generation unit may store the same update application time information as that of the sample storing the immediately preceding value-based difference information in the sample storing the initial value-based difference information or the updated information. Also, the sample storing the initial value-based difference information or the updated information may store the same update application time information as that of the sample storing the immediately preceding value-based difference information.

[0190] Similar to the case of Method 1-3-1, time information specifying the time when the update is executed is stored in the sample, and the sample storing the initial value-based difference information or the updated information stores the same time information as that of the sample storing the immediately preceding value-based difference information whose update application timing is the same as that of the initial value-based difference information. For example, in the case of FIG. 20, the same time information is stored in the sample of JSON Patch 82-6 and the sample of the next JSON Patch 101-1. Also, the same time information is stored in the sample of JSON Patch 82-9 and the sample of the next JSON Patch 101-2. Also, in the example of FIG. 28, the same time information is stored in the sample of JSON Patch 82-6 and the sample of the next Scene Description 83-1. Also, the same time information is stored in the sample of JSON Patch 82-9 and the sample of the next Scene Description 83-2. Also, in the example of FIG. 29, the same time information is stored in the sample of JSON Patch 82-6 and the sample of the next JSON Patch 101-1. Also, the same time information is stored in the sample of JSON Patch 82-9 and the sample of the next Scene Description 83-2.

[0191] By doing so, without parsing the inside of the sample, the client device can easily identify the immediately preceding value-based difference information corresponding to the initial value-based difference information or the updated information (where the update application timing is the same as the initial value-based difference information or the updated information).

[0192] <Method 2-3-2> Also, when applying Method 2-3, as shown in the 11th row from the top of the table shown in FIG. 23, in the management area of the file, the sample in which the initial value-based difference information or the updated information is stored may be managed as a sample not used during normal playback (Method 2-3-2).

[0193] For example, the file generation unit may store, in the management area of the update file, management information for managing the sample in which the initial value-based difference information or the updated information is stored as a sample not used during normal playback. Also, the update information acquisition unit may acquire the initial value-based difference information or the updated information based on the management information stored in the management area of the update file for managing the sample in which the initial value-based difference information or the updated information is stored as a sample not used during normal playback.

[0194] By managing, in the management area of the update file that stores the update information, the sample in which the initial value-based difference information or the updated information is stored as a sample not used during normal playback, the client device can easily omit the acquisition of the sample in which the initial value-based difference information or the updated information is stored without parsing the inside of the sample during normal playback. That is, when sequentially acquiring the samples of the update file as normal playback, the client device can easily refrain from acquiring the sample in which the initial value-based difference information or the updated information is stored based on this management information, and can acquire only the sample in which the immediately preceding value-based difference information is stored.

[0195] In other words, during random access, the client device can easily obtain a sample in which the initial value reference difference information or the updated information is stored without parsing the inside of the sample. That is, when performing random access, the client device can easily obtain a sample in which the initial value reference difference information or the updated information is stored based on this management information.

[0196] <Method 2-3-2-1> When applying such a method 2-3-2, as shown in the 12th row from the top of the table shown in FIG. 23, identification information indicating whether or not it is a sample used during normal playback may be stored in the management area of the file for each sample (Method 2-3-2-1).

[0197] For example, management information for managing a sample in which the initial value reference difference information or the updated information is stored as a sample not used during normal playback may include identification information indicating whether or not it is a sample used during that normal playback.

[0198] For example, similar to the case of Method 1-3-2-1, the sample_has_redundancy flag described in the SampleDependencyTypeBox may be used as identification information indicating whether the sample is a sample to be used during normal playback. That is, in the SampleDependencyTypeBox, for samples in which the initial value-based difference information or the updated information is stored, a sample_has_redundancy flag with a value of "1" (sample_has_redundancy flag = 1) is stored. Then, during normal playback, the client device omits the acquisition of samples with a value of "1" for this sample_has_redundancy flag. By doing so, during normal playback, the client device can easily omit the acquisition of samples in which the initial value-based difference information or the updated information is stored without parsing the inside of the sample. That is, when sequentially acquiring samples of the update file as normal playback, the client device can easily, based on this sample_has_redundancy flag, refrain from acquiring samples in which the initial value-based difference information or the updated information is stored and acquire only samples in which the immediately preceding value-based difference information is stored.

[0199] <Method 2-3-3> Also, when applying Method 2-3, as shown in the 13th row from the top of the table shown in FIG. 23, in the management area of the file, samples in which the initial value-based difference information or the updated information is stored may be managed as randomly accessible samples (Method 2-3-3).

[0200] For example, the file generation unit may store, in the management area of the update file, management information for managing samples in which the initial value-based difference information or the updated information is stored as randomly accessible samples. Also, the update information acquisition unit may acquire the initial value-based difference information or the updated information based on the management information for managing samples in which the initial value-based difference information or the updated information is stored, which is stored in the management area of the update file, as randomly accessible samples.

[0201] For example, in the same manner as in the case of <Method 2-2-3>, a sample in which initial value-based difference information or updated information is stored is described as a sync sample in a sync sample box (SyncSampleBox). By doing so, the client device can easily identify a randomly accessible sample (without parsing the inside of the sample) by referring to the sync sample box. That is, the client device can more easily identify a sample in which initial value-based difference information or updated information is stored based on this information.

[0202] <Method 2-4> When applying Method 2, as shown in the 14th row from the top of the table shown in FIG. 23, the initial value-based difference information or the updated information may be replaced with the immediately preceding value-based difference information (Method 2-4). Note that the immediately preceding value-based difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected, as described above in <Method 1-1>.

[0203] For example, the file generation unit may replace the immediately preceding value-based difference information stored in the update file, whose update application time is the same as the initial value-based difference information or the updated information, with the initial value-based difference information or the updated information. Also, the update information acquisition unit may acquire the initial value-based difference information or the updated information stored in the update file, in which the immediately preceding value-based difference information whose update application time is the same as the initial value-based difference information or the updated information has been replaced.

[0204] When the initial value-based difference information is replaced with the immediately preceding value-based difference information as a random access point, the initial value-based difference information is stored as in the example of FIG. 22. Therefore, similar to the case of Method 1-4, the data amount of the update file can be reduced compared to the cases of Method 2-2 and Method 2-3.

[0205] On the other hand, as a random access point, when the updated information is replaced with the immediately preceding value-based difference information, the updated information is stored as in the example of FIG. 30, for example. In the example of FIG. 30, the sample of the scene description 83-1 (updated information) is replaced with the sample of the JSON patch 82-6. Similarly, the sample of the scene description 83-2 (updated information) is replaced with the sample of the JSON patch 82-9.

[0206] Therefore, in the case of normal playback, the scene description of the update application timing of samples other than the samples of the random access point is generated using the immediately preceding value-based difference information (JSON patch 82) and the scene description at the timing immediately before that. On the other hand, as the scene description of the update application timing of the samples of the random access point, the updated information (scene description 83) is applied both in the case of normal playback and in the case of random access.

[0207] By doing so, the data amount of the update file can be reduced compared to the cases of Method 1-2 and Method 1-3.

[0208] Note that the selection of whether to store the initial value-based difference information or the updated information as a random access point may be made in any data unit. For example, this selection may be made in sequence units as in the examples of FIGS. 22 and 30. That is, in this case, the initial value-based difference information and the updated information do not coexist in the same track (only one of them is stored).

[0209] On the other hand, for each sample (for each random access point), a selection may be made as to whether to store the initial value-based difference information or the updated information. An example in that case is shown in FIG. 31. In the example of FIG. 31, the sample of JSON patch 101-1 is replaced with the sample of JSON patch 82-6 as the data of the random access point. Also, the sample of scene description 83-2 is replaced with the sample of JSON patch 82-9 as the data of the random access point. That is, JSON patch 101-1 and scene description 83-2 are stored in the same track (Track #1). JSON patch 101-1 is the initial value-based difference information, and scene description 83-2 is the updated information. That is, in this case, the initial value-based difference information and the updated information can be mixed in the same track.

[0210] Note that in the case of this method 2-4, since there is no immediately preceding value-based difference information where the initial value-based difference information or the updated information and the update application timing are the same as in the case of method 2-3, the sample dropout redundancy flag is not necessary (can be omitted).

[0211] <Method 2-4-1> Also, when applying method 2-4, as shown in the bottom row of the table shown in FIG. 23, in the management area of the file, the sample in which the initial value-based difference information or the updated information is stored may be managed as a randomly accessible sample (method 2-3-3).

[0212] For example, the file generation unit may store, in the management area of the update file, management information for managing the sample in which the initial value-based difference information or the updated information is stored as a randomly accessible sample. Also, the update information acquisition unit may acquire the initial value-based difference information or the updated information based on the management information for managing the sample in which the initial value-based difference information or the updated information is stored, which is stored in the management area of the update file, as a randomly accessible sample.

[0213] For example, similar to the case of <Method 2-2-3>, samples in which initial value-based difference information or updated information is stored are described as sync samples in a sync sample box (SyncSampleBox). By doing so, the client device can easily identify randomly accessible samples (without parsing the inside of the samples) by referring to the sync sample box. That is, the client device can more easily identify samples in which the initial value-based difference information or the updated information is stored based on this information.

[0214] <4. First Embodiment> <File Generation Device> Each technique of the present technology described above can be applied to any device. FIG. 32 is a block diagram showing an example of the configuration of a file generation device which is an aspect of an information processing device to which the present technology is applied. The file generation device 300 shown in FIG. 28 is a device that generates information for distributing 3D object content. For example, the file generation device 300 generates a 3D object content file to be distributed, or generates a scene description file (scene description) of the 3D object content.

[0215] Note that in FIG. 32, main components such as a processing unit and a data flow are shown, and not all of the components shown in FIG. 32 are necessarily included. That is, in the file generation device 300, there may be a processing unit not shown as a block in FIG. 32, or there may be a process or a data flow not shown as an arrow or the like in FIG. 32.

[0216] As shown in FIG. 32, the file generation device 300 includes a control unit 301 and a file generation processing unit 302. The control unit 301 controls the file generation processing unit 302. The file generation processing unit 302 performs processing related to file generation under the control of the control unit 301. For example, the file generation processing unit 302 generates a 3D object file, which is a content file for storing data of the 3D object to be distributed. In addition, the file generation processing unit 302 generates a scene description file for storing a scene description corresponding to the 3D object and an update file thereof. The file generation processing unit 302 outputs the generated file to the outside of the file generation device 300. For example, the file generation processing unit 302 uploads the generated file to a distribution server or the like.

[0217] The file generation processing unit 302 includes an input unit 311, a preprocessing unit 312, an encoding unit 313, a file generation unit 314, a recording unit 315, and an output unit 316.

[0218] The input unit 311 acquires data of the 3D object and supplies it to the preprocessing unit 312. The preprocessing unit 312 generates a scene description and update information thereof using the data of the 3D object. The preprocessing unit 312 supplies the generated information to the file generation unit 314. In addition, the preprocessing unit 312 supplies the data of the 3D object to the encoding unit 313.

[0219] The encoding unit 313 encodes the data of the 3D object supplied from the preprocessing unit 312 and generates encoded data (bitstream). The encoding unit 313 supplies the generated encoded data of the 3D object to the file generation unit 314.

[0220] The file generation unit 314 acquires the encoded data of the 3D object supplied from the encoding unit 313. In addition, the file generation unit 314 acquires information (for example, a scene description and update information thereof) supplied from the preprocessing unit 312.

[0221] The file generation unit 314 generates a 3D object file for storing the encoded data of the 3D object supplied from the encoding unit 313. Further, the file generation unit 314 generates a scene description file for storing the scene description supplied from the preprocessing unit 312. Furthermore, the file generation unit 314 generates an update file for storing the update information supplied from the preprocessing unit 312.

[0222] The file generation unit 314 supplies the generated files to the recording unit 315. The recording unit 315 has an arbitrary recording medium such as a hard disk or a semiconductor memory, and records the files supplied from the file generation unit 314 on the recording medium. Further, the recording unit 315 reads out the files recorded on the recording medium in accordance with a request from the control unit 301 or the output unit 316, or at a predetermined timing, and supplies them to the output unit 316.

[0223] The output unit 316 acquires the files supplied from the recording unit 315, and outputs the files to the outside of the file generation apparatus 300 (for example, a distribution server or a playback apparatus).

[0224] <Flow of File Generation Process 1> This file generation apparatus 300 can generate files by applying the present disclosure described above, for example, in <2. Transmission of Initial Value-Based Difference Information>. For example, the file generation apparatus 300 can generate files by applying the above-described Method 1. An example of the flow of the file generation process executed by the file generation apparatus 300 in that case will be described with reference to the flowchart of FIG. 33.

[0225] When the file generation process is started, the input unit 311 of the file generation apparatus 300 acquires the data of the 3D object in step S301.

[0226] In step S302, the preprocessing unit 312 generates a scene description, which is spatial arrangement information for arranging one or more 3D objects in a 3D space, using the 3D object data acquired in step S301. Further, the file generation unit 314 generates a scene description file for storing the scene description.

[0227] In step S303, the preprocessing unit 312 updates the scene description generated in step S302 along the time axis, and generates previous-value-based difference information as update information applied during normal playback.

[0228] In step S304, the file generation unit 314 generates an update file for storing the update information, and stores the update information (previous-value-based difference information) generated in step S303 in the update file. At this time, the file generation unit 314 stores the previous-value-based difference information in samples of a track for storing the previous-value-based difference information.

[0229] In step S305, the preprocessing unit 312 applies the method 1 described above in <Method 1> to generate initial-value-based difference information as update information for the scene description generated in step S302.

[0230] In step S306, the file generation unit 314 applies the method 1 described above in <Method 1> and stores the initial-value-based difference information generated in step S303 in the update file as samples of random access points.

[0231] In step S307, the encoding unit 313 encodes the 3D object data acquired in step S301 to generate encoded data (bitstream) of the 3D object.

[0232] In step S308, the file generation unit 314 generates a 3D object file which is a content file, and stores the encoded data of the 3D object generated in step S307 in the 3D object file.

[0233] In step S309, the recording unit 315 records the scene description file generated in step S302 on a recording medium. Also, the recording unit 315 records the update file generated in step S304 and in which update information is stored in steps S304 and S306 on the recording medium. Further, the recording unit 315 records the 3D object file generated in step S308 on the recording medium.

[0234] In step S310, the output unit 316 reads out the various files recorded in step S309 from the recording medium, and outputs the read files outside the file generation apparatus 300 at a predetermined timing. For example, the output unit 316 may transmit (upload) the files read from the recording medium to other apparatuses such as a distribution server or a playback apparatus via a communication medium such as a network. Also, the output unit 316 may record the files read from the recording medium on an external recording medium such as a removable medium. In that case, the output files may be supplied to other apparatuses (such as a distribution server or a playback apparatus) via the external recording medium, for example.

[0235] When the process of step S310 ends, the file generation process ends.

[0236] As described above, by executing each process, the file generation apparatus 300 can suppress an increase in the processing load of a client apparatus that randomly accesses a scene description, as described above in <Method 1>.

[0237] In addition, when Method 1 is applied in step S306, the file generation unit 314 may apply Method 1-1 described above in <Method 1-1>. That is, the file generation unit 314 may store the initial value reference difference information in a track different from the immediate previous value reference difference information of the update file. By doing so, the client device can more easily obtain samples of the initial value reference difference information independently of the samples of the immediate previous value reference difference information by specifying the track.

[0238] Also, when Method 1-1 is applied in step S306, the file generation unit 314 may apply Method 1-1-1 described above in <Method 1-1-1>. That is, the file generation unit 314 may generate an initial value file for storing the initial value of the spatial arrangement information, and store link information indicating a link to the track for storing the initial value reference difference information in the initial value file as link information for random access. By doing so, the client device can more easily identify the track in which appropriate update information is stored.

[0239] Also, when Method 1 is applied in step S306, the file generation unit 314 may apply Method 1-2 described above in <Method 1-2>. That is, the file generation unit 314 may store the initial value reference difference information in the same sample of the same track as the immediate previous value reference difference information of the update file. By doing so, the client device can obtain update information (immediate previous value reference difference information or initial value reference difference information) using the same link information both during normal playback and during random access.

[0240] Also, when Method 1-2 is applied in step S306, the file generation unit 314 may apply the Method 1-2-1 described above in <Method 1-2-1>. That is, the file generation unit 314 may store the immediately preceding value-based difference information and the initial value-based difference information as lists each having the same sample as an element. By doing so, the client device can more easily acquire the immediately preceding value-based difference information and the initial value-based difference information.

[0241] Also, when Method 1-2 is applied in step S306, the file generation unit 314 may apply the Method 1-2-2 described above in <Method 1-2-2>. That is, the file generation unit 314 may store management information (for example, SubSampleInformationBox) that manages each of the immediately preceding value-based difference information and the initial value-based difference information as sub-samples in the management area of the update file. By doing so, the client device can more easily acquire the immediately preceding value-based difference information and the initial value-based difference information.

[0242] Also, when Method 1-2-2 is applied in step S306, the file generation unit 314 may apply the Method 1-2-2-1 described above in <Method 1-2-2-1>. That is, the above-described management information may include identification information (for example, codec_specific_parameters) that identifies the sub-sample in which the immediately preceding value-based difference information is stored and the sub-sample in which the initial value-based difference information is stored. By referring to the value of this identification information, the client device can easily identify (without parsing the inside of the sample) whether it is the sub-sample in which the immediately preceding value-based difference information is stored or the sub-sample in which the initial value-based difference information is stored.

[0243] Also, when Method 1-2 is applied in step S306, the file generation unit 314 may apply the Method 1-2-3 described above in <Method 1-2-3>. That is, the file generation unit 314 may store management information (SyncSampleBox) for managing samples in which the initial value reference difference information is stored as randomly accessible samples in the management area of the update file. Based on this management information, the client device can more easily identify the samples in which the initial value reference difference information is stored.

[0244] Also, when Method 1 is applied in step S306, the file generation unit 314 may apply the Method 1-3 described above in <Method 1-3>. That is, the file generation unit 314 may store the initial value reference difference information in different samples of the same track that are the same as the immediately preceding value reference difference information of the update file. By doing so, the client device can select the immediately preceding value reference difference information or the initial value reference difference information by selecting a sample.

[0245] Also, when Method 1-3 is applied in step S306, the file generation unit 314 may apply the Method 1-3-1 described above in <Method 1-3-1>. That is, the file generation unit 314 may store the same update application time information as the sample in which the immediately preceding value reference difference information is stored in the sample in which the initial value reference difference information is stored. By doing so, the client device can easily identify the immediately preceding value reference difference information corresponding to the initial value reference difference information without parsing the inside of the sample.

[0246] Also, when Method 1-3 is applied in step S306, the file generation unit 314 may apply the Method 1-3-2 described above in <Method 1-3-2>. That is, the file generation unit 314 may store, in the management area of the update file, management information that manages the samples storing the initial value reference difference information as samples not used during normal playback. By doing so, the client device can easily omit acquiring the samples storing the initial value reference difference information (without parsing the inside of the samples) based on this management information during normal playback. In other words, when performing random access, the client device can easily acquire the samples storing the initial value reference difference information based on this management information (without parsing the inside of the samples).

[0247] Also, when Method 1-3-2 is applied in step S306, the file generation unit 314 may apply the Method 1-3-2-1 described above in <Method 1-3-2-1>. That is, the above-described management information may include identification information (for example, sample_has_redundancy_flag) indicating whether the sample is a sample used during normal playback. By doing so, the client device can easily omit acquiring the samples storing the initial value reference difference information (without parsing the inside of the samples) based on this identification information during normal playback.

[0248] Also, when Method 1-3 is applied in step S306, the file generation unit 314 may apply the Method 1-3-3 described above in <Method 1-3-3>. That is, the file generation unit 314 may store, in the management area of the update file, management information (for example, SyncSampleBox) that manages the samples storing the initial value reference difference information as randomly accessible samples. By doing so, the client device can more easily identify the samples storing the initial value reference difference information based on this management information.

[0249] Also, when Method 1 is applied in step S306, the file generation unit 314 may apply the Method 1-4 described above in <Method 1-4>. That is, the file generation unit 314 may replace the immediately preceding value reference difference information whose update application time is the same as the initial value reference difference information and is stored in the update file with the initial value reference difference information. By doing so, the data amount of the update file can be reduced compared to the cases of Method 1-2 and Method 1-3.

[0250] Also, when Method 1-4 is applied in step S306, the file generation unit 314 may apply the Method 1-4-1 described above in <Method 1-4-1>. That is, the file generation unit 314 may store management information (for example, SyncSampleBox) that manages the sample in which the initial value reference difference information is stored as a randomly accessible sample in the management area of the update file. By doing so, the client device can more easily identify the sample in which the initial value reference difference information is stored based on this management information.

[0251] Note that as long as there is no contradiction, any plurality of the above-described various methods may be combined and applied. Also, the above-described various methods may be combined and applied with any other method not described above.

[0252] <Flow of File Generation Process 2> Also, this file generation device 300 can generate a file by applying the present disclosure described above in, for example, <3. Transmission of Initial Value Reference Difference Information or Updated Information>. For example, the file generation device 300 can generate a file by applying the above-described Method 2. An example of the flow of the file generation process executed by the file generation device 300 in that case will be described with reference to the flowchart of FIG. 34.

[0253] When the file generation process is started, the input unit 311 of the file generation device 300 acquires the data of the 3D object in step S351.

[0254] In step S352, the preprocessing unit 312 generates a scene description, which is spatial arrangement information for arranging one or more 3D objects in a 3D space, using the data of the 3D object acquired in step S351. Further, the file generation unit 314 generates a scene description file for storing the scene description.

[0255] In step S353, the preprocessing unit 312 updates the scene description generated in step S352 along the time axis, and generates previous-value-based difference information as update information applied during normal playback.

[0256] In step S354, the file generation unit 314 generates an update file for storing the update information, and stores the update information (previous-value-based difference information) generated in step S353 in the update file. At this time, the file generation unit 314 stores the previous-value-based difference information in samples of a track for storing the previous-value-based difference information.

[0257] In step S355, the preprocessing unit 312 applies the method 2 described above in <Method 2>, and selects and generates initial-value-based difference information or updated information as update information for the scene description generated in step S352.

[0258] In step S356, the file generation unit 314 applies the method 2 described above in <Method 2>, and stores the initial-value-based difference information or updated information generated in step S353 in the update file as samples of random access points.

[0259] In step S357, the encoding unit 313 encodes the data of the 3D object acquired in step S351, and generates encoded data (bit stream) of the 3D object.

[0260] In step S358, the file generation unit 314 generates a 3D object file which is a content file, and stores the encoded data of the 3D object generated in step S357 in the 3D object file.

[0261] In step S359, the recording unit 315 records the scene description file generated in step S352 on a recording medium. Also, the recording unit 315 records the update file generated in step S354 and in which update information is stored in steps S354 and S356 on the recording medium. Further, the recording unit 315 records the 3D object file generated in step S358 on the recording medium.

[0262] In step S360, the output unit 316 reads out the various files recorded in step S359 from the recording medium, and outputs the read files outside the file generation apparatus 300 at a predetermined timing. For example, the output unit 316 may transmit (upload) the files read from the recording medium to other apparatuses such as a distribution server or a playback apparatus via a communication medium such as a network. Also, the output unit 316 may record the files read from the recording medium on an external recording medium such as a removable medium. In that case, the output files may be supplied to other apparatuses (such as a distribution server or a playback apparatus) via the external recording medium, for example.

[0263] When the processing of step S360 ends, the file generation processing ends.

[0264] As described above, by executing each process, the file generation apparatus 300 can suppress an increase in the processing load of a client apparatus that randomly accesses a scene description in more diverse cases as described above in <Method 2>.

[0265] In addition, when Method 2 is applied in step S356, the file generation unit 314 may apply Method 2-1 described above in <Method 2-1>. That is, the file generation unit 314 may store the initial value reference difference information or the updated information in a track different from the immediately preceding value reference difference information of the update file. By doing so, the client device can more easily obtain the initial value reference difference information or the sample of the scene description independently of the sample of the immediately preceding value reference difference information by specifying the track.

[0266] Also, when Method 2-1 is applied in step S356, the file generation unit 314 may apply Method 1-1-1 described above in <Method 2-1-1>. That is, the file generation unit 314 may generate an initial value file for storing the initial value of the spatial arrangement information, and store the link information indicating the link to the track for storing the initial value reference difference information or the updated information as the link information for random access in the initial value file. By doing so, the client device can more easily identify the track in which the appropriate update information is stored.

[0267] Also, when Method 2 is applied in step S356, the file generation unit 314 may apply Method 2-2 described above in <Method 2-2>. That is, the file generation unit 314 may store the initial value reference difference information or the updated information in the same sample of the same track as the immediately preceding value reference difference information of the update file. By doing so, the client device can obtain the update information using the same link information both during normal playback and during random access.

[0268] Also, when Method 2-2 is applied in step S356, the file generation unit 314 may apply the Method 2-2-1 described above in <Method 2-2-1>. That is, the file generation unit 314 may store the immediately preceding value-based difference information and the initial value-based difference information or the updated information as lists each having the same sample as an element. By doing so, the client device can more easily acquire the immediately preceding value-based difference information and the initial value-based difference information or the updated information.

[0269] Also, when Method 2-2 is applied in step S356, the file generation unit 314 may apply the Method 2-2-2 described above in <Method 2-2-2>. That is, the file generation unit 314 may store management information (for example, SubSampleInformationBox) that manages each of the immediately preceding value-based difference information, the initial value-based difference information, and the updated information as sub-samples in the management area of the update file. By doing so, the client device can more easily acquire the immediately preceding value-based difference information, the initial value-based difference information, and the updated information.

[0270] Also, when Method 2-2-2 is applied in step S356, the file generation unit 314 may apply the Method 2-2-2-1 described above in <Method 2-2-2-1>. That is, the above-described management information may include identification information (for example, codec_specific_parameters) that identifies the sub-sample in which the immediately preceding value-based difference information is stored and the sub-sample in which the initial value-based difference information or the updated information is stored. By referring to the value of this identification information, the client device can easily identify (without parsing inside the sample) whether it is the sub-sample in which the immediately preceding value-based difference information is stored or the sub-sample in which the initial value-based difference information or the updated information is stored.

[0271] Also, when Method 2-2 is applied in step S356, the file generation unit 314 may apply the Method 2-2-3 described above in <Method 2-2-3>. That is, the file generation unit 314 may store in the management area of the update file management information (SyncSampleBox) that manages samples storing initial value reference difference information or updated information as randomly accessible samples. Based on this management information, the client device can more easily identify samples storing initial value reference difference information or updated information.

[0272] Also, when Method 2 is applied in step S356, the file generation unit 314 may apply the Method 2-3 described above in <Method 2-3>. That is, the file generation unit 314 may store initial value reference difference information or updated information in different samples of the same track as the immediately preceding value reference difference information of the update file. By doing so, the client device can select an immediately preceding value reference difference information and an initial value reference difference information or updated information by selecting a sample.

[0273] Also, when Method 2-3 is applied in step S356, the file generation unit 314 may apply the Method 2-3-1 described above in <Method 2-3-1>. That is, the file generation unit 314 may store the same update application time information as the sample storing the immediately preceding value reference difference information in the sample storing the initial value reference difference information or the updated information. By doing so, the client device can easily identify the immediately preceding value reference difference information corresponding to the initial value reference difference information or the updated information without parsing the inside of the sample.

[0274] Also, when Method 2-3 is applied in step S356, the file generation unit 314 may apply the Method 2-3-2 described above in <Method 2-3-2>. That is, the file generation unit 314 may store, in the management area of the update file, management information that manages samples storing initial value reference difference information or updated information as samples not used during normal playback. By doing so, the client device can easily omit acquiring samples storing initial value reference difference information (without parsing the inside of the samples) based on this management information during normal playback. In other words, when performing random access, the client device can easily acquire samples storing initial value reference difference information or updated information based on this management information (without parsing the inside of the samples).

[0275] Also, when Method 2-3-2 is applied in step S356, the file generation unit 314 may apply the Method 2-3-2-1 described above in <Method 2-3-2-1>. That is, the above-described management information may include identification information (for example, sample_has_redundancy_flag) indicating whether it is a sample used during normal playback. By doing so, the client device can easily omit acquiring samples storing initial value reference difference information or updated information (without parsing the inside of the samples) based on this identification information during normal playback.

[0276] Also, when Method 3-3 is applied in step S356, the file generation unit 314 may apply the Method 2-3-3 described above in <Method 2-3-3>. That is, the file generation unit 314 may store, in the management area of the update file, management information (for example, SyncSampleBox) that manages samples storing initial value reference difference information or updated information as randomly accessible samples. By doing so, the client device can more easily identify samples storing initial value reference difference information or updated information based on this management information.

[0277] Also, when Method 2 is applied in step S356, the file generation unit 314 may apply Method 2-4 described above in <Method 2-4>. That is, the file generation unit 314 may replace the immediately preceding value reference difference information stored in the update file, whose update application time is the same as the initial value reference difference information or the updated information, with the initial value reference difference information or the updated information. By doing so, compared with the cases of Method 2-2 and Method 2-3, the data volume of the update file can be reduced.

[0278] Also, when Method 2-4 is applied in step S356, the file generation unit 314 may apply Method 2-4-1 described above in <Method 2-4-1>. That is, the file generation unit 314 may store management information (for example, SyncSampleBox) for managing the sample in which the initial value reference difference information or the updated information is stored as a randomly accessible sample in the management area of the update file. By doing so, the client device can more easily identify the sample in which the initial value reference difference information or the updated information is stored based on this management information.

[0279] Note that as long as there is no contradiction, any plurality of the above-described various methods may be combined and applied. Also, the above-described various methods may be combined and applied with any other method not described above.

[0280] <5. Second Embodiment> <Client Device> FIG. 35 is a block diagram showing an example of the configuration of a client device, which is an aspect of an information processing apparatus to which the present technology is applied. The client device 400 shown in FIG. 35 is a playback device that performs playback processing of 3D object content based on a scene description file (scene description). For example, the client device 400 plays back the data of the 3D object stored in the 3D object file generated by the file generation device 300. At that time, the client device 400 performs processing related to the playback based on the scene description.

[0281] In addition, FIG. 35 shows the main components such as the processing unit and the data flow, but not all of the components shown in FIG. 35. That is, in the client device 400, there may be a processing unit not shown as a block in FIG. 35, or a process or data flow not shown as an arrow or the like in FIG. 35.

[0282] As shown in FIG. 35, the client device 400 includes a control unit 401 and a playback processing unit 402. The control unit 401 performs processes related to the control of the playback processing unit 402. The playback processing unit 402 performs processes related to the playback of 3D object data.

[0283] The playback processing unit 402 includes a file acquisition unit 411, a file processing unit 412, a decoding unit 413, a display information generation unit 414, a display unit 415, and a display control unit 416.

[0284] The file acquisition unit 411 acquires files from outside the client device 400, such as a distribution server or a file generation device 300. For example, the file acquisition unit 411 acquires a scene description file and its update file from outside the client device 400, and supplies them to the file processing unit 412. In addition, the file acquisition unit 411 acquires the 3D content file requested by the file processing unit 412 from outside the client device 400, and supplies it to the file processing unit 412.

[0285] The file processing unit 412 acquires various files supplied from the file acquisition unit 411 and performs processing on the acquired files. For example, the file processing unit 412 acquires a scene description file and its update file supplied from the file acquisition unit 411. Further, the file processing unit 412 extracts a scene description from the scene description file. Also, the file processing unit 412 extracts update information (such as difference information based on the previous value, difference information based on the initial value, and updated information) from the update file. Further, the file processing unit 412 updates the scene description using the update information as necessary, and generates the latest (at the timing of the processing target) scene description.

[0286] Also, the file processing unit 412 analyzes the scene description and identifies the data of the 3D object at the timing of the processing target necessary for reproduction. Then, the file processing unit 412 requests the file acquisition unit 411 to acquire the data of the identified 3D object. Further, the file processing unit 412 acquires the 3D object file supplied from the file acquisition unit 411 based on the request, and extracts the data (encoded data) of the 3D object stored in the file. The file processing unit 412 supplies the extracted data (encoded data) of the 3D object to the decoding unit 413. Also, the file processing unit 412 supplies information useful for generating display information included in the scene description or the like to the display control unit 416.

[0287] The decoding unit 413 decodes the encoded data of the 3D object supplied from the file processing unit 412. The decoding unit 413 supplies the data of the 3D object obtained by the decoding to the display information generation unit 414.

[0288] The display information generation unit 414 acquires the data of the 3D object supplied from the decoding unit 413. Further, the display information generation unit 414 performs rendering of the data of the 3D object according to the control of the display control unit 416, and generates a display image or the like. The display information generation unit 414 supplies the generated display image or the like to the display unit 415.

[0289] The display unit 415 has a display device, and displays the display image supplied from the display information generation unit 414 using the display device.

[0290] The display control unit 416 acquires information such as a scene description supplied from the file processing unit 412. The display control unit 416 controls the display information generation unit 414 based on the information.

[0291] <Flow of client processing 1> This client device 400 can perform processes related to analysis of a scene description, reproduction of 3D object data, etc. by applying the present disclosure described above, for example, in <2. Transmission of initial value reference difference information>. For example, the client device 400 can perform processing by applying the above-described method 1. An example of the flow of client processing executed by the client device 400 in that case will be described with reference to the flowcharts of FIGS. 36 and 37.

[0292] When the client processing is started, the file acquisition unit 411 of the client device 400 acquires a scene description file in step S401 of FIG. 36. The initial value of the scene description is stored in this scene description file. The file acquisition unit 411 supplies the acquired scene description file to the file processing unit 412. The file acquisition unit 411 extracts the scene description (initial value) from the scene description file.

[0293] In step S402, the control unit 401 determines whether to perform random access on the scene description. For example, if it is determined to perform random access based on an instruction from a user, an application, or the like, the process proceeds to step S403.

[0294] In step S403, the file processing unit 412 analyzes the scene description and identifies a random access point as the processing target timing. That is, the file processing unit 412 identifies a sample to be randomly accessed from among the prepared random access points.

[0295] In step S404, the file processing unit 412 requests the file acquisition unit 411 to acquire the initial value reference difference information of the random access point identified in step S403. The file acquisition unit 411 acquires an update file including the requested initial value reference difference information from outside the client device 400, such as a distribution server or the file generation device 300. That is, the file acquisition unit 411 requests a distribution server or the like to supply an update file (initial value reference difference information) including the initial value reference difference information requested by the file processing unit 412. Then, the file acquisition unit 411 acquires the update file (initial value reference difference information) supplied in response to the request. The file processing unit 412 applies the method 1 described above in <Method 1>, extracts and acquires the initial value reference difference information from the acquired update file.

[0296] In step S405, the file processing unit 412 applies the method 1 described above in <Method 1>, updates the scene description using the acquired initial value reference difference information, and generates a scene description at the processing target timing. That is, the file processing unit 412 generates a scene description (spatial arrangement information) at the update application time of the initial value reference difference information (i.e., the processing target timing) by reflecting the update of the initial value reference difference information in the initial value of the scene description, for example.

[0297] When the process of step S405 ends, the process proceeds to step S406. Also, in step S402, if it is determined that the access is not a random access (i.e., it is normal playback), the process proceeds to step S406.

[0298] In step S406, based on the scene description at the processing target timing, the file processing unit 412 requests the file acquisition unit 411 to acquire the encoded data of the 3D object at the processing target timing. The file acquisition unit 411 acquires from outside the client device 400 a 3D object file that stores the encoded data of the 3D object at the requested processing target timing. For example, the file acquisition unit 411 requests a distribution server or the like to supply a 3D object file including the encoded data of the 3D object requested by the file processing unit 412. Then, the file acquisition unit 411 acquires the 3D object file supplied in response to the request and supplies it to the file processing unit 412. The file processing unit 412 extracts the encoded data of the 3D object at the processing target timing from the 3D object file.

[0299] In step S407, the decoding unit 413 decodes the encoded data of the 3D object at the processing target timing obtained in step S406.

[0300] In step S408, the display information generation unit 414 performs processing related to the generation of a display image according to the control of the display control unit 416. For example, based on the scene description at the processing target timing obtained in step S405, the display information generation unit 414 arranges the 3D object at the processing target timing obtained in step S407 in a 3D space and performs rendering to generate a display image.

[0301] In step S409, the display unit 415 displays the display image generated in step S408. When the process of step S409 ends, the process proceeds to step S421 in FIG. 37.

[0302] In step S421 of FIG. 37, the control unit 401 determines whether to perform a random access. If it is determined to perform a random access, the process proceeds to step S403 of FIG. 36. Also, in step S421 of FIG. 37, if it is determined not to perform a random access (i.e., to perform normal reproduction), the process proceeds to step S422.

[0303] In step S422, the control unit 401 determines whether to end the client process. If it is determined not to end the client process, the process proceeds to step S423.

[0304] In step S423, the control unit 401 switches the processing target timing to the next timing. That is, the control unit 401 switches the processing target sample to the next sample.

[0305] In step S424, the file acquisition unit 411 acquires the immediately preceding value reference difference information of the sample at the processing target timing from outside the client device 400. For example, the file acquisition unit 411 requests the distribution server or the like to supply the immediately preceding value reference difference information of the sample at the processing target timing. Then, the file acquisition unit 411 acquires an update file including the immediately preceding value reference difference information of the sample at the processing target timing supplied in response to the request, and supplies it to the file processing unit 412. The file processing unit 412 acquires the immediately preceding value reference difference information of the sample at the processing target timing from the update file.

[0306] In step S425, the file processing unit 412 updates the scene description using the immediately preceding value reference difference information of the sample at the processing target timing acquired by the process of step S424, and generates the scene description at the processing target timing.

[0307] When the process of step S425 ends, the process returns to step S406 in FIG. 36. Also, in step S422 of FIG. 37, when it is determined that the client process ends, the client process ends.

[0308] As described above, by executing each process, the client device 400 can suppress an increase in the processing load of the client device 400 that randomly accesses the scene description, as described above in <Method 1>.

[0309] Note that when Method 1 is applied in step S404, the file processing unit 412 may apply Method 1-1 described above in <Method 1-1>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire initial value-based difference information stored in a track different from the immediately preceding value-based difference information of the update file. By doing so, the client device 400 can more easily acquire samples of the initial value-based difference information independently of the samples of the immediately preceding value-based difference information by designating a track.

[0310] Also, when Method 1-1 is applied in step S404, the file processing unit 412 may apply Method 1-1-1 described above in <Method 1-1-1>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information using link information indicating a link to a track storing the initial value-based difference information, which is stored as link information for random access in an initial value file storing an initial value of the spatial arrangement information (scene description). By doing so, the client device 400 can more easily identify a track in which appropriate update information is stored.

[0311] Also, when Method 1 is applied in step S404, the file processing unit 412 may apply the Method 1-2 described above in <Method 1-2>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information stored in the same sample of the same track as the immediate value-based difference information of the update file. By doing so, the client device 400 can obtain update information (immediate value-based difference information or initial value-based difference information) using the same link information both during normal playback and during random access.

[0312] Also, when Method 1-2 is applied in step S404, the file processing unit 412 may apply the Method 1-2-1 described above in <Method 1-2-1>. That is, the immediate value-based difference information and the initial value-based difference information may be stored in the sample as a list having each of them as an element. By doing so, the client device 400 can more easily acquire the immediate value-based difference information and the initial value-based difference information.

[0313] Also, when Method 1-2 is applied in step S404, the file processing unit 412 may apply the Method 1-2-2 described above in <Method 1-2-2>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information based on management information (for example, SubSampleInformationBox) that manages each of the immediate value-based difference information and the initial value-based difference information as sub-samples and is stored in the management area of the update file. By doing so, the client device 400 can more easily acquire the immediate value-based difference information and the initial value-based difference information.

[0314] Also, when Method 1-2-2 is applied in step S404, the file processing unit 412 may apply the Method 1-2-2-1 described above in <Method 1-2-2-1>. That is, the above-described management information may include identification information (e.g., codec_specific_parameters) for identifying a sub-sample in which the immediately preceding value-based difference information is stored and a sub-sample in which the initial value-based difference information is stored. By referring to the value of this identification information, the client device 400 can easily (without parsing inside the sample) identify whether it is a sub-sample in which the immediately preceding value-based difference information is stored or a sub-sample in which the initial value-based difference information is stored.

[0315] Also, when Method 1-2 is applied in step S404, the file processing unit 412 may apply the Method 1-2-3 described above in <Method 1-2-3>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information based on management information (e.g., SyncSampleBox) that manages a sample in which the initial value-based difference information is stored in the management area of the update file as a randomly accessible sample. Based on this management information, the client device 400 can more easily identify the sample in which the initial value-based difference information is stored.

[0316] Also, when Method 1 is applied in step S404, the file processing unit 412 may apply the Method 1-3 described above in <Method 1-3>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information stored in a different sample of the same track as the immediately preceding value-based difference information of the update file. By doing so, the client device 400 can select the immediately preceding value-based difference information or the initial value-based difference information by selecting a sample.

[0317] Also, when Method 1-3 is applied in step S404, the file processing unit 412 may apply the Method 1-3-1 described above in <Method 1-3-1>. That is, the sample in which the initial value-based difference information is stored may store the same update application time information as the sample in which the immediately preceding value-based difference information is stored. By doing so, the client device 400 can easily identify the immediately preceding value-based difference information corresponding to the initial value-based difference information without parsing the inside of the sample.

[0318] Also, when Method 1-3 is applied in step S404, the file processing unit 412 may apply the Method 1-3-2 described above in <Method 1-3-2>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information based on management information that manages the sample in which the initial value-based difference information is stored, which is stored in the management area of the update file, as a sample not used during normal playback. By doing so, the client device 400 can easily omit the acquisition of the sample in which the initial value-based difference information is stored (without parsing the inside of the sample) based on this management information during normal playback. In other words, when performing random access, the client device 400 can easily acquire the sample in which the initial value-based difference information is stored (without parsing the inside of the sample) based on this management information.

[0319] Also, when Method 1-3-2 is applied in step S404, the file processing unit 412 may apply the Method 1-3-2-1 described above in <Method 1-3-2-1>. That is, the above-described management information may include identification information (for example, sample_has_redundancy_flag) indicating whether it is a sample used during normal playback. By doing so, the client device 400 can easily omit the acquisition of the sample in which the initial value-based difference information is stored (without parsing the inside of the sample) based on this identification information during normal playback.

[0320] Also, when Method 1-3 is applied in step S404, the file processing unit 412 may apply the Method 1-3-3 described above in <Method 1-3-3>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information based on management information (for example, SyncSampleBox) that manages, as randomly accessible samples, the samples in which the initial value-based difference information to be stored in the management area of the update file is stored. By doing so, the client device 400 can more easily identify the samples in which the initial value-based difference information is stored based on this management information.

[0321] Also, when Method 1 is applied in step S404, the file processing unit 412 may apply the Method 1-4 described above in <Method 1-4>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information in which the immediately preceding value-based difference information whose update application time is the same as the initial value-based difference information is replaced in the update file. By doing so, compared with the cases of Method 1-2 and Method 1-3, the data amount of the update file can be reduced.

[0322] Also, when Method 1-4 is applied in step S404, the file processing unit 412 may apply the Method 1-4-1 described above in <Method 1-4-1>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information based on management information (for example, SyncSampleBox) that manages, as randomly accessible samples, the samples in which the initial value-based difference information to be stored in the management area of the update file is stored. By doing so, the client device 400 can more easily identify the samples in which the initial value-based difference information is stored based on this management information.

[0323] Note that, as long as there is no contradiction, any plurality of the above-described various methods may be combined and applied. Also, the above-described various methods may be combined and applied with any other method not described above.

[0324] <Flow of Client Processing 2> This client device 400 can perform processes related to analysis of scene descriptions, reproduction of 3D object data, etc. by applying the present disclosure described above, for example, in <3. Transmission of Initial Value Reference Difference Information or Updated Information>. For example, the client device 400 can perform processing by applying Method 2 described above. An example of the flow of client processing executed by the client device 400 in that case will be described with reference to the flowcharts of FIGS. 38 and 39.

[0325] When the client processing is started, the file acquisition unit 411 of the client device 400 acquires a scene description file in step S451 of FIG. 38. The initial value of the scene description is stored in this scene description file. The file acquisition unit 411 supplies the acquired scene description file to the file processing unit 412. The file acquisition unit 411 extracts the scene description (initial value) from the scene description file.

[0326] In step S452, the control unit 401 determines whether to perform random access to the scene description. For example, if it is determined to perform random access based on an instruction from a user, application, etc., the process proceeds to step S453.

[0327] In step S453, the file processing unit 412 analyzes the scene description and identifies a random access point as the processing target timing. That is, the file processing unit 412 identifies a sample for random access from among the prepared random access points.

[0328] In step S454, the file processing unit 412 requests the file acquisition unit 411 to acquire the initial value reference difference information or the updated information of the random access point specified in step S453. The file acquisition unit 411 acquires an updated file including the requested initial value reference difference information or updated information from outside the client device 400, such as a distribution server or the file generation device 300. That is, the file acquisition unit 411 requests a distribution server or the like to supply an updated file including the initial value reference difference information or updated information requested by the file processing unit 412. Then, the file acquisition unit 411 acquires the supplied updated file (initial value reference difference information or updated information) in response to the request. The file processing unit 412 applies the method 2 described above in <Method 2>, extracts and acquires the initial value reference difference information or updated information from the acquired updated file.

[0329] In step S455, the file processing unit 412 applies the method 2 described above in <Method 2>, updates the scene description using the acquired initial value reference difference information or updated information, and generates a scene description at the processing target timing. That is, the file processing unit 412 generates a scene description (spatial arrangement information) at the time of applying the update of the initial value reference difference information or updated information (that is, at the processing target timing) by, for example, reflecting the update of the initial value reference difference information in the initial value of the scene description or applying the updated information.

[0330] When the process of step S455 ends, the process proceeds to step S456. Also, in step S452, if it is determined that the access is not a random access (that is, normal playback), the process proceeds to step S456.

[0331] In step S456, based on the scene description at the processing target timing, the file processing unit 412 requests the file acquisition unit 411 to acquire the encoded data of the 3D object at the processing target timing. The file acquisition unit 411 acquires, from outside the client device 400, a 3D object file that stores the encoded data of the 3D object at the requested processing target timing. For example, the file acquisition unit 411 requests a distribution server or the like to supply a 3D object file including the encoded data of the 3D object requested by the file processing unit 412. Then, the file acquisition unit 411 acquires the supplied 3D object file in response to the request and supplies it to the file processing unit 412. The file processing unit 412 extracts the encoded data of the 3D object at the processing target timing from the 3D object file.

[0332] In step S457, the decoding unit 413 decodes the encoded data of the 3D object at the processing target timing obtained in step S456.

[0333] In step S458, the display information generation unit 414 performs processing related to the generation of a display image according to the control of the display control unit 416. For example, based on the scene description at the processing target timing obtained in step S455, the display information generation unit 414 arranges the 3D object at the processing target timing obtained in step S457 in a 3D space and performs rendering to generate a display image.

[0334] In step S459, the display unit 415 displays the display image generated in step S458. When the processing of step S459 ends, the processing proceeds to step S471 in FIG. 39.

[0335] In step S471 of FIG. 39, the control unit 401 determines whether to perform a random access. If it is determined to perform a random access, the process proceeds to step S453 of FIG. 38. Also, in step S471 of FIG. 39, if it is determined not to perform a random access (i.e., to perform normal playback), the process proceeds to step S472.

[0336] In step S472, the control unit 401 determines whether to end the client process. If it is determined not to end the client process, the process proceeds to step S473.

[0337] In step S473, the control unit 401 switches the processing target timing to the next timing. That is, the control unit 401 switches the processing target sample to the next sample.

[0338] In step S474, the file acquisition unit 411 acquires the immediately preceding value reference difference information of the sample at the processing target timing from outside the client device 400. For example, the file acquisition unit 411 requests the delivery server or the like to supply the immediately preceding value reference difference information of the sample at the processing target timing. Then, the file acquisition unit 411 acquires the update file including the immediately preceding value reference difference information of the sample at the processing target timing supplied in response to the request, and supplies it to the file processing unit 412. The file processing unit 412 acquires the immediately preceding value reference difference information of the sample at the processing target timing from the update file.

[0339] In step S475, the file processing unit 412 updates the scene description using the immediately preceding value reference difference information of the sample at the processing target timing acquired by the processing in step S474, and generates the scene description at the processing target timing.

[0340] When the processing in step S475 ends, the process returns to step S456 of FIG. 38. Also, in step S472 of FIG. 39, if it is determined to end the client process, the client process ends.

[0341] As described above, by executing each process, the client device 400 can suppress an increase in the processing load of the client device 400 that randomly accesses the scene description as described above in <Method 2>.

[0342] In addition, when Method 2 is applied in step S454, the file processing unit 412 may apply Method 2-1 described above in <Method 2-1>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire initial value-based difference information or updated information stored in a track different from the immediately preceding value-based difference information of the update file. By doing so, the client device 400 can more easily acquire samples of the initial value-based difference information or the updated information independently of the samples of the immediately preceding value-based difference information by designating a track.

[0343] Also, when Method 2-1 is applied in step S454, the file processing unit 412 may apply Method 2-1-1 described above in <Method 2-1-1>. That is, the file processing unit 412, which is a difference information acquisition unit, may use link information indicating a link to a track storing the initial value-based difference information or the updated information, which is stored as link information for random access in an initial value file storing the initial value of the spatial arrangement information (scene description), to acquire the initial value-based difference information or the updated information. By doing so, the client device 400 can more easily identify a track in which appropriate update information is stored.

[0344] Also, when Method 2 is applied in step S454, the file processing unit 412 may apply Method 2-2 described above in <Method 2-2>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information or updated information stored in the same sample of the same track as the immediately preceding value-based difference information of the update file. By doing so, the client device 400 can obtain update information (immediately preceding value-based difference information, initial value-based difference information, or updated information) using the same link information both during normal playback and during random access.

[0345] Also, when Method 2-2 is applied in step S454, the file processing unit 412 may apply Method 2-2-1 described above in <Method 2-2-1>. That is, the immediately preceding value-based difference information and the initial value-based difference information or updated information may be stored in the sample as lists each having these as elements. By doing so, the client device 400 can more easily acquire the immediately preceding value-based difference information and the initial value-based difference information or updated information.

[0346] Also, when Method 2-2 is applied in step S454, the file processing unit 412 may apply Method 2-2-2 described above in <Method 2-2-2>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information based on management information (for example, SubSampleInformationBox) that manages each of the immediately preceding value-based difference information and the initial value-based difference information or updated information as sub-samples and is stored in the management area of the update file. By doing so, the client device 400 can more easily acquire the immediately preceding value-based difference information, the initial value-based difference information, and the updated information.

[0347] Also, when Method 2-2-2 is applied in step S454, the file processing unit 412 may apply the Method 2-2-2-1 described above in <Method 2-2-2-1>. That is, the above-described management information may include identification information (for example, codec_specific_parameters) that identifies a subsample in which the immediately preceding value-based difference information is stored and a subsample in which the initial value-based difference information or the updated information is stored. By referring to the value of this identification information, the client device 400 can easily (without parsing the inside of the sample) identify whether it is a subsample in which the immediately preceding value-based difference information is stored or a subsample in which the initial value-based difference information or the updated information is stored.

[0348] Also, when Method 2-2 is applied in step S454, the file processing unit 412 may apply the Method 2-2-3 described above in <Method 2-2-3>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information or the updated information based on management information (for example, SyncSampleBox) that manages a sample in which the initial value-based difference information or the updated information is stored in the management area of the update file as a randomly accessible sample. Based on this management information, the client device 400 can more easily identify a sample in which the initial value-based difference information or the updated information is stored.

[0349] Also, when Method 2 is applied in step S454, the file processing unit 412 may apply the Method 2-3 described above in <Method 2-3>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information or the updated information stored in a different sample of the same track as the immediately preceding value-based difference information of the update file. By doing so, the client device 400 can select the immediately preceding value-based difference information, the initial value-based difference information, or the updated information by selecting a sample.

[0350] Also, when Method 2-3 is applied in step S454, the file processing unit 412 may apply the Method 2-3-1 described above in <Method 2-3-1>. That is, in the sample where the initial value-based difference information or the updated information is stored, the same update application time information as that in the sample where the immediately preceding value-based difference information is stored may be stored. By doing so, the client device 400 can easily identify the immediately preceding value-based difference information corresponding to the initial value-based difference information or the updated information without parsing the inside of the sample.

[0351] Also, when Method 2-3 is applied in step S454, the file processing unit 412 may apply the Method 2-3-2 described above in <Method 2-3-2>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information or the updated information based on the management information that manages the sample in which the initial value-based difference information or the updated information is stored in the management area of the update file as a sample not used during normal playback. By doing so, the client device 400 can easily omit the acquisition of the sample in which the initial value-based difference information or the updated information is stored (without parsing the inside of the sample) based on this management information during normal playback. In other words, when performing random access, the client device 400 can easily acquire the sample in which the initial value-based difference information or the updated information is stored (without parsing the inside of the sample) based on this management information.

[0352] Also, when Method 2-3-2 is applied in step S454, the file processing unit 412 may apply the Method 2-3-2-1 described above in <Method 2-3-2-1>. That is, the above-described management information may include identification information (for example, sample_has_redundancy_flag) indicating whether it is a sample used during normal playback. By doing so, the client device 400 can easily omit the acquisition of the sample in which the initial value-based difference information or the updated information is stored (without parsing the inside of the sample) based on this identification information during normal playback.

[0353] Also, when Method 2-3 is applied in step S454, the file processing unit 412 may apply the Method 2-3-3 described above in <Method 2-3-3>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information or the updated information based on management information (for example, SyncSampleBox) that manages samples in which the initial value-based difference information or the updated information stored in the management area of the update file can be randomly accessed as random accessable samples. By doing so, the client device 400 can more easily identify a sample in which the initial value-based difference information or the updated information is stored based on this management information.

[0354] Also, when Method 2 is applied in step S454, the file processing unit 412 may apply the Method 2-4 described above in <Method 2-4>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information or the updated information in which the time of applying the update is replaced with the immediately preceding value-based difference information that is the same as the initial value-based difference information or the updated information, and is stored in the update file. By doing so, compared with the cases of Method 2-2 and Method 2-3, the data amount of the update file can be reduced.

[0355] Also, when Method 2-4 is applied in step S454, the file processing unit 412 may apply the Method 2-4-1 described above in <Method 2-4-1>. That is, the file processing unit 412, which is a difference information acquisition unit, may acquire the initial value-based difference information or the updated information based on management information (for example, SyncSampleBox) that manages samples in which the initial value-based difference information or the updated information stored in the management area of the update file can be randomly accessed as random accessable samples. By doing so, the client device 400 can more easily identify a sample in which the initial value-based difference information or the updated information is stored based on this management information.

[0356] As long as there is no contradiction, any combination of multiple methods among the various methods described above may be applied. Also, the various methods described above may be applied in combination with any other method not described above.

[0357] <6. Supplementary Note> <Computer> The series of processes described above can be executed by hardware or by software. When the series of processes is executed by software, the program constituting the software is installed in a computer. Here, the computer includes a computer incorporated in dedicated hardware, and, for example, a general-purpose personal computer that can execute various functions by installing various programs.

[0358] FIG. 40 is a block diagram showing a configuration example of the hardware of a computer that executes the series of processes described above by a program.

[0359] In the computer 900 shown in FIG. 40, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0360] An input / output interface 910 is also connected to the bus 904. An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input / output interface 910.

[0361] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0362] In the computer configured as described above, the CPU 901 loads and executes, for example, a program stored in the storage unit 913 via the input / output interface 910 and the bus 904 into the RAM 903, thereby performing the series of processes described above. The RAM 903 also appropriately stores data and the like necessary for the CPU 901 to execute various processes.

[0363] The program executed by the computer can be recorded and applied, for example, on a removable medium 921 as a package medium or the like. In that case, the program can be installed in the storage unit 913 via the input / output interface 910 by mounting the removable medium 921 on the drive 915.

[0364] Also, this program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In that case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0365] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913.

[0366] <Objects to Which the Present Technology Can Be Applied> The present technology can be applied to any image encoding / decoding method.

[0367] In addition, the present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices.

[0368] Also, for example, the present technology can be implemented as a part of a configuration of a device such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using a plurality of processors, a unit (e.g., a video unit) using a plurality of modules, or a set (e.g., a video set) with other functions added to the unit.

[0369] Also, for example, the present technology can be applied to a network system composed of a plurality of devices. For example, the present technology may be implemented as cloud computing that is processed in a shared and collaborative manner by a plurality of devices via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (moving images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, and an IoT (Internet of Things) device.

[0370] Note that in this specification, a system means a collection of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, both a plurality of devices housed in separate housings and connected via a network, and a single device in which a plurality of modules are housed in one housing are systems.

[0371] <Fields and Applications Applicable to the Present Technology> Systems, devices, processing units, etc. to which the present technology is applied can be used in any fields such as transportation, medical care, crime prevention, agriculture, livestock industry, mining, beauty, factories, home appliances, meteorology, natural monitoring, etc. Also, their applications are arbitrary.

[0372] For example, the present technology can be applied to systems and devices used for providing ornamental content or the like. Also, for example, the present technology can also be applied to systems and devices used for transportation, such as traffic situation monitoring and automatic driving control. Further, for example, the present technology can also be applied to systems and devices used for security. Also, for example, the present technology can be applied to systems and devices used for automatic control of machines and the like. Further, for example, the present technology can also be applied to systems and devices used for agriculture and livestock industry. Also, the present technology can be applied to systems and devices for monitoring natural conditions such as volcanoes, forests, oceans, and wild organisms. Further, for example, the present technology can also be applied to systems and devices used for sports.

[0373] <Others> In this specification, the "flag" is information for identifying a plurality of states, and includes not only information used for identifying two states of true (1) or false (0), but also information capable of identifying three or more states. Therefore, the values that this "flag" can take may be, for example, two values of 1 / 0, or three or more values. That is, the number of bits constituting this "flag" is arbitrary and may be 1 bit or a plurality of bits. Also, identification information (including flags) is assumed not only in the form of including the identification information in a bit stream, but also in the form of including the difference information of the identification information with respect to a certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also the difference information with respect to the reference information.

[0374] Also, various types of information (such as metadata) related to the encoded data (bitstream) may be transmitted or recorded in any form as long as it is associated with the encoded data. Here, the term "associate" means, for example, enabling the use (linking) of one data when processing the other data. That is, the data associated with each other may be grouped as one data or may be individual data. For example, the information associated with the encoded data (image) may be transmitted on a transmission path different from that of the encoded data (image). Also, for example, the information associated with the encoded data (image) may be recorded on a recording medium different from that of the encoded data (image) (or a different recording area of the same recording medium). Note that this "association" may be for a part of the data instead of the entire data. For example, an image and the information corresponding to the image may be associated with each other in any unit such as a plurality of frames, one frame, or a part within a frame.

[0375] Note that in this specification, terms such as "synthesize", "multiplex", "add", "integrate", "include", "store", "insert", "plug in", "insert" mean, for example, grouping a plurality of things into one, such as grouping encoded data and metadata into one data, and mean one method of the above-mentioned "associate".

[0376] Also, the embodiments of the present technology are not limited to the above-described embodiments, and various changes are possible without departing from the gist of the present technology.

[0377] For example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configuration described as a plurality of devices (or processing units) above may be combined and configured as one device (or processing unit). Of course, a configuration other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or another processing unit).

[0378] Also, for example, the program described above may be executed on any device. In that case, it suffices that the device has necessary functions (such as function blocks) and can obtain necessary information.

[0379] Also, for example, each step of one flowchart may be executed by one device, or may be executed by a plurality of devices sharing the work. Furthermore, when a plurality of processes are included in one step, the plurality of processes may be executed by one device, or may be executed by a plurality of devices sharing the work. In other words, the plurality of processes included in one step can also be executed as processes of a plurality of steps. Conversely, the processes described as a plurality of steps can also be executed as one step combined together.

[0380] Also, for example, the program executed by a computer may be such that the processing of the steps of writing the program is executed in time series along the order described in this specification, or may be executed in parallel, or individually executed at a necessary timing such as when a call is made. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of writing this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0381] Also, for example, a plurality of technologies related to the present technology can be independently implemented individually as long as there is no contradiction. Of course, any plurality of the present technologies can also be implemented in combination. For example, part or all of the present technology described in any one of the embodiments can be implemented in combination with part or all of the present technology described in other embodiments. Also, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0382] In addition, the present technology can also adopt the following configuration. (1) An update information generation unit that generates initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information, as update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space, A file generation unit that generates an update file for storing the update information and stores the initial value reference difference information in the update file as a random access point An information processing apparatus comprising: (2) The file generation unit stores the initial value reference difference information in a track different from the immediately preceding value reference difference information of the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected The information processing apparatus according to (1). (3) The file generation unit generates an initial value file for storing the initial value of the spatial arrangement information, and stores link information indicating a link to the track for storing the initial value reference difference information as the link information for random access The information processing apparatus according to (2). (4) The file generation unit stores the initial value reference difference information in the same sample of the same track as the immediately preceding value reference difference information of the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected The information processing apparatus according to (1). (5) The file generation unit stores the immediately preceding value reference difference information and the initial value reference difference information as lists each having the information as an element. The information processing apparatus according to (4). (6) The file generation unit stores, in a management area of the update file, management information for managing each of the immediately preceding value reference difference information and the initial value reference difference information as subsamples. The information processing apparatus according to (4) or (5). (7) The management information includes identification information for identifying the subsample in which the immediately preceding value reference difference information is stored and the subsample in which the initial value reference difference information is stored. The information processing apparatus according to (6). (8) The file generation unit stores, in a management area of the update file, management information for managing the sample in which the initial value reference difference information is stored as a randomly accessible sample. The information processing apparatus according to any one of (4) to (7). (9) The file generation unit stores the initial value reference difference information in a different sample of the same track as the immediately preceding value reference difference information of the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected. The information processing apparatus according to (1). (10) The file generation unit stores the same update application time information as that of the sample in which the immediately preceding value reference difference information is stored in the sample in which the initial value reference difference information is stored. The information processing apparatus according to (9). (11) The file generation unit stores, in a management area of the update file, management information for managing the sample in which the initial value reference difference information is stored as a sample not used during normal reproduction. The information processing apparatus according to (9) or (10). (12) The management information includes identification information indicating whether the sample is a sample used during normal reproduction. The information processing apparatus according to (11). (13) The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information is stored as a randomly accessible sample. The information processing apparatus according to any one of (9) to (12). (14) The file generation unit replaces, in the update file, the immediately preceding value reference difference information, in which the update application time is the same as that of the initial value reference difference information, with the initial value reference difference information. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected. The information processing apparatus according to (1). (15) The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information is stored as a randomly accessible sample. The information processing apparatus according to (14). (16) Generate initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information, as update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space. Generate an update file for storing the update information, and store the initial value reference difference information in the update file as a random access point. Information processing method.

[0383] (21) A difference information acquisition unit that acquires initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information, stored as a random access point in an update file for storing update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space. A spatial arrangement information generation unit that generates the spatial arrangement information at the update application time of the initial value reference difference information by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information. An information processing apparatus comprising the above. (22) The difference information acquisition unit acquires the initial value-based difference information stored in a track different from the immediately preceding value-based difference information of the update file, The immediately preceding value-based difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to (21). (23) The difference information acquisition unit uses the link information indicating a link to the track storing the initial value-based difference information, which is stored as link information for random access in an initial value file storing the initial value of the spatial arrangement information, to acquire the initial value-based difference information. The information processing apparatus according to (22). (24) The difference information acquisition unit acquires the initial value-based difference information stored in the same sample of the same track as the immediately preceding value-based difference information of the update file, The immediately preceding value-based difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to (21). (25) The immediately preceding value-based difference information and the initial value-based difference information are stored in the sample as lists each having the information as an element. The information processing apparatus according to (24). (26) The difference information acquisition unit acquires the initial value-based difference information based on management information that manages each of the immediately preceding value-based difference information and the initial value-based difference information as subsamples and is stored in a management area of the update file. The information processing apparatus according to (24) or (25). (27) The management information includes identification information for identifying the subsample storing the immediately preceding value-based difference information and the subsample storing the initial value-based difference information. The information processing apparatus according to (26). (28) The difference information acquisition unit acquires the initial value reference difference information based on management information that manages, as samples that can be randomly accessed, the samples in which the initial value reference difference information is stored and that are stored in the management area of the update file. The information processing apparatus according to any one of (24) to (27). (29) The difference information acquisition unit acquires the initial value reference difference information stored in different samples of the same track as the immediately preceding value reference difference information of the update file. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to (21). (30) In the sample in which the initial value reference difference information is stored, the same update application time information as that of the sample in which the immediately preceding value reference difference information is stored is stored. The information processing apparatus according to (29). (31) The difference information acquisition unit acquires the initial value reference difference information based on management information that manages, as samples not used during normal playback, the samples in which the initial value reference difference information is stored and that are stored in the management area of the update file. The information processing apparatus according to (29) or (30). (32) The management information includes identification information indicating whether the sample is a sample used during normal playback. The information processing apparatus according to (31). (33) The difference information acquisition unit acquires the initial value reference difference information based on management information that manages, as samples that can be randomly accessed, the samples in which the initial value reference difference information is stored and that are stored in the management area of the update file. The information processing apparatus according to any one of (29) to (32). (34) The difference information acquisition unit acquires the initial value reference difference information stored in the update file, in which the initial value reference difference information is replaced with the immediately preceding value reference difference information having the same update application time as the initial value reference difference information. The immediately preceding value-based difference information is difference information based on the spatial arrangement information reflecting the updates up to the immediately preceding update information. The information processing apparatus according to (21). (35) The difference information acquisition unit acquires the initial value-based difference information based on management information that manages, as samples that can be randomly accessed, the samples in which the initial value-based difference information is stored and that are stored in the management area of the update file. The information processing apparatus according to (34). (36) Acquire initial value-based difference information, which is difference information based on the initial value of the spatial arrangement information, stored as a random access point in an update file that stores update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space. By reflecting the update of the initial value-based difference information in the initial value of the spatial arrangement information, generate the spatial arrangement information at the update application time of the initial value-based difference information. Information processing method.

[0384] (41) An update information generation unit that generates, as update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, initial value-based difference information, which is difference information based on the initial value of the spatial arrangement information, or updated information in which the update of the initial value-based difference information is reflected in the initial value of the spatial arrangement information. A file generation unit that generates an update file for storing the update information and stores the initial value-based difference information or the updated information in the update file as a random access point. An information processing apparatus comprising the above. (42) The file generation unit stores the initial value-based difference information or the updated information in a track different from the immediately preceding value-based difference information of the update file. The immediately preceding value-based difference information is difference information based on the spatial arrangement information reflecting the updates up to the immediately preceding update information. The information processing apparatus according to (41). (43) The file generation unit generates an initial value file for storing the initial value of the spatial arrangement information, and stores link information indicating a link to the track for storing the initial value reference difference information or the updated information as the link information for random access. The information processing apparatus according to (42). (44) The file generation unit stores the initial value reference difference information or the updated information in the same sample of the same track as the immediately preceding value reference difference information of the update file. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which updates up to the immediately preceding update information are reflected. The information processing apparatus according to (41). (45) The file generation unit stores the immediately preceding value reference difference information and the initial value reference difference information or the updated information as lists each having these as elements. The information processing apparatus according to (44). (46) The file generation unit stores management information for managing each of the immediately preceding value reference difference information and the initial value reference difference information or the updated information as subsamples in the management area of the update file. The information processing apparatus according to (44) or (45). (47) The management information includes identification information for identifying the subsample in which the immediately preceding value reference difference information is stored and the subsample in which the initial value reference difference information or the updated information is stored. The information processing apparatus according to (46). (48) The file generation unit stores management information for managing the sample in which the initial value reference difference information or the updated information is stored as a randomly accessible sample in the management area of the update file. The information processing apparatus according to any one of (44) to (47). (49) The file generation unit stores the initial value reference difference information or the updated information in a different sample of the same track as the immediately preceding value reference difference information of the update file. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to (41). (50) The file generation unit stores the same update application time information as that of the sample in which the immediately preceding value reference difference information is stored in the sample in which the initial value reference difference information or the updated information is stored. The information processing apparatus according to (49). (51) The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information or the updated information is stored as a sample not used during normal reproduction. The information processing apparatus according to (49) or (50). (52) The management information includes identification information indicating whether the sample is a sample used during normal reproduction. The information processing apparatus according to (51). (53) The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information or the updated information is stored as a randomly accessible sample. The information processing apparatus according to any one of (49) to (52). (54) The file generation unit replaces the immediately preceding value reference difference information, whose update application time is the same as that of the initial value reference difference information, stored in the update file with the initial value reference difference information or the updated information. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to (41). (55) The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information or the updated information is stored as a randomly accessible sample. The information processing apparatus according to (54). As update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, initial value reference difference information that is difference information based on an initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information is generated. An update file for storing the update information is generated, and the initial value reference difference information or the updated information is stored in the update file as a random access point. An information processing method.

[0385] (61) An initial value reference difference information that is difference information based on an initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, which is stored as a random access point in an update file for storing update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space, is acquired as the update information by an update information acquisition unit. A spatial arrangement information generation unit that generates the spatial arrangement information at the update application time of the initial value reference difference information by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information or by applying the updated information. An information processing apparatus comprising the same. (62) The update information acquisition unit acquires the initial value reference difference information or the updated information stored in a track different from the immediately preceding value reference difference information of the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected. The information processing apparatus according to (61). (63) The update information acquisition unit uses link information indicating a link to the track storing the initial value reference difference information or the updated information, which is stored as random access link information in an initial value file storing the initial value of the spatial arrangement information, to acquire the initial value reference difference information or the updated information. The information processing apparatus according to (62). (64) The update information acquisition unit acquires the initial value reference difference information or the updated information stored in the same sample of the same track as the immediately preceding value reference difference information of the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to (61). (65) The immediately preceding value reference difference information and the initial value reference difference information or the updated information are stored in the sample as lists each having the respective information as elements. The information processing apparatus according to (64). (66) The update information acquisition unit acquires the initial value reference difference information or the updated information based on management information that manages, as subsamples, the immediately preceding value reference difference information and the initial value reference difference information or the updated information, each stored in a management area of the update file. The information processing apparatus according to (64) or (65). (67) The management information includes identification information for identifying the subsample in which the immediately preceding value reference difference information is stored and the subsample in which the initial value reference difference information or the updated information is stored. The information processing apparatus according to (66). (68) The update information acquisition unit acquires the initial value reference difference information or the updated information based on management information that manages, as a randomly accessible sample, the sample in which the initial value reference difference information or the updated information is stored in a management area of the update file. The information processing apparatus according to any one of (64) to (67). (69) The update information acquisition unit acquires the initial value reference difference information or the updated information stored in a different sample of the same track as the immediately preceding value reference difference information of the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to (61). (70) In the sample in which the initial value reference difference information or the updated information is stored, the same update application time information as that of the sample in which the immediately preceding value reference difference information is stored is stored. The information processing apparatus according to (69). (71) The update information acquisition unit acquires the initial value reference difference information or the updated information based on management information that manages, as samples not used during normal reproduction, the samples in which the initial value reference difference information or the updated information is stored and that are stored in the management area of the update file. The information processing apparatus according to (69) or (70). (72) The management information includes identification information indicating whether or not the sample is a sample used during normal reproduction. The information processing apparatus according to (71). (73) The update information acquisition unit acquires the initial value reference difference information or the updated information based on management information that manages, as samples that can be randomly accessed, the samples in which the initial value reference difference information or the updated information is stored and that are stored in the management area of the update file. The information processing apparatus according to any one of (69) to (72). (74) The update information acquisition unit acquires the initial value reference difference information or the updated information in which the immediately preceding value reference difference information whose update application time is the same as that of the initial value reference difference information is replaced, and is stored in the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected. The information processing apparatus according to (61). (75) The update information acquisition unit acquires the initial value reference difference information or the updated information based on management information that manages, as samples that can be randomly accessed, the samples in which the initial value reference difference information or the updated information is stored and that are stored in the management area of the update file. The information processing apparatus according to (74). Initial value reference difference information, which is difference information based on the initial value of the spatial arrangement information, stored as a random access point in an update file that stores update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, is acquired as the update information, The spatial arrangement information at the update application time of the initial value reference difference information is generated by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information or by applying the updated information. Information processing method.

Explanation of Signs

[0386] 300 File generation device, 301 Control unit, 302 File generation processing unit, 311 Input unit, 312 Preprocessing unit, 313 Encoding unit, 314 File generation unit, 315 Recording unit, 316 Output unit, 400 Client device, 401 Control unit, 402 Client processing unit, 411 File acquisition unit, 412 File processing unit, 413 Decoding unit, 414 Display information generation unit, 415 Display unit, 416 Display control unit

Claims

1. As update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, initial value reference difference information which is difference information based on the initial value of the spatial arrangement information, or an update information generation unit that generates updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, a file generation unit that generates an update file for storing the update information and stores the initial value reference difference information or the updated information in the update file as a random access point An information processing apparatus comprising:

2. The file generation unit stores the initial value reference difference information or the updated information in a track different from the immediately preceding value reference difference information of the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected The information processing apparatus according to claim 1.

3. The file generation unit generates an initial value file for storing the initial value of the spatial arrangement information, and stores link information indicating a link to the track for storing the initial value reference difference information or the updated information as the link information for random access The information processing apparatus according to claim 2.

4. The file generation unit stores the initial value reference difference information or the updated information in the same sample of the same track as the immediately preceding value reference difference information of the update file, The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected The information processing apparatus according to claim 1.

5. The file generation unit stores the immediately preceding value reference difference information and the initial value reference difference information or the updated information as lists each having them as elements The information processing apparatus according to claim 4.

6. The file generation unit stores management information for managing each of the immediately preceding value reference difference information and the initial value reference difference information or the updated information as subsamples in a management area of the update file The information processing apparatus according to claim 4.

7. The management information includes identification information for identifying the subsample in which the immediately preceding value reference difference information is stored and the subsample in which the initial value reference difference information or the updated information is stored The information processing apparatus according to claim 6.

8. The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information or the updated information is stored as a sample that can be randomly accessed. The information processing apparatus according to claim 4.

9. The file generation unit stores the initial value reference difference information or the updated information in a different sample on the same track as the immediately preceding value reference difference information of the update file. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to claim 1.

10. The file generation unit stores the same update application time information as the sample in which the immediately preceding value reference difference information is stored in the sample in which the initial value reference difference information or the updated information is stored. The information processing apparatus according to claim 9.

11. The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information or the updated information is stored as a sample not used during normal playback. The information processing apparatus according to claim 9.

12. The management information includes identification information indicating whether the sample is a sample used during normal playback. The information processing apparatus according to claim 11.

13. The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information or the updated information is stored as a sample that can be randomly accessed. The information processing apparatus according to claim 9.

14. The file generation unit replaces the immediately preceding value reference difference information, in which the update application time is the same as the initial value reference difference information, stored in the update file with the initial value reference difference information or the updated information. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the updates up to the immediately preceding update information are reflected. The information processing apparatus according to claim 1.

15. The file generation unit stores, in the management area of the update file, management information for managing the sample in which the initial value reference difference information or the updated information is stored as a sample that can be randomly accessed. The information processing apparatus according to claim 14.

16. As update information for updating spatial arrangement information for arranging one or more 3D objects in a 3D space, initial value reference difference information that is difference information based on the initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information is generated. An update file for storing the update information is generated, and the initial value reference difference information or the updated information is stored in the update file as a random access point. An information processing method.

17. An update information acquisition unit that acquires, as the update information, initial value reference difference information that is difference information based on the initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, and that is stored as a random access point in an update file that stores update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space. A spatial arrangement information generation unit that generates the spatial arrangement information at the update application time of the initial value reference difference information by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information or by applying the updated information. An information processing apparatus comprising the above.

18. The update information acquisition unit acquires the initial value reference difference information or the updated information stored in the same sample of the same track as the immediately preceding value reference difference information of the update file. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected. The information processing apparatus according to Claim 17.

19. The update information acquisition unit acquires the initial value reference difference information or the updated information stored in a different sample of the same track as the immediately preceding value reference difference information of the update file. The immediately preceding value reference difference information is difference information based on the spatial arrangement information in which the update up to the immediately preceding update information is reflected. The information processing apparatus according to Claim 17.

20. Initial value reference difference information that is difference information based on the initial value of the spatial arrangement information, or updated information in which the update of the initial value reference difference information is reflected in the initial value of the spatial arrangement information, and that is stored as a random access point in an update file that stores update information for updating the spatial arrangement information for arranging one or more 3D objects in a 3D space is acquired as the update information. Generate the spatial arrangement information at the update application time of the initial value reference difference information by reflecting the update of the initial value reference difference information in the initial value of the spatial arrangement information or by applying the updated information. Information processing method.

Images