Information processing device, information processing method, reproduction processing device and reproduction processing method

Abstract

An information processing device, a reproduction processing device, an information processing method, and a reproduction processing method are provided, by which a client device can efficiently select a content configuration. A preprocessing unit generates content configuration selection information for determining whether a reproduction process of the content is possible for one or more contents having a content configuration representing a virtual space, the content configuration being configured by one or more three-dimensional objects and spatial arrangement information thereof. A file generation unit generates a file containing data of the virtual space and the content configuration selection information.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method, a reproduction processing device and a reproduction processing method. Background Art

[0002] In current video distribution, two-dimensional videos for distributing movies and the like are mainly distributed. Hereinafter, two-dimensional videos may be referred to as two-dimensional (2D) content. In addition, 360-degree videos that can be viewed in all directions are also published on video publishing sites on the Internet. The ability to view in all directions indicates that the direction of sight can be freely selected. 360-degree videos are referred to as 3-degree-of-freedom (3DoF) videos or 3DoF content. In both 2D content and 3DoF content, the video of the two-dimensional encoding is basically distributed from a distribution server and displayed to the client.

[0003] There is also content called 3DoF+ content. 3DoF+ content is content that can be viewed in all directions similar to 3DoF content, and also allows a slight shift in the viewpoint position. It is assumed that the shift in the viewpoint position in 3DoF+ content is allowed within the range that the user can move his head in a seated state. In 3DoF+ content, the shift in the viewpoint position is achieved by using one or more two-dimensionally encoded videos.

[0004] In addition, the distribution of 6DoF videos, which are called 6DoF content, has been proposed as videos with a higher degree of freedom. 6DoF videos are videos that can be viewed in all directions in a three-dimensional space and can be viewed by walking around in the displayed three-dimensional space. Walking around in a three-dimensional space means that the viewpoint position can be freely selected. Hereinafter, a three-dimensional space may be referred to as a 3D space.

[0005] 6DoF content is three-dimensional content, in which a three-dimensional space is represented by one or more three-dimensional model data. The three-dimensional model data may be referred to as 3D model data, and the three-dimensional content may also be referred to as 3D content.

[0006] As an example of a method of distributing 6DoF content, the 6DoF content is sent as multiple object streams by configuring a three-dimensional space using multiple three-dimensional model data. At this time, configuration information about the three-dimensional space called scene description can be used. An example thereof is Moving Picture Experts Group (MPEG)-4 scene description. Scene description as a representation method is a method in which a scene is represented by a graph having a tree hierarchical structure called a scene graph, and the scene graph is represented in a binary format.

[0007] 6DoF content is a video material that represents a three-dimensional space using three-dimensional model data at each time. Examples of schemes for representing 6DoF content include the following three schemes.

[0008] One scheme is a representation scheme referred to as an object-based representation scheme in the present invention. In the object-based representation scheme, 6DoF content has a content configuration in which three-dimensional model data of each three-dimensional object such as a person or an object is arranged in a three-dimensional space to represent the entire three-dimensional space, and the three-dimensional object is an individual target object to be displayed in the video. The object-based representation scheme is characterized in that a client that performs reproduction of 6DoF content simultaneously processes the maximum number of three-dimensional model data among the three schemes. On the other hand, in the object-based representation scheme, the definition can be changed when displaying each three-dimensional object such as an individual person or an object. Therefore, it can be said that the object-based representation scheme is a configuration method in which, among the three methods, the client has a high degree of freedom in the reproduction process.

[0009] Another scheme is what is referred to as a space-based representation scheme in the present invention. In the space-based representation scheme, the 6DoF content has a content configuration in which the entire target three-dimensional space is represented as one three-dimensional model data without separating each three-dimensional object such as a person or an object into three-dimensional model data. The space-based representation scheme is characterized in that, among the three schemes, the client processes one three-dimensional model data when reproducing, requiring the lowest processing power. On the other hand, since the definition of the entire 6DoF content is fixed, it can be said that the client has extremely low degrees of freedom in the reproduction process.

[0010] Another scheme is a combination of a space-based representation scheme and an object-based representation scheme. Hereinafter, the representation scheme will be referred to as a hybrid representation scheme. In the hybrid representation scheme, the 6DoF content has a content configuration in which a specific three-dimensional object is represented by an independent three-dimensional model data, and a three-dimensional space other than the specific three-dimensional object is represented by one three-dimensional model data. In the hybrid representation scheme, the client uses multiple three-dimensional model data in the reproduction process, but the number of three-dimensional model data is less than the number of three-dimensional model data used in the object-based representation scheme. That is, in the hybrid representation scheme, the client is required to have a higher processing capability than that in the space-based representation scheme, but may have a lower processing capability than that in the object-based representation scheme. In addition, it can be similarly said that the client has a higher degree of freedom in the reproduction process than that in the space-based representation scheme, and has a lower degree of freedom in the reproduction process than that in the object-based representation scheme.

[0011] As described above, in each of the representation schemes, the 6DoF content has different content configurations. At this point, when creating a scene description to include several 6DoF contents in different representation schemes, it is preferred that the client selects the content configuration of the representation scheme with the highest degree of freedom as possible, so that the user's viewing experience can be further expanded.

[0012] Reference List

[0013] Non-patent literature

[0014] Non-patent document 1: "ISO / IEC 14496-11:2015", Information technology. Coding of audiovisual objects. Part 11: Sene description and application engine, 2015-11 Summary of the invention

[0015] Technical issues

[0016] However, in the case of selecting an appropriate content configuration from a scene description, the client selects the content configuration after performing various analyses. Examples of the analysis include analysis of the entire scene description, analysis of AdaptationSets in a media presentation description (MPD), etc. Such analysis includes analysis of parts that are not actually used, and therefore it can be said that the client device is less efficient in selecting content configurations.

[0017] Therefore, the present disclosure provides an information processing apparatus, an information processing method, a reproduction processing apparatus, and a reproduction processing method that enable a client device to efficiently select a content configuration.

[0018] Solution to the problem

[0019] According to the present disclosure, a preprocessing unit generates content configuration selection information for determining whether each of the content can be reproduced for one or more content, each of the content having a content configuration including one or more three-dimensional objects and spatial arrangement information thereof to represent a virtual space. A file generation unit generates a file including data about the virtual space and the content configuration selection information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a diagram showing the configuration of 6DoF content.

[0021] Figure 2 is a system configuration diagram of an example of a distribution system.

[0022] Figure 3 is a block diagram of a file generating device.

[0023] Figure 42 is a diagram for describing a method of storing content configuration selection information according to the first embodiment.

[0024] Figure 5 : is a diagram showing an example of the syntax of the extended switching node in the first embodiment.

[0025] Figure 6 is a block diagram of a client device.

[0026] Figure 7 is a flowchart of a document generation process performed by the document generation device according to the first embodiment.

[0027] Figure 8 is a flowchart of a reproduction process executed by the client device according to the first embodiment.

[0028] Fig. 9 is a diagram showing an ISOBMFF file of a scene description in modification (1) of the first embodiment.

[0029] Fig.10 : is a diagram showing an example of the syntax of content configuration selection information stored in SampleEntry in the modification (1) of the first embodiment.

[0030] Fig.11 is a diagram showing an example of a set of content configuration selection information.

[0031] Fig.12 is a diagram showing the Matroska media container format.

[0032] Fig.13 : is a diagram for describing a method of storing content configuration selection information according to modification (4) of the first embodiment.

[0033] Fig.14 is a diagram showing an example of the syntax of a RequiedPerformance node.

[0034] Fig.15 is a diagram showing an example of describing a 6DoFContentStruct Descriptor.

[0035] Fig.16 It is a diagram showing the semantics of CSC in the modification (5) of the first embodiment.

[0036] Fig.17 is a diagram for describing a method of storing content configuration selection information according to the second embodiment.

[0037] Fig.18 : is a diagram showing an example of the syntax of the extended switching node in the second embodiment.

[0038] Fig.19 is a diagram showing an example of the syntax of content configuration selection information stored in SampleEntry in modification (1) of the third embodiment.

[0039] Fig. 20 is a diagram showing the semantics of CSC in modification (2) of the third embodiment.

[0040] Fig.21 This is a diagram for describing a method of using configuration information stored in AdaptationSet of three-dimensional model data.

[0041] Fig. 22 It is a hardware configuration diagram of a computer. DETAILED DESCRIPTION

[0042] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In each of the following embodiments, the same parts are represented by the same figure numerals, and their description will not be repeated. In addition, the scope of the present technology disclosed herein is not limited to the embodiments, and the contents of the following non-patent documents disclosed at the time of filing the application are also incorporated herein.

[0043] Non-patent document 1: (as mentioned above)

[0044] Non-patent document 2: "ISO / IEC 14496-12:2015", Information technology. Coding of audiovisual objects. Part 12: ISO base media file format, 2015-12

[0045] Non-patent document 3: "ISO / IEC 23009-1:2014", Information Technology. Dynamic Adaptive Streaming over HTTP (DASH), Part 1: Media presentation description and segment formats, 2014-5

[0046] That is, the disclosure of the above-mentioned non-patent documents is also incorporated into this specification by reference. That is, the disclosure of the above-mentioned non-patent documents is also used as a basis for determination regarding support requirements. For example, although the structure / terminology used for the scene description described in non-patent document 1, the file structure described in non-patent document 2, and the terminology used for the MPEG-DASH standard described in non-patent document 3 are not directly described in the detailed description of the present invention, these are considered to fall within the scope of the disclosure of the present technology and meet the support requirements of the claims. Similarly, for example, even if technical terms such as parsing, grammar, and semantics are not directly described in the detailed description of the present invention, these are considered to fall within the scope of the disclosure of the present technology and meet the support requirements of the claims.

[0047] In addition, the present disclosure will be described according to the following item order.

[0048] 1. First Implementation

[0049] 1.1 Modification of the first embodiment (1)

[0050] 1.2 Modification of the First Embodiment (2)

[0051] 1.3 Modification of the First Embodiment (3)

[0052] 1.4 Modification of the First Implementation (4)

[0053] 1.5 Modification of the First Embodiment (5)

[0054] 2. Second Implementation

[0055] 2.1 Modification of the Second Embodiment (1)

[0056] 3. Third Implementation

[0057] 3.1 Modification of the Third Embodiment (1)

[0058] 3.2 Modification of the Third Embodiment (2)

[0059] 4. Fourth Implementation

[0060] [1. First embodiment]

[0061] When distributing 6DoF content each having a content configuration in an object-based representation scheme, a space-based representation scheme, or a hybrid representation scheme, for example, a scene description file, an MPD file, and a 3D model data file such as Figure 1 The configuration shown is distributed. Figure 1is a diagram showing the configuration of 6DoF content. In such a current content configuration, the client can determine whether the client device can fully demonstrate its own reproduction capability based on the following three indexes.

[0062] The first index is an index for determining whether the scene description file and the three-dimensional model data file can be decoded separately. The second index is an index for determining whether the scene description file and the three-dimensional model data file can be decoded together. The third index is an index for determining whether data rendering can be performed after the decoding process. Rendering refers to the arrangement and display in three-dimensional space. In the case where the client device determines whether the reproduction process can be performed for each content configuration based on such an index, the use of the following information can be considered.

[0063] The first information is the @mimeType attribute and the @codecs attribute stored in the AdaptationSet, which represents the scene description in the AdaptationSet included in the MPD file. Based on this information, the client device determines whether the scene description can be decoded. More specifically, it can be determined whether the client device supports the file format of the scene description using the @mimeType attribute. In addition, it can be determined whether the client device is compatible with the codec that encodes the scene description using the @codecs attribute. Therefore, it is determined which format of the scene description is created between the MPEG-4 scene description format and the GL Transmission Format (glTF) 2.0 format, and it is determined whether the client device can reproduce the scene description.

[0064] The second information is a sceneProfileLevelIndication filed stored when the scene description is represented in the ISO Base Media File Format (ISOBMFF). Based on this information, the client device determines whether data rendering can be performed after the scene description is decoded. This information includes information for determining the reproduction processing capability of the client device, which is used to reconstruct a three-dimensional space from a scene graph (hierarchical structure) represented by the scene description (data about the scene graph). For example, in the case of a point cloud, the sceneProfileLevelIndication filed includes the maximum number of points for each scene, and in the case of a mesh, the sceneProfileLevelIndication filed includes the maximum number of vertices, the maximum number of faces, and the maximum number of vertices for each scene's face. That is, based on this information, it is determined what degree of reproduction processing capability is required for the entire scene.

[0065] The third information is the number of external 3D model data files used to configure the scene obtained from the scene graph represented by the scene description file. Based on this information, the client device determines whether the scene description file and the 3D model data file can be decoded. For example, when the number of decoders for 3D model data of the client device itself is greater than the number of external 3D model data files used to configure the scene, the client device determines that reproduction is available. In this case, the greater the number of decoders used, the higher the reproduction processing capability required by the client device.

[0066] The fourth information is the @mimeType attribute and the @codecs attribute stored in the AdaptationSet, which represents each three-dimensional model data in the AdaptationSet included in the MPD file. The @mimeType attribute includes, for example, information about the file format of the stored 3D model data. In addition, the @codecs attribute includes information about which codec the 3D model data is encoded by and information about the profile or level of the codec. Based on this information, the client device determines whether each three-dimensional model data can be decoded. More specifically, it can be determined whether the client device supports the file format of each three-dimensional model data using the @mimeType attribute. In addition, it can be determined whether the client device is compatible with the codec that encodes each three-dimensional model data using the @codecs attribute.

[0067] In addition, in the case where the @codecs attribute includes information on the reproduction compatibility of the three-dimensional model data, the client device can determine whether the rendering of each three-dimensional model data can be performed. In this case, for example, in the case where the three-dimensional model data is a point cloud, the @codecs attribute includes the maximum number of points, and in the case where the three-dimensional model data is a mesh, the @codecs attribute includes the maximum number of vertices of a face, the maximum number of faces, and the maximum number of vertices.

[0068] The fifth information is the @bandwidth attribute in the Representation included in the MPD file stored for each 3D model data. Based on this information, the client device determines whether each 3D model data can be decoded. For example, by using this information, the client device can determine whether the bit rate is a bit rate that can reproduce the 3D model data alone or a bit rate that can reproduce the 3D model data in the entire scene.

[0069] Among the above information, the first information, the fourth information, and the fifth information are used as the first index, the third information, the fourth information, and the fifth information are used as the second index, and the second information and the fourth information are used as the third index.

[0070] Here, the content creator hopes to provide users with content that can be reproduced with as high a degree of freedom as possible, thereby increasing the value of the content. At the same time, preferably, the number of reproducible client devices is large. Therefore, the content creator may consider preparing multiple content configurations to distribute 6DoF content. Hereinafter, content configurations based on object, space, and hybrid representation schemes will be referred to as object-based content configurations, space-based content configurations, and hybrid content configurations, respectively.

[0071] For example, in the case where the content creator prepares a hybrid content configuration and a space-based content configuration, if the reproduction processing capability of the client device is high, the client device selects the hybrid content configuration for reproduction, but if the reproduction processing capability of the client device is low, the client device selects the space-based content configuration for reproduction. In this case, the scene description is created to include two content configurations. In this case, conventionally, in order to select a content configuration, the client device analyzes the entire scene description and analyzes the information described in the AdaptationSet of the MPD for the three-dimensional model data used to configure the scene using the first information to the fifth information. This processing is not efficient because the content configuration that will not be actually used in each scene is analyzed.

[0072] However, in the 6DoF content distribution system according to the related art, no information is provided to the client device for determining which of the contents having different content configurations can be reproduced. For this reason, unless decoding and rendering are actually performed, it is difficult for the client device to determine whether the reproduction process is available. Therefore, a system that enables a client device to efficiently select a content configuration will be described.

[0073] [Configuration of System According to First Embodiment]

[0074] Figure 2 1 is a system configuration diagram of an example of a distribution system. The distribution system 100 includes a file generation device 1 as an information processing device, a client device 2 as a reproduction processing device, and a Web server 3. The file generation device 1, the client device 2, and the Web server 3 are connected to a network 4. Then, the file generation device 1, the client device 2, and the Web server 3 can communicate with each other via the network 4. Here, although Figure 1 1 and 2 , each device is shown as one device, but the distribution system 100 may include a plurality of file generation devices 1 and a plurality of client devices 2 .

[0075] The file generation device 1 generates 6DoF content. The file generation device 1 uploads the generated 6DoF content to the Web server 3. Here, although it is described in the present embodiment that the Web server 3 provides the 6DoF content to the client device 2, the distribution system 100 may adopt another configuration. For example, the file generation device 1 may include the function of the Web server 3 to store the generated 6DoF content therein by itself, and provide the stored 6DoF content to the client device 2.

[0076] The web server 3 holds the 6DoF content uploaded from the file generating device 1. Then, the web server 3 provides the 6DoF content specified by the request from the client device 2.

[0077] The client device 2 sends a request to the web server 3 to send the 6DoF content thereto. The client device 2 then obtains the 6DoF content specified by the sent request from the web server 3. The client device 2 then decodes the 6DoF content to generate a video so that the video is displayed on a display device such as a monitor.

[0078] Here, 6DoF content will be described. 6DoF content represents a three-dimensional space with one or more three-dimensional objects. The three-dimensional objects are represented using a coordinate system in a bounding box normalized by the local coordinate system of the 6DoF content, and the three-dimensional objects are compressed and encoded into a bitstream. Scene description is used to arrange the bitstream in a three-dimensional space.

[0079] There are multiple standards for scene description. Basically, the scene that displays each three-dimensional object at each time is represented by a graph in a tree hierarchy called a scene graph, and the scene graph is represented in a binary format or a text format. Here, the scene graph is spatial display control information, and information related to the display of three-dimensional objects is configured by defining nodes as constituent units and hierarchically combining multiple nodes. The nodes include nodes for information about coordinate transformation from one coordinate system to another, nodes for information about the position and size of the three-dimensional object, and nodes for information about access to the three-dimensional object and audio data.

[0080] Note that in the following description, it is assumed that the 6DoF content includes scene description data, which is spatial display control information and media data of multiple three-dimensional objects (for example, represented by mesh data and texture data of the three-dimensional objects). In addition, the 6DoF content may include audio data. The media data about the three-dimensional objects may also be applied in another format such as a point cloud. In addition, in the present embodiment, the scene description file is based on MPEG-4 scene description (ISO / IEC14496-11).

[0081] MPEG-4 scene description data is obtained by binarizing a scene graph in a format called Binary Format for Scenes (BIFS). A predetermined algorithm can be used to transform a scene graph into BIFS. In addition, by storing a scene description in ISOBMFF, a scene can be specified at each time, making it possible to represent a three-dimensional object whose position and size change.

[0082] [Configuration of Document Generation Device According to First Embodiment]

[0083] Next, the file generating device 1 will be described in detail. Figure 3 is a block diagram of a file generation device. Figure 3 As shown, the file generation device 1 as an information processing apparatus includes a generation processing unit 10 and a control unit 11. The control unit 11 performs processing for controlling the generation processing unit 10. For example, the control unit 11 uniformly controls the operation timing and the like of each part of the generation processing unit 10. The generation processing unit 10 includes a data input unit 101, a preprocessing unit 102, an encoding unit 103, a file generation unit 104, and a transmission unit 105.

[0084] The data input unit 101 receives input of original information, meta information, etc. for generating a three-dimensional object. The data input unit 101 outputs the acquired original information to the preprocessing unit 102. The data input unit 101 receives input of data. The data received by the data input unit 101 includes a 3D object and meta information such as information about the arrangement of the 3D object. The data input unit 101 outputs the acquired data to the preprocessing unit 102.

[0085] The preprocessing unit 102 receives an input of data including 3D objects and metadata such as information about the arrangement of the 3D objects from the data input unit 101. The preprocessing unit 102 then determines a bitstream configuration based on the acquired data, and generates a scene graph using the metadata of each 3D object and information about access to the bitstream. The metadata includes control information such as which codec is used for compression.

[0086] In addition, the preprocessing unit 102 generates content configuration selection information for each content configuration, the content configuration selection information including any of the first to fifth information for one or more content configurations. The content configuration selection information provides an index of the reproduction processing capability required to reproduce the scene of each content configuration.

[0087] Then, the preprocessing unit 102 stores the content configuration selection information for each content configuration in the scene description. Therefore, the client device 2 can select a content configuration capable of reproduction processing by using the content configuration selection information. Hereinafter, the storage of the content configuration selection information according to the present embodiment will be described in detail.

[0088] Figure 4 1 is a diagram for describing a method for storing content configuration selection information according to the first embodiment. Figure 4 As shown, the pre-processing unit 102 arranges sub-nodes under the switch node in the scene description for each content configuration. Figure 4 For example, content configuration 301 is a content configuration in a hybrid representation scheme, and content configuration 302 is a content configuration in a space-based representation scheme. The pre-processing unit 102 extends the switch node to store information to be used for determining whether decoding and rendering can be performed for the entire scene in each content configuration as content configuration selection information.

[0089] Figure 5 1 is a diagram showing an example of the syntax of the extended switch node in the first embodiment. For example, the preprocessing unit 102 indicates a plurality of content configurations in the choice field of the switch node. In addition, the preprocessing unit 102 newly adds a Points field, a VertivesParFace field, a Faces field, an Indices field, a Num3DmodeData field, a 3DmodeIDataMimeType field, a 3DmodeDataCodec field, and a Bitrate field indicating content configuration selection information for each content configuration. Then, the preprocessing unit 102 stores the value of each content configuration by storing the value in the newly added field in the order of the content configuration indicated in the choice field.

[0090] Points is the number of points in the point cloud. VertivesParFace is the number of vertices of the face of the mesh. Faces is the number of faces of the mesh. Indices is the number of vertices of the mesh. Points, VertivesParFace, Faces, and Indices correspond to the second information. Num3DmodelData is the number of externally referenced three-dimensional model data. Num3DmodelData corresponds to the third information. 3DmodelDataMimeType is the MimeType of the externally referenced three-dimensional model data. 3DmodelDataCodec is the codec of the externally referenced three-dimensional model data. Num3DmodelData and 3DmodelDataCodec correspond to the fourth information. Bitrate is the bit rate of the three-dimensional model data including the external reference. Bitrate corresponds to the fifth information.

[0091] The preprocessing unit 102 outputs the three-dimensional object and the generated scene graph to the encoding unit 103. The preprocessing unit 102 also outputs metadata to the file generating unit 104.

[0092] The encoding unit 103 receives the input of the three-dimensional object and the scene graph from the pre-processing unit 102. Then, the encoding unit 103 encodes the three-dimensional object to generate a bit stream. In addition, the encoding unit 103 encodes the acquired scene graph to generate a scene description. Thereafter, the encoding unit 103 outputs the generated bit stream and scene description to the file generation unit 104.

[0093] The file generation unit 104 receives inputs of a bitstream and a scene description from the encoding unit 103. The file generation unit 104 also receives inputs of metadata from the preprocessing unit 102. Then, the file generation unit 104 stores the acquired bitstream in an ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the bitstream. The file generation unit 104 also stores data on the scene description in an ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the scene description.

[0094] In addition, the file generation unit 104 generates a media presentation description (MPD) file based on the data acquired from the pre-processing unit 102. In the MPD file, meta information of the media type and 6DoF content, such as information on segment files of moving images or sounds, is stored.

[0095] The transmission unit 105 acquires the ISOBMFF file of the bit stream and scene description, and the MPD file from the file generation unit 104 , and sends and uploads the acquired files to the Web server 3 .

[0096] [Configuration of Client Device According to First Embodiment]

[0097] Figure 6 is a block diagram of the client device. Figure 6 As shown, the client device 2 includes a reproduction processing unit 20 and a control unit 21. The control unit 21 controls the operation of each part of the reproduction processing unit 20. For example, the control unit 21 uniformly controls the operation timing of each part of the reproduction processing unit 20. The reproduction processing unit 20 includes a file acquisition unit 201, a measurement unit 202, a file processing unit 203, a decoding processing unit 204, a display control unit 205, a display information generation unit 206, and a display unit 207.

[0098] The file acquisition unit 201 acquires an MPD file corresponding to the 6DoF content to be reproduced from the Web server 3. Then, the file acquisition unit 201 acquires information on the scene description of the 6DoF content to be reproduced based on the MPD file.

[0099] The file acquisition unit 201 accesses the Web server 3 and acquires the ISOBMFF file storing the scene description of the 6DoF content to be displayed. Then, the file acquisition unit 201 outputs the ISOBMFF file storing the scene description to the file processing unit 203.

[0100] In addition, the file acquisition unit 201 acquires information about the bit stream selected by the file processing unit 203 from the file processing unit 203. Then, the file acquisition unit 201 accesses the Web server 3 and acquires the segment file of the selected bit stream. Thereafter, the file acquisition unit 201 outputs the acquired segment file of the bit stream to the file processing unit 203.

[0101] The measuring unit 202 measures the transmission band of the transmission path between the client device 2 and the Web server. Then, the measuring unit 202 outputs the result of measuring the transmission band to the file processing unit 203.

[0102] The file processing unit 203 receives an input of an MPD file corresponding to the 6DoF content to be reproduced from the file acquisition unit 201. Then, the file processing unit 203 parses the acquired MPD file to obtain information about the scene description of the 6DoF content to be reproduced. The file processing unit 203 also identifies a plurality of data to be used for adaptive distribution. For example, if adaptive distribution is used to switch the bit rate, information about the segment file of the bit stream corresponding to each bit rate is obtained. In this case, the file processing unit 203 outputs the information about the scene description of the 6DoF content to be reproduced to the file acquisition unit 201.

[0103] The file processing unit 203 receives an input of an ISOBMFF file storing a scene description from the file acquisition unit 201. The file processing unit 203 parses the acquired ISOBMFF file. Then, the file processing unit 203 acquires the exchange node of the scene description. Thereafter, the file processing unit 203 acquires content configuration selection information from the exchange node. Next, the file processing unit 203 selects a content configuration to be used from the acquired content configuration selection information according to the reproduction processing capability of the client device 2. Thereafter, the file processing unit 203 acquires a child node corresponding to the selected content configuration in the scene graph. Then, the file processing unit 203 acquires coordinate transformation information, layout information of three-dimensional objects, and access information in the scene of the selected content configuration.

[0104] In addition, the file processing unit 203 receives an input of a result of measuring the transmission band from the measurement unit 202. Then, the file processing unit 203 selects a segment file of a bitstream to be reproduced based on the result of parsing the scene description, the information indicating the transmission band acquired from the measurement unit 202, and the like. Then, the file processing unit 203 outputs information on the segment file of the selected bitstream to the file acquisition unit 201. In this case, adaptive distribution according to the bit rate is achieved by changing the segment file of the bitstream to be selected according to the transmission band.

[0105] Thereafter, the file processing unit 203 receives an input of the segment file of the selected bitstream from the file acquisition unit 201. Then, the file processing unit 203 extracts bitstream data from the acquired segment file of the bitstream, and outputs the extracted data to the decoding processing unit 204.

[0106] The decoding processing unit 204 receives an input of the bit stream data from the file processing unit 203. Then, the decoding processing unit 204 decodes the acquired bit stream data. Thereafter, the decoding processing unit 204 outputs the decoded bit stream data to the display information generating unit 206.

[0107] The display control unit 205 receives input of information about the viewpoint position and the line of sight direction of the operator from an input device not shown, and then outputs the acquired information about the viewpoint position and the viewpoint direction to the display information generation unit 206.

[0108] The display control unit 205 also receives input of information about what kind of three-dimensional object exists from the file processing unit 203. Alternatively, the operator may not input information about the viewpoint position and the line of sight, but input designated information indicating the three-dimensional object of interest using an input device. The display control unit 205 obtains designated information indicating the three-dimensional object of interest specified by the operator. Then, for example, in the case of tracking a three-dimensional object for viewing, the display control unit 205 outputs information about the viewpoint position and the viewpoint direction to the display information generation unit 206 to track the three-dimensional object designated as the designated information over time. Therefore, an image for tracking a three-dimensional object designated by the operator, for example, can be displayed. In addition, for example, in the case of displaying the position of a three-dimensional object, the display control unit 205 generates information for specifying a designated three-dimensional object in 6DoF content.

[0109] The display information generating unit 206 receives the scene description, decoded bit stream data and acquired information about the viewpoint position and viewpoint direction to generate display information. The display information generating unit 206 will be described in detail below.

[0110] The input of the bit stream data is received from the decoding processing unit 204. Then, the display information generation unit 206 arranges the three-dimensional object as the acquired bit stream data in the three-dimensional space based on the scene description. In addition, the display information generation unit 206 receives the input of information about the viewpoint position and the line of sight direction of the operator from the display control unit 205. Then, the display information generation unit 206 performs rendering of the three-dimensional object arranged in the three-dimensional space according to the viewpoint position and the line of sight direction to generate an image for display. Thereafter, the display information generation unit 206 provides the generated image for display to the display unit 207.

[0111] The display unit 207 includes a display device such as a monitor. The display unit 207 receives an input of an image for display generated by the display information generation unit 206. Then, the display unit 207 displays the acquired image for display on the display device.

[0112] [File Generation Process According to the First Embodiment]

[0113] Next, we will refer to Figure 7 The flow of the file generation process performed by the file generation device 1 according to the first embodiment is described in detail. Figure 7 is a flowchart of a document generation process performed by the document generation device according to the first embodiment.

[0114] The pre-processing unit 102 generates content configuration selection information (step S1). In this embodiment, the pre-processing unit 102 generates content configuration selection information to be stored in a switch node of a scene description graph.

[0115] Then, the pre-processing unit 102 generates data of a scene graph in which the content configuration selection information is stored in the switching node (step S2 ). The pre-processing unit 102 outputs the data of the scene graph in which the content configuration selection information is stored in the switching node to the encoding unit 103 .

[0116] Next, the encoding unit 103 encodes the data on the three-dimensional object to generate a bit stream of the three-dimensional object. In addition, the encoding unit 103 encodes the acquired scene graph to generate a scene description (step S3).

[0117] Next, the file generation unit 104 stores the acquired bitstream in the ISOBMFF file of each segment to generate a segment file of the bitstream. In addition, the file generation unit 104 stores data on scene description in the ISOBMFF file of each segment to generate a segment file of the scene description (step S4).

[0118] The transmission unit 105 outputs the segment file generated by the file generation unit 104 to the Web server 3 (step S5 ).

[0119] [Reproduction Processing Procedure According to the First Embodiment]

[0120] Next, we will refer to Figure 8 The flow of the reproduction process performed by the client device 2 according to the present embodiment is described. Figure 8 is a flowchart of a reproduction process executed by the client device according to the first embodiment.

[0121] The file acquisition unit 201 acquires the MPD file corresponding to the 6DoF content to be reproduced from the Web server 3 (step S11 ). The file acquisition unit 201 outputs the acquired MPD file to the file processing unit 203 .

[0122] The file processing unit 203 performs analysis processing (step S12) by parsing the MPD file input from the file acquisition unit 201. The file processing unit 203 specifies the scene description of the 6DoF content to be reproduced based on the analysis result.

[0123] Then, the file acquisition unit 201 acquires the scene description specified by the file processing unit 203. Next, the file processing unit 203 parses the scene description acquired by the file acquisition unit 201 to acquire the switching node. Next, the file processing unit 203 acquires content configuration selection information from the switching node (step S13).

[0124] Then, the file processing unit 203 selects a content configuration according to the reproduction processing capability of the client device 2 using the acquired content configuration selection information (step S14).

[0125] Next, the file processing unit 203 parses the child node corresponding to the selected content configuration. Thereafter, the file processing unit 203 obtains the segment file of the bitstream corresponding to the 6DoF content to be reproduced based on the parsing result (step S15). The decoding processing unit 204 decodes the segment file of the bitstream. Thereafter, the decoding processing unit 204 outputs the bitstream data to the display information generation unit 206.

[0126] The display control unit 205 outputs the input information about the viewpoint position and the line of sight direction to the display information generating unit 206. The display information generating unit 206 performs viewing processing by rendering a three-dimensional object using the information about the viewpoint position and the line of sight direction acquired from the display control unit 205 and adding the position information to generate an image for display and displays the generated image for display on the display unit 207 (step S16).

[0127] As described above, the file generation device according to the present embodiment stores the content configuration selection information in the switching node as the root node of the scene description, and provides the content configuration selection information to the client device. The client device obtains the switching node as the root node of the scene description to obtain the content configuration selection information, and selects the content configuration using the obtained content configuration selection information.

[0128] In a configuration in which content configuration selection information is not stored, the entire scene description has been analyzed to collect information for selecting a content configuration. In contrast, in a distribution system according to the present embodiment, a client device can obtain content configuration selection information once it obtains a switching node, and can obtain information for selecting a content configuration without analyzing other nodes of the scene description. Therefore, a content configuration can be selected efficiently.

[0129] Content creators can prepare multiple content configurations so that 6DoF content can be distributed corresponding to client devices with different reproduction capabilities. Then, the client device can efficiently select a content configuration according to its own reproduction processing capability.

[0130] [1.1 Modification of the first embodiment (1)]

[0131] The file generating device 1 according to the present modification is different from the file generating device according to the first embodiment in that content configuration selection information is stored in SampleEntry of the ISOBMFF file of the scene description.

[0132] The preprocessing unit 102 of the file generating device 1 according to the present modification generates content configuration selection information as metadata. Then, the preprocessing unit 102 sends the metadata including the content configuration selection information to the file generating unit 104.

[0133] The file generation unit 104 receives an input of a bitstream and a scene description from the encoding unit 103. The file generation unit 104 also receives an input of metadata including content configuration selection information from the pre-processing unit 102.

[0134] Regarding the bit stream, similarly to the first embodiment, the file generation unit 104 stores the acquired bit stream in the ISOBMFF file for each segment to be converted into a file, thereby generating a segment file of the bit stream.

[0135] On the other hand, when data on the scene description is stored in the ISOBMFF file, the file generation unit 104 acquires the content configuration selection information from the metadata. Then, the file generation unit 104 stores the content configuration selection information in the SampleEntry of the ISOBMFF file of the scene description.

[0136] For example, Fig. 9 As shown, the file generation unit 104 newly adds a 6DoFContentStructBox shown as BOX 303 to the SampleEntry of the ISOBMFF file of the scene description. Fig. 9 6DoFContentStructBox is a diagram showing an ISOBMFF file of a scene description in the modification (1) of the first embodiment. Then, the file generation unit 104 stores the content configuration selection information of each content configuration in 6DoFContentStructBox.

[0137] For example, the file generation unit 104 generates Fig.10 The content configuration selection information indicated by the syntax shown in is stored in 6DoFContentStructBox. Fig.10 is a diagram showing an example of the syntax of the content configuration selection information stored in SampleEntry in the modification (1) of the first embodiment. The file configuration selection information in this case is as follows Figure 5 The newly added fields in the first embodiment shown have the same information.

[0138] In this way, the file generation unit 104 stores the data on the scene description in the ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the scene description. Then, the file generation unit 104 outputs the ISOBMFF file of the scene description including the file configuration information to the transmission unit 105.

[0139] The file processing unit 203 of the client device 2 according to the present modification receives the input of the ISOBMFF file of the scene description from the file acquisition unit 201. Then, the Initialization Segment of the ISOBMFF file of the scene description is acquired. Next, the file processing unit 203 acquires the content configuration selection information from the 6DoFContentStructBox in the acquired Initialization Segment. Then, the file processing unit 203 uses the acquired content configuration selection information to select the content configuration to be used from the scene description.

[0140] As described above, the client device according to this modification can obtain the content configuration selection information before analyzing the scene description itself, thereby selecting the content configuration without analyzing the scene description itself. In this way, the client device according to this embodiment does not need to analyze the scene description itself to select the content configuration. The configuration of modification (1) is effective in the case where the reproduction processing capacity required for reproduction in the content configuration does not change every time period. In this case, there is no need to expand the scene description itself.

[0141] [1.2 Modification of the first embodiment (2)]

[0142] The file generating device 1 according to the present modification is different from the file generating device according to the first embodiment in that the value of each content configuration selection information is determined for grouping and which group each content configuration belongs to is indicated according to the grouping, thereby indicating the content configuration selection information of each content configuration.

[0143] Fig.11 is a diagram showing an example of a set of content configuration selection information. Fig.11 In the table of , groups to which 01 to 03 are assigned as RequiedPerformanceID, which is an identification number, are set. Then, for each group, it is indicated what value is set for each content configuration selection information.

[0144] The pre-processing unit 102 of the file generating device 1 according to the present embodiment has Fig.11 Then, the pre-processing unit 102 stores information on which group each content configuration indicated by the RequiredPerformanceID corresponds to in the switching node.

[0145] The file processing unit 203 of the client device 2 according to the present embodiment analyzes the exchange node of the scene description to obtain the RequiredPerformanceID of the group to which each content configuration belongs. Then, the file processing unit 203 selects the content configuration by determining the reproduction processing capability required for reproduction in each content configuration from the content configuration selection information assigned to the group to which each content configuration belongs.

[0146] As described above, the file generation device according to the present embodiment uses a group of content configuration selection information to notify content configuration selection information for each content configuration. Therefore, it is not necessary to generate detailed content configuration selection information for each content configuration, making the process of generating content configuration selection information simple. This scheme is also applicable to modification (1) of the first embodiment.

[0147] [1.3 Modification of the first embodiment (3)]

[0148] The file generating apparatus 1 according to the present modification is different from the file generating apparatus according to the first embodiment in that, when transmitting three-dimensional model data, the Matroska media container (http: / / www.matroska.org / ) is used as the file format instead of ISOBMFF.

[0149] Fig.126DoFContentStruct element with content configuration selection information is stored in TrackEntry element. At this time, file generation unit 104 sets element type to binary and sets Fig.10 SelectContentStructMetadata() shown in FIG. 1 stores binary data as Extensible Binary Meta Language (EBML) data.

[0150] The file processing unit 203 of the client device 2 according to the present modification obtains the Initialization Segment of the Matroska media container file including the scene description. Then, the file processing unit 203 obtains the content configuration selection information from the 6DoFContentStruct element included in the InitializationSegment to select the content configuration.

[0151] In this way, even when a container format different from that in modification (1) is used, content configuration selection information is provided so that the client device can select a container configuration. In addition, although container configuration selection information is generated for each container configuration in this modification, this modification is not limited thereto, and, for example, the element type may be stored as an integer to indicate RequirePerformanceID as in modification (2).

[0152] [1.4 Modification of the first embodiment (4)]

[0153] The file generating device 1 according to the present embodiment is different from the file generating device according to the first embodiment in that content configuration selection information is stored for each scene description of content configuration. Fig.13 : is a diagram for describing a method of storing content configuration selection information according to modification (4) of the first embodiment.

[0154] For example, Figure 4 As shown, the file generation unit 104 of the file generation device 1 according to the present modification newly defines a RequiedPerformance node 311 under the Group node of the content configuration 301, and newly defines a RequiedPerformance node 312 under the Group node of the content configuration 302. Then, the file generation unit 104 stores the content configuration selection information in the RequiedPerformance node 311 of the content configuration 301. In addition, the file generation unit 104 stores the content configuration selection information in the RequiedPerformance node 312 of the content configuration 302.

[0155] Fig.14 is a diagram showing an example of the syntax of the RequiredPerformance node. Specifically, the file generation unit 104 uses Fig.14 The syntax shown in defines the RequiredPerformance node as a node with content configuration selection information. Fig.14 As shown, information used as an index for determining whether the content configuration can be reproduced is registered in the RequiedPerformance node. In this case, Points also represents the number of points in the point cloud. VertivesParFace represents the number of vertices of the face of the mesh. Faces represents the number of faces of the mesh. Indices represents the number of vertices of the mesh. Num3DmodelData represents the number of externally referenced three-dimensional model data. 3DmodelDataMimeType represents the MimeType of the externally referenced three-dimensional model data. 3DmodelDataCodec represents the codec of the externally referenced three-dimensional model data. Bitrate represents the bit rate of the three-dimensional model data including the external reference.

[0156] The file processing unit 203 of the client device 2 according to the present modification obtains the RequiedPerformance node of each content configuration. At this stage, the file processing unit 203 does not obtain another child node under the group node. Next, the file processing unit 203 obtains the content configuration selection information of each content configuration from each RequiedPerformance node. Then, the file processing unit 203 selects the content configuration. Thereafter, the file processing unit 203 obtains and parses the group node and subsequent nodes of the selected content configuration.

[0157] As described above, according to the modified client device, the content configuration can be selected by acquiring and analyzing the RequiredPerformance node immediately below the group node of each content configuration. Therefore, compared with the case of analyzing the entire scene description, processing can be reduced. In addition, the content configuration selection information can be provided to the client device without changing the existing nodes.

[0158] Here, in this modification, the node storing the content configuration selection information is generated as a child node of the group node, but it may be a child node of another node as long as the other node is the root node of the content configuration. In addition, although the container configuration selection information is generated for each container configuration in this modification, this modification is not limited thereto, and for example, the RequiedPerformance node may be configured to hold the RequiedPerformanceID of modification (2).

[0159] [1.5 Modification of the first embodiment (5)]

[0160] The file generating device 1 according to the present modification is different from the file generating device according to the first embodiment in that content configuration selection information is stored in AdaptationSet of the MPD file indicating information about access to scene descriptions.

[0161] The preprocessing unit 102 of the file generating device 1 according to the present modification generates content configuration selection information. Then, the preprocessing unit 102 sends metadata including the content configuration selection information to the file generating unit 104.

[0162] The file generation unit 104 receives an input of a bitstream and a scene description from the encoding unit 103. The file generation unit 104 also receives an input of metadata including content configuration selection information from the pre-processing unit 102.

[0163] Then, the file generation unit 104 stores the acquired bitstream in the ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the bitstream. The file generation unit 104 also stores data on the scene description in the ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the scene description.

[0164] In addition, the file generation unit 104 generates an MPD file based on the data obtained from the pre-processing unit 102. At this time, the file generation unit 104 obtains the content configuration selection information included in the metadata. Then, the file generation unit 104 generates an MPD file based on the data obtained from the pre-processing unit 102. Figure 4 The scene description shown is defined in the AdaptationSet 320 of the MPD file Fig.15 The 6DoFContentStructdescriptor shown in FIG. Fig.15 6DoFContentStruct Descriptor is a diagram showing an example of describing 6DoFContentStruct Descriptor. Furthermore, according to the acquired content configuration selection information, the file generation unit 104 stores the CSC element of each content configuration in 6DoFContentStruct Descriptor, and registers the content configuration selection information as an attribute of the CSC element.

[0165] Fig.16 is a diagram showing the semantics of CSC in modification (5) of the first embodiment. Fig.16 As shown, CSC describes the definition of information elements indicating the capabilities of content configuration. CSC includes one or more elements, up to 255 elements. Here, @Use indicates attribute information about whether each element to be used is optional or mandatory.

[0166] In the case where multiple CSC elements appear, the CSC elements are registered in the order of content configuration described in the choice field of the exchange node of the scene description. In this case, CSC@points represents the number of points in the point cloud. CSC@VertivesParFace represents the number of vertices of the face of the mesh. CSC@Faces represents the number of faces of the mesh. CSC@Indices represents the number of vertices of the mesh. CSC@Num3DmodelData represents the number of externally referenced three-dimensional model data. CSC@3DmodelDataMimeType represents the MimeType of the externally referenced three-dimensional model data. CSC@3DmodelDataCodec represents the codec of the externally referenced three-dimensional model data. CSC@Bitrate represents the bit rate of the three-dimensional model data including the external reference.

[0167] The file processing unit 203 of the client device 2 according to this embodiment obtains the content configuration selection information by parsing the MPD file. That is, the file processing unit 203 can obtain the content configuration selection information when obtaining the MPD file, and can determine whether there is reproducible content in the scene description.

[0168] As described above, according to the client device of this embodiment, it is possible to select a content configuration that can be reproduced without obtaining a scene description. Therefore, the content configuration can be selected efficiently. However, in this case, since the content configuration selection information is stored in the MPD file, it can be said to be effective when the reproduction processing capacity required to reproduce the content configuration does not change every time period. In addition, although the container configuration selection information is generated for each container configuration in the present modification, the present modification is not limited to this, and for example, the attributes of the CSC element can be configured to store information indicating the RequiedPerformanceID as in modification (2).

[0169] [2. Second Embodiment]

[0170] The file generating device 1 according to the present embodiment is different from the file generating device according to the first embodiment in that the scene description of each content configuration is configured in a separate file, and the content configuration selection information is stored in the AdaptationSet of each scene description. Figure 3 The file generation device 1 according to this embodiment is also shown in the block diagram of FIG. Figure 6 The client device 2 according to the present embodiment is also shown in the block diagram of In the following description, the description of the operation of each unit that is the same as that of each unit in the first embodiment may be omitted.

[0171] Fig.172 is a diagram for describing a method for storing content configuration selection information according to a second embodiment. Fig.17 As shown, the preprocessing unit 102 of the file generation device 1 according to the present modification generates scene descriptions 331 and 332 for each content configuration. The scene description 331 is a scene description of the content configuration #1. In addition, the scene description 332 is a scene description of the content configuration #2. Then, the preprocessing unit 102 generates content configuration selection information for each of the content configuration #1 and the content configuration #2, and outputs the generated content configuration selection information to the file generation unit 104.

[0172] The file generation unit 104 acquires the scene descriptions 331 and 332 from the encoding unit 103. In addition, the file generation unit 104 acquires, from the pre-processing unit 102, content configuration selection information for each of content configuration #1 and content configuration #2.

[0173] Then, the file generation unit 104 stores the scene descriptions 331 and 332 in the ISOBMFF file. Fig.15 The 6DoFContentStruct Descriptor shown in FIG. 3 stores information of each content configuration in the AdaptationSet of the MPD file of each of the scene description 331 and the scene description 332 .

[0174] The file processing unit 203 of the client device 2 according to the present embodiment acquires the MPD file from the file acquisition unit 201. Then, the file processing unit 203 acquires the content configuration selection information included in each AdaptationSet of the MPD file of each of the scene description 331 and the scene description 332. Then, the file processing unit 203 selects the content configuration to be used by using the acquired content configuration selection information. Thereafter, the file processing unit 203 acquires the scene description of the selected content configuration from the Web server 3 via the file acquisition unit 201.

[0175] As described above, the client device according to the present embodiment can select a content configuration when acquiring an MPD file. Meanwhile, in the scheme according to the first embodiment and its modifications, since a scene description including a content configuration that is not used is acquired, data that is not used is acquired. In this way, since the client device according to the present embodiment does not need to acquire data about a scene description of an unnecessary content configuration, the client device according to the present embodiment can efficiently select a content configuration compared to the client device according to the first embodiment and its modifications. However, in this case, since the content configuration selection information is stored in the MPD file, it can be said to be effective in the case where the reproduction processing capacity required to reproduce the content configuration does not change every time period.

[0176] [2.1 Modification of Second Embodiment (1)]

[0177] The file generating device 1 according to the present modification is different from the file generating device according to the second embodiment in that content configuration selection information is stored in SampleEntry of the ISOBMFF file of the scene description.

[0178] The preprocessing unit 102 of the file generating device 1 according to the present modification generates content configuration selection information. Then, the preprocessing unit 102 sends metadata including the content configuration selection information to the file generating unit 104.

[0179] The file generation unit 104 receives an input of a bit stream and a scene description from the encoding unit 103. In addition, the file generation unit 104 receives an input of metadata including content configuration selection information from the pre-processing unit 102.

[0180] Then, the file generation unit 104 stores the acquired bitstream in the ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the bitstream. The file generation unit 104 also stores data on the scene description in the ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the scene description.

[0181] When the scene description is stored in an ISOBMFF file, the file generation unit 104 stores the content configuration selection information included in the metadata in the SampleEntry of the ISOBMFF file of the scene description. Fig. 9 and Fig.10 The storage method shown in stores content configuration selection information.

[0182] The file processing unit 203 of the client device 2 according to the present embodiment obtains the ISOBMFF file of the scene description from the file acquisition unit 201. Next, the file processing unit 203 obtains the Initialization Segment of the ISOBMFF file of the scene description obtained. Then, the file processing unit 203 obtains the content configuration selection information from the 6DoFContentStructBox included in the Initialization Segment of the ISOBMFF file of the scene description. Then, the file processing unit 203 selects the content configuration to be used by using the obtained content configuration selection information. Thereafter, the file processing unit 203 obtains the scene description of the selected content configuration from the Web server 3 via the file acquisition unit 201.

[0183] As described above, the client device according to the present embodiment obtains the content configuration selection information from the 6DoFContentStructBox included in the Initialization Segment of the ISOBMFF file of the scene description. Thereafter, the client device only needs to obtain the data of the scene description of the content configuration to be used, and does not need to obtain the data of the scene description of other content configurations that are not used. Therefore, the content configuration can be selected efficiently.

[0184] Here, as a file format for sending 3D model data, a Matroska media container may be used instead of ISOBMFF. When a Matroska media container is used, the file generation unit 104 may store content configuration selection information and provide the content configuration selection information to the client device in the same manner as in modification (3) of the first embodiment.

[0185] [3. Third embodiment]

[0186] In the distribution of 6DoF content, the maximum processing capability, bit rate, definition, etc. can be changed by adaptability. Then, the reproduction processing capability required by the client changes due to the adaptation of the bit rate, etc. of each 3D model data in one content configuration. For example, when a stream with the largest change in the number of faces of the mesh of the 3D model data in a certain content configuration is selected, the client is required to have a high reproduction processing capability, but when a stream with the smallest change in the number of faces of the mesh of the 3D model data in a certain content configuration is selected, the reproduction processing capability required by the client can be reduced.

[0187] That is, a content creator can increase the number of clients that can reproduce by providing content that is adapted in a content configuration. However, no information is provided for the client to determine whether the content adapted in the content configuration can be reproduced by the client. Therefore, a system that enables a client to efficiently select a content configuration in the presence of an adaptation in the content configuration will be described.

[0188] The file generation device 1 according to the present embodiment is different from the file generation device according to the first embodiment in that information indicating the minimum required reproduction processing capacity is generated and stored together with information indicating the maximum reproduction processing capacity required for reproduction, and is provided to the client device 2. Figure 3 The file generation device 1 according to this embodiment is also shown in the block diagram of FIG. Figure 6 The client device 2 according to the present embodiment is also shown in the block diagram of In the following description, the description of the operation of each unit that is the same as that of each unit in the first embodiment may be omitted.

[0189] The preprocessing unit 102 of the file generating device 1 according to the present embodiment generates content configuration selection information for each content configuration. Next, the preprocessing unit 102 expands the Fig.18 The illustrated scenario describes a switching node to store content configuration selection information for each content configuration.

[0190] Fig.18 is a diagram showing an example of the syntax of the extended exchange node in the second embodiment. MaxPoints is the maximum number of points of the point cloud. MinPoints is the minimum number of points of the point cloud. MaxVertivesParFace is the maximum number of vertices of the face of the mesh. MinVertivesParFace is the minimum number of vertices of the face of the mesh. MaxFaces is the maximum number of faces of the mesh. MinFaces is the minimum number of faces of the mesh. MaxIndices is the maximum number of vertices of the mesh. MinIndices is the minimum number of vertices of the mesh. MaxNum3DmodelData is the maximum number of three-dimensional model data for external reference. MinNum3DmodelData is the minimum number of three-dimensional model data for external reference. Max3DmodelDataCodec is the maximum value of the codec of the three-dimensional model data for external reference. Min3DmodelDataCodec is the minimum value of the codec of the three-dimensional model data for external reference. MaxBitrate is the maximum bit rate of the three-dimensional model data including the external reference. MinBitrate is the minimum bit rate of the three-dimensional model data including the external reference.

[0191] Here, the content configuration information indicating the maximum value is information indicating the reproduction processing capability that can reliably reproduce the content. In addition, the content configuration information indicating the minimum value is information indicating the reproduction processing capability that can reproduce the content when adaptation is performed, and can be said to be information indicating the reproduction processing capability that can reproduce a part of the content.

[0192] The pre-processing unit 102 includes Fig.18 The scene graph of the exchange nodes represented by the syntax shown in is output to the encoding unit 103.

[0193] The file processing unit 203 of the client device 2 according to the present embodiment acquires the exchange node from the acquired scene description. Then, the file processing unit 203 acquires the system configuration selection information of each system configuration from the exchange node. Then, the file processing unit 203 selects the system configuration to be used by using the acquired system configuration selection information. In this case, even if the client device 2 does not meet the reproduction processing capability required based on the content configuration selection information indicating the maximum value in a certain system configuration, as long as the reproduction processing capability required based on the content configuration selection information indicating the minimum value is met, the file processing unit 203 can select the content configuration.

[0194] Thereafter, the file processing unit 203 parses the group node and subsequent nodes of the selected system configuration of the scenario description.

[0195] In the first embodiment, since the content configuration requires the maximum reproduction processing capability regardless of adaptation, the content configuration that can be reproduced if adaptation is performed therein is not selected. In contrast, even if it is difficult to perform reproduction with the selected content configuration when the highest reproduction processing capability is required, the client device according to the present embodiment can select a content configuration in which reproduction is available when adaptation is performed for reproduction.

[0196] Therefore, the content creator may also consider the adaptation in the content configuration so that the 6DoF content can be distributed corresponding to the client devices with different reproduction capabilities. That is, the content creator may increase the number of client devices that can reproduce with respect to one content configuration. In addition, the client device may consider the adaptation in the content configuration to select the content configuration that can be reproduced according to its own reproduction processing capability.

[0197] [3.1 Modification of the third embodiment (1)]

[0198] The file generating device 1 according to the present modification is different from the file generating device according to the third embodiment in that content configuration selection information is stored in SampleEntry of the ISOBMFF file of the scene description.

[0199] The file generation unit 104 receives the input of the content configuration selection information from the pre-processing unit 102. The file generation unit 104 adds 6DoFContentStructBox to the SampleEntry of the ISOBMFF file of the scene description. Then, the file generation unit 104 adds the 6DoFContentStructBox to the SampleEntry of the ISOBMFF file of the scene description. Fig.19 The content configuration selection information indicated by the syntax shown in is stored in 6DoFContentStructBox. Fig.19 is a diagram showing an example of the syntax of content configuration selection information stored in SampleEntry in modification (1) of the third embodiment.

[0200] Thereafter, the file generation unit 104 outputs the ISOBMFF file including the scene description of the file configuration information to the transmission unit 105 .

[0201] The file processing unit 203 of the client device 2 according to the present modification receives an input of an ISOBMFF file of a scene description from the file acquisition unit 201. Then, the Initialization Segment of the ISOBMFF file of the scene description is acquired. Next, the file processing unit 203 acquires content configuration selection information from the 6DoFContentStructBox in the acquired Initialization Segment. Then, the file processing unit 203 uses the acquired content configuration selection information to select a content configuration to be used from the scene description. In this case, even if the client device 2 does not meet the reproduction processing capability required based on the content configuration selection information indicating the maximum value in a certain system configuration, the file processing unit 203 can select the content configuration as long as the reproduction processing capability required based on the content configuration selection information indicating the minimum value is met.

[0202] As described above, in a solution where content configuration selection information is stored in the SampleEntry of the ISOBMFF file of the scene description, the client device can also select a content configuration available for reproduction when performing adaptation of the reproduction. Therefore, the content creator can also consider adaptation in the content configuration so that 6DoF content can be distributed corresponding to client devices with different reproduction capabilities.

[0203] [3.2 Modification (2) of the third embodiment]

[0204] The file generating device 1 according to the present modification is different from the file generating device according to the second embodiment in that content configuration selection information is stored in AdaptationSet of the MPD file indicating information about access to scene descriptions.

[0205] The preprocessing unit 102 of the file generating device 1 according to the present modification generates content configuration selection information. Then, the preprocessing unit 102 sends metadata including the content configuration selection information to the file generating unit 104.

[0206] The file generation unit 104 receives an input of a bitstream and a scene description from the encoding unit 103. The file generation unit 104 also receives an input of metadata including content configuration selection information from the pre-processing unit 102.

[0207] Then, the file generation unit 104 stores the acquired bitstream in the ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the bitstream. The file generation unit 104 also stores data on the scene description in the ISOBMFF file of each segment to be converted into a file, thereby generating a segment file of the scene description.

[0208] In addition, the file generation unit 104 generates an MPD file based on the data obtained from the preprocessing unit 102. At this time, the file generation unit 104 defines 6DoFContentStructdescriptor in the AdaptationSet of the scene description of the MPD file, such as Fig. 20 Then, according to the content configuration selection information, the file generation unit 104 stores the CSC element of each content configuration in the 6DoFContentStruct Descriptor, and registers the content configuration selection information as an attribute of the CSC element. Fig. 20 is a diagram showing the semantics of CSC in modification (2) of the third embodiment. Fig. 20 As shown, CSC describes the definition of information elements indicating the capabilities of content configuration.

[0209] Thereafter, the file generation unit 104 outputs the ISOBMFF file including the scene description of the file configuration information to the transmission unit 105 .

[0210] The file processing unit 203 of the client device 2 according to the present embodiment obtains the content configuration selection information by parsing the MPD file. Then, the file processing unit 203 selects the content configuration to be used from the scene description using the obtained content configuration selection information. In this case, even if the client device 2 does not meet the reproduction processing capability required based on the content configuration selection information indicating the maximum value in a certain system configuration, as long as the reproduction processing capability required based on the content configuration selection information indicating the minimum value is met, the file processing unit 203 can select the content configuration.

[0211] As described above, in a scheme where content configuration selection information is stored in AdaptationSet of a scene description in an MPD file, a client device can also select a content configuration available for reproduction when performing adaptation of the reproduction. Therefore, a content creator can also consider adaptation in content configuration so that 6DoF content can be distributed corresponding to client devices with different reproduction capabilities of the reproduction.

[0212] Note that although container configuration selection information is generated for each container configuration in the second embodiment and its modifications (1) and (2), this is not limiting, and for example, the maximum value and the minimum value may be expressed using RequiedPerformanceID.

[0213] [4. Fourth embodiment]

[0214] The file generation device 1 according to the present embodiment is different from the file generation devices according to the first to third embodiments in that each configuration information of the three-dimensional model data is stored in the Representation included in the AdaptationSet of the three-dimensional model data in the MPD. Figure 3 The file generation device 1 according to this embodiment is also shown in the block diagram of FIG. Figure 6 The client device 2 according to the present embodiment is also shown in the block diagram of In the following description, the description of the operation of each unit that is the same as that of each unit in the first embodiment may be omitted.

[0215] The file generating unit 104 of the file generating device 1 according to the present embodiment acquires the content configuration selection information of each content configuration from the preprocessing unit 102. Then, when generating the MPD file, the file generating unit 104 stores the configuration information of the three-dimensional model data in each Representation of the AdaptationSet of the three-dimensional model data.

[0216] The file generation unit 104 stores the number of points in the Representation@numPoint attribute as the configuration information of the three-dimensional model data in the case of a point cloud, for example. In addition, the file generation unit 104 stores the number of vertices of a face in the Representation@vpf attribute, the number of faces in the Representation@numFase attribute, and the number of vertices in the Representation@numIndices attribute in the case of a mesh.

[0217] In the first to third embodiments and their respective examples, by storing such configuration information, the file processing unit 203 of the client device 2 can select a content configuration that can be reproduced independently of the selection based on the content configuration selection information. Here, an example has been described in which each configuration information of the three-dimensional model data is stored in Representation. However, the file generation unit 104 may store the configuration information in another element such as AdaptationSet or Preselection.

[0218] In addition, in addition to storing the configuration information, the file generation unit 104 may also store the minimum and maximum values ​​in the AdaptationSet of the three-dimensional model data of the MPD. For example, in the case of a point cloud, the file generation unit 104 stores the maximum and minimum values ​​of the number of points in the AdaptationSet@MaxNumPoint attribute and the AdaptationSet@MinNumPoint attribute, respectively. In addition, in the case of a mesh, the file generation unit 104 stores the maximum and minimum values ​​of the number of vertices of a face in the AdaptationSet@MaxVpf attribute and the AdaptationSet@MinVpf attribute, respectively, stores the maximum and minimum values ​​of the number of faces in the AdaptationSet@MaxNumFace attribute and the AdaptationSet@MinFace attribute, respectively, and stores the maximum and minimum values ​​of the number of vertices in the AdaptationSet@MaxNumIndices attribute and the AdaptationSet@MinIndices attribute, respectively.

[0219] The configuration according to this embodiment can be used to determine the reproduction processing capability required for reproduction when the 3D model data is transmitted using an MPD file without using a scene description. Fig.21 In the MPD file, there is an AdaptationSet 340 regarding the 3D model data of the point cloud. Fig.21 This is a diagram for describing a method of using configuration information stored in AdaptationSet of three-dimensional model data.

[0220] For example, Fig.21 As shown, the file generation unit 104 stores information on the maximum value and the minimum value of the number of points in the AdaptationSet 340 of the three-dimensional model data. In addition, the file generation unit 104 stores the number of points in Representation 341 to Representation 343 included in the AdaptationSet 340.

[0221] When parsing the MPD file, the file processing unit 203 of the client device 2 can determine whether there is a Representation capable of reproduction processing at the level of AdaptationSet 340. In the case where there is a Representation capable of reproduction processing, the file processing unit 203 can select the Representation capable of reproduction processing from the reproduced Representations 341 to 343.

[0222] Alternatively, the file generation unit 104 may store the configuration information of the three-dimensional model data in another place. For example, the file generation unit 104 newly defines a 3DmModelDataMetadataBox in the SampleEntry of the ISOBMFF file of the three-dimensional model data. Then, in the case where the three-dimensional model data is a point cloud, the file generation unit 104 may store the number of points in the newly defined 3DmModelDataMetadataBox, and in the case where the three-dimensional model data is a mesh, the file generation unit 104 may store the number of vertices, the number of faces, and the number of vertices of the face in the newly defined 3DmModelDataMetadataBox.

[0223] In addition, the file generation unit 104 can use the Matroska media container instead of ISOBMFF as the file format. In this case, the file generation unit 104 newly stores the 3DmodelDataMetadata element of the configuration information in the TrackEntry element. At this time, the file generation unit 104 sets the element type to binary. As EBML data, in the case where the 3D model data is a point cloud, the file generation unit 104 stores the number of points as binary data, and in the case where the 3D model data is a mesh, the file generation unit 104 stores the number of vertices of the face, the number of faces, and the number of vertices as binary data.

[0224] In each of the above-mentioned embodiments and modifications thereof, it has been described that there are multiple content configurations, but the above-mentioned embodiments and modifications thereof can also be applied to the case of determining whether a content configuration is a content configuration that can be reproduced. In addition, even in the case where the client device does not have the reproduction processing capability for 6DoF content with specific three-dimensional model data, the content configuration selection information can be used. For example, a content creator can distribute content in a configuration using one three-dimensional model data and a spherical video. In this case, by adding information indicating a spherical video that is not three-dimensional model data as the content configuration selection information, the client device can use the content configuration selection information to determine whether the content has a content configuration that can be reproduced.

[0225] [Hardware Configuration]

[0226] Fig. 22 is a hardware configuration diagram of a computer. The file generating device 1 and the client device 2 can be Fig. 22 The computer 90 shown in FIG. 9 is implemented. In the computer 90, a processor 91, a memory 92, a network interface 93, a nonvolatile memory 94, an input / output interface 95, and a display interface 96 are connected to each other via a bus.

[0227] For example, external devices such as input devices, output devices, storage devices, and drives are connected to the input / output interface 95. The input device is, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output device is, for example, a speaker, an output terminal, etc. The storage device is, for example, a hard disk, a random access memory (RAM) disk, etc. The drive drives a removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory. In addition, a display 98 as a display device is connected to the display interface 96.

[0228] The network interface 93 is connected to an external network. The file generation device 1 and the client device 2 are connected to each other via the network interface 93. In addition, the file generation device 1 and the client device 2 are connected to the Web server 3 via the network interface 93. The nonvolatile memory 94 is a built-in auxiliary storage device such as a hard disk or a solid state drive (SSD).

[0229] In the computer 90 configured as described above, the processor 91 loads a program stored in the nonvolatile memory 94 into the memory 92 via the bus to execute the program, thereby executing the above-mentioned series of processing. The memory 92 also appropriately stores data and the like required for the processor 91 to execute various types of processing.

[0230] The program executed by the processor 91 may be recorded in a removable medium, for example, as a package medium for application, etc. In this case, by mounting the removable medium on a drive as an external device 97 , the program can be installed in the nonvolatile memory 94 via the input / output interface 95 .

[0231] In addition, the program may be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program may be received through the network interface 93 and installed in the nonvolatile memory 94.

[0232] Alternatively, the program may be installed in the nonvolatile memory 94 in advance.

[0233] Although the embodiments of the present disclosure are described above, the technical scope of the present disclosure is not limited to the above embodiments, and various modifications can be made without departing from the gist of the present disclosure. In addition, components of different embodiments and modifications can be appropriately combined.

[0234] Note that the effects described in this specification are merely examples and are not limiting, and other effects may be provided.

[0235] Note that the present technology may also have the following configurations. (1)

[0237] An information processing device, comprising:

[0238] a pre-processing unit that generates, for one or more contents, content configuration selection information for determining whether each of the contents is reproducible, each of the contents having a content configuration including one or more three-dimensional objects and spatial arrangement information thereof to represent a virtual space; and

[0239] A file generating unit generates a file including data on the virtual space and the content configuration selection information. (2)

[0241] An information processing device according to (1), wherein the preprocessing unit sets the content configuration selection information to include availability identification information, and the availability identification information is used to determine whether decoding and rendering can be performed for the entire virtual space in a reproduction processing device that reproduces the content. (3)

[0243] The information processing device according to (1) or (2), wherein the pre-processing unit stores the content configuration selection information in a scene description. (4)

[0245] The information processing device according to (3), wherein the pre-processing unit stores the content configuration selection information in a switching node of the scene description. (5)

[0247] The information processing device according to (3), wherein the pre-processing unit stores the content configuration selection information of each content configuration in the scene description. (6)

[0249] The information processing device according to (2), wherein:

[0250] The pre-processing unit generates the content configuration selection information as metadata, and

[0251] The file generation unit generates a content file storing the content configuration selection information. (7)

[0253] The information processing device according to (6), wherein the file generation unit generates the content file as an ISO Base Media File Format (ISOBMFF) file, and stores the content configuration selection information in a 6DoFConentStructBox of a SampleEntry of the content file. (8)

[0255] An information processing device according to (2), wherein the preprocessing unit has the content configuration selection information of each group of the content configuration determined in advance, and the content configuration selection information of the group to which each of the contents belongs is set as the content configuration selection information of each of the contents. (9)

[0257] The information processing device according to (2), wherein:

[0258] The pre-processing unit generates the content configuration selection information as metadata, and

[0259] The file generating unit generates a metadata file storing the content configuration selection information. (10)

[0261] The information processing device according to (9), wherein the file generation unit generates the metadata file as a media presentation description (MPD) file, and stores the content configuration selection information in an AdaptationSet of the MPD file. (11)

[0263] The information processing device according to (2), wherein:

[0264] For a plurality of contents with different content configurations, the preprocessing unit generates a different scene description for each content configuration, and

[0265] The file generation unit stores the content configuration selection information in the 6DoFContentStructBox of the AdaptationSet of the MPD file or the SampleEntry of the ISOBMFF file for each of the scene descriptions. (12)

[0267] The information processing device according to any one of (1) to (11), wherein the pre-processing unit sets information indicating a reproduction processing capability with which the content can be reproduced as the content configuration selection information. (13)

[0269] The information processing device according to (12), wherein the pre-processing unit sets the content configuration selection information to include information indicating a reproduction processing capability with which a part of the content can be reproduced. (14)

[0271] An information processing method for causing a computer to perform the following operations:

[0272] generating, for one or more contents, content configuration selection information for determining whether each of the contents is reproducible, each of the contents having a content configuration including one or more three-dimensional objects and spatial arrangement information thereof to represent a virtual space; and

[0273] A file including data about the virtual space and the content configuration selection information is generated. (15)

[0275] A reproduction processing device, comprising:

[0276] a file acquisition unit that acquires, for one or more contents, a file including content configuration selection information for determining whether each of the contents is reproducible and data on the contents, each of the contents having a content configuration including one or more three-dimensional objects and spatial arrangement information thereof to represent a virtual space;

[0277] a file processing unit that acquires the content configuration selection information from the file acquired by the file acquisition unit, determines whether each of the contents can be reproduced based on the acquired content configuration selection information, and selects the content to be reproduced; and

[0278] A reproduction unit that reproduces the content selected by the file processing unit. (16)

[0280] A reproduction processing method is used to enable a computer to perform the following operations:

[0281] acquiring, for one or more contents, a file including content configuration selection information for determining whether each of the contents is reproducible, each of the contents having a content configuration including one or more three-dimensional objects and spatial arrangement information thereof to represent a virtual space;

[0282] acquiring the content configuration selection information from the acquired file, determining whether each of the contents is reproducible based on the acquired content configuration selection information, and selecting the content to be reproduced;

[0283] Obtain data about the selected content; and

[0284] The selected content is reproduced using the acquired data.

[0285] Reference numerals list

[0286] 1 File generation equipment

[0287] 2 Client Devices

[0288] 3 Web Server

[0289] 4 Network

[0290] 10 Generate processing unit

[0291] 11 Control Unit

[0292] 20 Reproduction Processing Unit

[0293] 21 Control unit

[0294] 101 Data Input Unit

[0295] 102 Preprocessing Unit

[0296] 103 Coding Unit

[0297] 104 File generation unit

[0298] 105 Transmission Unit

[0299] 201 File acquisition unit

[0300] 202 measurement unit

[0301] 203 File Processing Unit

[0302] 204 decoding processing unit

[0303] 205 Display control unit

[0304] 206 Display information generation unit

[0305] 207 Display unit

Claims

1. An information processing device, comprising: a preprocessing unit that generates content configuration selection information from content, wherein the content includes one or more three-dimensional objects and spatial arrangement information of the one or more three-dimensional objects, and the content configuration selection information is used to select a content configuration capable of reproduction processing; and a file generating unit, which generates a file including the content and the content configuration selection information; The pre-processing unit sets the content configuration selection information to include availability identification information, and the availability identification information is used to determine whether decoding and rendering can be performed on the content.

2. The information processing device according to claim 1, wherein: The pre-processing unit stores the content configuration selection information in a scene description.

3. The information processing device according to claim 2, wherein: The pre-processing unit stores content configuration selection information for each content configuration in the scene description.

4. The information processing device according to claim 1, wherein: The pre-processing unit generates the content configuration selection information as metadata, and The file generation unit generates a content file storing the content configuration selection information.

5. The information processing device according to claim 4, wherein: The file generation unit generates the content file as an ISO base media file format ISOBMFF file, and stores the content configuration selection information in a 6DoFConentStructBox of a SampleEntry of the content file.

6. The information processing device according to claim 1, wherein: The preprocessing unit has content configuration selection information of each group in which the content configuration is determined in advance, and sets the content configuration selection information of the group to which each of the contents belongs as the content configuration selection information of each of the contents.

7. The information processing device according to claim 1, wherein: The pre-processing unit generates the content configuration selection information as metadata, and The file generating unit generates a metadata file storing the content configuration selection information.

8. The information processing device according to claim 7, wherein: The file generation unit generates the metadata file as a media presentation description MPD file, and stores the content configuration selection information in an AdaptationSet of the MPD file.

9. The information processing device according to claim 1, wherein: For a plurality of contents with different content configurations, the preprocessing unit generates a different scene description for each content configuration, and The file generation unit stores the content configuration selection information in the 6DoFContentStructBox of the AdaptationSet of the MPD file or the SampleEntry of the ISOBMFF file for each of the scene descriptions.

10. The information processing device according to claim 1, wherein: The pre-processing unit sets information indicating a reproduction processing capability with which the content can be reproduced as the content configuration selection information.

11. The information processing device according to claim 10, wherein: The pre-processing unit sets the content configuration selection information to include information indicating a reproduction processing capability with which a part of the content can be reproduced.

12. An information processing method, for causing a computer to perform the following operations: Generating content configuration selection information from the content, wherein: The content includes one or more three-dimensional objects and spatial arrangement information of the one or more three-dimensional objects, and the content configuration selection information is used to select a content configuration capable of performing a reproduction process; as well as generating a file including the content and the content configuration selection information, The content configuration selection information is configured to include availability identification information, and the availability identification information is used to determine whether decoding and rendering can be performed on the content.

13. A reproduction processing device, comprising: A file acquisition unit, which acquires a file including content and content configuration selection information generated from the content, wherein the content includes one or more three-dimensional objects and spatial arrangement information of the one or more three-dimensional objects, and the content configuration selection information is used to select a content configuration capable of performing a reproduction process; a file processing unit that acquires the content configuration selection information from the file acquired by the file acquisition unit, determines whether each of the content configurations can be reproduced based on the acquired content configuration selection information, and selects the content configuration that can be reproduced; and a reproduction unit that reproduces content corresponding to the content configuration selected by the file processing unit, The content configuration selection information is configured to include availability identification information, and the availability identification information is used to determine whether decoding and rendering can be performed on the content.

14. A reproduction processing method, configured to cause a computer to perform the following operations: Obtaining a file including content and content configuration selection information generated from the content, wherein: The content includes one or more three-dimensional objects and spatial arrangement information of the one or more three-dimensional objects, and the content configuration selection information is used to select a content configuration capable of performing a reproduction process; acquiring the content configuration selection information from the acquired file, determining whether each of the content configurations can be reproduced based on the acquired content configuration selection information, and selecting the content configuration that can be reproduced; as well as reproducing the content corresponding to the selected content configuration, The content configuration selection information is configured to include availability identification information, and the availability identification information is used to determine whether decoding and rendering can be performed on the content.

Images