Information processing device, information processing method, and program medium

Abstract

The present invention relates to an information processing device, method, and program capable of efficient processing on a client. From 3D data representing a three-dimensional structure, identification information associated with a stream of shape information representing the shape of the three-dimensional structure and a stream of attribute information representing the attributes of the three-dimensional structure is generated for each of the streams. A file containing multiple sets of the generated identification information is generated, and the file has a sample structure that clearly shows the boundaries between the shape information and the attribute information. The present invention can be applied, for example, to a data generation device that generates data for distributing point clouds, or to a data playback device for playing back point clouds.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. More specifically, the present disclosure relates to an information processing device, an information processing method, and a program for enabling a client to efficiently perform processing. Background Art

[0002] In the past, as disclosed in Non-Patent Document 1, a method for compressing a point cloud, which is a collection of points located in a three-dimensional space and having both shape information and attribute information (particularly color information), has been specified.

[0003] One method of point cloud compression involves separating point cloud data into a shape (indicating a 3D shape) and attributes (e.g., color and reflectance) and encoding the separated information. This method is called G-PCC (Shape-based Point Cloud Coding).

[0004] It is expected that the G-PCC stream generated by this encoding method will be downloaded for reproduction and distributed over an IP (Internet Protocol) network. Therefore, as disclosed in Non-Patent Document 2, MPEG (Moving Picture Experts Group) has begun research on distribution technology within the existing framework of ISOBMFF / DASH (ISO Base Media File Format / Dynamic Adaptive Streaming over HTTP) to reduce the impact on existing distribution platforms and to implement services as soon as possible.

[0005] Reference List

[0006] Non-patent literature

[0007] Non-patent document 1:

[0008] MPEG-I Part 5Point Cloud Compression(ISO / IEC 23090-5)

[0009] Non-patent document 2:

[0010] w17675, First idea on Systems technologies for Point Cloud Coding, April 2018, San Diego, USA Summary of the Invention

[0011] Technical issues

[0012] Furthermore, the G-PCC stream generated in the past was a single, continuous stream, constructed in a way that prevented the client from accessing shapes and attributes independently of each other. Therefore, even in scenarios where only a shape or only one of multiple attributes was needed, the client was not allowed to access and obtain the required content. Therefore, before performing the necessary processing, the client was required to obtain all shapes and attributes that comprise the G-PCC stream. This caused processing overhead for the client and made it difficult for the client to perform processing efficiently.

[0013] The present disclosure has been made in view of the above circumstances, and is intended to enable a client to efficiently perform processing.

[0014] Solution to the problem

[0015] According to a first aspect of the present disclosure, an information processing device is provided, which includes a file generating unit, which is configured to generate, based on 3D data representing a three-dimensional structure, identification information related to a stream of shape information indicating the shape of the three-dimensional structure and identification information related to a stream of attribute information indicating the attributes of the three-dimensional structure, for each of a stream of shape information indicating the shape of the three-dimensional structure and a stream of attribute information indicating the attributes of the three-dimensional structure, respectively, and the file generating unit is configured to generate a file including the generated multiple pieces of identification information.

[0016] According to the first aspect of the present disclosure, an information processing method or program is also provided, which includes: based on 3D data representing a three-dimensional structure, generating identification information related to the stream of shape information indicating the shape of the three-dimensional structure and identification information related to the stream of attribute information indicating the attributes of the three-dimensional structure for each of the streams of shape information indicating the shape of the three-dimensional structure and the stream of attribute information indicating the attributes of the three-dimensional structure; and generating a file including the generated multiple pieces of identification information.

[0017] Therefore, according to the first aspect of the present disclosure, identification information related to each stream of shape information indicating the shape of the three-dimensional structure and identification information related to each stream of attribute information indicating the attributes of the three-dimensional structure are generated based on 3D data representing the three-dimensional structure. Then, a file including the generated plurality of pieces of identification information is generated.

[0018] According to a second aspect of the present disclosure, an information processing device is provided, which includes: an extraction unit configured to extract the shape information and the attribute information in an identifiable manner according to usage and based on the identification information from a file including identification information related to a stream of shape information indicating the shape of a three-dimensional structure represented by 3D data and identification information related to a stream of attribute information indicating the attributes of the three-dimensional structure, wherein the identification information is generated separately for each of the streams of shape information indicating the shape of the three-dimensional structure and the streams of attribute information indicating the attributes of the three-dimensional structure based on the 3D data; and a construction unit configured to construct a three-dimensional structure by using the shape information or the attribute information both extracted by the extraction unit.

[0019] According to a second aspect of the present disclosure, an information processing method or program is also provided, which includes: from a file including identification information related to a stream of shape information indicating the shape of a three-dimensional structure represented by 3D data and identification information related to a stream of attribute information indicating the attributes of the three-dimensional structure, identifiably extracting the shape information and the attribute information according to usage and based on the identification information, wherein the identification information is generated separately for each of the streams of shape information indicating the shape of the three-dimensional structure and the streams of attribute information indicating the attributes of the three-dimensional structure based on the 3D data; and constructing a three-dimensional structure by using the shape information or attribute information that has been extracted.

[0020] Therefore, according to a second aspect of the present disclosure, a file is provided that includes identification information related to each stream of shape information indicating the shape of a three-dimensional structure represented by 3D data, and identification information related to each stream of attribute information indicating the attributes of the three-dimensional structure. The identification information is generated from the 3D data, and the shape information and attribute information are identifiably extracted from the file based on the identification information according to a usage scenario. Then, a three-dimensional structure is constructed using the extracted shape information or attribute information. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] [ Figure 1 ] Figure 1 is a diagram showing a conventional flow structure.

[0022] [ Figure 2 ] Figure 2 is a diagram illustrating a typical PC sample that is redefined.

[0023] [ Figure 3 ] Figure 3 is a diagram showing a typical PC sample.

[0024] [ Figure 4 ] Figure 4is a diagram showing an example of information stored in a header.

[0025] [ Figure 5 ] Figure 5 is a diagram illustrating an example in which a field is added to signal an attribute type.

[0026] [ Figure 6 ] Figure 6 is a diagram showing a typical attribute parameter set.

[0027] [ Figure 7 ] Figure 7 is a diagram showing an example in which a G-PCC stream is stored in one track of ISOBMFF.

[0028] [ Figure 8 ] Figure 8 This is a diagram showing the definition of codec_specific_parameters.

[0029] [ Figure 9 ] Figure 9 This is a diagram showing the structure of a newly defined PCSampleEntry (PC sample entry).

[0030] [ Figure 10 ] Figure 10 is a diagram showing an example of a PCParamSetBox (PC parameter set box).

[0031] [ Figure 11 ] Figure 11 is a block diagram illustrating a configuration example of an information processing apparatus for executing file generation processing.

[0032] [ Figure 12 ] Figure 12 This is a block diagram illustrating a configuration example of an information processing device for executing point cloud (Point Cloud) reproduction processing.

[0033] [ Figure 13 ] Figure 13 This is a flowchart of the description document generation process.

[0034] [ Figure 14 ] Figure 14 This is a flowchart illustrating point cloud reproduction processing.

[0035] [ Figure 15 ] Figure 15 is a diagram showing a typical ISOBMFF structure stored into one track by the second recognition enabling method.

[0036] [ Figure 16 ] Figure 16is a diagram showing an alternative example of the ISOBMFF structure stored in one track.

[0037] [ Figure 17 ] Figure 17 is a diagram showing an example in which a shape bitstream and an attribute bitstream are stored in two tracks of ISOBMFF.

[0038] [ Figure 18 ] Figure 18 This is a diagram depicting the typical syntax of PCMultiStreamBox.

[0039] [ Figure 19 ] Figure 19 is a diagram showing a first example of an ISOBMFF structure stored into two tracks by the second recognition enabling method.

[0040] [ Figure 20 ] Figure 20 is a diagram showing an alternative example of PCMultiStreamBox.

[0041] [ Figure 21 ] Figure 21 is a diagram showing a typical syntax of a PCAttributeTrackGroupBox.

[0042] [ Figure 22 ] Figure 22 is a diagram showing another alternative example of PCMultiStreamBox.

[0043] [ Figure 23 ] Figure 23 is a diagram illustrating an example of signaling a shape track and an attribute track associated by a track group.

[0044] [ Figure 24 ] Figure 24 is a diagram showing an example of a newly defined PCStreamGroupBox.

[0045] [ Figure 25 ] Figure 25 is a diagram showing a second example of an ISOBMFF structure stored into two tracks by the second recognition enabling method.

[0046] [ Figure 26 ] Figure 26 This is a diagram illustrating DASH signaling.

[0047] [ Figure 27 ] Figure 27 is another diagram illustrating DASH signaling.

[0048] [ Figure 28 ] Figure 28 This is a diagram showing an overview of the SubSampleInformationBox (subsample information box).

[0049] [ Figure 29 ] Figure 29 This is a diagram showing an overview of a sample group (Sample Group).

[0050] [ Figure 30 ] Figure 30 is a block diagram showing a configuration example of a data generating device.

[0051] [ Figure 31 ] Figure 31 is a block diagram showing a configuration example of a data reproducing apparatus.

[0052] [ Figure 32 ] Figure 32 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present technology is applied. DETAILED DESCRIPTION

[0053] Specific embodiments of applying the present technology are described in detail below with reference to the accompanying drawings.

[0054] <Existing stream structure>

[0055] Before describing the flow structure to which this technology is applied, Figure 1 Describe the existing flow structure.

[0056] Figure 1 The structure of a stream encoded by G-PCC is shown in FIG. Such a stream is called a G-PCC stream (or simply a PC stream).

[0057] For example, a G-PCC stream consists of a stream as a sequence of point cloud frames arranged in decoding order. Here, a point cloud frame (hereinafter also referred to as a PC frame) represents a point cloud displayed simultaneously. A PC frame is a shape bitstream ( Figure 1 Geom) and attribute bitstream ( Figure 1 Attr indicated in ).

[0058] It should be noted that a PC frame has one shape bitstream and multiple attribute bitstreams (in Figure 1In the example of two attribute streams). The SPS (Sequence Parameter Set) stores meta information of each G-PCC stream sequence as common information required to decode the shape bitstream and the attribute bitstream. The GPS (Geometry Parameter Set) stores information required to decode the shape bitstream, and the APS (Attribute Parameter Set) stores information required to decode the attribute bitstream.

[0059] The client uses separate decoders to decode the shape bitstream and attribute bitstream in the G-PCC stream. The client first decodes the shape bitstream to generate point cloud content lacking color. Thereafter, by referring to the decoded shape bitstream information, the client decodes the attribute bitstream to add attributes such as color and reflection to the content based on the referenced information.

[0060] On the other hand, in addition to the general use case of reproducing a G-PCC stream of colored point cloud data, there are also use cases that do not require attribute information such as color and reflectance, and only use the three-dimensional shape information from the point cloud data. For example, there may be a use case in which colored map information obtained by LiDAR (Light Detection and Ranging) and a camera is retained as point cloud data, and only terrain information (i.e., three-dimensional shape information) is extracted from this point cloud data and used for vehicle driving control.

[0061] In the case where there are multiple pieces of attribute information such as color and reflection, there may be use cases where, for example, it is desirable to use shape information along with the color attribute without requiring the reflection attribute. There may also be use cases where, given multiple color attributes, it is desirable to extract and use only one color attribute.

[0062] However, in the G-PCC stream structure, for example, when there is no clear boundary information between the shape and multiple attributes, decoding the PC frame provided by the attributes requires processing a single bitstream that includes the shape and multiple attributes. Therefore, in such a use case, the client is required to obtain the entire shape and attributes that make up the PC frame and decode the stream in sequence from the beginning. Having to decode the unused attribute bitstream reduces processing efficiency.

[0063] To address the issue of reduced efficiency, the present disclosure proposes a method by which information related to the generated shape bitstream and attribute bitstream is regenerated as identification information to be included in a file generated for storing the identification information. The specific embodiments discussed below are arranged to enable clients to independently access shapes and attributes, thereby improving processing efficiency in the above-mentioned use cases.

[0064] <Definition of PC Sample>

[0065] Refer to the following Figure 2 to describe the point cloud sample (hereinafter referred to as PC sample), which is redefined to correspond to the sample defined by ISOBMFF.

[0066] For example, a PC frame, which is a unit of the point cloud data to be simultaneously displayed, is defined as being composed of one PC sample. That is, the G-PCC stream is composed of multiple PC samples, and each PC sample has a structure similar to the structure of the Figure 2 shown PC frame. Similar to a PC frame, a client can decode a PC sample to construct the point cloud for simultaneous display.

[0067] In this way, the PC sample is composed of the shape bitstream and the attribute bitstream that make up the point cloud data to be simultaneously displayed.

[0068] <First Identification Enablement Method>

[0069] Refer to the following Figures 3 to 6 to describe the sample structure that specifies the boundaries of the shape and attributes, as the first identification enablement method that enables the client to identify the shape and attributes for separation.

[0070] Figure 3 Shows a typical structure of a PC sample composed of one shape subsample and two attribute subsamples.

[0071] As Figure 3 [[ID=)31]]shown, a PC sample is composed of a shape subsample, a first attribute subsample, and a second attribute subsample arranged continuously. Each subsample is constructed to have header information and a corresponding bitstream, and the header stores information about the bitstream for separating it.

[0072] Figure 4 Shows an example of the information stored in the Figure 3 header

[0073] As Figure 4 shown, the header stores size information (size_of_bitstream) indicating the size of the shape bitstream or the attribute bitstream and type information (type) indicating the type of each bitstream that makes up the PC sample, as identification information. For example, when the information indicating the bitstream type is 0, it indicates the shape bitstream. When the information indicating the bitstream type is 1, it indicates the attribute bitstream. The identification information identifies each bitstream and is generated for each bitstream.

[0074] It should be noted that, for example, in the case of ISOBMFF, SPS, GPS, and APS that are not stored in the PC sample are stored in the sample entry (see later discussion). Figure 9 and Figure 10 ).

[0075] The above signaling enables the client to identify the boundaries of each bitstream based on the header. Therefore, the client can extract only the shape bitstream from the PC sample for independent decoding. Similarly, the client can extract only the required attribute bitstream from the PC sample for decoding. The client can then easily input the shape bitstream and attribute bitstream into different corresponding decoders for decoding.

[0076] In addition, in Figure 5 In the case where the bold letters in the header indicate type = 1 (i.e., the bitstream is an attribute bitstream), the header may be extended to add a field (attribute_type) that signals the attribute type. For example, if the attribute type signaled field is 0, the attribute type is indicated as color. If the attribute type signaled field is 1, the attribute type is indicated as reflection. Alternatively, the attribute type signaled field may be used to indicate other types.

[0077] As another alternative, the high-level syntax of the G-PCC stream may be extended to signal the attribute type.

[0078] For example, the attribute parameter set can be extended to signal attr_type, such as Figure 6 The attribute parameter set to be referenced is signaled as an identifier in the attribute bitstream. Incidentally, the semantics are similar to those of attribute_type described above.

[0079] As described above, the first recognition enabling method enables the client to identify the shape and attributes with reference to the header and access the identified content separately. This makes it possible to efficiently perform processing even when only desired attributes are used.

[0080] <Second Identification Enabling Method>

[0081] Refer to the following Figures 7 to 24 A track extension of ISOBMFF for storing subsample information, shapes, and attributes is described as a second recognition enabling method that enables the client to recognize shapes and attributes for separation.

[0082] First refer to Figures 7 to 15 This section describes the case where the shape bitstream and attribute bitstream of the G-PCC stream are stored in one track of ISOBMFF using the second identification enabling method.

[0083] Figure 7 An example is shown in which a G-PCC stream is stored in one ISOBMFF track.

[0084] For example, ISOBMFF's moov stores a redefined PCSampleEntry (see Figure 9 ). PCSampleEntry is composed of Figure 10 The PCParamSetBox structure shown in .

[0085] In addition, ISOBMFF's moov stores subs (subsamples) (SubSampleInformationBox). Figure 7 As shown, the SubSampleInformationBox can be used to signal the boundaries of the shape subsamples and attribute subsamples #0 to #N in the PC sample. Incidentally, reference will be made to the later discussed Figure 28 SubSampleInformation (subsample information) is explained by referring to the summary of SubSampleInformationBox in .

[0086] The samples stored in the mdat of ISOBMFF are such as Figure 3 The header of a PC sample stores size information and type information.

[0087] Here, if Figure 8 As shown, SubSampleInformationBox defines codec_specific_parameters as subsample information fixed for each codec. That is, when the value of codec_specific_parameters is 0, the subsample is indicated as a shape subsample. When the value of codec_specific_parameters is 1, the subsample is indicated as an attribute subsample.

[0088] It should be noted that the subsample information may be provided in units of consecutive attribute subsample groups.In addition, codec_specific_parameters may be further extended to signal the attribute bitstream type of the subsample (ie, color attribute, reflectance attribute, etc.).

[0089] Figure 9 The PCSampleEntry structure redefined in the present disclosure is shown.

[0090] exist Figure 9 In the structure shown, the sample entry for the ISOBMFF track storing the G-PCC stream is, for example, 'pcbs'.

[0091] PCParamSetBox Storage Figure 10 The various parameter sets (sequence parameter sets, shape parameter sets, and attribute parameter sets) indicated in the PCSampleEntry. For example, when decoding PC samples, these parameter sets are referenced. Generally, parameter sets are not information to be modified in units of PC samples. Therefore, storing parameter sets in PCSampleEntry eliminates the need to repeatedly signal the same information in each PC sample, which reduces file size. Incidentally, locations where parameter sets can be stored include seq_parameter_set_rbsp(), geometry_parameter_set(), and attribute_parameter_set() in PCParamSetBox.

[0092] The above signaling enables the client to identify the boundary between shape subsamples and attribute subsamples (or between attribute subsamples) without parsing the subsample content. In other words, the client can independently extract and decode only the shape bitstream from the PC sample through a simplified process that only refers to the system layer signaling. Similarly, the client can extract and decode only the necessary attribute bitstream from the PC sample. In addition, the client can easily input the shape bitstream and attribute bitstream into different corresponding decoders.

[0093] <Configuration Example of Information Processing Apparatus>

[0094] Figure 11 : is a block diagram showing a configuration example of an information processing apparatus that performs file generation processing involving generating a PC stream from point cloud data and generating a file defined by ISOBMFF for storing the generated PC stream.

[0095] like Figure 11 As shown, the information processing apparatus 11 includes a separation unit 21 , an encoding unit 22 , a PC stream generation unit 23 , and a file generation unit 24 .

[0096] The separation section 21 separates shapes and attributes from the input point cloud data, and supplies the separated shapes and attributes to the encoding section 22 .

[0097] The encoding unit 22 separately encodes the shape and attributes provided by the separation unit 21 to generate a shape bitstream and an attribute bitstream. At this time, the encoding unit 22 generates parameter sets (sequence parameter set, shape parameter set, and attribute parameter set) to be referenced when decoding the shape bitstream and the attribute bitstream. For example, the encoding unit 22 uses an octree to encode the shape and uses lifting coding or region-adaptive Haar transform to encode the attributes. At this time, the encoding unit 22 can use two encoders 25-1 and 25-2 to encode the shape and attributes in parallel.

[0098] The PC stream generation unit 23 generates a shape bit stream and an attribute bit stream such as a PC frame by interleaving the shape bit stream and the attribute bit stream encoded by the encoding unit 22 in units constituting the PC frame. Figure 3 Then, the PC stream generation unit 23 generates a PC stream composed of a plurality of PC samples and supplies the generated PC stream to the file generation unit 24.

[0099] The file generation unit 24 stores the shape bitstream and the attribute bitstream in one track of a file defined by ISOBMFF, thereby generating a file. At this time, the file generation unit 24 generates size information indicating the size of the shape bitstream or the attribute bitstream and type information indicating the type of each bitstream constituting the PC sample, and stores the generated information in the header of the generated file as the above-mentioned identification information.

[0100] Therefore, the information processing apparatus 11 configured as described above generates a PC stream from point cloud data, and outputs a file in which the PC stream is stored in one track.

[0101] Furthermore, when the information processing apparatus 11 stores the G-PCC stream in a file defined by ISOBMFF that signals metadata, the file sample defined by ISOBMFF is a PC sample (see Figure 3 and Figure 7 ).

[0102] Figure 12 This is a block diagram illustrating a configuration example of an information processing apparatus that performs point cloud reproduction processing on the client side of reproducing content, the processing involving generating a display image from a file to reproduce point cloud data.

[0103] like Figure 12 As shown, the information processing device 12 includes an extraction unit 31 , a decoding unit 32 , a construction unit 33 , and a display processing unit 34 .

[0104] The extraction unit 31 extracts the shape bitstream and attribute bitstream corresponding to the reproduction time from the file based on the information notified in the ISOBMFF box (e.g., the identification information described above), and provides the extracted bitstream to the decoding unit 32. In this case, the extraction unit 31 can identify the shape and attribute according to the various usage scenarios described above and extract only the required shape or attribute. Alternatively, the extraction unit 31 can extract the shape bitstream and attribute bitstream corresponding to the reproduction time from the file based on the identification information stored in the header of the subsample, and provide the extracted bitstream to the decoding unit 32.

[0105] The decoding section 32 decodes the shape bitstream and the attribute bitstream supplied from the extraction section 31 with reference to each parameter set, and supplies the decoded shape and attribute to the construction section 33. At this time, the decoding section 32 may decode the shape bitstream and the attribute bitstream individually and in parallel using two decoders 35-1 and 35-2.

[0106] The construction section 33 constructs point cloud data by using the shape and attributes supplied from the decoding section 32 .

[0107] The display processing section 34 renders the point cloud data constructed by the construction section 33 in correspondence with a display device of a client, thereby generating a display image and causing a display device (not shown) to display the generated display image.

[0108] Thus, the information processing apparatus 12 configured as described above reproduces point cloud data from a file, renders the point cloud data into a display image, and displays the image.

[0109] <Example of File Generation Processing and Point Cloud Reproduction Processing>

[0110] Figure 13 Is explained by Figure 11 Flowchart of a file generation process for generating a file from point cloud data, performed by the information processing device 11 in FIG.

[0111] For example, the process starts when point cloud data is input to the information processing device 11. In step S11, the separation unit 21 separates the shape and attributes from the input point cloud.

[0112] In step S12, the encoding unit 22 encodes the shape and attribute separated by the separation unit 21 in step S11 to generate a shape bitstream and an attribute bitstream. At this time, the encoding unit 22 generates a parameter set.

[0113] In step S13 , the PC stream generation unit 23 generates a PC stream by interleaving the shape bitstream and the attribute bitstream generated in step S12 in units (PC samples) constituting a PC frame.

[0114] In step S14, the file generation unit 24 generates a file by storing the PC stream generated in step S13 in an ISOBMFF file that signals a frame containing metadata. Furthermore, the file generation unit 24 generates size and type information and stores this information in the header of the file. At this point, the ISOBMFF sample is a PC sample.

[0115] Figure 14 Is explained by Figure 12 1 is a flowchart of a point cloud reproduction process performed by the information processing device 12 to generate a display image from a file generated by the information processing device 11 so as to reproduce the generated display image.

[0116] For example, the process starts when a file is supplied to the information processing device 12 from the start of the process. In step S21, the extraction unit 31 extracts the shape bitstream and attribute bitstream corresponding to the reproduction time from the file based on information signaled by the frame in the ISOBMFF (e.g., the identification information described above). Alternatively, the extraction unit 31 may extract the shape bitstream and attribute bitstream corresponding to the reproduction time from the file based on identification information stored in the subsample header.

[0117] In step S22, by referring to the parameter set, the decoding section 32 separately decodes the shape bitstream and the attribute bitstream extracted in step S21. At this time, the shape bitstream and the attribute bitstream are separately decoded using a dual decoder instance.

[0118] In step S23 , the construction section 33 constructs point cloud data by using the shape and attributes obtained by decoding in step S22 .

[0119] In step S24 , the display processing unit 34 presents the point cloud data constructed in step S23 as a display image in coordination with the display device of the client, and causes the display image to be displayed.

[0120] In step S25, the extraction unit 31 determines whether the PC stream has reached its end. If the PC stream has not reached its end, control returns to step S21. If the PC stream has reached its end, the process ends.

[0121] The above-described file generation processing and point cloud reproduction processing contribute to efficient execution of processing on the client side.

[0122] Figure 15 A first example of an ISOBMFF structure when a shape bitstream and an attribute bitstream are stored in one track of ISOBMFF by the second recognition enabling method is shown.

[0123] As an alternative example, without using PC samples, the shape bitstream and attribute bitstream that make up a PC frame can each be stored as a sample in ISOBMFF, such as Figure 16 At this point, the shape subsample and attribute subsample become shape samples and attribute samples respectively.

[0124] As another alternative, the shape samples and attribute samples that make up one PC frame can be signaled using sample groups. Similarly, the boundaries between shape samples and attribute samples can be signaled using sample groups. Figure 29 To illustrate the sample group.

[0125] Here, in the case where PC samples that are not motion pictures and have no temporal information (i.e., not a stream of multiple consecutive PC samples but a stream of a single PC sample) are stored in ISOBMFF, similar information only needs to be signaled in MedaDataBox.

[0126] Next, refer to Figures 17 to 25 It describes how to use the second identification enabling method to store the shape bitstream and attribute bitstream of the G-PCC stream separately into each track of the ISOBMFF.

[0127] Figure 17 An example of two tracks is shown, in which the G-PCC stream is divided into a shape bitstream and an attribute bitstream, the shape bitstream is stored in one track of ISOBMFF, and the attribute bitstream is stored in another track of ISOBMFF.

[0128] Therefore, when the shape bitstream and attribute bitstream are stored separately in two tracks as described above, the client can process only the track storing the shape bitstream to easily decode the shape bitstream alone. Similarly, the client can process only the track storing one attribute bitstream to easily decode the necessary attribute bitstream alone. Furthermore, the client can easily input the shape bitstream and attribute bitstream into different corresponding decoders.

[0129] like Figure 17 As shown, the track storing the shape bitstream is called the shape track or main track, and the track storing the attribute bitstream is called the attribute track or sub-track. The association between these tracks is signaled through track references and the newly defined PCMultiStreamBox.

[0130] Also in Figure 17 In the example shown, only the main track has a PCParamSetBox, which includes a sequence parameter set, a shape parameter set, and an attribute parameter set. Alternatively, both the main track and the sub-track can have PCParamSetBoxes. In this case, the PCParamSetBox in the main track can include a sequence parameter set and a shape parameter set, while the PCParamSetBox in the sub-track can include an attribute parameter set.

[0131] Furthermore, sample groups indicating the boundaries of PC frames may be signaled.In addition, one shape track may be associated with multiple attribute tracks of different bit rates.

[0132] Figure 18 Shows the typical syntax of PCMultiStreamBox.

[0133] For example, if isGeometryStream=1, the shape track is indicated. Otherwise, the attribute track is indicated. In the case of the shape track, the track_id of the attribute track associated with it is signaled.

[0134] The above signaling with the indicated track references allows the client to identify which is the shape track by simply parsing the TrackReferenceBox of any one track. Thereafter, the client can parse the PCMultiStreamBox of the shape track to identify all associated attribute tracks. In other words, the client can identify the entire structure by simply parsing at most two tracks. In particular, this simplifies the handling of cases where multiple attribute tracks are associated.

[0135] Alternatively, in the case where PCMultiStreamBox is extended with isGeometryBitstream=0 (i.e., the bitstream is an attribute bitstream), a field (attribute_type) may be added to signal the attribute type (color, reflection, etc.). As another alternative, the attribute track may be arranged to include multiple attribute bitstreams corresponding to multiple attribute types.

[0136] Figure 19 An example of an ISOBMFF structure is shown, in which the shape bitstream and the attribute bitstream are stored separately into two tracks of the ISOBMFF through the second identification enabling method.

[0137] As another alternative example of PCMultiStreamBox syntax, multiple attribute tracks can be commonly signaled by a track group and associated with the attribute track group through attribute_track_id, such as Figure 20 shown.

[0138] In this case, if Figure 21 As shown, the track group newly defines PCAttributeTrackGroupBox (PC attribute track group box), where track_group_type is 'pctg'.

[0139] In addition, if Figure 22 As shown in bold letters in [ ], PCMultiStreamBox can be supplemented with isInGeometryStream, which indicates whether the attributes are stored in the same track as the shape. This provides a box that can signal the case where both the shape and the attributes are stored in one track as described above.

[0140] Alternatively, instead of PCMultiStreamBox, one can Figure 23 The shape track and attribute track associated by the track group are shown signaled.

[0141] In addition, if Figure 24 As shown, the track group newly defines PCStreamGroupBox (PC stream group box), where track_group_type is 'pcgp'. At this time, the tracks of pcgp with the same track_group_id constitute a single PC stream. For example, if isGeometry = 0, the track is indicated as an attribute track. If isGeometry = 1, the track is indicated as a shape track.

[0142] Alternatively, in case PCStreamGroupBox is extended and isGeometry=0 (ie, the bitstream is an attribute bitstream), a field (attribute_type) may be added to signal the attribute type (color, reflection, etc.).

[0143] Figure 25 A second example of ISOBMFF is shown when the shape bitstream and the attribute bitstream are stored in one track of ISOBMFF through the second recognition enabling method.

[0144] The second identification enabling method described above enables the client to identify shapes and attributes by referring to information related to subsamples (SubSampleInformationBox) or information indicating the association relationship between tracks (PCMultiStreamBox), and to obtain separate access to the identified content. Therefore, even in the case of using only the desired shape or necessary attributes, the client can perform processing efficiently.

[0145] <Third Identification Enabling Method>

[0146] Refer to the following Figure 26 and Figure 27 Explained is an association method in DASH MPD (Media Presentation Description) as a third identification enabling method for enabling a client to identify shapes and attributes so as to separate them in a case where the shapes and attributes are stored in separate tracks.

[0147] For example, in DASH MPD, SupplementalProperty or EssentialProperty@schemeIdUri==“urn:mpegI:pointcloud” is defined as a PC component descriptor, such as Figure 26In addition, in DASH MPD, Representation indicates an attribute or a shape.

[0148] Additionally, Representation@dependencyId is used to provide an association from the property representation to the shape representation. This is required for dependency relationships, as this relationship dictates that properties can only be mapped when a shape graphic exists at rendering time.

[0149] It should be noted that in case the representation is a property, a property type descriptor (SupplementalProperty or EssentialProperty@schemeIdUri == "urn:mpeg:mpegI:pc_attribute_type") may be added to signal the property type (color, reflection, etc.).

[0150] This signaling allows the client to refer to the PC component descriptor when obtaining DASH-delivered content and independently obtain the shape bitstream that constitutes the PC stream. Similarly, the client can independently decode the attribute bitstream. Furthermore, depending on network bandwidth, for example, the client can switch the image quality of the attribute bitstream associated with the shape bitstream.

[0151] In addition, if Figure 27 As shown, Representation@associationID can be used to provide an association from a shape representation to an attribute representation. Here, the associationType is "patt". Since the shape itself can represent a colorless point cloud, an association using associationtId is appropriate.

[0152] The third identification enabling method described above enables the client to identify shapes and attributes based on associations in the DASH MPD and obtain separate access to the identified content. Therefore, even in a use case where only desired attributes are used, the client can perform processing efficiently. For the third identification enabling method, as with the second identification enabling method, such as Figure 3 The samples of the PC samples detected in the ISOBMFF are stored in the mdat of the ISOBMFF, and the header of the sample stores the size information and type information.

[0153] Here, Figure 28 An overview of the SubSampleInformationBox is shown.

[0154] like Figure 28As shown, a continuous specific byte area in a sample is defined as a subsample. In addition, the definition of a subsample is fixed for each encoding codec. For example, in the case of HEVC, the NAL unit becomes a subsample. In addition, in the case of SubSampleInformationBox, information can be added to each subsample.

[0155] also, Figure 29 An overview of the sample group is shown.

[0156] like Figure 29 As shown, the grouping_type of the Sample To Group Box indicates the grouping_type of the associated Sample Group Description Box. Sample_count and group_description_index are signaled for each entry. The group_description_index indicates the index of the associated Group Entry, and the sample_count indicates the number of samples belonging to the group entry.

[0157] <System Configuration>

[0158] Refer to the following Figure 30 and Figure 31 The system configuration of a data generating device and a data reproducing device to which the present technology is applied will be described.

[0159] Figure 30 is a block diagram showing a configuration example of a data generating device.

[0160] like Figure 30 As shown, the data generation device 51 includes a control unit 61 and a file generation unit 62. The file generation unit 62 includes a data input unit 71, a data encoding and generation unit 72, an MPD file generation unit 113, a recording unit 74, and an upload unit 75. The data encoding and generation unit 72 includes a pre-processing unit 76, an encoding unit 77, and a segment file generation unit 78.

[0161] For example, the pre-processing unit 76 corresponds to Figure 11 The above-mentioned separation unit 21 in . The pre-processing unit 76 performs processing to separate the shape and attributes from the input point cloud.

[0162] The encoding unit 77 corresponds to Figure 11 The encoding unit 22 in the above-mentioned encoding unit 77 performs a process of encoding the shape bitstream, the attribute bitstream and the parameter set.

[0163] The segment file generation unit 78 corresponds to Figure 11The above-mentioned PC stream generating section 23 and file generating section 24 in the .segment file generating section 78 performs processing of generating a PC stream and further generating a file in which the PC stream is stored in ISOBMFF.

[0164] Figure 31 is a block diagram showing a configuration example of a data reproducing apparatus.

[0165] like Figure 31 As shown, the data reproduction device 52 includes a control unit 81 and a reproduction processing unit 82. The reproduction processing unit 82 includes an MPD file acquisition unit 91, an MPD file processing unit 92, a segment file acquisition unit 93, a display control unit 94, a data analysis and decoding unit 95, and a display unit 96. The data analysis and decoding unit 95 includes a segment file processing unit 97, a decoding unit 98, and a display information generation unit 99.

[0166] For example, the segment file processing unit 97 corresponds to Figure 12 The segment file processing unit 97 extracts the shape bitstream and attribute bitstream corresponding to the playback time according to the usage.

[0167] In addition, the decoding unit 98 corresponds to Figure 12 The decoding unit 98 performs processing for decoding the shape bitstream and the attribute bitstream separately.

[0168] In addition, the display information generating unit 99 corresponds to Figure 12 The display information generating unit 99 executes the processing of constructing a point cloud, rendering the point cloud, and displaying the rendered point cloud as a display image.

[0169] As described above, this technology defines a sample structure that specifies the boundaries between shapes and attributes, thereby enabling clients to easily and independently access shapes and attributes. Furthermore, this technology separates the point cloud stream into a shape bitstream and an attribute bitstream, storing them in two ISOBMFF tracks, allowing clients to process each track separately. Furthermore, as described above, this technology defines an association method in the DASH MPD when shapes and attributes are stored in separate tracks, allowing clients to easily and independently access shapes and attributes.

[0170] This enables clients to decode shapes individually or only the desired attributes out of multiple attributes.

[0171] Thus, for example, in use cases where attribute information such as color and reflection is not required and only three-dimensional shape information in the point cloud is used, or where only one of multiple pieces of attribute information such as color and reflection is to be used along with the shape for preview purposes, the client can perform processing efficiently.

[0172] <Computer Configuration Example>

[0173] The above-mentioned series of processing (information processing method) can be executed by hardware or software. In the case of executing the series of processing by software, for example, a program constituting the software is installed in a general-purpose computer.

[0174] Figure 32 : is a block diagram showing a configuration example of a typical computer in which a program for executing the above-described series of processes is installed.

[0175] The program may be recorded in advance on the hard disk 105 or the ROM 103 serving as a recording medium within the computer.

[0176] Alternatively, the program may be stored (recorded) on a removable recording medium 111 driven by the drive 109. Such a removable recording medium may be provided as what is generally called packaged software. Here, the removable recording medium 111 includes, for example, a floppy disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto-Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

[0177] It should be noted that, in addition to installation into the computer from the above-mentioned removable recording medium 111, the program can also be downloaded into the computer via a communication network or a broadcasting network and installed onto the internal hard disk 105. That is, the program can be transferred to the computer from a download site, for example, wirelessly via a satellite for digital satellite broadcasting, or by wire through a network such as a LAN (Local Area Network) and the Internet.

[0178] The computer includes a CPU (Central Processing Unit) 102 . The CPU 102 is connected to an input / output interface 110 via a bus 101 .

[0179] When a user inputs a command via the input / output interface 110 by operating the input unit 107, for example, the CPU 102 accordingly executes a program stored in the ROM (Read Only Memory) 103. Alternatively, the CPU 102 loads a program from the hard disk 105, wherein the program is stored in the RAM (Random Access Memory) 104, and executes the loaded program.

[0180] In this way, the CPU 102 performs processing according to the above flowchart or in a manner determined by the configuration in the above block diagram. Then, the CPU 102 causes the processing result to be output from the output unit 106, sent from the communication unit 108, and recorded to the hard disk 105 via the input / output interface 110.

[0181] It should be noted that the input section 107 includes a keyboard, a mouse, a microphone, etc. Furthermore, the output section 106 includes, for example, an LCD (Liquid Crystal Display) and a speaker.

[0182] In this description, the processing performed by the computer according to the program does not need to be performed in chronological order as illustrated in the flowchart. That is, the processing performed by the computer according to the program may include processing to be performed in parallel or individually (for example, parallel processing or object-oriented processing).

[0183] In addition, the program may be processed by a single computer (processor) or by multiple computers on a distributed basis. The program may also be transmitted to a remote computer or multiple computers to be executed thereby.

[0184] In addition, in this specification, the term "system" refers to a collection of multiple components (e.g., devices or modules (parts)). It does not matter whether all components are housed in the same housing. Therefore, a system may include multiple devices housed in separate housings and interconnected via a network, or a single device in a single housing that houses multiple modules.

[0185] In addition, any configuration described as a device (or processing unit) in the preceding paragraphs can be divided into multiple devices (or processing units). In contrast, the configuration described above as multiple devices (or processing units) can be unified into one device (or processing unit). In addition, it is clear that the configuration of each device (or processing unit) can be supplemented with one or more configurations other than those described above. In addition, part of the configuration of a device (or processing unit) can be included in the configuration of another device (or processing unit), as long as the configuration and operation remain substantially the same for the entire system.

[0186] Furthermore, the present technology may be implemented in a cloud computing setting, for example, where multiple networked devices collaboratively process a single function on a shared basis.

[0187] In addition, the above-mentioned program can be executed by any device. In such a case, the device only needs to be arranged to have necessary functions (eg, functional blocks) and acquire necessary information.

[0188] Furthermore, for example, each step discussed with reference to the flowcharts above can be performed by a single device or by multiple devices on a shared basis. Furthermore, where a single step includes multiple processes, these processes can be performed by a single device or by multiple devices on a shared basis. In other words, multiple steps included in a single step can be performed as a process of multiple steps. In contrast, a process described as consisting of multiple steps can be performed as a single step.

[0189] It should be noted that each program executed by the computer can be processed chronologically, i.e., in the order depicted in this specification, in parallel with other programs, or in another appropriately timed manner (e.g., when the program is called as needed). That is, the above steps can be performed in an order different from the order discussed above, as long as there are no conflicts between the steps. In addition, the processing of the steps of a given program can be performed in parallel or in combination with the processing of other programs.

[0190] It should be noted that the various techniques discussed in this specification can be implemented independently of one another, as long as there are no conflicts between them. Clearly, any number of these techniques can be implemented in combination. For example, some or all of the techniques discussed in connection with one embodiment can be implemented in combination with some or all of the techniques described in connection with another embodiment. Furthermore, some or all of any of the techniques discussed above can be implemented in combination with another technique not described above.

[0191] <Typical combination of configuration components>

[0192] It should be noted that the present technology can preferably be implemented in the following configurations: (1)

[0194] An information processing device, comprising:

[0195] a file generating unit configured to generate, based on 3D data representing a three-dimensional structure, identification information regarding a stream of shape information indicating a shape of the three-dimensional structure and identification information regarding a stream of attribute information indicating an attribute of the three-dimensional structure, for each of a stream of shape information indicating a shape of the three-dimensional structure and a stream of attribute information indicating an attribute of the three-dimensional structure;

[0196] The file generating section is configured to generate a file including the generated plurality of pieces of identification information. (2)

[0198] The information processing apparatus according to the above paragraph (1), wherein the identification information includes size information indicating a size of each of the streams and type information indicating a type of each of the streams. (3)

[0200] The information processing apparatus according to the above paragraph (1) or (2), wherein the file generating section generates the file in a sample structure that specifies a boundary between the shape information and the attribute information. (4)

[0202] An information processing device according to any one of the above paragraphs (1) to (3), wherein the file generation unit stores size information indicating the size of the subsample constituting the file and type information indicating the type of the subsample as the identification information in the header of the shape information and the header of the attribute information of the subsample. (5)

[0204] An information processing device according to any one of the above paragraphs (1) to (3), wherein the file generation unit stores the stream of the shape information and the stream of the attribute information into one track of ISOBMFF (ISO Base Media File Format), and transmits the boundary of the shape information and the attribute information with a signal by using information related to subsamples constituting the file. (6)

[0206] An information processing device according to the above paragraph (5), wherein the file generation unit stores size information indicating the size of the subsample and type information indicating the type of the subsample as the identification information in the header of the shape information and the header of the attribute information of the subsample stored in mdat of the ISOBMFF. (7)

[0208] An information processing device according to any one of the above paragraphs (1) to (3), wherein the file generation unit separates the stream of the shape information and the stream of the attribute information from each other to store each of the streams separately into one track of the ISOBMFF, and transmits the shape information and the attribute information with a signal by using information indicating the association relationship between the tracks. (8)

[0210] An information processing device according to any one of the above paragraphs (1) to (3), wherein the file generation unit separates the stream of the shape information and the stream of the attribute information from each other to store each of the streams separately into one track of ISOBMFF, and transmits the shape information and the attribute information with a signal by using MPD (Media Presentation Description) of DASH (Dynamic Adaptive Streaming over HTTP). (9)

[0212] An information processing device according to the above paragraph (8), wherein the file generation unit stores size information indicating the size of the subsample constituting the file and type information indicating the type of the subsample as the identification information in the header of the shape information and the header of the attribute information of the subsample stored in mdat of ISOBMFF. (10)

[0214] An information processing method, comprising:

[0215] causing an information processing apparatus to generate, for each of a stream of shape information indicating the shape of the three-dimensional structure and a stream of attribute information indicating the attributes of the three-dimensional structure, identification information related to the stream of shape information indicating the shape of the three-dimensional structure and identification information related to the stream of attribute information indicating the attributes of the three-dimensional structure, based on 3D data representing the three-dimensional structure; and

[0216] The information processing device is caused to generate a file including the generated plurality of pieces of identification information. (11)

[0218] A program for causing a computer of an information processing device to execute information processing, the information processing comprising:

[0219] generating, based on 3D data representing a three-dimensional structure, identification information regarding a stream of shape information indicating a shape of the three-dimensional structure and identification information regarding a stream of attribute information indicating an attribute of the three-dimensional structure, for each of a stream of shape information indicating a shape of the three-dimensional structure and a stream of attribute information indicating an attribute of the three-dimensional structure; and

[0220] A file including the generated plurality of pieces of identification information is generated. (12)

[0222] An information processing device, comprising:

[0223] an extraction unit configured to identifiably extract, from a file including identification information regarding a stream of shape information indicating a shape of a three-dimensional structure represented by 3D data and identification information regarding a stream of attribute information indicating attributes of the three-dimensional structure, the shape information and the attribute information according to a usage situation and based on the identification information, wherein the identification information is generated separately for each of the stream of shape information indicating the shape of the three-dimensional structure and the stream of attribute information indicating attributes of the three-dimensional structure based on the 3D data; and

[0224] A construction section is configured to construct the three-dimensional structure by using the shape information or the attribute information both extracted by the extraction section. (13)

[0226] An information processing method, comprising:

[0227] causing an information processing apparatus to identifiably extract, from a file including identification information regarding a stream of shape information indicating a shape of a three-dimensional structure represented by 3D data and identification information regarding a stream of attribute information indicating an attribute of the three-dimensional structure, the shape information and the attribute information according to a usage situation and based on the identification information, wherein the identification information is generated separately for each of the stream of shape information indicating the shape of the three-dimensional structure and the stream of attribute information indicating an attribute of the three-dimensional structure based on the 3D data; and

[0228] The information processing device is caused to construct the three-dimensional structure by using the shape information or the attribute information that have been extracted. (14)

[0230] A program for causing a computer of an information processing device to execute information processing, the information processing comprising causing the computer to:

[0231] extracting, from a file including identification information regarding a stream of shape information indicating a shape of a three-dimensional structure represented by 3D data and identification information regarding a stream of attribute information indicating an attribute of the three-dimensional structure, the shape information and the attribute information in an identifiable manner according to a usage situation and based on the identification information, wherein the identification information is generated separately for each of the stream of shape information indicating the shape of the three-dimensional structure and the stream of attribute information indicating an attribute of the three-dimensional structure based on the 3D data; and

[0232] The three-dimensional structure is constructed using the shape information or the attribute information that have been extracted.

[0233] It should be noted that the embodiments of the present disclosure are not limited to the embodiments discussed above, and various modifications or changes can be made within the scope of the present disclosure. In addition, the advantageous effects described in this specification are only examples, not limitations of the present disclosure, and the present disclosure may also provide other advantages.

[0234] Reference Signs List

[0235] 11 and 12 Information processing devices

[0236] 21 Separation Department

[0237] 22 Coding Department

[0238] 23 PC flow generation unit

[0239] 24 Document Generation Department

[0240] 31 Extraction Department

[0241] 32 Decoding Unit

[0242] 33 Construction Department

[0243] 34 Display processing unit

[0244] 51 Data generation device

[0245] 52 Data reproduction device

[0246] 61 Control Department

[0247] 62 Document Generation Department

[0248] 71 Data Input Unit

[0249] 72 Data Coding and Generation Department

[0250] 73 MPD file generation unit

[0251] 74 Records Department

[0252] 75 Upload Department

[0253] 76 Pretreatment Department

[0254] 77 Coding Department

[0255] 78 Segment file generation unit

[0256] 81 Control Department

[0257] 82 Reproduction Processing Unit

[0258] 91 MPD file acquisition unit

[0259] 92 MPD file processing unit

[0260] 93. Document Acquisition Section

[0261] 94 Display control unit

[0262] 95 Data Analysis and Decoding Department

[0263] 96 Display

[0264] 97 Section File Processing Department

[0265] 98 Decoding Department

[0266] 99 Display information generation unit

Claims

1. An information processing device, comprising: a file generating unit configured to generate, based on 3D data representing a three-dimensional structure, identification information regarding a stream of shape information indicating a shape of the three-dimensional structure and identification information regarding a stream of attribute information indicating an attribute of the three-dimensional structure, for each of a stream of shape information indicating a shape of the three-dimensional structure and a stream of attribute information indicating an attribute of the three-dimensional structure; The file generating section is configured to generate a file including the generated plurality of pieces of identification information, wherein the file generation unit stores the stream of the shape information and the stream of the attribute information in one track of the ISO base media file format, and signals a boundary between the shape information and the attribute information by using information related to subsamples constituting the file, or The file generation unit separates the stream of the shape information and the stream of the attribute information from each other to store each of the streams separately in one track of the ISO base media file format, and transmits the shape information and the attribute information by signaling using a media presentation description for dynamic adaptive streaming over HTTP, or by signaling using information indicating an association relationship between tracks.

2. The information processing device according to claim 1, wherein The identification information includes size information indicating the size of each of the streams and type information indicating the type of each of the streams.

3. The information processing device according to claim 1, wherein The file generating unit generates the file in a sample structure that specifies a boundary between the shape information and the attribute information.

4. The information processing device according to claim 1, wherein: The file generation section stores, as the identification information, size information indicating the size of a subsample constituting the file and type information indicating the type of the subsample in a header of the shape information and a header of the attribute information of the subsample. The information processing device according to claim 1 , wherein: The file generation unit stores size information indicating the size of the subsample and type information indicating the type of the subsample as the identification information in a header of the shape information and a header of the attribute information of the subsample stored in mdat of the ISO base media file format.

6. An information processing method, comprising: causing an information processing apparatus to generate, for each of a stream of shape information indicating a shape of the three-dimensional structure and a stream of attribute information indicating attributes of the three-dimensional structure, identification information related to the stream of shape information indicating the shape of the three-dimensional structure and identification information related to the stream of attribute information indicating attributes of the three-dimensional structure, based on 3D data representing the three-dimensional structure; as well as causing the information processing device to generate a file including the generated plurality of pieces of identification information, wherein the stream of shape information and the stream of attribute information are stored into one track of the ISO base media file format, and a boundary of the shape information and the attribute information is signaled by using information about subsamples constituting the file, or The shape information stream and the attribute information stream are separated from each other so that each of the streams is stored separately in one track of the ISO base media file format, and the shape information and the attribute information are signaled by using a media presentation description for dynamic adaptive streaming over HTTP, or by using information indicating an association relationship between tracks.

7. An information processing device comprising: an extraction unit configured to identifiably extract, from a file including identification information regarding a stream of shape information indicating a shape of a three-dimensional structure represented by 3D data and identification information regarding a stream of attribute information indicating attributes of the three-dimensional structure, the shape information and the attribute information according to a usage situation and based on the identification information, wherein the identification information is generated separately for each of the stream of shape information indicating the shape of the three-dimensional structure and the stream of attribute information indicating attributes of the three-dimensional structure based on the 3D data; and a construction section configured to construct the three-dimensional structure by using the shape information or the attribute information both extracted by the extraction section, wherein the stream of shape information and the stream of attribute information are stored into one track of the ISO base media file format, and a boundary of the shape information and the attribute information is signaled by using information about subsamples constituting the file, or The shape information stream and the attribute information stream are separated from each other so that each of the streams is stored separately in one track of the ISO base media file format, and the shape information and the attribute information are signaled by using a media presentation description for dynamic adaptive streaming over HTTP, or by using information indicating an association relationship between tracks.

8. An information processing method, comprising: causing an information processing apparatus to identifiably extract, from a file including identification information regarding a stream of shape information indicating a shape of a three-dimensional structure represented by 3D data and identification information regarding a stream of attribute information indicating an attribute of the three-dimensional structure, the shape information and the attribute information according to a usage situation and based on the identification information, wherein the identification information is generated separately for each of the stream of shape information indicating the shape of the three-dimensional structure and the stream of attribute information indicating an attribute of the three-dimensional structure based on the 3D data; and causing the information processing device to construct the three-dimensional structure by using the shape information or the attribute information that have been extracted, wherein the stream of shape information and the stream of attribute information are stored into one track of the ISO base media file format, and a boundary of the shape information and the attribute information is signaled by using information about subsamples constituting the file, or The shape information stream and the attribute information stream are separated from each other so that each of the streams is stored separately in one track of the ISO base media file format, and the shape information and the attribute information are signaled by using a media presentation description for dynamic adaptive streaming over HTTP, or by using information indicating an association relationship between tracks.

9. A computer-readable medium having recorded thereon a program, the program, when executed by a computer, causing the computer to perform a method, the method comprising: generating, based on 3D data representing a three-dimensional structure, identification information related to the stream of shape information indicating the shape of the three-dimensional structure and identification information related to the stream of attribute information indicating the attributes of the three-dimensional structure, for each of a stream of shape information indicating the shape of the three-dimensional structure and a stream of attribute information indicating the attributes of the three-dimensional structure; as well as generating a file including the generated plurality of pieces of identification information, wherein the stream of shape information and the stream of attribute information are stored into one track of the ISO base media file format, and a boundary of the shape information and the attribute information is signaled by using information about subsamples constituting the file, or The shape information stream and the attribute information stream are separated from each other so that each of the streams is stored separately in one track of the ISO base media file format, and the shape information and the attribute information are signaled by using a media presentation description for dynamic adaptive streaming over HTTP, or by using information indicating an association relationship between tracks.

10. A computer-readable medium having recorded thereon a program, the program, when executed by a computer, causing the computer to perform a method, the method comprising causing the computer to: From a file including identification information related to a stream of shape information indicating a shape of a three-dimensional structure represented by 3D data and identification information related to a stream of attribute information indicating an attribute of the three-dimensional structure, the shape information and the attribute information are identifiably extracted according to a usage situation and based on the identification information, wherein the identification information is generated for each of a stream of shape information indicating a shape of the three-dimensional structure and a stream of attribute information indicating an attribute of the three-dimensional structure, respectively, based on the 3D data; as well as constructing the three-dimensional structure using the shape information or the attribute information that have been extracted, wherein the stream of shape information and the stream of attribute information are stored into one track of the ISO base media file format, and a boundary of the shape information and the attribute information is signaled by using information about subsamples constituting the file, or The shape information stream and the attribute information stream are separated from each other so that each of the streams is stored separately in one track of the ISO base media file format, and the shape information and the attribute information are signaled by using a media presentation description for dynamic adaptive streaming over HTTP, or by using information indicating an association relationship between tracks.

Images