Information processing device, information processing method, reproduction processing device, and reproduction processing method

Abstract

An information processing device, an information processing method, a reproduction processing device, and a reproduction processing method are provided, which reduce the amount of information and provide a high-quality viewing experience to a viewer. A preprocessing unit is used to separately generate subsample information for decoding each subsample in a content stream, wherein the content stream includes a plurality of subsamples as component streams. A file generation unit is used to generate a file including the subsample information generated by the preprocessing unit.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method, a reproduction processing device, and a reproduction processing method. Background Art

[0002] In recent years, Moving Picture Experts Group Dynamic Adaptive Streaming over Hypertext Transfer Protocol (MPEG-DASH) and the like have begun to spread as basic technologies for streaming services on the Internet. In addition, in streaming using MPEG-DASH, for example, a technology for storing video data and audio data in an International Organization for Standardization Base Media File Format for Distribution (ISOBMFF) file is used. Here, ISOBMFF is a standard for the MPEG-4 file format.

[0003] In addition, MPEG-IPart 5 video-based point cloud compression (ISO / IEC 23090-5) specifies a compression method for point clouds, which are a collection of points in three-dimensional space that have both position information and attribute information (especially color information). One of the compression methods is a method called video-based point cloud coding (V-PCC) as described below. In V-PCC, the point cloud is divided into multiple regions, and each region is projected onto a plane to generate attribute images including color information, geometric images including depth information, occupancy images, and information metadata for reconstructing point clouds using patches. The metadata includes a patch sequence and a sequence parameter set. The three images are then encoded by the video codec, and a total of four component streams are generated. In addition, in the case of video content with multiple audio information and subtitle information, multiple component streams are also stored in individual tracks in V-PCC. In the following, video content decoded using V-PCC technology is referred to as "V-PCC content."

[0004] However, since multiple component streams in V-PCC content include common elements such as time information and codec information, file management information is redundantly stored in the case where the component streams are stored in each track. Therefore, in a method of storing the corresponding component streams in different tracks, it is conceivable that management information such as moov and voof of ISOBMFF is redundant.

[0005] In order to distinguish common information from information in a component stream specifically used for management and reduce the amount of information, a method of using a Muxed track has been proposed in which a plurality of component streams are stored in one track.

[0006] Reference List

[0007] Non-patent literature

[0008] Non-patent document 1: ISO / IEC 14496-12:2015 Information technology. Coding of audio-visual object. Part 12: ISO base media file format, December 2015. Summary of the invention

[0009] Technical issues

[0010] However, in the proposed method using multiplexed tracks, although the amount of information can be reduced, it is difficult to associate codec information switched according to each component stream. Therefore, in the proposed method using multiplexed tracks, it is difficult to dynamically change the encoding parameters of the component streams.

[0011] Therefore, the present disclosure provides an information processing device, an information processing method, a reproduction processing device, and a reproduction processing method that reduce the amount of information and provide viewers with a high-quality viewing experience.

[0012] Solution to the problem

[0013] According to the present disclosure, the pre-processing unit separately generates SubSample information for decoding each SubSample in a content stream including a plurality of SubSamples as component streams. The file generating unit generates a file including the SubSample1 information generated by the pre-processing unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a system configuration diagram of an example of a distribution system.

[0015] Figure 2 It is a block diagram of a file generating device.

[0016] Figure 3 is a diagram showing an example of the syntax of SubSampleEntryBox.

[0017] Figure 4 is a diagram showing an example of the syntax of SubSampleToChunkBox.

[0018] Figure 5 is a diagram showing each piece of information stored in an ISOBMFF file according to the first embodiment.

[0019] Figure 6 is a diagram showing a storage state of SubSampleEntryBox.

[0020] Figure 7is a diagram showing a storage state of SubSampleEntryBox and SubSampleToChunkBox depending on the presence or absence of a fragment.

[0021] Figure 8 is a block diagram of a client device.

[0022] Fig. 9 This is a flowchart of a file generation process executed by the file generation device according to the first embodiment.

[0023] Fig.10 This is a flowchart of the reproduction process executed by the client device according to the first embodiment.

[0024] Fig.11 is a diagram showing an example of the syntax of SubSampleToHandlerBox and SubSampleEntryBox according to the modification example of the first embodiment.

[0025] Fig.12 Is a diagram used to describe the storage state of a parameter set.

[0026] Fig.13 is a diagram showing the storage status of SubSampleEntryBox and tr_flags depending on the presence or absence of a slice.

[0027] Fig.14 This is a flowchart of a file generation process executed by the file generation device according to the second embodiment.

[0028] Fig.15 This is a flowchart of a reproduction process executed by the client device according to the second embodiment.

[0029] Fig.16 is a diagram showing the format of the Matroska media container.

[0030] Fig.17 It is a hardware configuration diagram of a computer. DETAILED DESCRIPTION

[0031] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that in each of the following embodiments, the same reference numerals are assigned to the same components, and repeated descriptions will be omitted. In addition, the scope disclosed in the present technology is not limited to the contents of the embodiments, and includes the contents described in the following non-patent documents known at the time of filing.

[0032] Non-patent document 1: (the above-mentioned non-patent document 1)

[0033] Non-patent document 2: N18413, WD of ISO / IEC 23090-10 Carriage of PC data, 126th MPEG meeting, Geneva, Switzerland, March 2019.

[0034] Non-patent document 3: N18180, Technologies under consideration on carriage of PC data, 126th MPEG Conference, Geneva, Switzerland, March 2019.

[0035] Non-patent document 4: m47257, PCC file format Consideration, 126th MPEG meeting, Geneva, Switzerland, March 2019.

[0036] The contents described in the above-mentioned non-patent documents are also incorporated into the present embodiment by reference. That is, the contents described in the above-mentioned non-patent documents are also used as the basis for determining the support requirements. For example, even in the case where the terms used in the file structure described in non-patent document 1 and the V-PCC file structure described in non-patent document 2 are not directly defined in the detailed description of the present invention, they fall within the scope of the present disclosure and meet the support requirements of the claims. In addition, for example, even if technical terms such as parsing, grammar, and semantics are not directly defined in the detailed description of the present invention, they are within the scope of the present disclosure and meet the support requirements of the claims.

[0037] Furthermore, the present disclosure will be described according to items in the following order.

[0038] 1. First Implementation

[0039] 1.1 Modification of the First Embodiment

[0040] 2. Second Implementation

[0041] 2.1 Modification of the Second Embodiment

[0042] 3. Third Implementation

[0043] [1. First embodiment]

[0044] (Configuration of Distribution System According to First Embodiment)

[0045] Figure 11 is a system configuration diagram of an example of a distribution system. The distribution system 100 includes: a file generation device 1 as an information processing device, a client device 2 as a reproduction processing device, and a Web server 3. The file generation device 1, the client device 2, and the Web server 3 are connected to a network 4. Then, the file generation device 1, the client device 2, and the Web server 3 can communicate with each other via the network 4. Here, although Figure 1 Although one file generation device 1 and one client device 2 are shown in FIG. 1 , the distribution system 100 may include a plurality of file generation devices 1 and a plurality of client devices 2 .

[0046] The file generating device 1 generates video content as data for providing a video. The content stream of the video content generated by the file generating device 1 according to the present embodiment includes a plurality of component streams. For example, the video content generated by the file generating device 1 according to the present embodiment includes V-PCC content. The file generating device 1 uploads the generated video content to the Web server 3. Here, the present embodiment describes a configuration in which the Web server 3 provides the video content to the client device 2, but the distribution system 100 may adopt another configuration. For example, another configuration may be configured such that the file generating device 1 includes the function of the Web server 3, stores the generated video content in the file generating device 1 itself, and provides the generated video content to the client device 2.

[0047] The web server 3 stores the video content uploaded from the file generating device 1. Then, the web server 3 provides the designated video content according to the request from the client device 2.

[0048] The client device 2 sends a video content transmission request to the Web server 3. The client device 2 then acquires the video content specified by the transmission request from the Web server 3. The client device 2 then decodes the video content to generate a display video, and displays the display video on a display device such as a monitor.

[0049] (Configuration of File Generating Device According to First Embodiment)

[0050] Next, the details of the document generating device 1 will be described. Figure 2 is a block diagram of a file generating device. Figure 2 As shown, a file generation device 1 as an information processing device includes a file generation processing unit 10, a control unit 11, and a transmission unit 12. The control unit 11 performs processing related to the control of the file generation processing unit 10. For example, the control unit 11 controls the operation timing of each unit of the file generation processing unit 10 and the like in a centralized manner. The file generation processing unit 10 includes a data acquisition unit 101, a preprocessing unit 102, an encoding unit 103, and a file generation unit 104.

[0051] The data acquisition unit 101 acquires raw data of video content for displaying a video. The raw data of the video content includes control information and image data of each image included in an image sequence as a series of images. For example, the control information includes information about time information of each piece of image data. The data acquisition unit 101 outputs the control information and the image data included in the acquired image sequence of the video content to the preprocessing unit 102.

[0052] The preprocessing unit 102 receives an input of control information and image data included in an image sequence from the data acquisition unit 101. Then, the preprocessing unit 102 determines a bitstream configuration of the image sequence based on the acquired data. In this case, the preprocessing unit 102 determines to store data of a plurality of component streams in one track.

[0053] Specifically, each of the multiple component streams is called a "SubSample", and the data of the bit stream of one Sample including multiple SubSamples is stored in one track. Hereinafter, the track storing the data of multiple component streams is called a "Muxed track". For example, in the case of decoding using V-PCC, the preprocessing unit 102 determines that the data of one Sample is stored in a Muxed track, where the component stream of each of the attribute image, geometry image, occupancy image, and metadata is a SubSample. Here, the case of performing segmentation of dividing one content into multiple pieces will be described.

[0054] Next, the preprocessing unit 102 generates a SampleEntry including management information, such as the encoding parameters of the Sample in the Muxed track. For example, the encoding parameters of the Sample in the Muxed track include information indicating that the Sample is a Muxed track, information indicating that the codec of the Sample is V-PCC, and the like.

[0055] In addition, the pre-processing unit 102 generates SubSample related information by using the control information. Here, the SubSample related information is information related to the SubSample corresponding to each component included in the Muxed track. For example, the SubSample related information includes SubSampleEntryBox.

[0056] Figure 3is a diagram showing an example of the syntax of the SubSampleEntryBox. component_id is an identifier for uniquely identifying each component stream stored in the Muxed track. Here, in the present embodiment, component_id is determined to correspond to the logical position order of the SubSample in the Sample. Therefore, since the component_id is associated with the logical position of the SubSample, the SubSampleEntry corresponding to the component_id can be specified. However, the present invention is not limited to this, and the preprocessing unit 102 can associate the component_id with the SubSampleEntry by setting the same identifier in the codec_specific_parameter of the SubSampleInfomationBox and the muxted_stream_specific_type of the SubSumpleEntryBox as the access information of the SubSample. The component_id corresponds to the tack_id of the moov in the Sample.

[0057] Figure 3 The handler_type in the syntax of represents the handler information as the media attribute information of each component stream. The handler information corresponds to an example of the "second related information". For example, in the case of video, the handler_type is set to vide, and in the case of audio, the handler_type is set to soun. The category of SubSampleEntry is determined by this handler_type.

[0058] Figure 3 The muxed_stream_specific_type in the syntax is type information for identifying the type of each component stream in the Muxed track. For example, in the V-PCC Sample, since the attribute image, geometry image, and occupancy image are the same video media, the same attribute "vide" is set to hanlder_type. Therefore, each component stream can be identified by setting attr in the attribute image, geom in the geometry image, occu in the occupancy image, etc. to muxed_stream_specific_type. complement_id and muxed_stream_specific_type correspond to the example of "first related information".

[0059] Figure 3The SubSampleEntry() in the syntax has a structure similar to that of Track's SampleEntry(). When handler_type is vide, VisualSampleEntry is set to SubSampleEntry, and when handler_type is soun, AudiolSampleEntry is set to SubSampleEntry. That is, in SubSampleEntry, the category of SubSampleEntry is defined for each handler_type. A SubSampleEntryBox is defined for each component stream. For example, in the case of decoding using V-PCC, a SubSampleEntryBox is stored for each component stream of attribute images, geometry images, occupancy images, and metadata. Note that the SubSampleEntryBox can be defined to have a for loop structure so that the information of each component stream can be set in one box structure.

[0060] also, Figure 2 The pre-processing unit 102 in the Muxed track generates SubSample corresponding information. The SubSample corresponding information is information used to associate each SubSample included in the Sample of the Muxed track with each SubSample related information.

[0061] For example, the preprocessing unit 102 sets component_id, which is identification information of the component stream of the SubSample, and an arbitrary number for identifying the SubSampleEntry assigned to the SubSample as SubSample corresponding information. To this end, SubSampleToChunkBox corresponding to each SubSample is set as SubSample corresponding information.

[0062] Figure 4 is a diagram showing an example of the syntax of SubSampleToChunkBox. Figure 4The first_sub_sample_number in the syntax of indicates the number of Samples to which the SubSampleEntry specified by the following sub_sample_entry_inedx is applied, and more specifically, indicates the number of the first SubSample among the multiple SubSamples that constitute the Sample and are arranged in time series. In addition, sub_sample_entry_inedx indicates the identification information of the SubSampleEntry applied from the number of the first SubSample indicated by the SubSampleEntry to the number of SubSamples before the number indicated by first_sub_sample_number in the next SubSampleToChunkBox. This information corresponds to an example for associating the SubSample with the information of the SubSampleEntry.

[0063] In this way, the pre-processing unit 102 can set the SubSampleEntry corresponding to each SubSample by generating SubSample corresponding information. This enables the SubSampleEntry to be dynamically set for each SubSample.

[0064] Here, it may be arranged that a SubSampleToChunkBox may be generated for each component stream, but the pre-processing unit 102 may define the SubSampleToChunkBox to have a for loop structure so that each component stream may be arranged in a box structure.

[0065] Furthermore, the preprocessing unit 102 generates metadata for each image and metadata indicating access information of a bit stream. In addition, the preprocessing unit 102 also generates control information such as information on what codec is used for compression as metadata.

[0066] Then, the preprocessing unit 102 outputs each piece of image data included in the image sequence and metadata such as codec information to the encoding unit 103. In addition, the preprocessing unit 102 outputs metadata including SubSample related information, SubSample corresponding information, information of each image, access information of the bitstream, etc. to the file generation unit 104.

[0067] The encoding unit 103 receives an input of image data of each image included in the image sequence. Then, the encoding unit 103 encodes the image data of each image in the image sequence to generate a coded stream. In the present embodiment, the encoding unit 103 generates a plurality of component streams as coded streams. For example, in the case where decoding is performed by V-PCC, the encoding unit 103 generates four component streams of an attribute image, a geometric image, an occupancy image, and metadata as coded streams. Each component stream includes a bit stream for each corresponding image or metadata. The data of the bit stream is stored in a track of each frame or each associated frame. The encoding unit 103 according to the present embodiment stores the data of a plurality of bit streams in one track to generate a Muxed track. The encoding unit 103 outputs the generated coded stream to the file generation unit 104.

[0068] The file generating unit 104 receives an input of an encoded stream including a plurality of component streams from the encoding unit 103. In addition, the file generating unit 104 receives an input of metadata including SubSample related information and SubSample corresponding information from the pre-processing unit 102.

[0069] The file generation unit 104 stores the encoding parameters of the SubSample included in the Muxed track in the SampleEntryBox. In addition, the file generation unit 104 redefines the SubSampleEntryBox storing SubSample related information, and stores the SubSampleEntryBox in the SampleTableBox in the Moov of the ISOBMFF file.

[0070] In addition, the file generation unit 104 redefines the SubSampleToChunkBox storing the SubSample corresponding information, and stores the SubSampleToChunkBox in the moof which is the file header area of ​​the ISOBMFF when performing the segmentation. Therefore, the file generation unit 104 associates the SubSample stored in each mdat in the segmented state with each SubSampleEntry stored in the moov.

[0071] Here, the present embodiment has described the case where slicing is performed, but the configuration of the present embodiment can also be applied to the case where slicing is not performed. In this case, the file generation unit 104 sets SubSmapleToChunkBox in SampleTableBox.

[0072] The file generation unit 104 stores the SubSampleEntryBox and SubSampleToChunkBox in the ISOBMFF file for each fragment together with the image data and the metadata of each component stream, thereby creating a file. Specifically, the file generation unit 104 generates an ISOBMFF file, which includes (moov) as management information, (mdat) as sliced ​​video information, and management information (moof) for each video information. Mdat is a data area in the ISOBMFF file. In addition, moov and moof are header areas in the ISOBMFF. As described above, the file generation unit 104 then stores the SubSampleEntryBox in the Sampletable of moov, and also stores the SubSampleToChunkBox in the TrackFragment of each moof. In this way, the file generation unit 104 generates a fragment file of the video content.

[0073] Here, we will refer to Figure 5 Each piece of information stored by the file generation unit 104 is collectively described. Figure 5 is a diagram showing each piece of information stored in an ISOBMFF file according to the first embodiment. Figure 5 One set of moof and mdat is shown, but in reality, there are multiple sets of moof and mdat having similar structures.

[0074] A plurality of samples including sample 131 and sample 132 are stored in mdat. Each sample including sample 131 and sample 132 is a Muxed track. Then, sample 131 includes a plurality of SubSamples including SubSample 141 to SubSample 143 for different component streams. Similarly, sample 132 also includes a plurality of SubSamples including SubSample 144 to SubSample 146 for different component streams.

[0075] SampletableBox 110 is set in moov. Then, SampleEntryBox 111 is set in SampleTableBox 110. Then, SampleEntry 112 and SampleEntry 113 are stored in SampleEntryBox 111. Figure 5SampleEntry 112 and SampleEntry 113 are shown, but in reality, there are SampleEntries corresponding in number to the number of Samples including Sample 131 to Sample 132. SampleEntries including SampleEntry 112 and SampleEntry 113 store V-PCC information of each Sample including Sample 131 to Sample 132, information indicating a Muxed track, and the like.

[0076] In addition, SampleTableBox 110 stores a plurality of sub-SubSampleEntryBoxes including SubSampleEntryBox 114. A plurality of SubSampleEntryBoxes including SubSampleEntryBox 114 are provided for each type of component stream included in the Muxed track. Then, SubSampleEntryBox 114 stores a plurality of SubSampleEntries including SubSampleEntry 115 and SubSampleEntry 116. Figure 5 SubSampleEntry 115 and SubSampleEntry 116 are shown, but in reality, there are as many SubSampleEntries as the number of patterns of variables corresponding to all Samples including Sample 131 to Sample 132. A category is defined for each handler_type in each of SubSampleEntries including SubSampleEntry 115 and SubSampleEntry 116.

[0077] The TrackFragment 120 is provided in the moof. Then, the TrackRunBox 121 is provided in the TrackFragment 120. The TrackRunBox 121 has information for associating each Sample including the Sample 131 to the Sample 132 with the SampleEntry 112 to the SampleEntry 113.

[0078] In addition, a plurality of SubSampleToChunkBoxes including SubSampleToChunkBox 122 are provided in TrackFragment 120. The plurality of SubSampleToChunkBoxes including SubSampleToChunkBox 122 have information for associating the respective SubSamples including SubSample 141 to SubSample 146 with SubSampleEntry 115 and SubSampleEntry 116.

[0079] For example, it is assumed that sub_sample_number as the number of SubSample 141 is 1, and sub_sample_number as the number of SubSample 144 is 16. Furthermore, it is assumed that: in SubSampleToChunk 122, sub_sample_entry_index is 1 in the case where first_sub_sample_numbe is 1, and sub_sample_entry_index is 2 in the case where first_sub_sample_numbe is 16. Furthermore, it is assumed that sub_sample_entry_index of SubSumpleEntry 115 is 1, and sub_sample_entry_index of sub SubSumpleEntry 116 is 2.

[0080] A case where the client device 2 continuously decodes Sample 131 and Sample 132 under this condition will be described. The client device 2 decodes SubSamples including SubSample 141 to SubSample 143 included in Sample 131. At this time, for example, when decoding SubSample 141, the client device 2 specifies SubSampleToChunkBox 122 having information of SubSample 141 based on first_sub_sample_number. Then, the client device 2 specifies SubSampleEntryBox 114 to be used according to component_id stored in SubSampleToChunkBox 122, and further specifies SubSunpleEntry 115 managing information of SubSample 141 in SubSampleEntryBox 114 using sub_sample_entry_index. Then, the client device 2 decodes SubSample 141 using information such as handler_type stored in SubSunpleEntry 115.

[0081] Next, the client device 2 continues to decode Sample 132, and similarly specifies SubSumpleEntry 116 that manages information of SubSample 146. Then, the client device 2 decodes SubSample 144 using the information of SubSunpleEntry 116. In this way, the client device 2 can decode a Muxed track using SubSampleEntryBox and SubSampleToChunkBox.

[0082] Figure 6 is a diagram showing a storage state of SubSampleEntryBox. Figure 6 FIG. 2 shows the storage state of SubSmapleEntryBox in another method. Figure 6 As shown, the file generation unit 104 sets SubSmapleEntryBox in BOX 151 of the moov indicated by BOX 150 , and sets SubSampleToChunkBox in BOX 152 .

[0083] in addition, Figure 7is a diagram showing the storage state of SubSampleEntryBox and SubSampleToChunkBox depending on the presence or absence of fragmentation. In the case where movie fragmentation is not performed, Figure 7 As shown in the file 160 of , there is one moov and one mdat for one video content. In this case, the file generation unit 104 stores SubSampleEntryBox and SubSampleToChunkBox in moov, as indicated by BOX 161.

[0084] On the other hand, in the case of performing movie segmentation, such as Figure 7 As shown in the file 170 of , for one video content, there is one moov and multiple sets of one moof and one mdat. In this case, the file generation unit 104 sets the SubSampleEntryBox indicated by BOX171 in the moov. In addition, the file generation unit 104 sets the SubSampleToChunkBoxes indicated by BOX 172 to BOX174 in the corresponding moof.

[0085] Back to Figure 2 , the description will be continued. Thereafter, the file generation unit 104 outputs the segment file of the video content including the sample of the Muxed track to the transmission unit 12.

[0086] The transmission unit 12 receives an input of a segment file of video data from the file generation unit 104. Then, the transmission unit 12 uploads the acquired segment file of video data to the Web server 3.

[0087] (Configuration of Client Device According to First Embodiment)

[0088] Figure 8 is a block diagram of a client device. Figure 8 As shown, the client device 2 includes a reproduction processing unit 20 and a control unit 21. The control unit 21 controls the operation of each unit of the reproduction processing unit 20. For example, the control unit 21 controls the operation timing of each unit of the reproduction processing unit 20 in a centralized manner. The reproduction processing unit 20 includes a file acquisition unit 201, a file processing unit 202, a decoding processing unit 203, a display control unit 204, a display information generation unit 205, and a display unit 206.

[0089] The file acquisition unit 201 receives an input of a video content acquisition request from the file processing unit 202. Then, the file acquisition unit 201 acquires a segment file of the designated video content from the Web server 3. Thereafter, the file acquisition unit 201 outputs the acquired segment file to the file processing unit 202.

[0090] The file processing unit 202 receives input of an operation command for specifying video content to be reproduced or for random access from the display control unit 204. Then, the file processing unit 202 selects video content in response to the operation command, and outputs a request to acquire the selected video content to the file acquisition unit 201.

[0091] Thereafter, the file processing unit 202 receives an input of a segment file of the video content for which a transmission request has been made from the file acquisition unit 201. Then, the file processing unit 202 extracts data of the encoded stream from the acquired segment file, and outputs the data to the decoding processing unit 203. In addition, the file processing unit 202 obtains metadata from the acquired segment file, and outputs the metadata to the display information generation unit 205.

[0092] The decoding processing unit 203 receives an input of data of the encoded stream from the file processing unit 202. Then, the decoding processing unit 203 specifies the SubSampleEntry corresponding to the SubSample of each component stream by using the information stored in the SubSampleEntryBox and the SubSampleToChunkBox, and decodes the SubSample of each component stream by using the specified SubSampleEntry. Thus, the decoding processing unit 203 decodes each component stream included in the encoded stream. Thereafter, the decoding processing unit 203 outputs the decoded data of each component stream to the display information generating unit 205.

[0093] The display control unit 204 receives an operation command from the operator through an input device (not shown). Then, in the case of an operation command to switch video content, the display control unit 204 outputs the operation command to the file processing unit 202. In addition, in the case of an operation command to move the viewpoint, the display control unit 204 outputs the acquired operation command to the display information generation unit 205.

[0094] The display information generating unit 205 receives input of data of a decoded bit stream for each component stream from the decoding processing unit 203. Also, the display information generating unit 205 receives input of metadata from the file processing unit 202. Furthermore, the display information generating unit 205 receives input of an operation command from the display control unit 204.

[0095] Then, the display information generating unit 205 performs presentation in response to the operation command using the acquired bit stream and metadata, and generates a display image. Thereafter, the display information generating unit 205 supplies the generated display image to the display unit 206.

[0096] The display unit 206 includes a display device such as a monitor. The display unit 206 receives an input of a display image generated by the display information generation unit 205. Then, the display unit 206 causes the display device to display the acquired display image.

[0097] (File Generation Process According to First Embodiment)

[0098] Next, we will refer to Fig. 9 The flow of the file generation process executed by the file generation device 1 according to the first embodiment is described in detail. Fig. 9 This is a flowchart of a file generation process executed by the file generation device according to the first embodiment.

[0099] The data acquisition unit 101 acquires the original data of the video content from the Web server 3. Then, the preprocessing unit 102 outputs the image data to be encoded and the control information included in the acquired original data acquired by the data acquisition unit 101 to the encoding unit 103. The preprocessing unit 102 sets the encoding parameters of the Muxed track in the SampleEntry by using the control information included in the acquired original data (step S101).

[0100] Next, the preprocessing unit 102 sets the encoding parameters of each component in SubSampleEntry (step S102). That is, the preprocessing unit 102 generates SubSample related information in which SubSampleEntry of each component is set together with component_id, muxed_stream_specific_type, and handler_type_type.

[0101] In addition, the preprocessing unit 102 generates SubSample corresponding information according to the codec information of the SubSample and the arrangement of the Sample and the SubSample (step S103). Then, the preprocessing unit 102 outputs the image data and metadata to the encoding unit 103. In addition, the preprocessing unit 102 outputs the metadata including the SubSample related information and the SubSample corresponding information to the file generation unit 104.

[0102] The encoding unit 103 receives input of image data and metadata from the data acquisition unit 101. Then, the encoding unit 103 encodes the image data and metadata to generate encoded data of each component stream (step S104). Thereafter, the encoding unit 103 outputs the encoded data of each component stream to the file generation unit 104.

[0103] The file generation unit 104 receives an input of the encoded data of each component stream from the encoding unit 103. In addition, the file generation unit 104 receives an input of metadata including SubSample related information and SubSample corresponding information from the pre-processing unit 102. Then, the file generation unit 104 acquires codec information of each SubSample in each component stream (step S105).

[0104] Next, the file generation unit 104 creates a Muxed track by collecting the SubSamples of each component as Samples, sets the Samples in mdat, and sets the Sample information (step S106). That is, the file generation unit 104 stores the SampleEntry of each Muxed track in the SampleEntryBox set in the SampleTable of moov.

[0105] Next, the file generation unit 104 stores SubSampleToCunkBox in moof (step S107 ).

[0106] Then, the file generation unit 104 generates an ISOBMFF file having a Muxed track including a component stream for each component (step S108 ) Thereafter, the transmission unit 108 uploads the segment file of the video content generated by the file generation unit 104 to the Web server 3 .

[0107] (Procedure of Reproduction Processing According to First Embodiment)

[0108] Next, we will refer to Fig.10 The flow of the reproduction process performed by the client device 2 according to the first embodiment is described. Fig.10 This is a flowchart of the reproduction process executed by the client device according to the first embodiment.

[0109] The file acquisition unit 201 acquires the segment file of the video content to be reproduced from the Web server 3. The file processing unit 202 parses the segment file of the video content acquired by the file acquisition unit 201. Then, the file processing unit 202 acquires Sample and SampleEntry from the ISOBMFF file. Thereafter, the file processing unit 202 outputs the acquired Sample and SampleEntry to the decoding processing unit 203. In addition, the file processing unit 202 outputs metadata to the display information generation unit 205. The decoding processing unit 203 acquires Sample and SampleEntry and sets decoding (step S201).

[0110] Next, the decoding processing unit 203 specifies the corresponding SubSampleEntry from SubSampleToCunkBox for each SubSample multiplexed in the acquired Sample, sets decoding of each component, and decodes the data of each component stream (step S202).

[0111] The display information generating unit 205 acquires the decoded data of each component stream from the decoding processing unit 203. In addition, the display information generating unit 205 acquires the operation command from the display control unit 204. Then, the display information generating unit 205 performs a display process of performing visualization using the data of each component stream to generate a display image in response to the operation command, and causing the display unit 207 to display the display image (step S203).

[0112] Thereafter, the file processing unit 202, the decoding processing unit 203, the display control unit 204, and the display information generation unit 205 determine whether to continue the reproduction process (step S204). For example, in the case where a stop command or the like is input from the user, the file processing unit 202, the decoding processing unit 203, the display control unit 204, and the display information generation unit 205 determine not to continue the reproduction process. In the case where the reproduction process is continued (step S204: Yes), the video reproduction process returns to step S201. On the other hand, in the case where the decoding of all the image data of the video content is completed (step S204: No), the file processing unit 202, the decoding processing unit 20, the display control unit 204, and the display information generation unit 205 end the video reproduction process.

[0113] As described above, the file generation device according to the present embodiment defines the media attributes of each component stream included in the Muxed track, and associates the attributes with the SubSampleEntry including the attribute information of the codec of each component stream. In addition, the file generation device associates the attribute information of the codec of each component stream that can be changed dynamically with the SubSampleEntry of each component stream by using SubSampleToChunkBox. Therefore, since a plurality of component streams are included in one track, the time information of each component stream and the like can be shared, and the amount of information can be reduced. That is, the amount of information can be reduced and a high-quality viewing experience can be provided to the viewer. In addition, for each component stream centrally stored in a single track, the mechanism of ISOBMFF that has been implemented using the track so far can be implemented, in which the handler information as the media attribute information of the track is defined, the SampleEntry representing the codec attribute information is defined, and the codec information of the Sample is dynamically associated with the SampleEntry.

[0114] [1.1 Modification of the First Embodiment]

[0115] Next, a modification of the first embodiment will be described. In the first embodiment, handler information as type information of a component stream corresponding to each SubSample is stored in SubSampleEntryBox. However, SampleEntry stores Sample specific information, and therefore preferably, SubSampleEntry stores SubSample specific information.

[0116] Therefore, in the present modification, information of a processing procedure of a component stream corresponding to each SubSample included in a Muxed track is stored in a header area of ​​a file different from the SubSampleEntryBox. Fig.11 is a diagram showing an example of the syntax of SubSampleToHandlerBox and SubSampleEntryBox according to the modification example of the first embodiment.

[0117] The preprocessing unit 102 redefines and generates SubSample handler information indicating the contents of SubSampleHandlerBox represented by the syntax 181. In this case, the preprocessing unit 102 sets component_id which is identification information of SubSample and handler_type which is information of a handler of a component stream corresponding to each SubSample in the SubSample handler information.

[0118] In addition, the preprocessing unit 102 generates SubSample related information indicating the content of the SubSampleEntryBox represented by the syntax 182. In this case, the preprocessing unit 102 sets component_id as identification information of the SubSample and Muxed_stream_specific_type indicating the type of the component stream corresponding to each SubSample in the SubSample related information. In addition, the preprocessing unit 102 also sets SubSampleEntry() in the SubSample related information.

[0119] That is, SubSample specific information is set in the SubSample related information. Meanwhile, in the SubSample handler information, information of the handler of the component stream corresponding to the SubSample is stored. Both the SubSample handler information and the SubSample related information are linked to each other through component_id and to the SubSample corresponding information.

[0120] The file generation unit 104 receives an input of metadata including SubSample related information, SubSample handler information, and SubSample corresponding information from the preprocessing unit 102. Then, the file generation unit 104 redefines the SubSampleEntryBox represented by the syntax 181 and stores the SubSample related information, and stores the SubSampleEntryBox in moov. In addition, the file generation unit 104 redefines the SubSampleHandlerBox represented by the syntax 182 and stores the SubSample handler information, and stores the SubSampleHandlerBox in moov. In addition, the file generation unit 104 stores the SubSampleToChunkBox including the SubSample corresponding information in moof. Then, the file generation unit 104 generates an ISOBMFF file storing each component stream.

[0121] By configuring the SubSampleHandlerBox and the SubSampleEntryBox in this way, the SubSampleEntryBox can also be configured to follow the relationship between Sample and SampleEntry.

[0122] [2. Second Embodiment]

[0123] The file generation device 1 according to the present embodiment stores a parameter set of a SubSample in the SubSample, and transmits the parameter set together with the component stream. Figure 2 The file generation device 1 according to this embodiment is also shown in the block diagram of Figure 6 The client device 2 according to the present embodiment is also shown in the block diagram of In the following description, the description of the operation of the same units as those of the first embodiment will be omitted.

[0124] Fig.12 is a diagram used to describe the storage state of a parameter set. Fig.12 As shown, the pre-processing unit 102 determines that the parameter set 211 is stored in the SubSample. The parameter set 211 includes information for initializing the SubSample of each component stream in the Muxed track and information for associating the SubSample with the SubSampleEntry. For example, the pre-processing unit 102 stores the parameter set 211 in the SubSample by using the supplemental enhancement information (SEI) of the codec.

[0125] As described above, the preprocessing unit 102 according to the present embodiment transmits information equivalent to the SampleEntry related information of each component stream in the inbound stream by using the parameter set of the codec and SEI. On the other hand, in the moof as in the first embodiment, the information related to the SampleEntry of each SubSample can also be stored by setting the SubSampleToChunkBox under the TrackFragmentBox. However, in the case of using TrackFragmentBox, since the SubSampleToChunkBox is used to identify the link between the SubSample and the SubSample entry, the consumption of moof is large.

[0126] That is, compared with the case of using SubSampleToChunkBox, the preprocessing unit 102 according to the present embodiment can reduce the consumption of moof. In this case, it is necessary to notify the client to transmit the parameter set 211 in the inbound stream. However, when a new BOX is defined, the consumption of moof is generated, and therefore, it is preferable to use the reserved bits of the existing BOX. Therefore, the preprocessing unit 102 according to the present embodiment performs the following definition.

[0127] The preprocessing unit 102 redefines the undefined bits of tr_frag of TrackTunBox by allocating bits to notification of the parameter set 211 for the SubSample for each component stream transmitted in the inbound stream. The information used for the notification corresponds to an example of "storage notification information". In the present embodiment, the preprocessing unit 102 allocates bit 0x800000 of tr_flags to notification of the parameter set 211 for the SybSample for each component stream transmitted in the inbound stream. Then, the preprocessing unit 102 notifies the file generation unit of the storage of the parameter set 211 in the SubSample and the enabling of the 0x800000 bit of tr_flag. Therefore, the preprocessing unit 102 does not need to store SubSampleToChunkBox in the moof which is the header area of ​​each segment.

[0128] In addition, the pre-processing unit 102 generates SubSampleEntry for each component stream. In this case, since the parameter set 211 is transmitted in the inbound stream, the pre-processing unit 102 sets capability information indicating whether each SubSample can be used by the application that performs the reproduction process in the SubSampleEntry. Examples of the capability information include information indicating the type of codec, image size information, and frame table information.

[0129] The file generation unit 104 receives a notification from the pre-processing unit 102 that the parameter set 211 is stored in SubSampleEntry and SubSample and that the bit 0x800000 of tr_flag is enabled. Fig.12 As shown, the file generation unit 104 then stores the SubSampleEntry of each component stream in moov.

[0130] In addition, if Fig.12 As shown, the file generation unit 104 stores the parameter set 211 in the SubSample. In addition, the file generation unit 104 enables bit 0x800000 of tr_flag in TrackRunBox as indicated by BOX 212, and indicates that the parameter set 211 of each SubSample is transmitted together with the data of the component stream in the mdat area.

[0131] Fig.13 : is a diagram showing the storage state of SubSampleEntryBox and tr_flag depending on the presence or absence of segmentation. In the case where movie segmentation is not performed, as Fig.13 As shown in the file 220 , the file generation unit 104 stores the SubSampleEntryBox indicated by BOX221 in moov.

[0132] On the other hand, in the case of performing movie segmentation, such as Fig.13 As shown in the file 230 of moov, the file generation unit 104 sets the SubSampleEntryBox indicated by BOX 231 in moov. In addition, the file generation unit 104 sets the value of bit 0x800000 of tr_flag to enable in the TrackRunBoxes indicated by BOX 232 to BOX 234 of each moof.

[0133] Meanwhile, in the client device 2, the file processing unit 202 acquires the parameter set included in each SubSample when parsing the segment file. Then, the file processing unit 202 outputs the parameter set to the decoding processing unit 203 together with the bit stream of each component stream.

[0134] The decoding processing unit 203 receives an input of a parameter set together with the bit stream of each component stream from the file processing unit 202. Then, the decoding processing unit 203 initializes each bit stream using the acquired parameter set. Thereafter, the decoding processing unit 203 decodes the data of each component stream using the encoding parameters stored in the SubSampleEntry.

[0135] (File Generation Process According to Second Embodiment)

[0136] Next, we will refer to Fig.14 The flow of the file generation process executed by the file generation device 1 according to the second embodiment will be described in detail. Fig.14 1 is a flowchart of a file generation process executed by the file generation device according to the second embodiment.

[0137] The data acquisition unit 101 acquires the original data of the video content from the Web server 3. Then, the preprocessing unit 102 outputs the image data to be encoded and the control information included in the acquired original data acquired by the data acquisition unit 101 to the encoding unit 103. The preprocessing unit 102 sets the encoding parameters of the Muxed track in the SampleEntry by using the control information included in the acquired original data (step S301).

[0138] Next, the pre-processing unit 102 sets the encoding parameters of each component in SubSampleEntry (step S302).

[0139] The encoding unit 103 receives input of image data and metadata from the data acquisition unit 101. Then, the encoding unit 103 encodes the image data and metadata to generate encoded data of each component stream (step S303). Thereafter, the encoding unit 103 outputs the encoded data of each component stream to the file generation unit 104.

[0140] Next, the file generation unit 104 sets the codec information in the SubSample of each component stream (step S304). Thus, the parameter set of each SubSample is bound to the bitstream.

[0141] Next, the file generation unit 104 creates a Muxed track by collecting the SubSamples of the respective components as one Sample, sets the Sample in mdat, and sets the Sample information (step S305).

[0142] Next, the file generation unit 104 sets the value of bit 0x800000 of tr_flag of moof to enable (step S306 ).

[0143] Then, the file generation unit 104 generates an ISOBMFF file having a Muxed track including a component stream of each component (step S307 ) Thereafter, the transmission unit 108 uploads the segment file of the video content generated by the file generation unit 104 to the Web server 3 .

[0144] (Procedure of Reproduction Processing in Second Embodiment)

[0145] Next, we will refer to Fig.15 The flow of the reproduction process performed by the client device 2 according to the second embodiment is described. Fig.15 This is a flowchart of a reproduction process executed by the client device according to the second embodiment.

[0146] The file acquisition unit 201 acquires the segment file of the video content to be reproduced from the Web server 3. The file processing unit 202 parses the segment file of the video content acquired by the file acquisition unit 201. Then, the file processing unit 202 acquires Sample and SampleEntry from the ISOBMFF file. Thereafter, the file processing unit 202 outputs the acquired Sample and SampleEntry to the decoding processing unit 203. In addition, the file processing unit 202 outputs metadata to the display information generation unit 205. The decoding processing unit 203 acquires Sample and SampleEntry and sets decoding (step S401).

[0147] Next, the decoding processing unit 203 acquires a parameter set added to the bitstream for each of the plurality of SubSamples stored in the acquired Sample. Then, the decoding processing unit 203 decodes each component stream using the parameter set and the information of the corresponding SubSampleEntry (step S402).

[0148] The display information generating unit 205 acquires the decoded data of each component stream from the decoding processing unit 203. In addition, the display information generating unit 205 acquires the operation command from the display control unit 204. Then, the display information generating unit 205 performs a display process of performing visualization using the data of each component stream to generate a display image in response to the operation command, and causing the display unit 207 to display the display image (step S403).

[0149] Thereafter, the file processing unit 202, the decoding processing unit 203, the display control unit 204, and the display information generation unit 205 determine whether to continue the reproduction process (step S404). For example, in the case where a stop command or the like is input from the user, the file processing unit 202, the decoding processing unit 203, the display control unit 204, and the display information generation unit 205 determine not to continue the reproduction process. In the case where the reproduction process is continued (step S404: Yes), the video reproduction process returns to step S401. On the other hand, in the case where the decoding of all the image data of the video content is completed (step S404: No), the file processing unit 202, the decoding processing unit 20, the display control unit 204, and the display information generation unit 205 end the video reproduction process.

[0150] As described above, the file generation device according to the present embodiment transmits the parameter set of each SubSample by binding to the bitstream. That is, the file generation device according to the present embodiment transmits the codec attribute information of each component stream in the stream, and the codec attribute information changes dynamically in the SubSample of each component stream. Therefore, it is possible to omit associating each SubSample with the SubSampleEntry by setting the SubSampleToChunkBox in the header area at the time of slicing, and it is possible to reduce the consumption caused by the management information at the time of slicing.

[0151] [2.1 Modification of Second Embodiment]

[0152] In the second embodiment, tr_flag of TrackRunBox is used to indicate an inbound stream, but the storage location of tr_flag is not limited thereto. For example, the file generation unit 104 may define a flag for indicating an inbound stream as information of the entire fragment by using tr_flag of TrackFragmentHeaderBox in the same hierarchy as TrackRunBox.

[0153] In addition, in the case where SubSampleToChunkBox is not set under TrackFragmentBox, the file generation unit 104 can perform definition using the semantics of SubSampleEntry that transmits each SubSample in an inbound stream.

[0154] As described above, in addition to using tr_flags of TrackRunBox, parameters of each SubSample can be transmitted in the inbound stream, and in this case, consumption caused by management information at the time of slicing can be reduced. Information for notification other than the case of using tr_flag also corresponds to an example of "storing notification information".

[0155] The structure described in each of the above-mentioned embodiments and modifications can be applied to a stream including a plurality of component streams other than V-PCC.

[0156] [3. Third embodiment]

[0157] The above-mentioned embodiment and its modification have described the case where storage is performed in ISOBMFF. However, even by using Fig.16 In the case where transmission is performed using the Matroska media container (http: / / www.matroska.org / ) shown in FIG. 4 , it is also possible to define media attributes for each of a plurality of component streams, and to associate codec attribute information for each component stream with data of each component stream. Fig.16 : is a diagram showing the format of the Matroska media container. In this case, the file generation unit 104 stores the information stored in the SubSampleEntryBox and the information stored in the SubSampleToChunkBox of each of the embodiment and the modification example in an element newly defined in the TrackEntry element.

[0158] [Hardware Configuration]

[0159] Fig.17 is a hardware configuration diagram of a computer. The file generation device 1 and the client device 2 can be composed of Fig.17 The computer 90 shown in FIG. 9 is implemented. In the computer 90, a processor 91, a memory 92, a network interface 93, a nonvolatile storage device 94, an input / output interface 95, and a display interface 96 are connected to each other via a bus.

[0160] For example, external devices such as an input device, an output device, a storage device, and a drive are connected to the input / output interface 95. For example, the input device is a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. For example, the output device is a speaker, an output terminal, etc. For example, the storage device is a hard disk, a random access memory (RAM) disk, etc. For example, the drive drives a removable medium disk, an optical disk, a magneto-optical disk, or a semiconductor memory. In addition, a display 98 as a display device is connected to the display interface 96.

[0161] The network interface 93 is connected to an external network. The file generation device 1 and the client device 2 are connected to each other via the network interface 93. In addition, the file generation device 1 and the client device 2 are connected to the Web server 3 via the network interface 93. The nonvolatile storage 94 is a built-in auxiliary storage device, such as a hard disk or a solid state drive (SSD).

[0162] In the computer 90 configured as described above, for example, the processor 91 loads the program stored in the nonvolatile storage device 94 into the memory 92 via the bus and executes the program, thereby executing the above-mentioned series of processing. The memory 92 also appropriately stores data and the like required for the processor 91 to execute various types of processing.

[0163] For example, the program executed by the processor 91 can be applied by recording it on a removable medium as a package medium, etc. In this case, by attaching the removable medium to a drive as an external device 97, the program can be installed in the nonvolatile storage device 94 via the input / output interface 95.

[0164] In addition, the program may be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program may be received through the network interface 93 and installed in the nonvolatile storage device 94.

[0165] Alternatively, the program may be installed in the nonvolatile storage device 94 in advance.

[0166] The embodiments of the present disclosure have been described above, but the technical scope of the present disclosure is not limited to the embodiments described above, and various changes can be made without departing from the gist of the present disclosure. In addition, components of different embodiments and modifications can be appropriately combined.

[0167] It should be noted that the effects described in this specification are merely examples and not limitations, and other effects may be obtained.

[0168] Note that the present technology may also have the following configurations.

[0169] (1) An information processing device comprising:

[0170] a pre-processing unit that separately generates SubSample information for decoding each SubSample in a content stream, the content stream including a plurality of SubSamples as component streams; and

[0171] A file generating unit generates a file including the SubSample information generated by the pre-processing unit.

[0172] (2) An information processing device according to (1), wherein the preprocessing unit causes SubSample related information to be included in the SubSample information, the SubSample related information corresponding to each of a plurality of SubSamples and including identification information of the SubSample, type information of a plurality of component streams, media attribute information of the SubSample, and codec attribute information.

[0173] (3) An information processing device according to (2), wherein the file generation unit inserts the SubSample related information into the header area of ​​a file format having a single header area and a single data area to generate a file in which a single content stream generated from a plurality of component streams is inserted into the data area.

[0174] (4) An information processing device according to (2), wherein the file generation unit inserts the SubSample related information into a first header area including initialization information of the following file format to generate a file in which a single content stream generated from a plurality of component streams is divided and inserted into each data area, and the file format has the first header area, a plurality of data areas, and a plurality of second header areas corresponding to the data areas.

[0175] (5) The information processing device according to (2), wherein the file generation unit stores the SubSample related information in a SubSampleEntryBox.

[0176] (6) The information processing device according to (2), wherein the preprocessing unit causes SubSample corresponding information to be further included in the SubSample information, the SubSample corresponding information indicating a corresponding relationship between a plurality of SubSamples and the SubSample related information.

[0177] (7) The information processing device according to (6), wherein the file generation unit stores the SubSample corresponding information in a SubSampleToChunkBox of the file.

[0178] (8) An information processing device according to (2), wherein the preprocessing unit generates first related information and second related information, the first related information includes identification information of the SubSample and type information of the component stream in the SubSample related information, and the second related information includes identification information of the SubSample and media attribute information of the SubSample.

[0179] (9) The information processing device according to (8), wherein the file generation unit stores the first related information in a SubSamplehandlerBox, and stores the second related information in a SubSampleEntryBox.

[0180] (10) The information processing device according to (1), wherein the preprocessing unit causes a parameter set for decoding to be included in the SubSample information.

[0181] (11) The information processing device according to (10), wherein the file generation unit causes the SubSample information to be included in a component stream.

[0182] (12) The information processing device according to (11), wherein the file generation unit generates storage notification information indicating that the SubSample information is included in a component stream, and stores the storage notification information in a TrackRunBox.

[0183] (13) An information processing method, causing a computer to execute the following processing:

[0184] Separately generating SubSample information for decoding each SubSample in a content stream including a plurality of SubSamples as component streams; and

[0185] Generates a file including the generated SubSample information.

[0186] (14) A reproduction processing device comprising:

[0187] A file processing unit that acquires SubSample information for decoding each SubSample as a component stream from a header area of ​​a file generated according to a file format having the header area and a data area in which a content stream including a plurality of SubSamples is stored; and

[0188] A decoding processing unit is configured to decode the content stream based on the SubSample information obtained by the file processing unit.

[0189] (15) A reproduction processing method causes a computer to execute the following processing:

[0190] acquiring SubSample information for decoding each SubSample as a component stream from a header area of ​​a file generated according to a file format having the header area and a data area in which a content stream including a plurality of SubSamples is stored; and

[0191] The content stream is decoded based on the acquired SubSample information.

[0192] Reference numerals list

[0193] 1 File generation device

[0194] 2 Client Devices

[0195] 3 Web Server

[0196] 10 File generation processing unit

[0197] 11 Control Unit

[0198] 12 Transmission Unit

[0199] 20 Reproduction Processing Unit

[0200] 21 Control unit

[0201] 100 Distribution System

[0202] 101 Data Acquisition Unit

[0203] 102 Preprocessing Unit

[0204] 103 Coding Unit

[0205] 104 File generation unit

[0206] 201 File acquisition unit

[0207] 202 File Processing Unit

[0208] 203 Decoding processing unit

[0209] 204 Display control unit

[0210] 205 Display information generation unit

[0211] 206 Display Unit

Claims

1. An information processing device, comprising: a pre-processing unit that separately generates SubSample information for decoding each SubSample in a content stream, the content stream including a plurality of SubSamples as component streams; as well as A file generating unit, which generates a file including the SubSample information generated by the pre-processing unit, wherein the preprocessing unit causes SubSample related information to be included in the SubSample information, the SubSample related information corresponding to each of the plurality of SubSamples and including identification information of the SubSample, type information of the plurality of component streams, media attribute information of the SubSample, and codec attribute information, The file generation unit stores the SubSample related information in SubSampleEntryBox. wherein the pre-processing unit further causes SubSample corresponding information to be included in the SubSample information, the SubSample corresponding information indicating a corresponding relationship between a plurality of SubSamples and the SubSample related information, and The file generation unit stores the SubSample corresponding information in the SubSampleToChunkBox of the file.

2. The information processing device according to claim 1, wherein: The file generation unit inserts the SubSample related information into the header area of ​​a file format having a single header area and a single data area to generate a file in which a single content stream generated from a plurality of component streams is inserted into the data area.

3. The information processing device according to claim 1, wherein: The file generation unit inserts the SubSample related information into a first header area including initialization information of the following file format to generate a file in which a single content stream generated from multiple component streams is divided and inserted into each data area, and the file format has the first header area, multiple data areas, and multiple second header areas corresponding to the data areas.

4. The information processing device according to claim 1, wherein: The preprocessing unit generates first relevant information and second relevant information, wherein the first relevant information includes identification information of the SubSample and type information of the component stream in the SubSample relevant information, and the second relevant information includes identification information of the SubSample and media attribute information of the SubSample.

5. The information processing device according to claim 4, wherein: The file generation unit stores the first related information in a SubSamplehandlerBox, and stores the second related information in a SubSampleEntryBox.

6. The information processing device according to claim 1, wherein: The pre-processing unit causes a parameter set for decoding to be included in the SubSample information.

7. The information processing device according to claim 6, wherein: The file generating unit causes the SubSample information to be included in a component stream.

8. The information processing device according to claim 7, wherein: The file generation unit generates storage notification information indicating that the SubSample information is included in a component stream, and stores the storage notification information in a TrackRunBox.

9. An information processing method, causing a computer to perform the following processing: Separately generating SubSample information for decoding each SubSample in a content stream including a plurality of SubSamples as component streams; and Generate a file including the generated SubSample information, in, SubSample related information is included in the SubSample information, the SubSample related information corresponds to each of a plurality of SubSamples and includes identification information of the SubSample, type information of a plurality of component streams, media attribute information of the SubSample, and codec attribute information, The SubSample related information is stored in SubSampleEntryBox. Wherein, SubSample corresponding information is included in the SubSample information, and the SubSample corresponding information indicates a corresponding relationship between a plurality of SubSamples and the SubSample related information, and The SubSample corresponding information is stored in the SubSampleToChunkBox of the file.

10. A reproduction processing device, comprising: A file processing unit, which acquires SubSample information for decoding each SubSample as a component stream from a header area of ​​a file generated according to the following file format, the file format having the header area and a data area, in which a content stream including a plurality of SubSamples is stored; as well as a decoding processing unit, which decodes the content stream based on the SubSample information obtained by the file processing unit, The SubSample related information is included in the SubSample information, the SubSample related information corresponds to each of the multiple SubSamples and includes identification information of the SubSample, type information of the multiple component streams, media attribute information and codec attribute information of the SubSample, The SubSample related information is stored in SubSampleEntryBox. Wherein, SubSample corresponding information is included in the SubSample information, and the SubSample corresponding information indicates a corresponding relationship between a plurality of SubSamples and the SubSample related information, and The SubSample corresponding information is stored in the SubSampleToChunkBox of the file.

11. A reproduction processing method, causing a computer to perform the following processing: acquiring SubSample information for decoding each SubSample as a component stream from a header area of ​​a file generated according to a file format having the header area and a data area in which a content stream including a plurality of SubSamples is stored; and decoding the content stream based on the obtained SubSample information, in, SubSample related information is included in the SubSample information, the SubSample related information corresponds to each of a plurality of SubSamples and includes identification information of the SubSample, type information of a plurality of component streams, media attribute information of the SubSample, and codec attribute information, The SubSample related information is stored in SubSampleEntryBox. Wherein, SubSample corresponding information is included in the SubSample information, and the SubSample corresponding information indicates a corresponding relationship between a plurality of SubSamples and the SubSample related information, and The SubSample corresponding information is stored in the SubSampleToChunkBox of the file.

Images