Reproduction method and device

Abstract

To provide a method for receiving data in which a coded stream is divided into a plurality of files and playing back the accumulated data.SOLUTION: A playback method is to play back a plurality of audio files and a plurality of video files each scalably coded on the basis of data in a predetermined file format, and the playback method including: a first acquisition step of acquiring control information for playing back the plurality of audio files and the plurality of video files, the control information associating the plurality of audio files with the plurality of video files, the control information including information indicating the order of playing back the plurality of audio files and the plurality of video files and information indicating values used for deriving the start time to play back the plurality of audio files and the time to play back the plurality of video files; and a playback step of sequentially playing back predetermined files of the plurality of audio files and the plurality of video files based on the control information.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to a playback method and a playback device. [Background technology]

[0002] Conventionally, a technique for transmitting encoded data in a predetermined transmission format is known. The encoded data is generated by encoding content including video data and audio data based on a video encoding standard such as High Efficiency Video Coding (HEVC).

[0003] The predetermined transmission format includes, for example, MPEG-2 TS (Moving Picture Experts Group-2 Transport Stream) or MMT (MPEG Media Transport) (see Non-Patent Document 1). For example, Patent Document 1 discloses a technique for transmitting encoded media data packet by packet in accordance with MMT. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Information technology - High efficiency coding and media delivery in heterogeneous environment - Part1:MPEG media transport(MMT), ISO / IEC DIS 23008-1 Summary of the Invention [Problem to be solved by the invention]

[0005] However, no consideration is given to a playback device for receiving and storing data in which one stream is divided into multiple files, such as in the MMT system.

[0006] Therefore, an object of the present invention is to provide a method for playing back data that has been received and stored as a single coded stream divided into a plurality of files. [Means for solving the problem]

[0007] In order to solve the above problem, a playback method according to one embodiment of the present invention is a playback method for playing a plurality of audio files and a plurality of video files, each of which is scalably encoded in data units of a predetermined file format, and includes a first acquisition step of acquiring control information for playing the plurality of audio files and the plurality of video files, the control information associating the plurality of audio files and the plurality of video files, the control information including information indicating the order in which the plurality of audio files and the plurality of video files are to be played and information indicating values ​​used to derive the playback start times of each of the plurality of audio files and the plurality of video files, and a playback step of sequentially playing predetermined files among the plurality of audio files and the plurality of video files based on the control information.

[0008] In addition, a playback device according to one embodiment of the present invention is a playback device that plays a plurality of audio files and a plurality of video files, each of which is scalably encoded in data units of a predetermined file format, and is equipped with a first acquisition unit that acquires control information for playing the multiple audio files and the multiple video files, the control information associating the multiple audio files and the multiple video files, the control information including information indicating the order in which the multiple audio files and the multiple video files are played and information indicating values ​​used to derive the playback start times of each of the multiple audio files and the multiple video files, and a playback unit that sequentially plays predetermined files among the multiple audio files and the multiple video files based on the control information.

[0009] In order to solve the above problem, a playback method according to one embodiment of the present invention is a playback method for playing a plurality of files, the files being scalably encoded in data units of a predetermined file format, the playback method including: a first acquisition step of acquiring control information for playing the plurality of files, the control information associating the plurality of files, the control information including information indicating the order in which the plurality of files are to be played and information indicating a value used to derive the playback start time of each of the plurality of files; and a playback step of sequentially playing predetermined files of the plurality of files based on the control information.

[0010] In addition, a playback device according to one embodiment of the present invention is a playback device that plays a plurality of files, the files being scalably encoded in data units of a predetermined file format, and is equipped with: a first acquisition unit that acquires control information for playing the plurality of files, the control information associating the plurality of files, the control information including information indicating the order in which the plurality of files are to be played and information indicating a value used to derive the playback start time of each of the plurality of files; and a playback unit that sequentially plays predetermined files of the plurality of files based on the control information.

[0011] In order to solve the above problem, a playback device according to one embodiment of the present invention includes a first receiving unit that receives control information stored in a content transmission device from the content transmission device in order to play a plurality of segments corresponding to a data unit defined in the MPEG-DASH standard and including a plurality of video segments and a plurality of audio segments, an extraction unit that extracts from the control information a playback start timing that corresponds to a first video segment among the plurality of video segments and a first audio segment among the plurality of audio segments, a second receiving unit that receives the first video segment and the first audio segment from the content transmission device, and a playback unit that simultaneously plays back the first video segment and the first audio segment based on the playback start timing, wherein the control information has a flag indicating whether the control information is updatable or not.

[0012] In addition, a content transmission device according to one embodiment of the present invention includes an acquisition unit that acquires an encoded stream having a plurality of segments corresponding to data units defined in the MPEG-DASH standard and including a plurality of video segments and a plurality of audio segments; a storage that stores the plurality of segments; and a transmission unit, wherein the storage stores control information including a playback start timing that corresponds to a first video segment among the plurality of video segments and a first audio segment among the plurality of audio segments, and the storage stores a flag that indicates whether the control information is updatable and is included in the control information, the transmission unit transmits the control information to the playback device so that the playback device simultaneously plays back the first video segment and the first audio segment based on the playback start timing, and the transmission unit transmits the first video segment and the first audio segment to the playback device.

[0013] In order to solve the above problem, a playback method according to one embodiment of the present invention includes receiving control information stored in a content transmission device from the content transmission device in order to play a plurality of segments corresponding to a data unit defined in the MPEG-DASH standard and including a plurality of video segments and a plurality of audio segments, extracting from the control information a playback start timing that corresponds to a first video segment among the plurality of video segments and a first audio segment among the plurality of audio segments, receiving the first video segment and the first audio segment from the content transmission device, and simultaneously playing back the first video segment and the first audio segment based on the playback start timing, wherein the control information has update information indicating whether or not the control information is updated.

[0014] In addition, a content transmission method according to another aspect of the present invention acquires an encoded stream having a plurality of segments corresponding to data units defined in the MPEG-DASH standard and including a plurality of video segments and a plurality of audio segments, stores the plurality of segments in a storage, stores in the storage control information including a playback start timing that corresponds to a first video segment among the plurality of video segments and a first audio segment among the plurality of audio segments, stores in the storage update information that indicates whether the control information is updated and is included in the control information, transmits the control information to a playback device so that the playback device simultaneously plays back the first video segment and the first audio segment based on the playback start timing, and transmits the first video segment and the first audio segment to the playback device.

[0015] In order to solve the above problem, an accumulation method according to one embodiment of the present invention includes a receiving step of receiving an encoded stream including a plurality of files scalably encoded in data units of a predetermined file format, a storage step of storing the received plurality of files from the encoded stream in a memory unit, and a generating step of generating control information used to play the stored plurality of files, the control information associating the plurality of files, wherein the plurality of files are played in sequence, and the control information includes information indicating the order in which the plurality of files are to be played and information indicating a value used to derive the playback start time of each of the plurality of files.

[0016] In addition, a playback method according to another aspect of the present invention is a playback method for playing a plurality of files, the files being scalably encoded in data units of a predetermined file format, the playback method including: a first acquisition step of acquiring control information for playing the plurality of files, the control information associating the plurality of files, the control information including information indicating the order in which the plurality of files are to be played and information indicating a value used to derive the playback start time of each of the plurality of files; an analysis step of analyzing the acquired control information; a second acquisition step of sequentially acquiring predetermined files from the plurality of files based on a result of the analysis step; and a playback step of sequentially playing the acquired predetermined files to play the plurality of files in sequence.

[0017] In order to solve the above problem, an accumulation method according to one embodiment of the present invention includes a receiving step of receiving an encoded stream including a plurality of files, which are data units in a predetermined file format; a storage step of storing the received plurality of files from the encoded stream in a memory unit; and a generating step of generating first control information used to play the stored plurality of files, the first control information associating the plurality of files.

[0018] In addition, these general or specific aspects may be realized by a playback method, a storage device, a playback device, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized by any combination of the storage method, the playback method, the storage device, the playback device, the integrated circuit, the computer program, and the recording medium. Effect of the Invention

[0019] The playback method of the present invention can play back stored encoded data. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram for explaining the data structure of an encoded stream in MMT. [Diagram 2] FIG. 1 is a diagram for explaining the data structure of an encoded stream in MMT. [Diagram 3] 1 is a block diagram showing a configuration of a storage device according to an embodiment; [Figure 4] 1 is a flowchart of a storage method according to an embodiment. [Diagram 5] 13 is a diagram showing data obtained by receiving data encoded using the MMT method and obtaining MPUs for each media from MMT packets. [Figure 6] FIG. 13 is a diagram showing the structure of a media file A21 when the media file A21 is formed by accumulating files of media A (video). [Figure 7] 11 is a diagram for explaining a specific example of storing a file and generating control information based on a received file and second control information. FIG. [Figure 8] FIG. 13 is a diagram illustrating an example of a random access table. [Figure 9] 1 is a block diagram showing a configuration of a playback device according to an embodiment. [Figure 10] 4 is a flowchart of a reproducing method according to an embodiment. [Figure 11] FIG. 13 is a diagram for explaining data in a media file when the data in the media file is rearranged. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] (Findings on which the present invention is based) The present invention relates to a storage method or device for receiving data transmitted using the MMT method currently being standardized by MPEG and storing the received data, or a playback method or device for playing back the stored data. The MMT method is a multiplexing method for multiplexing and packetizing video and audio data, and transmitting the multiplexed data over one or more transmission paths for broadcasting, communication, etc.

[0022] In conventional systems using the MPEG-2 TS method as a multiplexing method, there is a method for directly storing TS packets, but there is no sufficient consideration given to a method for receiving and storing data packetized by the MMT method, or a method for playing back stored data.

[0023] The MPU (Media Processing Unit) generated using the MMT method divides video, audio, etc. into appropriate sizes for each media and stores the data in MP4 format.

[0024] Conventional MP4 files store video, audio, and other tracks in a single MP4 file, and store information indicating the relationship between the tracks in the MP4 internal header. In contrast, MPUs are MP4 files in which a program is segmented by media, and the MPU itself does not contain information indicating the relationship between media. As a result, related video and audio cannot be played together.

[0025] In addition, if MPUs are stored as multiple MP4 files, the number of files increases, making file management and file access during playback cumbersome, and increasing the amount of processing required for playback.

[0026] In order to solve such problems, an accumulation method according to one embodiment of the present invention includes a receiving step of receiving an encoded stream including a plurality of files, which are data units in a predetermined file format; a storage step of storing the received plurality of files from the encoded stream in a memory unit; and a generating step of generating first control information used to play the accumulated plurality of files, the first control information associating the plurality of files.

[0027] According to this, even if data is stored as a plurality of files, the plurality of files can be stored in association with each other, so that the amount of processing involved in the reproduction process can be reduced.

[0028] For example, the encoded stream may include second control information, and in the generating step, information indicating an order in which the multiple files are to be played may be generated as the first control information by analyzing the second information.

[0029] For example, the plurality of files may include a first file belonging to a first medium and a second file belonging to a second medium, and the generating step may further include generating information associating the first file and the second file as the first control information by analyzing the second control information.

[0030] For example, each of the multiple files may be a random access unit file that can be randomly accessed, and in the generation step, time information indicating the playback start time of each of the multiple files may be generated as the first control information by analyzing the second control information.

[0031] For example, in the generating step, information that associates the first file and the second file, which have the same playback start time, may be generated as the first control information.

[0032] For example, the generating step may further generate, as the first control information, information indicating recording positions in the storage section of the plurality of files when the plurality of files are stored in the storage section in the storing step.

[0033] Also, a playback method for playing a plurality of files, each of which is a data unit of a predetermined file format and stored in a memory unit, may be realized by the playback method including a first acquisition step of acquiring control information for playing the plurality of files, the control information associating the plurality of files, an analysis step of analyzing the acquired control information, a second acquisition step of acquiring a predetermined file from the plurality of files from the memory unit based on a result of the analysis step, and a playback step of playing the acquired predetermined file.

[0034] In addition, these general or specific aspects may be realized in a storage device, a playback device, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or in any combination of a storage device, a playback device, an integrated circuit, a computer program, or a recording medium.

[0035] Hereinafter, a transmitting method and a receiving method according to an embodiment of the present invention will be specifically described with reference to the drawings.

[0036] Note that the embodiments described below each show a specific example of the present invention. The numerical values, shapes, materials, components, the arrangement and connection of the components, steps, and the order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in an independent claim showing a superordinate concept are described as optional components.

[0037] (Embodiment) Hereinafter, a storage method (storage device) and a playback method (playback device) according to an embodiment will be described with reference to the drawings. In the embodiment, as an example, a storage method for receiving encoded data according to MMT and storing the received encoded data, and a playback method for playing back the stored encoded data will be described.

[0038] First, the data structure of an encoded stream in MMT will be described. Figures 1 and 2 are diagrams for explaining the data structure of an encoded stream in MMT.

[0039] As shown in Fig. 1, the coded data is composed of a plurality of access units (AUs). The coded data is, for example, AV data coded based on a video coding standard such as HEVC. Specifically, the coded data includes video data, audio data, and associated subtitles, superimposed text, metadata, still images, files, and the like. When the coded data is video data, one AU is a unit equivalent to one picture (one frame).

[0040] In MMT, the coded data is converted into MP4 data (with an MP4 header) in units of GOPs (Group Of Pictures) according to the MP4 file format (ISO Base Media Data Format). That is, the coded data has a number of sets (GOPs), each of which is made up of a number of access units. A GOP is a random access point in the coded data, and the first access unit in the GOP in decoding order corresponds to an IDR picture in HEVC or AVC, or an I picture in non-IDR.

[0041] The MP4 header included in the MP4 data describes the relative values ​​of the presentation time (the above-mentioned PTS) and the decoding time (the above-mentioned DTS) of the access unit. The MP4 header also describes the sequence number of the MP4 data. Note that MP4 data (MP4 file) is an example of an MPU, which is a data unit defined in the MMT standard. In the MPU, it is possible to transmit only the sample data in the MPU without transmitting the MP4 header. In this case, the MPU also corresponds to a random access unit, and the samples that constitute the MPU are associated with the MPU. In addition, a random access point, which is the first access unit of the random access unit, is placed at the beginning of the MPU. In addition, the MPU may be composed of multiple GOPs. In addition, the MPU is a collection of access units of a single media of video or audio, and is encapsulated in the form of an MP4 file for each media. Note that the "media" referred to here indicates the type of content, such as video, audio, and subtitles.

[0042] 2, an encoded stream 10 in MMT includes signaling information 11 and a plurality of MMT packets 12. In other words, the encoded stream 10 is a packet sequence of the MMT packets 12.

[0043] The coded stream 10 (MMT stream) is one of one or more streams that make up one MMT package. The MMT package corresponds to, for example, one broadcast program content.

[0044] The signaling information 11 includes, for example, information indicating a correspondence relationship between a plurality of assets and a packet ID. Note that an asset is a data entity including data with the same transport characteristics, and is, for example, one of video data and audio data.

[0045] Specifically, the signaling information 11 is, for example, CI (Composition Information) and MPT (MMT Package Table) in MMT. Note that the signaling information 11 is PMT (Program Map Table) and PAT (Program Association Table) in MPEG2-TS, and MPD (Media Presentation Description) in MPEG-DASH.

[0046] The signaling information 11 may also include time information. The time information is information for determining the PTS or DTS of an access unit. The time information is specifically, for example, the PTS or DTS as the absolute time of the first access unit in the MPU. More specifically, the time information can indicate, in the case of PTS, the absolute value of the PTS of the first access unit in presentation order in the MPU, and in the case of DTS, the absolute value of the DTS of the first access unit in decoding order in the MPU. The time information does not have to be stored in the signaling information 11.

[0047] For example, assuming that a set includes multiple pictures, the first access unit to be decoded in the decoding order of the set is decoded at the time indicated by the DTS. At this time, time information indicating the time (DTS) based on a reference clock at which the decoding of the first access unit of the set is performed may be stored as time information in the coded stream 10.

[0048] The reference clock is NTP (Network Time Protocol) when transmitting an encoded stream in the MMT format, and PCR (Program Clock Reference) when transmitting an encoded stream in the MPEG2-TS format. Here, the NTP may be a reference clock set by the transmitting device, and does not necessarily have to match the NTP value of an NTP server commonly used on the Internet.

[0049] The MMT packet 12 is data in which MP4 data has been packetized. In the embodiment, one MMT packet 12 includes one MP4 data (MPU). As shown in Fig. 2, the MMT packet 12 includes a header 12a (MTT packet header (corresponding to a TS packet header in the case of MPEG2-TS)) and a payload 12b.

[0050] MP4 data is stored in the payload 12b. Note that the payload 12b may store divided MP4 data.

[0051] The header 12a is auxiliary information related to the payload 12b. For example, the header 12a includes a packet ID and time information. The time information here is a relative value of the presentation time (PTS) or decoding time (DTS) of the MP4 data.

[0052] The packet ID is an identification number that indicates an asset of data included in the MMT packet 12 (payload 12b). The packet ID is an identification number that is unique to each asset that constitutes an MMT package.

[0053] In this way, the encoded stream includes time information (DTS or PTS) indicating the time when the encoded data is decoded or presented, and the encoded data (ID1_#0, ID2_#0, ID1_#1, ID1_#2, ID2_#1, ID2_#2, ... in Figure 2).

[0054] [Storage method] Fig. 3 is a block diagram showing a configuration of a storage device according to an embodiment of the present invention Fig. 4 is a flowchart showing a storage method according to an embodiment of the present invention.

[0055] 3, the storage device 100 includes a receiving unit 110, a storage unit 120, a generating unit 130, and a memory unit 140. The components of the storage device 100 are specifically realized by a microcomputer, a processor, or a dedicated circuit. The memory unit 140 does not need to be included in the storage device 100, and may be a device separate from the storage device 100.

[0056] In the storage method according to the embodiment, as shown in FIG. 4, the receiving section 110 of the storage device 100 receives an encoded stream including a plurality of files, which are data units in a predetermined file format (S11: receiving step).

[0057] Next, the accumulation unit 120 of the accumulation device accumulates the received multiple files of the encoded stream in the memory unit 140 (S12: accumulation step). Specifically, multiple data units in the MP4 file format (i.e., MP4 files) are obtained from the received packets constituting the received encoded stream 10.

[0058] Next, the generating unit 130 of the storage device generates first control information that is used to play the stored MP4 files and associates the MP4 files (S13: generating step). The first control information generated in step S13 includes the following three pieces of control information:

[0059] The three types of control information are information indicating the recording positions of multiple MP4 files in the storage unit 140 (control file A31 described later), program information (control file B32 described later), and information indicating a random access table (control file C33). That is, in step S13, three steps are performed: a step of generating control file A31, a step of generating control file B32, and a step of generating control file C33.

[0060] That is, for example, in the generating step, information indicating the order in which a plurality of MP4 files are to be played back may be generated as the first control information by analyzing the signaling information (second control information).

[0061] Also, for example, in the generating step, the signaling information may be analyzed to generate information associating a first file whose media is video with a second file whose media is audio as the first control information. More specifically, each of the multiple MP4 files is a random access unit file that can be randomly accessed, and in the generating step, the signaling information may be analyzed to generate time information indicating the playback start time of each of the multiple MP4 files as the first control information (control file B32). In addition, in the generating step, information associating a first file and a second file whose playback start time is the same may be generated as the first control information (control file C33).

[0062] In addition, in the generating step, information indicating the recording positions in storage unit 140 of the multiple MP4 files when the multiple MP4 files are stored in storage unit 140 in the storing step may be generated as first control information (control file A31).

[0063] The control information generated in step S13 is stored in the storage unit 140 together with the multiple MP4 files, and when reception of the encoded stream for a predetermined period of time has ended, storage ends.

[0064] The first control information and the second control information are different control information. The first control information is control information used to play back a plurality of stored MP4 files. The second control information is control information used to play back a coded stream in real time when the coded stream is received. That is, the first control information is control information used to record (store) broadcasted content and play back the recorded content. The second control information is control information used to play back content received in real time by broadcast or communication, for example.

[0065] Next, the details of each step of the accumulation method will be specifically described with reference to the drawings.

[0066] Fig. 5 is a diagram showing data obtained by receiving data encoded using the MMT method and acquiring MPUs for each media from the MMT packets. Fig. 6 is a diagram showing the structure of a media file A21 when a file of media A (video) is accumulated to form the media file A21.

[0067] The MMT packets 12 in the received encoded stream 10 are filtered by packet ID. After removing headers 12a such as the MMT packet header and payload header, the filtered MMT packets 12 become a collection of multiple MP4 files for each medium. The header of the MP4 file describes an identifier for each medium, sequence numbers of MP4 files that make up the same medium, and the like.

[0068] In addition to the MP4 file, multiple types of signaling information are received. The signaling information includes, for example, a table showing the relationship between media, the playback time of the first access unit of the MP4, information related to the startup of an application, etc. Some types of signaling information are received only once, while others are received repeatedly one or more times.

[0069] FIG. 7 is a diagram for explaining a specific example of storing a file and generating control information based on a received file and second control information.

[0070] In the storage method of this embodiment, the acquired MP4 file is directly stored in a file for each medium. At the same time, control information indicating the relationship between the media and other information required for playing the program are generated as control files A31 to C33 (first control information). When playing the program, the MP4 file for each medium is analyzed while referring to the control information, and video and audio are played.

[0071] First, we will explain an example of how one program is transmitted over a single transmission path using the MMT method, and how it is received and stored at a receiver. The program is composed of media A and media B, and the information that composes the program (information indicating the relationship between the media) is stored in the signaling information.

[0072] The MP4 files acquired in the receiving step are stored in a media file for each medium, and are accumulated in the storage unit 140. For example, M MP4 files of medium A with sequence numbers #1 to #M are stored in order in media file A21, and N MP4 files of medium B with sequence numbers #1 to #N are stored in order in media file B22 for medium B.

[0073] The media file may be set with a predetermined file name that identifies the media. The media file may also store within the media file an identifier that identifies the type of media of the file stored in the media file. As the identifier for identifying the type of media, for example, an asset ID that is assigned to each of a plurality of media by the MMT method may be used, or an identifier corresponding to the asset ID may be used. When the identifier is stored within the media file, a new mp4 box may be created and the identifier may be stored within it.

[0074] In the generating step, when storing an MP4 file in a media file, information for identifying the recording position of each MP4 file is generated and stored in a control file (control file A31 in FIG. 7). The position identifying information for identifying the recording position of each MP4 file is indicated, for example, by an offset amount by the number of bytes from the beginning of the media file to the first byte of the MP4 file of each sequence number for the media identifier. When the MMT method is used, the position identifying information indicates the recording position of the MP4 file for the asset ID.

[0075] In the receiving step, signaling information describing information about the media that make up the program is received, and the acquired signaling information is stored in a control file (e.g., control file B32 in FIG. 2). The signaling information stores the file names of the media files that store the MP4 files of the media that make up the program. There are signaling information that is sent only once, signaling information that is sent repeatedly, and signaling information that is sent in multiple parts. In the generating step, in any case, the information required for playing the program is acquired (generated) from the acquired signaling information and stored in a file.

[0076] In the generating step, a random access table may be created so that random access is possible during playback. The random access table is a list of randomly accessible points, and indicates, for example, the MP4 sequence numbers of media A and media B corresponding to the random access points.

[0077] FIG. 8 is a diagram illustrating an example of the random access table.

[0078] If media A is video and media B is audio, the random access table is a table that associates the MP4 sequence number of the video MP4 file with the MP4 sequence number of the audio MP4 file that is accessed (played) at the same time as the random access point of the video MP4 file.

[0079] Specifically, the random access table is generated in the following process. First, the random access point of the video MP4 file is generated based on the time information when the time information of the first access unit of the video MP4 file is transmitted as signaling information. Then, the MP4 sequence number of the audio MP4 file having the same time information as the time of the random access point of the generated video MP4 file is searched for, and the MP4 sequence number of the searched audio MP4 file is associated with the MP4 sequence number of the video MP4 file used in the search, thereby generating a random access table. That is, in the generating step, the random access table generated in this way is stored in a control file (for example, control file C33 in FIG. 7).

[0080] The random access table does not need to be a list of all randomly accessible points, but may be configured with random access points at regular intervals (that is, a portion of random access points thinned out so as to have regular intervals).

[0081] The random access table may further include time information corresponding to each random access point. In this case, the time information is a relative time from the start time of the media (the start time of the program). More specifically, when the time information of the first access unit of an MP4 file is a time stamp added based on NTP or STC, the time information is the relative time of the MP4 file calculated from the start time of the media.

[0082] In addition, when accumulation is started halfway through a program, the relative times corresponding to the multiple MP4 files are calculated based on the time of the accumulation start data, and the relative times are stored in the random access table. This time information may also be generated based on the timestamp described in the MP4 file header.

[0083] When layout information or application information is signaled, the signaling information may be stored in a file.

[0084] If a time stamp is attached to the signaling information, it may be converted into a relative time from the beginning time of the media and stored together with the signaling information.

[0085] In the generating step, event information such as layout information and application start information may be stored in the program information and random access table. In this case, the event information may be presented based on time stamp information during playback.

[0086] Even if the time interval of the audio MP4 file does not match the time interval (in GOP units) of the video MP4 file, the audio MP4 sequence numbers for the video random access points are searched for and a random access table is generated, making it easy to randomly access the video MP4 file and the audio MP4 file that should be played simultaneously with the video MP4 file during playback.

[0087] If there are multiple random access points in a video MP4 file, the number of random access points contained in the MP4 file may be stored in the header information and transmitted. In this case, the random access table associates each of the multiple random access points in the video MP4 file with the MP4 sequence number of the audio MP4 file. The audio MP4 file is further associated in the random access table with information indicating which random access point it is among the multiple random access points in the video MP4 file.

[0088] [How to play] Next, a playback method for playing back a plurality of MP4 files stored in storage unit 140 will be described.

[0089] Fig. 9 is a block diagram showing a configuration of a playback device according to an embodiment of the present invention, and Fig. 10 is a flowchart of a playback method according to an embodiment of the present invention.

[0090] 9, the playback device 200 includes a first acquisition section 210, an analysis section 220, a second acquisition section 230, and a playback section 240. The components of the playback device 200 are specifically realized by a microcomputer, a processor, a dedicated circuit, or the like.

[0091] First, the playback device 200 determines whether the file to be played back is a file including multiple MP4 files (media file) or a single MP4 file (S21).

[0092] When it is determined that the target file is a media file (S21: Media file), the first acquisition unit 210 of the playback device 200 acquires first control information that is control information for playing back the multiple MP4 files and associates the multiple MP4 files (S22: First Acquisition Step). Specifically, the first acquisition unit 210 acquires the first control information stored in the storage unit 140.

[0093] Next, in steps S23 to S25 following step S22, the analysis section 220 of the playback device 200 analyzes the first control information acquired in step S22 (analysis step).

[0094] More specifically, the analysis section 220 has a program information analysis section 221, a random access table analysis section 222, and a recording position analysis section 223, and these analysis sections 221 to 223 perform the processes of steps S23 to S25, respectively.

[0095] The program information analysis unit 221 analyzes the control file B32 of the first control information, and identifies the media files that constitute the program (S23).

[0096] Next, the random access table analysis unit 222 analyzes the control file C33 of the first control information, and obtains the MP4 sequence number of the random access point for each medium (S24).

[0097] Next, the recording position analysis unit 223 analyzes the control file A31 of the first control information and acquires position information indicating the recording position of the MP4 file in the memory unit 140, which is associated with the MP4 sequence number of the MP4 file acquired in step S24 (S25).

[0098] Next, the second acquisition section 230 acquires a predetermined MP4 file from among the multiple MP4 files from the storage section 140 based on the result of the analysis step (location information of the MP4 file) (S26: second acquisition step).

[0099] Next, the playback unit 240 of the playback device 200 plays the acquired MP4 files (S27: playback step). More specifically, the playback unit 240 has an MP4 file analysis unit 241 and a media decoding / presentation unit 242. In step S27, first, the MP4 file analysis unit 241 analyzes the multiple MP4 files in order for each media. Then, the media decoding / presentation unit 242 plays the multiple MP4 files by decoding and presenting them based on the timestamp information analyzed by the MP4 file analysis unit 241.

[0100] On the other hand, if it is determined that the target file is one MP4 file (S21: MP4 file), the MP4 file is analyzed, and the MP4 file is decoded and presented based on the time stamp information of the MP4, thereby playing the MP4 file (S28). Note that step S21 is not necessarily required. Also, step S28 may not be required.

[0101] (Features) According to the storage method and playback method of this embodiment, it is possible to store and play back a program coded in the MMT format.

[0102] Furthermore, according to the storage method of the present embodiment, a plurality of received MP4 files can be stored together in a file without changing the structure of the MP4 files.

[0103] Furthermore, according to the storage method of the present embodiment, control information such as program information is generated and stored, thereby making it possible to play back a plurality of stored MP4 files.

[0104] Furthermore, according to the storage method of the present embodiment, random access playback can be easily performed by generating a random access table.

[0105] Furthermore, according to the storage method of the present embodiment, layout information, application startup information, and the like can be presented during playback.

[0106] (Modification) The accumulation method described above is an example and is not limited to this.

[0107] The first control information may be stored in multiple files for each function, or the control information for two or more functions may be stored together in one file. Also, the control information for a single medium may be stored together with the medium in a media file.

[0108] The media file and the control file may be specified by a predetermined file name, and an identifier for identifying the contents of the media or control information stored in the media file and the control file may be stored in the file.

[0109] In addition, in the playback method, the control file in which the program information is stored is used as the entry point, but it is also possible to specify a separate file to be the entry point, and specify the control file in which the program information is stored in the entry point file.

[0110] Also, although the offset amount indicating the recording position of the MP4 file is described as being indicated by the number of bytes, other methods may be used. For example, the offset amount may specify a predetermined data unit and indicate an index number of the data unit. Also, the recording position of the MP4 file may be an offset amount from the first byte of the media file to the first byte of the MP4 file. Also, the recording position of the MP4 file may be differential information of the recording position from the first byte of the previous and next MP4 files.

[0111] Although the above description has been given with an example of data encoded in the MMT format, it is also possible to store and play back programs consisting of multiple segmented MP4 files, such as MPEG-DASH, using the method of this embodiment. Also, if an MP4 file encoded in the MMT format or MPEG-DASH format is The method of this embodiment can also be used for storage and playback when data is transmitted in MPU mode, GFD mode, or other protocols such as FLUTE.

[0112] Also, after storing multiple MP4 files in a media file in the order in which they were received, the order of the data (MP4 files) in the media file may be changed. When the data in the media file is rearranged, the first control information or the MP4 file header is appropriately modified in accordance with the rearrangement, and information indicating the rearrangement is stored in the media file or in a control file separate from the media file. Also, the data in the media file does not need to be rearranged after accumulation, and the data may be temporarily stored in memory when multiple MP4 files are received, and accumulated while rearranging.

[0113] For example, the header parts of multiple MP4 files may be collected in the first half of the file, and the data parts of multiple MP4 files may be collected in the second half of the file. Also, as shown in FIG. 11, if there is header information common to multiple MP4 files or header information unnecessary for storage and playback (e.g., 'ftyp', 'mmpu' box, etc.), the duplicated header information may be deleted. FIG. 11 is a diagram for explaining data in a media file when the data in the media file is rearranged. (a) of FIG. 11 is a diagram showing the structure of data in a media file before the data is rearranged. (b) of FIG. 11 is a diagram showing the structure of data in a media file after the data is rearranged.

[0114] Also, for example, if the initialization information differs for each MP4 file, the header needs to be modified. That is, the initialization information of the Nth MP4 file is compared with the initialization information of the first MP4 file, and if the initialization information of the Nth MP4 file differs from the initialization information of the first MP4 file, the initialization information of the Nth MP4 file is added to the initialization information of the first MP4 file, and when processing the Nth MP4 file, the reference destination in 'moof' of the Nth MP4 file may be changed so that the added initialization information can be referenced to the initialization information of the first MP4 file.

[0115] Also, a plurality of MP4 files of media may be stored in one media file. For example, a video MP4 file and an audio MP4 file may be stored alternately in random access units.

[0116] In addition, if the playback time is long, the media file may be divided into a number of files for storage. In this case, the divided files of the audio file may be divided to match the division time intervals of the video file.

[0117] In addition, when receiving and storing MP4 files transmitted using multiple transmission paths, the files may be stored without synchronizing the delay time between the transmission paths, and the stored MP4 files may be synchronized by, for example, sorting them based on the MP4 sequence numbers.

[0118] In addition, multiple MP4 files may be stored in advance in the transmission server using the media file and control file formats shown in this embodiment, and the receiver (storage device 100) may download and store the files from the transmission server via a broadcast transmission path or a communication transmission path.

[0119] In addition, MP4 files acquired by downloading from a broadcast transmission path or a communication transmission path and MP4 files acquired by streaming may be stored separately, and the MP4 files may be rearranged after storage.

[0120] In addition, when storing MP4 files including scalably encoded data, even if the media IDs of the enhancement layer and the base layer are different, the MP4 files of each layer may be stored in one media file, or each layer may be stored in a separate media file. When storing MP4 files of the base layer and the enhancement layer in one media file, the MP4 files of the corresponding base layer and enhancement layer may be stored alternately. Also, media of only the base layer may be stored. The control file describing the program information stores an identifier that indicates that the media file is scalably encoded data and enables identification of the hierarchical level.

[0121] In addition, when storing multiple received MP4 files, an identifier or flag indicating the state of the stored media may be provided for the stored media file. For example, an identifier or flag indicating whether the stored media corresponding to the stored media is complete may be provided in the media file or the control file. The flag is stored in the media file or the control file. Whether the media is complete can be determined, for example, from the media sequence number of the media identifier. The media sequence number is, for example, an MPU sequence number indicating the order of the MPU in the asset, a sequence number indicating the order of the packets in the asset, a sequence number indicating the order of the MMT packets, or a sequence number of the fragmented payload, and is indicated in the header of the data unit, MMT packet, or MMT payload. For this reason, a flag can be set according to the result of the determination. In addition, an identifier or flag indicating that the stored media is free of errors may be set. If there is an error in some MP4 files of the stored media, or if there is an MP4 file that cannot be obtained due to packet loss, etc., only a specific MP4 file may be downloaded from the server and stored, or the media may be supplemented by downloading each media file.

[0122] Additionally, an identifier or flag may be provided to indicate that the stored media is ready to be rearranged or that rearrangement has been completed, and an identifier or flag may be provided to indicate that the stored media is ready to be played back or listened to.

[0123] For example, it may be determined that the media is ready for rearrangement if there are no errors or omissions, or, for example, a program that requires rearrangement, it may be determined that the program is ready for playback only if rearrangement is complete.

[0124] In addition, content producers may transmit signaling information relating to conditions for storing, sorting, and playing / playing back programs, as well as permitted storage formats, and use this information in combination with identifiers and flags indicating the storage status to specify or restrict the operation of stored content.

[0125] In addition, content producers may assign unique IDs to programs and media in advance to uniquely specify them, and the receiver may include the unique ID along with the program or media. For example, it may be possible to play back stored programs by specifying the unique ID from data broadcasting, web content, or an application.

[0126] Also, for example, when a program is composed of media A transmitted by broadcast and stored media B, the program information indicates the storage device as the transmission path information (location information) of media B, and specifies the unique ID assigned as the media ID. When a storage device is indicated as location information in the receiver, it searches for the unique ID of the media from the program information in the storage device registered in advance, and plays it in sync with media A transmitted by broadcast.

[0127] Alternatively, multiple MP4 files may be analyzed, the elementary streams reconstructed, and then converted into a single MP4 file. Multiple media are stored in the MP4 file as multiple tracks. A random access table, program information, control information indicating recording positions, etc. may be generated based on the results of analyzing the MP4 header.

[0128] It is also possible to directly store MMT packets. When storing MMT packets directly, a timestamp indicating the arrival time of the MMT packet is added at the time of storage. The timestamp may be added before the MMT packet header, or may be stored inside the MMT packet header (for example, in the NTP timestamp area). When MMT packets transmitted from multiple transmission paths are rearranged in sequence, a timestamp is added after rearrangement. When there is unnecessary header information in the MMT packet header, MMT payload header, MFU header, etc., the unnecessary header may be deleted before storage. When a header is deleted, information indicating that the header has been deleted is stored. When playing back stored data, the data is played back based on the timestamp. Program information and random access tables may also be generated.

[0129] In addition, the MMT method can store data in a plurality of storage formats (MMT packets, MP4 file format), and may be configured to allow a selection from a plurality of storage methods. Data may be stored using a plurality of storage methods.

[0130] Each storage method has its own characteristics in terms of the amount of stored data, data structure, playback processing amount, etc. The user may be able to select from multiple storage methods. The receiver may also automatically select a storage method depending on the playback purpose. For example, when transferring a stored program to another device, a method of directly storing MMT packets that can transfer the MMT packets as is is selected.

[0131] The transmitting side may transmit signaling information that specifies a storage format and a storage method, and the receiver may store the data in the specified storage format and by the specified storage method.

[0132] (others) Although the method of storing data transmitted using the MMT method has been shown, it is also possible to receive and store signals transmitted by media constituting one program over multiple transmission paths using different multiplexing methods or formats. In this case, the control file containing the program information stores an identifier that can identify the multiplexing method or multiplexing format.

[0133] Furthermore, the random access table indicates randomly accessible points for a certain random access point with an identifier that can be identified by each multiplexing method. For example, if a program is composed of an MP4 file (MPU) packetized using the MMT method and an MP4 file (segment) packetized using the DASH method, a table of MPU sequence numbers and segment sequence numbers for a certain random access point is created. The random access table may be generated in advance on the transmitting side and transmitted via broadcast or communication, or may be created while analyzing the data after reception.

[0134] When transmitting data over multiple transmission paths using different multiplexing methods for each media, the media may be divided at the same time. By transmitting data at the same division time, random access points can be easily created.

[0135] Also, the data may be converted into one of the formats after storage, or the sender may specify the format in which the data is to be stored.

[0136] In each of the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that realizes the transmitting device, receiving device, etc. of each of the above embodiments is a program such as the following.

[0137] In other words, this program causes a computer to execute an accumulation method including a receiving step of receiving an encoded stream including a plurality of files, which are data units of a predetermined file format; a storage step of storing the received plurality of files from the encoded stream in a memory unit; and a generating step of generating first control information used to play the accumulated plurality of files, the first control information associating the plurality of files.

[0138] In addition, this program may cause a computer to execute a playback method for playing a plurality of files, each of which is a data unit in a predetermined file format and is stored in a memory unit, the playback method including a first acquisition step of acquiring control information for playing the plurality of files, the control information associating the plurality of files, an analysis step of analyzing the acquired control information, a second acquisition step of acquiring a predetermined file from the plurality of files from the memory unit based on a result of the analysis step, and a playback step of playing the acquired predetermined file.

[0139] In the above embodiment, the processes executed by a specific processing unit may be executed by another processing unit. The order of multiple processes may be changed, or multiple processes may be executed in parallel.

[0140] In addition, a comprehensive or specific aspect of the present invention may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, etc. In addition, a comprehensive or specific aspect of the present invention may be realized as any combination of a system, a method, an integrated circuit, a computer program, or a recording medium.

[0141] Although the transmission method and the reception method according to one or more aspects of the present invention have been described based on the embodiments, the present invention is not limited to these embodiments. As long as they do not deviate from the spirit of the present invention, various modifications conceived by those skilled in the art to the present embodiments and forms constructed by combining components of different embodiments may also be included within the scope of one or more aspects of the present invention. [Industrial Applicability]

[0142] The present invention is useful as a playback device or the like that can play back stored encoded data. [Explanation of symbols]

[0143] 10 Encoded Stream 11 Signaling Information 12 MMT packets 12a Header 12b Payload 21 Media File A 22 Media File B 31 Control File A 32 Control File B 33 Control File C 100 Storage Device 110 Receiving unit 120 Storage Unit 130 Generation part 140 Storage section 200 Playback device 210 First acquisition part 220 Analysis Department 221 Program Information Analysis Unit 222 Random Access Table Analysis Unit 223 Recording Position Analysis Unit 230 Acquisition Department 240 Playback Department 241 MP4 File Analysis Unit 242 Media Decoding and Presentation Unit

Claims

1. A playback method for playing a plurality of audio files and a plurality of video files, each of which is scalably encoded in data units of a predetermined file format, comprising the steps of: a first acquisition step of acquiring control information for playing the audio files and the video files, the control information associating the audio files and the video files with each other, the control information including information indicating an order in which the audio files and the video files are to be played and information indicating values ​​used to derive playback start times for the audio files and the video files; a reproducing step of sequentially reproducing predetermined files among the plurality of audio files and the plurality of video files based on the control information; A playback method including:

2. A playback device that plays back a plurality of audio files and a plurality of video files, each of which is scalably encoded in data units of a predetermined file format, comprising: a first acquisition unit that acquires control information for playing the audio files and the video files, the control information associating the audio files and the video files, the control information including information indicating an order in which the audio files and the video files are to be played and information indicating a value used to derive a playback start time for each of the audio files and the video files; a playback unit that sequentially plays predetermined files out of the plurality of audio files and the plurality of video files based on the control information; A playback device comprising:

Images