Regeneration method and regeneration device

Abstract

To provide a regeneration method capable of regenerating accumulated data by receiving data with an encoded stream split into multiple files.SOLUTION: A regeneration method is a method that regenerates multiple audio files and multiple video files in which each scalably encoded in a data unit of a given file format. The regeneration method includes: a first acquisition step of acquiring a piece of control information that includes information that is a piece of control information for regenerating multiple audio and video files and is a piece of control information for linking multiple audio and video files, which represent a value used for deriving the playback start time for multiple audio files and multiple video files; and a playback step of sequentially regenerating the specified files among multiple audio files and multiple video files on the basis of the control information.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a playback method and a playback apparatus.

Background Art

[0002] Conventionally, a technique for transmitting encoded data in a predetermined transmission format has been known. The encoded data is generated by encoding content including video data and audio data based on a moving image encoding standard such as HEVC (High Efficiency Video Coding).

[0003] Examples of the predetermined transmission format include MPEG-2 TS (Moving Picture Experts Group-2 Transport Stream) or MMT (MPEG Media Transport) (see Non-Patent Document 1). For example, Patent Document 1 discloses a technique for transmitting encoded media data packet by packet according to MMT.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, a playback apparatus for data that receives and accumulates data in which one stream is divided into a plurality of files, such as the MMT method, has not been considered.

[0006] Therefore, an object of the present invention is to provide a method for playing back data obtained by receiving and storing data in which one encoded stream is divided into a plurality of files, and the like. **Means for Solving the Problems**

[0007] To solve the above problems, a playback method according to an aspect of the present invention is a playback method for playing back a plurality of audio files and a plurality of video files, each of which is scalably encoded in data units of a predetermined file format, the method including: a first acquisition step of acquiring control information for playing back the plurality of audio files and the plurality of video files, the control information being control information for associating the plurality of audio files and the plurality of video files and including information indicating values used for deriving respective playback start times of the plurality of audio files and the plurality of video files; and a playback step of sequentially playing back a predetermined file among the plurality of audio files and the plurality of video files based on the control information.

[0008] Also, a playback apparatus according to an aspect of the present invention is a playback apparatus for playing back a plurality of audio files and a plurality of video files, each of which is scalably encoded in data units of a predetermined file format, the apparatus including: a first acquisition unit that acquires control information for playing back the plurality of audio files and the plurality of video files, the control information being control information for associating the plurality of audio files and the plurality of video files and including information indicating values used for deriving respective playback start times of the plurality of audio files and the plurality of video files; and a playback unit that sequentially plays back a predetermined file among the plurality of audio files and the plurality of video files based on the control information.

[0009] In order to solve the above problems, a playback method according to an aspect of the present invention is a playback method for playing a plurality of audio files and a plurality of video files, each of which is scalable encoded in a data unit of a predetermined file format, the method comprising: a first acquisition step of acquiring control information for playing the plurality of audio files and the plurality of video files, the control information being control information for associating the plurality of audio files and the plurality of video files, the control information including information indicating an order in which the plurality of audio files and the plurality of video files are to be played, and information indicating values used for deriving respective playback start times of the plurality of audio files and the plurality of video files; and a playback step of sequentially playing a predetermined file among the plurality of audio files and the plurality of video files based on the control information.

[0010] Further, a playback device according to an aspect of the present invention is a playback device for playing a plurality of audio files and a plurality of video files, each of which is scalable encoded in a data unit of a predetermined file format, the device comprising: a first acquisition unit that acquires control information for playing the plurality of audio files and the plurality of video files, the control information being control information for associating the plurality of audio files and the plurality of video files, the control information including information indicating an order in which the plurality of audio files and the plurality of video files are to be played, and information indicating values used for deriving respective playback start times of the plurality of audio files and the plurality of video files; and a playback unit that sequentially plays a predetermined file among the plurality of audio files and the plurality of video files based on the control information.

[0011] In order to solve the above problems, a playback method according to an aspect of the present invention is a method for playing back files that are scalably encoded in data units of a predetermined file format, and that play back a plurality of said files, the method comprising: a first acquisition step of acquiring control information for playing back the plurality of files, the control information being control information for associating the plurality of files and including information indicating the order in which the plurality of files are to be played back and information indicating values used to derive the playback start time of each of the plurality of files; and a playback step of sequentially playing back a predetermined file among the plurality of files based on the control information.

[0012] Also, a playback apparatus according to an aspect of the present invention is a playback apparatus for playing back files that are scalably encoded in data units of a predetermined file format, and that play back a plurality of said files, the apparatus comprising: a first acquisition unit that acquires control information for playing back the plurality of files, the control information being control information for associating the plurality of files and including information indicating the order in which the plurality of files are to be played back and information indicating values used to derive the playback start time of each of the plurality of files; and a playback unit that sequentially plays back a predetermined file among the plurality of files based on the control information.

[0013] In order to solve the above problems, a playback apparatus according to an aspect of the present invention includes: a first reception unit that receives, from a content transmission apparatus, control information stored in the content transmission apparatus in order to play back a plurality of segments corresponding to data units defined in the MPEG-DASH standard and including a plurality of video segments and a plurality of audio segments; an extraction unit that extracts, from the control information, a playback start timing in which a first video segment among the plurality of video segments is associated with a first audio segment among the plurality of audio segments; a second reception unit that receives the first video segment and the first audio segment from the content transmission apparatus; and a playback unit that simultaneously plays back the first video segment and the first audio segment based on the playback start timing, wherein the control information has a flag indicating whether the control information is updatable.

[0014] Also, a content transmission device according to an aspect of the present invention includes an acquisition unit that acquires an encoded stream having a plurality of segments corresponding to data units defined in the MPEG-DASH standard and including a plurality of video segments and a plurality of audio segments, a storage that stores the plurality of segments, and a transmission unit. The storage stores control information including a playback start timing in which a first video segment among the plurality of video segments is associated with a first audio segment among the plurality of audio segments. The storage stores a flag indicating whether the control information is updatable and included in the control information. The transmission unit transmits the control information to the playback device so that the playback device simultaneously plays back the first video segment and the first audio segment based on the playback start timing. The transmission unit transmits the first video segment and the first audio segment to the playback device.

[0015] To solve the above problems, a playback method according to an aspect of the present invention receives control information stored in a content transmission device from the content transmission device in order to play back a plurality of segments corresponding to data units defined in the MPEG-DASH standard and including a plurality of video segments and a plurality of audio segments, extracts a playback start timing in which a first video segment among the plurality of video segments is associated with a first audio segment among the plurality of audio segments from the control information, receives the first video segment and the first audio segment from the content transmission device, simultaneously plays back the first video segment and the first audio segment based on the playback start timing, and the control information has update information indicating whether it is updated.

[0016] Also, a content transmission method according to another aspect of the present invention acquires an encoded stream having a plurality of segments corresponding to data units defined in the MPEG-DASH standard and including a plurality of video segments and a plurality of audio segments, stores the plurality of segments in a storage, stores control information including a playback start timing in which a first video segment among the plurality of video segments is associated with a first audio segment among the plurality of audio segments in the storage, stores in the storage information indicating whether the control information is updated and update information included in the control information, transmits the control information to a playback device so that the playback device simultaneously plays back the first video segment and the first audio segment based on the playback start timing, and transmits the first video segment and the first audio segment to the playback device.

[0017] To solve the above problems, an accumulation method according to one aspect of the present invention includes a receiving step of receiving an encoded stream including a plurality of files that are scalably encoded in data units of a predetermined file format, an accumulating step of accumulating the received plurality of files in a storage unit, and a generating step of generating control information for use in playing back the accumulated plurality of files, the control information associating the plurality of files, wherein the plurality of files are played back in order, and the control information includes information indicating the order in which the plurality of files are played back and information indicating values used for deriving the playback start time of each of the plurality of files.

[0018] Also, a playback method according to another aspect of the present invention is a playback method for a file that is scalable-encoded in data units of a predetermined file format and that plays a plurality of said files, the method including: a first acquisition step of acquiring control information for playing the plurality of files, the control information being control information for associating the plurality of files and including information indicating an order in which the plurality of files are to be played and information indicating values used to derive a playback start time for each of the plurality of files; an analysis step of analyzing the acquired control information; a second acquisition step of sequentially acquiring a predetermined file among the plurality of files based on the result of the analysis step; and a playback step of playing the plurality of files in order by sequentially playing the acquired predetermined file.

[0019] To solve the above problems, an accumulation method according to one aspect of the present invention includes: a reception step of receiving an encoded stream including a plurality of files that are data units of a predetermined file format; an accumulation step of accumulating the received plurality of files in a storage unit; and a generation step of generating first control information for use in playing the accumulated plurality of files, the first control information being control information for associating the plurality of files.

[0020] Note that these general or specific aspects may be implemented by a playback method, an accumulation device, a playback device, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be implemented by any combination of an accumulation method, a playback method, an accumulation device, a playback device, an integrated circuit, a computer program, and a recording medium.

Effects of the Invention

[0021] The playback method and the like of the present invention can play back accumulated encoded data.

Brief Description of the Drawings

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0023] (Knowledge Underlying the Present Invention) The present invention relates to an accumulation method or accumulation device for receiving data transmitted using the MMT method being standardized by MPEG and accumulating the received data, or a playback method or playback device for playing back the accumulated data. The MMT method is a multiplexing method for multiplexing video and audio while packetizing and transmitting them over one or more transmission paths such as broadcasting and communication.

[0024] In a conventional system using the MPEG-2 TS method as a multiplexing method, there is a method of directly accumulating TS packets. However, the method of receiving and accumulating data packetized in the MMT method or the method of playing back the accumulated data has not been sufficiently considered.

[0025] The MPU (Media Processing Unit) generated by the MMT method is data that divides video, audio, etc. into appropriate sizes for each medium and stores them in the MP4 format.

[0026] Conventional MP4 files store tracks such as video and audio in one MP4 file and store information indicating the relationship of the tracks in the header inside the MP4. In contrast, the MPU is an MP4 file in which the program is segmented for each medium, and the MPU itself does not have information indicating the relationship between media. For this reason, related video and audio cannot be played back together.

[0027] Also, when accumulating MPUs as a plurality of MP4 files, there are problems that the number of files increases, file management and file access during playback become complicated, and the processing amount related to the playback process increases.

[0028] In order to solve such problems, an accumulation method according to an aspect of the present invention includes a receiving step of receiving an encoded stream including a plurality of files that are data units of a predetermined file format, an accumulation step of accumulating the plurality of received files in the encoded stream in a storage unit, and a generation step of generating first control information used for playing back the plurality of accumulated files, the first control information associating the plurality of files.

[0029] According to this, even data accumulated as a plurality of files can be accumulated by associating the plurality of files, so that the processing amount related to the playback process can be reduced.

[0030] For example, the encoded stream may include second control information, and in the generating step, by analyzing the second information, information indicating the order in which the plurality of files are to be played back may be generated as the first control information.

[0031] For example, the plurality of files may include a first file belonging to a first medium and a second file belonging to a second medium, and in the generating step, by further analyzing the second control information, information associating the first file and the second file may be generated as the first control information.

[0032] For example, each of the plurality of files is a file of a random access unit that is randomly accessible, and in the generating step, by analyzing the second control information, time information indicating the playback start time of each of the plurality of files may be generated as the first control information.

[0033] For example, in the generating step, information associating the first file and the second file, which have the same playback start time, may be generated as the first control information.

[0034] For example, in the generating step, further, information indicating the recording positions of the plurality of files in the storage unit when the plurality of files are stored in the storage unit in the storing step may be generated as the first control information.

[0035] Also, a playback method for playing back a plurality of files that are files in data units of a predetermined file format and are stored in a storage unit, the playback method including: a first acquisition step of acquiring control information for associating the plurality of files, which is control information for playing back the plurality of files; an analysis step of analyzing the acquired control information; a second acquisition step of acquiring a predetermined file from the plurality of files from the storage unit according to the result of the analysis step; and a playback step of playing back the acquired predetermined file.

[0036] Note that these general or specific aspects may be implemented in a storage device, a playback device, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be implemented by any combination of a storage device, a playback device, an integrated circuit, a computer program, or a recording medium.

[0037] Hereinafter, a transmission method and a reception method according to an aspect of the present invention will be specifically described with reference to the drawings.

[0038] Note that all of the embodiments described below are specific examples of the present invention. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps, order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present invention. In addition, among the components in the following embodiments, the components not described in the independent claims indicating the most general concept are described as optional components.

[0039] (Embodiment) Hereinafter, a storage method (storage device) and a playback method (playback device) according to the embodiment will be described with reference to the drawings. In the embodiment, as an example, a storage method of receiving encoded data according to MMT and storing the received encoded data, and a playback method of playing back the stored encoded data will be described.

[0040] First, the data structure of the encoded stream in MMT will be described. FIGS. 1 and 2 are diagrams for explaining the data structure of the encoded stream in MMT.

[0041] As shown in FIG. 1, the encoded data consists of a plurality of access units (AUs). The encoded data is, for example, AV data encoded based on a video encoding standard such as HEVC. Specifically, the encoded data includes video data, audio data, and accompanying subtitles, text super, metadata, still images, and files. When the encoded data is video data, one AU corresponds to one picture (one frame).

[0042] In MMT, the encoded data is MP4-formatted (an MP4 header is added) according to the MP4 file format (ISO base media data format) in units of GOP (Group Of Picture). That is, the encoded data has a plurality of sets (GOPs), each of which is composed of a plurality of access units. A GOP is a random access point in the encoded data, and the first access unit in decoding order in the GOP corresponds to an IDR picture of HEVC or AVC, or a non-IDR I picture.

[0043] The MP4 header included in the MP4 data describes the relative values of the presentation time (PTS as described above) and decoding time (DTS as described above) of the access unit. Also, the MP4 header describes the sequence number of the MP4 data. Note that the MP4 data (MP4 file) is an example of an MPU, which is a data unit defined in the MMT standard. In the MPU, only the sample data in the MPU may be transmitted without transmitting the MP4 header. Even in this case, the MPU corresponds to a random access unit, and the samples constituting the MPU are associated with the MPU. Also, at the head of the MPU, a random access point, which is the first access unit of the random access unit, is arranged. Also, the MPU may be composed of a plurality of GOPs. Also, the MPU is a set of access units of a single media such as video or audio, and is encapsulated in the form of an MP4 file for each media. Here, the "media" refers to the type of content such as video, audio, and subtitles.

[0044] As shown in FIG. 2, the encoded stream 10 in MMT includes signaling information 11 and a plurality of MMT packets 12. In other words, the encoded stream 10 is a packet sequence of MMT packets 12.

[0045] The encoded stream 10 (MMT stream) is one of one or more streams that constitute one MMT package. The MMT package corresponds to, for example, one broadcast program content.

[0046] The signaling information 11 includes, for example, information indicating the correspondence between a plurality of assets and packet IDs. Note that an asset is a data entity including data with the same transport characteristics, and is, for example, any one of video data and audio data.

[0047] Specifically, the signaling information 11 is, for example, CI (Composition Information) and MPT (MMT Package Table) in MMT. Note that the signaling information 11 is PMT (Program Map Table) or PAT (Program Association Table) in MPEG2-TS, and MPD (Media Presentation Description) in MPEG-DASH, etc.

[0048] Also, the signaling information 11 may include time information. The time information is information for determining the PTS or DTS of an access unit. Specifically, the time information is, for example, the PTS or DTS as the absolute time of the first access unit in the MPU. More specifically, in the case of PTS, the absolute value of the PTS of the first access unit in the presentation order in the MPU can be indicated, and in the case of DTS, the absolute value of the DTS of the first access unit in the decoding order in the MPU can be indicated. Also, the time information may not be stored in the signaling information 11.

[0049] For example, assuming that a plurality of pictures constitute one set, the first access unit to be decoded first in the decoding order of the set is decoded at the time indicated by the DTS. At this time, the time information indicating the time (DTS) based on the reference clock for performing the decoding performed on the first access unit of the set may be stored as the time information of the encoding stream 10.

[0050] Note that the reference clock is NTP (Network Time Protocol) when transmitting the encoding stream in the MMT method, and PCR (Program Clock Reference) when transmitting the encoding stream in the MPEG2-TS method. Here, the NTP may be any reference clock set by the transmitting device, and does not necessarily have to match the NTP value in the NTP server generally used on the Internet.

[0051] The MMT packet 12 is data in which MP4 data is packetized. In the embodiment, one MMT packet 12 contains one MP4 data (MPU). As shown in FIG. 2, the MMT packet 12 includes a header 12a (corresponding to an MTT packet header (TS packet header in the case of MPEG2-TS)) and a payload 12b.

[0052] The MP4 data is stored in the payload 12b. Note that the payload 12b may store the split MP4.

[0053] The header 12a is accessory information regarding the payload 12b. For example, the header 12a includes a packet ID and time information. The time information here is the relative value of the presentation time (PTS) or the decoding time (DTS) of the MP4 data.

[0054] The packet ID is an identification number indicating the asset of the data included in the MMT packet 12 (payload 12b). The packet ID is a unique identification number for each asset constituting the MMT package.

[0055] Thus, the encoded stream includes time information (DTS or PTS) indicating the time to perform the process of decoding or presenting the encoded data, and the encoded data (ID1_#0, ID2_#0, ID1_#1, ID1_#2, ID2_#1, ID2_#2, ··· in FIG. 2).

[0056] [Storage method] FIG. 3 is a block diagram showing the configuration of the storage device according to the embodiment. FIG. 4 is a flowchart of the storage method according to the embodiment.

[0057] As shown in FIG. 3, the storage device 100 includes a receiving unit 110, a storage unit 120, a generating unit 130, and a storage unit 140. Note that the components of the storage device 100 are specifically realized by a microcomputer, a processor, a dedicated circuit, or the like. Note that the storage unit 140 may not be included in the storage device 100 and may be a device separate from the storage device 100.

[0058] In the storage method according to the embodiment, as shown in FIG. 4, the receiving unit 110 of the storage device 100 receives an encoded stream including a plurality of files that are data units in a predetermined file format (S11: receiving step).

[0059] Next, the storage unit 120 of the storage device stores the plurality of received files in the encoded stream in the storage unit 140 (S12: storage step). Specifically, a plurality of data units in the MP4 file format (that is, MP4 files) are acquired from the received packets constituting the received encoded stream 10.

[0060] Next, the generating unit 130 of the storage device generates first control information for associating a plurality of MP4 files, which is control information used to reproduce the stored plurality of MP4 files (S13: generating step). The first control information generated in step S13 includes the following three types of control information.

[0061] The three types of control information are information indicating the recording positions of a plurality of MP4 files in the storage unit 140 (control file A31 described later), program information (control file B32 described later), and information indicating a random access table (control file C33). That is, in step S13, three steps are performed: a step of generating control file A31, a step of generating control file B32, and a step of generating control file C33.

[0062] That is, for example, in the generation step, by analyzing the signaling information (second control information), information indicating the order in which a plurality of MP4 files are to be played back may be generated as the first control information.

[0063] Also, for example, in the generation step, by analyzing the signaling information, information associating a first file whose media is video and a second file whose media is audio may be generated as the first control information. More specifically, each of the plurality of MP4 files is a file of a random access unit that can be randomly accessed. In the generation step, by analyzing the signaling information, time information indicating the playback start time of each of the plurality of MP4 files may be generated as the first control information (control file B32). Also, in the generation step, information associating the first file and the second file having the same playback start time may be generated as the first control information (control file C33).

[0064] Also, in the generation step, further, information indicating the recording positions of the plurality of MP4 files in the storage unit 140 when the plurality of MP4 files are stored in the storage unit 140 in the accumulation step may be generated as the first control information (control file A31).

[0065] Then, the control information generated in step S13 is stored in the storage unit 140 together with the plurality of MP4 files, and when the reception of the encoded stream for a predetermined time ends, the accumulation ends.

[0066] Note that the first control information and the second control information are different control information. The first control information is control information used to play a plurality of stored MP4 files. The second control information is control information used when playing a received encoded stream in real time. That is, the first control information is, for example, control information used when recording (storing) broadcast content and playing the recorded content. Also, the second control information is, for example, control information used when playing content received in real time by broadcast or communication.

[0067] Next, the details of each step of the storage method will be specifically described with reference to the figures.

[0068] FIG. 5 is a diagram showing data obtained by receiving data encoded using the MMT method and acquiring an MPU for each media from the MMT packet. FIG. 6 is a diagram showing the configuration of the media file A21 when the file of media A (video) is stored to constitute the media file A21.

[0069] Among the received MMT packets 12 in the encoded stream 10, the MMT packets 12 are filtered by the packet ID. After removing the headers 12a such as the MMT packet header and the payload header from the filtered MMT packets 12, they become a set of a plurality of MP4 files for each media. In the header of the MP4 file, an identifier for each media, a sequence number of MP4 files constituting the same media, and the like are described.

[0070] In addition to the MP4 files, a plurality of types of signaling information are received. The signaling information includes, for example, a table indicating the relationship between media, the playback time of the access unit at the head of the MP4, information related to the startup of the application, and the like. The signaling information may be received only once or may be received repeatedly one or more times.

[0071] FIG. 7 is a diagram for explaining a specific example of accumulating a file and generating control information based on the received file and second control information.

[0072] In the storage method of the present embodiment, the acquired MP4 file is directly stored in the file for each medium. At the same time, control information indicating the relationship between media and other information necessary for playing back programs are generated as control files A31 to C33 (first control information). When playing back a program, while referring to the control information, the MP4 of the file for each medium is analyzed to play back video and audio.

[0073] First, an operation of transmitting one program through a single transmission path using the MMT method and receiving and storing it in a receiver will be described as an example. The program is composed of media A and media B, and it is assumed that information indicating the relationship between media (information indicating the relationship between media) constituting the program is stored in signaling information.

[0074] The MP4 file acquired in the receiving step is stored in the storage unit 140 by storing it in the media file for each medium. For example, M MP4 files of media A with sequence numbers from #1 to #M are sequentially stored in the media file A21, and N MP4 files of media B with sequence numbers from #1 to #N are sequentially stored in the media file B22 for media B.

[0075] The media file may be set with a predetermined file name that can identify the media. Further, the media file may store an identifier for identifying the type of media of the file stored in the media file in the media file. As the identifier for identifying the type of media, for example, the asset ID assigned to each of a plurality of media by the MMT method may be used, or an identifier corresponding to the asset ID may be used. When storing an identifier in the media file, a new mp4 box may be provided and stored therein.

[0076] In the generation step, when storing an MP4 file in a media file, information for specifying the recorded position for each MP4 is generated and stored in a control file (control file A31 in FIG. 7). The position specifying information for specifying the recorded position for each MP4 file is indicated, for example, by an offset amount based on the number of bytes from the head of the media file to the head byte of the MP4 file of each sequence number with respect to the media identifier. When using the MMT method, the position specifying information indicates the recording position of the MP4 file with respect to the asset ID.

[0077] In the reception step, signaling information in which information on the media constituting the program is described is received, and the acquired signaling information is stored in a control file (for example, control file B32 in FIG. 2). The signaling information stores the file name of the media file that stores the MP4 files of the media constituting the program. The signaling information includes signaling information that is transmitted only once, signaling information that is repeatedly transmitted, or signaling information that is transmitted in a divided manner. In the generation step, in any case, information necessary for reproducing the program is acquired (generated) from the acquired signaling information and stored in a file.

[0078] In the generation step, at the time of reproduction, a random access table may be created so that random access is possible. The random access table is a list of random accessable points, and indicates, for example, the respective MP4 sequence numbers of media A and media B corresponding to the random access points.

[0079] FIG. 8 is a diagram showing an example of a random access table.

[0080] When media A is video and media B is audio, the random access table is a table in which the MP4 sequence number of the MP4 file of the video is associated with the MP4 sequence number of the MP4 file of the audio that is accessed (played back) at the same time as the random access point of the MP4 file of the video.

[0081] The random access table is specifically generated by the following processing procedure. First, for the random access points of the MP4 file of the video, when the time information of the access unit at the beginning of the MP4 file of the video is transmitted as signaling information, the random access points are generated based on the time information. Then, the MP4 sequence number of the audio MP4 file with the same time information as the time of the generated random access points of the video MP4 file is searched, and the random access table is generated by associating the MP4 sequence number of the searched audio MP4 file with the MP4 sequence number of the video MP4 file used in the search. That is, in the generation step, the generated random access table is stored in a control file (for example, the control file C33 in FIG. 7).

[0082] Note that the random access table does not necessarily need to be a list of all random accessable points, and may be composed of random access points at regular intervals (that is, some randomly selected access points thinned out at regular intervals).

[0083] In addition, the random access table may further be associated with time information corresponding to each random access point. Note that in this case, the time information is the relative time from the start time of the media (the start time of program playback). More specifically, when the time information of the access unit at the beginning of the MP4 file is a timestamp given based on NTP or STC, the time information is the relative time of the MP4 file calculated from the start time of the media.

[0084] In addition, when starting accumulation from the middle of the program, the relative time corresponding to each of the plurality of MP4 files is calculated based on the time of the data at the start of accumulation, and the relative time is stored in the random access table. Also, this time information may be generated based on the timestamp described in the MP4 file header.

[0085] When layout information and application information are signaled, the signaling information may be stored in a file.

[0086] When a timestamp is attached to the signaling information, it may be converted to a relative time from the start time of the media and stored together with the signaling information.

[0087] In the generation step, event information such as layout information and application startup information may be stored in program information and a random access table. In this case, it may be presented based on timestamp information during playback.

[0088] Even if the time interval of the audio MP4 file does not match the time interval (GOP unit) of the video MP4 file, the MP4 sequence number of the audio with respect to the random access point of the video is searched to generate a random access table, so that during playback, the video MP4 file and the audio MP4 file that should be played simultaneously with the video MP4 file can be easily accessed randomly.

[0089] When there are a plurality of random access points in the video MP4 file, the number of random access points included in the MP4 file may be stored in the header information and transmitted. In this case, the random access table has each of the plurality of random access points of the video MP4 file associated with the MP4 sequence number of the audio MP4 file. And the audio MP4 file is further associated with information indicating which random access point among the plurality of random access points of the video MP4 file it is in the random access table.

[0090] [Playback method] Next, a playback method for playing a plurality of MP4 files stored in the storage unit 140 will be described.

[0091] FIG. 9 is a block diagram showing the configuration of a playback device according to an embodiment. FIG. 10 is a flowchart of a playback method according to an embodiment.

[0092] As shown in FIG. 9, the playback device 200 includes a first acquisition unit 210, an analysis unit 220, a second acquisition unit 230, and a playback unit 240. Note that the components of the playback device 200 are specifically realized by a microcomputer, a processor, a dedicated circuit, or the like.

[0093] First, in the playback device 200, it is determined whether the file to be played back is a file (media file) including a plurality of MP4 files or a single MP4 file (S21).

[0094] When it is determined that the target file is a media file (S21: media file), the first acquisition unit 210 of the playback device 200 acquires first control information for playing back a plurality of MP4 files and associating the plurality of MP4 files (S22: first acquisition step). Specifically, the first acquisition unit 210 acquires the first control information stored in the storage unit 140.

[0095] Next, the analysis unit 220 of the playback device 200 analyzes the first control information acquired in step S22 in steps S23 to S25 following step S22 (analysis step).

[0096] More specifically, the analysis unit 220 includes a program information analysis unit 221, a random access table analysis unit 222, and a recording position analysis unit 223, and these analysis units 221 to 223 perform the processes of steps S23 to S25, respectively.

[0097] The program information analysis unit 221 analyzes the control file B32 in the first control information and identifies the media files constituting the program (S23).

[0098] Next, the random access table analysis unit 222 analyzes the control file C33 among the first control information, and acquires the MP4 sequence number of the random access point for each medium (S24).

[0099] Next, the recording position analysis unit 223 analyzes the control file A31 among the first control information, and acquires position information indicating the recording position in the storage unit 140 of the MP4 file, which is associated with the MP4 sequence number of the MP4 file acquired in step S24 (S25).

[0100] Next, the second acquisition unit 230 acquires a predetermined MP4 file from the storage unit 140 based on the result of the analysis step (position information of the MP4 file) (S26: second acquisition step).

[0101] Next, the playback unit 240 of the playback device 200 plays back the acquired MP4 file (S27: playback step). More specifically, the playback unit 240 includes an MP4 file analysis unit 241 and a media decoding / presentation unit 242. In step S27, first, the MP4 file analysis unit 241 analyzes a plurality of MP4 files in order for each medium. Then, the media decoding / presentation unit 242 decodes and presents the plurality of MP4 files based on the time stamp information analyzed by the MP4 file analysis unit 241, thereby performing playback.

[0102] On the other hand, when it is determined that the target file is one MP4 file (S21: MP4 file), the MP4 file is analyzed, and based on the time stamp information of the MP4, the MP4 file is played back by decoding and presenting it (S28). Note that step S21 is not necessarily required. Also, step S28 is not necessarily required.

[0103] (Feature) According to the storage method and playback method according to the present embodiment, a program encoded in the MMT format can be stored and played back.

[0104] Also, according to the storage method according to the present embodiment, a plurality of received MP4 files can be collectively stored in a file without changing the configuration of the MP4 files.

[0105] Also, according to the storage method according to the present embodiment, by generating and storing control information such as program information, it becomes possible to play a plurality of stored MP4 files.

[0106] Also, according to the storage method according to the present embodiment, by generating a random access table, random access playback becomes possible easily.

[0107] Also, according to the storage method according to the present embodiment, layout information, application start information, etc. can be presented at the time of playback.

[0108] (Modification example) The storage method described above is an example and is not limited thereto.

[0109] The first control information may be stored in a plurality of files for each function, or the control information of two or more functions may be collectively stored in one file. Also, the control information related to a single medium may be stored together with the medium in the media file.

[0110] Also, the media file and the control file may be made identifiable by a predetermined file name. Also, the media file and the control file may have an identifier for identifying the content of the media or the control information stored in the file stored in the file.

[0111] Also, in the playback method, although the control file in which the program information is stored is used as the entry point, a separate entry point file may be defined, and the program information or the control file in which the program information is stored may be specified in the entry point file.

[0112] Also, although it was explained that the offset amount indicating the recording position of the MP4 file is expressed in bytes, other methods may be used. The offset amount may, for example, define a predetermined data unit and indicate the index number of the data unit. Also, the recording position of the MP4 file may be the offset amount from the first byte of the media file to the first byte of the MP4 file. Also, the recording position of the MP4 file may be the difference information of the recording positions from the first bytes of the MP4 files before and after.

[0113] Note that although the data encoded in the MMT format was used as an example, the method of this embodiment can also be used for accumulation and playback when accumulating a program composed of a plurality of segmented MP4 files such as MPEG-DASH. Also, when an MP4 file encoded in the MMT format or the MPEG-DASH format is transmitted in the MPU mode or the GFD mode of the MMT protocol, or in another protocol such as FLUTE, the method of this embodiment can be used for accumulation and playback. In addition, after accumulating a plurality of MP4 files in the order of reception in the media file, the order of the data (MP4 files) in the media file may be changed. When the data in the media file is rearranged, the first control information and the MP4 file header are appropriately corrected along with the rearrangement, and information indicating the rearrangement is stored in the media file or in a control file separate from the media file. Also, it is not necessary to rearrange the data in the media file after accumulation, and the data may be temporarily stored in memory when receiving a plurality of MP4 files and accumulated while rearranging.

[0114] Also, after accumulating a plurality of MP4 files in the media file in the order of reception, the order of the data (MP4 files) in the media file may be changed. When the data in the media file is rearranged, the first control information and the MP4 file header are appropriately corrected along with the rearrangement, and information indicating the rearrangement is stored in the media file or in a control file separate from the media file. Also, it is not necessary to rearrange the data in the media file after accumulation, and the data may be temporarily stored in memory when receiving a plurality of MP4 files and accumulated while rearranging.

[0115] For example, it may be rearranged such that the header parts of a plurality of MP4 files and the first half of the files are gathered, and the data parts of the plurality of MP4 files are gathered in the second half of the files. Further, as shown in FIG. 11, when there is header information common to a plurality of MP4 files or header information unnecessary for storage / playback (e.g., 'ftyp','mmpu' box, etc.), the overlapping header information may be deleted. Note that FIG. 11 is a diagram for explaining the data in the media file when the data in the media file is rearranged. FIG. 11(a) is a diagram showing the configuration of the data in the media file before rearranging the data. FIG. 11(b) is a diagram showing the configuration of the data in the media file after rearranging the data.

[0116] Also, for example, when the initialization information is different for each MP4 file, modification of the header is required. That is, when comparing the initialization information of the Nth MP4 file with the initialization information of the first MP4 file and the initialization information of the Nth MP4 file is different from the initialization information of the first MP4 file, the initialization information of the Nth MP4 file is added to the initialization information of the first MP4 file, and when processing the Nth MP4 file, the reference destination in the'moof' of the Nth MP4 file may be changed so that the added initialization information can be referred to with respect to the initialization information of the first MP4 file.

[0117] Also, a plurality of media MP4 files may be stored in one media file. For example, a configuration may be adopted in which an MP4 file of video and an MP4 file of audio are alternately stored in units of random access.

[0118] Also, when the playback time is long, the media file may be divided into a plurality of files and stored. In that case, the divided files of the audio file may be divided in accordance with the division time interval of the video file.

[0119] Also, when receiving and storing an MP4 file transmitted using a plurality of transmission paths, it may be stored without synchronizing the delay times between the transmission paths, and synchronization processing such as rearranging the stored MP4 file based on the MP4 sequence number may be performed.

[0120] Also, a plurality of MP4 files may be stored in advance in the transmission server in the formats of the media files and control files shown in this embodiment, and the receiver (storage device 100) may download and store the files from the transmission server via a broadcast transmission path or a communication transmission path.

[0121] Also, the MP4 files obtained by downloading from a broadcast transmission path or a communication transmission path and the MP4 files obtained by streaming may be stored respectively, and the MP4 files may be rearranged after storage.

[0122] Also, when storing an MP4 file including scalable-encoded encoded data, even if the media IDs in the enhancement layer and the base layer are different, the MP4 files of each layer may be stored in one media file, or may be stored in separate media files for each layer. When storing the MP4 files of the base layer and the enhancement layer in one media file, the corresponding MP4 files of the base layer and the enhancement layer may be stored alternately. Also, only the media of the base layer may be stored. The control file for describing the program information stores that the media file is scalable-encoded data and an identifier that enables identification of the hierarchical level.

[0123] Also, when storing a plurality of received MP4 files, an identifier or flag indicating the state of the stored media may be provided for the stored media files. For example, an identifier or flag indicating whether the stored media corresponding to the media file is in a complete state without shortage may be provided in the media file or the control file. The flag is stored in the media file or the control file. Whether the media is in a complete state without shortage can be determined, for example, from the sequence number of the media in the identifier of the media. The sequence number of the media is, for example, the MPU sequence number indicating the order of the MPUs in the asset, the sequence number indicating the order of the packets in the asset, the sequence number indicating the order of the MMT packets, or the sequence number of the fragmented payload, and is indicated by the header of the data unit, MMT packet, or MMT payload. Therefore, a flag can be provided according to the result of the determination. Also, an identifier or flag indicating that there is no error in the stored media may be provided. If there is an error in some of the stored media MP4 files, or if there is an MP4 file that cannot be obtained due to packet loss or the like, only the specific MP4 file may be downloaded and stored from the server, or the media may be complemented by downloading each media file.

[0124] Also, an identifier or flag indicating that the stored media is in a state where it can be reordered or that the reordering is complete may be provided. Also, an identifier or flag indicating that the stored media can be played and viewed may be provided.

[0125] For example, it may be determined that the media is in a state where it can be reordered when the media is in a state without errors or shortages. Or, for example, in the case of a program that requires reordering, it may be set that the media can be played and viewed only when the reordering is complete.

[0126] In addition, the content producer may transmit signaling information regarding the accumulation of programs, sorting, conditions for playback viewing, and permitted accumulation formats, etc., and use it in combination with identifiers, flags, etc. indicating the accumulation state, to specify or restrict the operation of the accumulated content.

[0127] In addition, the content producer may pre-assign a unique ID that can uniquely specify a program or media, and the receiving side may use the unique ID together with the program or media. For example, by specifying a unique ID from data broadcasts, web content, applications, etc., it becomes possible to play back the accumulated program.

[0128] Also, for example, when composing a program from media A transmitted by broadcast and media B that is stored, the program information indicates the storage device as the transmission path information (location information) of media B, and specifies the unique ID assigned as the media ID. When the storage device is indicated as the location information in the receiver, the media-specific ID is searched from the program information in the pre-registered storage device, and it is played back in synchronization with media A transmitted by broadcast.

[0129] Also, after analyzing a plurality of MP4 files and reconstructing the elementary stream, it may be converted into one MP4 file. The plurality of media are stored in the MP4 file as a plurality of tracks. Based on the result of analyzing the MP4 header, control information indicating a random access table, program information, recording position, etc. may be generated.

[0130] It is also possible to directly store MMT packets. When directly storing MMT packets, a timestamp indicating the arrival time of the MMT packet is added during storage. The timestamp may be added before the MMT packet header, or stored inside the MMT packet header (for example, in the NTP timestamp area). Also, when arranging MMT packets transmitted from multiple transmission paths in sequence order, the timestamp is added after the arrangement. If there is unnecessary header information in the MMT packet header, MMT payload header, MFU header, etc., it may be stored after deleting the unnecessary header. When deleting the header, information indicating that the header has been deleted is stored. When playing back the stored data, the data is played back based on the timestamp. Also, program information and a random access table may be generated.

[0131] Also, the MMT method can store in a plurality of storage formats (MMT packets, MP4 file format), and may be configured to be selectable from a plurality of storage methods. It may be stored using a plurality of storage methods.

[0132] Each storage method has its own characteristics in terms of the amount of stored data, data configuration, playback processing amount, etc. The selection of the storage method may be made by the user from among a plurality of selection methods, or may be automatically selected by the receiver according to the playback application. For example, when transferring the stored program to another device, select the method of directly storing MMT packets that can transfer the MMT packets as they are.

[0133] Signaling information specifying the storage format and storage method may be transmitted on the transmitting side, and the receiver may store in the specified storage format and storage method.

[0134] (Others) The method of accumulating data transmitted using the MMT method has been shown. However, it is also possible to receive and accumulate signals transmitted through a plurality of transmission paths and in different multiplexing methods or formats by a medium constituting one program. In this case, an identifier capable of specifying the multiplexing method and the multiplex format is stored in a control file including program information.

[0135] In addition, in the random access table, for a certain random access point, random accessable points are indicated by identifiers distinguishable by each multiplexing method. For example, when a program is composed of an MP4 file (MPU) packetized by the MMT method and an MP4 file (segment) packetized using the DASH method, a table of the MPU sequence number and the segment sequence number for a certain random access point is created. The random access table may be generated in advance on the transmission side and transmitted by broadcast or communication, or may be created while analyzing the received data.

[0136] When transmitting through a plurality of transmission paths using different multiplexing methods for each medium, the time for dividing the medium may be the same. By transmitting with the same time for division, random access points can be easily created.

[0137] In addition, format conversion may be performed to any format after accumulation. The transmission side may specify in which format to accumulate.

[0138] In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software for realizing the transmission device, the reception device, etc. of each of the above embodiments is the following program.

[0139] That is, this program causes a computer to execute a storage method including: a reception step of receiving an encoded stream including a plurality of files that are data units in a predetermined file format; a storage step of storing the plurality of received files in a storage unit; and a generation step of generating first control information for use in playing back the plurality of stored files, the first control information associating the plurality of files.

[0140] Further, this program may cause a computer to execute a playback method for playing back a file that is a data unit in a predetermined file format and that is one of the plurality of files stored in the storage unit, the playback method including: a first acquisition step of acquiring control information for playing back the plurality of files, the control information associating the plurality of files; an analysis step of analyzing the acquired control information; a second acquisition step of acquiring a predetermined file from the plurality of files in the storage unit according to the result of the analysis step; and a playback step of playing back the acquired predetermined file.

[0141] In the above embodiment, the processing executed by a specific processing unit may be executed by another processing unit. Also, the order of a plurality of processes may be changed, or a plurality of processes may be executed in parallel.

[0142] Note that the comprehensive or specific aspect of the present invention may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. Further, the comprehensive or specific aspect of the present invention may be realized by any combination of a system, a method, an integrated circuit, a computer program, or a recording medium.

[0143] As described above, the transmission method and the reception method according to one or more aspects of the present invention have been described based on the embodiments. However, the present invention is not limited to these embodiments. Without departing from the spirit of the present invention, various modifications conceived by those skilled in the art to these embodiments, or forms constructed by combining components in different embodiments may also be included within the scope of one or more aspects of the present invention.

Industrial Applicability

[0144] The present invention is useful as a playback device or the like that can play back the stored encoded data.

Explanation of Signs

[0145] 10 Encoded stream 11 Signaling information 12 MMT packet 12a Header 12b Payload 21 Media file A 22 Media file B 31 Control file A 32 Control file B 33 Control file C 100 Storage device 110 Receiver 120 Storage unit 130 Generation unit 140 Memory unit 200 Playback device 210 First acquisition unit 220 Analysis unit 221 Program information analysis unit 222 Random access table analysis unit 223 Recording position analysis unit 230 Acquisition unit 240 Playback unit 241 MP4 file analysis unit 242 Media decoding / presentation unit

Claims

1. A playback method for playing back a plurality of audio files and a plurality of video files, each of which is scalably encoded in data units of a predetermined file format, comprising the steps of: a first acquisition step of acquiring control information for playing the audio files and the video files, the control information associating the audio files and the video files with each other, the control information including information indicating values ​​used to derive playback start times of the audio files and the video files; a reproducing step of sequentially reproducing predetermined files among the plurality of audio files and the plurality of video files based on the control information; A playback method including:

2. A playback device that plays back a plurality of audio files and a plurality of video files, each of which is scalably encoded in data units of a predetermined file format, comprising: a first acquisition unit that acquires control information for playing the audio files and the video files, the control information associating the audio files and the video files with each other, the control information including information indicating a value used to derive a playback start time of each of the audio files and the video files; a playback unit that sequentially plays predetermined files out of the plurality of audio files and the plurality of video files based on the control information; A playback device comprising:

Images