Playback method and playback device

Abstract

To provide a playback method sufficiently applicable to broadcasting.SOLUTION: A playback method includes receiving control packets storing control information. In receiving the control packets, a plurality of pieces of time information corresponding to a plurality of MPUs, respectively, are stored and received, respectively, in control packets transmitted at time before time indicated by the time information. The time information is information indicating presentation time of a leading access unit in presentation order corresponding to a corresponding one of the MPUs. The time information stored in the received control packets is changed in accordance with transmission time of the control packets. A plurality of pieces of time information are stored in the received control packets. Time information corresponding to a first MPU of the plurality of MPUs is stored in a plurality of control packets transmitted at time points differing from each other.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a method for transmitting coded data and a method for reproducing the data. [Background technology]

[0002] The MPEG-2 TS (Moving Picture Experts Group-2 Transport Stream) method is a media transport method that is widely used in current broadcasting systems. However, this method has various limitations when coordinating broadcasting and communications, so the MPEG (Moving Picture Experts Group) is currently standardizing MMT (MPEG Media Transport) as a new media transport method that assumes the use of a variety of networks (see Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Information technology - High efficiency coding and media delivery in heterogeneous environment - Part1:MPEG media transport(MMT), ISO / IEC DIS 23008-1 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the MMT data transmission method in Non-Patent Document 1 has the problem that it cannot be fully applied to broadcasting.

[0005] Therefore, the present invention provides a data transmission method and the like that can be fully applied to broadcasting. [Means for solving the problem]

[0006] A playback method according to one embodiment of the present invention is a playback method implemented by a playback device that plays video data transmitted using the MPU (Media Processing Unit) format in MMT (MPEG Media Transport), wherein one of the MPUs corresponds to a plurality of access units, the method receives a plurality of data packets storing the plurality of access units corresponding to the MPU, receives a control packet storing control information, and plays the plurality of access units according to the control information, and in receiving the control packets, each of a plurality of time information corresponding to each of the plurality of MPUs is stored in the control packet transmitted at a time earlier than the time indicated by the time information and received at least once, the time information is information indicating the presentation time of the first access unit in the presentation order corresponding to the corresponding MPU, the time information stored in the received control packet is changed according to the transmission time of the control packet, the received control packet stores a plurality of time information, and the time information corresponding to a first MPU of the plurality of MPUs is stored in a plurality of control packets transmitted at different times.

[0007] A data transmission method according to one embodiment of the present invention is a data transmission method performed by a transmitting device, which generates a plurality of coded data units, each consisting of a plurality of samples, reference time information indicating a reference time, and start time information indicating the start time, which is the time at which the first sample in the coded data unit is presented or decoded, and executes a transmission process to transmit the generated plurality of coded data units, the reference time information, and the start time information, wherein in the transmission process, the generated plurality of coded data units are transmitted in packets different from the generated reference time information and the start time information, the multiple coded data units are transmitted sequentially, the start time information indicates the start time of each of a plurality of specific coded data units, which are multiple coded data units among the generated plurality of coded data units, whose presentation or decoding begins after the start time information is transmitted, and each of the generated plurality of coded data units indicates the time at which each sample other than the first sample in the coded data unit is presented or decoded as a relative value to the time of the other samples in the coded data unit.

[0008] A transmission method according to one embodiment of the present invention is a playback method implemented by a playback device that plays video data transmitted using the MPU (Media Processing Unit) format in MMT (MPEG Media Transport), wherein one MPU corresponds to a plurality of access units, receives a plurality of data packets storing the plurality of access units corresponding to the MPU, periodically receives control packets storing control information, and plays the plurality of access units according to the control information, and in receiving the control packets, each of a plurality of time information corresponding to the MPU is stored in the control packet transmitted at a time earlier than the time indicated by the time information and received at least once, the time information being information indicating the decoding time of the first access unit in decoding order contained in the corresponding MPU, the time information stored in each of the periodically received control packets is changed according to the transmission time of the control packet, and the time information is stored in the periodically received control packets.

[0009] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0010] The data transmission method of the present invention is well suited for broadcasting. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram for explaining the problems that can be expected with the data transmission method. [Figure 2] FIG. 2 is a block diagram of the data transmission device according to this embodiment. [Figure 3] FIG. 3 is a flowchart showing the operation of the data transmitting device according to this embodiment. [Figure 4] FIG. 4 is an explanatory diagram for explaining the data transmission method according to this embodiment. [Figure 5] FIG. 5 is a block diagram of the leading time information generating unit in this embodiment. [Figure 6] FIG. 6 is a flowchart showing the operation of the head time information generating unit in this embodiment. [Figure 7] FIG. 7 is a block diagram of the data reproduction device according to the present embodiment. [Figure 8] FIG. 8 is a flowchart showing the operation of the data reproducing device in this embodiment. [Figure 9] FIG. 9 is a block diagram of the first PTS determination unit in this embodiment. [Figure 10] FIG. 10 is a flowchart showing the operation of the leading PTS determination unit in this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] (Findings that form the basis of the present invention) The present inventors have found that the data transmission method of Non-Patent Document 1 described in the "Background Art" section has the following problems.

[0013] In MMT, a media transmission format currently being standardized by MPEG, MPUs (Media Processing Units), which are coded data units of coded content, are configured in a format that cannot indicate the presentation time stamp (PTS) or decoding time stamp (DTS) of the MPU. Therefore, a data playback device cannot determine when to play the MPU simply by acquiring the MPU.

[0014] Therefore, it is assumed that the data transmission device transmits the start time information of the MPU to the data reproduction device. The start time information indicates, for example, the time (start PTS) at which presentation (or display) of the MPU begins. Note that this start PTS is indicated in the same format as absolute time. In other words, when a data reproduction device receives and reproduces MMT data, it requests an MPU already stored in a server (data transmission device) or the like, along with the start time information, before reproducing the data. Then, upon receiving the request, the data transmission device transmits the MPU to the data reproduction device along with the start time information. As a result, the data reproduction device reproduces the MPU so that presentation of the MPU begins at the time of the start PTS indicated in the start time information.

[0015] However, in cases where MMT data is generated and transmitted sequentially, and the transmitted data is received and played back sequentially, there is a problem in that it is not possible to obtain an appropriate starting PTS from the starting time information. This problem will be explained with reference to Figure 1.

[0016] FIG. 1 is an explanatory diagram for explaining the problems with the data transmission method that are assumed from the above-mentioned Non-Patent Document 1 and the like.

[0017] The data transmitting device transmits an asset, which is a data sequence consisting of multiple MPUs (MPU#1, MPU#2, MPU#3, ...) in MMT, as shown in Figure 1. At time T_get, the first PTSs of MPU#1 and MPU#2 are determined.

[0018] Here, the data playback device acquires the start time information at time T_get, and receives data from each MPU starting with MPU #3, which starts processing first after time T_get. In this case, the start time information indicates only the start PTSs (PTSab_1, PTSab_2) of MPU #1 and MPU #2, but not the start PTS (PTSab_3) of MPU #3. Therefore, the data playback device cannot acquire the start PTS of MPU #3.

[0019] Thus, when a data playback device starts receiving and playing an MPU distributed via broadcasting or a communication network in response to a user's request, it is unable to properly acquire the first PTS of the received MPU. In other words, it is unable to determine the start time for decoding and displaying (or outputting, in the case of audio) the MPU. Furthermore, if the first PTS cannot be acquired, it is impossible to synchronize audio and video playback based on their respective PTSs, for example.

[0020] In other words, the data transmission method and data reproduction method, as well as the data transmission device and data reproduction device, envisaged here have the problem that they cannot be adequately applied to broadcasts that require reproduction at a predetermined time.

[0021] Conventional MMT was designed to transmit MMT files (MP4 files), so the PTS and DTS of the first sample displayed or decoded in an MPU (Media Processing Unit) of an asset configured according to MMT were known. Therefore, a data playback device can acquire in advance the first PTS and DTS of the MPU where playback will begin. However, when a data transmission device generates and transmits MMT data sequentially, and a data playback device plays the transmitted data, the data playback device cannot acquire in advance the absolute time, such as the first PTS of the MPU where playback will begin, and is therefore unable to determine the timing of playback start. In other words, there is a problem in that this method cannot be fully applied to MMT data transmitted for live broadcasts, etc.

[0022] In order to solve such problems, a data transmission method according to one embodiment of the present invention includes a generation step of generating a plurality of coded data units, each consisting of a plurality of samples, reference time information indicating a reference time for setting the current time, and start time information indicating the start time, which is the time at which the first sample in the coded data unit is presented or decoded, and a transmission step of transmitting the generated plurality of coded data units, the reference time information, and the start time information, wherein the start time information indicates the start time of each of a plurality of specific coded data units, which are coded data units among the generated plurality of coded data units, whose presentation or decoding begins after the start time information is transmitted, and each of the generated plurality of coded data units indicates the time at which each sample other than the first sample in the coded data unit is presented or decoded as a relative value to the time of the other samples in the coded data unit.

[0023] As a result, the start time information indicates the start time (e.g., start PTS) of each of multiple specific encoded data units (e.g., multiple specific MPUs) whose presentation or decoding will begin after the start time information is transmitted. Therefore, when the data playback device receives the start time information, it can acquire the start time of each of multiple specific encoded data units whose presentation or decoding will begin after that time, i.e., the appropriate start time. As a result, the data playback device can appropriately start presenting or decoding multiple specific encoded data units from their start times. Therefore, this is fully applicable to broadcasts that require playback at a predetermined time. Furthermore, because the start time information indicates the start time of not just one, but each of multiple specific encoded data units, even if the data playback device is unable to receive one specific encoded data unit, it can start presenting or decoding the other specific encoded data unit at the appropriate time as long as it can start receiving the other specific encoded data unit.

[0024] The head time information may indicate the head time of at least one of the plurality of generated coded data units that is to be transmitted immediately after the head time information is transmitted.

[0025] This allows the data reproduction device to quickly start presenting or decoding the coded data unit at an appropriate time.

[0026] In addition, in the transmitting step, the leading time information may be transmitted periodically.

[0027] This makes it possible to notify the data reproduction device of the start times of many coded data units.

[0028] The start time information may indicate the start time of the plurality of specific coded data units that will start to be presented or decoded by the time the next start time information following the start time information of the start time information is transmitted.

[0029] As a result, the multiple start times indicated by the start time information are only the start times of each of multiple specific encoded data units whose presentation or decoding begins at a time close to the time when the start time information is transmitted, so that only the accurate start times can be indicated in the start time information.

[0030] In addition, the data transmission method may further include a calculation step of calculating the final reception time by adding the propagation time required from when the data is transmitted until it is received by the data reproduction device and the delay time from when the start time information is received by the data reproduction device until reception of the encoded data unit is first started to the time when the start time information is transmitted to the time when the start time information is transmitted, and the start time information may indicate the start times of all encoded data units among the generated plurality of encoded data units whose presentation or decoding begins between the time when the start time information is transmitted and the final reception time as the start times of the plurality of specific encoded data units.

[0031] As a result, since the start time information indicates multiple start times taking into account propagation time and delay time, it is possible to prevent a situation where the start time information does not indicate any start time of an encoded data unit that can be presented or decoded after the data reproduction device receives the start time information.

[0032] Also, a data reproduction method according to one aspect of the present invention includes a receiving step of receiving a plurality of coded data units each consisting of a plurality of samples, reference time information indicating a reference time for setting a current time, and start time information indicating a start time which is a time at which a first sample in the coded data unit is to be presented or decoded, and a reproduction step of reproducing the plurality of coded data units, wherein the start time information indicates the start time of a plurality of specific coded data units which are coded data units whose presentation or decoding starts after the start time information is received among the plurality of received coded data units, and each of the plurality of received coded data units is a specific coded data unit. The time at which each of a plurality of non-leading samples, which are samples other than the leading sample in a unit, is presented or decoded is indicated as a relative value to the time of the other samples in the coded data unit, and in the playback step, previous samples are played back so that the plurality of samples included in the plurality of specific coded data units are each presented or decoded at a timing according to the current time set according to the reference time indicated by the reference time information, the leading time of each of the plurality of specific coded data units indicated by the leading time information, and the relative value of each of the plurality of non-leading samples indicated by each of the plurality of received coded data units.

[0033] For example, in the reproduction step, when reproducing a first sample in a current coded data unit that is one of the plurality of specific coded data units, the first sample is presented or decoded when a current time set according to a reference time indicated by the reference time information reaches the start time of the current coded data unit that is indicated by the start time information. Also, in the reproduction step, when reproducing a current non-starting sample that is one of the plurality of non-starting samples in the current coded data unit, a processing time for presenting or decoding the current non-starting sample is calculated, and the current time reaches the calculated processing time, and the processing time is calculated by adding at least the relative value of the current non-starting sample among the relative values ​​of the plurality of non-starting samples indicated by the current coded data unit to the start time of the current coded data unit that is indicated by the start time information.

[0034] This allows all samples (access units) contained in multiple coded data units to be presented or decoded at a predetermined time, making it fully applicable to broadcasting that requires playback at a predetermined time.

[0035] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or as any combination of a system, a method, an integrated circuit, a computer program, or a recording medium.

[0036] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0037] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

[0038] (Embodiment) FIG. 2 is a block diagram of the data transmission device according to this embodiment.

[0039] The data transmission device 10 in this embodiment is a device that can be fully applied to broadcasting, and includes a generation unit 11 and a transmission unit 12.

[0040] The generator 11 generates a plurality of coded data units each consisting of a plurality of samples, reference time information indicating a reference time for setting the current time, and start time information indicating a start time at which the first sample in the coded data unit is presented or decoded. The transmitter 12 transmits the generated coded data units, reference time information, and start time information.

[0041] Here, the start time information indicates the start time of each of a plurality of specific coded data units, which are coded data units whose presentation or decoding starts after the start time information is transmitted, among the plurality of coded data units. Also, each of the plurality of coded data units indicates the time at which each of the samples other than the first sample in the coded data unit is presented or decoded relative to the time of the other samples in the coded data unit.

[0042] FIG. 3 is a flowchart showing the operation of the data transmitting device 10 in this embodiment.

[0043] First, the generating unit 11 generates a plurality of coded data units, reference time information, and start time information (step S11). Next, the transmitting unit 12 transmits the generated plurality of coded data units, reference time information, and start time information (step S12).

[0044] For example, an encoded data unit is an MPU in MMT, and a sample is an access unit. The reference time is a common time between the data transmitting device 10 and a data reproduction device that receives and reproduces data from the data transmitting device 10, or a time for synchronizing between the data transmitting device 10 and the data reproduction device. For example, the reference time is UTC (Coordinated Universal Time) or STC (System Time Clock). The start time information indicates, for each MPU that starts presentation or decoding after the start time information is transmitted, the MPU's identification information (such as a sequence number) and the start time, which is the time at which the first access unit in the MPU is presented or decoded, in association with each other. Note that the first sample (access unit) in an encoded data unit (MPU) is the first sample in presentation order or the first sample in decoding order. In other words, if the start time indicated by the start time information is the time at which the first sample is presented (first PTS), then the first sample is the first sample in presentation order. Furthermore, if the start time indicated by the start time information is the time at which the start sample is decoded (start DTS), then the start sample is the start sample in decoding order.

[0045] In the data transmission device 10 and data transmission method according to the present embodiment, the start time information indicates the start times (e.g., start PTSs) of multiple specific encoded data units (e.g., multiple specific MPUs) whose presentation or decoding starts after the start time information is transmitted. Therefore, upon receiving the start time information, the data reproduction device can acquire the start times of each of the multiple specific encoded data units whose presentation or decoding starts after that time, i.e., the appropriate start times. As a result, the data reproduction device can appropriately start presenting or decoding multiple specific encoded data units from their start times. Therefore, this method is fully applicable to broadcasts that require playback at a predetermined time. Furthermore, since the start time information indicates the start times of not just one, but multiple specific encoded data units, even if the data reproduction device is unable to receive one specific encoded data unit, it can appropriately start presenting or decoding the other specific encoded data units as long as it can start receiving the other specific encoded data units.

[0046] The data transmitting device 10 and the data transmitting method according to this embodiment will be described in detail below. In the following description, an MPU, an access unit, and a first PTS will be used as specific examples of the coded data unit, the sample, and the first time, respectively.

[0047] The leading time information in this embodiment includes at least the leading PTS of the MPU that is transmitted immediately after the transmission time of the leading time information.

[0048] The first PTS indicates the PTS of the first sample in the MPU in presentation (display) order, in the same format as absolute time. In the case of MMT, absolute time is expressed in UTC (Coordinated Universal Time), and the value obtained from UTC using NTP (Network Time Protocol) is used. Absolute time is not limited to UTC, and any reference clock that can express absolute time, such as the STC (System Time Clock) in the MPEG-2 system, can be used.

[0049] FIG. 4 is an explanatory diagram for explaining the data transmission method according to this embodiment.

[0050] As shown in Fig. 4, the start time information d1 stores the start PTSs of MPUs (MPU #3, MPU #4, MPU #5) that are transmitted at time T_send but have not yet been transmitted at time T_send. Specifically, the start time information d1 indicates the sequence number of MPU #3, which is transmitted immediately after time T_send, in association with the start PTS of that MPU #3, the sequence number of MPU #4 in association with the start PTS of that MPU #4, and the sequence number of MPU #5 in association with the start PTS of that MPU #5. In Fig. 4, the number shown with # to the right of an MPU is the sequence number (identification information) of that MPU. The MPU with the sequence number indicated by this start time information d1 is the specific encoded data unit described above.

[0051] When the start time information d1 is transmitted at time T_send in this way, the data playback device acquires the start time information d1 and starts receiving, for example, data from MPU #4. At this time, the data playback device acquires the start PTS (PTSab_4) of MPU #4 from the start time information d1 and starts presenting MPU #4 when the current time becomes time PTSab_4.

[0052] If an MMT package is made up of multiple assets (a sequence of multiple MPUs for each video and audio), the data transmitting device 10 transmits start time information d1 for each asset. The start time information d1 may indicate the start DTS instead of the start PTS of the MPU, or may indicate both the start PTS and the start DTS. Here, the start DTS indicates the DTS of the first sample in decoding order within the MPU in the same format as absolute time.

[0053] Furthermore, in this embodiment, the start time information d1 in an MPU of MMT is described as an example, but the start time information d1 may indicate the start time of an encoded data unit other than an MPU. In other words, this embodiment can be applied to any format in which the PTS or DTS of an access unit (equivalent to a sample in MP4, MMT, or MPEG-DASH) in content cannot be represented in the same format as absolute time, such as an MPEG-DASH (Dynamic Adaptive Streaming over HTTP) segment or an MP4 movie fragment. When transmitting an MPU or an MPEG-DASH segment, only the sample data constituting the MPU may be transmitted without transmitting header information such as moov or mdat. In this case, the relative value of the PTS or DTS in the sample data in the MPU from the start time information d1 may be transmitted as information separate from the MPU. Furthermore, the data transmission method of this embodiment can be applied to cases in which content is received and played via streaming (including multicast, broadcast, etc.) or download over a broadcast or communication network.

[0054] Here, the number or range of the leading PTSs of the MPU indicated in the leading time information d1 will be described in detail.

[0055] The generation unit 11 of the data transmission device 10 stores in the head time information d1 at least the first PTS of a plurality of MPUs that indicates a time that is the same as the last reception time T_last or a time that is earlier than the last reception time T_last. The last reception time T_last is defined as the time obtained by adding the maximum delay time Max_delay and the propagation time Trans_time to the transmission time T_send, as shown in the following (Equation 1).

[0056] T_last = T_send + Trans_time + Max_delay (Formula 1)

[0057] In (Equation 1), T_send is the transmission time of the start time information d1, Trans_time is the data propagation time on the transmission path (the time it takes from when the data is transmitted until it is received by the data playback device), and Max_delay is the maximum delay time from when the start time information d1 is received by the data playback device until the MPU first starts receiving.

[0058] If an MPU is a random access unit (the beginning of an MPU is a random access point), when a data playback device is able to obtain the beginning data of an MPU for the first time after receiving beginning time information d1, it can decode data including that MPU from that MPU.

[0059] For example, even if the data playback device starts receiving immediately after the first data of MPU #3 shown in Fig. 4, it cannot start decoding from MPU #3, but can start decoding from MPU #4. In other words, there is a time lag of up to the playback time length of the MPU from the start of reception until decoding can start. Therefore, when calculating the maximum delay time Max_delay, the generation unit 11 of the data transmission device 10 may calculate the maximum delay time Max_delay by adding the maximum value of the playback time length of a random access unit such as an MPU to the delay time from receiving the first time information d1 until reception of the MPU data can start.

[0060] If the MPU is not a random access unit, the generation unit 11 of the data transmission device 10 may determine the maximum delay time Max_delay based on the time from when the start time information d1 is received to when the data at the random access point starts to be received.

[0061] When determining the transmission time T_send, the generation unit 11 of the data transmission device 10 may determine the transmission time T_send not to be the exact transmission time of the start time information d1, but to be an estimated value of the transmission time, or the time at which the start time information d1 is generated.

[0062] When the generation time of the leading time information d1 is set as the transmission time T_send, the generation unit 11 determines the last reception time T_last taking into account the difference between the generation time and the actual transmission time. For example, when the transmission unit 12 packetizes the leading time information d1 or the MPU (e.g., MMT packets, TS packets, or RTP packets) and transmits them, it multiplexes these data packets. At this time, the packets of the leading time information d1 and the MPU packets may not be multiplexed in ascending order of the time the data contained in those packets was generated. In other words, a case may occur in which an MPU generated after the leading time information d1 is transmitted before the leading time information d1.

[0063] Therefore, the generation unit 11 may determine the last received time T_last based on the amount of fluctuation in the transmission time caused by the change in the order of packets when multiplexing packets. However, in the case of data acquired by download (such as MPEG-DASH content), such a problem does not occur because packet multiplexing is not performed.

[0064] Alternatively, the generating unit 11 may periodically generate the leading time information d1, and the transmitting unit 12 may transmit the leading time information d1 each time the leading time information d1 is generated. Note that the leading PTSs indicated by the leading time information d1 that is transmitted sequentially are updated according to the transmission time T_send of the leading time information d1.

[0065] If the transmission period is T, after the first time information d1 is transmitted at time T_send, the next first time information d1 is transmitted at time (T_send+T). The next first time information d1 indicates at least the first PTS of the MPU that will be transmitted immediately after time (T_send+T). Therefore, the first time information d1 transmitted at time T_send does not need to indicate the first PTS of the MPU after time (T_send+T). Therefore, the generation unit 11 may compare the time calculated from (Equation 1) with (T_send+T) and determine the smaller value as T_last (or, if they are the same, either value).

[0066] In broadcasting, the propagation time Trans_time is constant, so the generation unit 11 can determine the propagation time Trans_time depending on the broadcasting method, such as terrestrial broadcasting or satellite broadcasting. In particular, in terrestrial broadcasting, the propagation time Trans_time may be ignored. The maximum delay time Max_delay may also be defined according to operational regulations for the broadcasting service. Furthermore, when the data reproduction device operates in synchronization with a reference time, there is no need to take the propagation time Trans_time into consideration, so the propagation time Trans_time can be ignored.

[0067] On the other hand, in a communication network, the propagation time Trans_time varies depending on the congestion state of the network, etc. Therefore, the generation unit 11 may set the propagation time Trans_time assuming the worst case, or may adaptively set the propagation time Trans_time after estimating the propagation time from the transmission server (data transmission device 10) to the data reproduction device.

[0068] Furthermore, there are cases where the start time information d1 and the MPU are received on different channels (for example, the start time information d1 is received via HTTP, and the MPU is received via the MMT protocol or RTP (Real-time Transport Protocol)). In such cases, after acquiring the start time information d1, the data transmitting device 10 may receive data of an MPU that was transmitted before the transmission time of the start time information d1 (for example, in FIG. 4, after receiving the start time information d1, data of MPU #1 and MPU #2 is received). Therefore, the generating unit 11 may include the start PTS of an MPU that is transmitted before the transmission time T_send in the start time information d1.

[0069] In broadcasting, the start time information d1 is transmitted in an MPEG-2 TS section or an MMT message in MMT, so an MPU with a start PTS indicating a time earlier than the transmission time T_send will not be received after that transmission time T_send. However, a similar problem can occur when the start time information d1 is obtained from a transmission path other than broadcasting, such as via a communication network.

[0070] In the above example, the generation unit 11 calculated the last received time T_last using the above (Equation 1), but the last received time T_last may be determined by adding a predetermined length of time to the sending time T_send without performing such a calculation.

[0071] Here, a method for generating the leading time information d1 will be described in detail.

[0072] The generating unit 11 includes a head time information generating unit that generates head time information d1.

[0073] FIG. 5 is a block diagram of the leading time information generating unit in this embodiment.

[0074] The head time information generating unit 100 includes a start MPU determining unit 101, a head PTS acquiring unit 103, a determining unit 104, and an information generating unit .

[0075] The start MPU determination unit 101 determines the MPU transmitted immediately after the transmission time T_send of the start time information d1 as the start MPU, and notifies the start PTS acquisition unit 103 of the identification information of the start MPU.

[0076] The start PTS acquisition unit 103 acquires auxiliary information and, based on the auxiliary information, sequentially acquires the start PTSs of each MPU after the start MPU indicated by the identification information notified by the start MPU determination unit 101. Then, every time the start PTS acquisition unit 103 acquires a start PTS, it notifies the determination unit 104 of the acquired start PTS.

[0077] The determining unit 104 determines whether the leading PTS notified from the leading PTS acquiring unit 103 indicates a time before the last reception time T_last.

[0078] The information generating unit 106 generates the leading time information d1 based on the determination result of the determining unit 104.

[0079] FIG. 6 is a flowchart showing the operation of the head time information generating section 100 in this embodiment.

[0080] First, the start MPU determination unit 101 determines a start MPU that will be transmitted immediately after the transmission time T_send of the start time information d1. Note that this start MPU is the i=0th MPU in a string of multiple MPUs included in the MMT asset, and is represented as MPU(0). i is a sequence number (identification information). At this time, the start MPU determination unit 101 also notifies the start PTS acquisition unit 103 of the identification information of the determined start MPU. Upon acquiring the identification information from the start MPU determination unit 101, the start PTS acquisition unit 103 thereby acquires the start PTS of the start MPU (MPU(0)) indicated by the identification information by referring to auxiliary information as necessary (step S101).

[0081] Next, the leading PTS acquisition unit 103 sets the number i to 1 (step S102), and acquires the leading PTS of MPU(1), which is the i=1th MPU, by referring to auxiliary information as necessary (step S103). The leading PTS acquisition unit 103 also notifies the determination unit 104 of the acquired leading PTS.

[0082] Next, when the determination unit 104 acquires the leading PTS of MPU(1) from the leading PTS acquisition unit 103, it determines whether the leading PTS indicates a time before the last reception time T_last (step S104). Note that the propagation time Trans_time in the above (Equation 1) differs depending on the transmission path over which the MMT data is transmitted, for example, between broadcasting and a communication network. Therefore, the determination unit 104 may set a different value for the propagation time Trans_time depending on the transmission path over which the MMT data (MPU) is transmitted. For example, when two assets are transmitted via broadcasting and a communication network, respectively, the determination unit 104 sets the propagation time Trans_time for broadcasting and the communication network for each of the two assets.

[0083] Here, if the determination unit 104 determines that the leading PTS indicates a time before the last reception time T_last (Yes in step S104), it notifies the leading PTS acquisition unit 103 of the determination result. As a result, the leading PTS acquisition unit 103 increments the number i (step S105). That is, in the above example, since the number i=1, the leading PTS acquisition unit 103 updates the number i to 2. Then, the leading PTS acquisition unit 103 repeatedly executes the process from step S103. That is, the leading PTS acquisition unit 103 acquires the leading PTS of the MPU identified by the updated number i.

[0084] On the other hand, if the determination unit 104 determines that the leading PTS does not indicate a time before the last reception time T_last (No in step S104), it notifies the information generation unit 106 of the determination result. As a result, the information generation unit 106 generates leading time information d1 that stores the leading PTS of each of the 0th to i-th MPUs (step S106). The leading time information d1 generated in this manner indicates, for each of the 0th to i-th MPUs, the identification information (e.g., sequence number) of that MPU and the leading PTS of that MPU in association with each other. Note that the information generation unit 106 may store, in the leading time information d1, the leading PTS of the 0th MPU (start MPU) and the playback durations of the start MPU and each of the subsequent MPUs (MPUs following the start MPU). Furthermore, the information generating unit 106 may store, in the start time information d1, the start PTS of the 0th MPU (start MPU) and a flag indicating whether the playback duration of the start MPU and each subsequent MPU is constant. In this case, if the flag indicates that the playback duration is constant (flag = 1), the information generating unit 106 stores only the start PTS and playback duration of the start MPU and the flag in the start time information d1. On the other hand, if the flag indicates that the playback duration is not constant (flag = 0), the information generating unit 106 stores, in the start time information d1, not only the start PTS and flag of the start MPU but also the playback duration of the start MPU and each subsequent MPU.

[0085] The process of step S103 shown in FIG. 6 will now be described in detail.

[0086] When obtaining the first PTS of the ith MPU (MPU(i)), the method for obtaining that first PTS differs depending on the following three cases: Case 1, where MPU(i-1) already exists; Case 2, where MPU(i-1) does not exist but MPU(i-1) can be generated from already existing encoded data; and Case 3, where even the encoded data does not exist and encoding processing is required to generate MPU(i-1).

[0087] First, the first case will be described.

[0088] In the first case, the head PTS acquisition unit 103 acquires PTS(i), which is the start time of MPU(i), by adding the sum of the playback durations of the samples that make up MPU(i-1) to PTS(i-1), which is the start PTS of MPU(i-1). At this time, the head PTS acquisition unit 103 analyzes the MP4 header in MPU(i-1) to acquire the above-mentioned sum of playback durations. Note that this sum of playback durations is treated as, for example, the above-mentioned auxiliary information.

[0089] Next, the second case will be described.

[0090] If the coded data is video data, the start PTS acquisition unit 103 determines one or more random access units constituting MPU(i-1) from the coded data. Then, the start PTS acquisition unit 103 acquires PTS(i) by adding the sum of the playback times of the samples constituting the determined one or more random access units (for example, the auxiliary information described above) to PTS(i-1). Note that a random access unit corresponds to a sequence starting from an IDR picture in MPEG-4 AVC or HEVC (High Efficiency Video Coding), or a GOP in MPEG-2 Video.

[0091] Alternatively, the first PTS acquisition unit 103 may determine the random access units that constitute the MPU using a playback time length that is preset for the MPU. For example, if the playback time length of the MPU is preset to 2 seconds and the playback time length of each random access unit is 1 second, the first PTS acquisition unit 103 determines two random access units that constitute the MPU.

[0092] For example, the frame rate of the encoded data may change during a commercial (commercial message) portion of a broadcast. When the frame rate changes (e.g., from 24 Hz to 60 Hz), the playback time length of the random access unit also changes. Therefore, the first PTS acquisition unit 103 may determine the random access unit that constitutes the MPU taking this change into consideration.

[0093] On the other hand, if the encoded data is audio data or voice data, the encoded data can be decoded from any access unit that constitutes the encoded data. Therefore, the first PTS acquisition unit 103 determines the playback time length of each access unit based on the sampling frequency. Then, the first PTS acquisition unit 103 determines the access units that constitute MPU(i-1) based on the preset playback time length of the MPU. The first PTS acquisition unit 103 then acquires PTS(i) by adding the sum of the playback times of those access units to PTS(i-1).

[0094] Next, the third case will be described.

[0095] When the encoded data is video data, the first PTS acquisition unit 103 presets the structure of the random access unit at the time of encoding (for example, the number of access units constituting the random access unit, the reference structure between the access units, etc.).The first PTS acquisition unit 103 then calculates the playback duration of the random access unit based on the preset structure, and acquires PTS(i) by adding the playback duration to PTS(i-1) as the above-mentioned auxiliary information, for example.

[0096] In this case, an encoding unit (not shown) that generates encoded data by encoding video data encodes the video data in accordance with a predetermined random access unit structure. Note that the encoding unit may or may not be included in the data transmission device 10.

[0097] Alternatively, a predetermined random access unit structure (structure A) may differ from the actual structure (structure B) of the coded data generated by coding by the coding unit. In this case, the data transmission device 10 may calculate the difference between the first PTS of the MPU corresponding to structure A and the first PTS of the MPU corresponding to structure B, and store information indicating the difference (difference information) in the header information of the MPU (such as a Box in 'moof') or the header of the MMT packet. For example, suppose that the random access unit of structure A is made up of 15 access units, the random access unit of structure B is made up of 18 access units, and the playback time length of the access units is 20 msec. In structure B, the playback time length is 60 msec (= (18 - 15) × 20 msec) longer than the estimated value, so the data transmission device 10 stores the difference information indicating that it is 60 msec longer than the estimated value in the above-mentioned header, etc.

[0098] A data reproduction device that receives such difference information modifies the start PTS indicated in the start time information d1 based on the difference information. Furthermore, when calculating the start PTSs of each MPU that follows the MPU corresponding to the modified start PTS, the data reproduction device also reflects the difference information in the start PTS indicated in the start time information d1. For example, the data reproduction device checks for the presence of difference information, and if present, determines a start PTS that reflects the difference information.

[0099] Here, the information generating unit 106 may include, in the start time information d1, information indicating whether the first PTS is a predicted value determined based on a predetermined random access unit structure. This information may be included as information common to each start PTS in the start time information d1, or may be included as information set individually for each start PTS.

[0100] Furthermore, if the structure of a random access unit changes due to a change in frame rate, etc., the playback time length of the random access unit also changes. Therefore, the data transmission device 10 may also store information indicating the change in playback time length in the above-mentioned header, etc.

[0101] On the other hand, if the encoded data is audio data or voice data, the first PTS acquisition unit 103 determines PTS(i) in the same manner as in the second case described above, based on the predetermined playback time length of the MPU. Note that, when the sampling frequency of an access unit constituting MPU(i-1) is switched, the playback time length of the access unit changes. Therefore, the data transmission device 10 may also store information indicating the change in playback time length in the above-mentioned header, etc.

[0102] In addition, in step S101 of Figure 6, when the leading PTS acquisition unit 103 acquires the leading PTS of MPU(0), the leading PTS acquisition unit 103 also acquires the leading PTS of MPU(0) by performing processing similar to the processing of step S103 described above based on the information of the MPU immediately preceding MPU(0).

[0103] FIG. 7 is a block diagram of the data reproduction device according to the present embodiment.

[0104] The data reproducing device 20 in this embodiment is a device that can be fully applied to broadcasting, and includes a receiving unit 21 and a reproducing unit 22.

[0105] The receiver 21 receives a plurality of coded data units, each consisting of a plurality of samples, reference time information indicating a reference time for setting the current time, and start time information d1 indicating a start time at which the first sample in the coded data unit is to be presented or decoded. The player 22 plays back the received coded data units.

[0106] Here, the start time information d1 indicates the start time of a plurality of specific coded data units, which are coded data units that start to be presented or decoded after the start time information d1 is received, among the plurality of received coded data units. Also, each of the plurality of received coded data units indicates the time at which each of a plurality of non-starting samples, which are samples other than the start sample in the coded data unit, is presented or decoded relative to the time of the other samples in the coded data unit.

[0107] Specifically, the playback unit 22 plays back the multiple samples contained in the multiple specific coded data units so that the multiple samples are each presented or decoded at a timing according to the current time set according to the reference time indicated by the reference time information, the start time of the multiple specific coded data units indicated by the start time information d1, and the relative values ​​of each of the multiple non-start samples indicated by each of the multiple received coded data units.

[0108] FIG. 8 is a flowchart showing the operation of data reproduction device 20 in this embodiment.

[0109] First, the receiving unit 21 receives a plurality of coded data units, reference time information, and start time information d1 (step S21). Next, the reproducing unit 22 reproduces the received plurality of coded data units (step S22). That is, the reproducing unit 22 reproduces the plurality of samples included in the plurality of specific coded data units so that the plurality of samples are respectively presented or decoded at timings according to the current time set according to the reference time indicated by the reference time information, the start times of the respective plurality of specific coded data units indicated by the start time information d1, and the relative values ​​of the respective plurality of non-start samples indicated by each of the received plurality of coded data units.

[0110] For example, as described above, the coded data unit is an MPU in MMT, and the sample is an access unit. The reference time is a common time between the data transmitting device 10 and the data reproducing device 20, or a time for synchronizing the data transmitting device 10 and the data reproducing device 20. For example, the reference time is UTC or STC. For each MPU whose presentation or decoding starts after the transmission of this start time information d1, the start time information d1 indicates, in association with the MPU's identification information (such as a sequence number) and the start time, which is the time when the first access unit in the MPU is presented or decoded. Note that the first sample (access unit) in a coded data unit (MPU) is the first sample in presentation order or the first sample in decoding order.

[0111] The data reproduction device 20 and data reproduction method of this embodiment can present or decode all samples (access units) contained in multiple encoded data units at a predetermined time, and can be fully applied to broadcasts that require reproduction at a predetermined time.

[0112] The data reproducing device 20 and the data reproducing method according to this embodiment will be described in detail below.

[0113] The playback unit 22 includes a leading PTS determination unit that determines the leading PTS of each MPU to be presented or decoded based on the leading time information d1.

[0114] FIG. 9 is a block diagram of the first PTS determination unit in this embodiment.

[0115] The leading PTS determination unit 200 includes a program information acquisition unit 201 , a leading time information acquisition unit 202 , a leading MPU acquisition unit 203 , a leading PTS judgment unit 204 , a first acquisition unit 205 , and a second acquisition unit 206 .

[0116] The program information acquisition unit 201 acquires MMT program information (specifically, MPT) from the MMT package received by the receiving unit 21. This program information is information transmitted from the data transmitting device 10, and the program information acquisition unit 201 may acquire this program information via the receiving unit 21, or may acquire it without via the receiving unit 21. Furthermore, this MMT program information includes identification information for each asset constituting the MMT package, and access information for each asset (information required to acquire the asset, such as the ID of the MMT packet that transmits the asset data, the URL to which the asset data is transmitted, etc.).

[0117] The leading time information acquisition unit 202 acquires program information from the program information acquisition unit 201, and based on the program information, acquires leading time information d1 of each asset constituting the MMT package from the MMT package. The leading time information acquisition unit 202 then notifies the leading MPU acquisition unit 203 that leading time information d1 has been acquired, and outputs the leading time information d1 to the leading PTS determination unit 204.

[0118] The start time information d1 may be stored in an MMT message and transmitted, similar to the program information. When MMT data is transmitted using MPEG-2 TS, the start time information d1 may be stored as section data, such as a descriptor in a PMT (Program Map Table). When MPEG-DASH is used, the start time information d1 may be stored as segment information in an MPD (Media Presentation Description). Alternatively, the start time information d1 may be transmitted separately. The program information and the start time information d1 may be acquired from a transmission path different from the MPU. In this case, for example, the data reproduction device 20 may access a server on a communication network to acquire the program information or the start time information d1 of the MPU transmitted by broadcasting before receiving the MPU. In this case, the difference between the transmission time of the start time information d1 and the start time of reception, presentation, or decoding of the MPU increases compared to when the start time information d1 and the MPU are transmitted over the same transmission path. Therefore, the data transmitting device 10 may take this difference into consideration when determining the MPU corresponding to the leading PTS stored in the leading time information d1.

[0119] When notified by the head time information acquisition unit 202 that head time information d1 has been acquired, the head MPU acquisition unit 203 acquires the sequence number of the MPU that first received the head data of the MPU (head access unit) after acquiring the head time information d1. Then, the head MPU acquisition unit 203 notifies the head PTS determination unit 204 of the sequence number.

[0120] The head PTS determination unit 204 determines whether the head PTS of the MPU having the sequence number notified by the head MPU acquisition unit 203 is included in the head time information d 1 output from the head time information acquisition unit 202 .

[0121] If the first PTS determination unit 204 determines that the first PTS is included, the first acquisition unit 205 acquires the first PTS of the MPU with the above-mentioned sequence number from the first time information d1 and outputs the acquired first PTS.

[0122] If the first PTS determination unit 204 determines that the first PTS is not included, the second acquisition unit 206 calculates the first PTS of the MPU of the above-mentioned sequence number based on the first time information d1 and outputs the calculated first PTS.

[0123] For audio, initialization information for decoding can be stored as sample data, separate from headers such as moov and moof. In this case, decoding can start from any sample, so decoding or presentation can start from the first received sample without waiting for the beginning of the MPU. The header of an MMT packet indicates the sequence number of the MPU to which the sample data stored in the payload belongs, and the sample's position in the MPU in decoding order. Therefore, the DTS or PTS of a sample can be determined based on this information.

[0124] FIG. 10 is a flowchart showing the operation of the head PTS determination unit 200 in this embodiment.

[0125] First, the program information acquisition unit 201 acquires the program information (step S201). Next, the start time information acquisition unit 202 acquires the start time information d1 from the MMT package based on the program information (step S202).

[0126] Then, the head MPU acquisition unit 203 acquires the sequence number (SQN) of the MPU that first received the head data of the MPU after the head time information d1 was acquired (step S203). Next, the head PTS determination unit 204 determines whether or not the head PTS of the MPU (MPU(SQN)) identified by the acquired sequence number (SQN) is included in the head time information d1 (step S204). In other words, the head PTS determination unit 204 determines whether or not the head PTS associated with the acquired sequence number (SQN) is included in the head time information d1.

[0127] Here, if it is determined that the first PTS is included in the first time information d1 (Yes in step S204), the first acquisition unit 205 acquires the first PTS included in the first time information d1 (step S205).

[0128] 6, if the first PTS is set on the assumption that the video data or audio data will be coded based on a predetermined coding operation, and the actual coding operation differs from the predetermined coding operation, as in the third case, a difference occurs in the first PTS. That is, a difference occurs between the first PTS, which is a predicted value included in the first time information d1, and the first PTS, which is an actual value.

[0129] Since such a case may occur, the first acquisition unit 205 interprets information included in the MMT package indicating whether the first PTS is a predicted value, and if it is determined to be a predicted value, acquires difference information indicating the difference between the predicted value and the actual value. The first acquisition unit 205 may then acquire the first PTS reflecting the difference information by reflecting the difference information in the first PTS included in the first time information d1. Alternatively, the first acquisition unit 205 may determine whether difference information exists, and if so, reflect the difference information in the first PTS included in the first time information d1. Alternatively, if difference information is not transmitted from the data transmission device 10, the first acquisition unit 205 may correct the first PTS included in the first time information d1 using the first PTS of the MPU subsequent to the MPU (SQN). For example, the first PTS corresponding to the pre-decoding MPU (MPU_pre) subsequent to the MPU (SQN) may be indicated in the first time information repeatedly transmitted after the first time information d1. In this case, if the leading PTS corresponding to the MPU (MPU_pre) is not a predicted value, the first acquisition unit 205 determines the leading PTS of the MPU (SQN) using the leading PTS corresponding to the MPU (MPU_pre).

[0130] On the other hand, if it is determined in step S204 that the first PTS is not included in the first time information d1 (No in step S204), the second acquisition unit 206 calculates the first PTS of the MPU (SQN) based on the first time information d1 (step S206).

[0131] Here, the process of step S206 will be described in detail.

[0132] When calculating the first PTS of an MPU (SQN), the second acquisition unit 206 first calculates the playback duration (DUR) of the MPU whose first PTS is indicated in the first time information d1. The second acquisition unit 206 may then calculate the first PTS of the MPU (SQN) by assuming that the playback duration of each MPU after that MPU is equal to the DUR. For example, assume that the first time information d1 stores the first PTSs (PTSab_1, PTSab_2, PTSab_3) of each MPU (specific coded data unit) from MPU #1 to MPU #3. In this case, the second acquisition unit 206 calculates the playback duration of MPU #1 using (PTSab_2 - PTSab_1) and the playback duration of MPU #2 using (PTSab_3 - PTSab_2). The second acquisition unit 206 then assumes that the playback time of MPU #3 and each MPU subsequent to MPU #3 is equal to the playback time (DUR) of MPU #2. That is, DUR = (PTSab_3 - PTSab_2). Here, the leading PTS of MPU (SQN = 5), i.e., MPU #5, is the leading PTS of MPU #3 plus the playback times of MPU #3 and MPU #4. The playback times of MPU #3 and MPU #4 are also assumed to be DUR. Therefore, the second acquisition unit 206 calculates the leading PTS of MPU #5 as leading PTS = (PTSab_3 + 2 × DUR).

[0133] When the leading PTS of the MPU (SQN) is estimated in this manner, the data reproduction device 20 can start presenting (displaying) or decoding the MPU (SQN) based on the estimated leading PTS. Here, the data reproduction device 20 may acquire other leading time information subsequent to the leading time information d1 and acquire the leading PTS of the MPU before decoding from that other leading time information. In this case, the data reproduction device 20 may set the PTS of the access unit to be displayed first in the MPU before decoding based on the leading PTS acquired from that other following leading time information. For example, the data reproduction device 20 sets the PTS of that access unit to the leading PTS acquired from the other following leading time information. Thus, even if there is an error in the leading PTS estimated at the start of playback for the MPU before decoding, the data reproduction device 20 can correct the error using the other following leading time information.

[0134] Furthermore, if it is determined in step S204 that the leading PTS is not included in the leading time information d1, the data playback device may start decoding and presenting (displaying) the MPU (SQN) at a predetermined time. For example, after buffering the encoded data of the MPU (SQN), the decoding of the leading sample (access unit) in decoding order is completed, and the sample is presented at the earliest possible time for presentation (display). In this case, the leading PTS of the following MPU may also be adjusted based on other following leading time information.

[0135] Also, even if the sequence number of an MPU (SQN) is earlier than the sequence number of an MPU corresponding to the first PTS indicated by the first time information d1, the second acquisition unit 206 can calculate the first PTS of that MPU (SQN) using a method similar to that described above.

[0136] If it is determined in step S204 that the leading PTS is not included in the leading time information d1, the second acquisition unit 206 may acquire the leading time information again. For example, in broadcasting, the data transmission device 10 periodically transmits leading time information. Each time the data transmission device 10 transmits leading time information, it updates the leading time information so that the leading PTS of the MPU transmitted around the transmission time of the leading time information is indicated in the leading time information. Therefore, even if the leading time information d1 does not include the leading PTS of the MPU(SQN), the second acquisition unit 206 of the data reproduction device 20 can acquire the leading PTS of the MPU(SQN) by acquiring other leading time information transmitted temporally after the leading time information d1. If the sequence number of the MPU(SQN) is earlier than the sequence number of the MPU corresponding to the leading PTS indicated by the leading time information d1, the leading PTS of the MPU(SQN) cannot be acquired using the other leading time information. Therefore, in this case, the second acquisition unit 206 acquires the start PTS of the MPU (SQN) by assuming the playback duration of each MPU based on the start time information d1, as described above. Alternatively, the second acquisition unit 206 may separately acquire start time information indicating the start PTS of the MPU (SQN) from a server or the like.

[0137] Furthermore, the playback unit 22 of the data playback device 20 starts presenting the first access unit in the presentation order included in the MPU corresponding to the first PTS, at the time indicated by the first acquisition unit 205 or the second acquisition unit 206. That is, when playing back the first access unit in the target MPU, which is one of the multiple MPUs indicated by the start time information d1, the playback unit 22 presents (decodes) the first access unit when the current time, which is set according to the reference time indicated by the reference time information, reaches the first PTS (first DTS) of the target MPU, which is indicated by the start time information d1.

[0138] Here, a plurality of access units are arranged in the MPU, and the MPU indicates the time at which each of a plurality of non-first access units, which are access units other than the first access unit in the MPU, is presented (decoded) as a relative value (difference) from the time at which the immediately preceding access unit in the MPU is presented (decoded).

[0139] Therefore, when starting presentation of the nth access unit in presentation order included in an MPU, playback unit 22 first accumulates the relative values ​​of each access unit (each non-first access unit) from the first access unit (the access unit next to the first access unit) to the nth access unit. Then, playback unit 22 calculates the PTS of the nth access unit by adding the accumulated result of these relative values ​​to the first PTS of that MPU. Then, playback unit 22 starts presentation of the nth access unit when the current time reaches the calculated PTS.

[0140] That is, when playing back a target non-first access unit that is one of the plurality of non-first access units in the target MPU, the playback unit 22 first calculates the processing time at which to present (decode) the target non-first access unit. Next, the playback unit 22 presents (decodes) the target non-first access unit when the current time reaches the calculated processing time. In calculating the processing time described above, the playback unit 22 calculates the processing time by adding the relative value of at least the target non-first access unit, of the relative values ​​of the plurality of non-first access units indicated by the target MPU, to the first PTS (first DTS) of the target MPU, which is indicated by the first time information d1.

[0141] As described above, in the data transmission method and data reproduction method of this embodiment, MMT data is generated and transmitted sequentially, and even when the transmitted data is reproduced, the leading PTS of the received MPU can be obtained by referring to the leading time information. Also, even if the leading PTS of the MPU that started reception is not included in the first received leading time information, the leading PTS of the MPU can be estimated based on the content of the leading time information, or the leading PTS can be determined by repeatedly receiving the leading time information.

[0142] Although the data transmission method and data reproduction method according to the present invention have been described using the embodiments, the present invention is not limited to these.

[0143] For example, in the above-described embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software realizing the data transmitting device 10 and the data reproducing device 20 of the above-described embodiments is a program that causes a computer to execute the steps shown in FIG. 3 or FIG. 8, respectively.

[0144] Furthermore, there are cases where the MPU data that constitutes the assets of an MMT package is transmitted using a combination of transmission paths with different characteristics, such as broadcasting and communication networks. In such cases, there are cases where only the data received from either the broadcasting or communication network transmission path is decoded and played back, and cases where the data received from both the broadcasting and communication network transmission paths is decoded and played back.

[0145] Therefore, the data transmitting device 10 may separately store and transmit the start time information corresponding to the MPU of the asset transmitted by broadcasting and the start time information corresponding to the MPU of the asset transmitted over the communication network. In this case, the data reproducing device 20 can select and process the start time information according to the reproduction mode.

[0146] For example, the data transmitting device 10 stores the start time information in the MMT in a table. At this time, the data transmitting device 10 may store the start time information corresponding to each transmission path of the broadcast and the communication network in separate tables. These tables are identified by identification information. The identification information may be, for example, a table ID. The data transmitting device 10 may transmit the tables corresponding to the broadcast and the communication network over their respective transmission paths. Alternatively, the data transmitting device 10 may transmit the tables over their corresponding transmission paths, for example, by transmitting the table for broadcast over broadcast and the table for the communication network over the communication network. Furthermore, the data transmitting device 10 may signal or pre-specify the ID of the table to be used. Furthermore, in cases where broadcast is used as a service entry, the data transmitting device 10 may transmit both tables over broadcast. Furthermore, the data transmitting device 10 may store the start time information in an MMT descriptor or the like.

[0147] Furthermore, the present invention is not limited to the above-described embodiment and can be more generalized. For example, the data transmitting device 10 may store and transmit tables such as an MPT (MMT Package Table) or an MCIT (MMT Composition Information Table), i.e., information on an asset data acquisition method, a decoding time, or a presentation time, or information on an asset data display method, separately for each transmission path, such as broadcasting or a communication network. The data transmitting device 10 may transmit these tables via one of the transmission paths, or may transmit the tables for broadcasting via broadcast and the tables for the communication network via the corresponding transmission paths, such as communicating via the communication network. For example, in MMT, the 0th subset of the MPT or MCIT can be used for broadcasting, and the 1st subset can be used for the communication network.

[0148] Furthermore, the data transmission method and data playback method of the present invention can also be applied to cases where data is transmitted using conventional MPEG-2 TS for broadcasting and data obtained via a communication channel (communication network) using MMT, MPEG-DASH, or the like, and the data acquired via broadcasting and the communication network are handled in a unified manner. In this case, the start time information corresponding to MMT, MPEG-DASH, or the like transmitted via the communication channel may be stored in the section data of the TS. For example, a new descriptor may be defined, stored in the PMT, and transmitted. Alternatively, the communication server may determine the start time information independently of the broadcasting and transmit it via the communication channel.

[0149] Furthermore, when transmitting an asset consisting of multiple MPUs to the data reproduction device 20, the data transmission device 10 in this embodiment packetizes and transmits encoded data including the MPUs and start time information d1. The data reproduction device 20 receives the packetized encoded data. Even when a packet loss occurs, the data reproduction device 20 in this embodiment can start presenting the MPUs at the correct time after the packet loss. That is, when an MPU packet is lost, the data reproduction device 20 may not be able to obtain the relative value (time difference) of the access unit (non-first access unit) included in the MPU. In such a case, it is unable to determine the PTS of each access unit included in the MPU after the access unit. However, by receiving start time information d1 indicating the start PTS of the MPU following the MPU where the packet loss occurred, the data reproduction device 20 in this embodiment can start presenting the subsequent MPU from that start PTS. As a result, the data reproduction device 20 can quickly restore playback of the encoded data transmitted from the data transmission device 10 according to the correct time.

[0150] The following cases are also included in the present invention.

[0151] (1) Each of the above devices is specifically a computer system consisting of a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. Each device achieves its function when the microprocessor operates in accordance with the computer program. Here, a computer program is composed of a combination of multiple instruction codes that indicate commands to a computer to achieve a predetermined function.

[0152] (2) Some or all of the components constituting each of the above devices may be configured as a single system LSI (Large Scale Integration). A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple components on a single chip, and specifically, is a computer system configured to include a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its functions when the microprocessor operates in accordance with the computer program.

[0153] (3) Some or all of the components constituting each of the above devices may be configured as an IC card or a standalone module that can be attached to each device. The IC card or module is a computer system composed of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned ultra-multifunctional LSI. The IC card or module achieves its functions when the microprocessor operates according to a computer program. The IC card or module may be tamper-resistant.

[0154] (4) The present invention may be embodied as the methods described above, as a computer program for implementing these methods on a computer, or as a digital signal comprising the computer program.

[0155] The present invention may also be the computer program or the digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray (registered trademark) Disc), semiconductor memory, etc. Alternatively, the present invention may be the digital signal recorded on such a recording medium.

[0156] Furthermore, the present invention may also be such that the computer program or the digital signal is transmitted via a telecommunications line, a wireless or wired communication line, a network such as the Internet, or data broadcasting.

[0157] The present invention may also be a computer system having a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates in accordance with the computer program.

[0158] The program or the digital signal may also be implemented by another independent computer system by recording it on the recording medium and transferring it, or by transferring it via the network or the like.

[0159] (5) The above-described embodiments and modifications may be combined with each other.

[0160] Although the data transmission method and data reproduction method according to one or more aspects of the present invention have been described based on the embodiments, the present invention is not limited to these embodiments. As long as they do not deviate from the spirit of the present invention, various modifications conceivable by those skilled in the art to the present embodiments, or configurations constructed by combining components of different embodiments, may also be included within the scope of one or more aspects of the present invention. [Industrial Applicability]

[0161] The data transmission method according to the present invention has the effect of being fully applicable to broadcasting, and can be applied to devices or equipment that transport media such as video data and audio data, for example. [Explanation of symbols]

[0162] 10 Data transmission device 11 Generation part 12 Transmitter 20 Data playback device 21 Receiving unit 22 Playback Department 100 Leading time information generation unit 101 Start MPU determination section 103 First PTS acquisition part 104 Judgment section 106 Information generation section 200 First PTS determination section 201 Program Information Acquisition Department 202 First time information acquisition unit 203 First MPU acquisition part 204 First PTS judgment section 205 First Acquisition Section 206 Second Acquisition Section

Claims

1. A playback method implemented by a playback device that plays video data transmitted in a format of MPU (Media Processing Unit) in MMT (MPEG Media Transport), wherein one MPU corresponds to a plurality of access units; receiving a plurality of data packets storing a plurality of access units corresponding to the MPU; receiving a control packet containing control information; Reproducing the plurality of access units in accordance with the control information; In receiving the control packet, each of a plurality of pieces of time information corresponding to each of the plurality of MPUs is stored in the control packet transmitted at a time earlier than the time indicated by the time information and received at least once, the time information is information indicating the presentation time of the first access unit in the presentation order corresponding to the corresponding MPU, the time information stored in the received control packet is changed according to the transmission time of the control packet, and the received control packet stores a plurality of pieces of time information; The time information corresponding to a first MPU among the plurality of MPUs is stored in a plurality of control packets transmitted at mutually different times. How to play.

2. A playback device that plays back video data transmitted using an MPU (Media Processing Unit) format in MMT (MPEG Media Transport), wherein one MPU corresponds to a plurality of access units; a receiving unit that receives a plurality of data packets storing a plurality of access units corresponding to the MPU and receives a control packet storing control information; a playback unit that plays back the plurality of access units in accordance with the control information, In receiving the control packet, each of a plurality of pieces of time information corresponding to each of the plurality of MPUs is stored in the control packet transmitted at a time earlier than the time indicated by the time information and received at least once, the time information is information indicating the presentation time of the first access unit in the presentation order corresponding to the corresponding MPU, the time information stored in the received control packet is changed according to the transmission time of the control packet, and the received control packet stores a plurality of pieces of time information; The time information corresponding to a first MPU among the plurality of MPUs is stored in a plurality of control packets transmitted at mutually different times. playback device.

Images