Transmission method and transmission device

Abstract

To provide a transmission method of a content capable of regenerating a content that combines broadcasting and communication on a receiving side, even if timing of receiving the content through the communication is delayed.SOLUTION: The transmission method includes: a generation step of generating a content in a format conforming to MMT; a content transmitting step of transmitting the content in the format generated in the generation step; and an information transmitting step of, when transmitting the content using both a broadcast wave and a communication channel, transmitting information indicating that there is a content to be transmitted via the communication channel, information indicating the format of meta information for playback control of the content to be transmitted via the communication path, and auxiliary information including location information indicating an acquisition destination of the content to be transmitted via the communication path by using the broadcast wave. In the generation step, auxiliary information is generated including the same in message information which is information about the acquisition of the content.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a transmission method and a reception method.

Background Art

[0002] Conventionally, the main transmission path for distributing content is a broadcast wave, and as a media transport method widely used in a broadcast system using a broadcast wave, for example, there is MPEG-2 TS (Moving Picture Experts Group-2 Transport Stream).

[0003] On the other hand, with the recent progress of network technology, it has become possible to distribute content using a communication path such as the Internet. That is, it has become possible to distribute content not only using a broadcast wave but also using a communication path, and the transmission paths for distributing content are diversifying.

[0004] For example, Non-Patent Document 1 discloses MMT (MPEG Media Transport) and the like as a new media transport method assuming distribution of content using both broadcast and communication (see Non-Patent Document 1). For example, Patent Document 1 discloses a technique for accessing communication content based on data obtained from a broadcast, mainly using the broadcast.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

[0006] However, while there is discussion about acquiring information related to content reception (service information) and initiating content reception in order to distribute content using both broadcasting and telecommunications, rapid access to content via telecommunications is not taken into consideration, resulting in issues such as delays in the timing of initiating content reception via telecommunications.

[0007] Therefore, the object of the present invention is to provide a content transmission method that allows the receiving side to play content that combines broadcasting and communication, even if there is a delay in the start timing of receiving content via communication. [Means for solving the problem]

[0008] To solve the above problems, a transmission method according to one aspect of the present invention is a content transmission method that can transmit content using broadcast waves and a communication channel, comprising: a generation step of generating the content in a format conforming to MMT (MPEG Media Transport); a content transmission step of transmitting the content in the format generated in the generation step; and, when transmitting content using broadcast waves and a communication channel respectively, an information transmission step of transmitting auxiliary information including information indicating that there is content to be transmitted by the communication channel, information indicating the format of metadata for playback control of the content transmitted by the communication channel, and location information indicating the acquisition destination of the content transmitted by the communication channel, using at least the broadcast waves, wherein in the generation step, the auxiliary information is generated by including it in message information which is information relating to the acquisition of the content.

[0009] These general or specific embodiments may be implemented using a data receiving method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or they may be implemented using any combination of a data transmission method, a data receiving method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0010] According to the present invention, it is possible to realize a content transmission method that allows the receiving side to play content that combines broadcasting and communication even if there is a delay in the start timing of receiving content via communication. [Brief explanation of the drawing]

[0011] [Figure 1A] This figure shows an example of the data structure of service information in the broadcast-communication collaboration service of Embodiment 1. [Figure 1B] This figure shows an example of the data structure of service information in the broadcast-communication collaboration service of Embodiment 1. [Figure 2] This figure shows an example of the overview of the transmission path identification descriptor in Embodiment 1. [Figure 3A] This figure shows another example of the data structure of service information in the broadcast-communication collaboration service of Embodiment 1. [Figure 3B] This figure shows another example of the data structure of service information in the broadcast-communication collaboration service of Embodiment 1. [Figure 4A] This flowchart shows an example of the operation of the receiving side in the broadcast-communication collaboration service of Embodiment 1. [Figure 4B] This flowchart shows another example of the operation of the receiving side in the broadcast-communication collaboration service of Embodiment 1. [Figure 5] This is a block diagram showing an example of the configuration of the receiving device in Embodiment 1. [Figure 6A] This figure shows an example of the data structure of service information in a broadcast-communication collaborative service in a modified example 1 of Embodiment 1. [Figure 6B]It is a diagram showing an example of the data structure of service information in the broadcast communication cooperation service of Modification Example 1 of Embodiment 1. [Figure 7A] It is a flowchart showing an example of the operation on the receiving side in the broadcast communication cooperation service of Modification Example 1 of Embodiment 1. [Figure 7B] It is a flowchart showing another example of the operation on the receiving side in the broadcast communication cooperation service of Modification Example 1 of Embodiment 1. [Figure 8] It is a block diagram showing an example of the configuration of the receiving device in Modification Example 1 of Embodiment 1. [Figure 9] It is a flowchart showing an example of the operation on the receiving side in the broadcast communication cooperation service of Modification Example 2 of Embodiment 1. [Figure 10] It is a block diagram showing an example of the configuration of the receiving device in Modification Example 2 of Embodiment 1. [Figure 11] It is a diagram showing an example of the data structure of service information in the broadcast communication cooperation service of Modification Example 4 of Embodiment 1. [Figure 12] It is a flowchart showing an example of the operation on the receiving side in the broadcast communication cooperation service of Modification Example 4 of Embodiment 1. [Figure 13A] It is a diagram showing an example of the syntax of the location information descriptor in Modification Example 4 of Embodiment 1. [Figure 13B] It is a diagram showing an example of the syntax of the location information descriptor in Modification Example 4 of Embodiment 1. [Figure 13C] It is a diagram showing an example of the syntax of the location information descriptor in Modification Example 4 of Embodiment 1. [Figure 13D] It is a diagram showing an example of the syntax of the location information descriptor in Modification Example 4 of Embodiment 1. [Figure 14] It is a flowchart showing an example of the operation on the receiving side in the broadcast communication cooperation service of Modification Example 6 of Embodiment 1. [Figure 15] It is a flowchart showing another example of the operation on the receiving side in the broadcast communication cooperation service of Modification Example 6 of Embodiment 1. [Figure 16] This flowchart shows an example of the operation of the receiving side in a broadcast communication collaboration service in a modified example 7 of Embodiment 1. [Figure 17] This is a block diagram showing an example of the configuration of a receiving device in modified example 7 of Embodiment 1. [Figure 18A] This figure shows an example of the description of default playback control information in Embodiment 2. [Figure 18B] This figure shows an example of video display according to the layout information of the default playback control information shown in Figure 18A. [Figure 19A] This figure shows an example of application control information in Embodiment 2. [Figure 19B] This figure shows an example of video display according to the layout information of the application control information shown in Figure 19A. [Figure 20A] This figure shows another example of how to describe the default playback control information in Embodiment 2. [Figure 20B] This figure shows an example of video display according to the layout information of the default playback control information shown in Figure 20A. [Figure 21A] This figure shows an example of application control information in Embodiment 2. [Figure 21B] This figure shows an example of video display according to the layout information of the application control information shown in Figure 21A. [Figure 22] This is a flowchart showing an example of the operation of the receiving side in Embodiment 2. [Figure 23] This is a block diagram showing an example of the configuration of the receiving device in Embodiment 2. [Figure 24] This figure shows an example of the data structure of service information in the broadcast-communication collaboration service in Embodiment 3. [Figure 25] This figure shows an example of the information included in the program configuration information descriptor of Embodiment 3. [Figure 26] This figure shows an example of the information included in the location information descriptor of Embodiment 3. [Figure 27] This figure shows an example of a location type included in the location information descriptor of Embodiment 3. [Figure 28] This figure shows an example of the information included in the asset configuration information descriptor of Embodiment 3. [Figure 29] This is a flowchart showing the reception method in the broadcast-communication collaboration service of Embodiment 3. [Figure 30] This is a block diagram showing an example of the configuration of the receiving device in Embodiment 3. [Modes for carrying out the invention]

[0012] (Knowledge that formed the basis of this invention) Currently, services that distribute content using both broadcasting and telecommunications (broadcast-telecommunications integrated services) are being considered. Among these, a method that primarily uses broadcasting and accesses content obtained from telecommunications (hereinafter referred to as telecommunications content) based on data obtained from broadcasting is considered promising. In broadcast-telecommunications integrated services, the operation at the start of viewing on the receiving side when receiving content is to acquire service information, and then start receiving content using encoded audio and video data, or a data carousel in an MPEG-2 system, similar to conventional services that distribute content using only broadcasting.

[0013] However, the service information obtainable through conventional services does not take into account factors such as rapid access to communication content or the operation of selectively receiving only broadcast content. Therefore, when using conventional service information to provide broadcast-communication collaboration services, there are challenges such as increased processing involved in analyzing the service information and delays in the start of communication content acquisition.

[0014] To solve these problems, a transmission method according to one aspect of the present invention is a content transmission method that can transmit content using broadcast waves and a communication channel, and when transmitting content using broadcast waves and a communication channel respectively, includes an information transmission step of transmitting auxiliary information for synchronizing the content via the broadcast waves and the content via the communication channel, which, when received by the receiving side, causes the synchronization, at least using the broadcast waves.

[0015] According to this embodiment, a content transmission method can be realized in which the receiving side can play back content that uses both broadcasting and communication even if there is a delay in the timing of the start of receiving content via communication. More specifically, when content is transmitted using broadcast waves and a communication channel, auxiliary information is transmitted to synchronize the content transmitted using broadcast waves and a communication channel, so that when the receiving side receives the auxiliary information, the receiving side can synchronize the content.

[0016] Here, for example, in the information transmission step, the auxiliary information may be transmitted prior to transmitting the content, and the auxiliary information may further include location information indicating where the content is to be obtained or information indicating where the location information is to be obtained.

[0017] Furthermore, for example, in the auxiliary information transmission step, the auxiliary information may also include and transmit difference information between the reference clock of the content via the broadcast wave and the reference clock of the content via the communication channel.

[0018] Furthermore, for example, in the auxiliary information transmission step, if the reference clocks of the content are different, the receiving side may be synchronized by transmitting the auxiliary information to synchronize the reference clock of the content via the communication channel with the reference clock of the content via the broadcast wave based on the difference information.

[0019] Furthermore, for example, the transmission method may include a generation step of generating the content in a format conforming to MMT (MPEG Media Transport), and a content transmission step of transmitting the content in the format generated in the generation step.

[0020] Furthermore, for example, in the generation step, the auxiliary information may be included in the message information, which is information related to the acquisition of the content, during generation.

[0021] Furthermore, in order to solve the above problems, a receiving method according to one aspect of the present invention includes a receiving step of receiving content transmitted using a broadcast wave and a communication channel, respectively, and a playback step of performing the synchronization process and playing the content when auxiliary information for synchronizing the content via the broadcast wave and the content via the communication channel is received.

[0022] Here, for example, in the receiving step, the auxiliary information may be received prior to receiving the content, and if the auxiliary information includes location information indicating the source of the content, the content may be received by acquiring the content based on the location information.

[0023] Furthermore, for example, in the receiving step, the auxiliary information may be received prior to receiving the content, and if the auxiliary information includes information indicating the source of location information indicating the source of the content, the location information may be obtained based on the information indicating the source of the location information, and the content may be received by obtaining the content from the obtained location information.

[0024] Furthermore, for example, in the playback step, in the reception step, the auxiliary information including difference information between the reference clock of the content via the broadcast wave and the reference clock of the content via the communication channel may be received, and if the reference clock of the content via the broadcast wave and the reference clock of the content via the communication channel are different, the content may be synchronized by synchronizing the reference clock of the content via the communication channel with the reference clock of the content via the broadcast wave based on the difference information, and the content may be played back.

[0025] Furthermore, in order to solve the above problems, a transmitting device according to one aspect of the present invention is a content transmitting device capable of transmitting content using broadcast waves and a communication channel, and when transmitting content using broadcast waves and a communication channel respectively, it includes an information transmitting unit that transmits auxiliary information for synchronizing the content transmitted by the broadcast waves and the content transmitted by the communication channel, the auxiliary information for causing the synchronization when received by the receiving side, using at least the broadcast waves.

[0026] Furthermore, in order to solve the above problems, a receiving device according to one aspect of the present invention comprises a receiving unit that receives content transmitted using broadcast waves and a communication channel, respectively, and a playback unit that, when it receives auxiliary information for synchronizing the content transmitted via broadcast waves and the content transmitted via the communication channel, performs the synchronization process and plays the content.

[0027] These general or specific embodiments may be implemented using a transmission method, a transmission device, a reception method, a reception device, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or they may be implemented using any combination of a data reception method, an integrated circuit, a computer program, or a recording medium.

[0028] The following will specifically describe a transmission method and a reception method, etc., according to one aspect of the present invention, with reference to the drawings.

[0029] The embodiments described below are all specific examples of the present invention. The numerical values, shapes, materials, components, arrangement and connection configurations of the components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, those components that are not described in the independent claim representing the highest-level concept will be described as optional components.

[0030] (Embodiment 1) This embodiment describes a transmission method for transmitting and receiving content in a broadcast-communication collaborative service.

[0031] [How to send] In this embodiment, the transmitting side transmits content (data or assets) using broadcast waves and a communication channel, but transmits service information prior to transmitting the content.

[0032] [Service Information] Here, "service information" refers to a series of pieces of information related to content reception and metadata acquisition, such as information for acquiring audio, video, or data broadcasting-related data after channel selection, or EPG (Electronic Program Guide) information.

[0033] Currently, broadcasts are primarily transmitted using the MPEG-2 TS (Transport Stream) section, which includes PAT (Program Association Table), PMT (Program Map Table), NIT (Network Information Table), CAT (Conditional Access Table), or EIT (Event Information Table) as defined by ARIB (Association of Radio Industries and Businesses).

[0034] In this embodiment, MMT (MPEG Media Transport), standardized by MPEG, is used as an example of the multiplexing format on the broadcast side in a broadcast-linked service. However, this multiplexing format is not limited to MMT, and other multiplexing formats such as TS or MPEG-DASH (Dynamic Adaptive Streaming over HTTP) may also be used.

[0035] In this embodiment, service information is transmitted in a data structure that can be stored in a multiplexed format. For example, in MMT, service information is transmitted in the form of tables such as MPT (MMT Package Table) or message information such as PA (Package Access) messages. In each table, auxiliary information can be described using descriptors, similar to TS.

[0036] Figures 1A and 1B show an example of the data structure of service information in the broadcast-communication collaboration service of Embodiment 1. More specifically, Figure 1A shows an MPT that includes a transmission path identification descriptor, and Figure 1B shows an example in which, for example, information about the transmission path of the assets constituting the package is described as attribute information in the transmission path identification descriptor shown in Figure 1A.

[0037] (Attribute information) 1) The assets constituting the package may include attribute information indicating whether they are transmitted by (1) broadcasting only, or (2) a combination of broadcasting and communication.

[0038] Furthermore, the information may indicate whether the audio and video data of the main program is transmitted by (1) broadcasting only, or (2) a combination of broadcasting and communication. This information allows that even if the main program is transmitted using broadcasting only, the audio, video, still images, or metadata other than the main program, such as HTML files, can be obtained from the communication network.

[0039] 2) When audio or video data is transmitted using both broadcasting and telecommunications, information indicating the relationship between the data transmitted in each method may be included as attribute information.

[0040] For example, when achieving scalability (temporal resolution (e.g., 60fps → 120fps), spatial resolution (e.g., 4k → 8k), bit depth (e.g., 8bit → 10bit)), attribute information can be used to indicate that broadcast data is transmitted using the basic layer and communication data is transmitted using the extended layer. The attribute information may also indicate that while the frame rate is 60fps with broadcast data alone, it can be increased to 120fps by using communication data in conjunction. It may also indicate that backup data for broadcasting is transmitted via communication. This allows the transmitting side to use the attribute information to switch to transmitting data via communication if broadcast reception deteriorates due to rain attenuation or other factors.

[0041] Furthermore, attribute information may also indicate information for identifying related assets. For example, attribute information may indicate the asset ID of the base layer and the asset ID of the extension layer corresponding to the base layer.

[0042] In MMT, it is possible to transmit the same asset using multiple transmission lines. Therefore, attribute information may indicate whether the same asset is transmitted using both broadcast and communication. In this case, the identification information of the asset (such as an asset ID) may be shown separately. Information for identifying related assets may be shown by individual transmission line identification descriptors for each asset, for example, as shown in Figure 2. Here, Figure 2 is a diagram showing an example of the overview of a transmission line identification descriptor in Embodiment 1. The individual transmission line identification descriptors shown in Figure 2 may, for example, show information for identifying whether the asset is a basic layer or an extended layer asset, or, if the asset is an extended layer asset, they may show the ID of the corresponding basic layer asset.

[0043] Furthermore, attribute information may group broadcast assets and communication assets separately, and show a list of assets included in each group. If there are assets that are transmitted using both broadcast and communication, they may be grouped separately.

[0044] 3) Information indicating whether audio and video transmitted by broadcast and audio and video transmitted by communication are played back in sync may be included as attribute information.

[0045] 4) The audio and video clock information transmitted by broadcast and the audio and video clock information transmitted by communication may be included as attribute information to indicate whether they are the same.

[0046] Furthermore, attribute information may be presented as individual fields, or information indicating the service type may be defined to allow identification by type. Additionally, attribute information may be described in a format different from that of a descriptor.

[0047] Furthermore, the transmission path identification descriptor may be stored in a table or message that shows package-level information, which is different from the MPT. Alternatively, the contents of the above transmission path identification descriptor may be described as a data structure different from the descriptor.

[0048] Furthermore, when sending multiple packages, a table or message listing the packages may be defined, and information such as the transmission path identifier descriptor may be included as package attribute information.

[0049] Figures 3A and 3B show another example of the data structure of service information in the broadcast-communication collaboration service of Embodiment 1.

[0050] In other words, as shown in Figures 3A and 3B, the location information for each asset may be stored in separate MPTs for assets transmitted via broadcast and communication. In this case, the package attribute information can be stored in the MPT sent on the transmission path that serves as the service entry point. For example, if broadcast is the entry point, it is stored in the MPT sent via broadcast. These MPTs can also be identified by their table_id.

[0051] The MMT standard specifies that the table_id value of the reference MPT is zero. Therefore, it is possible to set the table_id of the MPT corresponding to the entry point transmission path to zero, and the table_id of the MPT corresponding to the other transmission path to 1 or greater.

[0052] Furthermore, the MPT of broadcast assets may be transmitted by broadcast, and the MPT of communication assets may be transmitted by communication.

[0053] Alternatively, the location information for broadcast and communication assets may be stored in a single MPT. In this case, the location information for each asset should be stored consecutively to facilitate identification. For example, if there are N1 broadcast assets and N2 communication assets, the information for the N1 broadcast assets is described consecutively first, followed by the information for the N2 communication assets. The transmission path identifier descriptor may also indicate that N1 assets are transmitted in broadcast mode and N2 assets are transmitted in communication mode.

[0054] In this way, the transmitting side includes a transmission path identification descriptor, which is auxiliary information, in the service information before sending it. As a result, the receiving side can obtain the service information including the auxiliary information, and based on the package attribute information described in the transmission path identification descriptor, it can determine in advance whether communication data is included, or the dependencies between broadcast data and communication data, without having to analyze the information for each asset.

[0055] In particular, when receiving communication data, there is the advantage of being able to reduce the delay time required to initiate the receiving process.

[0056] [Transmitter] For example, the transmitting device of this embodiment is a content transmitting device capable of transmitting content using broadcast waves and a communication channel, and generates package attribute information such as a transmission channel identification descriptor as auxiliary information, and transmits it included in the service information.

[0057] The transmitting device of this embodiment may transmit this service information periodically. When the content of this service information is updated, the updated content is reflected in the service information immediately after the update.

[0058] Furthermore, the entry point is not limited to broadcasting; it may also be communication, or it may be possible to enter from both broadcasting and communication. If entry is possible from both, then at least the package-level information must be transmitted from both broadcasting and communication transmitting devices.

[0059] Furthermore, when the receiving end receives content from a broadcast-communication linked service, it is not necessarily required to receive data transmitted through both transmission paths. For example, it is acceptable to receive and play only the broadcast data. In this case, since the information required by the receiving end when receiving communication data is not required when receiving broadcast data, the transmitting end may transmit the information required when receiving communication data during the communication. That is, the transmitting end may transmit package-level information through the entry point transmission path, and transmit information specific to each transmission path, such as broadcasting and communication, through their respective transmission paths (see, for example, Figure 2). In the following explanation, we will assume that broadcasting is the entry point.

[0060] In this way, service information specific to each transmission line can be generated and transmitted by the transmitting equipment on each transmission line. However, information transmitted by MPT, such as asset location information, can be transmitted via broadcast, as acquiring it all at once at the beginning reduces the delay time required for the start of asset acquisition on the communication side.

[0061] (Examples of information specific to communications) Examples of information specific to communications (such as the Internet and CDNs (Content Delivery Networks)) are shown.

[0062] 1) For example, information related to Forward Error Correction (FEC) in packets transmitted via communication, such as the FEC method and parameters. 2) Information related to QoS (Quality of Service) control, such as packet loss rate, packet arrival time jitter, RTT (Round Trip Time), and end-to-end delay in the communication transmission path. 3) Information regarding buffering from the time the asset data is received until decryption begins, such as buffering time and buffering amount.

[0063] Furthermore, in broadcasting, especially for mobile broadcasting (such as 1-segment broadcasting in Japan), FEC and QoS are important, but the parameters for these differ from those for the communication channel. Therefore, broadcast-specific information only needs to be transmitted during broadcasting.

[0064] On the other hand, in communications, it is possible to select multiple audio and video data sets according to the bandwidth of the communication network, such as bitrate and frame rate. In this case, information that associates the attribute information of the selectable data (such as bitrate) with the asset ID may be transmitted. Such association information may also be transmitted in broadcasts.

[0065] Furthermore, the information to be associated may include information indicating whether there are multiple selectable assets, or, if there are multiple selectable assets, a list of the selectable assets may be shown.

[0066] [How to receive] In this embodiment, the receiving side begins receiving (acquiring) content after obtaining service information. The receiving method in this embodiment will be explained below with reference to the diagrams.

[0067] Figure 4A is a flowchart illustrating an example of the operation of the receiving side in the broadcast-communication collaboration service of Embodiment 1. Figure 4A shows an example of the operation in which the receiving device acquires the transmission path identification descriptor of this embodiment and determines the asset to be received.

[0068] First, the MPT table, which is included in service information transmitted on the entry point transmission path, such as a PA message or an MPT message, is obtained, and the transmission path identification descriptor contained within it is also obtained.

[0069] Next, the system interprets the transmission path identifier descriptor information, including whether the asset is transmitted using both broadcast and communication, and, if both transmission paths are used, the dependencies between the broadcast and communication assets (step S101).

[0070] Messages such as PA messages are stored in the payload of packets such as MMT packets, and the type of data stored is indicated by the packet header. Therefore, when an MMT package is received, the ID number in the packet header (corresponding to packet_id in MMT packets) is first referenced to obtain the MMT packet containing the PA message.

[0071] Next, the system determines which assets to receive based on the terminal's playback capabilities or whether a communication channel is available (step S102).

[0072] Here, we will explain an example of how to determine assets.

[0073] 1) If the receiving device is not connected to a communication network, it will decide to receive only broadcast assets. In this case, assets transmitted using both broadcast and communication will not be received.

[0074] 2) When the basic layer of video is transmitted via broadcast and the extended layer via communication, if only the basic layer can be played back, the system will decide to receive only the broadcast; if both layers can be played back, it will decide to receive both the broadcast and the communication. For example, suppose that time scalability of 60fps can be achieved with the basic layer alone, and 120fps with the basic and extended layers. In this case, if the receiving device can only decode and display up to 60fps, it will receive only the broadcast; if it can decode and display up to 120fps, it will receive both the broadcast and the communication data.

[0075] 3) Depending on the bandwidth of the communication network to which the receiving device is connected, the receiving device will determine which assets can be received from among multiple assets with different bit rates transmitted via communication. Here, for example, information such as the bit rate of each asset will be transmitted separately in service information, etc. If the reception conditions are poor due to congestion of the communication network, etc., and there are no assets that can be received stably, the device may decide not to receive the data from the communication side.

[0076] 4) In addition, if backup data is transmitted during communication in preparation for a deterioration of the broadcast reception environment, the system may decide which assets to acquire when the reception environment deteriorates. In this case, the receiving device should monitor the reception environment and determine whether to receive the communication assets based on indicators such as the error rate in the received data, and then perform the reception process.

[0077] 5) When receiving data from a communication source that is to be played back in sync with the audio and video data of the main program transmitted by broadcast (such as data for extended layers in video), special actions such as buffering the data before playback begins may be required to ensure synchronization between the data received by broadcast and the data received by communication. In this case, it is also possible to receive only assets from the communication source that do not require strict synchronization with the data transmitted by broadcast (e.g., frame-by-frame synchronization). For example, it may be decided to obtain assets such as HTML data, still images, and videos that do not require strict synchronization from the communication source.

[0078] The following explanation will return to the flowchart in Figure 4A.

[0079] Next, in step S103, it is determined whether to receive the asset transmitted by communication. If it is determined to receive the asset (yes in S103), the process proceeds to S104. If it is determined not to receive the asset (no in S103), the process proceeds to S105.

[0080] In S104, assets are received from both the broadcast and communication transmission paths, while in S105, assets are received only from the broadcast.

[0081] Figure 4B is a flowchart showing another example of the operation on the receiving side in the broadcast-communication collaboration service of Embodiment 1.

[0082] Compared to Figure 4A, Figure 4B adds a step (step S106) to receive service information specific to the communication when it is transmitted during the communication. Other aspects are the same as those described in Figure 4A, so their explanation is omitted.

[0083] If there is any service information specific to the broadcast, it will be received separately in a step not shown in the diagram.

[0084] Furthermore, if the receiving device does not support broadcast-communication integrated services, it should only receive broadcast assets. In this case, if there are assets that are transmitted using both broadcast and communication, those assets will not be received.

[0085] [Receiving device] Figure 5 is a block diagram showing an example of the configuration of a receiving device in Embodiment 1. Figure 5 shows an example of the configuration of a receiving device that implements the receiving method described in Figure 4A.

[0086] The receiving device 100 shown in Figure 5 comprises an identification information acquisition unit 101, an asset determination unit 102, a determination unit 103, a broadcast receiving unit 104, and a communication receiving unit 105.

[0087] The identification information acquisition unit 101 has the function of realizing step S101 shown in Figure 4A. Specifically, the identification information acquisition unit 101 acquires service information transmitted in the transmission path that serves as the entry point, and acquires the transmission path identification descriptor (auxiliary information) contained therein. The identification information acquisition unit 11 then interprets the information of the transmission path identification descriptor.

[0088] The asset determination unit 102 has the function of realizing step S102 shown in Figure 4A, and determines the asset to be received based on the terminal's playback capability or whether a communication channel is available.

[0089] The determination unit 103 has the function of realizing step S103 shown in Figure 4A, and determines (determines) whether to receive the asset transmitted by communication. Specifically, the determination unit 103 determines whether to receive the communication data, and if it determines to receive it, the broadcast receiving unit 104 and the communication receiving unit 105 receive the asset data determined in step 102.

[0090] If the determination unit 103 determines that it will not receive communication data, it will receive the data using only the broadcast receiving unit 14.

[0091] (Variation 1) This modified example describes a case where broadcasts are transmitted using TS, and communications are transmitted using DASH or RTP (Real-time Transport Protocol).

[0092] Figures 6A and 6B show an example of the data structure of service information in a broadcast-communication collaborative service according to Modification 1 of Embodiment 1. Figure 6A shows an example in which information about data transmitted in communication is stored in the PMT. Figure 6B shows an example of the data structure of the transmission path identification descriptor in this modification.

[0093] In this modified example, the PMT, which is an example of service information, stores a transmission path identification descriptor that shows attribute information such as the one described in Figures 1A and 1B.

[0094] In program information for TS such as PMT, only the location information of the data transmitted by the TS is indicated. Therefore, in this modified example, when content data is transmitted using communication, the location information of the data on the communication side is stored in the transmission path identification descriptor. More specifically, the attribute information includes flag information indicating whether communication is being used. This allows the sender to choose whether or not to include the location information of the data on the communication side, depending on the value of the flag information. Note that location information is not limited to being stored in the transmission path identification descriptor. A separate descriptor may be defined to store location information.

[0095] (Location information) Location information refers to information that indicates where data is retrieved from; for example, a PID for TS (Time System), or a URL or URI for communications.

[0096] Location information such as DASH's MPD (Media Presentation Description) or RTP's SDP (Session Description Protocol) may be stored.

[0097] Furthermore, location information may not only store the actual location data such as MPD or SDP, but may also store information indicating the source from which to obtain the actual location data. Here, information indicating the source from which to obtain the actual location data is, for example, information indicating the URL for obtaining MPD. However, if information indicating the source from which to obtain the actual location data is stored in the location information, a delay will occur in order to separately obtain the actual location data. Therefore, from the perspective of reducing the delay until the start of receiving data on the communication side, it is desirable to store the actual data directly in the location information.

[0098] Furthermore, the MPD in DASH is large in size because it contains various information about the source of the content. Therefore, instead of storing the MPD as is in the location information, it may be better to store a subset of the information, including only the URL of the content source and information about the segment's DTS and PTS.

[0099] Furthermore, it is desirable that the system be able to handle updates to location information such as MPD.

[0100] For example, a version number may be assigned to the location information. This allows the location information to be retrieved again when the version number is updated. Alternatively, to check whether the version number has been updated, information such as the transmission path identification descriptor in the PMT, which is transmitted periodically, may be checked sequentially. However, since this sequential check process is computationally intensive, it may be better to generate separate section data (called a transmission path identification section) to store location information and other data, and transmit it periodically.

[0101] In the receiving device, it is possible to determine whether the location information has been updated by checking the version number in the transmission path identification section. Alternatively, attribute information and location information may be transmitted in both the program information (such as PMT) and the transmission path identification section. Alternatively, only location information may be stored in the transmission path identification section.

[0102] Furthermore, after starting to acquire data on the communication side based on location information obtained from the broadcast, the updated location information may be acquired during communication. In this case, since the source of the location information is required, the source of the location information may be stored together with the actual location information data in the broadcast's transmission path identification descriptor or the like.

[0103] The receiving device can obtain updated information during communication by periodically accessing the location information source.

[0104] Furthermore, if a mechanism such as message exchange exists between the DASH content distribution server and the receiving device, the server may issue a message to the receiving device indicating that the location information has been updated. The receiving device may then re-acquire the location information upon receiving the message.

[0105] While it was explained that information regarding data transmitted during communication is stored in the PMT, this is not the only option. Attribute information and location information may be transmitted in a section separate from the PMT.

[0106] Furthermore, the method of acquiring data on the communication side may be switched based on whether the data transmitted in the communication is to be played back in sync with the audio and video in the broadcast.

[0107] For example, in the case of synchronous playback, the broadcast program information can be accessed from the data on the communication side as described above. In the case of non-synchronous playback, information for accessing the data on the communication side may be transmitted via data broadcasting using an AIT (Application Information Table) as defined in the Hybridcast specification of ARIB (Association of Radio Industries and Businesses). The receiving device receives the AIT in the form of a table or carousel, and acquires the data on the communication side based on the contents of the AIT. When the acquired data on the communication side is received, playback begins when an event message transmitted via data broadcasting is received.

[0108] Furthermore, even when broadcast data and communication data are played back synchronously, information such as AIT may be used. In this case, the start and end times of the communication data playback will be indicated separately within the data broadcast. The receiving device will play back the communication data according to these times.

[0109] In DASH's MPD and RTP's SDP, the content playback start time or decoding start time can be indicated, and these times are specified based on a reference clock defined for each multiplexing and transmission method, such as UTC (Coordinated Universal Time). In addition, the PTS (Presentation Time Stamp) and DTS (Decoding Time Stamp) of individual video frames and audio samples are also set based on the reference clock. On the other hand, in broadcasting, the PCR (Program Clock Reference) is the reference clock, so when synchronizing playback of data from the broadcast side and data from the communication side, information indicating the correspondence between their respective reference clocks is required.

[0110] Therefore, information indicating the correspondence between the reference clocks of the broadcast data and the communication data may be included in the program information such as the PMT in the broadcast. For example, this information may indicate that the time when the PCR value in the broadcast is N1 corresponds to the time when the UTC value in the communication is N2. This information may also be included in MPD or SDP.

[0111] Furthermore, the transmission path identification descriptor may also indicate transport layer protocol information, such as whether the data from the communication side is transmitted via UDP (User Datagram Protocol) or TCP (Transmission Control Protocol). This allows the receiving device to open the ports used by each protocol, or to determine whether it supports the protocol being used.

[0112] Furthermore, the communication side may indicate the identification information of the multiplexing format in the data, for example, whether it is DASH or RTP.

[0113] [How to receive] The following diagram illustrates an example of the operation of a receiving device in this modified example, where the broadcast is transmitted using TS and the communication is transmitted using DASH or RTP (Real-time Transport Protocol).

[0114] Figure 7A is a flowchart showing an example of the operation of the receiving side in a broadcast-communication collaboration service in a modified example 1 of Embodiment 1. Figure 7A shows an example of the operation of the receiving device when attribute information and location information are stored in the broadcast program information.

[0115] The operation of each step (steps S201 to S205) is the same as in the flowchart in Figure 4A, but in Figure 7A, the data for the broadcast side and the communication side were unified using assets from the MMT package, whereas in Figure 7A, the broadcast side uses TS data and the communication side uses DASH or RTP data. Other aspects are as explained in Figure 4A, so a detailed explanation will be omitted.

[0116] Figure 7B is a flowchart illustrating another example of the operation of the receiving side in a broadcast-communication collaboration service in a modified example 1 of Embodiment 1. Figure 7B shows an example of operation where, in the broadcast program information, access information for obtaining attribute information and location information is stored, but the actual location information is not stored.

[0117] Step S301 is the same as step S201, so its explanation is omitted.

[0118] Next, in step S302, it is determined whether to receive data from the communication side. In step S303, if data from the communication side is received, the process proceeds to S304; if data from the communication side is not received, the process proceeds to S306.

[0119] In step S304, location information is obtained according to the source of the location information for the communication data included in the broadcast program information.

[0120] Meanwhile, in step S305, communication data is acquired based on the location information obtained in S304, as well as broadcast data. In S306, only the data transmitted by broadcast is acquired.

[0121] (Attribute information) The attribute information is the same as that described in Embodiment 1, but since Embodiment 1 used MMT as an example, the following will describe an example where broadcasting uses TS and communication uses DASH or RTP.

[0122] 1) The data constituting the content, such as audio and video, may include attribute information indicating whether it is transmitted by (1) broadcasting only, or (2) a combination of broadcasting and communication.

[0123] Furthermore, the information may also indicate whether the audio and video data is transmitted by (1) broadcasting only, or (2) a combination of broadcasting and communication. This information allows the audio, video, still images, or metadata such as HTML files to be obtained from a separate communication network even if the main program is transmitted using broadcasting only.

[0124] 2) When audio or video is transmitted using both broadcasting and telecommunications, information indicating the relationship between the data transmitted in each method may be included as attribute information.

[0125] Here, for example, when achieving scalability (temporal resolution (e.g., 60fps → 120fps), spatial resolution (e.g., 4k → 8k), bit depth (e.g., 8bit → 10bit)), attribute information can be used to indicate that broadcast data is transmitted using the basic layer and communication data is transmitted using the extended layer. The attribute information may also indicate, as a use case, that while the frame rate is 60fps with broadcast data alone, it can be increased to 120fps by using communication data in conjunction. Furthermore, the attribute information may indicate that backup data for broadcasting is transmitted via communication. This allows the transmitting side to use the attribute information to switch to transmitting data via communication if broadcast reception deteriorates due to rain attenuation or other factors.

[0126] Furthermore, attribute information may indicate information for identifying related encoded streams. For example, attribute information could indicate the PID of a TS packet containing basic layer data transmitted in broadcast, a segment in DASH data transmitted in communication, or video data identification information such as a track ID in MP4.

[0127] Furthermore, attribute information may indicate data from the broadcast side and data from the communication side that are played back synchronously with each other. This allows the transmitting side to transmit video from multiple viewpoints, or the parent and child images in a picture-in-picture, via broadcast and communication, respectively.

[0128] 3) Information indicating whether audio and video transmitted by broadcast and audio and video transmitted by communication are played back in sync may be included as attribute information.

[0129] 4) The audio and video clock information transmitted by broadcast and the audio and video clock information transmitted by communication may be included as attribute information to indicate whether they are the same.

[0130] 5) Information indicating whether the data on the communication side is live content may be included as attribute information.

[0131] For example, if the transmitted content is not live content, data from a time later than the current time (T1) can be acquired. Therefore, by starting to receive data at a time (T2) which is the sum of the time taken from the content acquisition request to the start of reception (or an estimated value thereof) and the time spent buffering the data (△T) added to the current time, the broadcast data can be played back without delay. In this case, the communication data will be played back from time T2. On the other hand, if the transmitted content is live content, the broadcast data may be buffered for △T, and both the broadcast and communication data may be played back at time T2.

[0132] Furthermore, if the communication side multicasts or broadcasts content using RTP or similar methods, data from after the current time cannot be obtained, regardless of whether the content is live content or not. Therefore, it may be advisable to include information indicating whether the communication side's data is multicast or broadcast in the attribute information.

[0133] [Receiving device] Next, we will describe an example of a receiving device configuration when broadcasts are transmitted using TS and communications are transmitted using DASH or RTP (Real-time Transport Protocol).

[0134] Figure 8 is a block diagram showing an example of the configuration of a receiving device in Modification 1 of Embodiment 1. Figure 8 shows an example of the configuration of a receiving device that realizes the receiving method described in Figure 7B.

[0135] The receiving device 300 shown in Figure 8 comprises an identification information acquisition unit 301, a determination unit 302, a communication combined determination unit 303, a Loc information acquisition unit 304, a broadcast receiving unit 305, and a communication receiving unit 306. Note that the configuration of the receiving device that realizes the receiving method shown in Figure 7A corresponds to the case in Figure 8 where the Loc information acquisition unit 304 is not present.

[0136] The identification information acquisition unit 301 has the function to realize step S301 shown in Figure 7B. Specifically, the identification information acquisition unit 301 acquires service information transmitted in the transmission path that serves as the entry point, and acquires the transmission path identification descriptor (auxiliary information) contained therein. The identification information acquisition unit 301 then interprets the information of the transmission path identification descriptor.

[0137] The determination unit 302 has the function of realizing step S302 shown in Figure 7B, and determines the data to be received based on the terminal's playback capability or whether a communication channel is available.

[0138] The communication-enabled determination unit 303 has the function of realizing step S303 shown in Figure 7B, and determines (determines) whether to receive data transmitted by communication.

[0139] The Loc information acquisition unit 304 has the function to realize step S304 shown in Figure 7B, and acquires location data from the communication side.

[0140] (Modification 2) This modified example describes an example of a reception method when attribute information and location information are stored in the broadcast program information and transmitted.

[0141] [How to receive] Figure 9 is a flowchart illustrating an example of the operation of the receiving side in a broadcast-communication collaborative service in a modified example 2 of Embodiment 1. Figure 9 shows an example of the operation from when attribute information and location information are stored in the broadcast program information and transmitted, until the receiving device determines whether to synchronize playback of the broadcast content and communication content, and then plays them back.

[0142] Note that the operation in Figure 9 is based on the operation in Figure 7A, but is described as an example where the transmitted data is not an MMT asset but rather general content.

[0143] First, program information transmitted in the entry point transmission path is obtained, the transmission path identification descriptor contained therein is obtained, and content attribute information and communication content location information are obtained (step S401).

[0144] If the actual location information is not stored in the program information, the actual location information data is obtained using the same method as described in Figure 7B.

[0145] Next, the data to be received is determined based on the regeneration capability of the receiving device or whether the communication channel is available (step S402).

[0146] Next, it is decided whether to acquire communication content (step S403). If it is decided to acquire communication content (yes in S403), the process proceeds to step S404, where data is received from both the broadcast and communication transmission paths. If it is decided not to receive data (no in S403), the process proceeds to step S409, where data is received only from the broadcast, and in step S410, only the broadcast content is played back.

[0147] Next, it is determined whether to synchronize playback of the broadcast content and the communication content (step S405).

[0148] If it is decided to play them synchronously (yes in S405), the reference clocks of the broadcast content and the communication content are synchronized in step S406, and then both are played synchronously in step S407.

[0149] Furthermore, the synchronization of the reference clock in S406 may be performed prior to S404. This is because, in cases such as when playback starts after pre-buffering the received content data, decoding and playback may begin after confirming that data with PTS T1 in the broadcast content and data with PTS T1 in the communication content (assuming that the PTS of both are already synchronized) have been received. When determining whether the data to be played back synchronously is available, it is necessary that the reference clocks of both are synchronized.

[0150] Furthermore, in synchronizing the reference clock in S406, it is possible to align it with the reference clock used in either broadcasting or communications. For example, if PCR (Program Clock Reference) is used in broadcasting and NTP (Network Time Protocol) is used in communications, the reference clocks for broadcasting and communications can be synchronized by converting the NTP-based DTS or PTS used in video and audio to the PCR standard. Alternatively, the DTS or PTS used in broadcasting and communications may be converted to synchronize with the unique clock used within the receiving device.

[0151] Furthermore, if the attribute information includes information indicating whether the reference clock information of multiple components is the same, and the receiving end can obtain from the attribute information that the reference clock information is different, the reference clocks for broadcasting and communication may be synchronized using the method described above.

[0152] In addition, Figure 9 illustrates the operation when attribute information and location information are stored in the broadcast program information and transmitted. However, if synchronous playback is not required in the first place, the transmission path identification descriptor describing this information may not be stored in the broadcast program information.

[0153] Furthermore, while this modified example explains that synchronous playback occurs when attribute information and location information are stored in the broadcast program information, it is not limited to this. If the program information includes a transmission path identification descriptor, the receiving device may perform synchronous playback of the stream in which location information is described in the transmission path identification descriptor and the broadcast stream. In this case, the transmission path identification descriptor does not need to include attribute information indicating whether the streams transmitted by broadcast and communication are to be played back synchronously with each other.

[0154] In other words, the reception method in this modified example may include a reception step of receiving content transmitted using broadcast waves and a communication channel, respectively, and a playback step of performing the synchronization process and playing the content if auxiliary information for synchronizing the content transmitted via broadcast waves and the content transmitted via the communication channel is received.

[0155] [Receiving device] Figure 10 is a block diagram showing an example of the configuration of a receiving device in a modified example 2 of Embodiment 1. Figure 10 shows an example of the configuration of a receiving device that realizes the receiving method described in Figure 9.

[0156] The receiving device 400 shown in Figure 10 comprises an identification information acquisition unit 401, a determination unit 402, a communication combined determination unit 403, a Loc information acquisition unit 404, a broadcast receiving unit 405, a communication receiving unit 406, a synchronization determination unit 407, and a playback unit 408.

[0157] The identification information acquisition unit 401 to the communication receiving unit 406 are the same as the identification information acquisition unit 301 to the communication receiving unit 306 described in Figure 8, so their explanation is omitted.

[0158] The synchronization determination unit 407 has the function of performing the processing shown in step S405 in Figure 9.

[0159] The playback unit 408 decodes and plays back the broadcast content or communication content based on the playback method determined by the determination results of the communication-assisted determination unit 403 and the synchronization determination unit 407.

[0160] (Variation 3) The following examples illustrate situations that differ from those described above.

[0161] [Other 1] The transmission path is not limited to a combination of broadcasting and communications; it may also be a combination of the same type of transmission path, such as broadcasting and broadcasting, or communications and communications.

[0162] Furthermore, if package-level information such as a transmission path identifier descriptor is not provided, the location information for each asset, as shown in Figures 1A and 1B, may be interpreted to determine whether each asset is transmitted by broadcast or communication, or whether there are any assets within the package that are transmitted by communication.

[0163] (Location information) Location information may include URL information from which the asset was obtained. If the source is a URL, it is possible to determine whether the asset will be transmitted via broadcast or telecommunications based on whether that URL is a specific URL predetermined in the broadcasting service.

[0164] For example, when broadcast assets are transmitted in the same stream as message information such as MPT, the location information will indicate the ID of the packet containing the asset data (packet_id in the case of an MMT packet) as the destination for the broadcast asset, and for assets transmitted via communication, the URL will be indicated as the destination. Therefore, the location information may be used to determine whether or not an asset is transmitted via communication depending on whether or not the destination is a URL.

[0165] (A modified version of the MPT description) Figures 1A and 1B above illustrate the case where location information and individual transmission path identifier descriptors are defined for each asset, but this is not the only case. A separate field indicating the encoding scheme of the asset may also be provided. The encoding scheme is essential information during decoding, and it is desirable to signal it using information that can be obtained prior to decoding, such as MPT. For example, information such as stream_type in an MPEG-2 system can be used.

[0166] Furthermore, if the encoding scheme allows for scalable encoding, the encoding scheme may be used along with the indication of whether the asset is a base layer or an extension layer.

[0167] Furthermore, there are cases where multiple scalable assets are included in the same MMT package, such as when there are two videos with time scalability. In this case, information indicating which base layer assets correspond to the extended layer assets may be provided. For example, the asset ID of the corresponding base layer asset may be provided for the extended layer assets. Alternatively, the extended layer and base layer assets may be grouped together, and a group ID may be assigned to each asset.

[0168] This information can be indicated in the program information for broadcasting, even when broadcasting uses TS and communication uses DASH or RTP. For example, in a PMT descriptor, information can be described indicating the dependency between the video stream transmitted in broadcasting and the video stream transmitted in communication (such as the communication-side video stream corresponding to the extension layer), or the encoding method of the video and audio on the communication side. It may also indicate whether the broadcast-side stream and the communication-side stream are played back synchronously.

[0169] (Information regarding transmission lines) Information indicating whether content data is transmitted by broadcast only, or by a combination of broadcast and communication, may be stored in program information such as an EPG (Electronic Program Guide).

[0170] For example, when selecting content to watch or scheduling a recording from the EPG, if the data transmitted via communication cannot be played back or recorded due to reasons such as not being connected to a communication network or the receiving device not supporting data acquisition via communication, a message indicating this may be displayed.

[0171] Furthermore, information indicating whether the data transmitted via communication is available prior to the program's start time may be stored. In particular, with download-type systems such as DASH, downloading the data on the communication side prior to the program's start allows both the broadcast data and the communication data to begin playback at the program's start time.

[0172] For example, if data from the communication side can be acquired prior to the program start time, the receiving device may start receiving data from the communication side before the program start time. In this case, the start time of reception is determined so that a predetermined buffered amount of data, or data equivalent to the buffering time, can be received at the program start time.

[0173] Furthermore, it may operate similarly in other applications, such as scheduled recording of TV programs.

[0174] In broadcasting, users can select multiple programs at any time. Even if communication data for the next program on the currently viewed channel is received in advance, the benefit of receiving the data in advance is negated when the viewing program switches. Therefore, the system may be configured to buffer the communication data in advance for all programs that will be viewable after the viewing time and whose data is transmitted using both broadcasting and communication. If it is not possible to receive data for all programs, the system may select and receive data for programs within the receivable range.

[0175] Furthermore, program information such as the EPG may include information indicating whether the broadcast data and the communication data are played back synchronously. If synchronous playback is possible, the communication data may be buffered in advance. If synchronous playback is not possible, the reception of the communication data may begin after the reception of the broadcast data has started.

[0176] Even if the EPG does not contain such information, similar information can be obtained by analyzing the MPT in MMT, or the PMT when using TS for broadcasting, and a similar message can be displayed.

[0177] Furthermore, this information may be stored in the control information used when demodulating signals transmitted over a transmission line, such as TMCC signals in the Japanese broadcasting system (ISDB-T).

[0178] This allows for a determination of whether reception is necessary during demodulation, thereby speeding up the startup of the communication side.

[0179] (format) Furthermore, the HTTP-based file format is not limited to DASH; it can also be HTTP Live Streaming (HLS) or Microsoft Smooth Streaming (MSS), and since these methods also have content management information equivalent to MPD, the relationship between the broadcast data and the communication data can be shown using a mechanism similar to DASH.

[0180] Furthermore, on the communication side, a TS stream may be used, or a format in which a 4-byte timestamp field is added to the beginning of the TS packet may be used. Here, timestamped TS is used in many standards such as BD (Blu-ray® Disc) and the IPTV Forum.

[0181] [Other 2] (Attribute information and location information) In attribute information and location information, the attributes of the entire content and the individual attributes of streams such as audio and video may be stored separately.

[0182] For example, when using DASH on the communication side, attribute information for the entire content, such as whether the broadcast content and communication content are played synchronously, or whether the broadcast content is the basic layer of scalability and the communication content is the extension layer, may be stored in a transmission path identification descriptor or similar. On the other hand, individual attributes for each stream, such as information for identifying the video and audio that are to be played synchronously, may be stored in the MPD.

[0183] As an example of attributes for each stream, when associating video in a broadcast with secondary audio transmitted in a communication, the MPD may associate the PID of the TS packet containing the video with the audio track ID and other information within the DASH data that identifies audio tracks, segments, or Adaptation Sets and Representations. Here, the media identification information on the broadcast side may include not only the PID, but also the ID of the transport stream containing the TS packet with that PID, or the service ID.

[0184] Furthermore, the contents of the MPD may be updated, and the update information is managed by the DASH content distribution server. Therefore, by describing individual stream attributes in the MPD, it is possible to reduce the exchange of information between the broadcast content transmission device and the communication content transmission server, while storing the overall attributes in information acquired at the start of broadcast reception, such as PMT, allows for the rapid initiation of the communication content reception process.

[0185] Furthermore, both overall and individual attributes may be described in either a descriptor such as PMT in broadcasting, or in MPD, or in both. For example, in broadcasting, they may be described in an application control information descriptor such as AIT.

[0186] (Attribute information) Furthermore, the attribute information may include information indicating whether the clock information of streams transmitted over different transmission paths, such as broadcasting and communications, is synchronized with each other, as well as a method for synchronizing them, or a method for synchronizing them. In this case, the receiving device may perform clock synchronization based on this information.

[0187] For example, information to identify the following three methods may be described as attribute information.

[0188] Method 1) The streams to be synchronized are based on a common clock, and synchronization of their clocks is not required.

[0189] Method 2) Synchronize based on clock synchronization information that is transmitted separately from the stream, such as a descriptor in a PMT.

[0190] Method 3) Synchronize by referencing a separate stream containing information for clock synchronization, such as the TS timeline extension (13818-1:2013 / AMD6(2nd WD)) currently being standardized in MPEG.

[0191] In Method 3, the PID of the TS packet containing the clock synchronization stream may be included in the attribute information.

[0192] (Modification 4) For example, in Figures 6A to 7B, we explained how to store location information and other data in cases where broadcasts are transmitted using TS and communications are transmitted using DASH or RTP.

[0193] This modified example will explain how location information is stored using a specific example.

[0194] Figure 11 shows an example of the data structure of service information in a broadcast-communication collaborative service according to Modification 4 of Embodiment 1. Figure 11 shows an example of a descriptor (location information descriptor) that indicates location information such as MPD. Note that location information may be included in the transmission path identification descriptor described above. Therefore, the location information descriptor may be considered equivalent to the transmission path identification descriptor.

[0195] In this modified example, the descriptor is stored in the PMT, or in section data different from the PMT.

[0196] The information indicated by this descriptor includes a transmission format that indicates the type of entity data referenced by the location information, location information, and a field indicating synchronization information between PCR and communication-side data in broadcasting.

[0197] In this modified example, location information refers to the reference point for the actual location data. Here, an example is shown where the actual location data is in MPD format, with MPD indicated as the transmission format and synchronization information between PCR and NTP indicated as the synchronization information.

[0198] Since MPD is location information used in DASH, the transmission format may be DASH, and the location may be the reference to the MPD. If the transmission format can be identified by the extension of the URL in the location information, the transmission format field may not be included. Furthermore, there are two ways to transmit MPD: within a broadcast or over a communication network. When transmitting within a broadcast, the private section of the MPEG-2 TS is used. Therefore, the location information of the MPD may indicate the identification information of the TS packet containing the MPD within the transport stream, such as the PID of the private section, when transmitting within a broadcast, and information such as a URL when transmitting over a communication network.

[0199] [How to receive] The following describes an example of the operation of a receiving device in this modified example, where the receiving method involves analyzing a descriptor indicating location information to synchronize the playback of broadcast and communication content.

[0200] Figure 12 is a flowchart showing an example of the operation of the receiving side in a broadcast communication collaboration service in a modified example 4 of Embodiment 1.

[0201] First, in step S801, the location information descriptor stored in the PMT or similar is analyzed.

[0202] Next, in step S802, it is determined whether an MPD exists within the broadcast. If an MPD exists within the broadcast (i.e., an MPD is transmitted by the broadcast) (yes in S802), the process proceeds to step S803, where the actual MPD data is obtained from the TS packet with the PID indicated by the location information. On the other hand, if an MPD does not exist within the broadcast (no in S802), the MPD is obtained from the communication server based on the URL indicated in the location information.

[0203] Next, in step S805, based on the analysis results of MPD, the data to be retrieved from the DASH content is determined, and that data is acquired by download (or progressive download, etc.).

[0204] Next, in step S806, the broadcast content and the communication content are played back in sync based on the synchronization information contained in the location information descriptor, or, if MPEG-2 TS timeline extension is used, the synchronization information obtained from the TS packet containing the timeline extension information.

[0205] Furthermore, when using timeline extension, synchronization information does not need to be included in the location information descriptor. Also, if synchronized playback of DASH content and broadcast content is not required, synchronization information does not need to be transmitted. In this case, whether synchronized playback of both is necessary may be determined based on whether synchronization information exists in the location information descriptor or in the data for timeline extension. Whether synchronized playback is necessary may be indicated separately.

[0206] Furthermore, if synchronized playback is not required, step S806 can determine the start and end of DASH content playback based on user instructions or control commands in the Hybridcast application. In this case, it is assumed that the decision of whether or not to acquire data via communication was made prior to S801.

[0207] Figure 12 illustrates an example of acquiring MPD data and playing DASH content, but the process is similar when acquiring and playing data in other formats such as RTP or TS.

[0208] Furthermore, in timeline extension, an access unit is defined to store timeline extension information, and both a descriptor indicating location information and a descriptor indicating synchronization information can be stored within the access unit.

[0209] Therefore, location information and synchronization information may be shown together by timeline extension without sending a location information descriptor. In the current timeline extension, it is not possible to show the PID as location information, so the field indicating the scheme type of the location information may be extended to enable signaling of the PID.

[0210] Furthermore, the maximum section size for PMT is limited to 1021 bytes, and depending on the URL in the location information, this limit may be exceeded. While it is possible to split and store PMT sections, it is particularly desirable for PMT to be stored in a single section.

[0211] Therefore, if the section size of the PMT exceeds 1021 bytes, the location information descriptor may be sent in a section different from the PMT. Furthermore, if the location information indicates a PID, it can be contained within the PMT size limit and included in the PMT; therefore, the section storing the location information may be switched based on whether the location information is a PID or a URL.

[0212] (Variation 5) In Modification 4, it was stated that location information can also be displayed in the TEMI (Timeline and Extend Media Information stream) access unit in timeline extension.

[0213] Specifically, temi_location_descriptor can be used to describe the URL of the content on the communication side. With Temi_location_descriptor, you can describe the URLs of multiple content items, and there are no data size constraints like the section size in PMT, allowing for flexible operation regarding URL description.

[0214] On the other hand, when location information is indicated by the PID of a TS packet within the transport stream, the data size of the location information is small, and the delay time associated with obtaining the location information can be reduced by referring to the location information descriptor and acquiring the location information during PMT analysis. Therefore, it is desirable to use both the location information descriptor and the temi_location_descriptor within the TEMI access unit as storage locations for the location information.

[0215] The following describes an example of the syntax (data structure) of the location information descriptor in this modified example.

[0216] Figure 13A shows an example of the syntax of a location information descriptor in Modification 4 of Embodiment 1. Figure 13A shows an example of the syntax of a location information descriptor for storing location information in either a location information descriptor or a TEMI access unit.

[0217] In this modified example, the synchronization information with PCR is described using the temi_timeline_descriptor of the TEMI access unit and is not included in the location information descriptor. The semantics for each field shown in Figure 13A will be explained.

[0218] "data_format" is the same as the transmission format shown in Figure 11. In other words, it indicates the format of the playback control metadata used in the service, such as MPD in DASH, playback control metadata in the IPTV Forum's VOD (Video On Demand) specification, or TTS (Time-stamp TS) as defined by the IPTV Forum, etc. It may also indicate identification information of the stream itself, rather than metadata, such as TTS or MP4 files, or the AV encoded data itself. It may also indicate MPT in MMT, PA messages or MMT packets, assets, etc.

[0219] For example, temporal scalability can be achieved in video encoding schemes such as H.265. Here, suppose a basic layer of 60fps is transmitted in broadcast mode, and encoding data for an extension layer is transmitted in communication mode to increase the frame rate from 60fps to 120fps. In this case, the location information of the communication content only needs to be the URL of the encoding stream of the extension layer. Information such as the resolution and encoding scheme in the encoding stream can be obtained in the broadcast data that transmits the basic layer.

[0220] The "location_type" parameter identifies whether location information is indicated by the PID of the TS in broadcasting or by the URL in communications. Here, location_type=0 indicates the PID of the TS in broadcasting. Note that the location information descriptor can also be used in formats other than TS, such as MPEG's MMT (MPEG Media Transport). In formats other than TS, "location_type" does not need to be used. For example, other information to identify the location, such as the packet_id of the MMT packet in MMT, may be used.

[0221] "PID" indicates the PID of the TS packet. If the broadcast consists of multiple transport streams, the identification numbers of the transport streams may also be stored.

[0222] "url_location" indicates whether the URL of the communication content is stored in the location information descriptor or in the TEMI access unit. In this modified example, if url_location=0, it is assumed that the URL is stored in the location information descriptor. If url_location is 1, the URL of the communication content is stored in the temi_location_descriptor in the TEMI access unit.

[0223] "url_length" indicates the byte length of "url_path". "url_path" indicates the data of the URL.

[0224] Figure 13B shows an example of the syntax of the location information descriptor in Modification 4 of Embodiment 1. Figure 13B shows a different example of the location information descriptor syntax from that shown in Figure 13A. The difference from the syntax shown in Figure 13A is that the data_format field does not exist when the location information is stored in the TEMI access unit.

[0225] Here, the Temi_location_descriptor can indicate the TEMI service type in a field called service_type. This service type corresponds to data_format. Therefore, the data_format information will be shown in the service_type field.

[0226] Note that in the syntax shown in Figure 13A, the data_format field and the service_type of temi_location_descriptor may both exist. In this case, both are considered to represent the same information.

[0227] In temi_location_descriptor, it is also possible to indicate only the URL of the communication content without signaling the service_type. Therefore, when using the syntax in Figure 13A, the data_format of the location information descriptor may be used as the transmission format, and the service_type of temi_location_descriptor may not be signaled.

[0228] Figure 13C shows an example of the syntax of the location information descriptor in Modification 4 of Embodiment 1. Figure 13C shows a different example from the syntax of the location information descriptor shown in Figures 13A and 13B.

[0229] The difference from Figure 13A lies in the initial conditional branching based on url_location. That is, if url_location=0 and location information is present in the location information descriptor, the broadcast TS PID or communication URL is stored according to location_type. On the other hand, if url_location=1, location_type is not described in the location information descriptor, and instead, the temi_location_descriptor in the TEMI access unit indicates location_type in the service_type field, and stores the broadcast TS PID or communication URL in temi_location_descriptor.

[0230] Figure 13D shows an example of the syntax of the location information descriptor in Modification 4 of Embodiment 1. Figure 13D shows a different example of the location information descriptor syntax from those shown in Figures 13A to 13C.

[0231] The difference from Figure 13A is that url_location and location_type have been integrated. If location_type=0, it indicates the broadcast PID, and if location_type=1, it indicates the communication URL. Furthermore, if location_type=2, it indicates that the temi_location_descriptor in the TEMI access unit stores either the TS PID or the communication URL in the broadcast.

[0232] Note that the syntax data and data structure of the location information descriptor are not limited to the examples above. For example, it may be used in combination with other data, such as by integrating the location type and format type. Also, if the transmission format can be identified by, for example, the extension of the URL in the location information, the transmission format field may be omitted. Furthermore, if, for example, no location information descriptor exists, the location information may be indicated by temi_location_descriptor, and the url_location field may be omitted.

[0233] (Experimental variation 6) The following is an example of location information different from that described in Modification Example 5.

[0234] (Other examples of location information) For example, if there are two or more different types of timelines within the same program, each timeline type may contain multiple TEMI streams.

[0235] In this case, multiple location information entries may be stored in the location information descriptor. When storing multiple location information entries, loops equal to the number of TEMI streams are created within the location information descriptor, and the location information corresponding to each TEMI stream is stored. Alternatively, the correspondence between multiple location information loops in the location information descriptor and multiple TEMI streams may be indicated by, for example, the order of the location information loops matching the order of the ES loops (PMT second loop) that represent the TEMI streams. Furthermore, if there are two or more timelines, the location information may not be stored in the location information descriptor but in the temi_location_descriptor of the TEMI access unit, or the location information descriptor may be stored in the ES loop (PMT second loop).

[0236] (Location information update) Furthermore, it is desirable that the system be able to handle updates to location information, such as metadata.

[0237] For example, to update the location information, you can add a reload flag to the location information descriptor in the PMT and set reload=1. The receiving device will then recognize that the location information has been updated when reload=1 is set, and will re-retrieve the PID, URL, etc., stored in the location information. If the location information such as the PID and URL has not been updated, and only the data has been updated, then only the data may be re-retrieved.

[0238] Furthermore, information indicating whether the location information itself, or the content of data such as MPD (Data Platform Data) whose source is indicated by the location information, has been updated, may be displayed independently.

[0239] Alternatively, location information descriptors within the PMT, which are transmitted periodically, may be checked sequentially. However, since this sequential checking process is resource-intensive, section data for storing location information descriptors and other elements, as well as event sections to notify of updates, may be generated separately and transmitted periodically.

[0240] The receiving device can determine whether the location information has been updated by checking the version number in the section. Furthermore, when transmitting location information in the broadcast section, updating the version number in the location information section indicates that the metadata has been updated.

[0241] Furthermore, after starting to acquire data on the communication side based on location information obtained from the broadcast, it is also possible to acquire updated location information during communication.

[0242] [How to receive] Next, we will describe an example of the operation of a receiving device in this modified example, where a descriptor indicating location information is analyzed and broadcast and communication content is played back in sync.

[0243] Figure 14 is a flowchart showing an example of the operation of the receiving side in a broadcast communication collaboration service in a modified example 6 of Embodiment 1.

[0244] Step S804 in Figure 12 has been changed to steps S906, S907, and S908 in Figure 14. The other operations (steps S901 to S905) are the same as those in Figure 12 (steps S801 to S803, S805, and S806), so their explanation is omitted.

[0245] In step S906, it is determined whether the URL for obtaining the MPD or other data is stored in the location information descriptor or the TEMI access unit. If it is in the location information descriptor (yes in S906), the process proceeds to step S907, where the location information descriptor is analyzed and the MPD is obtained from the communication server based on the URL indicated in the location information. On the other hand, if it is determined in step S906 that it is stored in the TEMI access unit (no in S906), the MPD is obtained from the communication server based on the URL indicated in the temi_location_descriptor in the TEMI access unit.

[0246] Although Figure 14 shows an example where MPD is indicated as the format information, broadcast content and communication content can be played back synchronously using a similar flow even with other metadata.

[0247] Figure 15 is a flowchart showing another example of the operation on the receiving side in a broadcast-communication collaboration service in Modification 6 of Embodiment 1. Figure 15 shows the operation when the format information is metadata other than MPD. More specifically, it shows the operation when the format information indicates the format of actual data such as a stream, rather than metadata such as MPD, and when steps S907 and S908 shown in Figure 14 obtain the URL of the actual data of the stream instead of the URL of the metadata.

[0248] First, in step S1001, the receiving device analyzes the location information descriptor.

[0249] Next, it is determined whether the data indicated in the format exists within the broadcast (step S1002). If the data exists within the broadcast (yes in S1002), the process proceeds to step S1003, and the PID indicated in the location information is obtained. Here, it is not assumed that the broadcast contains actual data such as streams, so if the data exists within the broadcast, it is confirmed that the format is metadata.

[0250] Next, metadata files are obtained from the broadcast stream based on the PID (step S1004), and the communication data to be obtained is determined (step S1005).

[0251] On the other hand, if data exists in the communication in step S1002 (the answer is no in S1002), the process proceeds to step S1008 to determine whether the URL of the communication data exists in the location information descriptor or in the temi_location_descriptor of the TEMI access unit. Then, depending on the case, the URL is obtained in step S1009 or step S1010.

[0252] In step S1011, it is determined whether the format is meta information. If the format is meta information (yes in S1011), the process proceeds to step S1012, where meta information is obtained from the communication stream based on the URL, and then the process proceeds to step S1005 to determine the communication data to be obtained.

[0253] After the communication data to be obtained is determined in step Sp1005 or when it is determined in step S1011 that the format is not meta information, the process proceeds to step S1006 to obtain the communication data.

[0254] Finally, in step S1007, the broadcast content and the communication content are synchronously played back based on the synchronization information included in the location information descriptor or the synchronization information indicated by the time line extension of MPEG-2 TS.

[0255] (Variant 7) So far, it has been described that it is determined whether the broadcast content and the communication content are synchronously played back, and when they are synchronously played back, the acquisition and synchronous playback of the broadcast content and the communication content are performed, but it is not limited to this.

[0256] In this variant, information indicating whether the reference clock information of the audio or video transmitted by broadcast and the reference clock information of the audio or video transmitted by communication are the same is included in the service information (transmission identification descriptor, etc.), and the operation based on this information will be described.

[0257] [Information on whether the reference clocks (time lines) of broadcast and communication are synchronized] When transmitting content using broadcast and communication using MMT, DASH, RTP, or hybrid broadcast, etc., information indicating whether the reference clock information of the audio or video transmitted by broadcast and the reference clock information of the audio or video transmitted by communication are the same may be stored in the transmission identification descriptor, program information, EPG, EIT, etc.

[0258] For example, if content transmitted by broadcast and communication operates based on a common reference clock (e.g., timestamping), information indicating that it operates based on a common reference clock is stored. Also, if content transmitted by broadcast and communication operates based on different reference clocks, information indicating that it operates based on different reference clocks is stored.

[0259] Furthermore, if it is indicated that different reference clock information is used for broadcasting and communication, information necessary for reference clock synchronization (e.g., timeline extension information) may be shown. In addition, the storage location of the information necessary for reference clock synchronization, the type of information necessary for reference clock synchronization, and the method of reference clock synchronization may be shown.

[0260] In this case, the synchronization of the reference clock can be made to match the reference clock used in either broadcasting or communications.

[0261] For example, if PCR (Program Clock Reference) is used in broadcasting and NTP (Network Time Protocol) is used in communications, the type and method of reference clock synchronization may be described as being able to synchronize the reference clocks in broadcasting and communications by converting the NTP-based DTS or PTS in video and audio to the PCR standard. Alternatively, the type and method of reference clock synchronization may be described as converting the broadcast and communications DTS or PTS so that they synchronize with the unique clock used within the receiving device.

[0262] Alternatively, identification information indicating the types and methods of information required for reference clock synchronization may be analyzed, and reference clock synchronization may be performed using a method based on that identification information.

[0263] Furthermore, if there are multiple streams with different reference clocks, information indicating that there are multiple reference clocks with different reference clocks may be stored in the descriptor. A descriptor may be shown for each different reference clock, or it may be shown in a single descriptor. Information indicating the correspondence between the reference clock and the program that uses that reference clock may also be stored.

[0264] Information indicating whether the reference clocks are the same may be shown as a descriptor as described above, or other descriptors, tables, sections, etc. Alternatively, it may be indicated by whether the information necessary for clock synchronization (e.g., timeline extension information) is present. If the information necessary for clock synchronization is present, it may be considered that the clock information is different. Alternatively, the difference in clock information may be indicated using attribute information of the communication content (e.g., information about format and type, or location information or extensions described in the URL) shown in the transmission identification descriptor, etc. In the case of Hybridcast, it may be stored in the AIT controlled section or within the application.

[0265] If the receiving device determines that different clock information is being used, it synchronizes the reference clock information for broadcast and communication. If it determines that common clock information is being used, it determines that clock synchronization is unnecessary. For example, the receiving device may receive auxiliary information including difference information between the reference clock of the content transmitted by the broadcast wave and the reference clock of the content transmitted by the communication channel, and if the reference clock of the content transmitted by the broadcast wave and the reference clock of the content transmitted by the communication channel are different, it may synchronize the reference clock of the content transmitted by the communication channel to the reference clock of the content transmitted by the broadcast wave based on that difference information.

[0266] Furthermore, the receiving device may determine whether to actually synchronize the reference clocks of the broadcast content and the communication content, taking into account not only the identification result of whether the reference clock information of the data stored in the transmission identification descriptor is the same, but also user selections and settings via the user interface, the intentions of the content provider, service provider, or broadcaster, or the specifications and features of the receiving device, or the intentions of the receiver manufacturer.

[0267] Furthermore, when the receiving device determines that it should synchronize with the reference clock, it may actually start the synchronization after the determination, or it may start without waiting for the determination, when it acquires information about the synchronization. Alternatively, it may synchronize in conjunction with the time when it starts acquiring the communication content (for example, a certain amount of time before acquiring the content, or at the same time as acquiring the content).

[0268] Furthermore, if the clock regeneration of the reference clock (for example, a broadcast PCR) has not been completed (i.e., the clock has not been made usable by smoothing out effects such as jitter), the synchronization of the clock with the referenced clock may be started once the clock regeneration of the referenced clock information is completed.

[0269] If the EPG indicates whether the reference clocks for broadcast and communication are synchronized, then, similar to pro-buffering for communication content, it may be possible to determine whether reference clock synchronization is necessary before broadcasting begins and to initiate the synchronization of the reference clock. By synchronizing the reference clock earlier in this way, services can be delivered to viewers more quickly.

[0270] Furthermore, the information indicating whether the reference clocks are the same is not limited to the combination of broadcasting and communication; it can be applied when data is transmitted along the same path, or when data acquired from multiple paths such as broadcasting, communication, and storage formats uses different reference clock information.

[0271] Note that some or all of the functions and processes described in this modified example may be implemented by hardware or software, or a part of them can be implemented as hardware or software.

[0272] For example, when implemented by software, functions such as instructing the functions and processes described in this modified example, notifying the state of the reception function by PUSH, and obtaining it by PULL may be packaged and provided as API functions.

[0273] The API function can be executed from an application. When implemented as an application, it may be implemented as a resident application or an application such as HTML5 may also be used. The above API may be implemented for data transfer and state notification between applications.

[0274] Here, as functions of API functions related to information on whether the reference clocks are synchronized, there are, for example, the following 1) to 9).

[0275] 1) Obtain information on whether the reference clock information of broadcast content and communication content is the same. 2) Obtain information on whether reference clock synchronization is necessary. 3) Obtain the types and synchronization methods of each reference clock information. 4) Obtain information necessary for clock synchronization (such as timeline information). 5) Obtain the acquisition destination of information necessary for clock synchronization. 6) Take the reference clock information of the reference source as an argument and return the synchronized reference destination clock information. For example, if synchronization is not achieved, a value indicating that synchronization is not achieved is returned. 7) Instruct the start of synchronization of each other's reference clocks. 8) Ob Obtain the state of whether the reference clock synchronization has been achieved. 9) Notify that the reference clock synchronization has been achieved.

[0276] [Receiving method] The following diagram illustrates an example of how, in this modified example, the receiving device operates when attribute information and location information are stored in the broadcast program information, and the receiving device determines whether to synchronize playback of the broadcast content and the communication content.

[0277] Figure 16 is a flowchart illustrating an example of the operation of the receiving side in a broadcast-communication collaboration service according to Modification 7 of Embodiment 1. The operation shown in the flowchart in Figure 16 is applicable to broadcast-communication collaboration services using any combination of multiplexing schemes, such as MMT, DASH, and RTP. It is also applicable to Hybridcast.

[0278] Figure 16 assumes that broadcast content and communication content are synchronized, and therefore omits the determination of whether or not to synchronize broadcast content and communication content.

[0279] Furthermore, steps S1101 and S1102 perform the same operations as steps S401 and S402 in Figure 9, so their explanation will be omitted.

[0280] Next, in step S1103, the receiving device decides whether to acquire the communication content. If it decides to acquire the communication content (yes in S1103), it proceeds to step S1104, where it receives data from both the broadcast and communication transmission lines, and in the following step S1105, it determines whether the reference clocks for the broadcast content and the communication content are the same or different.

[0281] If it is determined in step S1105 that the reference clocks are different (yes in S1105), the process proceeds to step S1106 to synchronize the reference clocks of the broadcast content and the communication content, and then in step S1107, both are played back in sync.

[0282] On the other hand, if it is determined in step S1105 that the reference clocks are the same (no in S1105), the reference clocks are not synchronized, and synchronized playback is performed using a common clock.

[0283] Furthermore, after step S1105, a process may be performed to determine whether synchronization of the reference clock is necessary. For example, depending on the type and method of clock synchronization, it may be determined whether the receiving device's capabilities support clock synchronization, whether the receiver's specifications include clock synchronization, or whether the user performs clock synchronization. Then, based on step S1105 and the above determination result, a comprehensive determination may be made as to whether to synchronize the reference clocks for broadcasting and communication, and the process may proceed to step S1106 or step S1107.

[0284] Furthermore, the synchronization of the reference clock in step S1106 may be performed prior to step S1104. This is because, in cases such as when pre-buffering the received content data before starting playback, it may be necessary to confirm that data for which PTS is T1 in broadcast content and data for which PTS is T1 in communication content (assuming that the PTS of both are already synchronized) have been received before decoding and starting playback. And, when determining whether the data to be played back in sync with each other is ready, it is necessary that the reference clocks of both are synchronized.

[0285] Furthermore, in step S1106, if clock synchronization is not possible due to reasons such as the inability to obtain clock synchronization information or the specifications of the receiver, it may not be possible to provide the user with a service that integrates broadcasting and communication. In such cases, it may be possible to determine whether to receive the data transmitted by communication in step S1103 by determining whether synchronization is essential or whether playback is acceptable without synchronization.

[0286] In step S1103, in addition to the information that can be identified by the transmission path identification descriptor, the system may also consider all other factors, such as the user's selection and settings, the intentions of the content provider, service provider, or broadcaster, or the specifications and features of the receiving device, or the intentions of the manufacturer, to decide whether or not to actually receive the data transmitted via communication.

[0287] [Receiving device] Next, an example of the configuration of a receiving device that realizes the operation shown in Figure 16 will be described. Figure 17 is a block diagram showing an example of the configuration of a receiving device in modified example 7 of Embodiment 1.

[0288] The receiving device 1100 shown in Figure 17 comprises an identification information acquisition unit 1101, a determination unit 1102, a communication combined determination unit 1103, a Loc information acquisition unit 1104, a broadcast receiving unit 1105, a communication receiving unit 1106, a reference clock determination unit 1107, and a playback unit 1108.

[0289] The identification information acquisition unit 1101 to the communication receiving unit 1106 are the same as the identification information acquisition unit 301 to the communication receiving unit 306 described in Figure 8, so their explanation is omitted.

[0290] The reference clock determination unit 1107 has the function of performing the processing of step S1105 shown in Figure 16. In addition, the reference clock determination unit 1107 may determine whether the reference clocks are different and then perform processing to determine whether synchronization of the reference clocks as described above is necessary.

[0291] The playback unit 1108, based on the method determined by the determination result in the reference clock determination unit 1107, synchronizes the reference clock if it is different, then decodes the broadcast content or communication content and plays it back.

[0292] It should be noted that the functions, receiver configuration, and receiving method described in this modified example are merely examples and are not limited to them; any method that can achieve similar functions and effects is acceptable.

[0293] [Effects of Embodiment 1, etc.] As described above, according to this embodiment, a content transmission method, reception method, transmission device, and reception device can be realized that allow the receiving side to play content that uses both broadcasting and communication even if there is a delay in the start timing of receiving content via communication.

[0294] Specifically, one aspect of this embodiment is a method for transmitting content that can transmit content using broadcast waves and a communication channel, and when transmitting content using broadcast waves and a communication channel, it includes an information transmission step of transmitting auxiliary information for synchronizing the content transmitted via broadcast waves and the content transmitted via the communication channel, which, when received by the receiving side, causes the synchronization, at least using the broadcast waves.

[0295] As a result, when content is transmitted using broadcast waves and communication channels, auxiliary information is transmitted to synchronize the content transmitted using broadcast waves and communication channels. When the receiving end receives the auxiliary information, it can synchronize the content itself.

[0296] Here, for example, in the information transmission step, the auxiliary information may be transmitted prior to transmitting the content, and the auxiliary information may further include location information indicating where the content is to be obtained or information indicating where the location information is to be obtained.

[0297] Furthermore, for example, in the auxiliary information transmission step, the auxiliary information may also include and transmit difference information between the reference clock of the content via the broadcast wave and the reference clock of the content via the communication channel.

[0298] Furthermore, for example, in the auxiliary information transmission step, if the reference clocks of the content are different, the receiving side may be synchronized by transmitting the auxiliary information to synchronize the reference clock of the content via the communication channel with the reference clock of the content via the broadcast wave based on the difference information.

[0299] Furthermore, for example, the transmission method may include a generation step of generating the content in a format conforming to MMT (MPEG Media Transport), and a content transmission step of transmitting the content in the format generated in the generation step.

[0300] Furthermore, for example, in the generation step, the auxiliary information may be included in the message information, which is information related to the acquisition of the content, during generation.

[0301] Furthermore, a receiving method in one aspect of this embodiment includes a receiving step of receiving content transmitted using a broadcast wave and a communication channel, respectively, and a playback step of performing the synchronization process and playing the content if auxiliary information for synchronizing the content transmitted via the broadcast wave and the content transmitted via the communication channel is received.

[0302] This allows, for example, the acquisition of service information transmitted in an entry point transmission path such as a broadcast wave, and if auxiliary information is included in that information, a synchronization process can be performed.

[0303] Here, for example, in the receiving step, the auxiliary information may be received prior to receiving the content, and if the auxiliary information includes location information indicating the source of the content, the content may be received by acquiring the content based on the location information.

[0304] Furthermore, for example, in the receiving step, the auxiliary information may be received prior to receiving the content, and if the auxiliary information includes information indicating the source of location information indicating the source of the content, the location information may be obtained based on the information indicating the source of the location information, and the content may be received by obtaining the content from the obtained location information.

[0305] Furthermore, for example, in the playback step, in the reception step, the auxiliary information including difference information between the reference clock of the content via the broadcast wave and the reference clock of the content via the communication channel may be received, and if the reference clock of the content via the broadcast wave and the reference clock of the content via the communication channel are different, the content may be synchronized by synchronizing the reference clock of the content via the communication channel with the reference clock of the content via the broadcast wave based on the difference information, and the content may be played back.

[0306] Furthermore, in one aspect of this embodiment, the transmitting device is a content transmitting device capable of transmitting content using broadcast waves and a communication channel, and when transmitting content using broadcast waves and a communication channel respectively, it includes an information transmitting unit that transmits auxiliary information for synchronizing the content transmitted by the broadcast waves and the content transmitted by the communication channel, the auxiliary information for causing the synchronization when received by the receiving side, using at least the broadcast waves.

[0307] Furthermore, in one aspect of this embodiment, the receiving device includes a receiving unit that receives content transmitted using broadcast waves and a communication channel, respectively, and a playback unit that, upon receiving auxiliary information for synchronizing the content transmitted via broadcast waves and the content transmitted via the communication channel, performs the synchronization process and plays the content.

[0308] Furthermore, as described above, according to this embodiment, identification information indicating whether content including audio and video is transmitted using both broadcasting and communication, and information indicating dependencies between data transmitted on both transmission paths when broadcasting and communication are used together, may be generated and transmitted as content management information. Here, for example, the information indicating dependencies between data may include whether the data transmitted on both transmission paths are played back synchronously. Also, the information indicating dependencies between data may include whether the clock information of the data transmitted on both transmission paths is the same.

[0309] As a result, the transmission path through which content, including audio and video, is transmitted, and the dependencies between data transmitted on different transmission paths, can be obtained at the start of content reception. This reduces the delay time involved in determining which assets to receive and initiating the acquisition of communication content.

[0310] Furthermore, according to this embodiment, location information of data transmitted during communication may be included in the content management information. Here, the location information of the transmitted data may be, for example, MPD in MPEG-DASH. Also, the content management information may store information indicating the source from which the location information is obtained, rather than the actual location information data such as MPD.

[0311] As a result, the transmission path through which content, including audio and video, is transmitted, and the dependencies between data transmitted on different transmission paths, can be obtained at the start of content reception. This reduces the delay time involved in determining which assets to receive and initiating the acquisition of communication content.

[0312] Furthermore, according to this embodiment, the receiving device may analyze content management information to determine the transmission path for receiving content and the data to be received on each transmission path. For example, if the clock information of the data transmitted on both transmission paths is different, the device may acquire the necessary auxiliary information for clock synchronization between the data, synchronize the DTS and PTS in each data, decode, and reproduce. Also, when synchronously reproducing data transmitted on both transmission paths, the device may acquire the necessary auxiliary information for clock synchronization between the data, synchronize the DTS and PTS in each data, decode, and reproduce.

[0313] As a result, receiving devices that only receive broadcast data can reproduce broadcast data using the same operation as conventional broadcast reception, while also supporting the reproduction of communication data.

[0314] Furthermore, the receiving device can be provided with a mechanism for synchronizing and reproducing broadcast and communication data.

[0315] Furthermore, even if the clocks of data transmitted across multiple transmission lines are not synchronized, the clocks can be synchronized and the data played back synchronously by acquiring the necessary auxiliary information for clock synchronization.

[0316] (Embodiment 2) Embodiment 1 describes whether the stream transmitted by broadcast and the stream transmitted by communication are subject to synchronous playback, or a transmission method for transmitting information indicating dependencies between streams transmitted by both transmission paths, and a reception method for receiving such information.

[0317] In order to enable flexible content playback in the receiving device that involves user interaction, such as when the user selects the audio or video to be played from a selection of available audio or video, or when switching from a single-view full-screen display to a multi-view display for video, it is desirable to control playback in the application.

[0318] Most applications are based on browsers using the widely adopted HTML (Hyper Text Markup Language). When an application uses HTML, playback control can be performed by interpreting the HTML. HTML also offers the advantage of allowing for flexible description of content selection, switching, and layout changes.

[0319] For example, in Hybridcast as defined by the IPTV Forum, identification information indicating the existence of an application, such as ait_identifier_info(), is transmitted in data that is always referenced when receiving a broadcast program, such as the PMT descriptor in the TS and the MPT descriptor in the MMT. In a Hybridcast-compatible receiving device, if this identification information exists, application control information such as AIT is obtained from information corresponding to sections or data carousels in the TS, MMT messages or data carousels in the MMT, or downloaded via the communication network. Then, based on the application control information, the application, such as an HTML file, is obtained.

[0320] Hereafter, application control information and data related to the application itself will be referred to as application-related data. Other descriptive languages ​​besides HTML, such as XML, may be used to describe the application.

[0321] [Default method for sending service information] However, some receivers are not compatible with the application, or even if they are compatible, there is a time lag between obtaining the application and starting it up, which occurs after the start of tuning in broadcasting or communication.

[0322] Therefore, information regarding the default settings for the stream to be played and the display layout may be transmitted not as application-related data, but as data that is always acquired when tuning, such as PMT or MPT, or as data that can be acquired with lower latency than the time it takes to start the application, such as sections of other TS or MMT messages. Furthermore, information for performing advanced playback control may be transmitted as application data such as HTML.

[0323] For example, information related to playback control of streams transmitted in broadcasting or communications includes (a) information on scalability between streams, (b) information on switching between streams such as multilingual audio or video with different bitrates, (c) information on simultaneous display, such as information indicating multiple video streams that can be displayed simultaneously, and information on the layout when simultaneous display is performed. The default values ​​of this information are referred to as default playback control information.

[0324] For receiving devices that do not support an application, playback should be performed based on default values. On the other hand, for receiving devices that do support an application, playback should start based on default values, and after the application is launched, the playback operation should be switched based on user actions or control commands of the application.

[0325] The application may switch playback behavior only for information that can be switched by user action. For example, while simultaneous display is generally switchable by the user, regarding time scalability, the receiver may automatically decide on the playback behavior, assuming that the extended layer will always be used if the extended layer stream is available. In this case, information regarding time scalability is transmitted only in the default playback control information.

[0326] In this case, while playback control information is primarily handled by the application in TS, scalability information may be transmitted in the default playback control information. In this instance, only the information necessary to associate the streams of the base layer and the extension layer may be transmitted in the default playback control information or by descriptors defined in MPEG-2 TS, and the application may decide whether or not to decode the extension layer based on the application's control commands or user actions.

[0327] Furthermore, the application may switch playback behavior only for layout-related information. For example, time scalability and stream switching do not involve layout changes. Therefore, this information is transmitted only in the default playback control information. On the other hand, information regarding simultaneous display is expected to be switched by user action, so the application should be able to control playback behavior for this information.

[0328] (Example of default playback control information description) Next, we will explain an example of how to describe playback control information when an application switches the playback operation based on the default playback control information. Here, the default playback control information is stored in a descriptor, and the application control information is described in HTML.

[0329] Figure 18A is a diagram showing an example of the description of default playback control information in Embodiment 2, and Figure 18B is a diagram showing an example of video display according to the layout information of the default playback control information shown in Figure 18A. That is, in the default playback control information shown in Figure 18A, the layout information indicates that the video with PID=100 should be displayed in full screen, as shown in Figure 18B.

[0330] Furthermore, the layout information may simply indicate full-screen display without specifying information to identify the stream. In this case, if there are two or more video streams, information to identify the video to be played by default (application control information) can be separately indicated, allowing the application to operate in full-screen mode for the default video.

[0331] Figure 19A is a diagram showing an example of application control information in Embodiment 2, and Figure 19B is a diagram showing an example of video display according to the layout information of the application control information shown in Figure 19A.

[0332] In other words, the application control information shown in Figure 19A sets up two display areas, Area 1 and Area 2, using the Layout tag, and as shown in Figure 19B, it indicates that the video with PID=100 will be displayed in Area 1 and the video with PID=200 will be displayed in Area 2.

[0333] Therefore, the receiving device that receives the HTML application control information shown in Figure 19A will display videos with PID=100 and PID=200 in area 1 and area 2, respectively. Note that the information used to identify the video may be other than the PID, such as the URL or the packet ID of the MMT packet.

[0334] As shown in the examples in Figures 19A and 19B, the default playback control information includes two types of information: scalability and layout. The scalability information indicates that the type of scalability is time scalability, with the video with PID=100 being the base layer and the video with PID=200 being the extended layer. The layout information indicates that the video with PID=100 is displayed in full screen.

[0335] If the frame rate is 60fps with only the base layer, and becomes 120fps with the extension layer, then in the default state, only the video with PID=100 will be decoded and the 60fps equivalent video will be played back in full screen. Note that if the transmitted stream corresponds to the base layer or extension layer of scalability is indicated separately by the stream attribute information, such as the stream type of the MPEG-2 system, then this information does not need to be described in the default playback control information. Also, if it is separately specified that only the base layer will be decoded and played back in the default state, then the layout information may only indicate that it will be displayed in full screen.

[0336] Figure 20A is a diagram showing another example of the description of default playback control information in Embodiment 2, and Figure 20B is a diagram showing an example of video display according to the layout information of the default playback control information shown in Figure 20A. Figure 21A is a diagram showing an example of application control information in Embodiment 2, and Figure 21B is a diagram showing an example of video display according to the layout information of the application control information shown in Figure 21A.

[0337] In Figure 21A, function A is defined by the Script tag. Function A is a function that instructs the system to decode and play back the time scalability extension layer when a button on the screen is pressed, and takes the index numbers that identify the base layer and extension layer as arguments. The Body tag describes that function A should be called when a specific button on the remote control is pressed.

[0338] When the receiving device receives the HTML application shown in Figure 21A, and the button is pressed, it decodes the extended layer stream with PID=200 and plays back the video at 120fps in full screen.

[0339] Streams such as audio and video that are played by default may be specified separately using MPEG-2 system or MMT descriptors. For example, it is possible to group the streams to be played by default and associate the group ID with the stream. Alternatively, a descriptor can be defined to indicate the streams to be played by default, and this descriptor can include a list of PIDs and other information of the streams to be played by default.

[0340] Furthermore, default values ​​for the default playback control information may be defined in advance, and the default playback control information may not be sent if the parameter value is equal to the default value. For example, if full-screen display is set as the default layout, then layout information does not need to be included when full-screen display is selected. In this case, if scalability is not used and stream switching is not supported, the default playback control information does not need to be sent.

[0341] [How to receive] In this embodiment, the receiving side operates after obtaining service information, analyzing the default playback control information and the playback control information provided by the application. The receiving method in this embodiment will be described below with reference to the figures.

[0342] Figure 22 is a flowchart illustrating an example of the operation of the receiving side in Embodiment 2. Figure 22 shows an example of the operation when analyzing default playback control information and playback control information provided by the application to determine functions such as scalability and stream switching, and the layout when presenting content.

[0343] First, in step S701, the receiving device selects the content to be viewed.

[0344] Next, in step S702, the receiving side analyzes the default playback control information, determines the stream to decode and play back, and also determines the presentation method, such as the layout.

[0345] Here, the default playback control information may be included in the data that is always acquired when tuning, such as PMT or MPT, or it may be transmitted separately from application-related data, such as in an MPEG-2 TS section or MMT message.

[0346] If default playback control information is transmitted via a section or message, a default value for the default playback control information may be set, and the system may operate based on that default value until the section or message is received. Alternatively, the stream to be decoded and played may be determined by referring to a descriptor that indicates the audio or video stream to be played by default.

[0347] Next, in step S703, playback is started according to the determined playback control parameters.

[0348] Next, in step S704, it is determined whether to receive the application. If it is not received (no in S704), the process is terminated. On the other hand, if it is received (yes in S704), the process proceeds to step S705, where it is determined whether the application indicates a playback control function.

[0349] In step S705, if the playback control function is not indicated (no in S705), the process terminates. If the playback control function is indicated (yes in S705), the process proceeds to step S706, and the playback control information is updated according to the control commands and user actions based on the playback control function indicated in the application.

[0350] Next, in step S707, playback is performed according to the updated playback control parameters, and the process is terminated.

[0351] [Receiving device] Figure 23 is a block diagram showing an example of the configuration of the receiving device in Embodiment 2. Figure 23 shows an example of the configuration of the receiving device that realizes the operation of each step described in Figure 22.

[0352] The receiving device 700 shown in Figure 23 comprises a receiving unit 701, a default information analysis unit 702, an application information receiving unit 703, an application playback control information analysis unit 704, and a playback control execution unit 705. Since these operations are the same as those described in Figure 22, their explanation is omitted.

[0353] (Variation 1) The mechanism for updating default playback control information via an application can be applied in all cases where content is transmitted using broadcasting alone, communication alone, or a combination of broadcasting and communication.

[0354] Furthermore, the concept of the method in this embodiment can also be applied to the MPEG-2 TS timeline extension (13818-3:2013 / AMD6), which is currently being standardized by MPEG. In the TS timeline extension, when transmitting content using multiple multiplexed streams including at least one TS (referred to as the base TS), information is provided to synchronize the base TS and the other multiplexed streams for decoding and displaying the content.

[0355] Specifically, a timeline extension access unit is defined that contains information linking the reference clock of a multiplexed stream different from the basic TS with the PCR of the basic TS. This timeline extension access unit is stored in a PES packet, which is then packaged into a TS packet and transmitted. In the timeline extension access unit, the timeline extension information is represented in the format of the MPEG-2 system descriptor. By transmitting the timeline extension information in PES packets in this way, even if a discontinuity occurs in the PCR, the reference clock with the updated PCR can be quickly synchronized.

[0356] However, since PCR discontinuities generally do not occur within a single program, if timeline extension information is received at the start of program reception and the reference clocks are synchronized, resynchronization within the program is often unnecessary.

[0357] Therefore, if it is guaranteed that no PCR discontinuities will occur within the program, default values ​​for timeline extension information may be stored in a PMT or similar, and the timeline extension access unit may not be transmitted. For example, a descriptor similar to the one used to store timeline extension information in the timeline extension access unit can be stored in a section of a PMT or similar, and used as the default value for timeline extension information. Alternatively, the timeline extension access unit and the default information stored in a PMT or similar may be used in combination.

[0358] Even when a timeline extension access unit is transmitted, if it is guaranteed that no discontinuities in PCR will occur within the program or up to a specific time, after clock synchronization is performed based on the timeline extension access unit information received immediately after channel selection, resynchronization may be omitted until within the program or up to a specific time.

[0359] Furthermore, when transmitting timeline extension information via a section, timeline extension information corresponding to the updated PCR may be transmitted for a certain period immediately after a PCR discontinuity occurs. In the receiving device, if the section's version number has been updated, resynchronization will be performed based on the timeline extension information contained in that section. Note that after a PCR discontinuity occurs but before resynchronization, clock synchronization between streams cannot be guaranteed. In this case, the PCR discontinuity may be detected separately based on information indicating the PCR discontinuity, such as the TS discontinuity indicator. If a discontinuity is detected, the access unit of the media to be synchronized with the basic TS may be assumed to be at a fixed frame rate and decoded and displayed until resynchronization.

[0360] [Effects of Embodiment 2, etc.] As described above, according to this embodiment, when transmitting playback control information, including layout information related to the display of video in content including audio and video, or information indicating a stream that can be switched during playback, the default value of the playback control information is stored in data that is always received when selecting a channel, and information for updating the default playback control operation is stored in information related to the application such as Hybridcast and transmitted.

[0361] Here, playback starts according to the default playback control values, and if the receiving device supports the application, the default playback control information is updated according to user actions, etc., based on the playback control functions transmitted as application data.

[0362] This reduces the delay time involved in determining parameters related to playback control, such as content layout.

[0363] (Embodiment 3) As described above, the MMT method allows video and audio to be multiplexed, packetized, and transmitted over one or more transmission paths such as broadcasting and communication. The receiving device can receive packets transmitted over one or more transmission paths, extract desired packets from the received packets based on program information, and decode and display them.

[0364] In other words, the MMT method makes it possible to construct a program by receiving media (video, audio, subtitles, etc.) transmitted through multiple transmission lines. However, it cannot construct a program using both data downloaded and stored via broadcasting or communication, and data streamed from broadcasting or communication.

[0365] Therefore, Embodiment 3 will describe the transmission and reception methods, etc., when a program is constructed using both data downloaded and stored via broadcasting or communication, and data streamed from broadcasting or communication.

[0366] In the following, we will explain the broadcast-side multiplexing format in broadcast-linked services using MMT (MPEG Media Transport), which is standardized under MPEG, as an example.

[0367] In MMT, program information is transmitted through tables such as the MPT (MMT Package Table) or message information such as PA (Package Access) messages. The MPT describes the assets that make up the program (single media such as video and audio) and the location information for transmitting those assets. It is also possible to transmit a single asset through multiple transmission paths.

[0368] In this embodiment as well, the transmitting side transmits content (data or assets) using broadcast waves and communication channels, but transmits service information prior to transmitting the content.

[0369] [Service Information] Figure 24 shows an example of the data structure of service information in the broadcast-communication collaboration service of Embodiment 3. Figure 25 shows an example of the information included in the program configuration information descriptor of Embodiment 3.

[0370] As shown in Figure 24, the MPT contains a program configuration information descriptor, which provides information about the assets that make up the package. This information may include the following:

[0371] (Information about assets) 1) The program configuration information descriptor may include information indicating whether or not the assets constituting the program include stored assets (hereinafter referred to as stored assets) as asset information. Here, stored assets are, for example, assets stored on HDDs, internal memory, SSDs, SD cards and other memory, Blu-ray discs, etc.

[0372] 2) If the assets that make up the program include stored assets, information indicating the program's configuration may be included in the program configuration information descriptor as information about the assets.

[0373] For example, the program may include information indicating its structure, such as whether the program consists only of stored assets, whether it consists of stored assets and transmitted assets (hereinafter referred to as transmitted assets), or whether it consists of assets containing stored packets and transmitted assets. Furthermore, if the program consists of scalable encoded video, the program may include information indicating that the transmitted assets include a basic layer and the stored assets include an extended layer. The program may also include information indicating whether decoding and playback of stored assets are essential or not for program playback. For example, if the content of the stored assets is additional information, playback may be considered not essential, and information indicating that playback of the stored assets is not essential may be included.

[0374] Furthermore, if, for example, the receiving device does not support storage, or if no storage assets have been stored, the program can be played back using assets obtained from broadcasts or communications, without using storage assets, based on information indicating that decoding and playback of storage assets is not essential for program playback.

[0375] 3) Information regarding accumulated assets may be included in the program configuration information descriptor as asset information.

[0376] Here, for example, a) attribute information such as the storage format of the stored asset, the video / audio encoding method, and the transmission method and multiplexing method used to transmit the stored asset may be included. In this case, attribute information may be assigned to the stored asset in advance, and this attribute information may be included, for example, in the header of the stored asset or in program information that indicates the configuration of the stored asset. The receiving device may enable playback when the attribute information stored in the program information matches the attribute information of the stored asset. Also, for example, b) information regarding the capabilities and functions of the receiver necessary to play back the program containing the stored asset may be included. Also, for example, c) information (scrambling method, viewing restriction, limited reception, copyright information) and keys for restricting playback or viewing of the program may be included. In this case, the receiving device may compare the viewing restriction information stored in the program information with the stored asset to determine whether playback or viewing of the stored asset is permitted. The receiving device may also store pre-encrypted assets and decrypt the stored assets based on the transmitted key.

[0377] 4) If the assets that make up the program consist of transmission assets and storage assets, time information for synchronizing playback of the transmission assets and storage assets may be included in the program configuration information descriptor as information about the assets.

[0378] This time information may, for example, indicate the offset amount between the reference time of the transmitted asset and the reference time of the timestamp assigned to the stored asset. Alternatively, if the timestamp assigned to the stored asset is assigned based on PCR (Program Clock Reference) or NTP (Network Time Protocol), it may indicate the relative time to the transmitted reference time (PCR or NTP). Furthermore, if the stored asset is stored in a format with a random access table, the time information may be a time information table of the transmitted asset relative to the random access point of the stored asset.

[0379] (Location information) Within the MPT (Multiple Partition Table), location information for each asset is described. For example, if an asset exists in multiple locations, then multiple location entries will be described within the MPT.

[0380] Figure 26 shows an example of the information included in the location information descriptor of Embodiment 3. Figure 27 shows an example of the location type included in the location information descriptor of Embodiment 3.

[0381] As shown in Figure 26, the location information descriptor contains location information including the location type and an identifier that identifies the asset corresponding to that location type.

[0382] The location type represents the type of location information. For example, in the example shown in Figure 26, the location type value Value is 0xA0 (Value=0xA0), so as can be seen in Figure 27, it is stored locally. When the location type indicates that the data is stored locally, the location information will contain a local ID that uniquely identifies the stored data.

[0383] Here, the local ID may be, for example, the asset ID of the stored asset, or the packet ID. It may also be a 32-bit transport file identifier as defined in ARIB STD-B45, or a newly defined local ID may be used. Furthermore, various IDs such as network IDs and stream IDs may be combined and a local ID may be generated according to a predetermined naming convention. In addition, the local ID may be assigned in advance by the broadcasting station, transmitting station, content provider, etc., or the receiver may modify the ID and add additional information before assigning it. The receiving side may assign its own ID and prepare a table showing the correspondence with the ID assigned by the transmitting side to identify the data.

[0384] The above example uses the location type to indicate that the asset is stored locally and assigns a separate local ID, but the following method may also be used. For example, IPv4 may be selected as the location type and specified using a local-specific IP address. For example, if the IP address is 192.168.xxx.xxx, it may be assumed that the asset is stored locally, and the xxx.xxx part may be used to identify the stored asset.

[0385] Furthermore, a local ID will be assigned to the accumulated asset data (files). For example, it may be stored in the file header within the file. The local ID may be stored in advance by the broadcasting station, transmitting station, or content provider, or it may be stored on the receiving end.

[0386] The descriptor described above is just one example; a similar data structure may be used to construct the descriptor. Furthermore, the descriptors and identifiers in this embodiment may be stored in a table or message that displays package-level information, different from that of MPT. Other signaling information may also be used for transmission.

[0387] Although MMT was used as an example, the explanation is not limited to MMT; other formats such as TS and MPEG-DASH may also be used. For example, when using MPEG2-TS as the multiplexing method, the data may be stored in the PMT (Program Map Table). Also, when transmitting a combination of the MPEG2-TS method and other methods, the data may be stored in the TEMI (Timeline and External Media Information) access unit. When using MPEG-DASH, the data may be described in the MPD (Media presentation description).

[0388] (Asset configuration information descriptor) In the MMT method, it is also possible to transmit a single asset through multiple transmission lines. In this case, an asset configuration information descriptor that indicates the configuration of the asset may be stored.

[0389] Figure 28 shows an example of the information included in the asset configuration information descriptor of Embodiment 3. That is, the information indicating the asset configuration can include the following:

[0390] 1) The asset may be a storage asset, or the asset may include stored packets in the information indicating the asset's composition. Here, an asset includes stored packets when a single asset consists of packets transmitted from broadcast or communication and stored packets.

[0391] 2) If the asset is a storage asset, for example, a) information indicating the asset's configuration may include information such as whether the asset consists only of storage assets or consists of storage packets and transmission packets. Also, b) if the asset consists of scalable encoded video, information indicating the asset's configuration may include information such as whether the transmission packets include a basic layer and the storage packets include an extended layer. Also, c) as part of the asset's configuration, information indicating whether decoding and playback of the storage packets are mandatory or not may be included as part of the asset's configuration. For example, if the content of the storage packets is additional information, playback may be considered not mandatory, and it can be indicated that playback of the content of the storage packets is not mandatory.

[0392] This allows the program to be played back using packets transmitted from broadcasts or communications, without using stored packets, based on information indicating that decoding and playback of stored packets is not essential for asset playback, for example, if the receiving device does not support storage or does not store stored packets.

[0393] The asset configuration information descriptor described above is just one example; an asset configuration information descriptor may also be constructed using a data structure with similar functionality.

[0394] Furthermore, asset configuration information descriptors and identifiers may be stored in tables or messages that indicate package-level information, different from those used in MPT. They may also be transmitted using other signaling information.

[0395] Although MMT was used as an example, the explanation is not limited to MMT; other formats such as TS and MPEG-DASH may also be used. For example, when using MPEG2-TS as the multiplexing method, the data may be stored in the PMT (Program Map Table). Also, when transmitting a combination of the MPEG2-TS method and other methods, the data may be stored in the TEMI (Timeline and External Media Information) access unit. When using MPEG-DASH, the data may be described in the MPD (Media presentation description).

[0396] [Receiving method and receiving device] The following diagram illustrates an example of how the receiving device in this embodiment synchronizes and plays back stored assets.

[0397] Figure 29 is a flowchart showing the reception method in the broadcast-communication collaboration service of Embodiment 3. Figure 30 is a block diagram showing an example of the configuration of the receiving device in Embodiment 3.

[0398] The receiving device 30 shown in Figure 30 comprises a program information analysis unit 31, a stored asset identification unit 32, a stored asset information analysis unit 33, a determination unit 34, an asset acquisition unit 35, a synchronous presentation unit 36, and a synchronous control unit 37.

[0399] First, in step S11, the program information analysis unit 31 analyzes the program configuration information descriptor which serves as the entry point, and determines whether the program configuration includes stored assets.

[0400] Next, in step S12, the program information analysis unit 31 determines whether the elements constituting the program include stored assets. If it is determined that the elements constituting the program include stored assets (Yes in S12), the unit proceeds to step S13 and acquires information such as information indicating the program's structure, information about stored assets, and relative time information with respect to the stored assets. If it is determined that the program does not include stored assets (No in S12), the unit proceeds to step S17 and acquires transmitted assets transmitted by broadcasting or communication and performs synchronized playback.

[0401] Next, in step S14, the receiving device 30 obtains the location information and local ID of the stored asset. More specifically, if the asset is stored locally, the stored asset identification unit 32 searches for and identifies the asset corresponding to the local ID from among the assets stored in the receiving device 30. If there are multiple storage devices, the search may be performed from among the multiple storage devices, or only from within a specific storage device. The stored asset information analysis unit 33 then obtains the attribute information of the identified asset.

[0402] Next, in step S15, the determination unit 34 determines whether a program containing the stored assets can be played back, based on the information about the stored assets obtained in step S12 and the attribute information of the stored assets obtained in step S14.

[0403] If the program is determined to be playable (Yes in S15), the process proceeds to step S16, where the asset acquisition unit 35 acquires stored assets in addition to transmission assets such as broadcast and communication assets. Then, the synchronization control unit 37 performs synchronization control processing between the transmission assets and stored assets based on the acquired relative time information, and the synchronization presentation unit 36 ​​synchronizes and presents the program.

[0404] On the other hand, if it is determined in step S15 that the program cannot be played back (No in S15), the process proceeds to step S17, where the transmission asset transmitted via broadcast or communication is acquired, synchronized, and played back.

[0405] [Effects of Embodiment 3, etc.] As described above, according to this embodiment, it is possible to realize a method for generating program information, a method for playing back a program, and a playback device when multiplexing a service that links broadcasting and communication using a format such as the MMT (MPEG Media Transport) method, which is being standardized by the MPEG (Moving Picture Expert Group).

[0406] The MMT method multiplexes and packets video and audio, transmits them over one or more transmission lines such as broadcasting or telecommunications, and the receiving device receives the packets transmitted over one or more transmission lines, extracts the desired packets from the received packets based on program information, decodes them, and displays them.

[0407] The MMT method allows for the creation of programs by receiving media (such as video, audio, and subtitles) transmitted via multiple transmission lines. However, it cannot create programs using both data downloaded and stored via broadcasting or communication, and data streamed from broadcasting or communication.

[0408] Therefore, in this embodiment, the program information, which shows the structure of the program, includes an identifier indicating that one of the media constituting the program is stored data, and an identifier that can identify the stored media (file). Furthermore, the playback device analyzes the program information, identifies the media being transmitted via broadcast / communication and the stored data, and plays them back in synchronization.

[0409] This allows programs to be constructed using both data downloaded and stored via broadcasting and communications, and data streamed from broadcasting and communications. In addition to conventional broadcasting and communications integrated services, it enables the provision of services that integrate with stored files.

[0410] In this embodiment, a combination of transmission assets and storage assets was used as an example, but it is not limited to this. Transmission assets may be assets transmitted by broadcast, or assets transmitted by communication. They may also be assets transmitted by both broadcast and communication. As a service using storage assets, the following services may be provided. An identifier indicating the content of the service may be stored in the program information.

[0411] (Services using accumulated assets) 1) The assets that make up the program can be downloaded in advance, and only program information and time synchronization information (such as time reference information and relative values ​​of timestamps) can be transmitted during broadcasting, with the program being played back using the stored assets. In this case, the broadcaster can provide the program using the stored data to viewers in real time according to the program schedule. Viewers can view the stored data as if they were watching a program being broadcast in real time. Not only real-time broadcasting but also on-demand content provision is possible.

[0412] 2) Alternatively, program information and random access points may be downloaded and stored in advance along with the assets that make up the program, and the program may be configured in synchronization with the program information transmitted from broadcast or communication.

[0413] 3) It is also possible to provide a service where regular broadcast programs are transmitted using only broadcasting, and only during commercials are they linked with stored data. Commercial data could be downloaded in advance from broadcasting or communication, and then a program including the stored data could be provided to viewers when it is time for commercials. Alternatively, multiple sets of commercial data could be downloaded in advance, and the commercials to be provided could be selected according to the attributes and preferences of the viewers. Viewer attributes and preferences could be acquired and analyzed by the receiving device, or they could be acquired in cooperation with other apps or services.

[0414] 4) Broadcasters may allocate available frequency resources to other services by providing services linked to stored data. In this case, the program information may store information indicating that the frequency resources used for that service are being provided to other services.

[0415] 5) The system may have a function that allows viewers to switch to a program that is linked to stored assets. For example, a viewer who is only viewing transmitted assets may switch to a program that includes stored assets using a user interface such as a remote control.

[0416] 6) The receiving device or viewer may also configure whether it is permissible to access stored assets to configure a program. Alternatively, it may be stipulated that stored assets can only be accessed to configure a program if the program information is authenticated as trustworthy. The program information may include information such as keys that indicates the program information is trustworthy.

[0417] (EPG) 7) The information to be displayed as an EPG (Electronic Program Guide) may be obtained from program information or from signaling information specifically for EPGs.

[0418] 8) Information such as whether the program is provided by broadcast, communication, storage, a combination of these, or as an extended service may be displayed in the EPG. Alternatively, only the functions that the receiving device can provide may be displayed. For example, if the receiving device does not have the function of storage or receiving communications, information regarding storage or communications may not be displayed.

[0419] 9) In a program that includes stored assets, information indicating whether or not the program's stored assets are stored may be displayed on the EPG. Information indicating whether or not the program's stored assets can be presented may also be displayed on the EPG. This may be indicated by text or by the background color of the program in the EPG. Content that includes stored assets may display information regarding viewing restrictions or copyright.

[0420] 10) When reserving a program containing stored assets from the EPG, the EPG may indicate whether the stored assets can be downloaded from broadcast, downloaded via communication, or viewed via streaming. If the receiving device does not have a means of communication, only whether broadcast download is possible may be indicated. Viewers select and reserve from the available download reservation methods.

[0421] Download reservations can be made in advance by the viewer via the EPG, or the receiving device can automatically download the content. The viewer can choose the download method, or the receiving device can choose it automatically.

[0422] Furthermore, the EPG display and the functions of the receiving device may be changed depending on the time period during which downloads are possible via broadcast, downloads are possible via communication, and streaming is possible via communication.

[0423] In addition to the EPG display described above, the information may also be presented using other methods besides EPG.

[0424] Although the transmission method, reception method, etc., according to one or more embodiments of the present invention have been described above based on embodiments, the present invention is not limited to these embodiments. As long as they do not depart from the spirit of the present invention, various modifications that a person skilled in the art can conceive of may be applied to these embodiments, and forms constructed by combining components from different embodiments may also be included within the scope of one or more embodiments of the present invention.

[0425] For example, in each of the above embodiments, each component may be implemented by being composed of dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. [Industrial applicability]

[0426] The present invention is useful as a method for transmitting and receiving content that can transmit content using broadcast waves and communication channels. [Explanation of symbols]

[0427] 11. Identification Information Acquisition Unit 14, 104, 305, 405, 1105 Broadcast receiving unit 30, 100, 300, 400, 700, 1100 Receiver 31 Program Information Analysis Department 32. Accumulated Asset Identification Department 33. Accumulated Asset Information Analysis Department 34, 103 Judgment section 35. Asset Acquisition Section 36 Synchronized presentation section 37 Synchronization Control Unit 101, 301, 401, 1101 Identification Information Acquisition Unit 102 Asset Decision Department 105, 306, 406, 1106 Communication receiving unit 302, 402, 1102 Decision Section 303, 403, 1103 Communication-enabled determination unit 304, 404, 1104 Loc information acquisition section 407 Synchronization Determination Unit 408, 1108 Reproduction Department 701 Receiver 702 Default Information Analysis Unit 703 App Information Receiving Unit 704 Application Playback Control Information Analysis Unit 705 Playback control execution unit 1107 Reference Clock Determination

Claims

1. A method for transmitting content that enables the transmission of content using broadcast waves and communication channels, A generation step in which the aforementioned content is generated in a format conforming to MMT (MPEG Media Transport), A content transmission step that transmits the content in the format generated in the generation step, When transmitting the content using both a broadcast wave and a communication channel, the information transmission step includes transmitting, at least using the broadcast wave, information indicating that there is content to be transmitted via the communication channel, information indicating the format of metadata for playback control of the content transmitted via the communication channel, and auxiliary information including location information indicating the source of the content transmitted via the communication channel. In the generation step, the auxiliary information is generated by including it in the message information, which is information related to the acquisition of the content. Sending method.

2. A content transmission device capable of transmitting content using broadcast waves and communication channels, A generation unit that generates the aforementioned content in a format conforming to MMT (MPEG Media Transport), A content transmission unit that transmits the content in the format generated by the generation unit, When transmitting content using both broadcast waves and a communication channel, the system includes an information transmission unit that transmits, at least using the broadcast waves, information indicating that there is content to be transmitted via the communication channel, information indicating the format of metadata for playback control of the content to be transmitted via the communication channel, and auxiliary information including location information indicating the source of the content to be transmitted via the communication channel. The generation unit generates the auxiliary information by including it in the message information, which is information relating to the acquisition of the content. Transmitter.

Images