Receiving device, client terminal device, and program

Abstract

To allow a client terminal side to use accompanying information (event information and the like) accompanying video and audio in a system that provides web resources based on broadcast resources from a receiving device to a client terminal device.SOLUTION: In a receiving device, a transcoding unit transcodes at least any data of video resources and audio resources extracted from a received broadcast signal into data in a format available on a web platform, and outputs the data as web resources. An event processing unit generates event information including accompanying information accompanying at least one of video and audio based on the resources extracted from the broadcast signal. A web resource providing unit transmits the web resources output by the transcoding unit and the event information generated by the event processing unit to a cooperation destination client terminal device through communication.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a receiving device, a client terminal device, and a program. [Background technology]

[0002] In conventional technology, television receivers utilize various resources (video, audio, subtitles, etc.) contained in broadcast waves. The utilization of resources contained in such broadcasts has basically been limited to television receivers and the like that have a broadcast tuner.

[0003] In response to this, technologies that enable resources contained in broadcast waves to be utilized on web platforms are becoming increasingly widespread. Such technologies enable terminal devices other than television receivers (e.g., web client terminals) to utilize resources contained in broadcasts. For example, television receivers that comply with the ATSC 3.0 standard (ATSC stands for "Advanced Television Systems Committee"), a standard used outside of Japan, can function as distribution servers. In other words, such television receivers can distribute the received broadcast video to other terminal devices.

[0004] On the other hand, in Japanese broadcasting standards (ISDB-T, etc.), remote viewing services are provided by recording devices provided by various electronics manufacturers. Non-Patent Document 1 describes remote viewing.

[0005] One standardized technology is the Hybridcast device linkage function, which is a mechanism for linking television receivers and viewing terminals. This function is called "Hybridcast Connect" or "Hybrid Connect." The technical standard for the Hybridcast device linkage function provides discovery, connection, and data transmission / reception protocols. Non-Patent Document 2 includes the provisions for the Hybridcast device linkage function. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] "About Remote Viewing of Digital Broadcasting," Broadcasting Service Advancement Promotion Association, [online], [downloaded April 10, 2021], Internet<URL:https: / / www.apab.or.jp / remote-viewing / pdf / remote-viewing.pdf> [Non-patent document 2] "IPTV Forum Standards and Operational Guidelines," IPTV Forum, General Incorporated Association, IPTVFJ-STD0010 Version 2.3, IPTVFJ-STD0011 Version 2.6, IPTVFJ-STD0013 Version 2.9, [October 2, 2020], Internet<URL:https: / / www.iptvforum.jp / download / input.html> Summary of the Invention [Problem to be solved by the invention]

[0007] However, the conventional techniques have problems to be solved.

[0008] In the remote viewing mechanism described in Non-Patent Document 1, the combination of a television receiver and a viewing terminal (and the application program running on that terminal) is fixed for each manufacturer. The reason for this is thought to be that each manufacturer uses its own proprietary technology for discovery and connection protocols between the television receiver and the viewing terminal, and for content protection. In other words, there is a problem that the interoperability between television receivers and viewing terminals does not improve. Due to these circumstances, user convenience does not improve, and the spread of the technology is hindered.

[0009] The mechanism of the hybrid cast terminal linkage function described in Non-Patent Document 2 has a problem in that the viewing terminal cannot acquire broadcast wave resources to play videos or display subtitles.

[0010] In other words, in order to improve the interoperability between television receivers and viewing terminals using standard technology, it is desirable to expand the standard technology for Hybridcast's terminal linkage function so that the various resources contained in broadcast waves can be utilized by viewing terminals.

[0011] In this situation, the problem to be solved by the present invention is as follows: In addition to video and audio, broadcast signals contain various information such as event information, subtitle information, and program information. Furthermore, metadata information can be obtained by analyzing the video and audio. When broadcast content is made available on a viewing terminal, it is desirable to make available not only the video and audio but also the event information and other information contained in the broadcast signal as web resources on the viewing terminal. Note that event information in a broadcast signal is transmitted, for example, in the form of an EM (event message).

[0012] In other words, it is desirable to be able to utilize information other than video and audio (typically events) contained in broadcast waves, as well as metadata information obtained by analyzing video and audio, through a standard web interface. This information other than video and audio, and metadata information obtained by analyzing video and audio, will be collectively referred to as "accompanying information." In other words, it is desirable to be able to utilize the accompanying information about broadcasts received by a television receiving device (receiving terminal device) in a format that conforms to a standard web interface on a viewing terminal (client terminal device).

[0013] The present invention has been made based on the recognition of the above-mentioned problems, and aims to provide a receiving device, a client terminal device, and a program that enable accompanying information (event information, etc.) to be used on the client terminal device side in a system in which a receiving device (television receiver) provides web resources based on broadcast resources to a client terminal device (viewing terminal). [Means for solving the problem]

[0014] [1] In order to solve the above problem, a receiving device according to one aspect of the present invention comprises a receiving unit that receives a broadcast signal, an event processing unit that generates event information based on resources extracted from the broadcast signal, and a web resource providing unit that transmits the event information generated by the event processing unit to a linked client terminal device via communication.

[0015] [2] Furthermore, one aspect of the present invention is that the above-mentioned receiving device further includes a transcoding unit that transcodes at least one of data of a video resource or an audio resource extracted from the received broadcast signal into data in a format usable by a web platform and outputs the data as a web resource, wherein the event processing unit generates the event information including accompanying information associated with at least one of the video resource or the audio resource, and the web resource providing unit transmits the event information generated by the event processing unit to the client terminal device together with the web resource output by the transcoding unit.

[0016] [3] Also, one aspect of the present invention is that in the above-mentioned receiving device, the accompanying information includes any one of an event message, subtitles, text superimpositions, an application information table (AIT), service information (SI), and data broadcasting content extracted from the broadcast signal, or metadata obtained by analyzing at least one of the video resources or the audio resources extracted from the broadcast signal.

[0017] [4] Furthermore, one aspect of the present invention is that in the above-mentioned receiving device, the web resource providing unit transmits the event information to the client terminal device of the partner using the function of terminal linkage communication with the partner client terminal device in Hybridcast Connect (a terminal linkage function of Hybridcast).

[0018] [5] Furthermore, one aspect of the present invention is that in the above-mentioned receiving device, the accompanying information includes event message data extracted from the broadcast signal, and the web resource providing unit performs control to prevent the event information including the event message data of the same content from being sent multiple times to the client terminal device.

[0019] [6] Also, one aspect of the present invention is that in the above-mentioned receiving device, the web resource providing unit receives information on the playback position of the web resource in the client terminal device from the client terminal device, and transmits the event information associated with a specific playback position in the web resource to the client terminal device at a timing based on the received playback position information.

[0020] [7] Also, one aspect of the present invention is that in the above-mentioned receiving device, the web resource providing unit identifies each of the multiple client terminal devices and transmits the event information associated with a specific playback position in the web resource to each of the multiple client terminal devices at a timing based on the playback position information received from each of the client terminal devices.

[0021] [8] Furthermore, one aspect of the present invention is that in the above-mentioned receiving device, the event processing unit generates the event information as a media-timed event (MTE) and inserts the generated media-timed event into the web resource output by the transcoding unit, and the web resource providing unit transmits the web resource into which the media-timed event has been inserted to the client terminal device.

[0022] [9] Furthermore, one aspect of the present invention is that in the above-mentioned receiving device, the media-timed event generated by the event processing unit includes, as the accompanying information, any of subtitles, superimposed characters, an application information table (AIT), service information (SI), and data broadcasting content extracted from the broadcast signal.

[0023]

[10] In addition, one aspect of the present invention is that the above-mentioned receiving device further includes a data analysis unit that associates the subtitles with a playback position obtained by matching an audio analysis result, which is a result of analyzing the extracted audio resource, with the extracted subtitles, and the event processing unit inserts the media-timed event including the subtitles associated with the playback position into the web resource.

[0024]

[11] In addition, one aspect of the present invention is that the above-mentioned receiving device further includes a data analysis unit that generates metadata obtained by associating a video analysis result, which is the result of analyzing the extracted video resource, with program information extracted from the broadcast signal or information from an external web server device, and the event processing unit inserts the media-timed event including the metadata generated by the data analysis unit into the web resource.

[0025]

[12] Furthermore, a client terminal device according to one aspect of the present invention includes a receiving terminal cooperation control unit that receives the event information from the receiving device described in [1] above.

[0026]

[13] Furthermore, a client terminal device according to one aspect of the present invention comprises an application engine that runs a web application that receives and utilizes the web resources from the receiving device described in [2] above, and a receiving terminal cooperation control unit that executes the processing necessary to utilize the web resources as a cooperative operation with the receiving device using Hybridcast Connect (a Hybridcast terminal cooperation function), and receives event information from the receiving device that includes accompanying information associated with at least one of video or audio based on resources extracted from a broadcast signal received by the receiving device.

[0027]

[14] Furthermore, one aspect of the present invention is that in the above-mentioned client terminal device, the receiving terminal cooperation control unit receives the event information through terminal cooperation communication with the receiving device.

[0028]

[15] Furthermore, one aspect of the present invention is that the above-mentioned client terminal device determines whether the event information has been processed based on the identification information contained in the received event information, and performs control to suppress processing of the event information that has already been processed.

[0029]

[16] Another aspect of the present invention is that in the above-mentioned client terminal device, the event information is received as a media-timed event (MTE) included in the web resource received by the web application.

[0030]

[17] Another aspect of the present invention is a program for causing a computer having a receiving unit that receives broadcast signals to function as a receiving device described in any one of [1] to

[11] above.

[0031]

[18] Another aspect of the present invention is a program for causing a computer to function as the client terminal device described in any one of

[12] to

[16] above. [Effects of the Invention]

[0032] According to the present invention, it is possible to distribute, as event information, accompanying information that is associated with video resources or audio resources included in broadcast resources received by a receiving device to a client terminal device. [Brief explanation of the drawings]

[0033] [Figure 1] 1 is a block diagram showing a configuration of a system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a functional block diagram showing the functional configuration of a receiving terminal device according to the embodiment. [Figure 3] FIG. 2 is a functional block diagram showing the functional configuration of a client terminal device according to the embodiment. [Figure 4] FIG. 10 is a schematic diagram (part 1) showing an example of a sequence of requests and responses between a receiving terminal device and a client terminal device using the terminal linking function of the extended Hybridcast in the embodiment. [Figure 5] 5 is a schematic diagram (part 2) showing an example of a sequence of requests and responses between a receiving terminal device and a client terminal device, similar to FIG. 4 above. [Figure 6] 10 is a schematic diagram showing an example of data of an event message transmitted from a receiving terminal device to a client terminal device using the sendtext function of the Hybridcast terminal linkage function according to the embodiment. FIG. [Figure 7] 10 is a schematic diagram showing an example of an MTE that the event processing unit according to the embodiment inserts into video content for online distribution. FIG. [Figure 8] FIG. 10 is a schematic diagram showing an example of an MTE (Media Timed Event) in which subtitle data is inserted as an event in the embodiment. [Figure 9] 10 is an example of a list of MTE identifiers that can be used depending on the type (application, etc.) in the embodiment. [Figure 10] FIG. 2 is a block diagram showing an example of the internal configuration of each device constituting the system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0034] Next, an embodiment of the present invention will be described with reference to the drawings.

[0035] In this embodiment, the receiving terminal device 2 has a function for enabling broadcast wave resources to be utilized on a web platform. In addition, the client terminal device 3 has a function for utilizing web resources provided by the receiving terminal device 2. Each of the receiving terminal device 2 and the client terminal device 3 realizes the above functions through an extended function of Hybridcast Connect (Hybridcast terminal linking function). Note that Hybridcast is a standardized technology that combines broadcasting and communications.

[0036] In this embodiment, the video resources and audio resources extracted from the broadcast signal are distributed as online video content used on a standard web platform from the receiving terminal device 2 to the client terminal device 3. In addition, data other than video and audio (accompanying information, including event information, and also including meta-information obtained by analyzing the video and audio) can be distributed from the receiving terminal device 2 to the client terminal device 3 by at least one of two methods.

[0037] The first method is to use two-way terminal link communication in the Hybridcast terminal link function. This method will also be described in step S115 of Figure 4. When using this method, text data can be sent from either side to the other between a receiving terminal device 2 and a client terminal device 3 that are linked using the Hybridcast terminal link function technology, using the "sendtext" function. It goes without saying that data other than text data can also be sent as text data by appropriately coding it.

[0038] The second method is to use a Media Timed Event (MTE). A Media Timed Event is a standardized method for inserting event information associated with time into media data such as video. When this MTE is used, information can be transmitted from the receiving terminal device 2 to the client terminal device 3. In other words, the MTE is inserted into video content distributed as online video. Since the MTE is associated with a specific position (playback position, relative time from the beginning of the video) within the online video, it is suitable for storing data for synchronization with the video.

[0039] MTE is also described in the document "Requirements for Media Timed Events, W3C Interest Group Note, 25 June 2020" (https: / / www.w3.org / TR / media-timed-events / ).

[0040] FIG. 1 is a block diagram showing the configuration of a system according to this embodiment. As shown in the figure, the system 1 includes a receiving terminal device 2, a client terminal device 3, an antenna 4, and a router 8. The receiving terminal device 2 is also called a "receiving device." In the system 1, the receiving terminal device 2 and the client terminal device 3 work together using a hybridcast terminal linking function. This allows the client terminal device 3 to present content based on broadcast resources provided by the receiving terminal device 2. Note that while this figure shows one receiving terminal device 2, one client terminal device 3, and one router 8, the system 1 may include any number of each of these devices. For example, the system 1 may be configured by connecting multiple receiving terminal devices 2 or multiple client terminal devices 3 to a single local network. The functions of each device that make up the system 1 will be described below.

[0041] The receiving terminal device 2 receives a broadcast signal (radio frequency, RF) via an antenna 4. The receiving terminal device 2 demodulates and decodes the digital broadcast signal to extract resources such as video, audio, and subtitles contained in the broadcast signal. In other words, the receiving terminal device 2 is a so-called television receiver. The receiving terminal device 2 communicates with the client terminal device 3 via a local area network using the Internet Protocol (IP). In other words, the receiving terminal device 2 can communicate with the client terminal device 3 via a router 8. The receiving terminal device 2 can access the router 8 via wireless or wired communication. An application program (hereinafter sometimes referred to as an "app") can be run on the receiving terminal device 2. The receiving terminal device 2 operates in cooperation with the client terminal device 3 using the terminal cooperation function of Hybridcast. The detailed functional configuration of the receiving terminal device 2 will be described later with reference to another diagram.

[0042] The client terminal device 3 is a terminal device that cooperates with the receiving terminal device 2. Specifically, the client terminal device 3 may be a device such as a PC (personal computer), a tablet device, a smartphone, or a smart speaker. The client terminal device 3 can run an app. An app for operating in cooperation with the receiving terminal device 2 runs on the client terminal device 3. The client terminal device 3 operates in cooperation with the receiving terminal device 2 using the terminal cooperation function of Hybridcast. The client terminal device 3 transmits various requests to the receiving terminal device 2 for viewing web resources. The client terminal device 3 can use the web resources according to the results of the requests. In other words, the client terminal device 3 presents the web resources to the user (viewer). Note that the client terminal device 3 itself does not need to have a tuner function for directly receiving broadcast signals.

[0043] The antenna 4 is for receiving the above-mentioned broadcast signals. The antenna 4 receives the broadcast signals as airwaves and supplies them to the receiving terminal device 2 as electric signals.

[0044] The router 8 is one of the devices that make up the local area network. The router 8 forwards IP packets sent from a source device connected to the local area network to a destination device. In the illustrated configuration, the router 8 forwards IP packets, enabling the receiving terminal device 2 and the client terminal device 3 to communicate with each other.

[0045] 2 is a functional block diagram showing the functional configuration of the receiving terminal device 2. The receiving terminal device 2 has a function for enabling resources such as video, audio, subtitles, event information, etc. contained in broadcast waves (broadcast signals), or other resources, to be utilized on a web platform. Specifically, as shown in the figure, the receiving terminal device 2 is composed of a tuner unit 201, a descrambler 202, a demultiplexer 203, a data broadcasting processing unit 211, a video decoder unit 212, an audio decoder unit 213, a subtitle decoder unit 214, a data broadcasting engine 221, a communication unit 231, a streaming receiving unit 232, a demultiplexer 233, a video decoder unit 242, an audio decoder unit 243, a subtitle decoder unit 244, an application control unit 251, an application engine 252, an application launcher 253, a transcoding unit 261, a web resource providing unit 262, a client terminal cooperation control unit 263, a video output unit 271, an audio output unit 272, a data analysis unit 281, and an event processing unit 282.

[0046] Each functional unit of the receiving terminal device 2 is configured using, for example, electronic circuits. At least some of the functions of the receiving terminal device 2 can be realized by a computer and a program. Each functional unit also has a storage means as needed. The storage means is, for example, a variable in the program or a memory allocated by the execution of the program. Furthermore, non-volatile storage means such as a magnetic hard disk drive or a solid state drive (SSD) may be used as needed. The functions of each unit of the receiving terminal device 2 are as follows:

[0047] Note that there are also communications between functional units that are omitted in Fig. 2. These will be explained separately in this embodiment.

[0048] Tuner unit 201 receives a broadcast signal, tunes (selects a channel) the received broadcast signal, and passes the selected broadcast signal to descrambler 202. The received broadcast signal is a broadcast signal such as ISDB-T (terrestrial digital broadcasting) or ISDB-S (satellite digital broadcasting). Tuner unit 201 is also called a "receiving unit."

[0049] The descrambler 202 descrambles the received broadcast signal and outputs the descrambled signal. The descrambler 202 passes the descrambled signal to the demultiplexer 203.

[0050] The demultiplexer 203 extracts and outputs individual resource signals from the multiplexed broadcast signal. Specifically, the demultiplexer 203 extracts and outputs the data broadcast signal, video signal, audio signal, subtitle signal, and other signals as needed from the signal output from the descrambler 202.

[0051] The data broadcast processing unit 211 processes the data broadcast signal extracted by the demultiplexer 203. The data broadcast signal is content described in, for example, BML (Broadcast Markup Language).

[0052] The video decoder unit 212 receives the video resources extracted by the demultiplexer 203 and decodes them as video. The video decoder unit 212 passes the video resulting from the decoding to the video output unit 271 and the transcode unit 261.

[0053] The audio decoder unit 213 receives the audio resource extracted by the demultiplexer 203 and decodes it as audio. The audio decoder unit 213 passes the audio resulting from the decoding to the audio output unit 272 and the transcode unit 261.

[0054] The subtitle decoder unit 214 receives and decodes the subtitle resources extracted by the demultiplexer 203. The subtitle decoder unit 214 passes the subtitle data resulting from the decoding to the video output unit 271. Furthermore, the subtitle decoder unit 214 may also pass the subtitle data to the transcode unit 261.

[0055] The data broadcasting engine 221 processes the content output from the data broadcasting processing unit 211. The data broadcasting engine 221 passes the processing result to the video output unit 271 as a video.

[0056] The communication unit 231 communicates with external devices. The communication unit 231 communicates with the outside world using, for example, the Internet Protocol (IP). The function of the communication unit 231 enables the receiving terminal device 2 to communicate with the client terminal device 3 and the like. The communication unit 231 can also communicate with an external (for example, outside the home) server device (such as a web editing information management server device 91) described below.

[0057] The streaming receiving unit 232 receives a signal of content that is distributed via streaming via communication (the Internet, etc.) The streaming receiving unit 232 passes the received signal to the demultiplexer 233.

[0058] The demultiplexer 233 extracts data of each resource from the content signal received by the streaming receiving unit 232. The extracted data includes video, audio, subtitles, etc. The demultiplexer 233 passes the video data to the video decoder unit 242, the audio data to the audio decoder unit 243, and the subtitle data to the subtitle decoder unit 244. The demultiplexer 233 may also extract resources of types other than video, audio, and subtitles.

[0059] The video decoder unit 242 decodes the video data passed from the demultiplexer 233 and outputs the decoded video.

[0060] The audio decoder unit 243 decodes the audio data passed from the demultiplexer 233 and outputs the decoded audio.

[0061] The subtitle decoder unit 244 decodes the subtitle data passed from the demultiplexer 233 and outputs the decoded subtitles (text).

[0062] The application control unit 251 controls application programs running on the receiving terminal device 2. Specifically, the application control unit 251 starts and stops specific application programs and performs other management operations. The application control unit 251 performs control based on Hybridcast technology (regulations).

[0063] The application engine 252 is an environment for running application programs that run on the receiving terminal device 2. The application programs are, for example, content written in HTML5. The content written in HTML5 includes page definitions for presentation on a screen or the like and executable code.

[0064] The application launcher 253 has a function for launching a specific application program on the application engine 252 .

[0065] The transcoding unit 261 receives the decoded video and audio data and converts (transcodes) it into a format (net video distribution format) for distribution via communication. Examples of the converted net video distribution format include MPEG-DASH (DASH is an abbreviation for "Dynamic Adaptive Streaming over HTTP") and HLS (HTTP Live Streaming). In other words, the transcoding unit 261 transcodes resource data (at least one of video resources and audio resources) extracted from the received broadcast signal into data in a format usable by the web platform, and outputs the data as web resources. The data output by the transcoding unit 261 is passed to the web resource providing unit 262 and can be further provided to the client terminal device 3. Note that parameters used when the transcoding unit 261 transcodes content can be controlled and set by the application control unit 251 and the client terminal cooperation control unit 263.

[0066] Web resource providing unit 262 receives content in the form of online video distribution output from transcoding unit 261 and manages distribution to external devices (client terminal device 3, etc.). Web resource providing unit 262 can also store the video content output by transcoding unit 261 for at least a predetermined period of time. Web resource providing unit 262 can provide the video content to client terminal device 3, etc. via communication unit 231. In other words, in response to a viewing request from an external client terminal device 3, web resource providing unit 262 transmits the web resource output by transcoding unit 261 to client terminal device 3 via communication.

[0067] Furthermore, the web resource providing unit 262 generates endpoint information for a web client (such as the client terminal device 3) to use video content. The endpoint information is information that indicates the access destination of the video distribution content as seen from the web client side. The endpoint information generated by the web resource providing unit 262 is information that can be passed to the client terminal device 3 side via the client terminal cooperation control unit 263. This endpoint information allows the client terminal device 3 to know the access destination for obtaining the video content.

[0068] The web resource providing unit 262 of this embodiment receives and manages information of an event message (EM) included in a broadcast signal received by the receiving terminal device 2. The web resource providing unit 262 communicates with the client terminal cooperation control unit 263 and provides event information associated with the event message to the client terminal device 3 via communication. The event information is data generated by the event processing unit 282. A specific method by which the web resource providing unit 262 distributes the event information to the client terminal device 3 is as follows. First, the web resource providing unit 262 may transmit the event information to the cooperative client terminal device 3 using the function of terminal cooperation communication with the cooperative client terminal device 3 in the terminal cooperation function of Hybridcast. Second, the web resource providing unit 262 transmits a media-timed event (MTE) provided from the event processing unit 282 (described later) to the client terminal device 3. The event processing unit 282 inserts the MTE into video and audio content (web resources) distributed to the client terminal device 3.

[0069] The client terminal cooperation control unit 263 controls cooperation between the receiving terminal device 2 and the client terminal device 3. Specifically, the client terminal cooperation control unit 263 controls cooperation between the receiving terminal device 2 and the client terminal device 3 by using the terminal cooperation function of Hybridcast. The terminal cooperation function of Hybridcast itself is an existing technology, but the client terminal cooperation control unit 263 realizes a function that is an extension of the terminal cooperation function of Hybridcast as a function unique to this embodiment.

[0070] Specifically, the client terminal cooperation control unit 263 controls cooperation with the client terminal device 3 using the Hybridcast terminal cooperation function (including extended functions), and transmits information necessary for using web resources to the client terminal device. The client terminal cooperation control unit 263 transmits at least information on the location (endpoint, etc.) for using the web resources to the client terminal device 3. In other words, the client terminal cooperation control unit 263 communicates with the web resource providing unit 262 and transmits the results of processing for cooperation between terminals to the client terminal device 3. In this case, the client terminal cooperation control unit 263 transmits information to the client terminal device 3 via the API of the Hybridcast terminal cooperation function, or transmits information via the WebSocket protocol.

[0071] The client terminal cooperation control unit 263 performs the following process in the extended hybridcast terminal cooperation function. That is, the client terminal cooperation control unit 263 provides the client terminal device 3 with the endpoint information described above. The client terminal cooperation control unit 263 also provides the client terminal device 3 with an API (Application Program Interface) for using web resources transcoded from broadcast resources. The client terminal cooperation control unit 263 also acquires information for controlling the transcoding unit 261 passed from the client terminal device 3, and passes this control information (parameters for transcoding processing, etc.) to the transcoding unit 261 via the application control unit 251. That is, the client terminal cooperation control unit 263 controls the transcoding unit 261 based on information from the client terminal device 3. The cooperation between the receiving terminal device 2 and the client terminal device 3 will be described in more detail later.

[0072] The video output unit 271 integrates the videos passed from the video decoder unit 212, the subtitle decoder unit 214, the video decoder unit 242, the subtitle decoder unit 244, the data broadcasting engine 221, etc., and outputs the integrated video. In other words, the video output unit 271 presents the video to the user (viewer). Specifically, the video output unit 271 outputs a video signal to be presented to a display device or the like. Note that in this embodiment, the receiving terminal device 2 may be configured not to have the video output unit 271. Even in this case, the receiving terminal device 2 can provide the video (including information equivalent to subtitles) to the client terminal device 3 as a web resource.

[0073] The audio output unit 272 outputs the audio passed from the audio decoder unit 213 or the audio decoder unit 243 to the outside. Specifically, the audio output unit 272 outputs an audio signal to a speaker, earphones, or the like. Note that in this embodiment, the receiving terminal device 2 may be configured not to have the audio output unit 272. Even in this case, the receiving terminal device 2 can provide the audio to the client terminal device 3 as a web resource.

[0074] The data analysis unit 281 analyzes data and manages metadata associated with video data and audio data. Specifically, the data analysis unit 281 analyzes video data received and extracted by the receiving terminal device 2, and manages information about people or objects included in the video, superimposed text (text) synchronized with the video, and the like as metadata. The data analysis unit 281 also analyzes audio data received and extracted by the receiving terminal device 2, and manages information about the content of utterances included in the audio data (such as the speaker and the text of the utterance) as metadata. The data analysis unit 281 can provide the metadata managed in association with the video data and audio data to the client terminal device 3. Specifically, the data analysis unit 281 cooperates with the web resource providing unit 262 to provide the above metadata to the client terminal device 3.

[0075] The data analysis unit 281 can, for example, compare the subtitle data extracted from the broadcast signal with the analysis results of the audio data extracted from the broadcast signal (for example, the results of analysis including voice recognition processing, etc.), and generate such subtitle processing results.

[0076] The data analysis unit 281 also has a function of matching the above metadata with program information (information such as EPG-API) received by the receiving terminal device 2. EPG is an abbreviation for "Electronic Programming Guide."

[0077] The video analysis and audio analysis techniques used by the data analysis unit 281 are existing techniques.

[0078] The event processing unit 282 performs processing related to the event information contained in the received broadcast signal.

[0079] The event processing unit 282 has a function to generate media timed event (MTE, MediaTimedEvents) data as event metadata for online video distribution. MTE data is MPD or EMSG. MPD stands for "Media Presentation Description." EMSG is a format for storing in-band messages. The event processing unit 282 inserts the MTE into the online video (video and audio) output by the transcoding unit 261. The event processing unit 282 works in conjunction with the web resource providing unit 262. In other words, when the web resource providing unit 262 provides web resources to the client terminal device 3, the event processing unit 282 inserts the MTE into the online video to be provided.

[0080] Instead of transmitting an event using an MTE, the event processing unit 282 can also transmit text data representing information about the event to the client terminal device 3 using two-way terminal linkage communication in the terminal linkage function of Hybridcast.

[0081] That is, the event processing unit 282 generates event information including accompanying information accompanying at least one of video and audio based on resources extracted from the broadcast signal. Here, the accompanying information is any of an event message, subtitles, superimposed characters, application information table (AIT), service information (SI), and data broadcasting content extracted from the broadcast signal. Alternatively, the accompanying information is metadata obtained by analyzing at least one of the video resources and audio resources extracted from the broadcast signal.

[0082] With the above-described functional configuration, the receiving terminal device 2 can present (use) resources such as video, audio, and subtitles received as broadcast signals on a web browser (web platform) of the client terminal device 3. In other words, by using the receiving terminal device 2, broadcast resources can be used on the web platform.

[0083] The receiving terminal device 2 may optionally have the following functions: The receiving terminal device 2 functions as a web server for the client terminal device 3, and the web server may be configured to store video data (e.g., mp4) generated by the transcoding unit 261. This may enable, for example, rewinding and viewing (jumping to a specific playback position) based on a request from the client terminal device 3, or video-on-demand (VOD) viewing. The receiving terminal device 2 may also have a video playback player function or HTML content capable of playing video. This allows video playback even when the client terminal device 3 does not have the video playback player function. The receiving terminal device 2 may also be configured to use chunked-transfer-encoding when delivering video. This makes it possible to achieve ultra-low latency delivery in CMAF (Common Media Application Format).

[0084] 3 is a block diagram showing a schematic internal functional configuration of the client terminal device 3. As shown in the figure, the client terminal device 3 includes a communication unit 331, an application engine 352, an application launcher 353, and a receiving terminal cooperation control unit 363. The client terminal device 3 can be realized using electronic circuits. At least some of the functions of the client terminal device 3 may be realized by a computer and a program. The functions of each unit are as follows:

[0085] It should be noted that the functions constituting the client terminal device 3 do not necessarily have to all be realized on the same hardware, and the client terminal device 3 may be realized in a form distributed (loosely coupled) across multiple devices (hardware).

[0086] The communication unit 331 communicates with external devices. The communication unit 331 communicates with the outside world using, for example, the Internet Protocol (IP). The function of the communication unit 331 enables the client terminal device 3 to communicate with the receiving terminal device 2 and the like.

[0087] The application engine 352 executes application programs. As shown in the figure, for example, a web application is one of the programs executed on the application engine 352. Specifically, the application engine 352 runs a web application that receives and uses web resources from the receiving terminal device 2.

[0088] The web application has the function of displaying web content, including content written in HTML (HyperText Markup Language) and MPEG-DASH video playback players.

[0089] The application launcher 353 launches a specific application program on the application engine 352. The application launcher 353 launches the above-mentioned web application based on, for example, a user operation.

[0090] The receiving terminal cooperation control unit 363 performs cooperation operation with the receiving terminal device 2 using the Hybridcast terminal cooperation function. Specifically, the receiving terminal cooperation control unit 363 acquires information for using web resources provided by the receiving terminal device 2 via the API of the Hybridcast terminal cooperation function. Because the Hybridcast terminal cooperation function is a standardized technology, it becomes possible for the receiving terminal device 2 and the client terminal device 3 to cooperate with each other in a manner that is independent of the device manufacturer, etc.

[0091] That is, the receiving terminal cooperation control unit 363 uses the terminal cooperation function of Hybridcast to execute the processing required to use the web resource as a cooperative operation with the receiving terminal device 2. Specifically, the receiving terminal cooperation control unit 363 receives at least information on the location (endpoint, etc.) for using the web resource from the receiving terminal device 2.

[0092] 4 and 5 are schematic diagrams showing an example of a sequence of requests and responses between the receiving terminal device 2 and the client terminal device 3 using the terminal linking function of the extended Hybridcast in this embodiment.

[0093] Of the processes from steps S101 to S115 and steps S121 to S132 shown in the figure, steps S101 to S115 are performed using APIs (requests, responses, etc.) that also exist in conventional Hybridcast terminal linkage functions. However, at least a portion of the conventional requests or responses may be extended for this embodiment. In other words, not all of the processes from steps S101 to S115 described below belong to the prior art. Furthermore, steps S121 to S132 are performed using APIs (requests and responses) of the extended Hybridcast terminal linkage function that are particularly specific to this embodiment.

[0094] Furthermore, the receiving terminal device 2 and the client terminal device 3 do not necessarily execute all of the procedures from steps S101 to S115 and steps S121 to S132 in sequence. As shown in the figure, all processes except for step S115 are integrated as a pair of request and response. The process of step S115 is two-way communication, and the specific steps therein depend on the process content and data content to be applied.

[0095] Below, FIG. 4 and FIG. 5 will be described in turn.

[0096] 4, the client terminal device 3 transmits a device discovery request to the receiving terminal device 2. In response to the request, the receiving terminal device 2 transmits a device discovery response to the client terminal device 3 in step S102.

[0097] In step S103, the client terminal device 3 transmits an authentication request to the receiving terminal device 2. In response to the request, the receiving terminal device 2 transmits an authentication response to the client terminal device 3 in step S104.

[0098] In step S105, the client terminal device 3 transmits a request for obtaining available media to the receiving terminal device 2. In response to the request, in step S106, the receiving terminal device 2 transmits a response to the client terminal device 3 for obtaining available media.

[0099] In step S107, the client terminal device 3 transmits a request for acquiring a list of organized services to the receiving terminal device 2. In response to the request, in step S108, the receiving terminal device 2 transmits a response to the client terminal device 3 for acquiring a list of organized services.

[0100] In step S109, the client terminal device 3 transmits a request for channel selection and HC application startup to the receiving terminal device 2. In response to the request, in step S110, the receiving terminal device 2 transmits a response to the channel selection and HC application startup request to the client terminal device 3. Note that "HC" is an abbreviation for hybrid cast.

[0101] In step S111, the client terminal device 3 transmits a request for acquiring the activation status to the receiving terminal device 2. In response to the request, in step S112, the receiving terminal device 2 transmits a response to the client terminal device 3 for acquiring the activation status.

[0102] In step S113, the client terminal device 3 transmits a request to acquire the receiver status to the receiving terminal device 2. In response to the request, in step S114, the receiving terminal device 2 transmits a response to acquire the receiver status to the client terminal device 3. Note that the term "receiver" here refers to the receiving terminal device 2.

[0103] In step S115, the client terminal device 3 and the receiving terminal device 2 communicate bidirectionally. This enables the client terminal device 3 and the receiving terminal device 2 to operate in cooperation with each other. In other words, the client terminal device 3 and the receiving terminal device 2 perform terminal cooperation communication.

[0104] 5, in step S121, the client terminal device 3 transmits a web resource device discovery request to the receiving terminal device 2. In response to the request, in step S122, the receiving terminal device 2 transmits a web resource device discovery response to the client terminal device 3. This "web resource device discovery" request and response are procedures that do not exist in conventional hybridcast terminal linkage function technology.

[0105] In step S123, the client terminal device 3 transmits a request to acquire whether the web resource is available to the receiving terminal device 2. In response to the request, in step S124, the receiving terminal device 2 transmits a response to acquire whether the web resource is available to the client terminal device 3. This request and response to acquire whether the web resource is available is a procedure that does not exist in conventional hybridcast terminal linkage function technology.

[0106] In step S125, the client terminal device 3 transmits a request to acquire a list of web services to the receiving terminal device 2. In response to the request, in step S126, the receiving terminal device 2 transmits a response to acquire a list of web services to the client terminal device 3. This request and response to acquire a list of web services is a procedure that does not exist in conventional hybridcast terminal linkage function technology.

[0107] In step S127, the client terminal device 3 transmits a web service activation request to the receiving terminal device 2. In response to the request, in step S128, the receiving terminal device 2 transmits a web service activation request response to the client terminal device 3. This "web service activation request" request and response is a procedure that does not exist in conventional hybridcast terminal linkage function technology.

[0108] In step S129, the client terminal device 3 transmits a request to acquire the web service activation status to the receiving terminal device 2. In response to the request, in step S130, the receiving terminal device 2 transmits a response to acquire the web service activation status to the client terminal device 3. This request and response to "acquire the web service activation status" is a procedure that does not exist in conventional hybridcast terminal linkage function technology.

[0109] In step S131, the client terminal device 3 transmits a request to acquire the web service status to the receiving terminal device 2. In response to the request, in step S132, the receiving terminal device 2 transmits a response to acquire the web service status to the client terminal device 3. This request and response to acquire the web service status is a procedure that does not exist in conventional hybridcast terminal linkage function technology.

[0110] Next, we will explain various variations (modified examples) of the process of delivering accompanying information (event information, etc.) from the receiving terminal device 2 to the client terminal device 3. The variations explained below are 19 types of methods, from (1) to (19).

[0111] (1) Transmission of event information (EM) to client terminal device 3 (part 1) Next, details of the event information from the receiving terminal device 2 to the client terminal device 3 will be described. Event information is transmitted from the receiving terminal device 2 to the client terminal device 3 using the "sendtext" (text transmission) function of the Hybridcast terminal linkage function. This sendtext processing is performed in the terminal linkage communication (two-way communication) of step S115 in FIG. 4.

[0112] The procedure is as follows: That is, the client terminal device 3 has already completed pairing with the server function (client terminal linkage control unit 263) in the receiving terminal device 2 using the Hybridcast terminal linkage function. In other words, the procedures such as device discovery and authentication described in Fig. 4 have already been completed. This enables communication between the receiving terminal device 2 and the client terminal device 3 using websocket, which is included in the API group of the Hybridcast terminal linkage function.

[0113] On this premise, an event message is included in the broadcast signal received by the tuner unit 201 of the receiving terminal device 2. The event message separated and extracted by the demultiplexer 203 is passed to the web resource providing unit 262 via the event processing unit 282 etc., and is managed there. The web resource providing unit 262 uses the hybridcast terminal linkage function of the client terminal linkage control unit 263 to deliver the event message to the client terminal device 3. At this time, the sendtext function described above is used.

[0114] FIG. 6 is a schematic diagram showing an example of event message data sent from a receiving terminal device 2 to a client terminal device 3 using the sendtext function in the Hybridcast terminal linkage function. As shown in the figure, this data is written in, for example, JSON format. JSON stands for "JavaScript Object Notation." The data shown in the figure has a block structure. Lines 2 to 16 of this data are a block labeled "message." This message has the following two elements:

[0115] The first element has a label "devid" and contains data of the ID of the terminal to which the message is to be sent. "dev" here is an abbreviation for "device." In reality, by the time this message is sent, pairing between the receiving terminal device 2 and the client terminal device 3 has been completed. In other words, an appropriate terminal ID known to the client terminal cooperation control unit 263 is set as the data for "devid."

[0116] The second element is data labeled "sendTextToCompanionDevice" (sending text to a linked device). Beneath this "sendTextToCompanionDevice" element is a block labeled "text". This "text" contains the following seven elements: The first element (line 6) has a label "$type" and its value is the string "event_message". This indicates that the type of data being transmitted is an event message. The second element (line 7) has a label "$name" and its value is the string "event_message". This indicates that the name of the data being transmitted is an event message. The third element (line 8) has a label "status" and its value is the numeric value 0. This data represents the status. The fourth element (line 9) has a label "private_data" and its value is a null string in this example. This data represents the content of the event message. The fifth element (line 10) has a label called "message_id" and a value of 147. This data is used to identify each individual message being transmitted. The sixth element (line 11) has a label called "message_version" and a value of 1. This data represents the version of the message. The seventh element (line 12) has a label called "message_group_id" and a value of 31. This data is message group identification information for handling the message. For example, if a message with the same private_data as above is sent multiple times, it is possible to distinguish whether the multiple messages are the same or different messages depending on whether the message_group_id is the same.

[0117] It is also possible to assign the meaning of the message to the ID (such as the first element ($type), second element ($name), or fifth element (message_id) above) in advance. As an example, when "type=emergency", the client terminal device 3 processes the message meaning as "private_data=emergency". This makes it possible to ensure that the meaning of the event is not missed even if Private_data is empty or the message is missing for some reason.

[0118] As described above, in this method, the event message is delivered from the receiving terminal device 2 to the client terminal device 3 using the sendtext function in the terminal linking function of Hybridcast. This makes it possible to use the event message in the client terminal device 3.

[0119] It should be noted that if the frequency of transmitting event messages from the receiving terminal device 2 to the client terminal device 3 is high, it is possible that the processing load of the event messages on the receiving terminal device 2 and the client terminal device 3 may become too large. Therefore, the method described below may be used.

[0120] (2) Transmission of event information (EM) to client terminal device 3 (part 2) The basic procedures of this method are based on the above-mentioned "(1) Transmission of event information (EM) to client terminal device 3 - Part 1." However, in order to reduce the processing load on the receiving terminal device 2 and the client terminal device 3, in this method, the receiving terminal device 2 has multiple optional functions for suppressing the transmission of duplicate event messages. The first and second options will be explained below. The system 1 may operate in either the first or second option, or may operate in both the first and second options.

[0121] As a first option, the receiving terminal device 2 performs processing to deliver an event message only when there is a paired client terminal device 3. In other words, if there is no paired partner (client terminal device 3), the event processing unit 282, web resource providing unit 262, client terminal cooperation control unit 263, and other units within the receiving terminal device 2 do not perform processing to deliver an event message. This makes it possible to reduce the processing load on the receiving terminal device 2 side.

[0122] As a second option, the receiving terminal device 2 manages event messages to prevent the same event message from being sent multiple times to the same client terminal device 3. The specific control procedure is as follows: The web resource providing unit 262 manages information related to the event messages that have been delivered for each client terminal device 3. Specifically, the web resource providing unit 262 stores and manages information such as a true / false value indicating whether a specific event message has been delivered to a specific client terminal device 3 and a numerical value indicating the number of deliveries, in association with information identifying the message.

[0123] Then, when there is an event message included in the broadcast signal, web resource providing unit 262 determines, based on the information it manages, whether or not the event message should be transmitted to each individual client terminal device 3. If the determination result shows that the event message should be transmitted, web resource providing unit 262 transmits the message to the target client terminal device 3. If the message should not be transmitted, web resource providing unit 262 does not transmit the message to the target client terminal device 3. An example of the method for making this determination is as follows. That is, web resource providing unit 262 determines whether or not a specific message should be sent based on information that identifies the event message (the message ID (message_id) and message group ID (message_group_id) shown in FIG. 6).

[0124] As an example, if the parameter value of the message ID (message_id) of an event message that can be sent is different from that of the previously delivered event message, web resource providing unit 262 sends that event message to client terminal device 3. If the parameter value of the message ID (message_id) is the same as that of the previously delivered event message, web resource providing unit 262 does not send that event message to client terminal device 3. Note that web resource providing unit 262 may perform a similar determination and control the need for delivery using a message group ID (message_group_id) instead of the message ID (message_id).

[0125] That is, in this method, the accompanying information includes event message data extracted from the broadcast signal. Also, the web resource providing unit 262 performs control to prevent event information including the event message data of the same content from being sent multiple times to the client terminal device.

[0126] If a predetermined condition is met, the web resource providing unit 262 may clear (initialize) the above-mentioned management information stored therein and manage the sending date of the event message again from the initial state.

[0127] As described above, according to this technique, it is possible to reduce the processing load on both the receiving terminal device 2 and the client terminal device 3 when an event message is sent from the receiving terminal device 2 to the client terminal device 3.

[0128] However, the methods described so far have an unresolved problem. That is, with each of the above methods, it is not possible to deliver an event message according to the video playback status on the client terminal device 3 side. In other words, for example, when the client terminal device 3 requests the receiving terminal device 2 side to reload the data of the online video, the client terminal device 3 is unable to receive an event message that matches the video playback status after the reload. Therefore, the method described below may be used.

[0129] (3) Transmission of event information (EM) to client terminal device 3 (Part 3, control of events related to content) In order to solve the above-mentioned problems, this method distributes event messages from the receiving terminal device 2 (server side) based on the status of generation control on the client terminal device 3 side. In other words, this method controls the timing of event distribution. As an example, we will explain the case of a performance related to a specific event in the distribution of a live sports video. In other words, this method enables the following optional functions.

[0130] In this method, the receiving terminal device 2 delivers event messages according to the status of video playback control on the client terminal device 3 side. To this end, the receiving terminal device 2 manages the association between the playback position and the generated event message for each piece of content in the online video distribution. As an example, consider a case where a predetermined effect is performed on the viewing terminal (client terminal device 3) side at the moment a goal is scored in a soccer broadcast. The effect at the moment of the goal is, for example, the generation of visual or auditory effects such as LED lights or sound effects on the client terminal device 3 side. In this case, the receiving terminal device 2 manages the association between the playback position of the online video distribution (for example, the relative time from the beginning of the video (hh hour, mm minute, ss second, nnn, etc., where "nnn" is the decimal point of the second)) and a specific event message.

[0131] It should be noted that the receiving terminal device 2 does not manage events that are not associated with online streaming videos (for example, emergency earthquake alerts, etc.) by associating them with the playback positions described above.

[0132] Step 1: The client terminal device 3 uses the sendtext function described above to send information about the start and stop of playback (including pausing and resuming from a pause) of the online streaming video being played (a web resource provided by the receiving terminal device 2) to the receiving terminal device 2. Here, the client terminal device 3 performs the above transmission using the Hybridcast terminal linking function (terminal linking communication (bidirectional) shown in step S115 of FIG. 4).

[0133] The information (referred to as "video playback control information") transmitted from the client terminal device 3 to the receiving terminal device 2 in the above step 1 is, for example, as follows: That is, the video playback control information may include information on the timing at which playback started and the timing at which playback stopped (including pausing). The video playback information also includes information on the playback position (the above-mentioned relative time) of the video when playback started (including restarted). The video playback information may also include information on the new playback position when the playback position is changed in the client terminal device 3 (by rewinding, fast-forwarding, etc.).

[0134] Step 2: On the receiving terminal device 2 side, the client terminal cooperation control unit 263 transmits the video playback control information (information such as playback start, playback stop, and playback position) received from the client terminal device 3 to the web resource providing unit 262. As a result, the web resource providing unit 262 grasps information on whether the online streaming video is being played or stopped in the client terminal device 3, and information on the playback position.

[0135] Step 3: Then, in accordance with the received video playback control information, web resource providing unit 262 controls the distribution of an event message to client terminal device 3. In other words, when the playback position associated with the event arrives while video is being distributed to client terminal device 3 (playback position is in progress), web resource providing unit 262 distributes the associated event message to client terminal device 3 at that timing.

[0136] When playback of an online video stream on the client terminal device 3 side is stopped or resumed, the web resource providing unit 262 may initialize (clear) information for managing event messages as appropriate. As described above, the web resource providing unit 262 delivers event messages based on the playback position at that time on the client terminal device 3 side. Therefore, for example, even if a rewind operation (an operation to move the playback position back in time) is performed on the client terminal device 3 side, the web resource providing unit 262 can repeatedly send event messages at appropriate timing.

[0137] As mentioned above, disposable events (such as emergency earthquake alerts) are not associated with the playback position of the distributed online video. Therefore, it is sufficient for the web resource providing unit 262 to distribute such disposable events to the client terminal device 3 only once. In other words, disposable events do not need to be resent.

[0138] That is, in this technique, web resource providing unit 262 receives information about the playback position of a web resource in destination client terminal device 3 from client terminal device 3. Furthermore, web resource providing unit 262 transmits event information associated with a specific playback position in the web resource to client terminal device 3 at a timing based on the received playback position information.

[0139] As described above, this technique makes it possible to control the timing of notifications of events associated with the content of distributed online videos, etc. Furthermore, it also makes it possible to appropriately distribute event messages even when the playback position is changed (by rewinding, fast-forwarding, etc., if this is possible) on the client terminal device 3 side. In other words, it is possible to avoid situations where the timing of event transmission is off, where an event message that should be transmitted is not transmitted, or where an event message that should not be transmitted is transmitted (because it was skipped by fast-forwarding, etc.).

[0140] (4) Transmission of event information (EM) to client terminal devices 3 (Part 4, control of events associated with content in each of multiple client terminal devices 3) This method is a further variation of the above-mentioned event distribution based on playback control on the client terminal device 3 side. In this method, the receiving terminal device 2 manages video playback control information individually for each of the multiple client terminal devices 3.

[0141] For this reason, each time pairing is established based on a request for device discovery and authentication (see FIG. 4) from the client terminal device 3, the client terminal cooperation control unit 263 passes the terminal ID of that client terminal device 3 to the web resource providing unit 262. This allows the web resource providing unit 262 of the receiving terminal device 2 to hold a set of terminal IDs of paired client terminal devices 3 at any point in time.

[0142] Then, the web resource providing unit 262 manages video playback control information individually for each of the above terminal IDs (i.e., for each client terminal device 3 that is the pairing partner). Then, the web resource providing unit 262 can transmit an event message to each of the client terminal devices 3 based on the video playback control information that is individual for each terminal ID. Furthermore, the web resource providing unit 262 can manage which event messages (identified by message IDs and message group IDs) have already been transmitted for each terminal ID, and transmit the event messages based on the management status.

[0143] That is, in this technique, web resource providing unit 262 identifies each of the multiple client terminal devices 3. Web resource providing unit 262 transmits event information associated with a specific playback position in the web resource to each of the multiple client terminal devices 3 at a timing based on the playback position information received from each client terminal device 3.

[0144] In other words, a receiving terminal device 2 using this method can deliver event messages according to the specific situation (playback position, etc.) of each client terminal device 3, even when delivering videos as web resources to multiple client terminal devices 3.

[0145] (5) Transmission of event information (EM) to client terminal device 3 (Part 5, handling of message duplication on the client terminal device 3 side) This method uses a different method from the methods already described to control the delivery of duplicate messages so that they are not delivered multiple times. Specifically, in the method described above in "(2) Transmission of Event Information (EM) to Client Terminal Device 3 - Part 2," control was performed on the receiving terminal device 2 side (server side) to deliver duplicate messages only once. In this method, this control is not performed on the receiving terminal device 2 side, but similar control is performed on the client terminal device 3 side.

[0146] The processing by the client terminal device 3 in this technique is as follows. That is, the client terminal device 3 receives an event message from the receiving terminal device 2 through inter-terminal cooperative communication. As explained with reference to FIG. 6, the event message received by the client terminal device 3 has a message ID and a message group ID. The client terminal device 3 records and manages the message ID and message group ID of the received event message. The client terminal device 3 manages duplicate event messages based on the managed information. That is, the client terminal device 3 identifies the event message using either or both of the message ID and message group ID (referred to as identification information). The client terminal device 3 checks whether the identification information of the received event message matches the identification information of an event message that has already been received. If the identification information of the received event message does not match the identification information of an event message that has already been received, the client terminal device 3 executes processing according to the content of the event message (for example, processing to produce a performance according to the event content (such as a goal in soccer)). If the identification information of the received event message matches the identification information of an event that has already been received (processed), the client terminal device 3 does not perform any processing on the event message this time. In other words, the client terminal device 3 skips the event message.

[0147] As described above, in this method, the client terminal device 3 determines whether the received event information has already been processed based on the identification information contained in the event information, and performs control to prevent the processing of event information that has already been processed. Such control based on the identification information on the client terminal device 3 side makes it possible to prevent the same event from being processed multiple times.

[0148] It should be noted that the methods (1) to (5) described above allow the receiving terminal device 2 to distribute event information to the client terminal device 3 regardless of whether it receives video or audio and provides it to the client terminal device 3. In other words, regardless of whether video or audio is present, the receiving terminal device 2 can receive an event message included in the broadcast signal and distribute that event message to the client terminal device 3.

[0149] (6) Transmission of event information by MTE In this method, event information is distributed from the receiving terminal device 2 to the client terminal device 3 by the MTE.

[0150] Specifically, the process is as follows: The transcoding unit 261 converts the video and audio data extracted from the broadcast signal received by the receiving terminal device 2 into a format for online video distribution. The format for online video distribution is, for example, MPEG-DASH. The transcoding unit 261 passes the generated data in the format for online video distribution to the web resource providing unit 262 via the event processing unit 282.

[0151] On the other hand, when the receiving terminal device 2 receives an event message, the event processing unit 282 acquires the event message. The event processing unit 282 converts the information in the event message into MTE data and inserts the MTE into the data of the online video content passed from the transcoding unit 261. In this way, the data of the online video distribution content into which the MTE has been inserted is managed by the web resource providing unit 262. In response to a request from the client terminal device 3, the web resource providing unit 262 distributes the data of the online video distribution content into which the MTE has been inserted to the client terminal device 3.

[0152] FIG. 7 is a schematic diagram showing an example of an MTE that the event processing unit 282 inserts into video content for online distribution. As shown in the figure, the MTE is written in, for example, JSON format. Lines 1 to 15 in the figure represent one MTE. This MTE has the following data: "scheme_id_url" (line 2, scheme ID URL, identifier), "value" (line 3, value), "event_id" (line 4, event ID), "presentation_delta" (line 5, presentation delta), "time_scale" (line 6, timescale), "presentation_delta" (line 7, presentation delta), and "event_duration" (line 8, event duration). This MTE also has message data (message_data) written in lines 9 to 14. This message data includes a component tag ("component_tag", value "0x40"), a message group ID ("message_group_id", value 0), a message ID ("message_id", value 170), a message version ("message_version", value 1), and private data ("private_data", value is the string "event msg"). This message data is information that represents the content of the event.

[0153] In this method, the receiving terminal device 2 distributes event information as an MTE inserted into a video stream distributed over the Internet to the client terminal device 3. In other words, the event information is inserted at a specific playback position within the video stream.

[0154] Specifically, in this technique, the event processing unit 282 generates event information as a media-timed event (MTE). The event processing unit 282 inserts the generated media-timed event into the web resource output by the transcoding unit 261. The web resource providing unit 262 transmits the web resource into which the media-timed event has been inserted to the client terminal device 3. This allows the client terminal device 3 to grasp the timing of the event in relation to the playback position within the distributed online video. In other words, the client terminal device 3 can easily process the event in synchronization with a predetermined position within the distributed online video.

[0155] (7) Subtitle data is distributed as event information As explained with reference to Fig. 2, one of the resources that the receiving terminal device 2 receives as a broadcast signal is a subtitle resource. Digital subtitle data is also specified in the ARIB standard. In this method, the receiving terminal device 2 distributes event information including subtitle data to the client terminal device 3.

[0156] The specific processing is as follows: The subtitle decoder unit 214 of the receiving terminal device 2 decodes the subtitle resource extracted from the broadcast signal. The subtitle decoder unit 214 passes the subtitle data resulting from the decoding processing to the event processing unit 282. The event processing unit 282 generates MTE data including the subtitle data and inserts the MTE into the content of the online distribution video. The process of inserting the MTE into the online distribution video is as explained above in "(6) Transmission of event information using MTE". The event processing unit 282 passes the data of the online distribution video with the MTE inserted to the web resource providing unit 262. The web resource providing unit 262 manages the online distribution video and distributes it to the client terminal device 3.

[0157] This allows the client terminal device 3 to receive online streaming video content into which an MTE containing subtitle data has been inserted from the receiving terminal device 2. In other words, the client terminal device 3 can extract and use the subtitle data from the MTE inserted into the online streaming video.

[0158] FIG. 8 is a schematic diagram showing an example of an MTE in which subtitle data is inserted as an event. The MTE is written in JSON format, for example. Lines 1 to 15 in the figure represent one MTE. As in the case of FIG. 7, this MTE has the following data: "scheme_id_url," "value," "event_id," "presentation_delta," "time_scale," "presentation_delta," and "event_duration." This MTE also has message data (message_data) written in lines 9 to 14.

[0159] This message data includes data on the content identification information ("content") on line 10, the text ("text") on line 11, the start date and time ("startTime") on line 12, and the end date and time ("endTime") on line 13. The content identification information is information that identifies the content into which the subtitle text is to be inserted. The content identification information may be, for example, information that identifies a service. An example of the content identification information is the numerical value 32736. The text is the subtitle text extracted from the broadcast signal. An example of the text is "Got it. Leave it to me." The start date and time is data on the date and time when the presentation of the subtitle text begins. In the example shown, the start date and time is 12:58:37 on August 7, 2020, Japan time. The end date and time is data on the date and time when the presentation of the subtitle text ends. In the example shown, the end date and time is 12:58:42 on August 7, 2020, Japan time.

[0160] That is, the message data into which subtitle information is inserted in this method includes information for identifying the content, the subtitle text, and information on the timing of presenting the subtitles (start and end dates and times).

[0161] That is, in this method, the media-timed event generated by the event processing unit 282 includes subtitles extracted from the broadcast signal as accompanying information.

[0162] In addition, instead of storing the subtitle text in an MTE and distributing it from the receiving terminal device 2 to the client terminal device 3, the receiving terminal device 2 may distribute the subtitle text data to the client terminal device 3 using the sendtext function in the terminal linkage function of Hybridcast.

[0163] With this method, the receiving terminal device 2 can transmit the subtitle text data included in the broadcast signal to the client terminal device 3 as event information.

[0164] (8) Options for distributing subtitle data as event information (Part 1) In the method described above, subtitle data is distributed as event information from the receiving terminal device 2 to the client terminal device 3. In the present method, as an option, broadcast resources other than subtitle data extracted from the broadcast signal received by the receiving terminal device 2 are inserted as event information and distributed to the client terminal device 3. In the present method, broadcast resources that can be included in the event information include, for example, superimposed text, SI (service information), AIT (hybridcast application information table), BML (data broadcast content written in BML), etc.

[0165] In other words, in this method, the media-timed event generated by the event processing unit 282 includes, as the accompanying information, any of the following: superimposed text extracted from the broadcast signal, an application information table (AIT), service information (SI), and data broadcasting content.

[0166] In this method, the receiving terminal device 2 may deliver the above event information to the client terminal device 3 as an MTE, or may deliver it to the client terminal device 3 using the sendtext function in the terminal linking function of Hybridcast.

[0167] (9) Options for distributing subtitle data as event information (Part 2) This method is a further option for the method of distributing subtitle data as event information from the receiving terminal device 2 to the client terminal device 3. In this method, a URL is included in the message data (message_data) area in the MTE (lines 9 to 14 in FIG. 8). Specifically, for example, an element for describing the URL (using a label such as "url," for example) is provided in the block indicated by "message_data" in the data shown in FIG. 8.

[0168] The above URL may indicate, for example, a location in an external server device or a location in the receiving terminal device 2 itself. The above URL may indicate a location where subtitle text is stored, or may indicate a location where any other data is stored.

[0169] By using this method, even if there is an upper limit on the size of data in the MTE, the MTE can transmit information for accessing data that exceeds the upper limit from the receiving terminal device 2 to the client terminal device 3. In other words, this method can complement data that exceeds the upper limit on the size of the MTE.

[0170] (10) Processing options at the receiving terminal device 2 that distributes MTE (part 1) This method is one of the processing options at the receiving terminal device 2 side that distributes event information as MTE. In this method, the receiving terminal device 2 avoids duplication of events.

[0171] Specifically, the event processing unit 282 of the receiving terminal device 2 appropriately identifies the received event message contained in the broadcast signal. The event processing unit 282 determines whether the received event message represents the same event as a processed (distributed) event message based on the event identification information, event content, etc. If the event message is an unprocessed event message, the event processing unit 282 generates an MTE corresponding to the event message as usual and inserts it into the content of the online video distribution. If the event message is a processed event message, the event processing unit 282 suppresses the generation and insertion of an MTE based on the event message.

[0172] With this method, an event with the same content is delivered only once from the receiving terminal device 2 to the client terminal device 3. In other words, it is possible to prevent the same event from being delivered to the client terminal device 3 multiple times.

[0173] (11) Processing options at the receiving terminal device 2 that distributes MTE (part 2) This method is one of the processing options on the receiving terminal device 2 side that distributes event information as MTE. This method enables the receiving terminal device 2 to distribute one event repeatedly to one client terminal device 3.

[0174] Specifically, the event processing unit 282 of the receiving terminal device 2 ranks the received event messages contained in the broadcast signal according to their contents. This ranking indicates, for example, the importance of the event. The event processing unit 282 creates an MTE based on the event message and inserts it into the online distribution video. However, for event messages of a predetermined rank, the event processing unit 282 inserts the MTE representing that event multiple times into the online distribution video. As a result, the receiving terminal device 2 distributes the MTE representing the contents of an event that has been assigned a predetermined ranking to the client terminal device 3 multiple times.

[0175] This method increases the reliability of transmitting an event to the client terminal device 3, even when, for example, congestion in the communication network or other communication errors occurs. In other words, as an example, it becomes possible to more reliably transmit important events, such as an emergency earthquake alert, to the client terminal device 3.

[0176] (12) Processing options at the receiving terminal device 2 that distributes MTE (part 3) This method is one of the processing options on the receiving terminal device 2 side that distributes event information as MTE. In this method, the web resource providing unit 262 of the receiving terminal device 2 has an MTE parsing function. This eliminates the need for the web resource providing unit 262 to manage the MTE information inserted by the event processing unit 282.

[0177] (13) Processing options at the receiving terminal device 2 that delivers MTE (part 4) This method is one of the processing options on the receiving terminal device 2 side that distributes event information as an MTE. In this method, the event processing unit 282 of the receiving terminal device 2 distributes the MTE only to specific client terminal devices 3.

[0178] Specifically, the event processing unit 282 of the receiving terminal device 2 sets information (terminal ID, companion ID) that identifies the client terminal device 3 to which the event is to be delivered in the MTE as the value of "scheme_id_url". As a result, the web resource providing unit 262 delivers the MTE only to the client terminal device 3 specified by the above "scheme_id_url". This method makes it possible to narrow down the destinations to which the MTE is delivered to only specific client terminal devices 3. Note that the identifying information may also be written in the "message_data" section on the 9th line of Figure 7.

[0179] The above "scheme_id_url" is an element on the second line of the MTE shown in FIG. 7 and the second line of the MTE shown in FIG.

[0180] (14) Processing options on the client terminal device 3 side that receives the MTE distribution This method is one of the processing options available on the client terminal device 3 side that receives event information distributed as an MTE. In this method, the processing on the client terminal device 3 side prevents duplication of events.

[0181] Specifically, the receiving terminal cooperation control unit 363 of the client terminal device 3 handles the received MTE. Based on the contents (such as identification information and information indicating the event contents) included in the received MTE, the receiving terminal cooperation control unit 363 determines whether the received event is the same as a previously received (processed) event.

[0182] If the event is an unprocessed event, the receiving terminal cooperation control unit 363 controls the client terminal device 3 to execute processing according to the event as usual (for example, effects such as LED lights that match the content of the online video content).

[0183] If the event has already been processed, the receiving terminal cooperation control section 363 does not perform any processing such as presentation based on the received MTE. That is, the client terminal device 3 skips reading the MTE.

[0184] With this method, even if the same event is distributed multiple times as an MTE, the client terminal device 3 receiving the distribution can process the event only once, thereby avoiding event duplication.

[0185] (15) Processing options for MTE This method is one of the processing options for handling MTEs. In this method, multiple "scheme_id_url" are used to handle MTE distribution and other tasks.

[0186] Specifically, when the event processing unit 282 of the receiving terminal device 2 generates an MTE based on event information extracted from a broadcast signal, it sets a different value of "scheme_id_url" in the MTE depending on the type of MTE.

[0187] Figure 9 is an example of a list of "scheme_id_url" (identifiers) that can be used according to type (purpose, etc.). In the example shown, the identifiers first correspond to classifications into multiple categories. Examples of categories here are EM (event message), subtitles, superimposed text, and general-purpose. Of these, "EM" can be further divided into multiple uses. That is, "EM" includes uses such as emergency information related to people's names, stage directions, and personalized content. Furthermore, the "subtitle" category corresponds to the use of subtitle information. The "superimposed text" category corresponds to uses such as news and election information. The "general-purpose" category corresponds to the use of general-purpose content. Each of the uses listed here has its own identifier.

[0188] That is, the identifier for the use "Emergency information concerning human life" is "urn:scte:scte35:2013:emg". The identifier for the use "Direction" is "urn:scte:scte35:2013:pro". The identifier for the use "Personalized content" is "urn:scte:scte35:2013:personal". The identifier for the use "Subtitle information" is "urn:scte:scte35:2013:text". The identifier for the use "News, election information" is "urn:scte:scte35:2013:news". The identifier for the use "General purpose" is "urn:scte:scte35:2013:general". Note that the correspondence between categories, uses, and identifiers ("scheme_id_url") illustrated here is merely an example. Other combinations of categories, uses, and identifiers may be used instead.

[0189] That is, in this method, an identifier ("scheme_id_url") according to the purpose of the MTE is set in the MTE on the receiving terminal device 2 side. The receiving terminal device 2 distributes the MTE to the client terminal device 3. The client terminal device 3 to which the MTE is distributed may be one device or multiple devices. The receiving terminal cooperation control unit 363 in the client terminal device 3 that receives this MTE refers to the identifier set in the MTE to distribute the MTE to multiple functional units, process only MTEs with specific identifiers, or prioritize the MTEs to control the processing order.

[0190] As described above, with this method, the receiving terminal device 2 can assign a different identifier to each MTE. The identifier represents, for example, the purpose. However, the identifier may also be used as information corresponding to something other than the purpose. The client terminal device 3 that receives the MTE can allocate the MTE and control the processing priority of the MTE based on the identifier. In other words, the system 1 using this method can handle MTEs in a variety of ways.

[0191] (16) Acquire metadata through analysis and distribute the metadata to the client terminal device 3 In this technique, the receiving terminal device 2 analyzes the video and audio content received as a broadcast signal, manages the analysis results as metadata, and distributes the metadata to the client terminal device 3.

[0192] Specifically, the data analysis unit 281 of the receiving terminal device 2 analyzes the video and audio extracted from the broadcast signal. The technology used by the data analysis unit 281 to analyze the video and audio is an existing technology. The data analysis unit 281 manages the data of the analysis results as metadata related to the broadcast video and audio. The data analysis unit 281 manages the metadata as needed by associating it with the playback position of the video and audio (for example, relative time with the beginning of the content set to zero).

[0193] The data analysis unit 281, for example, analyzes video and identifies people appearing in the video (using technology such as face recognition) (identifying performer names, actor names, character names, etc.), and manages the information identifying the people as metadata. The data analysis unit 281 also analyzes video and identifies objects included in the video (using technology such as object recognition), and manages the information identifying the objects as metadata. The data analysis unit 281 also analyzes transliteration resources and manages character information (words, etc.) included in transliteration images (videos) as metadata (using technology such as character recognition). The data analysis unit 281 also analyzes audio and manages the text of spoken content (lines, etc.) as metadata using technology such as voice recognition, and manages the results of speaker identification using technology based on acoustic features as metadata.

[0194] The data analysis unit 281 passes some or all of the metadata it manages to the web resource providing unit 262. The web resource providing unit 262 can distribute the metadata received from the data analysis unit 281 to the client terminal device 3 using the two-way terminal link communication function of the Hybridcast terminal link function. The terminal link communication function used here is the function shown in step S115 of FIG. 4. The receiving terminal device 2 can send the metadata to the client terminal device 3 through the above-mentioned terminal link communication using the sendtext function of the Hybridcast terminal link function. In other words, according to this technique, metadata related to the video and audio of the broadcast received by the receiving terminal device 2 can be transmitted from the receiving terminal device 2 to the client terminal device 3.

[0195] In this method, it is not necessary to distribute all of the metadata managed by the data analysis unit 281 to the client terminal device 3. Metadata appropriately selected by the data analysis unit 281, the web resource providing unit 262, or the like may be distributed to the client terminal device 3.

[0196] Furthermore, in this technique, the receiving terminal device 2 does not necessarily have to distribute video and audio resources to the client terminal device 3. Furthermore, the client terminal device 3 does not necessarily have the function of decoding and playing back video and audio. Even in such cases, by using this technique, it is possible to distribute metadata related to video and audio from the receiving terminal device 2 to the client terminal device 3. In other words, it is possible to distribute metadata related to video and audio from the receiving terminal device 2 to a client terminal device 3 that does not have the function of playing back video and audio.

[0197] (17) Distribution of metadata to client terminal device 3 by MTE In the method described above, the metadata generated by the data analysis unit 281 is distributed to the client terminal device 3 using the linked terminal communication (sendtext) function in the terminal linking function of Hybridcast. Instead, in this method, the metadata is stored in the MTE data and distributed from the receiving terminal device 2 to the client terminal device 3.

[0198] Specifically, the process is as follows: The data analysis unit 281 generates and manages metadata related to resources such as video and audio using the method already described. The data analysis unit 281 passes necessary data from the managed metadata to the event processing unit 282. The event processing unit 282 can create an MTE using the metadata passed from the data analysis unit 281. The event processing unit 282 inserts the MTE created here into the video or the like (online distribution video) passed from the transcoding unit 261. The event processing unit 282 passes the online distribution video with the MTE inserted to the web resource providing unit 262. In response to a request from the client terminal device 3, the web resource providing unit 262 distributes the online distribution video with the MTE inserted to the client terminal device 3. In this way, the client terminal device 3 can obtain the metadata stored in the MTE.

[0199] In this method, metadata is embedded in the MTE, making it easy to synchronize specific playback positions in online streaming videos with the metadata.

[0200] (18) Improving the accuracy of metadata (analysis results) (Part 1) This method is one of the data analysis processes described above as methods (16) and (17) that improve the accuracy of subtitles.

[0201] Specifically, the data analyzer 281 of the receiving terminal device 2 acquires subtitle data (ARIB subtitles) extracted from the broadcast signal. As described above, the data analyzer 281 analyzes the audio extracted from the broadcast signal (speech recognition processing) and acquires text of the speech recognition result. The data analyzer 281 manages the text of the speech recognition result in association with the playback position of the original audio (relative time within the content). The text of the ARIB subtitle data and the text of the speech recognition result may not perfectly match, but they have a correlation. Therefore, the data analyzer 281 can match (align) the text of the ARIB subtitle data and the text of the speech recognition result. This allows the data analyzer 281 to add, to the subtitle data, information on the time (playback position) associated with the corresponding speech recognition result.

[0202] Using this information, the data analysis unit 281 generates a TTML file for a web platform based on the ARIB subtitle data. TTML is an abbreviation for "Timed Text Markup Language." The TTML file generated by the data analysis unit 281 contains text based on the ARIB subtitle data. The TTML file also contains information about the time (relative time within the content, playback position) assigned to each fragment of that text. The data format of the TTML file itself is standardized.

[0203] That is, in this technique, the data analyzer 281 matches the audio analysis result, which is the result of analyzing the audio resource extracted from the broadcast signal, with the subtitles extracted from the broadcast signal. The data analyzer 281 associates the subtitles with the playback position obtained by the matching. Then, the event processor 282 inserts a media-timed event including the subtitles associated with the playback position into the web resource.

[0204] As described above, this technique improves the time accuracy of subtitle text. Furthermore, this improved accuracy of subtitle text data can be associated with time information and distributed from the receiving terminal device 2 to the client terminal device 3. On the client terminal device 3 side, it is also possible to synchronize the timing of the video being played with the timing of the presentation of the subtitle text.

[0205] (19) Improving the accuracy of metadata (analysis results) (Part 2) This method further improves the accuracy of the video analysis results from the data analysis processing described above as methods (16) and (17).

[0206] Specifically, as described above, the data analysis unit 281 of the receiving terminal device 2 analyzes the video extracted from the broadcast signal to obtain information for identifying a person, obtain information for identifying an object, and recognize (identify) characters included in superimposed text. The data analysis unit 281 stores and manages the analysis results of these videos, etc. as metadata. In this method, the data analysis unit 281 also obtains information such as an outline of the broadcast program, names of performers, and filming locations from program information (such as EPG-API data) extracted from the received broadcast signal. The data analysis unit 281 may also obtain publicly available information related to the broadcast program from an external web server device, etc.

[0207] The data analysis unit 281 associates information acquired from the program information (EPG) with each data item included in the metadata acquired as a result of the video analysis, using methods such as keyword matching and concept matching. Furthermore, the data analysis unit 281 associates information acquired from an external web server device or the like with each data item included in the metadata acquired as a result of the video analysis, using methods such as keyword matching and concept matching. In this way, the data analysis unit 281 enriches the information in the metadata generated based on the video analysis. In other words, the data analysis unit 281 enhances the metadata.

[0208] The receiving terminal device 2 can include the metadata enhanced by the data analysis unit 281 in the event information by the method already described and distribute it to the client terminal device 3. In this case, the receiving terminal device 2 may transmit the event information to the client terminal device 3 by terminal link communication in the terminal link function of Hybridcast (step S115 in FIG. 4), or may transmit it to the client terminal device 3 as MTE. In this way, the client terminal device 3 can obtain the enhanced metadata.

[0209] That is, in this technique, the data analysis unit 281 associates the video analysis result, which is the result of analyzing the video resource extracted from the broadcast signal, with the program information extracted from the broadcast signal or information from an external web server device. The data analysis unit 281 generates metadata (the above-mentioned advanced metadata) obtained by this association. The event processing unit 282 inserts a media-timed event including the metadata (advanced metadata) generated by the data analysis unit 281 into the web resource.

[0210] FIG. 10 is a block diagram showing an example of the internal configuration of each device (receiving terminal device 2, client terminal device 3, and other necessary server devices, etc.) constituting system 1 in this embodiment. Each device can be realized using a computer. As shown in the figure, the computer includes a central processing unit 901, RAM 902, input / output port 903, input / output devices 904 and 905, etc., and bus 906. The computer itself can be realized using existing technology. The central processing unit 901 executes instructions contained in a program read from RAM 902, etc. In accordance with each instruction, the central processing unit 901 writes data to RAM 902, reads data from RAM 902, and performs arithmetic and logical operations. RAM 902 stores data and programs. Each element contained in RAM 902 has an address and can be accessed using the address. RAM is an abbreviation for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices, etc. Input / output devices 904 and 905 are input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via an input / output port 903. A bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from / to RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906.

[0211] When implementing at least some of the functions of each device using a computer, a program for implementing these functions is recorded on a computer-readable recording medium, and the program recorded on the recording medium is loaded and executed by the computer system. Note that the term "computer system" here includes hardware such as an operating system and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, DVD-ROMs, and USB flash drives, as well as storage devices such as hard disks built into the computer system. In other words, a "computer-readable recording medium" may be a non-transitory computer-readable recording medium. Furthermore, the term "computer-readable recording medium" may also include media that temporarily and dynamically store programs, such as communication lines used when transmitting programs over networks like the Internet or telephone lines, or media that store programs for a fixed period of time, such as volatile memory within the server or client computer systems. The program may be a program that implements some of the functions described above, or it may be a program that can be implemented in combination with a program already stored in the computer system.

[0212] As described above, in this embodiment, the client terminal device 3 cooperates with the receiving terminal device 2. The client terminal device 3 receives, from the receiving terminal device 2, event information that includes accompanying information that accompanies at least one of video and audio, based on resources extracted from the broadcast signal received by the receiving terminal device 2.

[0213] As described above, in the client terminal device 3, the receiving terminal cooperation control unit 363 may receive the event information through terminal cooperation communication with the receiving terminal device 2. As described above, in the client terminal device 3, a web application running in the application engine 352 receives the web resource. The client terminal device 3 may receive the event information as a media-timed event (MTE) included in the received web resource.

[0214] [Variations] In the embodiment described above, the receiving terminal device 2 was equipped with the transcoding unit 261. This transcoding unit 261 transcoded at least one of video and audio and provided it as a web resource. The receiving terminal device 2 then transmitted this video and audio web resource to the client terminal device 3. In order for the receiving terminal device 2 to transmit event information based on a broadcast signal to the client terminal device 3, the receiving terminal device 2 does not necessarily need to be equipped with the transcoding unit 261. For example, when transmitting an EM (event message) from the receiving terminal device 2 to the client terminal device 3 using the sendtext function in the terminal linkage function of Hybridcast, a video or audio web resource is not required. In other words, a configuration is also possible in which the receiving terminal device 2 does not include the transcoding unit 261 for transcoding video and audio.

[0215] The above has described in detail an embodiment of the present invention (including modified examples) with reference to the drawings, but the specific configuration is not limited to this embodiment, and also includes designs within the scope that do not deviate from the gist of the present invention. [Industrial Applicability]

[0216] The present invention can be used in industries that provide content such as video content, and industries that manufacture or sell equipment for that purpose. However, the scope of use of the present invention is not limited to the examples given here. [Explanation of symbols]

[0217] 1 System 2. Receiving terminal device (receiving device) 3. Client terminal device 4 Antennas 8. Router 201 Tuner unit (receiving unit) 202 Descrambler 203 Demultiplexer 211 Data broadcasting processing unit 212 Video decoder section 213 Audio decoder section 214 Subtitle decoder section 221 Data Broadcasting Engine 231 Communications Department 232 Streaming Receiver 233 Demultiplexer 242 Video decoder section 243 Audio decoder section 244 Subtitle Decoder 251 Application control unit 252 Application Engine 253 Application Launcher 261 Transcoding Section 262 Web Resources Department 263 Client terminal cooperation control unit 271 Video output section 272 Audio output section 281 Data Analysis Department 282 Event Processing Unit 331 Communications Department 352 Application Engine 353 Application Launcher 363 Receiving terminal cooperation control unit 901 Central Processing Unit 902 RAM 903 Input / Output Ports 904,905 Input / Output Devices 906 Bus

Claims

1. a receiving unit for receiving a broadcast signal; an event processing unit that generates event information based on resources extracted from the broadcast signal; a web resource providing unit that transmits the event information generated by the event processing unit to a client terminal device as a collaboration destination via communication; a transcoding unit that transcodes at least one of the data of the video resource and the audio resource extracted from the received broadcast signal into data in a format usable by a web platform and outputs the data as a web resource; Equipped with the event processing unit generates the event information including accompanying information accompanying at least one of the video resource and the audio resource; the web resource providing unit transmits the event information generated by the event processing unit to the client terminal device together with the web resource output by the transcoding unit. Receiving device.

2. The accompanying information is Any of an event message, subtitles, superimposed characters, an application information table (AIT), service information (SI), and data broadcasting content extracted from the broadcast signal; or metadata obtained by analyzing at least one of the video resource or the audio resource extracted from the broadcast signal; Including, 2. The receiving device according to claim 1.

3. the web resource providing unit transmits the event information to the client terminal device of the collaboration destination using a function of terminal collaboration communication with the client terminal device of the collaboration destination in a terminal collaboration function of Hybridcast; 3. The receiving device according to claim 1 or 2.

4. the associated information includes event message data extracted from the broadcast signal; the web resource providing unit performs control to prevent the event information including the event message data of the same content from being transmitted multiple times to the client terminal device.

4. The receiving device according to claim 3.

5. the web resource providing unit receives information on a playback position of the web resource in the client terminal device from the client terminal device, and transmits the event information associated with a specific playback position in the web resource to the client terminal device at a timing based on the received information on the playback position.

5. The receiving device according to claim 3 or 4.

6. the web resource providing unit identifies each of the plurality of client terminal devices, and transmits the event information associated with a specific playback position in the web resource to each of the plurality of client terminal devices at a timing based on the information of the playback position received from each of the client terminal devices.

6. The receiving device according to claim 5.

7. the event processing unit generates the event information as a media-timed event (MTE) and inserts the generated media-timed event into the web resource output by the transcoding unit; the web resource providing unit transmits the web resource into which the media-timed event is inserted to the client terminal device; 2. The receiving device according to claim 1.

8. The media-timed event generated by the event processing unit includes, as the accompanying information, any one of subtitles, superimposed text, an application information table (AIT), service information (SI), and data broadcasting content extracted from the broadcast signal.

8. The receiving device according to claim 7.

9. a data analysis unit that associates the subtitles with a playback position obtained by matching an audio analysis result, which is a result of analyzing the extracted audio resource, with the extracted subtitles; Furthermore, the event processing unit inserts the media-timed event, including the subtitle associated with the playback position, into the web resource; 9. The receiving device according to claim 8.

10. a data analysis unit that generates metadata obtained by associating a video analysis result, which is a result of analyzing the extracted video resource, with program information extracted from the broadcast signal or information from an external web server device; Furthermore, the event processing unit inserts the media-timed event including the metadata generated by the data analysis unit into the web resource; 10. The receiving device according to claim 9.

11. an application engine that runs a web application that receives and uses the web resource from the receiving device according to claim 1; a receiving terminal cooperation control unit that executes a process required for using the web resource as a cooperative operation with the receiving device by a terminal cooperation function of Hybridcast; Equipped with receiving, from the receiving device, event information including accompanying information accompanying at least one of video and audio based on resources extracted from the broadcast signal received by the receiving device; Client terminal equipment.

12. the receiving-terminal cooperation control unit receives the event information through terminal cooperation communication with the receiving device; 12. The client terminal device according to claim 11.

13. determining whether the event information has already been processed based on the identification information of the received event information, and performing control to inhibit processing of the event information that has already been processed; 13. The client terminal device according to claim 12.

14. receiving the event information as a media-timed event (MTE) included in the web resource received by the web application; 12. The client terminal device according to claim 11.

15. A computer including a receiving unit for receiving a broadcast signal, A receiving device according to any one of claims 1 to 10, A program to function as a

16. Computer, A client terminal device according to any one of claims 11 to 14. A program to function as a

Images