Source device, display device, and operation method of source device

Abstract

A source device and a display device are provided. The source device: transmits first media data acquired through an image receiving unit to the display device through a first transmission path; on the basis that the first transmission path is lost, converts the first media data into second media data; stores the second media data in a local web server; and, on the basis of a request of the display device, transmits the second media data stored in the local web server to the display device through a second transmission path which is different from the first transmission path.

Description

Source device, display device, and method of operation of the source device

[0001] The present disclosure relates to a source device, a display device, and a method of operating the source device.

[0002] Recently, wireless connection services are being researched that wirelessly connect a source device that shares a screen and a sink device that receives the screen. The source device may be a device that generates or transmits content. The sink device may be a device connected to the source device that receives and outputs content from the source device.

[0003] A source device can receive and process various input signals to deliver content to a display device, which is an example of a sink device. Input signals may include video, audio, and supplementary information. The source device can connect external source devices to itself, process the input signals, and transmit them wirelessly to the display device. For example, the source device can receive digital broadcast signals (DTV) or analog broadcast signals (ATV) from a broadcast network, digitize them, and transmit them to the display device. Alternatively, for example, the source device can be wired to external source devices to receive and process input signals from the external source devices via the wire, and then transmit them wirelessly to the display device. Accordingly, the source device can transmit high-resolution video and audio data to the display device without the need for other wired connections, such as HDMI cables or USB.

[0004] However, since wireless connection services are based on the premise that the wireless connection between the source device and the display device is stable, there is a limitation in that stable video viewing by the user cannot be guaranteed if the wireless connection is unstable or lost in an actual usage environment.

[0005] Meanwhile, ATSC (Advanced Television Systems Committee) 3.0, a digital TV broadcasting standard, is a North American next-generation broadcasting standard characterized by high data transmission rates and network convergence services, and has been adopted as the domestic UHD broadcasting standard. ATSC 3.0 uses the ROUTE (Real-Time Object Delivery over Unidirectional Transport) protocol and the MMT (MPEG Media Transport) protocol as transmission standards. Users can watch OTA (Over-The-Air) content through the broadcasting network.

[0006] A broadband IP channel refers to a channel that transmits broadcast signals via the Internet Protocol (IP), representing a method of providing broadcast services through broadband internet connections instead of traditional broadcast networks. Users are now able to watch content at their desired time through services such as IPTV and OTT (Over-The-Top).

[0007] A source device according to one embodiment of the present disclosure includes a video receiving unit, a communication unit, at least one processor, and a memory comprising one or more storage media for storing one or more instructions.

[0008] According to one embodiment of the present disclosure, the source device transmits the first media data obtained through the image receiver to the display device through the first transmission path by executing the one or more instructions individually or collectively by the at least one processor.

[0009] According to one embodiment of the present disclosure, the source device converts the first media data into the second media data by the at least one processor executing the one or more instructions individually or in combination, based on the loss of the first transmission path.

[0010] According to one embodiment of the present disclosure, the source device stores the second media data on a local web server by executing the one or more instructions individually or in combination by the at least one processor.

[0011] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device transmits the second media data stored in the local web server to the display device through a second transmission path different from the first transmission path, in response to a request by the display device.

[0012] A display device according to one embodiment of the present disclosure includes a communication unit, at least one processor, and a memory comprising one or more storage media for storing one or more instructions.

[0013] According to one embodiment of the present disclosure, the display device receives first media data from a source device through a first transmission path by executing the one or more instructions individually or collectively by the at least one processor.

[0014] According to one embodiment of the present disclosure, by the at least one processor executing the one or more instructions individually or in combination, the display device obtains second media metadata for accessing second media data based on the loss of the first transmission path.

[0015] According to one embodiment of the present disclosure, the display device requests the second media data from the source device based on the second media metadata by executing the one or more instructions individually or in combination by the at least one processor.

[0016] According to one embodiment of the present disclosure, the display device receives the second media data from the source device through a second transmission path different from the first transmission path by executing the one or more instructions individually or in combination.

[0017] A method of operation of a source device according to one embodiment of the present disclosure includes the steps of: acquiring first media data through a tuner unit; transmitting the first media data to a display device through a first transmission path; converting the first media data into second media data based on the fact that the first transmission path is lost; storing the second media data in a local web server; and transmitting the second media data stored in the local web server to the display device through a second transmission path different from the first transmission path, in response to a request from the display device.

[0018] In one embodiment of the present disclosure, a computer-readable recording medium is provided having a program for performing a method of operation of a source device on a computer, comprising the steps of: acquiring first media data through a tuner unit; transmitting the first media data to a display device through a first transmission path; converting the first media data into second media data based on the loss of the first transmission path; storing the second media data on a local web server; and transmitting the second media data stored on the local web server to the display device through a second transmission path different from the first transmission path, in response to a request from the display device.

[0019] The present disclosure can be easily understood from the combination of the following detailed description and the accompanying drawings, where reference numerals denote structural elements.

[0020] FIG. 1 is a schematic diagram of a system including a source device and a display device according to one embodiment of the present disclosure.

[0021] Figure 2 is a diagram showing the ATSC 3.0 protocol stack.

[0022] FIG. 3 is a block diagram of a source device and a display device according to one embodiment of the present disclosure.

[0023] FIG. 4 is a flowchart illustrating a method of operation of a source device according to one embodiment of the present disclosure.

[0024] FIG. 5 is a flowchart for explaining the operation method of a display device according to one embodiment of the present disclosure.

[0025] FIG. 6 is a flowchart illustrating a method of operation for a source device and a display device connected through a first transmission path and a second transmission path according to an embodiment of the present disclosure.

[0026] FIG. 7 is a diagram illustrating the operation of a source device and a display device connected by a first transmission path according to one embodiment of the present disclosure.

[0027] FIG. 8 is a diagram illustrating the operation of the first demuxer of FIG. 7.

[0028] FIG. 9 is a diagram for explaining the operation of a source device and a display device connected by a second transmission path according to one embodiment of the present disclosure.

[0029] FIG. 10a is a diagram illustrating the operation of the second demuxer of FIG. 9.

[0030] FIG. 10b is a diagram illustrating the operation of the packager of FIG. 9.

[0031] FIG. 11 is a diagram illustrating a second media data playback process between a source device and a display device according to one embodiment of the present disclosure.

[0032] FIG. 12 is a drawing for exemplarily illustrating a screen of content output to a display device in various scenarios according to one embodiment of the present disclosure.

[0033] FIG. 13 is a detailed block diagram showing a display device according to one embodiment of the present disclosure.

[0034] In the present disclosure, the expression “at least one of a, b, or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “a, b, and c all”, or variations thereof.

[0035] Embodiments of the present disclosure are described below in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0036] The terms used in this disclosure are described in their current, general form considering the functions mentioned herein; however, they may refer to various other terms depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Accordingly, the terms used in this disclosure should not be interpreted solely by their names, but should be interpreted based on the meaning of the terms and the overall content of this disclosure.

[0037] Furthermore, the terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit this disclosure.

[0038] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components in between.

[0039] The terms “above” and similar designations used in this specification, particularly in the claims, may indicate both singular and plural forms. Furthermore, unless there is a description explicitly specifying the order of the steps describing the method according to this disclosure, the described steps may be performed in a suitable order. This disclosure is not limited by the order in which the described steps are described.

[0040] Phrases such as "in some embodiments" or "in one embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment.

[0041] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function. Additionally, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as algorithms executed on one or more processors. Furthermore, the present disclosure may employ prior art for electronic configuration, signal processing, and / or data processing, etc. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations.

[0042] Furthermore, the connecting lines or connecting members between the components depicted in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In the actual device, connections between components may be represented by various alternative or added functional connections, physical connections, or circuit connections.

[0043] Additionally, terms such as "...part," "module," etc., as described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0044] In the present disclosure, "processor" may include various processing circuits and / or a plurality of processors. For example, the term "processor" as used herein, including in the claims, may include at least one processor and various processing circuits. In at least one processor, one or more processors may be configured to perform the various functions described herein in a distributed manner, individually and / or collectively. As used herein, "processor," "at least one processor," and "one or more processors" may be configured to perform various functions. However, these terms cover, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor can perform all functions. Additionally, at least one processor may include a combination of processors performing various functions of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0045] In the present disclosure, the term “user” refers to a person using a display device and may include a consumer, evaluator, viewer, administrator, or installer. Additionally, in the specification, “manufacturer” or “provider” may refer to a manufacturer that manufactures a display device and / or components included in the display device.

[0046] In the present disclosure, 'image' may include a still image, a graphic, a picture, a frame, a video composed of a plurality of consecutive still images, or a video.

[0047] In the present disclosure, a transmission path may represent a physical or logical path through which data travels from a transmitting device to a receiving device. For example, a transmission path may represent a routing path or a physical network connection on a network. For example, in a Wi-Fi Direct connection, a transmission path may represent a wireless communication path between two devices. Or, for example, in a connection between devices relayed by an Access Point (AP), a transmission path may represent a physical connection path or a wireless communication path between two devices.

[0048] In the present disclosure, a pipeline may represent a series of processes for processing and transmitting data. For example, the process of packetizing data received by a source device and sending it to a display device over a network may be implemented as a pipeline. Additionally, for example, the process of image processing of data received by a display device may be implemented as a pipeline.

[0049] In the present disclosure, decapsulation, depacketization, demuxing, or demultiplexing may include an operation of extracting data contained within a packet by analyzing (or parsing) information contained in the header of a packet, or an operation of separating data such as video, audio, and subtitles contained in a single transmission stream.

[0050] In the present disclosure, streaming data refers to a continuous stream of data generated from various sources. A streaming server may divide the streaming data into small media file fragments and transmit them to a client. For example, each media file may be stored in the form of fragmented MP4, but is not limited thereto. Each media file may be decoded independently.

[0051] In the present disclosure, a broadband IP server refers to a computer device that transmits data over a broadband (Internet network) based on a request received from a client.

[0052] In the present disclosure, a local web server refers to a computer device that provides services to a client within a local network. The local web server is used within an internal network or the same network and can provide content by responding to client requests via the HTTP protocol. In the present disclosure, a local network refers to a communication network distinct from the Internet network, and may mean a network where multiple devices on the same network can communicate directly without routing. For example, the fact that a source device and a display device are connected to a local network may mean that the two devices exist on the same local network and can communicate directly.

[0053] In the present disclosure, the content may be received by the device from a content provider, such as a broadcast signal, a streaming service, a Blu-ray player, or a game console. The content may include one or more of broadcast content received directly from a broadcasting station as an RF signal, broadcast content received through an external source device, and content received from a content provider server via the Internet.

[0054] The present disclosure will be described in detail below with reference to the attached drawings.

[0055] FIG. 1 is a schematic diagram of a system including a source device and a display device according to one embodiment of the present disclosure.

[0056] A system according to one embodiment of the present disclosure may include a source device (100) and a display device (200). The system may be devices that provide a wireless connection service between the source device (100) and the display device (200). Here, the display device (200) may be a sink device. The source device (100) and the display device (200) are wirelessly connected, and the source device (100) transmits (or provides) various content received from an external device to the display device (200), and the display device (200) may output content received (or acquired) from the source device (100).

[0057] A source device (100) according to one embodiment of the present disclosure may include a set-top box, a Blu-ray Disc player, a DVD (Digital Versatile Disc) player, a game device, a digital camera, a camcorder, a streaming device, a home theater, etc. Alternatively, the source device (100) may include various electronic devices such as a smartphone, a tablet PC, a mobile terminal, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigation device, an MP3 player, a wearable device, etc.

[0058] A display device (200) according to one embodiment of the present disclosure may be a TV, but this is merely one embodiment, and may be implemented as an electronic device capable of receiving a broadcast signal and displaying an image based on the broadcast signal. For example, the display device (200) may be implemented as various electronic devices such as a mobile phone, tablet PC, digital camera, camcorder, laptop computer, tablet PC, desktop, e-book reader, digital broadcast terminal, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player), navigation, MP3 player, wearable device, etc. In particular, the embodiments may be easily implemented in a display device with a large display, such as a TV, but are not limited thereto.

[0059] Additionally, the display device (200) may be fixed or movable and may be a digital broadcast receiver capable of receiving digital broadcasts. Furthermore, the display device (200) may be implemented not only as a flat display device, but also as a curved display device having a screen with curvature or a flexible display device with adjustable curvature. The output resolution of the source device (100) may include, for example, HD (High Definition), Full HD, Ultra HD, or a resolution sharper than Ultra HD.

[0060] A source device (100) according to one embodiment of the present disclosure can receive various content from an external device. For example, the content may refer to multimedia content and may include, but is not limited to, images, videos, audio, text, games, applications, broadcasts, etc. For example, the source device (100) can receive broadcast content from a broadcast network (broadcast, 30) through a tuner unit. The broadcast content may be various forms of media composed of a collection of media components provided to the user (e.g., video, audio, subtitles, service announcements, etc.). For example, the source device (100) can receive various content from an external device through an input / output unit. For example, the source device (100) can receive content from an external server or various applications, for example, an OTT service provider, through a communication unit.

[0061] A source device (100) according to one embodiment of the present disclosure may be connected to a display device (200) via a communication network. The source device (100) may transmit received content to the display device (200). The display device (200) may receive content from the source device (100) and may output the received content through a display.

[0062] According to one embodiment of the present disclosure, a source device (100) and a display device (200) may be connected via a first transmission path (10) or a second transmission path (20) depending on the network environment. For example, if the network environment supporting the first transmission path (10) is stable, the source device (100) may transmit first media data to the display device (200) via the first transmission path (10). The display device (200) may receive the first media data from the source device (100) via the first transmission path (10). Alternatively, if the network environment supporting the first transmission path (10) is unstable or the first transmission path (10) is lost, the source device (100) may transmit second media data to the display device (200) via the second transmission path (20). The display device (200) may receive the second media data from the source device (100) via the second transmission path (20).

[0063] A first transmission path (10) and a second transmission path (20) according to one embodiment of the present disclosure may be different paths connecting a source device (100) and a display device (200) through different communication networks. A first transmission path (10) according to one embodiment of the present disclosure may be a path for transmitting first media data through a first communication module (e.g., including a communication circuit) that enables direct communication between the source device (100) and the display device (200). For example, the source device (100) and the display device (200) may be connected to each other through short-range wireless communication such as a wireless LAN (e.g., Wi-Fi). A second transmission path (20) according to one embodiment of the present disclosure may be a path for transmitting second media data through a second communication module (e.g., including a communication circuit) that enables communication between the source device (100) and the display through an access point (AP, 50). For example, the source device (100) and the display device (200) can each be connected to the internet network (broadband, 40) through an access point (50) and indirectly connected to each other.

[0064] According to one embodiment of the present disclosure, the first media data and the second media data may be the same content, standardized (or formatted) by different communication protocols, and transmitted through different transmission paths. For example, the first media data may be data transmitted through a first transmission path (10), and the second media data may be data in which the content included in the first media data is transmitted through a second transmission path (20).

[0065] For example, the first media data may be data standardized to a first communication protocol corresponding to the method of receiving content received by the source device (100) from an external device. For example, if the content received by the source device (100) is broadcast content through the broadcast network (30), the first media data may be data standardized to a first communication protocol available in the broadcast network (30). The first media data may be transmitted from the source device (100) to the display device (200) through the first transmission path (10) in a state standardized to a first communication protocol (e.g., broadcast protocol) available in the broadcast network (30). For example, the second media data may be data standardized to a second communication protocol (e.g., Internet protocol) that is a standard available in the Internet network (40). The second media data may be data in which the format of the first media data has been converted to the format of the second media data so that the first media data received through the broadcast network (30) can be transmitted through the Internet network (40).

[0066] In one embodiment of the present disclosure, while the first transmission path (10) is maintained, the source device (100) can transmit the first media data to the display device (200) through the first transmission path (10) without separate processing of the first media data standardized to a first communication protocol available in the broadcasting network (30). In one embodiment of the present disclosure, while the first transmission path (10) is lost, the source device (100) can generate second media data by converting the format of the media file included in the first media data to a second communication protocol so that the first media data can be transmitted through the internet network (40), and transmit the second media data to the display device (200) through the second transmission path (20).

[0067] A source device (100) according to one embodiment of the present disclosure may establish a local web server within the source device (100) to transmit second media data through a second transmission path (20) and may operate as a broadband IP server. Additionally, a display device (200) may request second media data from the source device (100) through the second transmission path (20) and may operate as a web client to receive second media data.

[0068] The structure of a web server-web client is such that when the web server receives an HTTP request from a web client remotely via the internet network (broadband) (40), it sends data corresponding to an HTTP response to the web client in response to the HTTP request.

[0069] In one embodiment of the present disclosure, the source device (100) establishing a local web server and operating like a broadband IP server may indicate that the source device (100) establishes a local web server at its local host address (e.g., 127.0.0.1) and, upon receiving a request from a web client (e.g., a display device (200)), transmits a response to the web client. In one embodiment of the present disclosure, the display device (200) operating like a web client may indicate that the display device (200) is connected to a local network and accesses the local web server of the source device (100) using a URL (Uniform Resource Locator) containing the local host address (e.g., 127.0.0.1) of the source device (100). The display device (200) may request data from the source device (100) using the URL and receive data from the source device (100).

[0070] A source device (100) according to one embodiment of the present disclosure can extract segments contained in first media data standardized by a first communication protocol, generate second media data by standardizing the extracted segments by a second communication protocol, and store the second media data on a local web server.

[0071] A display device (200) according to one embodiment of the present disclosure may access a local web server and obtain second media metadata containing information for requesting segments included in the second media data. The second media metadata may include information regarding bibliographic details of the segments included in the second media data. For example, the second media metadata may include the local host address of the source device (100) (e.g., 127.0.0.1), the content type of each segment, the length of each segment, the playback start time of the segment, etc. Based on the second media metadata, the display device (200) may generate a URL including the local host address and the filename of the segment, request the segments from the local web server using the URL, and receive the segments from the local web server. Here, the filename of the segment may include the availability start time of the segment, which is described in detail in FIG. 11. The second media metadata may be referred to as a Media Presentation Description (MPD) file.

[0072] When a source device (100) and a display device (200) according to one embodiment of the present disclosure are connected through a first transmission path (10), the source device (100) can directly transmit a first media data to the display device (200) based on the first transmission path (10), thereby enabling fast and stable transmission of large-capacity media data.

[0073] In the event that a first transmission path (10) connected to a source device (100) and a display device (200) according to one embodiment of the present disclosure is lost, the source device (100) and the display device (200) may switch the transmission path to a second transmission path (20) connected between the source device (100) and the display device (200) via an internet network (40). When the first transmission path (10) is lost, the source device (100) operates like a broadband IP server, and the display device (200) operates like a web client, and can deliver content through the second transmission path (20). Accordingly, even if direct communication between the source device (100) and the display device (200) is unstable, data transmission between the source device (100) and the display device (200) is not interrupted, so stable video viewing by the user can be guaranteed.

[0074] Meanwhile, in the present disclosure, the first communication protocol is exemplified as a broadcasting protocol but is not limited thereto. For example, if the content received by the source device (100) is content received from an external device via an HDMI port, the first media data may be data standardized according to the HDMI standard. Additionally, for example, if the content received by the source device (100) is OTT content received via a communication unit, the first media data may be data standardized to format the OTT content.

[0075] Meanwhile, in one embodiment of the present disclosure, the source device (100) and the display device (200) can each identify whether the first transmission path (10) is lost. This is described in operation 430 of FIG. 4 regarding the operation method of the source device (100) and operation 530 of FIG. 5 regarding the operation method of the display device (200). Additionally, the first connection management module of the source device (100) (730 of FIG. 7 or 930 of FIG. 9) and the second connection management module of the display device (200) (740 of FIG. 7 or 940 of FIG. 9) are described. However, this is not limited thereto, and at least one of the source device (100) or the display device (200) may identify whether the first transmission path (10) is lost.

[0076] Figure 2 is a diagram showing the ATSC 3.0 protocol stack.

[0077] Referring to FIG. 2, a method of transmitting content through a broadcast network may include a method of transmitting MPUs (Media Processing Units) using the MMT (MPEG Media Transport) protocol (or MMTP (MMT protocol)) and a method of transmitting MPEG-DASH (Dynamic Adaptive Streaming over HTTP) segments using the ROUTE (Real time Object delivery over Unidirectional Transport) protocol. In the broadcast network, a UDP / IP-based MMT protocol or ROUTE protocol may be used.

[0078] For example, at the transmitter, MMTP can transmit data formatted in MPU format and MMT signaling information (MMT-specific signaling). This data is encapsulated in UDP format at the transport layer and encapsulated into an IP (Internet Protocol) packet at the network layer. Additionally, for example, at the transmitter, ROUTE can transmit data formatted in DASH segments and ROUTE signaling information (ROUTE-specific signaling). This data is encapsulated in UDP format and encapsulated into an IP packet. Here, the signaling information may be signaling information regarding the service. IP / UDP packets can be re-encapsulated at the link layer, and the encapsulated IP / UDP packets can be referred to as ALP (ATSC Link-Layer Protocol) packets. ALP packets are transmitted to the physical layer and can be transmitted to the network layer of the receiver. At the receiving end, data decapsulation (or demuxing, depacketization, parsing) can be performed sequentially from the link layer to the transport layer.

[0079] Content transmission methods over the Internet (broadband) network may include transmitting MPEG-DASH segments based on HTTP / TCP / IP protocol specifications. On the Internet network, the TCP / IP-based HTTP protocol may be used.

[0080] For example, at the transmitter, data in DASH segment format is encapsulated into an HTTP message, divided into TCP segments at the transport layer, and encapsulated into IP packets at the network layer. Media data transmitted over the Internet is used in communication between web servers and web clients. For instance, when a web server receives an HTTP request from a web client, it can transmit MPEG-DASH segments as an HTTP response. Each segment can be distinguished by the sender's port number, the receiver's port number, and the segment order. The segment order can be identified using an index or timestamp. This ensures that media data is not lost or out of order.

[0081] The first media data according to one embodiment of the present disclosure may correspond to Over-The-Air (OTA) streaming data received through a broadcasting network. The OTA streaming data may be in a form where the content is divided into media file fragments in MPU format or DASH segment format and transmitted. That is, the unit of the media file may be in MPU format or DASH segment format. The first media data may be generated in MPU format and converted into MMT packets, or generated in DASH segment format and converted into ROUTE packets. The first communication protocol may include an MMT / UDP / IP or ROUTE / UDP / IP protocol according to the ATSC 3.0 standard.

[0082] The second media data according to one embodiment of the present disclosure may correspond to Over-The-Top (OTT) streaming data that can be transmitted over an internet network. The OTT streaming data may be in a form where the content is divided into fragments of media files in DASH segment format and transmitted. That is, the unit of the media file may be in DASH segment format. The second media data may be generated in DASH segment format and packetized based on HTTP. The second communication protocol may include an HTTP / TCP / IP protocol.

[0083] In one embodiment of the present disclosure, the source device (100) may convert a broadcast protocol such as MMT or ROUTE (hereinafter referred to as 'MMT / ROUTE') into an internet protocol such as HTTP-based MPEG-DASH when the first transmission path is lost. In this case, the second media data may retain the media file fragments included in the first media data, but the format (e.g., length and structure, etc.) may be converted to conform to the DASH specification.

[0084] FIG. 3 is a block diagram of a source device and a display device according to one embodiment of the present disclosure.

[0085] Referring to FIG. 3, a system according to one embodiment of the present disclosure may include a source device (100), a display device (200), a first network (310) connecting the source device (100) and the display device (200), and a second network (320) connecting the source device (100) and the display device (200) to the Internet.

[0086] The first network (310) is a short-range communication network that enables communication between a source device (100) and a display device (200) located in close proximity, and may utilize, for example, WLAN (or Wi-Fi), Bluetooth, BLE (Bluetooth Low Energy), Soft AP, NFC, WFD (Wi-Fi Direct), etc. Soft AP is an abbreviation for software enabled access point, which refers to software that enables a computer, rather than a router, to function as a wireless access point. The first network (310) may support a first transmission path connecting the source device (100) and the display device (200) through a first communication module provided in the source device (100) and a first communication module provided in the display device (200). For example, the first network (310) may be Wi-Fi 7 (Wi-Fi 7) that enables wireless communication with external devices through different frequency bands (e.g., 2.4 GHz band, 5 GHz band, and 6 GHz band) and channels via multi-link operation (MLO) technology, but is not limited thereto.

[0087] The second network (320) may include a WLAN communication network (or Wi-Fi communication network) for connecting each of the source device (100) and the display device (200) to the internet network. The second network (320) may support a second transmission path connecting the source device (100) and the display device (200) through a second communication module provided in the source device (100) and a second communication module provided in the display device (200). In the present disclosure, the second network (320) is exemplified as being formed through an access point, but is not limited thereto. For example, the second network (320) may be replaced not only with an access point but also with various unspecified communication technologies different from the first network (310).

[0088] However, not limited thereto, the source device (100) and the display device (200) may be wired connected via an HDMI (High-Definition Multimedia Interface) port, an MHL (Mobile High-Definition Link) port, a USB (Universal Serial Bus) port, a DP (Display Port), a Thunderbolt port, a VGA (Video Graphics Array) port, an RGB port, a D-SUB (D-subminiature), a DVI (Digital Visual Interface), a component jack, a PC port, etc.

[0089] First, the source device (100) will be described.

[0090] A source device (100) according to one embodiment of the present disclosure may include a processor (110), a tuner unit (120), an input / output unit (130), a communication unit (140), and a memory (150). However, not all of the illustrated components are essential components. The source device (100) may be implemented with more components than those illustrated, or with fewer components. In one embodiment of the present disclosure, the source device (100) may receive content through at least one of the tuner unit (120), the input / output unit (130), or the communication unit (140), each of which may be referred to as an 'image receiving unit'.

[0091] The tuner unit (120) can select only the frequency of the channel to be received by the source device (100) from among many radio wave components by tuning through amplification, mixing, resonance, etc. of broadcast content received via wired or wireless connection. The content received through the tuner unit (120) can be demuxed and decoded by the display device (200) described later to be separated into video, audio, and / or additional information (e.g., ESG (Electronic Service Guide)). The separated video, audio, and / or additional information can be stored in the memory (250) of the display device (200).

[0092] In one embodiment of the present disclosure, the tuner unit (120) may include a tuner and a demodulator. The tuner unit (120) may receive a radio frequency (RF) signal from a broadcast network through the tuner and demodulate the RF signal through the demodulator to obtain a first media data which is digital data. The first media data obtained through the tuner unit (120) may be transmitted to a display device (200) through a first transmission path without separate processing.

[0093] The input / output unit (130) can receive video (e.g., dynamic image signal or still image signal), audio (e.g., voice signal or music signal), and additional information from an external device, etc., under the control of the processor (110). The input / output unit (130) may include at least one of an HDMI port (High-Definition Multimedia Interface port), a component jack, a PC port, and a USB port. In addition to these, the input / output unit (130) may further include a DisplayPort (DP), Thunderbolt, and MHL (Mobile High-Definition Link). The input / output unit (130) may further include ports for separate output of video and audio.

[0094] In one embodiment of the present disclosure, the input / output unit (130) can receive first media data from an external device. The first media data received through the input / output unit (130) can be transmitted to a display device (200) via a first transmission path without separate processing.

[0095] The communication unit (140) can connect the source device (100) to peripheral devices, external devices, servers, mobile terminals, etc. under the control of the processor (110). The communication unit (140) may include various communication circuits included in at least one communication module. The communication unit (140) may include a short-range communication module, a wireless internet module, wired Ethernet, etc., corresponding to the performance and structure of the source device (100).

[0096] A short-range communication module is a module for short-range communication and may include, but is not limited to, a WLAN module (Wi-Fi module), a Bluetooth module, a Zigbee module, an infrared (IrDA, infrared Data Association) module, a WFD (Wi-Fi Direct) module, etc. A WLAN module can transmit and receive Wi-Fi signals with a surrounding device according to Wi-Fi communication standards. A Bluetooth module can receive Bluetooth signals transmitted from a surrounding device according to Bluetooth communication standards. A WFD (Wi-Fi Direct) module can support P2P (Peer-to-Peer) communication that enables direct connection between two devices without an access point. Wi-Fi may use frequencies in the 2.4 GHz band, 5 GHz band, or 6 GHz band. A short-range communication module may be used to communicate with a display device (200).

[0097] A wireless internet module is a module for wireless internet access and can be built into or external to a device. The wireless internet module may include a WLAN module, a Wibro (Wireless broadband) module, etc. The wireless internet module can be used for a source device (100) to communicate with a server device. A WLAN module can be used as a wireless internet module when it serves to connect to the internet through an access point.

[0098] A first communication module of a source device (100) according to one embodiment of the present disclosure may include a short-range communication module. A second communication module of a source device (100) according to one embodiment of the present disclosure may include a wireless internet module.

[0099] In one embodiment of the present disclosure, the communication unit (140) can receive first media data from an external device. The first media data received through the communication unit (140) can be transmitted to a display device (200) via a first transmission path without separate processing.

[0100] The processor (110) is electrically connected to the components included in the source device (100) and can perform operations or data processing regarding the control and / or communication of the components included in the source device (100). In one embodiment of the present disclosure, the processor (110) can load a request, command, or data received from at least one of the other components into memory for processing and store the processing result data in memory. According to one or more embodiments, the processor (110) may include at least one of a general-purpose processor such as a CPU (central processing unit), AP (application processor), DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU (graphic processing unit) or VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU (neural processing unit).

[0101] The processor (110) can process input data or control other configurations to process it according to data, operation rules, algorithms, methods, or models stored in memory (150). The processor (110) can perform operations of predefined operation rules, algorithms, methods, or models stored in memory (150) using the input data.

[0102] Memory (150) is electrically connected to the processor (110) and can store one or more modules, algorithms, operation rules, models, programs, instructions, or data related to the operation of components included in the source device (100). For example, memory (150) can store one or more modules, algorithms, operation rules, models, programs, instructions, or data for processing and controlling the processor (110). Memory (150) may include, but is not limited to, at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0103] A processor (110) according to one embodiment of the present disclosure can obtain first media data through a tuner unit (120) by executing one or more instructions stored in memory (150). The processor (110) can also obtain first media data through at least one of an input / output unit (130) or a communication unit (140) by executing one or more instructions stored in memory (150).

[0104] A processor (110) according to one embodiment of the present disclosure can transmit first media data to a display device (200) through a first transmission path by executing one or more instructions stored in a memory (150).

[0105] A processor (110) according to one embodiment of the present disclosure can convert first media data into second media data based on the loss of a first transmission path by executing one or more instructions stored in memory (150).

[0106] A processor (110) according to one embodiment of the present disclosure can store second media data on a local web server by executing one or more instructions stored in memory (150).

[0107] A processor (110) according to one embodiment of the present disclosure can transmit second media data stored in a local web server to the display device (200) via a second transmission path different from the first transmission path by executing one or more instructions stored in memory (150) in response to a request from the display device (200). For example, a display device (200) connected to the same network as the source device (100) (e.g., the second network (320)) can access the local web server of the source device (100) using a URL that includes the local IP address of the source device (100).

[0108] Next, the display device (200) is described.

[0109] A display device (200) according to one embodiment of the present disclosure may include a processor (210), a communication unit (220), a video processing unit (230), a display (240), and a memory (250). However, not all of the illustrated components are essential components. The display device (200) may be implemented with more components than those illustrated, or with fewer components.

[0110] The communication unit (220) can connect the display device (200) to peripheral devices, external devices, servers, mobile terminals, etc. under the control of the processor (210). The communication unit (220) may include various communication circuits included in at least one communication module. The communication unit (220) may include a short-range communication module, a wireless internet module, wired Ethernet, etc., corresponding to the performance and structure of the display device (200).

[0111] A short-range communication module is a module for short-range communication and may include, but is not limited to, a WLAN module (or Wi-Fi module), a Bluetooth module, a Zigbee module, an infrared (IrDA, infrared Data Association) module, a WFD (Wi-Fi Direct) module, etc. A WLAN module can transmit and receive Wi-Fi signals with nearby devices according to Wi-Fi communication standards. A Bluetooth module can receive Bluetooth signals transmitted from nearby devices according to Bluetooth communication standards. A WFD (Wi-Fi Direct) module can support P2P (Peer-to-Peer) communication that enables direct connection between two devices without an AP. Wi-Fi may use frequencies in the 2.4GHz, 5GHz, or 6GHz bands.

[0112] A wireless internet module is a module for wireless internet access and can be built into or external to a device. The wireless internet module may include a WLAN module, a Wibro (Wireless broadband) module, etc. The wireless internet module can be used for the display device (200) to communicate with a server device. A WLAN module can be used as a wireless internet module when it serves to connect to the internet through an AP.

[0113] A first communication module of a display device (200) according to one embodiment of the present disclosure may include a short-range communication module. A second communication module of a display device (200) according to one embodiment of the present disclosure may include a wireless internet module.

[0114] The video processing unit (230) processes video data to be displayed by the display (240) and can perform various image processing operations such as decoding, rendering, scaling, noise filtering, frame rate conversion, and resolution conversion on the video data. For example, the video processing unit (230) may include various image processing circuits. For example, the video processing unit (230) may include a media codec for processing video content. For example, the video processing unit (230) may include at least one of the first playback pipeline (720) of FIG. 7 and the second playback pipeline (920) of FIG. 9.

[0115] The display (240) can receive content from a broadcasting station, receive content from an external device such as an external server or external storage medium, or output content provided by various apps, such as an OTT service provider or a content provider. The display (240) can receive content from the source device (100) and display video-processed content.

[0116] A display device (200) according to one embodiment of the present disclosure can play first media data received from a source device (100) through a first transmission path by processing it through a video processing unit (230). The display device (200) can output the first media data to a display (240). A display device (200) according to one embodiment of the present disclosure can play second media data received from a source device (100) through a second transmission path by processing it through a video processing unit (230). The display device (200) can output the second media data to a display (240).

[0117] The processor (210) is electrically connected to the components included in the display device (200) and can perform operations or data processing regarding the control and / or communication of the components included in the display device (200). In one embodiment of the present disclosure, the processor (210) can load a request, command, or data received from at least one of the other components into memory for processing and store the processing result data in memory. According to one or more embodiments, the processor (210) may include at least one of a general-purpose processor such as a CPU (central processing unit), AP (application processor), DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU (graphic processing unit) or VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU (neural processing unit).

[0118] The processor (210) can process input data or control other configurations to process it according to data, operation rules, algorithms, methods, or models stored in memory (250). The processor (210) can perform operations of predefined operation rules, algorithms, methods, or models stored in memory (250) using the input data.

[0119] The memory (250) is electrically connected to the processor (210) and can store one or more modules, algorithms, operation rules, models, programs, instructions, or data related to the operation of components included in the display device (200). For example, the memory (250) can store one or more modules, algorithms, operation rules, models, programs, instructions, or data for processing and controlling the processor (210). The memory (250) may include, but is not limited to, at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0120] A processor (210) according to one embodiment of the present disclosure can receive first media data from a source device (100) through a first transmission path by executing one or more instructions stored in memory (250).

[0121] A processor (210) according to one embodiment of the present disclosure can obtain second media metadata for accessing second media data based on the loss of a first transmission path by executing one or more instructions stored in memory (250).

[0122] A processor (210) according to one embodiment of the present disclosure may request second media data from a source device (100) based on second media metadata by executing one or more instructions stored in memory (250).

[0123] A processor (210) according to one embodiment of the present disclosure can receive second media data from a source device (100) through a second transmission path different from a first transmission path by executing one or more instructions stored in a memory (250).

[0124] FIG. 4 is a flowchart illustrating a method of operation of a source device according to one embodiment of the present disclosure. The method of operation of a source device (100) according to one embodiment of the present disclosure can be performed by a processor (110 of FIG. 3) of the source device (100).

[0125] Referring to FIG. 4, in operation 410, the source device (100) can acquire the first media data.

[0126] A source device (100) according to one embodiment of the present disclosure can acquire first media data through a tuner unit (120). For example, the tuner unit (120) of the source device (100) can receive an RF (Radio Frequency) signal from a broadcast network and process the RF signal to output first media data, which is digital data. For example, the tuner unit (120) can select a signal of a specific channel from the RF signal received from the broadcast network, demodulate it, and acquire first media data corresponding to digital data. For example, the first media data may be data packetized into a first communication protocol, which is a standard available in the broadcast network. In this case, the first media data may be referred to as RF data, but is not limited thereto.

[0127] When a source device (100) according to one embodiment of the present disclosure receives an ATSC 3.0 broadcast signal, the first media data may be an ALP packet that is packetized into an ALP (ATSC Link-Layer Protocol), which is a Link-Layer protocol used in an ATSC 3.0 broadcast system. For example, the first media data may include a packet standardized to an MMT / UDP / IP protocol or a packet standardized to a ROUTE / UDP / IP protocol. For example, the first media data may include OTA (Over-The-Air) streaming data that is generated in MPU format and packetized into an MMT packet, or generated in DASH segment format and packetized into a ROUTE packet.

[0128] In one embodiment of the present disclosure, the first media data is exemplified as being received through a tuner unit (120), but is not limited thereto. For example, the first media data may be data received by the source device (100) from an external device through an input / output unit (130). In this case, the first media data may be data standardized to a communication protocol (e.g., HDMI standard protocol) that can be received through the input / output unit (130) (e.g., an HDMI port). Additionally, for example, the first media data may be data received from an external device through a communication unit (140).

[0129] According to one embodiment of the present disclosure, the first media data may include content such as video, audio, and / or additional information. The first media data may be separated into video, audio, and / or additional information by a display device (200). Based on the separated data, the display device (200) may display the content on a screen.

[0130] In operation 420, the source device (100) can transmit the first media data to the display device (200) through the first transmission path.

[0131] A source device (100) and a display device (200) according to one embodiment of the present disclosure may be connected via a first transmission path. The first transmission path may be a path for transmitting first media data based on a first communication module that enables direct communication between the source device (100) and the display device (200). Since the source device (100) can directly transmit the first media data to the display device (200) based on the first transmission path, large-capacity media data can be transmitted quickly and stably.

[0132] A first communication module according to one embodiment of the present disclosure may include a short-range communication module. For example, the first communication module may include a Wi-Fi 7 module. The first communication module may enable direct communication between Wi-Fi devices without passing through an access point (AP) that supports connection between devices equipped with a Wi-Fi 7 module (hereinafter referred to as Wi-Fi devices). However, it is not limited thereto, and the first communication module may include a WLAN module (or Wi-Fi module), a Bluetooth module, a Zigbee module, an infrared (IrDA, infrared Data Association) module, a WFD (Wi-Fi Direct) module, etc.

[0133] A source device (100) according to one embodiment of the present disclosure can transmit the first media data to a display device (200) through a first transmission path without separate processing of the first media data. For example, the source device (100) can transmit the first media data to the display device (200) without processing such as demuxing or parsing to extract data of a layer higher than the physical layer (e.g., internet layer, transport layer, etc.).

[0134] In one embodiment of the present disclosure, the first transmission path may be a wireless communication path. However, not limited thereto, the first transmission path may be a wired communication path. In this case, the first communication module may include wired Ethernet.

[0135] In operation 430, the source device (100) can identify whether the first transmission path with the display device (200) has been lost.

[0136] A source device (100) according to one embodiment of the present disclosure monitors a first transmission path and can identify whether the first transmission path is maintained, lost, or recovered. The source device (100) can identify whether the first transmission path is lost while transmitting first media data through the first transmission path.

[0137] A source device (100) according to one embodiment of the present disclosure may include a first connection management module (e.g., 730 in FIG. 7, or 930 in FIG. 9) that monitors the status of a first transmission path through a first communication module of the source device (100). The first connection management module may identify that the first transmission path is lost if a signal received or transmitted by the first communication module is not received or transmitted within a set time. Alternatively, the first connection management module may identify that the first transmission path is lost if the source device (100) transmits a connection status check signal to the display device (200), but there is no response from the display device (200). The method by which the first connection management module monitors the status of the first transmission path is not limited to the examples described above.

[0138] For example, while the first transmission path is maintained, a first pipeline (e.g., 710 in FIG. 7) for transmitting the first media data through the first transmission path may be activated. The source device (100) may deactivate the first pipeline (e.g., 710 in FIG. 7) when the first transmission path is lost. The source device (100) may activate a second pipeline (e.g., 910 in FIG. 9) for transmitting the second media data through the second transmission path when the first transmission path is lost.

[0139] Here, maintaining the first transmission path may indicate a case where the network environment supporting the first transmission path is stable. Losing the first transmission path may indicate a case where the first transmission path is disconnected or the network environment supporting the first transmission path is unstable. Recovering the first transmission path may indicate a case where the lost first transmission path is reconnected.

[0140] A source device (100) according to one embodiment of the present disclosure monitors a first transmission path, and if the first transmission path is disconnected or unstable, it may use an indirect communication method between devices using an internet network instead of a direct communication method between devices.

[0141] In operation 440, the source device (100) can convert the first media data into the second media data based on the fact that the first transmission path has been lost. In operation 450, the source device (100) can store the second media data on a local web server.

[0142] A source device (100) according to one embodiment of the present disclosure can generate second media data to be transmitted indirectly to a display device (200) via an internet network connected to a second transmission path when the first transmission path is lost. The source device (100) can store the second media data within the local web server by establishing a local web server inside the source device (100).

[0143] A source device (100) according to one embodiment of the present disclosure can generate data (second media data) packetized in a second communication protocol based on data (first media data) packetized in a first communication protocol. For example, the source device (100) can extract one or more media files contained in the first media data. The source device (100) can standardize one or more media files according to a second communication protocol different from the first communication protocol.

[0144] For example, the second communication protocol may be a standard available on the Internet (broadband). For example, the second media data may include packets standardized to the HTTP / TCP / IP protocol.

[0145] For example, the source device (100) can convert OTA streaming data, which is generated in MPU format and converted into MMT packets, or generated in DASH segment format and converted into ROUTE packets, to generate OTT streaming data that has a DASH segment format and can be transmitted as an HTTP message. The display device (200) can assign time information to each segment through an index or a timestamp, etc. This ensures that segments are not lost or out of order. This is explained in more detail in the second pipeline (910) of FIG. 9.

[0146] A source device (100) according to one embodiment of the present disclosure may establish a web server at its local host address (e.g., 127.0.0.1) and store second media data at the local host address. For example, the source device (100) may store one or more segments corresponding to the second media data at the local web server along with time information at which each segment was created. The time information of each segment may be used to match with playback start time information when the display device (200) requests the segment later.

[0147] In one embodiment of the present disclosure, the source device (100) may periodically generate segments according to the length of the segment (e.g., 2 seconds) from the time when the first transmission path is lost (e.g., 12:00:00) and store the generated segments on a local web server. The order or number of the segments stored on the local web server may be distinguished according to the time of storage.

[0148] Meanwhile, in one embodiment of the present disclosure, the source device (100) transmits the first media data received through the HDMI port to the display device (200) through the first transmission path, and even if the first transmission path is lost, the source device (100) may operate according to operation 440 and operation 450. For example, the source device (100) may receive the first media data standardized in HDMI through the HDMI port, convert the first media data standardized in HDMI to generate second media data standardized based on the second communication protocol, and store the second media data on a local web server. For example, the source device (100) may divide the first media data standardized in HDMI into segments at predetermined time intervals, add time information to each segment using a timestamp, and store the generated segments on a local web server. Accordingly, even while the source device (100) transmits the first media data received through the HDMI port to the display device (200), if direct communication between the source device (100) and the display device (200) is unstable, data transmission can be maintained through indirect communication between the source device (100) and the display device (200) to ensure stable video viewing for the user.

[0149] In operation 460, the source device (100) can transmit second media data to the display device (200) through the second transmission path.

[0150] A source device (100) and a display device (200) according to one embodiment of the present disclosure may be connected via a second transmission path. The second transmission path may be a path for transmitting second media data based on a second communication module that enables communication between the source device (100) and the display device (200) through an access point.

[0151] A second communication module according to one embodiment of the present disclosure may include a wireless internet module. For example, the second communication module may include a WLAN module (Wi-Fi module). The second communication module may be a module different from the first communication module. In one embodiment of the present disclosure, the second transmission path may be a wireless communication path. However, it is not limited thereto, and the second transmission path may be a wired communication path.

[0152] In one embodiment of the present disclosure, the source device (100) can operate as a broadband IP server by establishing a local web server. For example, when the source device (100) receives an HTTP request from a web client (e.g., a display device (200)) connected to the same network as the source device (100), it can transmit second media data to the web client in a streaming form over the internet network.

[0153] In one embodiment of the present disclosure, a source device (100) may receive a request for a segment from a display device (200) using a URL that includes information on the playback start time of the segment and location information of a local web server. In response to the segment request, the source device (100) may transmit a segment corresponding to the playback start time information of the segment among one or more segments to the display device (200) through a second transmission path. The source device (100) may identify a segment created at the playback start time requested by the display device (200) and transmit it to the display device (200).

[0154] In one embodiment of the present disclosure, the playback start time information of a segment may correspond to the time when the first transmission path is lost. The segment corresponding to the playback start time information of the segment may correspond to a segment generated by the source device (100) at the time when the first transmission path is lost. The source device (100) may sequentially transmit to the display device (200), starting from the segment generated at the time when the first transmission path is lost. This is explained in FIG. 11.

[0155] A source device (100) according to one embodiment of the present disclosure may delete a segment stored on a local web server if a predetermined condition is satisfied. For example, the source device (100) may delete a segment identified as having been transmitted to a client from the local web server. For example, the source device (100) may delete a segment from the local web server after a predetermined time has elapsed since storage. However, it is not limited thereto.

[0156] In one embodiment of the present disclosure, even when direct communication between the source device (100) and the display device (200) is unstable, the source device (100) and the display device (200) maintain data transmission using indirect communication through an access point, so that stable video viewing by the user can be guaranteed.

[0157] FIG. 5 is a flowchart illustrating a method of operation of a display device according to one embodiment of the present disclosure. The method of operation of a display device (200) according to one embodiment of the present disclosure may be performed by a processor (210 in FIG. 3) of the display device (200).

[0158] Referring to FIG. 5, in operation 510, the display device (200) can receive first media data from the source device (100) through a first transmission path.

[0159] A source device (100) and a display device (200) according to one embodiment of the present disclosure may be connected via a first transmission path. The first transmission path may be a path for receiving first media data based on a first communication module that enables direct communication between the source device (100) and the display device (200).

[0160] A display device (200) according to one embodiment of the present disclosure can directly receive first media data from a source device (100) based on a first transmission path, so that large volume media data can be received quickly and stably.

[0161] A first communication module according to one embodiment of the present disclosure may include a short-range communication module. For example, the first communication module may include a Wi-Fi 7 module. The first communication module may enable direct communication between Wi-Fi devices without passing through an access point (AP) that supports connection between devices equipped with a Wi-Fi 7 module (hereinafter referred to as Wi-Fi devices). However, it is not limited thereto, and the first communication module may include a WLAN module (or Wi-Fi module), a Bluetooth module, a Zigbee module, an infrared (IrDA, infrared Data Association) module, a WFD (Wi-Fi Direct) module, etc.

[0162] In one embodiment of the present disclosure, the first media data may be data standardized by a first communication protocol corresponding to the method of receiving content received by the source device (100) from an external device. For example, if the content received by the source device (100) is broadcast content through a broadcast network, the first media data may be data packetized by a communication protocol available in the broadcast network. For example, the first media data may include packets standardized by an MMT / UDP / IP protocol or packets standardized by a ROUTE / UDP / IP protocol. For example, the first media data may include OTA (Over-The-Air) streaming data generated in MPU format and packetized in MMT, or generated in DASH segment format and packetized in ROUTE.

[0163] However, this is not limited thereto, and in this case, if the content received by the source device (100) is content received from an external device through an HDMI port, the first media data may be data standardized according to the HDMI standard.

[0164] In one embodiment of the present disclosure, the first transmission path may be a wireless communication path. However, not limited thereto, the first transmission path may be a wired communication path. In this case, the first communication module may include wired Ethernet.

[0165] According to one embodiment of the present disclosure, the first media data may include content such as video, audio, and / or additional information. The first media data may be separated into video, audio, and / or additional information by a display device (200). Based on the separated data, the display device (200) may display the content on a screen.

[0166] In operation 520, the display device (200) can play the first media data.

[0167] A display device (200) according to one embodiment of the present disclosure may perform processing such as demuxing or parsing to extract data from a layer higher than the physical layer (e.g., internet layer, transport layer, etc.). The display device (200) may display content on a screen by performing processing such as decoding and rendering for the first media data.

[0168] In operation 530, the display device (200) can identify whether the first transmission path with the source device (100) has been lost.

[0169] A display device (200) according to one embodiment of the present disclosure can monitor a first transmission path and identify whether the first transmission path is maintained, lost, or recovered. The display device (200) can identify whether the first transmission path is lost while receiving first media data through the first transmission path.

[0170] A display device (200) according to one embodiment of the present disclosure may include a second connection management module (e.g., 740 in FIG. 7, or 940 in FIG. 9) that monitors the status of a first transmission path through a first communication module of the display device (200). The second connection management module may identify the first transmission path as lost if a signal received or transmitted by the first communication module is not received or transmitted within a set time. Alternatively, the second connection management module may identify the first transmission path as lost if the display device (200) transmits a connection status check signal to the source device (100), but there is no response from the source device (100). The method by which the second connection management module monitors the status of the first transmission path is not limited to the examples described above.

[0171] For example, while the first transmission path is maintained, a first playback pipeline (e.g., 720 in FIG. 7) for transmitting the first media data through the first transmission path may be activated. The display device (200) may disable the first playback pipeline when the first transmission path is lost. When the first transmission path is lost, the display device (200) may activate a second playback pipeline (e.g., 920 in FIG. 9) for transmitting the second media data through the second transmission path.

[0172] Here, "maintaining the first transmission path" may indicate a case where the first transmission path is stably connected. "Losing the first transmission path" may indicate a case where the first transmission path is disconnected or unstable. "Recovering the first transmission path" may indicate a case where the lost first transmission path is reconnected.

[0173] A display device (200) according to one embodiment of the present disclosure monitors a first transmission path, and if the first transmission path is disconnected or unstable, it may use an indirect communication method between devices using an internet network instead of a direct communication method between devices.

[0174] In operation 540, the display device (200) can generate second media metadata based on the fact that the first transmission path has been lost.

[0175] According to one embodiment of the present disclosure, the second media metadata may include information for a display device (200) to access a local web server and request one or more segments corresponding to the second media data. The second media metadata may include bibliographic details for the segments corresponding to the second media data. The second media metadata may correspond to a Media Presentation Description (MPD) file.

[0176] According to one embodiment of the present disclosure, the second media metadata may include location information of a local web server of a source device (100) and information on the availability start time of a segment. Here, the location information of the local web server may represent the local IP address of the local web server (e.g., 127.0.0.1). The display device (200) may have the local IP address of the source device (100) connected to the same network as the source device (100) stored in advance. Here, the availability start time information may be used as the filename of a segment to request a specific file among a plurality of files stored on the local web server. The availability start time information may include information used to determine the first segment to request from the local web server. For example, the display device (200) may update the availability start time to the point in time when the first transmission path is lost. In this case, the display device (200) may request from the segment stored on the local web server at the point in time when the first transmission path is lost.

[0177] However, this is not limited thereto, and the second media metadata may further include the start time, end time, period, filename, length, content type, etc. of each segment.

[0178] A display device (200) according to one embodiment of the present disclosure can generate second media metadata for second media data using channel list information received together with first media data through a first transmission path. The channel list information is received together with a broadcast signal received by a source device (100) through a tuner unit and may include a Service List Table (SLT). The channel list information received together with the first media data may be referred to as the first media metadata. The display device (200) can generate second media metadata for requesting second media data by adding local web server location information and playback start time information, etc., to the channel list information received together with the first media data.

[0179] In operation 550, the display device (200) may request second media data from the source device (100) based on second media metadata. In operation 560, the display device (200) may receive second media data from the source device (100) through a second transmission path.

[0180] A display device (200) according to one embodiment of the present disclosure may request a segment using a URL (e.g., http / 127.0.0.1 / start=a) that includes location information of a local web server of a source device (100) (e.g., 127.0.0.1) and a filename of a segment (e.g., start=a). The filename of the segment may be the playback start time of the segment, but is not limited thereto.

[0181] A display device (200) according to one embodiment of the present disclosure can access a local web server based on location information of the local web server.

[0182] A display device (200) according to one embodiment of the present disclosure may determine which point in time a segment among one or more segments stored on a local web server will be requested based on playback start time information. For example, the display device (200) may request a segment stored at a time corresponding to the playback start time information of the segment and receive the segment. That is, the display device (200) may request a segment stored at the same time as the playback start time of the source device (100) and receive the segment.

[0183] For example, the playback start time information may correspond to the time when the first transmission path is lost. The display device (200) may request a segment stored on a local web server at a time corresponding to the time when the first transmission path is lost. For example, if the first transmission path is lost at 12:00:00, the display device (200) may set 12:00:00 as the playback start time of the segment and request a segment stored on a local web server starting from 12:00:00.

[0184] In this case, a source device (100) according to one embodiment of the present disclosure may transmit to the display device (200) a segment stored at the same time as the playback start time, based on playback start time information requested by the display device (200). The order of each segment stored on a local web server may be distinguished according to the storage time of each segment. For example, if the playback start time is 12:00:00, the source device (100) may transmit to the display device (200) a segment stored at 12:00:00 in accordance with the request of the display device (200). Here, the time at which the segment is stored may mean the time at which the first media data is converted into the second media data and stored in a predetermined path of the local web server.

[0185] The playback start time information recorded in the display device (200) can be recorded according to standard time (e.g., UTC). Additionally, the creation time information recorded in the source device (100) can be recorded according to standard time (e.g., UTC). Since the web client and the local web server refer to the same standard time, they can be time-synchronized with each other.

[0186] The source device (100) can generate a segment from the point in time when the first transmission path is lost and transmit it through the second transmission path according to the request of the display device (200), so that the user's real-time video viewing can be guaranteed.

[0187] The operation of the source device (100) and the display device (200) requesting and transmitting segments is described in more detail in FIG. 11.

[0188] In operation 570, the display device (200) can play the second media data.

[0189] A display device (200) according to one embodiment of the present disclosure may perform processing such as demuxing or parsing on second media data. For example, the display device (200) may perform processing on second media data that has been packetized according to the specifications of a second communication protocol. The display device (200) may perform processing such as decoding and rendering on second media data to display content on a screen.

[0190] FIG. 6 is a flowchart illustrating a method of operation for a source device and a display device connected through a first transmission path and a second transmission path according to an embodiment of the present disclosure. In FIG. 6, descriptions that overlap with FIG. 4 and FIG. 5 are omitted. In FIG. 6, descriptions that overlap with the descriptions of the first pipeline (710), the second pipeline (910), the first playback pipeline (720), and the second playback pipeline (920) illustrated in FIG. 7 and FIG. 9, which will be described later, are omitted.

[0191] Referring to FIG. 6, in operation 610, the source device (100) can activate the first pipeline (710) when connected to the display device (200) via the first transmission path (10). In operation 615, the display device (200) can activate the first playback pipeline (720) when connected to the source device (100) via the first transmission path (10). The first pipeline (710) of the source device (100) according to one embodiment of the present disclosure may include appropriate logic, circuits, interfaces, and / or code for transmitting first media data received from an external device via the first transmission path (10). The first playback pipeline (720) of the display device (200) according to one embodiment of the present disclosure may include appropriate logic, circuits, interfaces, and / or code for playing first media data received via the first transmission path (10).

[0192] In operation 620, the source device (100) can transmit the first media data to the display device (200) through the first transmission path (10). The display device (200) can receive the first media data from the source device (100) through the first transmission path (10). In operation 625, the display device (200) can play the first media data using the first playback pipeline (720). Operation 620 may correspond to operation 420 of FIG. 4 and operation 510 of FIG. 5. Operation 625 may correspond to operation 520 of FIG. 5.

[0193] In operations 630 and 635, the source device (100) and the display device (200) can each identify whether the first transmission path (10) is lost. For example, the source device (100) can monitor the status of the first transmission path (10) through a first connection management module (e.g., 730 in FIG. 7, or 930 in FIG. 9) that monitors the status of the first transmission path (10) through the first communication module of the source device (100). Also, for example, the display device (200) can monitor the status of the first transmission path (10) through a second connection management module (e.g., 740 in FIG. 7, or 940 in FIG. 9) that monitors the status of the first transmission path (10) through the first communication module of the display device (200). Operation 630 may correspond to operation 430 of FIG. 4. Operation 635 can correspond to operation 530 of FIG. 5.

[0194] In operation 640, the source device (100) may activate a second pipeline (910) to be connected to the display device (200) via a second transmission path (20) when the first transmission path (10) is lost. However, when the source device (100) is not lost, it may maintain the activation state of the first pipeline (710) according to operation 610.

[0195] In operation 645, the display device (200) may activate a second playback pipeline (920) to be connected to the source device (100) via a second transmission path (20) when the first transmission path (10) is lost. However, when the first transmission path (10) is not lost, the display device (200) may maintain the activation state of the first playback pipeline (720) according to operation 615.

[0196] A second pipeline (910) of a source device (100) according to one embodiment of the present disclosure may include appropriate logic, circuits, interfaces, and / or code for generating second media data that can be transmitted via an internet network from first media data and transmitting the second media data through a second transmission path (20). A second playback pipeline (920) of a display device (200) according to one embodiment of the present disclosure may include appropriate logic, circuits, interfaces, and / or code for playing the second media data received through the second transmission path (20).

[0197] In operation 650, the source device (100) can generate second media data from first media data through the second pipeline (910). The source device (100) can store the second media data on a local web server. Operation 650 can correspond to operations 440 and 450 of FIG. 4.

[0198] In operation 655, the display device (200) can acquire second media metadata. Operation 655 can correspond to operation 540 of FIG. 5.

[0199] In operation 660, the display device (200) may request second media data from the source device (100) via the second transmission path (20) based on the second media metadata. The source device (100) may receive the request for second media data from the display device (200) via the second transmission path (20). Operation 660 may correspond to operation 550 of FIG. 5.

[0200] In operation 665, the source device (100) may transmit second media data to the display device (200) via the second transmission path (20) in response to a request. The display device (200) may receive the second media data from the source device (100) via the second transmission path (20). Operation 665 may correspond to operation 460 of FIG. 4 and operation 560 of FIG. 5.

[0201] In operation 670, the display device (200) can play the second media data using the second playback pipeline (920). Operation 670 can correspond to operation 570 of FIG. 5.

[0202] In operation 680, the source device (100) can identify whether the first transmission path (10) has been restored. If the first transmission path (10) has been restored, the source device (100) can activate the first pipeline (710) and deactivate the second pipeline (910) according to operation 610. The source device (100) can transmit the first media data to the display device (200) through the first transmission path (10) according to the first pipeline (710). However, if the first transmission path (10) has not been restored, the source device (100) can maintain the activation state of the second pipeline (910) according to operation 640.

[0203] In operation 685, the display device (200) can identify whether the first transmission path (10) has been restored. If the first transmission path (10) has been restored, the display device (200) can activate the first playback pipeline (720) and deactivate the second playback pipeline (920) according to operation 615. The display device (200) can receive the first media data from the source device (100) through the first transmission path (10) according to the first playback pipeline (720). However, if the first transmission path (10) has not been restored, the display device (200) can maintain the activated state of the second playback pipeline (920) according to operation 645.

[0204] In a system comprising a source device (100) and a display device (200) according to one embodiment of the present disclosure, resources (e.g., ALP Demux, ATSC3 Demux) that were used in the display device (200) while the first transmission path (10) is maintained may be switched to be used in the source device (100) while the first transmission path (10) is lost. For example, the first pipeline (710 in FIG. 7) and the first playback pipeline (720 in FIG. 7) may be logic implemented to transmit first media data received by the source device (100) to the display device (200) through the first transmission path (10), and for the display device (200) to process the first media data. On the other hand, the second pipeline (910) and the second playback pipeline (920) may be logic implemented such that when the first transmission path is lost, the source device (100) generates second media data from the first media data, transmits the second media data to the display device (200) through the second transmission path (20), and the display device (200) processes the second media data.

[0205] Meanwhile, the transmission rate per second of the first media data may be greater than the transmission rate per second of the second media data.

[0206] FIG. 7 is a diagram illustrating the operation of a source device and a display device connected to a first transmission path according to one embodiment of the present disclosure. FIG. 8 is a diagram illustrating the operation of the first demuxer of FIG. 7. In FIG. 7 and FIG. 8, the case where the source device (100) receives an ATSC 3.0 broadcast signal is described exemplarily, but the input signal received by the source device (100) is not limited to the example. For example, if the input signal received by the source device (100) is an HDMI signal, the first demuxer (722) may be replaced with a configuration for parsing and extracting data standardized according to the HDMI standard.

[0207] Referring to FIG. 7, a source device (100) and a display device (200) according to one embodiment of the present disclosure may be connected via a first transmission path. The first transmission path may be a path for transmitting first media data based on a first communication module that enables direct communication between the source device (100) and the display device (200). Here, the first transmission path may be a wireless communication path, but is not limited thereto.

[0208] A first transmission path according to one embodiment of the present disclosure may be connected through a first pipeline (710) of a source device (100) and a first playback pipeline (720) of a display device (200).

[0209] The first pipeline (710) of the source device (100) may include a tuner section (711) and a virtual output section (712). The first pipeline (710) may be connected to a first connection management module (730). The tuner section (711) may correspond to the tuner section (120) of FIG. 3.

[0210] The tuner unit (711) receives an RF (Radio Frequency) signal from a broadcast network and processes the RF signal to output a first media data, which is digital data. In this case, the first media data may be referred to as RF data, but is not limited thereto. The first media data may be transmitted to the first playback pipeline (720) through the first pipeline (710).

[0211] When a tuner unit (711) according to one embodiment of the present disclosure receives an ATSC 3.0 broadcast signal, a demodulator belonging to the tuner unit (711) can convert physical layer data to the next layer, the link layer (e.g., ALP (ATSC 3.0 Link Layer Protocol)). The first media data may be an ALP packet packetized into ALP (ATSC Link-Layer Protocol), which is a link-layer protocol used in the ATSC 3.0 broadcast system. The ALP packet may contain data of an upper layer (e.g., MMTP or IP). For example, IP packets may be loaded into the payload of the ALP packet. The header of the ALP packet includes a packet type field, and the packet type field may be used to identify the type of data packet at the receiver. For example, the first media data may include a packet standardized to the MMT / UDP / IP protocol or a packet standardized to the ROUTE / UDP / IP protocol. For example, the first media data may include OTA (Over-The-Air) streaming data generated in MPU format and packetized in MMT, or generated in DASH segment format and packetized in ROUTE.

[0212] In one embodiment of the present disclosure, the first media data is exemplified as being received through the tuner unit (711), but is not limited thereto. For example, the first media data may be data received from an external device through the input / output unit (130) of FIG. 3. For example, the first media data may be data standardized into a communication protocol that can be received through an HDMI port.

[0213] A first pipeline (710) according to one embodiment of the present disclosure can transmit extracted first media data to a first playback pipeline (720) through a first transmission path. Since 1,500 bytes are transmitted per packet and 1,400 packets are transmitted per second, 2 Mbtye of data can be transmitted per second.

[0214] The first playback pipeline (720) may include a virtual receiver (721), a first demuxer (722), a decoder (723), and an output unit (724). The first playback pipeline (720) may be connected to a second connection management module (740). The decoder (723) and the output unit (724) may be included in the video processing unit (230) of FIG. 3.

[0215] The first demuxer (722) can parse an ALP packet to identify that the data within the ALP packet is network streaming data of the IP packet type, and separate the data into each stream (video, audio, additional information, etc.). Specifically, referring to FIG. 8, the first demuxer (722) may include an ALP demuxer (810) that receives an ALP packet and extracts an IP / UDP packet. The ALP demuxer (810) can parse the received ALP packet to identify the packet type and extract it as an IP / UDP packet. The extracted IP / UDP packet may be transmitted to an ATSC3 demuxer (820). The first demuxer (722) may further include an ATSC3 demuxer (820) that receives the IP / UDP packet and assembles it into an MMT / ROUTE packet. A media file in the form of a fragment MP4 may be loaded into the payload of the MMT / ROUTE packet (see FIG. 10b). The MMT / ROUTE packet may be passed to an ATSC3 FFMPEG demuxer (830). The first demuxer (722) may further include an ATSC3 FFMPEG demuxer (830) that parses the MMT / ROUTE packet to extract an Elementary Stream (ES) from the fragment MP4. The ATSC3 FFMPEG demuxer (830) may be a demuxer that separates the compressed video and audio streams from the fragment MP4 and converts them into data that a decoder can recognize. The Video ES may be referred to as Encoded Video Data.

[0216] Each of the video stream and audio stream can be passed to a video decoder and an audio decoder within the decoder (723). Each of the video decoder and the audio decoder can decode the compressed media data and pass it to an output unit (724). The output unit (724) may include a rendering unit. When the compressed video stream (e.g., HEVC) is decoded, the output unit (724) can render it to the screen frame by frame.

[0217] A first transmission path according to one embodiment of the present disclosure may be a path for a virtual connection that connects two devices software-wise without a physical connection. The virtual connection may be a connection between two devices via wireless communication over a network. In this case, between the tuner section (711) of the first pipeline (710) and the first demuxer (722) of the first playback pipeline (720), a virtual output section (712) and a virtual receiver section (721), which are software modules for configuring each pipeline, may be included. The virtual output section (712) may operate as an output terminal in the input-output structure of the first pipeline (710). The virtual receiver section (721) may operate as an input terminal in the input-output structure of the first playback pipeline (720). However, the virtual output section (712) and the virtual receiver section (721) may be omitted.

[0218] A first connection management module (730) according to one embodiment of the present disclosure can monitor a first communication module of a source device (100). The first connection management module (730) can identify that a first transmission path has been lost if a signal received or transmitted by the first communication module is not received or transmitted within a set time. The method by which the first connection management module (730) monitors the status of the first transmission path is not limited to the examples described above. If the first connection management module (730) monitors the first communication module and determines that the first transmission path has been lost, it can transmit a signal to the source device (100) notifying it of the loss of the first transmission path. When the source device (100) receives a signal notifying it of the loss of the first transmission path, it can disable the first pipeline (710). When the source device (100) receives a signal notifying it of the loss of the first transmission path, it can enable the second pipeline (910) of FIG. 9, which will be described later. For example, the source device (100) can generate a disable signal for the first pipeline (710) and an enable signal for the second pipeline (910).

[0219] A second connection management module (740) according to one embodiment of the present disclosure can monitor a first communication module of a display device (200). The second connection management module (740) can identify that a first transmission path has been lost if a signal received or transmitted by the first communication module is not received or transmitted within a set time. The method by which the second connection management module (740) monitors the status of the first transmission path is not limited to the example described above. If the second connection management module (740) monitors the first communication module and determines that the first transmission path has been lost, it can transmit a signal to the display device (200) indicating the loss of the first transmission path. When the display device (200) receives a signal indicating the loss of the first transmission path, it can disable the first regeneration pipeline (720). When the display device (200) receives a signal indicating the loss of the first transmission path, it can enable the second regeneration pipeline (920) of FIG. 9, which will be described later. For example, the display device (200) can generate a disable signal for the first playback pipeline (720) and an enable signal for the second playback pipeline (920).

[0220] In one embodiment of the present disclosure, the source device (100) and the display device (200) are illustrated as including a connection management module that monitors each of the first communication modules, but are not limited thereto. For example, the connection management module may be provided in only at least one of the source device (100) and the display device (200).

[0221] A source device (100) according to one embodiment of the present disclosure can transmit to a display device (200) via a first transmission path without separate processing of the first media data obtained from a tuner unit (711). For example, the source device (100) may omit the operation of extracting data from a layer higher than the physical layer (e.g., internet layer, transmission layer, etc.). For example, the source device (100) can transmit to the display device (200) without processing such as demuxing or parsing of the first media data. In this case, the display device (200) can perform higher layer processing such as demuxing or parsing of the first media data, and after decoding and rendering of the first media data, output content corresponding to the first media data.

[0222] FIG. 9 is a diagram illustrating the operation of a source device and a display device connected to a second transmission path according to an embodiment of the present disclosure. FIG. 10a is a diagram illustrating the operation of the second demuxer of FIG. 9. FIG. 10b is a diagram illustrating the operation of the packager of FIG. 9. FIG. 9a, and FIG. 10b are described exemplarily in which the source device (100) receives an ATSC 3.0 broadcast signal, but the input signal received by the source device (100) is not limited to the example. For example, if the input signal received by the source device (100) is an HDMI signal, the second demuxer (912) may be replaced with a configuration for parsing and extracting data standardized according to the HDMI standard.

[0223] Referring to FIG. 9, a source device (100) and a display device (200) according to one embodiment of the present disclosure may be connected via a second transmission path. The second transmission path may be a path for transmitting second media data based on a second communication module that enables communication between the source device (100) and the display device (200) through an access point (50). The second transmission path may be a path used to transmit media data when a wireless communication connection supporting the first transmission path is lost. Here, the second transmission path may be a wireless communication path, but is not limited thereto.

[0224] A second transmission path according to one embodiment of the present disclosure may be connected through a second pipeline (910) of a source device (100) and a second playback pipeline (920) of a display device (200).

[0225] The second pipeline (910) of the source device (100) may include a tuner section (911), a second demuxer (912), a packager (913), and a local web server (914). The second pipeline (910) may be connected to a first connection management module (930). The tuner section (911) may correspond to the tuner section (120) of FIG. 3 and the tuner section (711) of FIG. 7. The first connection management module (930) may correspond to the first connection management module (730) of FIG. 7.

[0226] A tuner unit (911) according to one embodiment of the present disclosure can receive an RF signal from a broadcasting network and process the RF signal to output a first media data which is digital data. The first media data can be transmitted to a second demuxer (912).

[0227] When a tuner unit (911) according to one embodiment of the present disclosure receives an ATSC 3.0 broadcast signal, the first media data may be an ALP packet. The ALP packet may contain upper layer data (e.g., MMT or IP packet). For example, the first media data may include a packet standardized to an MMT / UDP / IP protocol or a packet standardized to a ROUTE / UDP / IP protocol.

[0228] The second demuxer (912) can parse the first media data and extract upper layer data. For example, the second demuxer (912) can parse an ALP packet and extract an MMT / ROUTE packet. Referring to FIG. 10a, the second demuxer (912) may include an ALP demuxer (1010) that receives an ALP packet and extracts it into an IP / UDP packet. The ALP demuxer (1010) may correspond to the ALP demuxer (810) of FIG. 8. The second demuxer (912) may further include an ATSC3 demuxer (1020) that receives an IP / UDP packet and assembles it into an MMT / ROUTE packet. The ATSC3 demuxer (1020) may correspond to the ATSC3 demuxer (820) of FIG. 8. The ATSC3 demuxer (1020) can deliver an MMT / ROUTE packet to a packager (913). A media file can be loaded into the MMT / ROUTE packet.

[0229] The packager (913) can convert the format of media files received through a broadcast network so that they can be transmitted over the Internet. For example, referring to FIG. 10b, the packager (913) can convert MMT / ROUTE packets into DASH segment format. The packager (913) can parse MMT / ROUTE packets, remove the MMT / ROUTE header, and extract the media files (e.g., MPU format or DASH segment format) contained in the payload of the MMT / ROUTE packets. For example, the packager (913) can convert the extracted media files in MPU format into media files in DASH segment format. The packager (913) can add HTTP headers to the media files in DASH segment format. Here, the format of the media files is exemplified as fragment MP4 format, but is not limited thereto. DASH segments can be stored on a local web server (914).

[0230] A local web server (914) may store DASH segments at the local host address (e.g., 127.0.0.1) of the source device (100). When the local web server (914) receives a request for a segment from a web client via the HTTP / TCP / IP protocol, it may transmit the segment in response to the request. Each segment has a unique index (or order) and can be used when the display device (200) requests media data at a specific point in time. For example, the order of each segment may be distinguished by the time of creation (or storage) of each segment.

[0231] For example, when a local web server (914) receives a segment request from a display device (200) connected to a source device (100) via an access point (50) and a second transmission path, it may transmit the segment to a second playback pipeline (920) of the display device (200). Each segment may be transmitted in kBytes per second. This may be a smaller amount than the first media data transmitted through the first communication module.

[0232] A second playback pipeline (920) according to one embodiment of the present disclosure may include a receiver (921), a third demuxer (922), a decoder (923), and an output unit (924). The second playback pipeline (920) may be connected to a second connection management module (940) and a metadata acquisition module (950). The decoder (923) and the output unit (924) may be included in the video processing unit (230) of FIG. 3. The second connection management module (940) may correspond to the second connection management module (740) of FIG. 7.

[0233] The receiver (921) can receive MPEG-DASH segments standardized to the HTTP / TCP / IP protocol, which is a standard available on the internet network (broadband). The receiver (921) can transmit the MPEG-DASH segments to the third demuxer (922).

[0234] The third demuxer (922) can parse MPEG-DASH segments standardized to the HTTP / TCP / IP protocol to separate and extract video and audio streams that the decoder (923) can recognize.

[0235] Each of the video stream and audio stream can be passed to a video decoder and an audio decoder within the decoder (923). Each of the video decoder and the audio decoder can decode the compressed media data and pass it to an output unit (924). The output unit (924) may include a rendering unit. When the compressed video stream (e.g., HEVC) is decoded, the output unit (924) can render it to the screen frame by frame.

[0236] A first connection management module (930) according to one embodiment of the present disclosure can monitor a first communication module of a source device (100). The first connection management module (930) can monitor the first communication module of the source device (100) and determine whether the lost first transmission path has been recovered. When the source device (100) obtains a signal indicating the recovery of the first transmission path through the first connection management module (930), it can deactivate the second pipeline (910) and activate the first pipeline (710) of FIG. 7 described above.

[0237] A second connection management module (940) according to one embodiment of the present disclosure can monitor a first communication module of a display device (200). Similar to the first connection management module (930) described above, the second connection management module (940) can notify the display device (200) of the recovery of the first transmission path when the first transmission path is recovered. In this case, the display device (200) can disable the second regeneration pipeline (920) and enable the first regeneration pipeline (720) of FIG. 7 described above.

[0238] A metadata acquisition module (950) according to one embodiment of the present disclosure can acquire second media metadata. When the first transmission path is lost, the metadata acquisition module (950) can convert the first media metadata received together with the first media data to generate second media metadata for requesting the second media data. The second playback pipeline (920) can identify which segment to request and in what order based on the generated second media metadata. This is described in FIG. 11.

[0239] FIG. 11 is a diagram illustrating a second media data playback process between a source device and a display device according to one embodiment of the present disclosure.

[0240] Referring to FIG. 11, the source device (100) can generate one or more segments containing streaming data and upload them to a local web server. The segments may correspond to MPEG-DASH segments. The segments may represent video fragments divided into specific time units (e.g., lengths of about 1 second to about 10 seconds). The source device (100) can store the MPEG-DASH segments on a local web server. The source device (100) can store the filenames, creation time information, etc., of the MPEG-DASH segments together. For example, the source device (100) can store the segments at a local IP address such as 127.0.0.1. The source device (100) can generate segments from the point in time when the first transmission path is lost and upload them to a local web server.

[0241] The display device (200) can generate second media metadata containing information that allows access to a local web server. The second media metadata may correspond to an MPD file. The display device (200) inputs the MPD file into a DASH player to access the source device (100), which is a local web server, and can access one or more segments stored on the local web server. The player implemented in the display device (200) can load the segments using the MPD file and play the content. The MPD file may be referred to as a media description file, a manifest file, or a playlist.

[0242] The display device (200) can generate an MPD file for the second media data using channel list information received together with the first media data through a first transmission path. The channel list information is included together with the broadcast signal received by the source device (100) through the tuner unit and may include a Service List Table (SLT). The channel list information received together with the first media data may be referred to as the first media metadata.

[0243] The display device (200) can generate an MPD file for the second media data by adding local web server location information and playback start time information, etc., to channel list information received together with the first media data. The MPD file for the second media data may be referred to as second media metadata as information for requesting the second media data. For example, the display device (200) can generate second media metadata for requesting the second media data by converting the first media metadata received together with the first media data.

[0244] The drawing shows a first MPD file, a second MPD file, and a third MPD file for requesting a first segment, a second segment, and a third segment.

[0245] The MPD file may include information such as the playback start time of the segment (1110), the length of the segment (1120), the address of the local web server where the segment is stored (1130), the segment ID and the time of the segment start (1140), and the content type (1150). For example, the playback start time (1110) represents a standard start time available for streaming services and may be a timeline reference point for a streaming session. For example, the playback start time (1110) may be 12:00:00. In one embodiment of the present disclosure, the playback start time (1110) may represent the time when the first transmission path is lost. The length of the segment (1120) may be 2 seconds. The address of the local web server (1130) may represent a base path where the segment can be downloaded. For example, the address of the local web server (1130) may be the BaseURL 127.0.0.1. In the segment ID and segment start time (1140), the segment ID represents an identifier for a segment within the timeline of a streaming session, and the segment start time may represent the point or location where the segment begins in the timeline. For example, the segment ID and segment start time (1140) may be a segment where the segment ID is P0 and playback starts at 0 seconds. The content type (1150) may be stream information such as video, audio, and subtitles present in the Adaptation set. However, not limited thereto, the MPD file may further include information of the entire segment timeline (e.g., duration of a single segment (d), total number of segments within the segment (repeat, r), start time of the first segment within the segment (timestamp, t)).

[0246] The display device (200) can identify which segment to request and in what order based on the MPD file. The display device (200) can request a segment corresponding to the playback start time using the BaseURL defined in the MPD file. The request format is an HTTP GET request, and a URL with playback start time information added to the BaseURL can be used, for example, such as http: / 127.0.0.1 / start=12:00:00. That is, the playback start time information can be used as the filename of a segment to request a specific file among multiple files stored on a local web server. The source device (100) can respond to the request and transmit the first segment stored at the time corresponding to the playback start time to the local web server. The response format may be HTTP POST. For example, the source device (100) can transmit starting from the segment stored at the time when the first transmission path was lost.

[0247] For example, if the first transmission path is lost at 12:00:00, the display device (200) may set 12:00:00 as the playback start time of the segment and request from the local web server the segment stored at 12:00:00. The source device (100) may identify the segment stored at 12:00:00 in response to the request of the display device (200) and transmit it to the display device (200).

[0248] The display device (200) can continue to request segments from the source device (100) and proceed with playback until all media defined in the adaptation set of the MPD file is played. The display device (200) can stop requesting segments when the first transmission path is restored.

[0249] FIG. 12 is a drawing for exemplarily illustrating a screen of content output to a display device in various scenarios according to one embodiment of the present disclosure.

[0250] Referring to 1210 in FIG. 12, when a source device (100) delivers real-time video content to a display device (200) in a stable network environment, media files are shown being played on the display device (200). For example, the media files can be played sequentially in real-time from media file 1 to media file 6.

[0251] Referring to 1220, in a scenario where the first transmission path is lost, media files are displayed on the display device (200) when there is no second transmission path. In this case, the media files (media file 3, media file 4, and media file 5) from the time the first transmission path is lost until the time the first transmission path is restored may not be displayed on the display device (200). Accordingly, there is a problem in that some of the real-time video content cannot be provided to the user.

[0252] Referring to 1230, media files are displayed on a display device (200) when a second transmission path exists in a scenario where the first transmission path is lost. In this case, the display device (200) can play media files (media file 4 and media file 5) from the time the first transmission path was lost until the time the first transmission path was restored, based on the second transmission path. Accordingly, the source device (100) and the display device (200) can ensure uninterrupted real-time playback of the content.

[0253] Meanwhile, ideally, when the time at which the first transmission path is lost and the time at which the second transmission path is connected are the same, all of media files 3, 4, and 5 can be played on the display device (200). However, if the time at which the second transmission path is connected is later than the time at which the first transmission path is lost, media file 3 stored between the time at which the first transmission path is lost and the time at which the second transmission path is connected may not be played. FIG. 12 illustrates a situation in which media file 3 is not displayed, but is not limited thereto.

[0254] FIG. 13 is a detailed block diagram showing a display device according to one embodiment of the present disclosure.

[0255] Referring to FIG. 13, the display device (1300) may include a processor (1301), memory (1302), tuner unit (1310), communication unit (1320), detection unit (1330), input / output unit (1340), video processing unit (1350), display (1360), audio processing unit (1370), audio output unit (1380), and input interface (1390). The processor (1301), memory (1302), communication unit (1320), video processing unit (1350), and display (1360) included in the display device (1300) may perform the same operations as the processor (210), memory (250), communication unit (220), video processing unit (230), and display (240) included in the display device (200) of FIG. 3.

[0256] The tuner unit (1310) can select only the frequency of the channel to be received by the display device (1300) from among many radio wave components by tuning through amplification, mixing, resonance, etc. of broadcast content received via wired or wireless connection. The content received through the tuner unit (1310) is decoded and separated into audio, video, and / or additional information. The separated audio, video, and / or additional information can be stored in memory (1302) under the control of the processor (1301).

[0257] The communication unit (1320) can connect the display device (1300) to a peripheral device, external device, server, mobile terminal, etc. under the control of the processor (1301). The communication unit (1320) may include at least one communication module capable of performing wireless communication. The communication unit (1320) may include at least one of a wireless LAN module (1321), a Bluetooth module (1322), and a wired Ethernet (1323) in accordance with the performance and structure of the display device (1300).

[0258] The wireless LAN module (1321) can transmit and receive Wi-Fi signals with a peripheral device according to the Wi-Fi communication standard. The Bluetooth module (1322) can receive Bluetooth signals transmitted from a peripheral device according to the Bluetooth communication standard.

[0259] The detection unit (1330) detects the user's voice, the user's image, or the user's interaction and may include a microphone, a camera unit, an optical receiver, and a sensing unit.

[0260] The input / output unit (1340) can receive video (e.g., dynamic image signal or still image signal), audio (e.g., voice signal or music signal), and additional information from an external device, etc., under the control of the processor (1301). The input / output unit (1340) may include one of an HDMI port (High-Definition Multimedia Interface port), a component jack, a PC port, and a USB port.

[0261] The audio processing unit (1370) performs processing on audio data. Various processing such as decoding, amplification, and noise filtering on audio data can be performed in the audio processing unit (1370).

[0262] The audio output unit (1380) can output audio included in the content received through the tuner unit (1310) under the control of the processor (1301), audio input through the communication unit (1320) or input / output unit (1340), and audio stored in the memory (1302). The audio output unit (1380) may include at least one of a speaker, headphones, or S / PDIF (Sony / Philips Digital Interface: output terminal).

[0263] The input interface (1390) can receive user input for controlling the display device (1300). The input interface (1390) may include, but is not limited to, various forms of user input devices including, a touch panel for detecting user touch, a button for receiving user push operation, a wheel for receiving user rotation operation, a keyboard, a dome switch, a microphone for voice recognition, a motion detection sensor for sensing motion, etc.

[0264] A source device according to one embodiment of the present disclosure includes at least one processor comprising an image receiving unit, a communication unit, and a processing circuit, and a memory comprising one or more storage media for storing one or more instructions. By the at least one processor according to one embodiment of the present disclosure executing the one or more instructions individually or collectively, the source device transmits first media data obtained through the image receiving unit to a display device through a first transmission path. By the at least one processor according to one embodiment of the present disclosure executing the one or more instructions individually or collectively, the source device converts the first media data into second media data based on the loss of the first transmission path. By the at least one processor according to one embodiment of the present disclosure executing the one or more instructions individually or collectively, the source device stores the second media data on a local web server. According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device transmits the second media data stored in the local web server to the display device through a second transmission path different from the first transmission path, in response to a request by the display device.

[0265] According to one embodiment of the present disclosure, the first transmission path may include a communication circuit and a path for transmitting the first media data through a first communication module that enables direct communication between the source device and the display device.

[0266] According to one embodiment of the present disclosure, the second transmission path may include a communication circuit and a path for transmitting the second media data through a second communication module that enables communication between the source device and the display through a broadband internet network.

[0267] The first media data according to one embodiment of the present disclosure may include data based on a first communication protocol that is a standard available in a broadcast network.

[0268] According to one embodiment of the present disclosure, the second media data may include data based on a second communication protocol that is a standard available on the internet (broadband).

[0269] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can extract one or more media files included in the first media data standardized according to the first communication protocol based on the loss of the first transmission path.

[0270] According to one embodiment of the present disclosure, the source device can generate the second media data by having the at least one processor execute the one or more instructions individually or in combination, thereby standardizing the one or more media files according to a second communication protocol different from the first communication protocol.

[0271] The second media data according to one embodiment of the present disclosure may include one or more segments.

[0272] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can receive a request for a segment from the display device using a URL that includes information on the playback start time of the segment and location information of the local web server.

[0273] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can transmit a segment corresponding to the playback start time information of the segment among the one or more segments to the display device through the second transmission path in response to a request of the segment.

[0274] The playback start time information of the segment according to one embodiment of the present disclosure may correspond to the time when the first transmission path is lost.

[0275] A segment corresponding to the playback start time information of the segment according to one embodiment of the present disclosure may correspond to a segment generated by the source device at the time when the first transmission path is lost.

[0276] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can transmit the first media data to the display device through the first transmission path based on the recovery of the first transmission path.

[0277] A display device according to one embodiment of the present disclosure includes a communication unit, at least one processor, and a memory comprising one or more storage media for storing one or more instructions.

[0278] According to one embodiment of the present disclosure, the display device receives first media data from a source device through a first transmission path by executing the one or more instructions individually or collectively by the at least one processor.

[0279] According to one embodiment of the present disclosure, by the at least one processor executing the one or more instructions individually or in combination, the display device obtains second media metadata for accessing second media data based on the loss of the first transmission path.

[0280] According to one embodiment of the present disclosure, the display device requests the second media data from the source device based on the second media metadata by executing the one or more instructions individually or in combination by the at least one processor.

[0281] According to one embodiment of the present disclosure, the display device receives the second media data from the source device through a second transmission path different from the first transmission path by executing the one or more instructions individually or in combination.

[0282] According to one embodiment of the present disclosure, the first transmission path may include a path for receiving the first media data through a first communication module that enables direct communication between the source device and the display device.

[0283] According to one embodiment of the present disclosure, the second transmission path may include a path for receiving the second media data through a second communication module that enables communication between the source device and the display through a broadband internet network.

[0284] The first media data according to one embodiment of the present disclosure may include data based on a first communication protocol that is a standard available in a broadcast network.

[0285] According to one embodiment of the present disclosure, the second media data may include data based on a second communication protocol that is a standard available on the internet (broadband).

[0286] According to one embodiment of the present disclosure, the display device can generate the second media metadata including location information of the local web server of the source device and playback start time information of the segment by executing the one or more instructions individually or in combination.

[0287] The second media data according to one embodiment of the present disclosure may include one or more segments.

[0288] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the display device can transmit a request for the segment to the source device using a URL that includes information on the playback start time of the segment and location information of the local web server.

[0289] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the display device can receive a segment corresponding to the playback start time information of the segment from the source device through the second transmission path in response to a request for the segment.

[0290] The playback start time information of the segment according to one embodiment of the present disclosure may correspond to the time when the first transmission path is lost.

[0291] A segment corresponding to the playback start time information of the segment according to one embodiment of the present disclosure may correspond to a segment stored on the local web server at the time when the first transmission path is lost.

[0292] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the display device can extract a video stream by parsing a packet that has been packetized according to a first communication protocol corresponding to the first media data upon receiving the first media data.

[0293] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the display device can extract a video stream by parsing a packet that has been packetized according to a second communication protocol corresponding to the second media data upon receiving the second media data.

[0294] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the display device can receive the first media data from the source device through the first transmission path based on the recovery of the first transmission path.

[0295] A method of operation of a source device according to one embodiment of the present disclosure includes the steps of: acquiring first media data through a tuner unit; transmitting the first media data to a display device through a first transmission path; converting the first media data into second media data based on the fact that the first transmission path is lost; storing the second media data in a local web server; and transmitting the second media data stored in the local web server to the display device through a second transmission path different from the first transmission path, in response to a request from the display device.

[0296] The step of converting the first media data into the second media data according to one embodiment of the present disclosure may include the step of generating the second media data by extracting one or more media files included in the first media data standardized according to a first communication protocol based on the loss of the first transmission path, and standardizing the one or more media files according to a second communication protocol different from the first communication protocol.

[0297] The second media data according to one embodiment of the present disclosure may include one or more segments.

[0298] The step of transmitting the second media data to the display device through the second transmission path according to one embodiment of the present disclosure may include: receiving a request for a segment from the display device using a URL including information on the playback start time of the segment and location information of the local web server; and, in response to the request for the segment, transmitting a segment corresponding to the playback start time information of the segment among the one or more segments to the display device through the second transmission path.

[0299] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0300] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Claims

1. In a source device (100), Video receiver; Communication unit (140); At least one processor (110) including a processing circuit; and It includes a memory (150) comprising one or more storage media for storing one or more instructions, and By having the above at least one processor (110) execute the above one or more instructions individually or collectively, the source device (100) The first media data obtained through the above-mentioned image receiving unit is transmitted to the display device (200) through the first transmission path (10), and Based on the fact that the first transmission path (10) is lost, the first media data is converted into second media data, and The above second media data is stored on a local web server, and A source device (100) that transmits the second media data stored in the local web server to the display device (200) through a second transmission path (20) different from the first transmission path (10) based on a request from the display device (200).

2. In Paragraph 1, The above communication unit (140) is, A first communication module including a communication circuit and enabling direct communication between the source device (100) and the display device (200), and It includes a second communication module that includes a communication circuit and enables communication between the source device (100) and the display through a broadband internet network, and The first transmission path (10) includes a path for transmitting the first media data through the first communication module, and The source device (100) includes a second transmission path (20) for transmitting the second media data through the second communication module.

3. In Paragraph 2, The above first media data includes data based on a first communication protocol, which is a standard available in a broadcast network, and The source device (100) includes data based on a second communication protocol, which is a standard available on the internet network (broadband), for the second media data.

4. In any one of paragraphs 1 to 3, By having at least one processor (110) execute the one or more instructions individually or in combination, the source device (100) A source device (100) that generates the second media data by extracting one or more media files included in the first media data standardized according to the first communication protocol based on the loss of the first transmission path (10), and standardizing the one or more media files according to the second communication protocol different from the first communication protocol.

5. In any one of paragraphs 1 through 4, The above second media data includes one or more segments, and By having at least one processor (110) execute the one or more instructions individually or in combination, the source device (100) A request for a segment is obtained from the display device (200) using a URL that includes information on the playback start time of the segment and location information of the local web server, and A source device (100) that transmits, based on a request for the above segment, a segment corresponding to the playback start time information of the above segment among the one or more segments to the display device (200) through the second transmission path (20).

6. In Paragraph 5, The playback start time information of the above segment corresponds to the time when the first transmission path (10) is lost, and The segment corresponding to the playback start time information of the above segment is a source device (100) that corresponds to a segment generated by the source device (100) at the time when the first transmission path (10) is lost.

7. In any one of paragraphs 1 through 6, By having at least one processor (110) execute the one or more instructions individually or in combination, the source device (100) A source device (100) that transmits the first media data to the display device (200) through the first transmission path (10) based on the recovery of the first transmission path (10).

8. In the display device (200), Communication unit (220); At least one processor (210) including a processing circuit; and It includes a memory (250) comprising one or more storage media for storing one or more instructions, and By the above at least one processor (210) executing the above one or more instructions individually or collectively, the display device (200) First media data is obtained from a source device (100) through a first transmission path (10), and Based on the fact that the first transmission path (10) is lost, second media metadata for accessing second media data is obtained, and Based on the second media metadata above, the second media data is requested from the source device (100), and A display device (200) that receives the second media data from the source device (100) through a second transmission path (20) different from the first transmission path (10).

9. In Paragraph 8, The above communication unit (220) is, A first communication module including a communication circuit and enabling direct communication between the source device (100) and the display device (200), and It includes a second communication module that includes a communication circuit and enables communication between the source device (100) and the display through an internet network, and The first transmission path (10) includes a path for receiving the first media data through the first communication module, and The display device (200) wherein the second transmission path (20) includes a path for obtaining the second media data through the second communication module.

10. In Paragraph 8 or 9, The above first media data includes data based on a first communication protocol, which is a standard available in a broadcast network, and The above second media data includes data based on a second communication protocol that is a standard available on the internet network (broadband), a display device (200).

11. In any one of paragraphs 8 through 10, By the above at least one processor (210) executing the above one or more instructions individually or in combination, the display device (200) is, A display device (200) that generates the second media metadata including location information of the local web server of the source device (100) and playback start time information of the segment.

12. In Paragraph 11, The above second media data includes one or more segments, and By the above at least one processor (210) executing the above one or more instructions individually or in combination, the display device (200) is, A request for the segment is transmitted to the source device (100) using a URL that includes the playback start time information of the segment and the location information of the local web server, and A display device (200) that, in response to a request for the above segment, obtains a segment corresponding to the playback start time information of the above segment from the source device (100) through the second transmission path (20).

13. In Paragraph 12, The playback start time information of the above segment corresponds to the time when the first transmission path (10) is lost, and A display device (200) for a segment corresponding to the playback start time information of the above segment, which corresponds to a segment stored in the local web server at the time when the first transmission path (10) is lost.

14. In Paragraph 10, By the above at least one processor (210) executing the above one or more instructions individually or in combination, the display device (200) is, Upon receiving the first media data, a packet is parsed according to a first communication protocol corresponding to the first media data to extract a video stream, and A display device (200) that, upon receiving the second media data, extracts a video stream by parsing a packet that has been packetized according to a second communication protocol corresponding to the second media data.

15. A step of transmitting the first media data obtained through the video receiver to the display device (200) through the first transmission path (10); Based on the fact that the first transmission path (10) is lost, a step of converting the first media data into second media data; The step of storing the above second media data on a local web server; and A computer-readable non-transient recording medium having a program recorded thereon for performing a method of operation of a source device (100) on a computer, the method comprising the step of transmitting the second media data stored on the local web server to the display device (200) through a second transmission path (20) different from the first transmission path (10) in response to a request from the display device (200).

Images