Source device and operation method of source device

Abstract

A source device and an operation method therefor are provided. The source device comprises: a wireless communication module for supporting direct communication with a display device; at least one processor; and a memory including one or more storage media for storing one or more instructions, wherein the source device identifies a first quantity of media packets generated to transmit media data to the display device, identifies a second quantity of media packets indicating the number of media packets transmitted to the display device through the wireless communication module, and adjusts a bitrate of the media data on the basis of a difference between the first quantity and the second quantity.

Description

Source device, method of operation of the source device

[0001] The present disclosure relates to a source device and a method of operating the source device.

[0002] Recently, wireless connection services are being researched that wirelessly connect a source device that shares a screen and a sink device that receives the screen. The source device may be a device that generates or transmits content. The sink device may be a device connected to the source device that receives and outputs content from the source device.

[0003] A source device can receive and process various input signals to deliver content to a display device, which is an example of a sink device. Input signals may include video, audio, and supplementary information. The source device can connect external source devices to itself, process the input signals, and transmit them wirelessly to the display device. For example, the source device can receive digital broadcast signals (DTV) or analog broadcast signals (ATV) from a broadcast network, digitize them, and transmit them to the display device. Alternatively, for example, the source device can be wired to external source devices to receive and process input signals from them via wires and transmit them wirelessly to the display device. Accordingly, the source device can transmit high-resolution video and audio data to the display device without other wired connections, such as HDMI cables or USB. The display device can output the content received from the source device to the display.

[0004] A source device according to one embodiment of the present disclosure includes a wireless communication module that supports direct communication with a display device, at least one processor, and a memory comprising one or more storage media that stores one or more instructions.

[0005] According to one embodiment of the present disclosure, the source device identifies a first quantity of media packets generated to transmit media data to the display device by having the at least one processor execute the one or more instructions individually or collectively.

[0006] According to one embodiment of the present disclosure, by executing the one or more instructions individually or in combination, the source device identifies a second quantity of media packets representing the quantity of media packets transmitted to the display device through the wireless communication module.

[0007] According to one embodiment of the present disclosure, the source device adjusts the bitrate of the media data based on the difference between the first quantity and the second quantity by executing the one or more instructions individually or in combination by the at least one processor.

[0008] A method of operation of a source device according to one embodiment of the present disclosure includes: identifying a first quantity of media packets generated to transmit media data to the display device; identifying a second quantity of media packets representing a quantity of the generated media packets transmitted to the display device through a wireless communication module that supports direct communication with the display device; and adjusting the bitrate of the media data based on the difference between the first quantity and the second quantity.

[0009] In one embodiment of the present disclosure, a computer-readable recording medium is provided on which a program for performing a method of operating a source device on a computer is recorded.

[0010] FIG. 1 is a schematic diagram of a system including a source device and a display device that support a wireless connection service according to one embodiment of the present disclosure.

[0011] FIG. 2 is a diagram showing a network stack structure used in a source device according to one embodiment of the present disclosure.

[0012] FIG. 3 is a block diagram of a source device and a display device according to one embodiment of the present disclosure.

[0013] FIG. 4 is a flowchart illustrating a method of operation of a source device according to one embodiment of the present disclosure.

[0014] FIG. 5 is a flowchart illustrating a method of operation for a source device according to one embodiment of the present disclosure to transmit media data using a network stack structure.

[0015] FIG. 6 is a diagram illustrating the operation of transmitting and receiving media data through a network between a source device and a display device according to one embodiment of the present disclosure.

[0016] FIG. 7 is a diagram illustrating the operation in a network stack structure of a source device and a display device according to one embodiment of the present disclosure.

[0017] FIG. 8 is a diagram showing the form of a packet corresponding to a network stack structure according to one embodiment of the present disclosure.

[0018] FIG. 9 is a flowchart illustrating a method of transmitting and receiving media data through a network between a source device and a display device according to one embodiment of the present disclosure.

[0019] FIG. 10 is a system block diagram of a source device and a display device according to one embodiment of the present disclosure.

[0020] In the present disclosure, the expression “at least one of a, b, or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “a, b, and c all”, or variations thereof.

[0021] Embodiments of the present disclosure are described below in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0022] The terms used in this disclosure are described in their current, general form considering the functions mentioned herein; however, they may refer to various other terms depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Accordingly, the terms used in this disclosure should not be interpreted solely by their names, but should be interpreted based on the meaning of the terms and the overall content of this disclosure.

[0023] Furthermore, the terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit this disclosure.

[0024] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components in between.

[0025] The terms “above” and similar designations used in this specification, particularly in the claims, may indicate both singular and plural forms. Furthermore, unless there is a description explicitly specifying the order of the steps describing the method according to this disclosure, the described steps may be performed in a suitable order. This disclosure is not limited by the order in which the described steps are described.

[0026] Phrases such as "in some embodiments" or "in one embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment.

[0027] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function. Additionally, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as algorithms executed on one or more processors. Furthermore, the present disclosure may employ prior art for electronic configuration, signal processing, and / or data processing, etc. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations.

[0028] Furthermore, the connecting lines or connecting members between the components depicted in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In the actual device, connections between components may be represented by various alternative or added functional connections, physical connections, or circuit connections.

[0029] Additionally, terms such as "...part," "module," etc., as described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0030] In the present disclosure, the “processor” may include various processing circuits and / or a plurality of processors. For example, the term “processor” as used herein, including in the claims, may include at least one processor and various processing circuits. In the at least one processor, one or more processors may be configured to perform the various functions described herein in a distributed manner, individually and / or collectively. As used herein, the “processor,” “at least one processor,” and “one or more processors” may be configured to perform various functions. However, these terms cover, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor can perform all functions. Additionally, the at least one processor may include a combination of processors performing various functions of the disclosed functions in a distributed manner. The at least one processor may execute program instructions to achieve or perform various functions.

[0031] In the present disclosure, the term “user” refers to a person using a display device and may include a consumer, evaluator, viewer, administrator, or installer. Additionally, in the specification, “manufacturer” or “provider” may refer to a manufacturer that manufactures a display device and / or components included in the display device.

[0032] In the present disclosure, 'image' may include a still image, a graphic, a picture, a frame, a video composed of a plurality of consecutive still images, or a video.

[0033] In the present disclosure, 'communication path' may represent a physical or logical path through which data travels from a transmitter (TX) to a receiver (RX). For example, a communication path may represent a routing path or a physical network connection on a network. For example, in a Wi-Fi Direct connection, a communication path may represent a wireless communication path between two devices. Or, for example, in a connection between devices relayed by an Access Point (AP), a communication path may represent a physical connection path or a wireless communication path between two devices.

[0034] In the present disclosure, Wi-Fi Direct may represent a method in which two devices are directly connected without passing through an Access Point (AP) using a Peer-to-Peer (P2P) communication method (or a Device-to-Device (D2D) communication method). For example, Wi-Fi Direct can transmit and receive data based on TCP / IP communication.

[0035] In the present disclosure, a device for providing wireless connection services transmits data generated by a user program for transmitting and receiving packets by attaching a communication protocol header. Examples of common communication protocol headers include IP (Internet Protocol), TCP (Transmission Control Protocol) / UDP (User Datagram Protocol), and Ethernet headers.

[0036] In the present disclosure, an application program, which is an upper layer in the Open Systems Interconnection (OSI) Layer 7, generates data to be transmitted and delivers it to a lower layer, and this process operates differently depending on the operating system (OS).

[0037] In the present disclosure, an operating system may refer to system software that manages the hardware and software resources of a computing device and provides common services for programs (processes). The operating system acts as an interface between the user and the hardware and enables various application programs to use the hardware efficiently.

[0038] In the present disclosure, the operating system may operate in dual modes including user mode and kernel mode. User mode is an execution mode in which operating system services cannot be provided and code in the kernel area cannot be executed. Application programs are executed in user mode. Application programs executed in user mode cannot access hardware resources. Kernel mode is an execution mode in which operating system services can be provided and code in the kernel area can be executed. When a processor executes instructions in kernel mode, it can access hardware resources.

[0039] In the present disclosure, packetization may include the operation of dividing data into small units of packets and adding a header containing information about the data to create a packet form. In the present disclosure, de-packetization may include the operation of analyzing (or parsing) information included in the header of a packet to extract data contained within the packet.

[0040] In the present disclosure, multiplexing or muxing may include an operation of converting independent data, such as video, audio, and subtitles, into a single transmission stream. In the present disclosure, demultiplexing or demuxing may include an operation of separating data, such as video, audio, and subtitles, included in a single transmission stream.

[0041] In the present disclosure, streaming data refers to a continuous stream of data generated from various sources. A source device may divide the streaming data into small media file chunks, packets, or segments and transmit them to a sink device.

[0042] In the present disclosure, the content may be received by the device from a content provider, such as a broadcast signal, a streaming service, a Blu-ray player, or a game console. The content may include one or more of broadcast content received directly from a broadcasting station as an RF signal, broadcast content received through an external source device, and content received from a content provider server via the Internet.

[0043] The present disclosure will be described in detail below with reference to the attached drawings.

[0044] FIG. 1 is a schematic diagram of a system including a source device and a display device that support a wireless connection service according to one embodiment of the present disclosure.

[0045] A system according to one embodiment of the present disclosure may include a source device (100) and a display device (200). The system may provide a wireless connection service between the source device (100) and the display device (200). The display device (200) may correspond to a sink device. In the present disclosure, the source device (100) may be referred to as a transmitter (TX) or a transmitting device. The display device (200) may be referred to as a receiver (RX) or a receiving device.

[0046] A source device (100) according to one embodiment of the present disclosure may include a set-top box, a Blu-ray Disc player, a DVD (Digital Versatile Disc) player, a game device, a digital camera, a camcorder, a streaming device, a home theater, etc. Alternatively, the source device (100) may include various electronic devices such as a smartphone, a tablet PC, a mobile terminal, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigation device, an MP3 player, a wearable device, etc.

[0047] A display device (200) according to one embodiment of the present disclosure may be a TV, but this is merely one embodiment, and may be implemented as an electronic device capable of receiving a broadcast signal and displaying an image based on the broadcast signal. For example, the display device (200) may be implemented as various electronic devices such as a mobile phone, tablet PC, digital camera, camcorder, laptop computer, tablet PC, desktop, e-book reader, digital broadcast terminal, PDA (Personal Digital Assistants), PMP (Portable Multimedia Player), navigation, MP3 player, wearable device, etc. In particular, the embodiments may be easily implemented in a display device with a large display, such as a TV, but are not limited thereto.

[0048] Additionally, the display device (200) may be a fixed or mobile device and may be a digital broadcast receiver capable of receiving digital broadcasts. Furthermore, the display device (200) may be implemented not only as a flat display device, but also as a curved display device having a screen with curvature or a flexible display device with adjustable curvature. The output resolution of the display device (200) may include, for example, HD (High Definition), Full HD, Ultra HD, or a resolution sharper than Ultra HD.

[0049] A source device (100) according to one embodiment of the present disclosure can receive various content from an external device. For example, the content may refer to multimedia content and may include, but is not limited to, images, videos, audio, text, games, applications, broadcasts, etc. For example, the source device (100) can receive broadcast content from a broadcast network through a tuner unit. The broadcast content may be various forms of media composed of a collection of media components provided to the user (e.g., video, audio, subtitles, service announcements, etc.). For example, the source device (100) can receive various content from an external device through an input / output unit. For example, the source device (100) can receive various applications, for example, OTT content provided by an OTT (Over-The-Top) service provider, through a communication unit.

[0050] A source device (100) according to one embodiment of the present disclosure may be connected to a display device (200) via a wireless communication network. The source device (100) may transmit received content to the display device (200). The display device (200) may receive content from the source device (100) and output the received content to a display. For example, the source device (100) may be directly connected to the display device (200) via a direct device-to-device connection method.

[0051] If data is not properly transmitted from the source device (100) to the display device (200), a buffer under-run problem may occur in the display device (200). A buffer under-run problem refers to a phenomenon where video or audio is interrupted because the receiver (e.g., display device (200)) does not receive enough data to play from the transmitter (e.g., source device (100)). A buffer refers to a space that temporarily stores received data. If the data stored in the receiver's buffer is exhausted, video output or audio output may be interrupted. For example, if the data stored in the buffer of the display device (200) is exhausted, the display device (200) may output a black screen, duplicate a previously output screen, or not output audio. In other words, if data processing (e.g., rendering) proceeds in a buffer under-run state where the data in the buffer of the display device (200) is exhausted, a problem of discontinuity in video playback of the display device (200) occurs.

[0052] FIG. 1 illustrates a case where a display device (200) receives data 1, data 2, data 3, and data 4 from a source device (100). Each data may be in the form of a packet, but is not limited thereto. The display device (200) may receive data 1, data 2, and data 3 from the source device (100) at a constant rate and store them in a buffer. The display device (200) may render the data stored in the buffer and output video and audio corresponding to data 1, data 2, and data 3. However, if the display device (200) fails to smoothly receive data 4 from the source device (100), a buffer underrun caused by data 4 may occur in the display device (200). Since the display device (200) processes data 4 while the buffer is depleted of data 4, video output may be interrupted.

[0053] To prevent buffer underrun, the source device (100) transmits the data at a lower bitrate, thereby enabling the display device (200) to play smooth content with a smaller amount of data. The method of the transmitter adjusting and transmitting the bitrate of the data according to the buffering state of the receiver may be referred to as Adaptive Bitrate Streaming (ABR streaming).

[0054] In the present disclosure, bitrate refers to the speed at which data is transmitted or processed, and may mean the amount of data transmitted per second. Bitrate is expressed in bps (bit per second). Lowering the bitrate means reducing the amount of data transmitted per second (bitrate). For example, an example is given in which an encoder compresses video consisting of 60 frames per second by setting the bitrate to 60 Mbps. If the bitrate of a video set to 60 Mbps is reduced by 30%, the bitrate may be lowered to 42 Mbps. In this case, the encoder may compress the bitrate per frame of the video consisting of 60 frames per second to 0.7 Mbps instead of 1 Mbps.

[0055] The source device (100) can identify resolution information or data compression rate information of the image corresponding to the adjusted bit rate in order to adjust the bit rate of the image, and can adjust the resolution or data compression rate of the image according to the identified resolution information or data compression rate information. For example, the source device (100) can identify resolution information of the image corresponding to the reduced bit rate in order to reduce the bit rate of the image. The source device (100) can lower the resolution of the image using the identified resolution information. For example, the source device (100) can identify data compression rate information corresponding to the reduced bit rate in order to reduce the bit rate of the image. The source device (100) can increase the data compression rate of the image using the identified data compression rate information.

[0056] The cause of the lack of data in the buffer of the display device (200) may include an unstable network condition between the source device (100) and the display device (200), a case where packets are not transmitted and accumulate in each layer of the network stack of the source device (100), or a CPU overload condition. For example, if the network condition between the source device (100) and the display device (200) is unstable, the number of packets (first quantity) prepared (or generated) by the source device (100) to transmit data may be 100, but the actual number (second quantity) transmitted from the source device (100) to the display device (200) may be 80. In this case, the difference of 20 between the first quantity and the second quantity may not be transmitted to the display device (200) and may accumulate in each layer of the network stack of the source device (100). The buffer of the display device (200) is short of data equivalent to 20 packets. There may be remaining space equivalent to 20 packets in the buffer of the display device (200). Alternatively, similarly, the first quantity and the second quantity may differ in the case of CPU overload conditions of the source device (100) and the display device (200), respectively.

[0057] The source device (100) can identify the difference between the first quantity and the second quantity to predict the buffering state of the receiver, the display device (200). The source device (100) can adjust the bitrate (streaming quality) of the data in real time according to the buffering state of the display device (200). If there is insufficient data in the buffer of the display device (200) (i.e., a buffer underrun is imminent), the source device (100) can lower the bitrate. If there is sufficient data in the buffer of the display device (200), the source device (100) can increase or maintain the bitrate. When switching to a lower bitrate, the amount of data transmitted is reduced, allowing less network bandwidth to be used. Therefore, by adjusting the bitrate, the continuity of video playback can be maintained without the buffer underrun problem of the display device (200).

[0058] In a direct connection method between devices according to one embodiment of the present disclosure, the source device (100) can identify a first quantity and a second quantity to predict whether the packet has been properly delivered to the display device (200). That is, the source device (100) can determine the remaining size (or remaining space) of the buffer of the display device (200) in real time on its own, even without receiving separate feedback from the display device (200). By predicting the buffering state of the display device (200) on its own, the source device (100) can adjust the bit rate. The source device (100) can prevent buffer underruns of the display device (200) and provide continuous video to the user.

[0059] A source device (100) according to one embodiment of the present disclosure can identify the amount of packets transmitted to each layer within the network stack of the source device (100) in order to identify a first quantity and a second quantity of packets. The network stack structure and the first quantity and second quantity of packets are further explained in FIG. 2.

[0060] FIG. 2 is a diagram showing a network stack structure used in a source device according to one embodiment of the present disclosure.

[0061] In the present disclosure, a 'network stack structure' used in network communication may include one or more layers. A 'layer' represents a logical structure for processing data transmission and reception processes by dividing them into stages, and each layer may include one or more modules that operate according to specific protocols and rules.

[0062] For example, a network stack structure may include an application layer (2100), a transport layer (2200), an internet layer (2300), and a network interface layer (or network access layer) (2400). The network interface layer (2400) may include an L2 driver (2420) and a network chip (2440). The application layer (2100) may be the highest layer, and the network interface layer (2400) may be the lowest layer. When a source device (100) according to one embodiment of the present disclosure uses a TCP / IP network stack, the transport layer (2200) may include a TCP layer, and the internet layer (2300) may include an IP layer.

[0063] Meanwhile, in the present disclosure, for convenience of explanation, the transport layer (2200), the internet layer (2300), and the network interface layer (2400) are referred to as the 'network stack', and the network stack may be described as a lower layer for the application layer (2100).

[0064] The application layer (2100) is a layer that generates data to be actually transmitted and may include a user application program. In the application layer (2100), the source device (100) may generate a packet by adding an application header to the data to be transmitted.

[0065] At the application layer (2100), the source device (100) can use a socket (2150) to transmit packets to a receiver via network communication through a network stack (e.g., 2200, 2300, 2400). The socket (2150) can serve as a channel (or interface) for transmitting and receiving data between devices. At the application layer (2100), the source device (100) can call a socket system to transmit packets to the lower layer, the transport layer (2200). When the socket system is called, user mode can be switched to kernel mode. Calling the socket system may correspond to creating (or executing) a socket (2150). The source device (100) can transmit packets through the socket (2150) to the receiver via the network stack (e.g., 2200, 2300, 2400) while kernel mode is running.

[0066] The transport layer (2200) may add a TCP header to a packet generated by the application layer (2100) to perform TCP communication and transmit it to the internet layer (2300). The TCP header may include the port address of the transmitter and the port address of the receiver. The payload may include data generated by the application layer (2100). The transport layer (2200) may run in kernel mode as part of an operating system. However, it is not limited thereto, and the transport layer (2200) may include a UDP layer for performing UDP communication.

[0067] The Internet layer (2300) may add an IP header to the packet generated by the transport layer (2200) to perform IP communication and pass it to the network interface layer (2400). The IP header may include the IP address of the transmitter and the IP address of the receiver. The Internet layer (2300) may run in kernel mode as part of an operating system.

[0068] The network interface layer (2400) can perform software transmission processing and physical transmission processing for packets generated at an upper layer (e.g., the internet layer (2300)). The network interface layer (2400) may include an L2 driver (2420) and a network chip (2440). The L2 driver (2420) may include software that provides an interface between hardware and an operating system (OS). For example, the L2 driver (2420) can connect the network chip (2440), which is hardware, with the TCP / IP layer (e.g., 2200, 2300), which is part of the operating system. The L2 driver (2420) may run in kernel mode as part of the operating system. The L2 driver (2420) can access the network chip (2440) while kernel mode is running. The L2 driver (2420) can add the physical address of the receiver (e.g., MAC (Media Access Control) address) to the header of the packet and transmit it to the lowest layer network chip (2440). The network chip (2440) may be hardware for connecting to the outside through a network. The network chip (2440) may transmit the packet transmitted from the L2 driver (2420) to the outside. The network chip (2440) may be referred to as a network interface card (NIC) or a wired / wireless communication module. The L2 driver (2420) may correspond to the data link layer or the second layer (L2) in the OSI 7-layer model. The network chip (2440) may correspond to the physical layer in the OSI 7-layer model.

[0069] For example, the L2 driver (2420) may include a Wi-Fi driver, and the network chip (2440) may include a Wi-Fi chip. In this case, the L2 driver (2420) may convert data received from the TCP / IP layer (e.g., 2200, 2300) into a form that the Wi-Fi chip can understand, or transmit data received from the Wi-Fi chip to an upper layer. However, not limited thereto, the L2 driver (2420) may include an Ethernet driver, and the network chip (2440) may include an Ethernet chip.

[0070] In one embodiment of the present disclosure, when a source device (100) is connected to a display device (200) via Wi-Fi Direct, a TCP header, an IP header, and a MAC header are added to data generated at the application layer (2100), and the data can be directly transmitted to the display device (200) in the form of packets via Wi-Fi Direct.

[0071] In the case of the Wi-Fi Direct method, unlike Wi-Fi methods that are connected indirectly through an access point (AP) or multiple routers, devices can be connected directly to each other. Accordingly, when a single packet is transmitted from the network chip (2440) of the network interface layer (2400) to the display device (200), the L2 driver (2420) can send a single packet down to the network chip (2440), and the internet layer (2300) can send a single packet down to the L2 driver (2420). In this case, the display device (200) can receive a single packet and store it in a buffer. That is, in the Wi-Fi Direct method, the number of packets (or transmission speed) transmitted to the display device (200) through the network chip (2440) of the source device (100) may be equal to at least one of the number of packets (or reception speed) received by the display device (200) from the source device (100), the number of packets transmitted from the internet layer (2300) to the L2 driver (2420), or the number of packets transmitted from the L2 driver (2420) to the network chip (2440).

[0072] Accordingly, the source device (100) can identify the quantity of packets received or transmitted by the L2 driver (2420) of the network interface layer (2400) in correspondence with the quantity of packets transmitted from the source device (100) to the display device (200) (second quantity). The source device (100) can predict (or estimate) the second quantity of packets directly transmitted from the network chip (2440) to the display device (200) by using the quantity of packets obtained through the L2 driver (2420).

[0073] Meanwhile, the quantity of packets generated based on media data (first quantity) may be the same as the quantity of packets transmitted from the application layer (2100) to the transmission layer (2200) through the socket (2150). Accordingly, the source device (100) can identify the quantity of packets transmitted from the application layer (2100) to the transmission layer (2200) through the socket (2150) in correspondence with the quantity of packets generated based on media data (first quantity).

[0074] The source device (100) can adjust the bit rate of the media data based on the difference between the first quantity and the second quantity.

[0075] FIG. 3 is a block diagram of a source device and a display device according to one embodiment of the present disclosure.

[0076] Referring to FIG. 3, a system according to one embodiment of the present disclosure may include a source device (100), a display device (200), and a network (310) connecting the source device (100) and the display device (200).

[0077] The network (310) is a short-range communication network that enables communication between a source device (100) and a display device (200) located in close proximity, and may include, for example, Wi-Fi Direct (WFD). For example, the network (310) may be Wi-Fi 7, which enables wireless communication with external devices through different frequency bands (e.g., 2.4 GHz band, 5 GHz band, and 6 GHz band) and channels via multi-link operation (MLO) technology, but is not limited thereto. Examples of the network (310) are not limited thereto and may use, for example, WLAN (or Wi-Fi), Bluetooth, BLE (Bluetooth Low Energy), soft AP, NFC, etc. Soft AP (Soft Access Point) is an abbreviation for software enabled access point, which refers to software that enables a computer, rather than a router, to function as a wireless access point.

[0078] First, the source device (100) will be described.

[0079] A source device (100) according to one embodiment of the present disclosure may include a processor (110), an image receiving unit (120), an image processing unit (130), a communication unit (140), and a memory (150). However, not all of the illustrated components are essential components. The source device (100) may be implemented with more components than those illustrated, or with fewer components.

[0080] The video receiving unit (120) can receive video from an external device. The video receiving unit (120) may include at least one of a tuner unit, a communication unit, and an input / output unit. The tuner unit can select broadcast content, etc., received via wired or wireless connection under the control of the processor (110) by tuning only the frequency of the channel to be received by the source device (100) among many radio wave components through amplification, mixing, resonance, etc. The content received through the tuner unit can be demuxed by the source device (100) and separated into video, audio, and / or additional information.

[0081] The input / output unit can receive video (e.g., dynamic image signal or still image signal), audio (e.g., voice signal or music signal), and additional information from an external device, etc., under the control of the processor (110). The input / output unit may include at least one of an HDMI port (High-Definition Multimedia Interface port), a component jack, a PC port, and a USB port. In addition to these, the input / output unit may further include a DisplayPort (DP), Thunderbolt, and MHL (Mobile High-Definition Link). The input / output unit may further include ports for separate output of video and audio.

[0082] The communication unit can connect the source device (100) to a peripheral device, external device, server, mobile terminal, etc. under the control of the processor (110). The communication unit can receive video from a content provider server, such as a streaming server.

[0083] In one embodiment of the present disclosure, the image receiving unit (120) may receive an image signal including media data from an external source device. The media data received through the image receiving unit (120) may be packetized into packets that are smaller pieces than the media data and transmitted to a display device (200) through a network (310).

[0084] The image processing unit (130) can perform processing on audio data and video data included in the image signal received from the image receiving unit (120). The image processing unit (130) may include an audio processing unit for performing various processing such as demuxing, encoding, and packetizing on the audio data. The image processing unit (130) may include a video processing unit for performing various processing such as demuxing, encoding, and packetizing on the video data. The video processing unit may include a media codec for processing video data.

[0085] The communication unit (140) can connect the source device (100) to peripheral devices, external devices, servers, mobile terminals, etc. under the control of the processor (110). The communication unit (140) may include various communication circuits included in at least one communication module. The communication unit (140) may include a short-range communication module, a wireless internet module, wired Ethernet, etc., corresponding to the performance and structure of the source device (100).

[0086] A short-range communication module is a module for short-range communication and may include, but is not limited to, a WLAN module (Wi-Fi module), a Bluetooth module, a Zigbee module, an infrared (IrDA, infrared Data Association) module, a WFD (Wi-Fi Direct) module, etc. A WLAN module can transmit and receive Wi-Fi signals with a surrounding device according to Wi-Fi communication standards. A Bluetooth module can receive Bluetooth signals transmitted from a surrounding device according to Bluetooth communication standards. A WFD (Wi-Fi Direct) module can support P2P (Peer-to-Peer) communication that enables direct connection between two devices without an access point. Wi-Fi may use frequencies in the 2.4 GHz band, 5 GHz band, or 6 GHz band. A short-range communication module may be used to communicate with a display device (200).

[0087] A wireless internet module is a module for wireless internet access and can be built into or external to a device. The wireless internet module may include a WLAN module, a Wibro (Wireless broadband) module, etc. The wireless internet module can be used for a source device (100) to communicate with a server device. A WLAN module can be used as a wireless internet module when it serves to connect to the internet through an access point.

[0088] A source device (100) according to one embodiment of the present disclosure may include a Wi-Fi Direct (WFD) module. The Wi-Fi Direct (WFD) module is connected to a network (310) and can directly transmit packets corresponding to media data to a display device (200). The WFD module may include a network chip implemented in hardware (e.g., 2440 of FIG. 2). The network chip may be referred to as a "wireless communication module."

[0089] The processor (110) is electrically connected to the components included in the source device (100) and can perform operations or data processing regarding the control and / or communication of the components included in the source device (100). In one embodiment of the present disclosure, the processor (110) can load a request, command, or data received from at least one of the other components into memory for processing and store the processing result data in memory. According to one or more embodiments, the processor (110) may include at least one of a general-purpose processor such as a CPU (central processing unit), AP (application processor), DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU (graphic processing unit) or VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU (neural processing unit).

[0090] The processor (110) can process input data or control other configurations to process it according to data, operation rules, algorithms, methods, or models stored in memory (150). The processor (110) can perform operations of predefined operation rules, algorithms, methods, or models stored in memory (150) using the input data.

[0091] Memory (150) is electrically connected to the processor (110) and can store one or more modules, algorithms, operation rules, models, programs, instructions, or data related to the operation of components included in the source device (100). For example, memory (150) can store one or more modules, algorithms, operation rules, models, programs, instructions, or data for processing and controlling the processor (110). Memory (150) may include, but is not limited to, at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0092] The memory (150) may include an application program (151) and a kernel (152). At least a portion of the kernel (152) may be referred to as an operating system (OS).

[0093] An application program (151) according to one embodiment may be executed in user mode. The application program (151) may enter kernel mode through a system call provided by the kernel (152) and access hardware resources. For example, a socket system call may be included as an example of a system call.

[0094] A kernel (152) according to one embodiment may control or manage hardware resources (e.g., processor (110), communication unit (140), or memory (150), etc.) used to execute operations or functions implemented in other programs (e.g., application program (151)). The kernel (152) may include software that is loaded into memory (e.g., RAM) and executed when the operating system boots. The kernel (152) may be executed in kernel mode.

[0095] A kernel (152) according to one embodiment may include a transport layer (e.g., 2200 in FIG. 2), an internet layer (e.g., 2300 in FIG. 2), and an L2 driver (e.g., 2420 in FIG. 2) included in a network stack. The transport layer, the internet layer, and the L2 driver may operate in kernel mode as part of an operating system. The kernel (152) may support an application program (151) to communicate with a network chip (e.g., 2440 in FIG. 2) included in a communication unit (140).

[0096] A processor (110) according to one embodiment of the present disclosure can obtain a media packet corresponding to media data through an image receiving unit (120) and an image processing unit (130) by executing one or more instructions stored in a memory (150).

[0097] A processor (110) according to one embodiment of the present disclosure can transmit a media packet to a display device (200) through a communication unit (140) by executing one or more instructions stored in a memory (150).

[0098] A processor (110) according to one embodiment of the present disclosure can identify a first quantity of media packets generated to transmit media data to a display device (200) by executing one or more instructions stored in memory (150).

[0099] A processor (110) according to one embodiment of the present disclosure can identify a second quantity of media packets received or transmitted by a network interface layer (e.g., 2400 of FIG. 2) by executing one or more instructions stored in memory (150) to correspond to a quantity (e.g., number) of media packets transmitted from a source device (100) to a display device (200) through a network chip.

[0100] A processor (110) according to one embodiment of the present disclosure can adjust the bit rate of media data based on the difference between a first quantity and a second quantity by executing one or more instructions stored in memory (150).

[0101] Next, the display device (200) is described.

[0102] A display device (200) according to one embodiment of the present disclosure may include a processor (210), a communication unit (220), an image processing unit (230), a display (240), and a memory (250). However, not all of the illustrated components are essential components. The display device (200) may be implemented with more components than those illustrated, or with fewer components.

[0103] The communication unit (220) can connect the display device (200) to peripheral devices, external devices, servers, mobile terminals, etc. under the control of the processor (210). The communication unit (220) may include various communication circuits included in at least one communication module. The communication unit (220) may include a short-range communication module, a wireless internet module, wired Ethernet, etc., corresponding to the performance and structure of the display device (200).

[0104] A short-range communication module is a module for short-range communication and may include, but is not limited to, a WLAN module (or Wi-Fi module), a Bluetooth module, a Zigbee module, an infrared (IrDA, infrared Data Association) module, a WFD (Wi-Fi Direct) module, etc. A WLAN module can transmit and receive Wi-Fi signals with nearby devices according to Wi-Fi communication standards. A Bluetooth module can receive Bluetooth signals transmitted from nearby devices according to Bluetooth communication standards. A WFD (Wi-Fi Direct) module can support P2P (Peer-to-Peer) communication that enables direct connection between two devices without an AP. Wi-Fi may use frequencies in the 2.4GHz, 5GHz, or 6GHz bands.

[0105] A wireless internet module is a module for wireless internet access and can be built into or external to a device. The wireless internet module may include a WLAN module, a Wibro (Wireless broadband) module, etc. The wireless internet module can be used for the display device (200) to communicate with a server device. A WLAN module can be used as a wireless internet module when it serves to connect to the internet through an AP.

[0106] A display device (200) according to one embodiment of the present disclosure may include a WFD module. The WFD module is connected to a network (310) and can receive packets corresponding to media data directly from a source device (100). The WFD module may include a network chip (e.g., 2440) implemented in hardware.

[0107] The image processing unit (230) processes video data to be displayed by the display (240) and can perform various image processing operations such as decoding, rendering, scaling, noise filtering, frame rate conversion, and resolution conversion on the video data. For example, the image processing unit (230) may include various image processing circuits. For example, the image processing unit (230) may include a media codec for processing video content. In addition, the image processing unit (230) processes audio data to be output by the microphone and can perform operations such as decoding on the audio data.

[0108] The display (240) can receive content from a broadcasting station, receive content from an external device such as an external server or external storage medium, or output content provided by various apps, such as an OTT service provider or a content provider. The display (240) can receive content from the source device (100) and display video-processed content.

[0109] The processor (210) is electrically connected to the components included in the display device (200) and can perform operations or data processing regarding the control and / or communication of the components included in the display device (200). In one embodiment of the present disclosure, the processor (210) can load a request, command, or data received from at least one of the other components into memory for processing and store the processing result data in memory. According to one or more embodiments, the processor (210) may include at least one of a general-purpose processor such as a CPU (central processing unit), AP (application processor), DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU (graphic processing unit) or VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU (neural processing unit).

[0110] The processor (210) can process input data or control other configurations to process it according to data, operation rules, algorithms, methods, or models stored in memory (250). The processor (210) can perform operations of predefined operation rules, algorithms, methods, or models stored in memory (250) using the input data.

[0111] The memory (250) is electrically connected to the processor (210) and can store one or more modules, algorithms, operation rules, models, programs, instructions, or data related to the operation of components included in the display device (200). For example, the memory (250) can store one or more modules, algorithms, operation rules, models, programs, instructions, or data for processing and controlling the processor (210). The memory (250) may include, but is not limited to, at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0112] The memory (250) may include an application program, a kernel, and a network driver, similar to the memory (150) of the source device (100) described above. Since this is identical to the memory (150), a description is omitted.

[0113] A processor (210) according to one embodiment of the present disclosure can receive a media packet from a source device (100) by executing one or more instructions stored in memory (250).

[0114] FIG. 4 is a flowchart illustrating a method of operation of a source device according to an embodiment of the present disclosure. The method of operation of a source device (100) according to an embodiment of the present disclosure may be performed by a processor (110 in FIG. 3) of the source device (100). FIG. 4 is described with reference to the application layer (2100), transport layer (2200), internet layer (2300), and network interface layer (2400) of FIG. 2 or FIG. 7.

[0115] Referring to FIG. 4, in operation 410, the source device (100) can acquire a media packet based on media data at the application layer (2100). The application layer (2100) may be a layer for executing a user application program.

[0116] The source device (100) can receive a video signal containing media data through a video receiving unit (120). The source device (100) can perform processing on the video signal through a video processing unit (130). The source device (100) can generate one or more media packets by dividing the media data into smaller units, such as packets. The media data may include video data and audio data corresponding to streaming data. The source device (100) can separate data such as video, audio, and additional information included in a single transmission stream. The source device (100) can encode each of the separated data such as video, audio, and additional information. The source device (100) can obtain video packets, audio packets, and additional information packets corresponding to each of the encoded data such as video, audio, and additional information. This is described in detail in FIG. 6.

[0117] In operation 420, the source device (100) can identify a first quantity of media packets obtained at the application layer (2100) according to operation 410. The first quantity of media packets can correspond to a quantity (e.g., number) of media packets transmitted from the application layer (2100) to the lower layer, the transport layer (2200), through a socket.

[0118] The source device (100) can transmit the acquired media packet to the network stack (e.g., 2200, 2300, 2400) through a socket. The socket can serve as a channel (or interface) for transmitting and receiving data between devices.

[0119] The source device (100) can call the socket system at the application layer (2100) to transmit packets to the lower layer, the transport layer (2200). When the socket system is called, the user mode can be switched to kernel mode. While kernel mode is running, the source device (100) can transmit packets to the transport layer (2200) through the socket. For example, the source device (100) can create (or run) a TCP socket and transmit media packets through the socket to the display device (200) via the network stack (e.g., 2200, 2300, 2400). The source device (100) can add headers corresponding to each layer to the media packets as they pass through the network stack (e.g., 2200, 2300, 2400). For example, the source device (100) may add a TCP header at the transport layer (2200), an IP header at the internet layer (2300), and a MAC header at the network interface layer (2400). The source device (100) may transmit a media packet containing the TCP header, the IP header, and the MAC header to a display device (200). The specific process of transmitting the media packet is further explained in operations 510 to 550 of FIG. 5.

[0120] The source device (100) can identify the quantity (first quantity) of media packets transmitted through a socket at the application layer (2100). The quantity (first quantity) of media packets transmitted through a socket may correspond to the quantity (e.g., number) of media packets transmitted from the application layer (2100) to the lower layer, the transport layer (2200). The source device (100) can track the amount of data transmitted through a socket at the application layer (2100). For example, the source device (100) may record and accumulate the amount (e.g., number of bytes) or number transmitted each time a media packet is transmitted through a socket. However, it is not limited thereto.

[0121] The quantity (first quantity) of media packets transmitted from the application layer (2100) of the source device (100) to the transmission layer (2200) can be identified in real time.

[0122] In the application layer (2100) of the source device (100), the quantity of video packets transmitted through the video socket (first quantity) and the quantity of audio packets transmitted through the audio socket (first quantity) can be identified, respectively, and this is explained in FIG. 5.

[0123] In operation 430, the source device (100) can identify a second quantity of media packets representing a quantity (e.g., number) transmitted to the display device (200) via the network chip (2440). The network chip (2440) may be referred to as a wireless communication module.

[0124] In one embodiment of the present disclosure, the network interface layer (2400) may include an L2 driver (2420) and a network chip (2440). A second quantity of media packets may correspond to at least one of the quantity (e.g., number) of media packets transmitted from the internet layer (2300) to the L2 driver (2420), or the quantity (e.g., number) of media packets transmitted from the L2 driver (2420) to the network chip (2440).

[0125] In one embodiment of the present disclosure, a source device (100) and a display device (200) may be connected via Wi-Fi Direct. As described in FIG. 2, unlike Wi-Fi methods that are indirectly connected through an access point (AP) or multiple routers, Wi-Fi Direct allows for direct connection between devices. Accordingly, when a packet is transmitted from the network chip (2440) of the network interface layer (2400) to the display device (200), the L2 driver (2420) can send a packet down to the network chip (2440), and the internet layer (2300) can send a packet down to the L2 driver (2420). In this case, the display device (200) can receive a packet and store it in a buffer. That is, in the Wi-Fi Direct method, the number of packets transmitted to the display device (200) through the network chip (2440) of the source device (100) may be equal to at least one of the number of packets received by the display device (200) from the source device (100), the number of packets transmitted from the internet layer (2300) to the L2 driver (2420), or the number of packets transmitted from the L2 driver (2420) to the network chip (2440).

[0126] The source device (100) can measure the quantity of media packets transmitted from the Internet layer (2300) to the L2 driver (2420) in correspondence with a second quantity (e.g., number) of media packets. The L2 driver (2420) may include a metering module for measuring the amount of packets received or transmitted within the L2 driver (2420) (see 740 in FIG. 7). For example, the source device (100) can measure the quantity (e.g., number) of packets transmitted from the Internet layer (2300) to the L2 driver (2420) through the metering module. However, it is not limited thereto, and the source device (100) may also measure the quantity (e.g., number) of packets transmitted from the L2 driver (2420) to the network chip (2440) through the metering module. The L2 driver (2420) can transmit a second quantity of media packets obtained through the metering module to the application layer (2100).

[0127] The source device (100) can predict (or estimate) a second quantity of packets that are directly transmitted from the network chip (2440) to the display device (200) by using the quantity (e.g., number) of packets obtained through the L2 driver (2420).

[0128] Accordingly, the source device (100) can predict the buffer state (or buffering) of the display device (200) by predicting a second quantity of media packets transmitted from the source device (100) to the display device (200).

[0129] In operation 440, the source device (100) can adjust the bit rate of the media data based on the difference between the first quantity and the second quantity.

[0130] The source device (100) can predict the buffering state of the display device (200) based on the difference between the quantity of media packets transmitted by the source device (100) itself through the socket (first quantity) and the quantity of packets transmitted to the display device (200) (second quantity).

[0131] For example, a first quantity corresponding to the quantity (e.g., number) of packets prepared by the source device (100) to transmit data, and a second quantity corresponding to the quantity (e.g., number) of packets transmitted to the actual display device (200) may be different from each other. The difference between the first quantity and the second quantity may occur due to unstable network conditions between the source device (100) and the display device (200) or CPU overload conditions.

[0132] For example, in a direct connection method between devices, if the difference between the first quantity and the second quantity is large, the number of packets actually transmitted to the receiver is less than the number of packets prepared for transmission from the transmitter, so there may be a state where there is insufficient data in the receiver's buffer. Accordingly, the source device (100) can predict the number of packets that are not delivered to the display device (200) (e.g., 20) based on the difference between the first quantity (e.g., 100) and the second quantity (e.g., 80). That is, the quantity corresponding to the remaining size of the buffer of the display device (200) may be 20.

[0133] The source device (100) can decrease, increase, or maintain the bitrate of the media data based on the difference between the first quantity and the second quantity. The adjustment of the bitrate of the media data can be performed in an encoder within the source device (100).

[0134] The source device (100) can adjust the bitrate of the media data to decrease when the difference between the first quantity and the second quantity is greater than or equal to a threshold value. For example, the source device (100) can adjust the bitrate of the media data to decrease by 30% when the difference between the first quantity and the second quantity is 20% of the first quantity (i.e., 20). The source device (100) can encode (compress) the media data by lowering the default bitrate set to 60 Mbps to 42 Mbps. In this case, a video signal consisting of 60 frames per second can be compressed to 0.7 Mbps per frame instead of 1 Mbps.

[0135] The source device (100) can adjust the bitrate of the media data to increase when the difference between the first quantity and the second quantity is less than a threshold value.

[0136] Alternatively, the source device (100) may divide the difference between the first quantity and the second quantity into three intervals, so that when the difference is greater than or equal to the first threshold value, the bitrate of the media data decreases, when the difference is less than the second threshold value which is smaller than the first threshold value, the bitrate increases, and when the difference is between the first threshold value and the second threshold value, the bitrate is maintained.

[0137] The source device (100) may determine whether to adjust the bit rate or at least one of the amount of bit rate adjustment based on the difference between the first quantity and the second quantity. For example, the source device (100) may increase the amount of bit rate reduction or increase so that the difference between the first quantity and the second quantity increases. If the difference between the first quantity and the second quantity is large, the source device (100) may change to a second bit rate lower than the originally set first bit rate, and if the difference is small, it may change back to a third bit rate (or first bit rate) higher than the second bit rate.

[0138] The source device (100) can acquire compressed media data based on an adjusted bit rate. The source device (100) can generate a media packet based on the compressed media data and transmit it to a display device (200) through a network stack (e.g., 2200, 2300, 2400) according to the operation described above.

[0139] Accordingly, in a direct connection method between devices using the Wi-Fi Direct method, the source device (100), which is the transmitter, can determine the quantity of packets (first quantity) transmitted from the application layer (2100) to the network stack (e.g., 2200, 2300, 2400) through a socket and the quantity of packets (second quantity) measured at the network interface layer (2400), and can predict whether the packets have been properly delivered to the receiver, the display device (200). That is, the source device (100) can determine the remaining size of the buffer of the display device (200) in real time on its own, even without receiving separate feedback from the display device (200). By predicting the buffering status of the display device (200) on its own, the source device (100) can adjust the bitrate. The source device (100) can prevent buffer underruns of the display device (200) and provide continuous video to the user.

[0140] In the present disclosure, the quantity is exemplified as a number, but is not limited thereto, and the quantity may include at least one of a capacity or a number.

[0141] In one embodiment of the present disclosure, if the bitrate adjustment amount of the media data (e.g., 30%) is higher than the difference ratio between the first quantity and the second quantity (i.e., difference relative to the first quantity) (e.g., 20%), the buffer underrun problem can be resolved quickly. However, the bitrate adjustment amount according to the quantity of packets is merely an example and is not limited to a specific value.

[0142] In one embodiment of the present disclosure, a source device (100) may identify resolution information or data compression rate information of an image corresponding to an adjusted bit rate in order to adjust the bit rate of an image, and may adjust the resolution or data compression rate of an image according to the identified resolution information or data compression rate information. For example, the source device (100) may lower the resolution of an image by using resolution information of an image corresponding to a reduced bit rate in order to reduce the bit rate of an image. For example, the source device (100) may increase the data compression rate of an image by using data compression rate information corresponding to a reduced bit rate in order to reduce the bit rate of an image. Conversely, the source device (100) may increase the resolution of an image in order to increase the bit rate of an image. Or, the source device (100) may lower the data compression rate of an image in order to increase the bit rate of an image.

[0143] In one embodiment of the present disclosure, the source device (100) may include a video encoder for video data and an audio encoder for audio data, respectively. The source device (100) may determine whether to adjust the bitrate and the amount of bitrate adjustment for each of the video packet and the audio packet. The source device (100) may adjust the bitrate for the video packet through the video encoder. The source device (100) may adjust the bitrate for the audio packet through the audio encoder. This is described in more detail in FIG. 5.

[0144] FIG. 5 is a flowchart illustrating a method of operation for a source device according to an embodiment of the present disclosure to transmit media data using a network stack structure. The method of operation of the source device (100) according to an embodiment of the present disclosure may be performed by the processor (110 in FIG. 3) of the source device (100). Operations 510 to 550 describe a method of transmitting packets, and operations 555 to 570 describe an adaptive bitrate streaming method. Operations 510 to 550 and operations 555 to 570 may be performed independently of each other without needing to occur sequentially.

[0145] Referring to FIG. 5, in operation 510, the source device (100) can obtain video packets and audio packets based on media data at the application layer (2100). The source device (100) can separate data such as video, audio, and additional information contained in a single transmission stream. The source device (100) can obtain video packets, audio packets, and additional information packets corresponding to each of the data such as video, audio, and additional information.

[0146] For example, the demuxer (601) of FIG. 6 can separate video data and audio data from a video signal received from an external source device. The video encoder (602) of FIG. 6 can receive video data from the demuxer (601) and obtain encoded video data. The video packager (604) of FIG. 6 can receive the encoded video data and convert it into a packet format suitable for network transmission to generate a video packet. The video packager (604) can store the generated video packet in the video buffer (606) of FIG. 6 or transmit it to the video sender (608) of FIG. 6.

[0147] For example, the audio encoder (603) of FIG. 6 can receive audio data from the demuxer (601) and obtain encoded audio data. The audio packager (605) of FIG. 6 can receive the encoded audio data and convert it into a packet format suitable for network transmission to generate an audio packet. The audio packager (605) can store the generated audio packet in the audio buffer (607) of FIG. 6 or transmit it to the audio sender (609) of FIG. 6.

[0148] In operation 515, the source device (100) may call the socket system at the application layer (2100) to create (or execute) video sockets and audio sockets corresponding to video packets and audio packets. A video socket may serve as a channel for transmitting video packets between devices. An audio socket may serve as a channel for transmitting audio packets between devices. A network communication path may be established between the source device (100) and the display device (200) for transmitting and receiving video packets and audio packets, respectively. When the socket system is called, it may switch from user mode to kernel mode.

[0149] In operation 520, the source device (100) can set an IP header field value to distinguish between the video packet and the audio packet, respectively, for each of the video packet and the audio packet to be transmitted. In operation 525, the source device (100) can transmit the video packet and the audio packet, respectively, to the TCP / IP layer (2200, 2300) through the video socket and the audio socket. For example, the video sender (608) of FIG. 6 can set an IP header field value and transmit the video packet to the TCP / IP layer (2200, 2300) through the video socket. The audio sender (609) of FIG. 6 can set an IP header field value and transmit the audio packet to the TCP / IP layer (2200, 2300) through the audio socket.

[0150] For example, the field value may contain information about the Differential Service Code Point (DSCP) field. The DSCP field may be information included in the IP header of the Internet layer (2300). The DSCP field may be information for managing the Quality of Service (QOS) of network traffic. For example, the source device (100) may set the DSCP field to 0x2c in hexadecimal for video packets and the DSCP field to 0x3c for audio packets. However, it is not limited thereto. The traffic priority of 0x3c may be higher than that of 0x2c. For example, the video sender (608) of FIG. 6 may set the DSCP field value together when transmitting video packets through a video socket. The audio sender (609) of FIG. 6 may set the DSCP field value together when transmitting audio packets through an audio socket. Each of the video packet and audio packet containing IP headers having different field values ​​can be stored separately in the video buffer (710 in FIG. 7) and audio buffer (720 in FIG. 7) of the L2 driver (2420) described later. That is, the DSCP field can be used to store media data by type (video, audio, additional information, etc.) in the L2 driver (2420).

[0151] In operation 530, the source device (100) may add a TCP header and an IP header to the video packet and audio packet, respectively, at the TCP / IP layer (2200, 2300). The TCP layer may add a TCP header to the packet received from the application layer (2100) and pass it to the IP layer. The IP layer may add an IP header to the packet received from the TCP layer. The IP header may include a DSCP field value to distinguish between the video packet and the audio packet set in operation 520. An example of an IP header having different DSCP field values ​​is shown in FIG. 8.

[0152] In operation 535, the source device (100) can deliver a video packet with a TCP / IP header and an audio packet with a TCP / IP header to the L2 driver (2420), respectively.

[0153] In operation 540, the source device (100) can add a MAC header to each of the video packet and audio packet through the L2 driver (2420). The L2 driver (2420) may include a video buffer (710 in FIG. 7) for storing video packets, an audio buffer (720 in FIG. 7) for storing audio packets, and an additional information buffer (730 in FIG. 7) for storing additional information. The L2 driver (2420) may store the video packet and audio packet separately in their respective buffers based on different DSCP field values ​​added to the IP header. For example, the L2 driver (2420) may store the video packet in the video buffer (710 in FIG. 7) and the audio packet in the audio buffer (720 in FIG. 7). Additionally, the L2 driver (2420) may store additional information in the additional information buffer (730 in FIG. 7).

[0154] In operation 545, the source device (100) can transmit a video packet with a MAC / TCP / IP header and an audio packet with a MAC / TCP / IP header to the network chip (2440), respectively, through the L2 driver (2420).

[0155] In operation 550, the source device (100) can perform P2P communication with the display device (200) through the network chip (2440). For example, the source device (100) can be connected to the display device (200) via Wi-Fi Direct. However, it is not limited thereto. The display device (200) can receive video packets from the source device (100) through the network communication path of the video packet established according to operation 515. The display device (200) can receive audio packets from the source device (100) through the network communication path of the audio packet established according to operation 515.

[0156] In operation 555, the source device (100) can identify a first quantity of video packets transmitted through a video socket. The source device (100) can identify a first quantity of audio packets transmitted through an audio socket. The source device (100) can identify the first quantity of video packets and the first quantity of audio packets separately. Each of the first quantity of video packets and the first quantity of audio packets can be identified and transmitted in real time. For example, the video sender (608) of FIG. 6 can identify a first quantity of video packets transmitted through a video socket. The video sender (608) can transmit the first quantity of video packets to the bitrate adjustment module (650) of FIG. 6. The audio sender (609) of FIG. 6 can identify a first quantity of audio packets transmitted through an audio socket. The audio sender (609) can transmit a first quantity of audio packets to the bitrate adjustment module (650) of FIG. 6.

[0157] In operation 560, the source device (100) can identify a second quantity of video packets through the L2 driver (2420). The source device (100) can identify a second quantity of audio packets through the L2 driver (2420). As described in operation 540, the L2 driver (2420) can store video packets and audio packets separately in their respective buffers based on different DSCP field values ​​added to the IP header. For example, video packets can be stored in the video buffer (710 in FIG. 7) of the L2 driver (2420), and audio packets can be stored in the audio buffer (720 in FIG. 7). The L2 driver (2420) can measure a second quantity of video packets delivered from the TCP / IP layer (2200, 2300) to the video buffer (710 in FIG. 7) through the metering module (740 in FIG. 7). The L2 driver (2420) can measure a second quantity of audio packets transmitted from the TCP / IP layer (2200, 2300) to the audio buffer (720 in FIG. 7) through the metering module (740 in FIG. 7).

[0158] In operation 565, the L2 driver (2420) of the source device (100) can transmit a second quantity of video packets and a second quantity of audio packets to the application layer (2100). The source device (100) can identify (or obtain) the second quantity of video packets and the second quantity of audio packets at the application layer (2100). Data can be transmitted and received between the L2 driver (2420) and the application layer (2100) via Inter-Process Communication (IPC). Each of the second quantity of video packets and the second quantity of audio packets can be measured and transmitted in real time. For example, the L2 driver (2420) can transmit the second quantity of video packets to the bitrate adjustment module (650) of FIG. 6. The L2 driver (2420) can transmit the second quantity of audio packets to the bitrate adjustment module (650) of FIG. 6.

[0159] In operation 570, the source device (100) can adjust the bitrate based on the difference between a first quantity and a second quantity at the application layer (2100). The source device (100) can adjust the bitrate of the video data based on the difference between a first quantity of video packets and a second quantity of video packets. The source device (100) can generate video packets again based on the video data compressed in response to the adjusted bitrate. The source device (100) can adjust the bitrate of the audio data based on the difference between a first quantity of audio packets and a second quantity of audio packets. The source device (100) can generate audio packets again based on the audio data compressed in response to the adjusted bitrate. That is, the source device (100) can adjust the bitrate and operate again according to operation 510.

[0160] For example, the bitrate adjustment module (650) of FIG. 6 can determine whether to adjust the bitrate of the video data and the amount of bitrate adjustment (or the adjusted bitrate value) based on the difference between a first quantity of video packets and a second quantity of video packets. The bitrate adjustment module (650) can transmit the adjusted bitrate value to the video encoder (602) of FIG. 6. The video encoder (602) can compress the video data in accordance with the adjusted bitrate value. The video encoder (602) can extract the compressed video data in accordance with the adjusted bitrate value. The extracted video data can be generated again into a video packet. For example, the video encoder (602) can compress the video data based on at least one of the resolution of the image or the data compression rate of the image corresponding to the adjusted bitrate value.

[0161] For example, the bitrate adjustment module (650) of FIG. 6 can determine whether to adjust the bitrate of the audio data and the amount of bitrate adjustment (or the adjusted bitrate value) based on the difference between a first quantity of audio packets and a second quantity of audio packets. The bitrate adjustment module (650) can transmit the adjusted bitrate value to the audio encoder (603) of FIG. 6. The audio encoder (603) can compress the audio data in response to the adjusted bitrate value. The audio encoder (603) can extract the compressed audio data in response to the adjusted bitrate value. The extracted audio data can be generated again into an audio packet.

[0162] FIG. 6 is a diagram illustrating the operation of transmitting and receiving media data over a network between a source device and a display device according to one embodiment of the present disclosure. FIG. 6 illustrates the application layer (2100) in detail. FIG. 6 briefly illustrates the network stack (2200, 2300, 2400), which will be further explained in FIG. 7.

[0163] Referring to FIG. 6, the source device (100) may include an image processing unit (e.g., 130 of FIG. 3) running at the application layer (2100). The image processing unit may include appropriate programs, logic, circuits, interfaces, and / or code for separating data such as video, audio, and additional information from a single transmission stream and processing them into packets. For example, the image processing unit may include a demuxer (601), a video encoder (602), an audio encoder (603), a video packager (604), an audio packager (605), a video buffer (606), an audio buffer (607), a video sender (608), and an audio sender (609).

[0164] The demuxer (601) may include appropriate programs, logic, circuits, interfaces, and / or code for separating data such as video, audio, and additional information contained in a single transmission stream. For example, when a source device (100) receives a video signal from an external source device through an HDMI input port, the demuxer (601) may parse data standardized according to the HDMI standard to extract data such as video, audio, and additional information.

[0165] The video encoder (602) may include appropriate programs, logic, circuits, interfaces, and / or code for encoding (compressing) raw video data into a specific format. For example, the video encoder (602) may encode using codecs such as H.264, H.265, etc., but is not limited thereto. For example, the video encoder (602) may receive video data extracted from the demuxer (601) and extract encoded video data.

[0166] The video packager (604) may include appropriate programs, logic, circuits, interfaces, and / or code for converting encoded video data into a packet format suitable for network transmission. For example, the video packager (604) may divide the encoded video stream into packets of a certain size and prepare them for transmission over a network. The video packets extracted by the video packager (604) may include a payload containing at least a portion of the encoded video stream and an application header. For example, the structure of a video packet is illustrated in FIG. 8.

[0167] The video buffer (606) may be a buffer that temporarily stores video data before it is transmitted over a network. The video buffer (606) may include at least one of a queue corresponding to a First-In-First-Out (FIFO) storage structure or a stack corresponding to a Last-In-First-Out (LIFO) storage structure. For example, the video buffer (606) may store video packets extracted by the video packager (604).

[0168] The video sender (608) may include appropriate programs, logic, circuits, interfaces, and / or code for creating a socket for network communication and sending video data through the socket. For example, the video sender (608) may create a video socket to establish a network communication path for sending and receiving video packets between a source device (100) and a display device (200). The video sender (608) may forward video packets received through the video packager (604) or video buffer (606) to a lower layer network stack (610) via the video socket. Video packets that pass through the network stack (610), have a header of the communication protocol added, and reach the lowest layer network chip, can be forwarded to the display device (200) via the network chip. The network stack (610) is further described in FIG. 7.

[0169] The audio encoder (603), audio packager (605), audio buffer (607), and audio sender (609) differ from the aforementioned video encoder (602), video packager (604), video buffer (606), and video sender (608) in that they generate and transmit audio packets from audio data instead of generating and transmitting video packets from video data. A description of this is omitted.

[0170] The display device (200) may include an image processing unit (e.g., 230 in FIG. 3) running at the application layer (2100). The image processing unit may include appropriate programs, logic, circuits, interfaces, and / or code for processing video packets and audio packets received from the source device (100) to generate an image signal for output to the display (200). For example, the image processing unit may include a muxer (621), a video decoder (622), an audio decoder (623), a video depacketizer (624), an audio depacketizer (625), a video buffer (626), an audio buffer (627), a video receiver (628), and an audio receiver (629). Each component may operate in opposition to the components provided in the image processing unit of the source device (100).

[0171] For example, a display device (200) can receive video packets from a source device (100) through a network stack (630). The display device (200) can forward the video packets received through the network stack (630) to an application layer (2100) through a video socket. A video receiver (628) can receive video packets from the network stack (630) through a video socket. The video receiver (628) can forward the received video packets to a video depacketizer (624) or a video buffer (626). The video buffer (626) may be a buffer that temporarily stores the received video data. The video depacketizer (624) can restore the packetized video data received through the network back to its original stream form. The video decoder (622) can restore the compressed video data back to the original frame. The muxer (621) can combine multiple data streams (e.g., video, audio, subtitles, etc.) into a single integrated stream. The video signal generated by the muxer (621) can be transmitted to the display (200) after undergoing additional rendering, etc.

[0172] The audio receiver (629), audio buffer (627), audio depacketizer (625), and audio decoder (623) differ from the aforementioned video receiver (628), video buffer (626), video depacketizer (624), and video decoder (622) in that they extract audio data from audio packets. A description of this is omitted. Meanwhile, the muxer (621) can combine video data received from the video decoder (622) and audio data received from the audio decoder (623) into a single integrated stream.

[0173] The source device (100) may further include a bitrate adjustment module (650). The bitrate adjustment module (650) may include appropriate programs, logic, circuits, interfaces, and / or code for the source device (100) to adjust the bitrate for video data or audio data. For example, a video sender (608) may identify a first quantity of video packets transmitted through a video socket and transmit it to the bitrate adjustment module (650). A network stack (610) may measure a second quantity of video packets transmitted within the network stack (610) and transmit it to the bitrate adjustment module (650). The bitrate adjustment module (650) may receive the first quantity of video packets and the second quantity of video packets. Based on the first quantity of video packets and the second quantity of video packets, the bitrate adjustment module (650) may determine whether to adjust the bitrate of the video data and the amount of bitrate adjustment (or the adjusted bitrate value). The bitrate adjustment module (650) can transmit the adjusted bitrate to the video encoder (602).

[0174] For example, the audio sender (609) can identify a first quantity of audio packets transmitted through an audio socket and transmit it to the bitrate adjustment module (650). The network stack (610) can measure a second quantity of audio packets transmitted within the network stack (610) and transmit it to the bitrate adjustment module (650). The bitrate adjustment module (650) can receive the first quantity of audio packets and the second quantity of audio packets. Based on the first quantity of audio packets and the second quantity of audio packets, the bitrate adjustment module (650) can determine whether to adjust the bitrate of the audio data and the amount of bitrate adjustment (or the adjusted bitrate value). The bitrate adjustment module (650) can transmit the adjusted bitrate to the audio encoder (603).

[0175] The bitrate adjustment module (650) can periodically receive first quantity information and second quantity information.

[0176] The video encoder (602) can compress video data to have an adjusted bitrate and extract the compressed video data. The extracted video data can then be generated into a video packet through the video packager (604).

[0177] The audio encoder (603) can compress audio data to have an adjusted bitrate and extract the compressed audio data. The extracted audio data can then be generated into an audio packet through the audio packager (605).

[0178] The first and second quantities of the packets will be further explained with reference to FIGS. 7 and FIGS. 8.

[0179] FIG. 7 is a diagram illustrating the operation in the network stack structure of a source device and a display device according to one embodiment of the present disclosure. FIG. 8 is a diagram showing the form of a packet corresponding to the network stack structure according to one embodiment of the present disclosure. FIG. 7 illustrates in detail the transmission layer (2200), the internet layer (2300), and the network interface layer (2400), which are lower layers of the application layer (2100). The network stack of the transmitter corresponds to the network stack (610) of FIG. 6, and the network stack of the receiver corresponds to the network stack (630) of FIG. 6. The packet illustrated in FIG. 8 may correspond to at least one of a video packet or an audio packet. In FIG. 8, descriptions that overlap with FIG. 2 are omitted or simplified.

[0180] As described in FIG. 6, the application layer (2100) may include a demuxer (601), a video encoder (602), an audio encoder (603), a video packager (604), an audio packager (605), a video buffer (606), an audio buffer (607), a video sender (608), and an audio sender (609). The video sender (608) of the application layer (2100) may create a video socket and transmit video packets to the transport layer (2200) through the video socket. The audio sender (609) may create an audio socket and transmit audio packets to the transport layer (2200) through the audio socket. Referring to FIG. 8, the form of the packet transmitted from the application layer (2100) to the transport layer (2200) may include an application header and at least a portion of stream data (represented as user data).

[0181] The transport layer (2200) can add a TCP header to the packet received from the application layer (2100) and deliver it to the internet layer (2300). The internet layer (2300) can add an IP header to the packet received from the transport layer (2200). The internet layer (2300) can deliver the video packet with the TCP / IP header added and the audio packet with the TCP / IP header added, respectively, to the L2 driver (2420). Referring to FIG. 8, the value of DSCP 1 can be set in the DSCP field of the IP header of the video packet. The value of DSCP 2 can be set in the DSCP field of the IP header of the audio packet. For example, if DSCP 1 is 0x2c and DSCP 2 is 0x3c, the video packet may have a lower priority than the audio packet in network traffic.

[0182] The L2 driver (2420) may include a video buffer (710) for storing video packets, an audio buffer (720) for storing audio packets, and an additional information buffer (730) for storing additional information. Each of the video buffer (710), the audio buffer (720), and the additional information buffer (730) may include at least one storage structure among a queue or a stack. The L2 driver (2420) may store video packets and audio packets separately in their respective buffers based on different DSCP field values ​​attached to the IP header. For example, the L2 driver (2420) may store video packets in the video buffer (710) and audio packets in the audio buffer (720). Additionally, the L2 driver (2420) may store additional information in the additional information buffer (730). When video packets and audio packets are stored separately using different DSCP field values, each of the video packets and audio packets may be managed independently. In other words, problems with video packets do not affect audio packets, and the same applies in the opposite case.

[0183] The L2 driver (2420) may include a metering module (740). The metering module (740) may include appropriate logic, circuits, interfaces, and / or code for measuring the amount of packets received or transmitted. The L2 driver (2420) may measure a second quantity of video packets delivered from the TCP / IP layer (2200, 2300) to the video buffer (710) through the metering module (740). The L2 driver (2420) may measure a second quantity of audio packets delivered from the TCP / IP layer (2200, 2300) to the audio buffer (720) through the metering module (740). Since the video packets and audio packets are stored in separate buffers, the metering module (740) may measure the second quantity of video packets and the second quantity of audio packets separately. The L2 driver (2420) can transmit the second quantity of video packets and the second quantity of audio packets measured through the metering module (740) to the application layer (2100).

[0184] Video packets and audio packets that have reached the network chip (2440) can be transmitted to the display device (200) through the network chip (2440). The display device (200) can receive video packets and audio packets through the network chip provided in the display device (200) and transmit them to the application layer via the L2 driver, IP layer, and TCP layer. The display device (200) can store video packets, audio packets, and additional information in respective buffers (e.g., video buffer, audio buffer, and additional information buffer).

[0185] FIG. 9 is a flowchart illustrating a method of transmitting and receiving media data through a network between a source device and a display device according to one embodiment of the present disclosure.

[0186] Referring to FIG. 9, the source device (100) differs from the embodiments of FIG. 4 and FIG. 5 described above in that it adjusts the bit rate based on a feedback signal received from the display device (200).

[0187] In operation 910, the source device (100) can acquire a media packet. The media packet may include a video packet and an audio packet. This corresponds to operation 510 of FIG. 5.

[0188] In operation 920, the source device (100) can transmit a media packet to the display device (200). P2P communication can be performed between the source device (100) and the display device (200). This corresponds to operation 550 of FIG. 5. The display device (200) can receive a media packet from the source device (100).

[0189] In operation 930, the display device (200) can manage its buffers in real time. For example, the display device (200) can manage the quantity of video packets stored in the video buffer (626 in FIG. 6) and the quantity of audio packets stored in the audio buffer (627 in FIG. 6). If the display device (200) determines that the quantity of packets stored in the video buffer (626 in FIG. 6) and the audio buffer (627 in FIG. 6) is insufficient, it can generate a feedback signal to be transmitted to the source device (100). The feedback signal can be generated at the application layer of the display device (200).

[0190] In operation 940, the display device (200) can create (or execute) a feedback socket at the application layer and transmit a feedback signal to the lower layer network stack (630 in FIG. 6) through the feedback socket. The feedback socket can serve as a channel for transmitting feedback signals between devices. Accordingly, a network communication path for transmitting feedback signals can be established. The display device (200) can transmit the feedback signal to the source device (100) through the network stack (630 in FIG. 6). The source device (100) can receive the feedback signal from the display device (200) through the established network communication path of the feedback signal.

[0191] In operation 950, the source device (100) can adjust the bit rate of the media data based on the feedback signal. Operation 950 can correspond to 570 of FIG. 5.

[0192] When the source device (100) adjusts the bit rate based on a feedback signal received from the display device (200), it is difficult to immediately resolve the buffer underrun problem of the display device (200). For example, if the network environment between the source device (100) and the display device (200) is unstable, it is difficult for the source device (100) to receive a feedback signal from the display device (200).

[0193] Accordingly, the source device (100) can predict the remaining size of the buffer of the display device (200) in real time on its own, even without receiving separate feedback from the display device (200), as described in FIGS. 4 and 5.

[0194] The source device (100) may operate as in FIGS. 4 and FIG. 5, and may also operate according to operations 910 to 940 simultaneously.

[0195] FIG. 10 is a system block diagram of a source device and a display device according to one embodiment of the present disclosure.

[0196] Referring to FIG. 10, the system (1000) may include a processor (1001), memory (1002), a tuner unit (1010), a communication unit (1020), a detection unit (1030), an input / output unit (1040), a video processing unit (1050), a display (1060), an audio processing unit (1070), an audio output unit (1080), and an input interface (1090).

[0197] The processor (1001), memory (1002), and communication unit (1020) included in the system (1000) may correspond to the processor (110), memory (150), and communication unit (140) included in the source device (100) of FIG. 3. The tuner unit (1010), at least part of the communication unit (1020), and input / output unit (1040) included in the system (1000) may correspond to the image receiving unit (120) included in the source device (100) of FIG. 3. The video processing unit (1050) and audio processing unit (1070) included in the system (1000) may correspond to the image processing unit (130) included in the source device (100) of FIG. 3.

[0198] The processor (1001), memory (1002), communication unit (1020), and display (1060) included in the system (1000) may correspond to the processor (210), memory (250), communication unit (140), and display (240) included in the display device (200) of FIG. 3. The video processing unit (1050) and audio processing unit (1070) included in the system (1000) may correspond to the image processing unit (230) included in the display device (200) of FIG. 3.

[0199] The tuner unit (1010) can select only the frequency of the channel to be received by the system (1000) from among many radio wave components by tuning through amplification, mixing, resonance, etc. of broadcast content received via wired or wireless connection. The content received through the tuner unit (1010) is decoded and separated into audio, video, and / or additional information. The separated audio, video, and / or additional information can be stored in memory (1002) under the control of the processor (1001).

[0200] The communication unit (1020) can connect the system (1000) to peripheral devices, external devices, servers, mobile terminals, etc. under the control of the processor (1001). The communication unit (1020) may include at least one communication module capable of performing wireless communication. The communication unit (1020) may include at least one of a wireless LAN module (1021), a Bluetooth module (1022), and a wired Ethernet (1023) in accordance with the performance and structure of the system (1000).

[0201] The wireless LAN module (1021) can transmit and receive Wi-Fi signals with a peripheral device according to the Wi-Fi communication standard. The Bluetooth module (1022) can receive Bluetooth signals transmitted from a peripheral device according to the Bluetooth communication standard.

[0202] The detection unit (1030) detects the user's voice, the user's image, or the user's interaction, and may include a microphone, a camera unit, an optical receiver, and a sensing unit.

[0203] The input / output unit (1040) can receive video (e.g., dynamic image signal or still image signal), audio (e.g., voice signal or music signal), and additional information from an external device, etc., under the control of the processor (1001). The input / output unit (1040) may include one of an HDMI port (High-Definition Multimedia Interface port), a component jack, a PC port, and a USB port.

[0204] The audio processing unit (1070) performs processing on audio data. Various processing such as decoding, amplification, and noise filtering on the audio data can be performed in the audio processing unit (1070).

[0205] The audio output unit (1080) can output audio included in the content received through the tuner unit (1010) under the control of the processor (1001), audio input through the communication unit (1020) or input / output unit (1040), and audio stored in the memory (1002). The audio output unit (1080) may include at least one of a speaker, headphones, or S / PDIF (Sony / Philips Digital Interface: output terminal).

[0206] The input interface (1090) can receive user input for controlling the system (1000). The input interface (1090) may include, but is not limited to, various forms of user input devices including, a touch panel for detecting user touch, a button for receiving user push operation, a wheel for receiving user rotation operation, a keyboard, a dome switch, a microphone for voice recognition, a motion detection sensor for sensing motion.

[0207] A source device according to one embodiment of the present disclosure includes a wireless communication module that supports direct communication with a display device, at least one processor, and a memory comprising one or more storage media that stores one or more instructions.

[0208] According to one embodiment of the present disclosure, the source device identifies a first quantity of media packets generated to transmit media data to the display device by having the at least one processor execute the one or more instructions individually or collectively.

[0209] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device identifies a second quantity of media packets representing the number of media packets transmitted to the display device through the wireless communication module.

[0210] According to one embodiment of the present disclosure, the source device adjusts the bitrate of the media data based on the difference between the first quantity and the second quantity by executing the one or more instructions individually or in combination by the at least one processor.

[0211] The first quantity of the media packet according to one embodiment of the present disclosure may correspond to the number of media packets transmitted from the application layer to the transport layer through a socket.

[0212] According to one embodiment of the present disclosure, the second quantity of the media packet may correspond to at least one of the number of media packets transmitted from the internet layer to the L2 driver, or the number of media packets transmitted from the L2 driver to the wireless communication module.

[0213] According to one embodiment of the present disclosure, a second quantity of media packets transmitted from the internet layer to the L2 driver may represent the number of media packets measured through the L2 driver.

[0214] A source device according to one embodiment of the present disclosure may be characterized by being connected to a display device via Wi-Fi Direct communication.

[0215] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can adjust the bit rate of the media data to decrease when the difference between the first quantity and the second quantity is greater than or equal to a threshold value.

[0216] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can adjust the bit rate of the media data to increase or maintain when the difference between the first quantity and the second quantity is less than the threshold value.

[0217] According to one embodiment of the present disclosure, depending on the difference between the first quantity and the second quantity, at least one of whether to adjust the bit rate or the amount of bit rate adjustment may be determined.

[0218] According to one embodiment of the present disclosure, by having the at least one processor execute the one or more instructions individually or collectively, the source device can transmit an adjusted bitrate value to an encoder based on the difference between the first quantity and the second quantity.

[0219] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can obtain encoded media data based on the adjusted bitrate value through the encoder.

[0220] According to one embodiment of the present disclosure, the source device can acquire video packets and audio packets based on media data at the application layer by executing the one or more instructions individually or in combination by the at least one processor.

[0221] According to one embodiment of the present disclosure, by executing one or more instructions individually or in combination, the source device can set an IP header field value for distinguishing between video packets and audio packets.

[0222] Each of the video packet and audio packet including the set IP header field value according to one embodiment of the present disclosure may be stored in a separate buffer.

[0223] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can separately identify a first quantity of the video packet and a first quantity of the audio packet, respectively.

[0224] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can separately identify a second quantity of the video packet and a second quantity of the audio packet, respectively.

[0225] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can adjust the bit rate of the video data based on the difference between a first quantity of the video packet and a second quantity of the video packet.

[0226] According to one embodiment of the present disclosure, by having at least one processor execute the one or more instructions individually or in combination, the source device can adjust the bit rate of the audio data based on the difference between a first quantity of the audio packet and a second quantity of the audio packet.

[0227] A method of operation of a source device according to one embodiment of the present disclosure includes: identifying a first quantity of media packets generated to transmit media data to the display device; identifying a second quantity of media packets representing the number of media packets transmitted to the display device through a wireless communication module that supports direct communication with the display device; and adjusting the bitrate of the media data based on the difference between the first quantity and the second quantity.

[0228] According to one embodiment of the present disclosure, the step of adjusting the bitrate of the media data based on the difference between the first quantity and the second quantity may include the step of adjusting the bitrate of the media data to decrease when the difference between the first quantity and the second quantity is greater than or equal to a threshold value, and the step of adjusting the bitrate of the media data to increase or maintain when the difference between the first quantity and the second quantity is less than the threshold value.

[0229] According to one embodiment of the present disclosure, the step of adjusting the bitrate of the media data based on the difference between the first quantity and the second quantity may include the step of obtaining a bitrate value adjusted based on the difference between the first quantity and the second quantity, and the step of obtaining encoded media data based on the adjusted bitrate value through an encoder.

[0230] A method of operation of a source device according to one embodiment of the present disclosure may further include the step of acquiring a video packet and an audio packet based on media data at the application layer; and the step of setting an IP header field value for distinguishing between the video packet and the audio packet.

[0231] Each of the video packet and audio packet, including the set IP header field value according to one embodiment of the present disclosure, may be stored in a separate buffer.

[0232] A method of operating a source device according to one embodiment of the present disclosure may further include the steps of separately identifying a first quantity of the video packet and a first quantity of the audio packet, respectively; separately identifying a second quantity of the video packet and a second quantity of the audio packet, respectively; adjusting the bitrate of the video data based on the difference between the first quantity of the video packet and the second quantity of the video packet; and adjusting the bitrate of the audio data based on the difference between the first quantity of the audio packet and the second quantity of the audio packet.

[0233] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0234] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Claims

1. In a source device, A wireless communication module that supports direct communication with a display device; At least one processor; and Memory comprising one or more storage media that store one or more instructions, and By the above at least one processor executing the above one or more instructions individually or collectively, the source device, Identifying a first quantity of media packets generated to transmit media data to the display device, and Identifying a second quantity of media packets indicating the number of media packets transmitted to the display device through the wireless communication module, and A source device that adjusts the bitrate of the media data based on the difference between the first quantity and the second quantity.

2. In Paragraph 1, A source device, wherein the first quantity of the above media packets corresponds to the number of media packets transmitted from the application layer to the transport layer through a socket.

3. In Paragraph 1 or 2, A source device in which the second quantity of the above media packets corresponds to at least one of the number of media packets transmitted from the internet layer to the L2 driver, or the number of media packets transmitted from the L2 driver to the wireless communication module.

4. In Paragraph 3, A source device in which the second quantity of media packets transmitted to the L2 driver in the above internet layer represents the number of media packets measured through the L2 driver.

5. In any one of paragraphs 1 through 4, The source device is characterized by being connected to the display device via Wi-Fi Direct communication.

6. In any one of paragraphs 1 through 5, By the above at least one processor executing the above one or more instructions individually or collectively, the source device, If the difference between the first quantity and the second quantity is greater than or equal to a threshold value, the bitrate of the media data is adjusted to decrease, and A source device that adjusts the bitrate of the media data to increase or maintain it when the difference between the first quantity and the second quantity is less than the threshold value.

7. In Paragraph 6, A source device in which at least one of whether to adjust the bit rate or the amount of bit rate adjustment is determined according to the difference between the first quantity and the second quantity.

8. In any one of paragraphs 1 through 7, By the above at least one processor executing the above one or more instructions individually or collectively, the source device, Based on the difference between the first quantity and the second quantity, an adjusted bitrate value is transmitted to the encoder, and A source device that obtains encoded media data based on the adjusted bitrate value through the encoder.

9. In any one of paragraphs 1 through 8, By the above at least one processor executing the above one or more instructions individually or collectively, the source device, In the above application layer, video packets and audio packets are obtained based on media data, and Set IP header field values ​​to distinguish between video and audio packets, and A source device in which a video packet and an audio packet, each containing the IP header field value set above, are stored in separate buffers.

10. In Paragraph 9, By the above at least one processor executing the above one or more instructions individually or collectively, the source device, The first quantity of the video packet and the first quantity of the audio packet are each identified separately, The second quantity of the video packet and the second quantity of the audio packet are each identified separately, Based on the difference between the first quantity of the video packet and the second quantity of the video packet, the bitrate of the video data is adjusted, and A source device that adjusts the bitrate of audio data based on the difference between a first quantity of the audio packet and a second quantity of the audio packet.

11. In the method of operating a source device, A step of identifying a first quantity of media packets generated to transmit media data to a display device; A step of identifying a second quantity of media packets indicating the number of media packets generated and transmitted to the display device through a wireless communication module that supports direct communication with the display device; and A method comprising the step of adjusting the bitrate of the media data based on the difference between the first quantity and the second quantity.

12. In Paragraph 11, A method in which the second quantity of the above media packets corresponds to at least one of the number of media packets transmitted from the internet layer to the L2 driver, or the number of media packets transmitted from the L2 driver to the wireless communication module.

13. In Paragraph 12, A method in which a second quantity of media packets transmitted from the above internet layer to the above L2 driver represents the number of media packets measured through the above L2 driver.

14. In any one of paragraphs 11 through 13, The step of adjusting the bitrate of the media data based on the difference between the first quantity and the second quantity is, A step of adjusting the bitrate of the media data to decrease when the difference between the first quantity and the second quantity is greater than or equal to a threshold value; and A method comprising the step of adjusting the bitrate of the media data to increase or maintain it when the difference between the first quantity and the second quantity is less than the threshold value.

15. A step of identifying a first quantity of media packets generated to transmit media data to a display device; A step of identifying a second quantity of media packets indicating the number of media packets generated and transmitted to the display device through a wireless communication module that supports direct communication with the display device; and A computer-readable non-transient recording medium having a program recorded thereon for performing a method of operation of a source device on a computer, comprising the step of adjusting the bitrate of the media data based on the difference between the first quantity and the second quantity.

Images