Electronic device and data transmission method thereof

Abstract

Disclosed is an electronic device. The electronic device comprises: a first communication interface; a second communication interface for wirelessly communicating with a display device; and at least one processor. The at least one processor: receives image data from an external device via the first communication interface; identifies the type of content included in the image data on the basis of information received from the external device via the first communication interface; changes the resolution or frame rate of the image data on the basis of the type of the content when the network quality between the electronic device and the display device is not good; and transmits, to the display device via the second communication interface, the image data that has had the resolution or the frame rate changed.

Description

Electronic devices and their data transmission methods

[0001] The present disclosure relates to an electronic device for transmitting data wirelessly and a data transmission method thereof.

[0002] With recent advancements in electronic technology, display devices such as TVs are now wirelessly connected to peripheral devices, providing a better aesthetic appearance.

[0003] Meanwhile, if the wireless network condition between the display device and the peripheral device deteriorates due to interference or other reasons, wireless communication between the display device and the peripheral device may be affected.

[0004] An electronic device according to an embodiment of the present disclosure includes a first communication interface, a second communication interface for wirelessly communicating with a display device, and one or more processors. The one or more processors receive image data from an external device through the first communication interface. The one or more processors identify the type of content included in the image data based on information received from the external device through the first communication interface. The one or more processors change the resolution or frame rate of the image data based on the type of content, based on the network quality between the electronic device and the display device. The one or more processors transmit the image data with the changed resolution or frame rate to the display device through the second communication interface. The one or more processors may change the resolution or frame rate of the image data based on the type of content when the network quality of the electronic device and the display device is poor. The poor network quality may mean that the network quality is lower than a preset value. The good network quality may mean that the network quality is higher than a preset value.

[0005] Additionally, the one or more processors can identify the type of the content based on ALLM (Auto Low Latency Mode) information received from the external device through the first communication interface.

[0006] Additionally, the content included in the above video data may be game content or general content.

[0007] Additionally, the one or more processors may change the resolution of the image data by downscaling the image data if the content is the game content, and may change the frame rate of the image data by using frame skipping if the content is the general content.

[0008] Additionally, if the content is the game content, the frame rate of the video data may not be changed, and if the content is the general content, the resolution of the video data may not be changed.

[0009] In addition, the data received from the external device may include the image data, audio data, and metadata. If the network quality between the electronic device and the display device is not good, the one or more processors may compress the image data with the changed resolution or frame rate using a video codec, compress the audio data using an audio codec, and transmit the compressed image data, the compressed audio data, and the metadata to the display device via the second communication interface. The metadata may include ALLM information.

[0010] In addition, the data received from the external device may include the image data, audio data, and metadata. If the network quality between the electronic device and the display device is good, the one or more processors may compress the image data using a video codec, compress the audio data using an audio codec, and transmit the compressed image data, the compressed audio data, and the metadata to the display device via the second communication interface. The metadata may include ALLM information.

[0011] Additionally, the one or more processors can receive information about network quality between the electronic device and the display device from the display device through the second communication interface, and identify the network quality between the electronic device and the display device based on the received information.

[0012] Additionally, the information about the network quality may include packet loss rate.

[0013] A method for transmitting data by an electronic device wirelessly communicating with a display device according to an embodiment of the present disclosure includes the steps of receiving image data from an external device, identifying a type of content included in the image data based on information received from the external device, changing a resolution or a frame rate of the image data based on the type of content based on network quality between the electronic device and the display device, and transmitting the image data with the changed resolution or frame rate to the display device. The changing of the resolution or frame rate of the image data may be based on poor network quality between the electronic device and the display device.

[0014] A non-transitory computer-readable medium storing computer instructions that, when executed by one or more processors of an electronic device that wirelessly communicates with a display device according to an embodiment of the present disclosure, cause the electronic device to perform an operation, the operation comprising: receiving image data from an external device; identifying a type of content included in the image data based on information received from the external device; changing a resolution or a frame rate of the image data based on the type of content based on a network quality between the electronic device and the display device; and transmitting the image data with the changed resolution or frame rate to the display device. The changing of the resolution or frame rate of the image data may be based on poor network quality between the electronic device and the display device.

[0015] FIG. 1 is a diagram schematically illustrating an electronic device transmitting image data according to one embodiment of the present disclosure.

[0016] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0017] FIG. 3 is a flowchart illustrating an operation of an electronic device transmitting image data to a display device according to one embodiment of the present disclosure.

[0018] FIG. 4 is a drawing for explaining the operation of an electronic device according to one embodiment of the present disclosure.

[0019] FIG. 5 is a block diagram illustrating the configuration of a display device according to one embodiment of the present disclosure.

[0020] FIG. 6 is a flowchart illustrating an operation of a display device obtaining information about network quality according to one embodiment of the present disclosure.

[0021] FIG. 7 is a flowchart illustrating an operation of a display device processing data received from an electronic device according to one embodiment of the present disclosure.

[0022] FIG. 8 is a drawing for explaining the operation of a display device according to one embodiment of the present disclosure.

[0023] FIG. 9 is a flowchart illustrating a data transmission method of an electronic device that wirelessly communicates with a display device according to one embodiment of the present disclosure.

[0024] It should be understood that the various embodiments of the present disclosure and the terminology used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, but rather to include various modifications, equivalents, or substitutes of the embodiments.

[0025] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0026] The singular form of a noun corresponding to an item may include one or more items, unless the context clearly indicates otherwise.

[0027] In this disclosure, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can each include any one of the items listed together in that phrase, or all possible combinations thereof. For example, "A or B," "at least one of A and B," or "at least one of A or B" can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0028] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0029] When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0030] The terms "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in this document, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0031] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0032] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0033] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0034] In some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0035] In the embodiments, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of "modules" or "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "module" or "part" that needs to be implemented as specific hardware.

[0036] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0037] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the attached drawings.

[0038] FIG. 1 is a diagram schematically illustrating an electronic device transmitting image data according to one embodiment of the present disclosure.

[0039] The electronic device (100) may be implemented as an electronic device of various types and forms that can be connected to a display device (200) and an external device (300). For example, the electronic device (100) may be replaced with an expression such as an OC (One Connect) box, but is not limited thereto.

[0040] The electronic device (100) receives data from an external device (300) and transmits the data received from the external device (300) to the display device (200). In this case, the data may include image data, audio data, and metadata. The metadata may include various information about the image and / or audio data.

[0041] The external device (300) is connected to the electronic device (100) and is implemented as one of various types and forms of electronic devices capable of transmitting video and audio data. For example, the external device (300) may be implemented as an electronic device capable of functioning as a source device, such as a console game machine, a Blu-ray disc player, a set-top box, a smartphone, a tablet PC, a laptop PC, etc., but is not limited thereto.

[0042] The display device (200) receives data from the electronic device (100), processes the received data, and outputs image and audio signals. For example, the display device (200) is implemented as one of various types and forms of electronic devices that can be connected to the electronic device (100) and can receive and output image and audio data. For example, the display device (200) is implemented as an electronic device that can function as a sink device such as a TV, monitor, or projector, but is not limited thereto.

[0043] An electronic device (100) and a display device (200) communicate wirelessly. At this time, the network quality between the electronic device (100) and the display device (200) may deteriorate due to interference or other reasons. If the network quality between the electronic device (100) and the display device (200) is not good, the electronic device (100) changes the resolution or frame rate of image data received from an external device (300) and transmits the image data with the changed resolution or frame rate to the display device (200).

[0044] Changing the resolution may include lowering the resolution of the image data using downscaling. Furthermore, changing the frame rate may include lowering the frame rate of the image data using frame skipping. That is, if the network quality between the electronic device (100) and the display device (200) is poor, the electronic device (100) lowers the resolution of the image data or lowers the frame rate of the image data to reduce the amount of data transmitted to the display device (200).

[0045] Specific operations of the electronic device (100) to change the resolution or frame rate of image data and transmit it to the display device (200) will be described in more detail through the drawings and descriptions thereof described below.

[0046] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0047] Referring to FIG. 2, the electronic device (100) may include a first communication interface (110), a second communication interface (120), a memory (130), and one or more processors (140). However, this configuration is exemplary, and it is obvious that some components may be omitted from this configuration or new components may be added when implementing the present disclosure.

[0048] The first communication interface (110) can communicate with an external device (300) under the control of one or more processors (140). For example, the first communication interface (110) can communicate with the external device (300) using a High-Definition Multimedia Interface (HDMI).

[0049] The first communication interface (110) includes an HDMI communication module. The HDMI communication module may include a communication circuit capable of performing data communication between an electronic device (100) and an external device (300) using an HDMI communication method.

[0050] The second communication interface (120) performs wireless communication with the display device (200) under the control of one or more processors (140). For example, the second communication interface (120) may communicate with the display device (200) using Wi-Fi.

[0051] The second communication interface (120) includes a Wi-Fi communication module. The Wi-Fi communication module includes a communication circuit capable of performing data communication between the electronic device (100) and the display device (200) using a Wi-Fi communication method. At this time, the Wi-Fi communication module performs Wi-Fi Direct communication for direct wireless communication between the electronic device (100) and the display device (200). However, the present invention is not limited thereto, and the Wi-Fi communication module performs wireless communication between the electronic device (100) and the display device (200) through an access point (AP).

[0052] Memory (130) may store instructions, data structures, and program codes that can be read by one or more processors (140). Operations performed by one or more processors (140) may be implemented by executing instructions or codes of a program stored in memory (130).

[0053] The memory (130) may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and may include a non-volatile memory including at least one of a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as a RAM (Random Access Memory) or an SRAM (Static Random Access Memory).

[0054] The memory (130) may store one or more instructions and / or programs that cause the electronic device (100) to operate according to the present disclosure.

[0055] One or more processors (140) control the overall operations of the electronic device (100). For example, one or more processors (140) are operatively connected to the first communication interface (110), the second communication interface (120), and the memory (130) to control the electronic device (100). In addition, one or more processors (140) execute one or more instructions of a program stored in the memory (130), thereby controlling the overall operations for the electronic device (100) to receive data and transmit the received data. One or more processors (140) may be configured as one or more processors.

[0056] The one or more processors (140) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (140) may control one or any combination of other components of the electronic device (100) and may perform operations related to communication or data processing.

[0057] For example, one or more processors (140) may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory (130).

[0058] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0059] One or more processors (140) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (140) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0060] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0061] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0062] For convenience of explanation, one or more processors (140) are referred to as processors (140) below.

[0063] FIG. 3 is a flowchart illustrating an operation of an electronic device transmitting image data to a display device according to one embodiment of the present disclosure.

[0064] The processor (140) receives image data from an external device (300) through the first communication interface (110) (S310).

[0065] The processor (140) identifies the type of content included in the image data (S315).

[0066] Content types can be categorized based on whether the content is game content or general content. Specifically, content contained in video data can be either game content or general content. General content can include various types of content, excluding game content, such as movies, dramas, photos, and broadcasts.

[0067] The processor (140) identifies the type of content included in the image data using information received from an external device (300) through the first communication interface (110).

[0068] For example, the external device (300) may be preset to transmit information indicating that game content is included to the electronic device (100) when game content is included in the data transmitted by the external device (300) to the electronic device (100).

[0069] For example, the information may include ALLM (Auto Low Latency Mode) information defined in HDMI (e.g., HDMI 2.1). ALLM information may be included in metadata.

[0070] For example, if game content is included in the data transmitted from the external device (300) to the electronic device (100), the external device (300) can use the ALLM information to transmit a signal to the display device (200) to automatically switch the mode of the display device (200) to a mode for gaming (e.g., low latency mode). ALLM can be replaced with expressions such as game mode, but is not limited thereto.

[0071] The processor (140) identifies the type of content based on ALLM information received from an external device (300) via the first communication interface (110).

[0072] For example, when ALLM information is received from an external device (300), the processor (140) identifies the image data received from the external device (300) as game content, and when ALLM information is not received from the external device (300), the processor (140) identifies the image data received from the external device (300) as general content.

[0073] The processor (140) identifies the network quality between the electronic device (100) and the display device (200) (S320).

[0074] For example, the processor (140) receives information about the network quality between the electronic device (100) and the display device (200) from the display device (200) through the second communication interface (120), and identifies the network quality between the electronic device (100) and the display device (200) based on the received information. In this case, the network may include a wireless network. The network quality may be replaced with expressions such as network status, network situation, channel quality, channel status, channel status, etc.

[0075] Information about network quality may include a packet loss rate. For example, the display device (200) may identify a packet loss rate based on packets received from the electronic device (100) and transmit information about the packet loss rate to the electronic device (100).

[0076] The processor (140) receives information on a packet loss rate from the display device (200) through the second communication interface (120), and identifies the network quality between the electronic device (100) and the display device (200) based on the packet loss rate.

[0077] For example, the processor (140) identifies the network quality as good if the packet loss rate is less than the threshold, and identifies the network quality as bad if the packet loss rate is greater than the threshold.

[0078] Meanwhile, while the aforementioned example described network quality as being measured based on packet loss rates, the present invention is not limited thereto. For example, network quality between the electronic device (100) and the display device (200) may be determined by various indicators such as bandwidth, time delay, and jitter.

[0079] In this disclosure, poor network quality may include low channel throughput due to interference or other reasons. Accordingly, poor network quality may result in packet loss, which may increase the packet loss rate.

[0080] If the network quality between the electronic device (100) and the display device (200) is good (S325-Y), the processor (140) transmits image data received from the external device (300) to the display device (200) via the second communication interface (120) (S330).

[0081] At this time, the electronic device (100) wirelessly transmits image data to the display device (200). At this time, wireless communication can be distinguished from wired communication that performs data communication using a cable in that data communication is performed by transmitting and receiving radio waves without using a cable.

[0082] For example, the processor (140) encodes image data received from an external device (300) using a video codec to compress the image data. Then, the processor (140) transmits the compressed image data to the display device (200) using the second communication interface (120).

[0083] If the network quality between the electronic device (100) and the display device (200) is not good, the processor (140) changes the resolution or frame rate of the image data among the data based on the type of content. Then, the processor (140) transmits the image data with the changed resolution or frame rate to the display device (200) through the second communication interface (120).

[0084] For example, if the network quality between the electronic device (100) and the display device (200) is poor (S325-N) and the content is game content (S335-Y), the processor (140) downscales the image data to change the resolution of the image data (S340). In this case, the frame rate of the image data is not changed. Downscaling the image data can be expressed as downscaling the image frames. The resolution of the downscaled image data can be preset. For example, if the resolution of the image data received from the external device (300) is 4k, the processor (140) can use downscaling to lower the resolution of the image data to 2k.

[0085] Then, the processor (140) transmits the image data with the changed resolution to the display device (200) through the second communication interface (120) (S345).

[0086] For example, the processor (140) encodes image data with a lower resolution using a video codec to compress the image data. Then, the processor (140) transmits the compressed image data to the display device (200) using the second communication interface (120).

[0087] For example, if the network quality between the electronic device (100) and the display device (200) is not good (S325-N) and the content is not game content, that is, if the content is general content (S335-N), the processor (140) changes the frame rate of the image data using frame skip (S350). In this case, the resolution of the image data is not changed. The frame skip may be used to lower the frame rate by skipping some frames among the image frames. The frame skip rate may be preset. For example, if the frame rate of the image data received from the external device (300) is 60 frames per second (fps) and the frame skip rate is 1 / 2, the processor (140) may lower the frame rate of the image data to 30 fps using frame skip.

[0088] Then, the processor (140) transmits the image data with the changed frame rate to the display device (200) through the second communication interface (120) (S355).

[0089] For example, the processor (140) encodes image data with a reduced frame rate using a video codec to compress the image data. Then, the processor (140) transmits the compressed image data to the display device (200) using the second communication interface (120).

[0090] In this way, according to the present disclosure, when the electronic device (100) transmits data to the display device (200), if the network quality between the electronic device (100) and the display device (200) is not good, the resolution or frame rate of the image data can be changed to reduce the amount of data transmitted to the display device (200), thereby enabling more stable data transmission.

[0091] At this time, since game content requires a faster response from the user than general content, it is important that the delay time taken for the image to be displayed by the image quality processing is minimized when the display device (200) performs image quality processing on the image data and then displays the image. Accordingly, if the content transmitted to the display device (200) is game content, the electronic device (100) changes the resolution of the image data to reduce the amount of data transmitted to the display device (200), and if the content transmitted to the display device (200) is general content, the electronic device (100) changes the frame rate of the image data to reduce the amount of data transmitted to the display device (200).

[0092] Meanwhile, data received by the electronic device (100) from the external device (300) may include image data, audio data, and metadata.

[0093] In this case, the processor (140) transmits audio data and metadata received from the external device (300) to the display device (200) through the second communication interface (120). That is, audio data and metadata are transmitted to the display device (200) together with image data.

[0094] For example, if the network quality between the electronic device (100) and the display device (200) is good, the processor (140) compresses image data using a video codec, compresses audio data using an audio codec, and transmits the compressed image data, compressed audio data, and metadata to the display device through the second communication interface.

[0095] In addition, if the network quality between the electronic device (100) and the display device (200) is not good, the processor (140) compresses image data with a changed resolution or frame rate using a video codec, compresses audio data using an audio codec, and transmits the compressed image data, compressed audio data, and metadata to the display device (200) via the second communication interface (120). In this case, the metadata may include ALLM information.

[0096] FIG. 4 is a drawing for explaining the operation of an electronic device according to one embodiment of the present disclosure.

[0097] Referring to FIG. 4, the electronic device (100) may include a preprocessing module (410), a video encoder (420), and an audio encoder (430). The components illustrated in FIG. 4 may be implemented as hardware controlled by a processor (140), or may be configurations implemented by the processor (140) executing a program or command stored in a memory (130).

[0098] The preprocessing module (410) can change the resolution or frame rate of the image data.

[0099] For example, if the content included in the data (e.g., video and audio data) received from the external device (300) is game content, the preprocessing module (410) can downscale the resolution of the video data received from the external device (300) to lower the resolution of the video data. In addition, if the content included in the data received from the external device (300) is general content, the preprocessing module (410) can lower the frame rate of the video data received from the external device (300) by using frame skip.

[0100] Video data with changed resolution or frame rate is transmitted to a video encoder (420) and encoded. The video encoder (420) compresses the video data using a video codec.

[0101] The audio encoder (430) compresses audio data received from an external device (300) using an audio codec.

[0102] Compressed image data, compressed audio data, and metadata are transmitted to the display device (200) by the second communication interface (120).

[0103] FIG. 5 is a block diagram illustrating the configuration of a display device according to one embodiment of the present disclosure.

[0104] Referring to FIG. 5, the display device (200) may include a communication interface (210), memory (220), a display (230), a speaker (240), and one or more processors (250). However, this configuration is exemplary, and it is to be understood that some components may be omitted or new components may be added in the present disclosure.

[0105] The communication interface (210) performs wireless communication with the electronic device (100) under the control of one or more processors (250). For example, the communication interface (210) communicates with the electronic device (100) using Wi-Fi.

[0106] The communication interface (210) may include a Wi-Fi communication module. The Wi-Fi communication module may include a communication circuit capable of performing data communication between the electronic device (100) and the display device (200) using a Wi-Fi communication method.

[0107] Memory (220) stores instructions, data structures, and program codes that can be read by one or more processors (250). Operations performed by one or more processors (250) can be implemented by executing instructions or codes of a program stored in memory (220).

[0108] The memory (220) may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and may include a non-volatile memory including at least one of a ROM, an EEPROM, a PROM, a magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as a RAM or SRAM.

[0109] The memory (220) stores one or more instructions and / or programs that cause the display device (200) to operate according to the present disclosure.

[0110] The display (230) displays an image under the control of one or more processors (250). The one or more processors (250) process image data received from the electronic device (100) and output the image on the screen of the display (230).

[0111] The display (230) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, the display (110) may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The display (230) may also include a driving circuit, a backlight unit, etc., which may be implemented as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. In addition, a touch sensor for detecting a touch operation may be arranged on the front of the display (230) to detect a touch input. Additionally, the display (230) may be implemented as a flat display, a curved display, a flexible display capable of folding or / and rolling, etc.

[0112] The speaker (240) outputs audio, notification sounds, voice messages, etc. under the control of one or more processors (250). For example, one or more processors (250) may process audio data received from the electronic device (100) and output audio signals through the speaker (240).

[0113] One or more processors (250) control the overall operations of the display device (200). For example, one or more processors (250) are connected to a communication interface (210), a memory (220), a display (230), and a speaker (240) to control the display device (200). In addition, one or more processors (250) execute one or more instructions of a program stored in the memory (220), thereby controlling the overall operations of the display device (200) to process received data and output image and audio signals. One or more processors (250) may be composed of one or more processors.

[0114] The one or more processors (250) may include one or more of a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator. The one or more processors (250) may control one or any combination of other components of the electronic device (100) and may perform operations related to communication or data processing.

[0115] For example, one or more processors (250) perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in memory (220).

[0116] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0117] One or more processors (250) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (250) are implemented as multicore processors, each of the multiple cores included in the multicore processor may include internal processor memory, such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0118] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0119] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0120] For convenience of explanation, in the following, one or more processors (250) are referred to as processors (250).

[0121] FIG. 6 is a flowchart illustrating an operation of a display device obtaining information about network quality according to one embodiment of the present disclosure.

[0122] The processor (250) can receive data from the electronic device (100) via the communication interface (210) (S610). In this case, the data is received by the display device (200) in packet form.

[0123] The processor (250) identifies the packet loss rate based on data received from the electronic device (100) (S620).

[0124] For example, the processor (250) can identify whether packet loss has occurred using an identifier of a packet received from the electronic device (100). For example, the identifier may be a number. If the number of packets sequentially received is not consecutive, the processor (250) can identify that packet loss has occurred and identify the number of lost packets. In addition, the processor (250) can identify a packet loss rate based on the number of lost packets and the total number of packets. The packet loss rate can be calculated by the ratio of the number of lost packets to the total number of packets. However, this is an example, and the display device (200) can also identify lost packets and the packet loss rate using various known methods.

[0125] The processor (250) transmits information on the packet loss rate to the electronic device (100) via the communication interface (210) (S630). The electronic device (100) identifies the network quality between the electronic device (100) and the display device (200) based on the information on the packet loss rate received from the display device (200).

[0126] Meanwhile, while the aforementioned example described the electronic device (100) identifying network quality, the present invention is not limited thereto. For example, the display device (200) may identify network quality based on the packet loss rate and transmit information about the network quality to the electronic device (100).

[0127] FIG. 7 is a flowchart illustrating an operation of a display device processing data received from an electronic device according to one embodiment of the present disclosure.

[0128] The processor (250) can receive data from the electronic device (100) via the communication interface (210) (S710). In this case, the data includes image data, audio data, and metadata.

[0129] The processor (250) can identify whether ALLM information exists in the data received from the electronic device (100) (S720). ALLM information is included in the metadata.

[0130] If ALLM information exists in the data received from the electronic device (100) (S720-Y), the processor (250) sets the mode of the display device (200) to low-delay mode (S730).

[0131] The low-delay mode may include a mode for minimizing the delay time due to image quality processing of image data during the process in which the display device (200) performs image quality processing on image data and displays the image on the screen.

[0132] For example, if the mode of the display device (200) is a low-latency mode, the processor (250) decodes compressed image data from among the data received from the electronic device (100) to obtain image data. Then, the processor (250) changes the resolution of the image data to match the resolution of the screen of the display (230). At this time, the processor (250) may change the resolution of the image data by upscaling or downscaling the image data. For example, if the resolution of the screen of the display (230) is 4k and the resolution of the image data received from the electronic device (100) is 2k, the processor (250) may upscale the image data to change the resolution of the image data to 4k. Upscaling the image data may be expressed as upscaling the image frames.

[0133] And, if the mode of the display device (200) is a low-delay mode, the processor (250) performs minimal image quality processing on the image data.

[0134] If ALLM information exists in the metadata received from the electronic device (100), the content received from the electronic device (100) corresponds to game content. In this case, the processor (250) may perform only the minimum quality processing among the quality processing that can be performed on the display device (200) in order to minimize the delay time required for the image to be displayed on the screen due to quality processing of the image data.

[0135] For example, when performing image quality processing on image data, if other image data (e.g., when performing image quality processing on an image frame, the previous image frame or the next image frame of the image frame) is used, it may take a relatively long time to perform image quality processing on the image data. In this case, the delay time required for the image to be displayed on the screen increases due to the image quality processing. Meanwhile, since game content requires a quick response from the user, if the delay time increases, the user will not be able to properly enjoy the game content. Therefore, considering the time it takes for the display device (200) to perform image quality processing, the type of image quality processing that the display device (200) performs in a low-delay mode may be preset.

[0136] The processor (250) can display images on the screen of the display (230) using image data that has been processed for quality. The display (230) can be implemented as a display that supports a high refresh rate or a variable refresh rate (VRR). Accordingly, game images with a high frame rate or a variable frame rate can be implemented on the screen of the display (230).

[0137] Additionally, the processor (250) decodes compressed audio data from among the data received from the electronic device (100) to obtain audio data. Then, the processor (250) outputs an audio signal through the speaker (240) using the audio data.

[0138] If ALLM information does not exist in the data received from the electronic device (100) (S720-N), the processor (250) can set the mode of the display device (200) to the normal mode.

[0139] The general mode may include a mode in which the display device (200) performs various image quality processing on image data to display high-quality images.

[0140] For example, when the mode of the display device (200) is a normal mode, the processor (250) decodes compressed image data from among the data received from the electronic device (100) to obtain image data. Then, the processor (250) changes the resolution of the image data to match the resolution of the screen of the display (230). At this time, the processor (250) may change the resolution of the image data by upscaling or downscaling the image data. For example, when the resolution of the screen of the display (230) is 4k and the resolution of the image data received from the electronic device (100) is 2k, the processor (250) may upscale the image data to change the resolution of the image data to 4k.

[0141] And, if the mode of the display device (200) is a normal mode, the processor (250) performs various image quality processing on the image data.

[0142] If ALLM information does not exist in the metadata received from the electronic device (100), the content received from the electronic device (100) corresponds to general content. In the case of general content, delay time is not a significant issue. Therefore, the processor (250) performs image quality processing that can be performed on the display device (200) on the image data to display high-quality images on the screen.

[0143] For example, the processor (250) can change the frame rate of the image data based on the refresh rate of the display (230). In this case, the refresh rate of the display (230) may be the maximum refresh rate of the display (230) or a refresh rate set by a user input. The processor (250) can change the frame rate of the image data to match the refresh rate of the display (230) using MEMC (Motion Estimation Motion Compensation). For example, if the frame rate of the image data received from the electronic device (100) is 30 fps and the refresh rate of the display (230) is 60 Hz, the processor (250) changes the frame rate of the image data to 60 fps.

[0144] In addition, the processor (250) may perform various image quality processing on the image data, such as noise reduction (NR) processing, compression artifact removal processing caused by compression of the image data, and motion judder removal, in order to display a high-quality image on the screen. However, the present invention is not limited thereto, and various image quality processing that can improve the image quality of the image displayed on the screen may be applied to the image data.

[0145] The processor (250) displays an image on the screen of the display (230) using image data that has been image-processed. In addition, the processor (250) decodes compressed audio data from among the data received from the electronic device (100) to obtain audio data. Then, the processor (250) outputs an audio signal through the speaker (240) using the audio data.

[0146] In the present disclosure, when the network quality between the electronic device (100) and the display device (200) is poor, the electronic device (100) reduces the amount of data transmitted to the display device (200) to ensure stable data transmission.

[0147] At this time, if the content transmitted to the display device (200) is game content, it is important to minimize the delay time in order to provide the user with fun and dynamic game content. Therefore, the electronic device (100) can change the resolution of the game image to reduce the transmission amount. For example, if the resolution is lowered from 4k to 2k, the amount of transmitted data is reduced to less than 1 / 4. When the display device (200) receives the game image from the electronic device (100), it can change the resolution of the game image to match the screen resolution, perform only minimal image quality processing, and display the game image on the screen at the original frame rate of the game image without image quality processing related to the frame rate. Accordingly, the delay time in the display device (200) can be minimized.

[0148] Additionally, if the content transmitted to the display device (200) is general content, the electronic device (100) lowers the frame rate of the image to reduce the amount of data transmitted. For example, if the frame rate is lowered from 60 fps to 30 fps, the amount of data transmitted can be reduced by half. When the display device (200) receives general images from the electronic device (100), it changes the resolution of the images to match the screen resolution and performs image quality processing and various image quality processing related to the frame rate to display high-quality images on the screen.

[0149] FIG. 8 is a drawing for explaining the operation of a display device according to one embodiment of the present disclosure.

[0150] Referring to FIG. 8, the display device (200) may include a video decoder (810), a scaler (820), a picture quality processing module (830), and an audio decoder (840). The components illustrated in FIG. 8 may be implemented as hardware controlled by a processor (250), or may be configurations implemented by the processor (250) executing a program or command stored in a memory (220).

[0151] The video decoder (810) decodes compressed image data received from the electronic device (100) to obtain image data.

[0152] The scaler (820) can change the resolution of the image data to match the resolution of the screen of the display (230). For example, the scaler (820) changes the resolution of the image data by upscaling or downscaling the resolution of the image data.

[0153] The image quality processing module (830) performs image quality processing on image data.

[0154] For example, if the data received from the electronic device (100) includes game content, the image quality processing module (830) performs the minimum image quality processing among the image quality processing that can be performed by the image quality processing module (830). In addition, if the data received from the electronic device (100) includes general content, the image quality processing module (830) performs the image quality processing that can be performed by the image quality processing module (830). The image quality-processed image data is transmitted to the display (230), and the display (230) can display an image on the screen of the display (230) using the image quality-processed image data.

[0155] An audio decoder (840) decodes compressed audio data received from an electronic device (100) to obtain audio data. The audio data is transmitted to a speaker (240), and the speaker (240) outputs an audio signal using the audio data.

[0156] FIG. 9 is a flowchart illustrating a data transmission method of an electronic device that wirelessly communicates with a display device according to one embodiment of the present disclosure.

[0157] Receives image data from an external device (S910).

[0158] The type of content included in the image data is identified based on information received from an external device (S920).

[0159] If the network quality between the electronic device and the display device is not good, the resolution or frame rate of the video data is changed based on the type of content (S930).

[0160] Image data with changed resolution or frame rate is transmitted to the display device (S940).

[0161] Step S920 identifies the type of content based on ALLM information received from an external device.

[0162] Additionally, the content included in the video data may be game content or general content.

[0163] Step S930 changes the resolution of the video data by downscaling the video data if the content is game content, and changes the frame rate of the video data by using frame skipping if the content is general content.

[0164] In this case, if the content is game content, the frame rate of the video data may not change, and if the content is the above general content, the resolution of the video data may not change.

[0165] Additionally, data received from an external device may include video data, audio data, and metadata. Step S940 may compress video data with a changed resolution or frame rate using a video codec, compress audio data using an audio codec, and transmit the compressed video data, compressed audio data, and metadata to the display device if the network quality between the electronic device and the display device is not good. In this case, the metadata may include ALLM information.

[0166] Additionally, the data received from the external device includes video data, audio data, and metadata. If the network quality between the electronic device and the display device is good, the data transmission method may further include the steps of compressing the video data using a video codec, compressing the audio data using an audio codec, and transmitting the compressed video data, compressed audio data, and metadata to the display device. In this case, the metadata may include ALLM information.

[0167] Additionally, the data transmission method further includes a step of receiving information on network quality between the electronic device and the display device from the display device and a step of identifying the network quality between the electronic device and the display device based on the received information.

[0168] In this case, information about network quality may include packet loss rates.

[0169] Meanwhile, the various embodiments described above may be implemented in a computer-readable recording medium or similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments, such as the procedures and functions described herein, may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0170] Meanwhile, computer instructions for performing processing operations of an electronic device according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When the computer instructions stored in such a non-transitory computer-readable medium are executed by a processor of a specific device, the computer instructions cause the specific device to perform processing operations in a modular display device (100) according to various embodiments described above.

[0171] A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0172] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

[0173] All features described in this specification may be combined in any combination with the features described herein.

Claims

1. In electronic devices, First communication interface; a second communication interface for wirelessly communicating with the display device; and Receives image data from an external device through the first communication interface, Identifying the type of content included in the image data based on information received from the external device through the first communication interface; Based on the network quality between the electronic device and the display device, changing the resolution or frame rate of the image data based on the type of the content, An electronic device comprising one or more processors that transmit image data having a changed resolution or frame rate to the display device through the second communication interface.

2. In paragraph 1, One or more of the above processors, An electronic device that identifies the type of content based on ALLM (Auto Low Latency Mode) information received from the external device through the first communication interface.

3. Or in paragraph 2, The content included in the above video data is an electronic device that is game content or general content.

4. In paragraph 3, One or more of the above processors, If the above content is the above game content, the resolution of the above video data is changed by downscaling the above video data, An electronic device that changes the frame rate of the video data by using frame skipping if the above content is the above general content.

5. In paragraph 3 or 4, If the above content is the above game content, the frame rate of the video data is not changed. An electronic device in which the resolution of the image data does not change if the above content is the above general content.

6. In the previous claims, The data received from the external device includes the image data, audio data and metadata, One or more of the above processors, If the network quality between the electronic device and the display device is not good, the video data whose resolution or frame rate is changed is compressed using a video codec, the audio data is compressed using an audio codec, and the compressed video data, the compressed audio data, and the metadata are transmitted to the display device through the second communication interface. The above metadata is an electronic device containing ALLM information.

7. In the previous claims, The data received from the external device includes the image data, audio data and metadata, One or more of the above processors, If the network quality between the electronic device and the display device is good, the image data is compressed using a video codec, the audio data is compressed using an audio codec, and the compressed image data, the compressed audio data, and the metadata are transmitted to the display device through the second communication interface. The above metadata is an electronic device containing ALLM information.

8. In the previous claims, One or more of the above processors, An electronic device that receives information about network quality between the electronic device and the display device from the display device through the second communication interface, and identifies the network quality between the electronic device and the display device based on the received information.

9. In paragraph 8, Information about the above network quality, electronic devices including packet loss rate.

10. A method for transmitting data between a display device and an electronic device that communicates wirelessly, A step of receiving image data from an external device; A step of identifying the type of content included in the image data based on information received from the external device; A step of changing the resolution or frame rate of the image data based on the type of the content, based on the network quality between the electronic device and the display device; and A data transmission method comprising: a step of transmitting image data with changed resolution or frame rate to the display device.

11. In paragraph 10, The above identifying step is, A data transmission method for identifying the type of content based on ALLM (Auto Low Latency Mode) information received from the external device.

12. In clause 10 or 11, The content included in the above video data is a data transmission method that is game content or general content.

13. In paragraph 12, The above changing steps are: If the content is the game content, a step of changing the resolution of the video data by downscaling the video data; and A data transmission method comprising: a step of changing the frame rate of the video data using frame skipping if the content is the general content.

14. In paragraph 12 or 13, If the above content is the above game content, the frame rate of the video data is not changed. A data transmission method in which the resolution of the image data is not changed if the above content is the above general content.

15. In any one of clauses 10 to 14, The data received from the external device includes the image data, audio data and metadata, The above transmitting step is, If the network quality between the electronic device and the display device is not good, a step of compressing the image data whose resolution or frame rate is changed using a video codec and compressing the audio data using an audio codec; and A step of transmitting the compressed image data, the compressed audio data and the metadata to the display device; The above metadata is a data transmission method including ALLM information.

Images