Electronic device, display device, and control method therefor

Abstract

This electronic device comprises: an input / output interface that receives uncompressed digital image content; a communication device that wirelessly transmits and receives data; at least one memory storing instructions; and at least one processor that obtains packet data corresponding to the digital image content and controls the communication device to transmit the packet data. The at least one processor executes the instructions to: separate video information, audio information, and meta information in the digital image content; in order to obtain encoded video information, perform encoding in units of preset blocks by using video information in units of preset lines among the video information; in order to obtain packet data, packetize each of the encoded video information, the audio information, and the meta information; and control the communication device to wirelessly transmit the packet data to a display device.

Description

Electronic devices, display devices and control methods thereof

[0001] The present disclosure relates to an electronic device, a display device, and a control method thereof, and more particularly, to an electronic device, a display device, and a control method thereof capable of wirelessly transmitting streaming data received through a wire.

[0002] The display device can process and display digital or analog video signals received from an external source or various video signals stored in compressed files of various formats in an internal storage device.

[0003] Meanwhile, the display device supports an HDMI terminal to receive various digital images, and receives image data from external devices (e.g., set-top boxes), game consoles, etc. through the HDMI terminal.

[0004] However, since this type of wired connection requires a separate cable and does not satisfy the aesthetic aspects of the display device, a method that can be connected wirelessly was required.

[0005] An electronic device according to one or more embodiments of the present disclosure includes an input / output interface for receiving uncompressed digital image content, a communication device for transmitting and receiving data wirelessly, at least one memory for storing instructions, and at least one processor for obtaining packet data corresponding to the digital image content and controlling the communication device to transmit the packet data, wherein the at least one processor executes the instructions for controlling the communication device to separate video information, audio information, and meta information from the digital image content, perform encoding in units of preset blocks using video information in units of preset lines among the video information to obtain encoded video information, packetize each of the encoded video information, the audio information, and the meta information to obtain the packet data, and transmit the packet data to a display device wirelessly.

[0006] The at least one processor may execute the instruction to perform an encoding operation using the buffered video information in units of preset lines when video information in units of preset lines among a plurality of video information constituting one frame in the digital image content is buffered.

[0007] The at least one processor may execute the instruction to, when encoding of one frame in the digital image content is completed, store information about the frame in the at least one memory, and perform an encoding operation using the video information of the preset line unit and the frame information corresponding to the preset line unit of the stored previous frame.

[0008] The at least one processor may include a plurality of encoding cores that perform encoding in preset block units for a plurality of different lines.

[0009] The at least one processor may execute the instruction to perform encoding at a bit rate based on the complexity in the input video information during the process of performing encoding in a preset block unit.

[0010] The at least one processor may execute the instruction to separate data other than a preset section within a signal received from the input / output interface into image information, and to separate meta information and audio information by checking the packet type of the signal within the preset section.

[0011] The at least one processor may execute the instruction to obtain a frame index based on a synchronization signal in a signal received from the input / output interface, and to add a header including the frame index to each of the encoded video information, the audio information, and the meta information.

[0012] The at least one processor may execute the instructions to merge a predetermined number of audio samples to obtain merged audio samples and an audio payload based on audio information including audio samples and headers corresponding to the audio samples, obtain a header corresponding to the audio payload using the header of each of the merged audio samples, and obtain a wireless audio packet by combining the audio payload and the header.

[0013] The at least one processor may execute the instruction to obtain a header corresponding to the audio payload, the header including frame index information corresponding to a location of the audio information.

[0014] The at least one processor may execute the instructions to merge the meta information in units of frames to obtain a meta payload, obtain a meta header including frame index information corresponding to the meta information, and combine the meta header and the meta payload to obtain a wireless meta packet.

[0015] The at least one processor may execute the instruction to obtain packet data including one of the encoded video information, the audio information, and the meta information and classification information indicating the type of data included in the data area.

[0016] The at least one processor may execute the instruction to control the communication device to transmit packet data corresponding to meta information among packet data with the highest priority.

[0017] Meanwhile, a method for controlling an electronic device using at least one processor according to one or more embodiments of the present disclosure includes a step of separating video information, audio information, and meta information in uncompressed digital image content, a step of performing encoding in a predetermined block unit using video information in a predetermined line unit among the video information to obtain encoded video information, a step of packetizing each of the encoded video information, the audio information, and the meta information to obtain packet data, and a step of controlling a communication device to wirelessly transmit the packet data to a display device.

[0018] The step of performing the encoding may include performing an encoding operation using the buffered video information in the preset line units when the video information in the preset line units among the plurality of video information constituting one frame in the digital image content is buffered.

[0019] Meanwhile, in a non-transitory computer-readable recording medium storing a program for executing a control method of an electronic device according to one or more embodiments of the present disclosure in at least one processor, the control method includes a step of separating video information, audio information, and meta information in uncompressed digital image content, a step of performing encoding in a preset block unit using video information in a preset line unit among the video information to obtain encoded video information, a step of packetizing each of the encoded video information, the audio information, and the meta information to obtain packet data, and a step of controlling a communication device to wirelessly transmit the packet data to a display device.

[0020] The above control method can perform an encoding operation using the video information of the preset line units buffered among a plurality of video information constituting one frame in the digital image content.

[0021] The control method may further include, when encoding of one frame in the digital image content is completed, a step of storing information about the frame in the at least one memory, and a step of performing an encoding operation using video information of the preset line unit and frame information corresponding to the preset line unit of the stored previous frame.

[0022] The above control method can perform encoding at a bit rate based on the complexity within the input video information during the process of performing encoding in a preset block unit.

[0023] The above control method may further include a step of separating data other than a preset section within a signal received from the input / output interface into image information, and a step of confirming a packet type of a signal within the preset section to separate meta information and audio information.

[0024] The control method may further include a step of obtaining a frame index based on a synchronization signal in a signal received from the input / output interface, and a step of adding a header including the frame index to each of the encoded video information, the audio information, and the meta information.

[0025] The above and other aspects, features, and advantages of the embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings. In the accompanying drawings:

[0026] FIG. 1 is a drawing for explaining the operation of an electronic device according to one or more embodiments of the present disclosure;

[0027] FIG. 2 is a block diagram illustrating a configuration of an electronic device according to one or more embodiments of the present disclosure;

[0028] FIG. 3 is a block diagram illustrating a configuration of a display device according to one or more embodiments of the present disclosure;

[0029] FIG. 4 is a block diagram illustrating a configuration of a display device according to one or more embodiments of the present disclosure;

[0030] FIG. 5 is a timing diagram of an HDMI signal according to one or more embodiments of the present disclosure;

[0031] FIG. 6 is a drawing for explaining an example of arrangement of metadata in an HDMI signal according to one or more embodiments of the present disclosure;

[0032] FIG. 7 is a drawing for explaining a method for wirelessly transmitting an HDMI signal according to one or more embodiments of the present disclosure;

[0033] FIG. 8 is a drawing for explaining a wireless transmission system for an image according to one or more embodiments of the present disclosure;

[0034] FIG. 9 is a diagram for explaining a processing method for each HDMI signal attribute according to one or more embodiments of the present disclosure;

[0035] FIG. 10 is a diagram illustrating an image encoding operation according to one or more embodiments of the present disclosure;

[0036] FIG. 11 is a drawing for explaining a line buffering operation according to one or more embodiments of the present disclosure;

[0037] FIG. 12 is a diagram illustrating a bit rate operation according to one or more embodiments of the present disclosure;

[0038] FIG. 13 is a diagram for explaining an operation of processing audio data according to one or more embodiments of the present disclosure;

[0039] FIG. 14 is a diagram for explaining the operation of a transmission buffer according to one or more embodiments of the present disclosure;

[0040] FIG. 15 is a drawing for explaining the operation of generating a control signal according to the present disclosure;

[0041] FIG. 16 is a diagram illustrating an example of a transmission packet according to one or more embodiments of the present disclosure;

[0042] FIG. 17 is a drawing for explaining a receiving operation according to one or more embodiments of the present disclosure;

[0043] FIG. 18 is a drawing for explaining the operation of generating a synchronization signal according to one or more embodiments of the present disclosure;

[0044] FIG. 19 is a flowchart for explaining a control operation of an electronic device according to one or more embodiments of the present disclosure, and

[0045] FIG. 20 is a flowchart illustrating a display method according to one or more embodiments of the present disclosure.

[0046] The present embodiments may be modified and have various embodiments, and specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the scope to specific embodiments, and it should be understood that various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure are included. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0047] In describing the present disclosure, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

[0048] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to further faithfully and completely convey the technical concepts of the present disclosure to those skilled in the art.

[0049] The terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the scope of the rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0050] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0051] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to instances where (1) at least one A is included, (2) at least one B is included, or (3) at least one A and at least one B are included.

[0052] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0053] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that said component may be directly coupled to said other component, or may be coupled via another component (e.g., a third component).

[0054] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between said component and said other component.

[0055] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0056] Instead, in some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0057] Elements described as 'modules' or 'parts', 'sub-components' or 'devices' may be physically implemented by analog and / or digital circuits including one or more of logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components and active electronic components.

[0058] According to various embodiments, operations performed by a module, program or other component may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0059] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0060] Meanwhile, a display device according to various embodiments of the present disclosure may include, for example, at least one of a smartphone, a tablet PC, a desktop PC, a laptop PC, and a projector.

[0061] In some embodiments, the display device may include at least one of, for example, a television, a digital video disk (DVD) player, an audio device, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a robot vacuum cleaner, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync™, Apple TV™, or Google TV™), a game console (e.g., Xbox™, PlayStation™), an electronic dictionary, an electronic key, a camcorder, or an electronic picture frame. The above-described device examples are examples, and any device including a display may be used.

[0062] Hereinafter, with reference to the attached drawings, embodiments according to the present disclosure will be described in detail so that a person having ordinary knowledge in the technical field to which the present disclosure pertains can easily implement the present disclosure.

[0063] FIG. 1 is a drawing for explaining the operation of an electronic device according to one or more embodiments of the present disclosure.

[0064] Referring to FIG. 1, a display system (1000) may include a source device (10), an electronic device (100), and a display device (200).

[0065] The source device (10) can output digital image content in a preset interface manner. Here, the digital image content may be movie content, music content, game content, or a web page, etc. In addition, the preset interface manner may be a method for inputting and outputting image (and audio) data in an uncompressed manner through a wire, for example, HDMI (High Definition Multimedia Interface). In the above, the above-described interface manner is set assuming HDMI, but the above-described interface manner may be DisplayPort, MHL (Mobile High-Definition Link), Thunderbolt, UTP, SATA, USB-C, etc., as long as it can input and output image data (or image content) in an uncompressed manner other than HDMI.

[0066] Such a source device (10) may be a set-top box, game console, home theater, PC, laptop, etc. that receives a broadcast signal from an external source, such as a wired signal or satellite signal. In addition to the above-described device, any device that can transmit digital image content using the above-described interface method may be used.

[0067] Here, the video content refers to a case where it includes video data, audio data, and metadata in accordance with the HDMI standard, as shown in FIG. 5. However, the HDMI standard may not include audio data and may only include video data, and thus it refers to a case where it is composed of video data, audio data, and metadata, a case where it is composed of video data and metadata, a case where it is composed of audio data and metadata, etc.

[0068] In the following, for ease of explanation, the case is assumed to include all three types of data (i.e., image data, audio data, and metadata), but one of the above-described types of data may be omitted during implementation.

[0069] For simplicity, the content transmitted via the aforementioned interface method is referred to as "video content" in the following description. Such video content may also be referred to as an HDMI signal, streaming data, streaming signal, high-resolution video signal, or high-resolution video content.

[0070] And data related to the video within the video content described above is referred to as video data. Such video data may also be referred to as frame data, screen data, screen signal, video signal, frame signal, pixel signal, pixel information, frame information, screen information, video information, video information, etc.

[0071] The electronic device (100) receives video content via a wired connection and transmits the video content wirelessly. For example, the electronic device (100) can receive uncompressed video content via the interface described above, separate data components within the input video content, compress the video data, and transmit the data via a commercially available wireless communication method. The specific configuration and operation of the electronic device (100) will be described below with reference to FIG. 2.

[0072] Here, the wireless communication method can be a high-speed wireless communication method. The latest HDMI cables can transmit 4K or 8K resolution video, supporting transmission speeds of 16 Gbps or higher (or 48 Gbps).

[0073] Therefore, using a wireless communication method that supports the aforementioned wired transmission speeds allows for easier transmission of the aforementioned video content. However, the currently widely used WiFi method has a communication speed similar to, or in some cases, insufficient for, the aforementioned transmission speeds.

[0074] In this regard, the present disclosure utilizes image compression technology to transmit image content output via the aforementioned high-speed interface method, even using currently used wireless communication methods. This image compression technology is described below with reference to FIGS. 10 to 12.

[0075] In addition, as previously explained, the above-described interface method transmits not only video but also audio signals and metadata, and therefore the data format and method required for transmitting the above-described video content are described in detail below.

[0076] The display device (200) can receive data transmitted wirelessly, restore image data, audio data, and metadata using the received data, and display an image using the restored image data. The specific configuration and operation of the display device (200) will be described later with reference to FIGS. 3 and 4.

[0077] As described above, the display system according to the present disclosure can wirelessly transmit video content that would normally be transmitted and received via wired channels, eliminating the need for wired cables. This can enhance the portability and aesthetics of the display device. Furthermore, by compressing the content for transmission, high-speed video data transmission and reception can be achieved using currently commonly used wireless communication methods.

[0078] Meanwhile, although the above description illustrates receiving digital video content and transmitting and receiving it wirelessly, it is also possible to receive analog content, convert it into digital format, and transmit it. In this case, the source device (100) and / or the electronic device (100) may support a VGA (Video Graphics Array) port, an RGB port, a D-SUB (D-subminiature), etc.

[0079] FIG. 2 is a block diagram illustrating a configuration of an electronic device according to one or more embodiments of the present disclosure.

[0080] Referring to FIG. 2, the electronic device (100) may include an input / output interface (110), a communication interface (or communication device) (120), (at least one) memory (130), and (at least one) processor (140).

[0081] The input / output interface (110) receives uncompressed digital video content. For example, the above-described input / output interface may be any one of HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), and DVI (Digital Visual Interface).

[0082] The electronic device (100) may be equipped with different types of input / output interfaces, and may receive digital image content through one of them and power for driving the electronic device (100) through the other. For example, the electronic device (100) may receive image content through HDMI and power through USB. Alternatively, the electronic device (100) may receive both the image content and power described above through a single interface (e.g., HDMI).

[0083] The communication device (120) can transmit and receive data wirelessly. Specifically, the communication device (120) is configured to connect to at least one external device (e.g., a display device) or the Internet, and can support wireless communication.

[0084] These communication devices (120) may include a Wi-Fi module, a Bluetooth module, a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.

[0085] Wi-Fi and Bluetooth modules can communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, connection information, such as the SSID and session key, is first transmitted and received. This information is then used to establish a connection and then transmit and receive various information.

[0086] In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0087] In addition, the communication device (120) may include an infrared communication module, etc., and may receive control commands for the electronic device (100) through the above-described infrared communication module, Wi-Fi module, Bluetooth module, etc. For example, operations such as stopping transmission of an image, starting transmission of an image, and selecting a display device to transmit an image may be received. Meanwhile, during implementation, the above-described control commands may be received through an input device included in the electronic device (100).

[0088] The communication device (120) can transmit a network packet generated by the process described below using a preset wireless transmission protocol. This is described later in FIG. 17.

[0089] The memory (130) can store received video content. In addition, the memory (130) can store compressed video information, transmission packets, etc., as well as video information, audio information, and meta information temporarily generated in the process described below.

[0090] The memory (130) may be implemented in the form of memory embedded in the electronic device (100) or may be implemented in the form of memory that can be attached or detached from the electronic device (100) depending on the purpose of data storage. For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in a memory that can be attached or detached from the electronic device (100).

[0091] Meanwhile, the memory embedded in the electronic device (100) may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)).

[0092] In addition, the memory that can be attached to the electronic device (100) may be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), an external memory that can be connected to a USB port (e.g., USB memory), etc.

[0093] The memory (130) may include various instructions necessary for the operation of the processor (140). Here, the instructions may include instructions related to overall operation control of the electronic device (100) according to a user's control command, instructions for separating data by type from received image content, instructions for compressing image data, instructions for packetizing data according to a wireless transmission method, etc.

[0094] The processor (140) controls the overall operation of the electronic device (100). Specifically, the processor (140) is connected to each component of the electronic device (100) (e.g., input / output interface (110), communication device (120), and memory (130)) and can control the overall operation of the electronic device (100).

[0095] The processor (140) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) for processing digital signals. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics-processing unit (GPU), a communication processor (CP), or an ARM processor, or may be defined by the relevant term. In addition, at least one processor (140) may be implemented as a system on chip (SoC) or large scale integration (LSI) having a built-in processing algorithm, or may be implemented in the form of a field programmable gate array (FPGA). In addition, at least one processor (140) may perform various functions by executing computer executable instructions stored in a memory. Meanwhile, at least one processor (140) may be implemented as a plurality of processors (for example, a CPU + It may also include DSP).

[0096] The processor (140) can classify the received packet data corresponding to the received video content into one of video information, audio information, and meta information. For example, the processor (140) can separate data outside of a preset section within a signal received from the input / output interface (110) into video information, and can separate meta information and audio information by checking the packet type of the signal within the preset section.

[0097] And the processor (140) can encode separated image information. Specifically, the processor (140) can perform encoding of a preset block unit by using video information of a preset line unit among the video information.

[0098] At this time, when video information of a preset line unit among a plurality of video information constituting one frame in digital image content is buffered, the processor (140) can perform an encoding operation using the buffered video information of a preset line unit.

[0099] Meanwhile, the processor (140) can determine whether to encode the frame in the form of a P frame or an I frame for each preset frame unit. This determination can be performed in preset cycle units. For example, the first frame may be an I frame, and the next two frames may be P frames in a repeating order.

[0100] When it is decided to encode video information in the form of an I frame, the processor (140) can store information about that frame in memory so that it can be used later in the encoding process of another P frame.

[0101] When it is determined to encode video information in the form of a P frame, the processor (140) can perform an encoding operation using video information in a preset line unit and frame information corresponding to a preset line unit of a stored previous frame.

[0102] Meanwhile, the above-described encoding operation can be performed using a plurality of encoding cores, and in this case, the processor (140) can control the plurality of encoding cores to perform encoding in preset block units for a plurality of different lines.

[0103] Additionally, during the encoding operation described above, the processor (140) may perform encoding at a bit rate based on the complexity within the input video information during the process of performing encoding in a preset block unit.

[0104] And the processor (140) can generate a frame index based on a synchronization signal within a signal received from an input / output interface.

[0105] And the processor (140) can store in memory a header including a frame index to each of the encoded video information, audio information, and meta information. The memory can be divided into multiple storage areas, and the video information, audio information, and meta information can be stored in different storage areas.

[0106] And the processor (140) generates packet data corresponding to compressed image information, audio information, and meta information in a wireless manner.

[0107] For example, if audio information includes audio samples and headers corresponding to the audio samples, the processor (140) can merge a preset number of audio samples to generate an audio payload, use the headers of each of the merged audio samples to generate a header corresponding to the audio payload, and combine the audio payload and the header to generate a wireless audio packet.

[0108] At this time, the processor (140) can generate a header corresponding to the audio payload to include frame index information corresponding to the location of the audio information.

[0109] And for the meta information, the processor (140) can generate a meta payload by merging the separated meta information in units of frames, generate a meta header including frame index information corresponding to the meta information, and generate a wireless meta packet by combining the meta header and the meta payload.

[0110] The processor (140) can control the communication device (120) to transmit the generated packet data. First, the processor (140) can generate packet data including one of encoded data, audio data, and metadata and identification information indicating the type of data included in the data area, and can attach a header to the packet data to generate a network packet. Then, the processor (140) can control the communication device (120) to transmit the generated network packet.

[0111] At this time, the processor (140) can control the communication device to transmit packet data corresponding to meta information among packet data with the highest priority. In addition, the processor (140) can control the communication device (120) to transmit audio information with a higher priority than video information. In other words, the processor (140) can perform the transmission operation in the priority order of meta information > audio information > video information.

[0112] As described above, the electronic device according to the present disclosure can wirelessly transmit video content that would normally be transmitted and received via wired channels, eliminating the need for wired cables. Furthermore, by compressing the content for transmission, high-speed video data transmission and reception can be achieved using currently commonly used wireless communication methods.

[0113] FIG. 3 is a block diagram illustrating a configuration of a display device according to one or more embodiments of the present disclosure.

[0114] Referring to FIG. 3, the display device (200) may include a communication interface (communication device) (210), a memory (220), a display (230), and a processor (240).

[0115] The communication device (210) is configured to connect to at least one external device (e.g., an electronic device (100)) or the Internet, and can be connected wirelessly. The communication device (210) can receive packet data corresponding to video content from the external device.

[0116] The communication device (210) can transmit and receive data wirelessly. Specifically, the communication device (210) is configured to connect to at least one external device (e.g., an electronic device) or the Internet, and can support wireless communication.

[0117] These communication devices (210) may include a Wi-Fi module, a Bluetooth module, a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.

[0118] Wi-Fi and Bluetooth modules can communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, connection information, such as the SSID and session key, is first transmitted and received. This information is then used to establish a connection and then transmit and receive various information.

[0119] In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0120] In addition, the communication device (210) may include a wired communication module (e.g., LAN), an infrared communication module, etc. In addition, the communication device (210) may receive control commands for the display device (200) from a user (or a remote control, a user terminal device) through the above-described infrared communication module, Wi-Fi module, Bluetooth module, etc. Meanwhile, during implementation, the above-described control commands may be received through an input device included in the display device (200).

[0121] A communication device (210) can wirelessly receive network packets related to video content. These network packets can be divided into a header region and a data region, and the data region can contain video information, audio information, or meta information. These network packets can be UDP packets.

[0122] The memory (220) can store data required for various embodiments. Depending on the purpose of data storage, the memory (220) may be implemented in the form of memory embedded in the display device (200) or may be implemented in the form of memory that can be attached or detached to the display device (200).

[0123] For example, data for driving the display device (200) may be stored in a memory embedded in the display device (200), and data for the extended functions of the display device (200) may be stored in a memory that can be detachably attached to the display device (200).

[0124] Meanwhile, the memory embedded in the display device (200) may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)).

[0125] In addition, the memory that can be detached from the display device (200) can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), an external memory that can be connected to a USB port (e.g., USB memory), etc.

[0126] The memory (220) may include various instructions necessary for the operation of the processor (240). For example, the above-described instructions may store instructions for controlling general control operations of the display device (200), instructions for restoring image data, audio data, and metadata using the received data when receiving content in a wireless manner as in the present disclosure, instructions for decoding using the received data in accordance with the image encoding method of the electronic device (100), etc.

[0127] The memory (220) can store various contents. Specifically, it can store data received through a communication device (210), and can store image data, audio data, metadata, etc. generated in the process described below.

[0128] The display (230) displays an image. The display (230) can be implemented as an LCD (Liquid Crystal Display), an LED (Light-Emitting Diode), an AMOLED (Active Matrix Organic Light-Emitting Diode), a CRT (Cathode-ray tube), etc.

[0129] The processor (240) controls the overall operation of the display device (200). Specifically, the processor (240) is connected to each component of the display device (200) (e.g., communication device (210), memory (220), display (230)) and can control the overall operation of the display device.

[0130] The processor (240) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes a digital signal. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics-processing unit (GPU), a communication processor (CP), or an ARM processor, or may be defined by the relevant term. In addition, at least one processor (240) may be implemented as a system on chip (SoC) or large scale integration (LSI) having a built-in processing algorithm, or may be implemented in the form of a field programmable gate array (FPGA). In addition, at least one processor (240) may perform various functions by executing computer executable instructions stored in a memory. Meanwhile, at least one processor (240) may be implemented as a plurality of processors (for example, a CPU + It may also include DSP).

[0131] When the processor (240) receives data through the communication device (210), it can restore image data, voice data, and metadata using the received data.

[0132] Specifically, the processor (240) can use the classification information within the received network packet to classify the information within the received network packet into video data, audio data, and metadata. In addition, the processor (240) can store each of the classified data in a memory area corresponding to each data type.

[0133] At this time, the processor (240) can remove the classification information indicating the type of data within the header and data of the received network packet from the received network packet and store it in memory.

[0134] And, if the image is compressed, the processor (240) can perform decoding in a manner corresponding to the compression method. Specifically, the processor (240) can perform decoding in a preset block unit using video data in a preset line unit among the video data.

[0135] And if the video compression method is a method using P frames, the processor (240) can store the generated frame image in memory as a previous frame image, and perform decoding using the video data of a preset line unit and the image information corresponding to the video data of the previous frame during the decoding process.

[0136] At this time, the processor (240) may generate a synchronization signal to synchronize the image and metadata, and output the frame image and metadata corresponding to the frame index of the frame image based on the generated synchronization signal.

[0137] And the processor (240) can control the display (230) so that the generated image is displayed.

[0138] Although only a brief configuration of the display device (200) has been described above, the display device (200) may further include a configuration as illustrated in FIG. 4. The specific configuration of the display device (200) will be described below with reference to FIG. 4.

[0139] FIG. 4 is a block diagram illustrating a configuration of a display device according to one or more embodiments of the present disclosure.

[0140] Referring to FIG. 4, the display device (200) may include a broadcast receiving unit (201), a signal separation unit (202), an A / V processing unit (203), an audio output unit (204), a GUI generation unit (205), a communication device (210), a memory (220), a display (230), a processor (240), an operation interface (250), and a microphone (260).

[0141] The configuration of the communication device (210), memory (220), display (230), and processor (240) of FIG. 4 is the same as that of FIG. 3, so overlapping descriptions are omitted and only the parts that are different from the above-described contents are additionally described.

[0142] The broadcast receiving unit (201) receives and demodulates a broadcast from a broadcasting station or satellite via wire or wireless.

[0143] The signal separation unit (202) separates the broadcast signal into a video signal, an audio signal, and an additional information signal. Meanwhile, in FIG. 3, the processor (240) is illustrated and described as separating data received through wireless communication into a video signal, an audio signal, and an additional information signal. However, when implemented, the signal separation unit (202) described above may perform the above-described operation, and when implemented, two signal separation units may exist in the display device (200). For example, one signal separation unit may separate various signals from a broadcast signal, and the other signal separation unit may be configured to separate video data, audio data, and metadata from data received through wireless communication.

[0144] And the signal separation unit (202) transmits the video signal and audio signal to the A / V processing unit (203).

[0145] The A / V processing unit (203) performs signal processing such as video decoding, video scaling, and audio decoding on the video signals and audio signals transmitted from the broadcast receiving unit (201), the communication device (210), and the memory (220). Specifically, video data and audio data received through the communication device described above and restored as video data can be input to the A / V processing unit (203). Meanwhile, although the processor (240) has been described above as performing decoding on the received data, the A / V processing unit (203) described above may perform the decoding upon implementation. In addition, although the A / V processing unit (203) has been described above as being a separate component distinct from the processor (240), upon implementation, it may be a module or processor core within the processor (140).

[0146] And the A / V processing unit (203) outputs a video signal to the GUI generation unit (205) and outputs an audio signal to the audio output unit (204).

[0147] On the other hand, when storing the received video and audio signals in the memory (220), the A / V processing unit (203) can output the video and audio to the memory (220) in a compressed form.

[0148] The audio output unit (204) converts the audio signal output from the A / V processing unit (203) into sound and outputs it through a speaker or to an external device (e.g., external speaker, sound bar) connected through an external output terminal.

[0149] The GUI generation unit (205) generates a GUI (Graphical User Interface) to be provided to the user. Here, the GUI may include at least one of the currently output channel information, source information (e.g., information indicating whether the content is a broadcast signal, received via wireless communication, or received via a wired interface), and volume information.

[0150] And the GUI generation unit (205) can add the generated GUI to the image output from the A / V processing unit (203). And the GUI generation unit (205) provides the image with the GUI added to the display (160).

[0151] The operation interface (250) is implemented as a touch screen, touchpad, key button, keypad, etc., and provides user operation of the display device (200). This operation interface (250) can be provided in the display device (200) and can receive user operation from an external remote control.

[0152] The microphone (260) can receive the user's voice when activated. For example, the microphone (260) can be formed integrally on the upper side, front side, side side, etc. of the display device (200). The microphone (260) can include various components such as a microphone that collects the user's voice in analog form, an amplifier circuit that amplifies the collected user's voice, an A / D conversion circuit that samples the amplified user's voice and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0153] When a user's voice is input through the microphone (260), the processor (240) can confirm the user's voice content and perform an operation corresponding to the voice content. Meanwhile, although the user's voice is input through the microphone (260) as described above, the microphone may be provided in a remote control for controlling the display device (200), and the user's voice input through the microphone provided in the remote control may be input and processed through the above-described communication device (210).

[0154] Additionally, the electronic device (200) may operate not only based on the configuration or remote control provided in the display device (200), but also according to control commands from a user terminal device. Such an application may provide a function that enables the user terminal device to be used as a remote control for the electronic device.

[0155] Fig. 5 is a timing diagram of an HDMI signal, and Fig. 6 is a drawing for explaining an example of arrangement of metadata in an HDMI signal.

[0156] Referring to FIGS. 5 and 6, an HDMI signal transmits a video signal (or video packet) in a Video Data Period. Specifically, the video data includes information about video pixels, and 8-bit video data can be converted to 10-bit and transmitted.

[0157] In the Data Island Period, audio signals and metadata are transmitted in the form of HDMI packets. These audio signals, metadata, etc. are transmitted in packet form and encoded using the TERC4 coding method, allowing 4-bit data to be converted to 10-bit data and transmitted. Additionally, InfoFrames can be transmitted in the Data Island.

[0158] Here, InfoFrames are information that conveys information such as vendor-specific information, auxiliary video information (e.g., chroma sampling, resolution, 3D format, etc.), source product description (e.g., name of video source, product type), audio (e.g., sampling rate, number of channels, encoding format, speaker mapping, etc.), MPEG source (e.g., bit rate, frame type), NTSC VBI (metadata such as subtitles), etc.

[0159] And the Control Period is the section in which control information (e.g., preamble, HYNC, VSYNC) is transmitted.

[0160] And referring to FIG. 6, it can be confirmed that the above-described data is transmitted as a data signal (610) in a digital manner in response to the trigger signal of the horizontal synchronization signal (HSYNC) (620) and the vertical synchronization signal (VSYNC) (630).

[0161] Additionally, the HDMI signal includes a vertical blanking period and a horizontal blanking period, and active video data is transmitted as information as shown in FIG. 5 at the vertical synchronization signal point and the horizontal synchronization point.

[0162] HDMI operating with these specifications supports a transmission speed of 18 Gbps for 4K video transmission in version 2.0, and a transmission speed of 48 Gbps in version 2.1.

[0163] However, WiFi only supports a throughput of 3.5 Gbps for WiFi 5 and up to 9.6 Gbps for WiFi 6. WiFi 4, currently used in real life, has a lower speed than the aforementioned throughput, limiting its ability to transmit HDMI signals directly via WiFi.

[0164] Accordingly, the present disclosure supports transmission of HDMI signals using a commonly used wireless communication method by reducing the size of data to be transmitted through encoding of the aforementioned video data. In addition, since HDMI transmits and receives not only video but also audio signals and metadata through a single cable, appropriate transmission and synchronization of not only video data but also the aforementioned audio signals and metadata are required, and the corresponding operations are described below.

[0165] FIG. 7 is a drawing for explaining a method for wirelessly transmitting an HDMI signal according to one or more embodiments of the present disclosure.

[0166] Referring to FIG. 7, the display system (1000) may include a source device (10), an electronic device (100), and a display device (200).

[0167] The source device (10) can output video content through an HDMI port. At this time, the source device (10) can output an HDMI signal in a manner that complies with the HDMI specification, i.e., in a form as illustrated in FIG. 5.

[0168] The electronic device (100) can be connected to the HDMI port (11) of the source device (10) and can receive video content through an input / output interface (110) corresponding to the HDMI port (11). In the following, for ease of explanation, the video content is expressed as being received, but the HDMI method outputs an HDMI signal in a streaming manner, and does not receive the entire content, but continuously receives an HDMI signal for the video content.

[0169] Meanwhile, although the illustrated example shows that the electronic device (100) is directly connected to the HDMI port of the source device (10), it may be connected via an HDMI cable during implementation. Furthermore, although the source device (10) and the electronic device (100) are illustrated and described as separate devices, the electronic device (100) may be implemented in a form built into the source device (10).

[0170] The electronic device (100) can separate video data, audio data, and metadata from a received HDMI signal, perform encoding on the video data, packetize the encoded video data, audio data, and metadata, and wirelessly transmit the packet data. Detailed operations and configurations for performing the above-described operations are described later in FIG. 8.

[0171] The display device (200) can receive data (or packets) transmitted wirelessly and restore image data, audio data, and metadata using the received data.

[0172] And the display device (200) can provide image data to an image processing device (241) that processes images, metadata to a device (243) that processes metadata, and audio data to an audio processing device (242). The detailed operation and configuration of the display device (200) that performs the above-described operation are described later in FIG. 8.

[0173] FIG. 8 is a diagram illustrating a wireless transmission system for an image according to one or more embodiments of the present disclosure.

[0174] Referring to FIG. 8, the wireless transmission system may include a transmitting device (300) and a receiving device (400).

[0175] The transmitting device (300) can receive video content via a wired connection and transmit data regarding the video content wirelessly. This transmitting device (300) may be the electronic device (100) described above.

[0176] The transmitting device (300) may include a transmitting control module (310), a video encoder (320), a metadata processing module (330), an audio processing module (340), a transmitting memory (350), and a wireless transmitting module (360).

[0177] The transmission control module (310) can generate a control signal for synchronization. For example, the transmission control module can generate control signals such as a meta enable signal, a video enable signal, a video enable signal, and a frame index. The generation operation of such control signals is described later in FIG. 15.

[0178] The transmission control module (310) can classify video signals (or video information, video packets), audio signals (or audio information, audio packets), and meta signals (or audio information, audio packets). This classification operation is described in detail in FIG. 9.

[0179] And the transmission control module (310) can provide the classified video signal to the video encoder (320), the audio signal to the audio processing module (340), and the meta signal to the meta data processing module (330).

[0180] The video encoder (320) can receive a video signal from the transmission control module (310) and encode the received video signal. In addition, the video encoder (320) can insert a control signal into a stream header. The specific operation of the video encoder (320) will be described below with reference to FIGS. 10 to 12. Such a video encoder may be referred to as a codec, a video compressor, a video compression module, a compression module, a codec module, etc.

[0181] The metadata processing module (330) can convert the metadata signal from the transmission control module (310) into a new format. For example, the metadata processing module (330) can collect and rearrange HDMI packets related to multiple metadata within a video frame to form a single packet. Here, the rearrangement can be performed by removing only the ECC portion of a 36-byte HDMI packet to make it 32-byte in size.

[0182] Metadata is not always transmitted for each video frame. Therefore, the metadata processing module (330) may insert a field indicating whether each rearranged data is valid or not in the reserved area of ​​the header of the generated wireless meta packet. In addition, the metadata processing module (330) may add index information to the header of the wireless meta packet to designate the location where the corresponding metadata was collected.

[0183] The operation of the metadata processing module (330) is described in detail in Fig. 14.

[0184] The audio processing module (340) can receive an audio signal from the transmission control module (310) and perform a process of converting it into a new format. The specific operation of the audio processing module (340) is described later in FIG. 13.

[0185] The transmission memory (350) can store video data, audio data, and metadata processed by each of the video encoder (320), the metadata processing module (330), and the audio processing module (340). At this time, the transmission memory (350) can have multiple storage spaces. At this time, the storage spaces can be a space for storing video data, a space for storing audio data, and a space for storing metadata.

[0186] The wireless transmission module (360) can access each memory space to read data, packetize the read data, and transmit it via wireless communication. The specific operation of the wireless transmission module (360) will be described later with reference to FIGS. 16 and 17.

[0187] The receiving device (400) can receive data wirelessly and use the received data to restore image data, audio data, and metadata. The receiving device (400) may be the display device (200) described above.

[0188] The receiving device (400) may include a receiving module (410), a video decoder (420), a metadata processing module (430), an audio processing module (440), a receiving memory (450), and a receiving control module (460).

[0189] The receiving module (410) can distinguish the received network packets and store data in the video / audio / meta areas of the receiving memory (450). Specifically, the receiving module (410) can check a distinction field indicating the type of data included in the corresponding data in the data area within the received network packet, and can use the identified distinction field to identify whether the corresponding data is video data, audio data, or meta data. Once this identification is completed, the receiving module can store the video data in the video area of ​​the receiving memory (450), store the audio data in the audio area of ​​the receiving memory (450), and store the meta data in the meta area of ​​the receiving memory (450).

[0190] Here, the network packet may be a UDP packet as described in Fig. 16, but is not limited thereto, and may also be a TCP packet, etc. In addition, the network packet may also be referred to as a wireless packet, a WiFi packet, etc.

[0191] The video decoder (420) can read the video data area of ​​the receiving memory (450), perform codec decoding, generate a video synchronization signal, and parse a control signal for synchronization. The operation of the video decoder is described below with reference to FIGS. 10 to 12. The video decoder (420) may also be referred to as a video decoder, a pixel decoder, a codec, a codec module, a decoding module, etc.

[0192] The metadata processing module (430) can read the metadata area of ​​the receiving memory (450), parse header information and metadata from a wireless metadata packet stored in the metadata area, and provide the parsed metadata to a configuration requiring metadata.

[0193] The audio processing module (440) can read the audio data area of ​​the receiving memory (450), parse the header information and audio samples in the audio packet, and provide the audio samples to a configuration (e.g., audio DSP) that processes the audio samples.

[0194] Receiving memory (450) stores each video / audio / meta data in a separate memory space

[0195] The reception control module (460) can perform video / audio / meta synchronization by using control signals to each processing unit based on the synchronization signal generated by the video processing unit. The operation of the reception control module (460) will be described later with reference to FIG. 18.

[0196] Meanwhile, such a receiving device (400) may be implemented in a dongle form without a display, and may provide wirelessly received data to a display device via a wired connection. In this case, the receiving device may be connected to an HDMI terminal of the display device, converting the restored signals into an HDMI signal format and providing them, or may be connected to a USB terminal and providing the aforementioned data as is to the display device.

[0197] FIG. 9 is a diagram for explaining a processing method for each HDMI signal attribute according to one or more embodiments of the present disclosure.

[0198] HDMI is a standard that assumes wired cables, and in order to wirelessly transmit the HDMI RX output signal, it requires the ability to distinguish, process, and store wired data signals received in real time through the cable.

[0199] To this end, the present disclosure classifies video signals (and sync signals) and HDMI Packets based on the Period of the HDMI specification, and HDMI Packets can classify audio signals and meta signals by referring to the Packet Type Value.

[0200] Specifically, referring to FIG. 9, first, it can be determined whether the current section is a Data Island Period (S910). Specifically, as described above in 5, if the current signal is not a Data Island Period section, and the signal of the corresponding section is a sync signal or a video signal, the corresponding signals can be transmitted to the video decoder (420).

[0201] If the current section is a Data Island Period, the packet type of the received signal is checked (S920), and if the packet type corresponds to an audio signal, the signal corresponding to it is transmitted to the audio processing module (340), otherwise, it can be transmitted to the metadata processing module (330). For example, according to the HDMI specification, packet types having values ​​of 0x02, ox07, Ox08, Ox09, Ox0B, Ox0C, Ox0E, OxOF are related to audio data, and thus, signals having the corresponding packet type values ​​can be transmitted to the audio processing module (340), and the remaining signals can be transmitted to the metadata processing module (330).

[0202] FIG. 10 is a diagram for explaining an image encoding operation according to one or more embodiments of the present disclosure.

[0203] Referring to FIG. 10, a video encoder (320) is positioned on the transmitting device side, and a video decoder (420) is positioned on the receiving device side.

[0204] The video encoder (320) may include a video buffer, a reference buffer, a stream buffer, and an encoding core.

[0205] The video encoder (320) can receive video pixels (i.e., video signals), video sync (horizontal synchronization signal, vertical synchronization signal) within an HDMI signal, a video enable signal, and a frame index signal from a transmission control module (310).

[0206] The video buffer stores the high-bandwidth HDMI video input signal (20) in the video buffer.

[0207] A reference buffer stores previous frames. This reference buffer can be implemented using DRAM. Specifically, this reference buffer is used to create P frames, which will be described later. If the encoding algorithm does not utilize P frames, the reference buffer can be omitted.

[0208] The encoding core can perform an encoding operation using data stored in the video buffer when the video enable signal has a high value and the vertical synchronization signal is triggered.

[0209] At this time, the encoding core can perform the encoding operation immediately after buffering only a few lines of the HDMI signal scanned in line units. For example, if the encoding processing unit of the video encoder (320) is a 16*16 block, only 16 lines can be buffered, and if 16 lines are buffered, the encoding operation can be performed.

[0210] Conventional encoders perform encoding while buffering video in frame units, but in the present disclosure, encoding is performed immediately when the line units required for encoding are buffered without waiting for the video to be stored in frame units, thereby minimizing delays that may occur due to video compression.

[0211] And the encoding core can use the information of the previous frame during the encoding operation described above to make the generated encoding data have a P frame format. Specifically, as an example of an encoding method, there is a method using a P frame, which is a technology for compressing an image by generating a P frame that only has information corresponding to the difference from the corresponding frame (or the previous frame) based on a key frame (i.e., an I frame) that has pixel information of all frames.

[0212] Therefore, the encoding core can perform encoding by periodically generating an I frame having all pixel information within the frame, and then generating a P frame after generating the I frame.

[0213] And the video encoder (320) can divide the encoded result into frames and store it in the transmission memory. At this time, the video encoder (320) can insert a frame index corresponding to the received frame index signal into the frame header.

[0214] Meanwhile, although the illustrated example shows one encoding core, the above-described encoding cores may be configured in multiples for faster encoding operations during implementation. For example, four encoding cores may be used, and if each encoding core performs encoding operations in units of 16*16 pixels, when 64 lines of image are stored in the image buffer, the four encoding cores can perform individual encoding operations using the 64 lines of image data. In this case, the encoding operations between multiple encoding cores will be described later with reference to FIG. 11.

[0215] And the encoding core can store the encoding result in the stream buffer. At this time, the encoding core can add a frame index corresponding to the current frame to the encoding result, generate a wireless video packet, and store the generated wireless video packet in the transmission memory.

[0216] The video decoder (420) may include a stream buffer, a reference buffer, a video buffer, and a decoding core.

[0217] A stream buffer can store wireless packets. This stream buffer can be the receive memory described above.

[0218] The decoding core can perform decoding using video data stored in the stream buffer. At this time, the video core can perform decoding using video data from the previous frame stored in the reference buffer. Furthermore, the decoding core can operate in a manner corresponding to the encoding core described above. That is, the decoding core can perform decoding operations in units of multiple line pixels.

[0219] The video decoded by the decoding core is stored in the video buffer and can be output in frame units.

[0220] Meanwhile, the video decoder (420) can generate a sync signal using the received video data for synchronization with metadata or audio data, and output a decoding result (30) between the generated sync signals. In addition, metadata can also be transmitted to the necessary configuration on the receiving device side in accordance with the aforementioned sync signal.

[0221] FIG. 11 is a diagram illustrating a line buffering operation according to one or more embodiments of the present disclosure.

[0222] Referring to FIG. 11, multiple encoding cores can perform image encoding in units of preset lines. For example, if one encoding core performs an encoding operation in units of 16*16 pixels, four encoding cores can perform an encoding operation in units of 64 lines.

[0223] Therefore, if 64 lines of video are buffered through the HDMI signal, the encoding core can immediately perform the encoding operation.

[0224] And once the video encoding of the 64 lines is completed, the video encoding operation for the next 64 lines can be performed.

[0225] When performed in this manner, when comparing the encoder input image (1110) and the decoder output image (1120), it can be confirmed that a width difference of about 64 lines occurs between the two images within a time range of 1 ms.

[0226] In this way, the present disclosure performs encoding and transmission operations on a line-by-line basis, thereby achieving low-latency operation.

[0227] And the plurality of encoding cores can perform encoding operations in the order shown in the image (1130). That is, the encoding operations are not performed by the plurality of encoding cores at the same time, but first, the first encoding core sequentially performs the encoding operation in units of 16 pixels from the left to the right, and the next encoding core can perform the encoding operation from a preset time point (for example, 3 shown) after the operation of the encoding core that operates previously. In the illustrated example, only two cores are illustrated, but the above-described method can be applied even when there are four cores, and the above-described operation can be applied in units of two cores.

[0228] Meanwhile, in the illustrated example, the encoding operation is depicted and described as being performed from top to bottom and from left to right of the image. However, this operation is because the image data of the HDMI signal is transmitted as pixel data in the above-described order (or scan method). Therefore, if the pixel transmission order of the HDMI signal differs from the above-described case, the operation may correspond to the changed order.

[0229] FIG. 12 is a diagram illustrating a bit rate operation according to one or more embodiments of the present disclosure.

[0230] Referring to FIG. 12, as described above, image data (1200) is processed in units of preset pixels (e.g., 16*16).

[0231] Meanwhile, in order to compress at a uniform bit rate for each frame, it is necessary to control the bit rate to be lower than a preset value per frame.

[0232] In addition, as explained above, since the operation of the present invention is not performed after loading the entire frame, but immediately after buffering a preset number of line data, the following two methods can be applied in each compression process.

[0233] First, a method is used to control compression of all blocks by allocating a uniform number of bits to each block (1210). While this method has low computational complexity, it can potentially cause image degradation in certain areas.

[0234] Second, for each block, regions with high image complexity are allocated higher bits, while regions with low image complexity are allocated lower bits. In this case, the cores can perform the above-described operations during the encoding process, taking into account the bit allocations from previous blocks and the remaining budget.

[0235] For example, the average bit rate can be set to 15 bits, and encoding can be performed by lowering the bit rate to 5 or 10 bits for areas with low video complexity to secure a margin, and encoding can be performed at a high bit rate in areas with high video complexity based on the margin secured. Conversely, if a high bit rate is used for areas with high video complexity in the early part of a line, the average bit rate can be matched by encoding the video at a low bit rate for areas with medium video complexity in the latter part of the line based on the high bit rate used.

[0236] FIG. 13 is a diagram for explaining an audio data processing operation according to one or more embodiments of the present disclosure. Specifically, FIG. 13 is a diagram for explaining the operation of the audio processing module (340) of FIG. 8.

[0237] The audio processing module (340) can generate a wireless audio packet for transmitting audio data.

[0238] A wireless audio packet may include a header (1311) and a payload (1312).

[0239] The payload (1312) is a collection of a preset number of audio packets (i.e., audio samples) from among HDMI packets, and the headers of the audio packets may be removed.

[0240] The header (1311) is a merged version of the headers of each audio sample constituting the payload, and may include header information of the aforementioned audio samples and a frame index indicating the frame position where the corresponding audio packet was received. For example, the aforementioned frame index may be located in a reserved area of ​​the header (1311).

[0241] In this way, wireless audio packets generated in the audio processing module (340) can be stored in the transmission memory (350).

[0242] The sizes of such payloads are as shown in Table 1.

[0243] Sampling Rate(Hz)# of per-channel audio samples in Wireless Audio PacketExample of Payload Size in Byte2CH8CH32K-48K5122048819248K-96K102440961638496K-192K204881923276819 2K-384K40961638465536384K-768K819232768131072768K-1536K16384655536262144

[0244] The wireless audio packet generated as described above includes frame index information corresponding to the location of the corresponding samples in the header, so that the receiving side can easily perform synchronization of the received wireless audio packet.

[0245] FIG. 14 is a diagram illustrating a configuration of a transmission memory according to one or more embodiments of the present disclosure.

[0246] Referring to FIG. 14, the transmission memory (350) stores wireless video packets, wireless audio packets, and wireless meta packets in individual spaces, and may have a memory area with a pre-allocated address for each packet type.

[0247] Accordingly, each of the video encoder (320), the metadata processing module (330), and the audio processing module (340) knows multiple address areas of the transmission memory (350) that it uses, and can sequentially use the multiple address areas one by one.

[0248] At this time, the video encoder (320) can perform a write operation in units of frames, the audio processing module (340) can perform a write operation in units of wireless audio meta packets, and the metadata processing module (330) can perform a write operation in units of wireless meta packets (or frames).

[0249] In addition, the receiving memory (450) may also have a structure similar to that of the transmitting memory. That is, the receiving module (410) described above may identify the type of wireless packet through the header of the received packet and store the received packet in different areas according to the identified type. For example, the header of the received packet may include information indicating video, and the received wireless packet may be stored in the video area of ​​the receiving memory (450).

[0250] Fig. 15 is a drawing for explaining the operation of generating a control signal according to the present disclosure.

[0251] Referring to FIG. 15, the arrangement of a synchronization signal (1510) within an HDMI signal and various data (1520, 1530, 1540) within an HDMI signal is shown.

[0252] Since HDMI signals transmit and receive data in the format described above, there are no separate synchronization signals for each piece of data. However, to support HDMI 2.1 specifications such as HDR and VRR, metadata is required for timing and alignment of video frames.

[0253] Although video data and audio data have different processing units, they require synchronization to align their starting points. In other words, a control signal (or synchronization signal) is required for video and audio synchronization tracking.

[0254] Accordingly, the present disclosure generates a control signal for synchronization for the reasons described above. Specifically, the control signal of the present disclosure can generate four control signals, such as a video enable signal (1550), an audio enable signal (1560), a meta enable signal (1570), and a frame index signal (1580), as illustrated.

[0255] The video enable signal (1550) may have a high value after the first vertical synchronization trigger, after the meta enable signal (1570) has a high value. Accordingly, the video decoder (420) described above may perform an encoding operation according to the subsequent vertical synchronization signal when the video enable signal has a high value.

[0256] The audio enable signal (1560) may have a high value after the first vertical synchronization trigger, after the meta enable signal (1570) has a high value. Accordingly, the audio processing module (440) described above may perform an operation for audio signal processing according to the subsequent vertical synchronization signal when the audio enable signal has a high value.

[0257] The meta enable signal (1570) is a signal for controlling the operation of the metadata processing module (430) and can have a high value when any start signal is triggered. When the first vertical synchronization signal is triggered while the meta enable signal is high, the metadata processing module (430) can perform a metadata processing operation.

[0258] The frame index signal (1580) can sequentially output a value that increases by 1 according to the trigger of the vertical synchronization signal when the video enable signal and / or the audio enable signal have a high value.

[0259] In the above, it was explained that the meta enable signal is triggered first. This is because in the HDMI specification, metadata outputs a value corresponding to the next frame, that is, while audio signals and video signals between vertical synchronization signals are information at the same point in time, metadata is data at the next point in time, and therefore the meta enable signal is triggered first.

[0260] When four control signals are generated in this way, each component of the transmitting device can store the data processed in each component in memory (1590).

[0261] Meanwhile, in order to perform the above-described operations in accordance with the HDMI transmission speed, low-latency processing is required. In other words, the processing of the video, audio, and metadata described above must also be performed within the frame units described above, in accordance with the frame units of the HDMI output image.

[0262] To achieve this, the processing delay of each component must be within 1ms, and the delay (e.g., 1ms) and bit-rate (e.g., the compressed stream size of each frame) must be processed to be uniform within each frame, so that the low-latency transmission and the video / audio meta of each frame can be synchronized.

[0263] FIG. 16 is a diagram illustrating an example of a transmission packet according to one or more embodiments of the present disclosure.

[0264] Referring to FIG. 16, a transmission packet (1610) includes a transmission header (1611) and transmission data (1613).

[0265] The transmission header (1611) may include transmission port information and reception port information.

[0266] Transmission data (1613) may include one of multiple wireless packets stored in the transmission memory. For example, one of a wireless video packet, a wireless audio packet, and a wireless meta packet may be included in the data described above. As described above, such wireless packets may have an additional identification field (H) indicating the type of each packet (i.e., whether it is a video packet, audio packet, etc.).

[0267] Meanwhile, although the illustrated example describes an example of transmitting data using UDP (User Datagram Protocol), other protocols such as TCP may be used in addition to the above-described protocol during implementation.

[0268] FIG. 17 is a diagram for explaining a packet transmission operation according to one or more embodiments of the present disclosure.

[0269] Referring to Figure 17, first, the size of the stream to be transmitted is confirmed (S1710). Specifically, the number of video data, audio data, and metadata constituting one frame can be confirmed.

[0270] And the number of streams to be transmitted and the number of streams transmitted are checked (S1720). Specifically, it is possible to check whether the transmission of data belonging to the corresponding frame has been completed.

[0271] If the transmission is not completed (S1720-yes), the data stored in the transmission memory is read (S1730), and it is checked whether there was a transmission start in the current frame (S1740).

[0272] If the current transmission start time is (S1740-Y), metadata is read first (S1750), packetized (S1790), and transmission can be performed (S1795).

[0273] Meanwhile, in the present disclosure, two packetizations are processed, and first, packetization is performed for each data rectification, and video packets, audio packets, and meta packets can be generated through the process.

[0274] Another packetization method is packetization for wireless transmission (or UDP packetization). Specifically, as shown in Figure 16, this packetization adds a UDP packet header and identification information to identify the type of the video packet, audio packet, and meta packet described above. Packets generated through this packetization method may be referred to as wireless video packets, wireless video packets, or wireless meta packets.

[0275] Meanwhile, in Fig. 17, the packet processing described above is illustrated and described as being processed by dividing it into two operations, but when implemented, it is also possible to generate video information, audio information, and meta information as wireless packets through a single operation.

[0276] If metadata is transmitted (S1740-N), wireless audio packets or wireless video packets can be transmitted by being packetized. At this time, the wireless transmission system (transmission device (300)) can give audio data a higher priority than video data, so that it can be transmitted first.

[0277] If all data transmission is complete (S1760-Y), the transmission history is checked and the current frame's transmission is completed (S1780). The buffer memory address can be increased, allowing transmission of the next frame's data to proceed.

[0278] FIG. 18 is a diagram for explaining an operation of generating a synchronization signal according to one or more embodiments of the present disclosure.

[0279] Referring to FIG. 18, the video decoder (420) can read a compressed stream from the receiving memory, decode it, and then transmit the final image to a subsequent block (e.g., a picture quality processing unit).

[0280] And, after the start of decoding, the video decoder (420) can generate a vertical synchronization signal (1810) and a horizontal synchronization signal (1820) at every frame interval, and output the image data by arranging it according to the generated vertical synchronization signal and horizontal synchronization signal.

[0281] Meanwhile, the operation of the video decoder (420) may be performed based on a start command output in response to the data reception operation of the receiving module. For example, if the index of the metadata and the image match, the metadata may be transmitted to the processor side that processes the metadata after the trigger of the sync signal described above.

[0282] You can use the V_SYNC interrupt signal, which is one of the output signals of the video processing unit.

[0283] FIG. 19 is a flowchart illustrating a control operation of an electronic device according to one or more embodiments of the present disclosure.

[0284] Referring to FIG. 19, uncompressed digital image content is received (S1910). Here, the uncompressed digital image content may be an HDMI data signal.

[0285] Video information, audio information, and meta information within the above digital image content are separated (S1920). Specifically, data outside of a preset section within the digital image content can be separated into video data, and meta data and audio data can be separated using packet type values ​​within the preset section. At this time, various control signals for synchronization of the data described below can be generated. For example, a meta enable signal, a video enable signal, an audio enable signal, and a frame index can be generated.

[0286] And the video data is encoded (1930). For example, encoding of a preset block unit can be performed using video information of a preset line unit among the video information. Specifically, when video information of a preset line unit among the plurality of video information constituting one frame within digital video content is buffered, an encoding operation can be performed using the buffered video information of a preset line unit.

[0287] And, packet data corresponding to the wireless method is generated from the compressed video data, audio data, and metadata (S1940). For example, packet data including one of the encoded data, audio data, and metadata in the data area and classification information indicating the type of data included in the data area can be generated.

[0288] The generated packet data can then be transmitted wirelessly. At this time, metadata can be transmitted with the highest priority, followed by audio data and then video data.

[0289] As described above, the control method of an electronic device according to the present disclosure enables wireless transmission of video content that would normally be transmitted and received via wired channels, eliminating the need for wired cables. Furthermore, by compressing the content for transmission, high-speed video data transmission and reception can be achieved using currently commonly used wireless communication methods.

[0290] FIG. 20 is a flowchart illustrating a display method according to one or more embodiments of the present disclosure.

[0291] Referring to Figure 20, a network packet is received via wireless communication (S2010). For example, the wireless communication method described above may be WiFi, and the network packet may be a network packet using the UDP protocol. These network packets may be divided into a header and data.

[0292] Using the identification information within the received network packet, the data within the received network packet is stored in one of the video area, audio area, and meta area of ​​the receiving memory (S2020). Specifically, using the identification information within the data area within the network packet, it is possible to identify whether the received network packet is related to video, audio, or meta data, and store the data of the received network packet in a memory area corresponding to the identified type based on the identification result. At this time, the field information indicating the above-described data type included in the corresponding data may be deleted and stored.

[0293] Data stored in the video area of ​​the receiving memory is decoded to generate video data and a sync signal corresponding to the video data. Specifically, decoding can be performed using the data stored in the video area. At this time, a horizontal synchronization signal and a vertical synchronization signal can be generated based on the corresponding video data, and a data signal can be output according to the timing of the generated synchronization signal.

[0294] A frame image is generated using the generated sync signal and video data (S2040). Specifically, this operation is a typical operation within a display device, and image quality improvement algorithms related to typical image processing can be performed to generate a frame image to be provided to the display.

[0295] The generated frame image is displayed on the display (S2050).

[0296] As described above, the method for controlling a display device according to the present disclosure can display images using images received via wireless communication, thereby eliminating the need for wired cables. This can improve the convenience of movement and aesthetics of the display device.

[0297] Meanwhile, the methods according to at least some of the various embodiments of the present disclosure described above may be implemented in the form of an application that can be installed on an existing electronic device (or display device).

[0298] Additionally, the methods according to at least some of the various embodiments of the present disclosure described above can be implemented with only a software upgrade or a hardware upgrade for an existing electronic device.

[0299] Additionally, the methods according to at least some of the various embodiments of the present disclosure described above may also be performed through an embedded server provided in an electronic device (or a display device), or an external server of at least one of the electronic devices.

[0300] Meanwhile, according to one or more embodiments of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The machine is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., an electronic device, a display device) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the 'non-transitory storage medium' means a tangible device and does not include a signal (e.g., an electromagnetic wave), and this term is used to refer to a case where data is stored semi-permanently in the storage medium and a case where data is stored temporarily. No distinction is made. For example, a 'non-transitory storage medium' may include a buffer in which data is temporarily stored. According to one or more embodiments, the methods according to the various embodiments disclosed in the present document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a commodity. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones).In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0301] Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored in the storage medium and operate according to the called instructions, and may include an electronic device (e.g., an electronic device, a display device) according to the disclosed embodiments.

[0302] When the above-described instruction is executed by the processor, the processor may directly or under the control of the processor use other components to perform the function corresponding to the above-described instruction. The instruction may include code generated or executed by a compiler or interpreter.

[0303] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, An input / output interface for receiving uncompressed digital video content; A communication device that transmits and receives data wirelessly; At least one memory to store instructions; At least one processor for obtaining packet data corresponding to the digital image content and controlling the communication device so that the packet data is transmitted; At least one processor of the above, Separate video information, audio information, and meta information within the above digital image content, In order to obtain encoded video information, encoding of preset block units is performed using preset line unit video information among the above video information, In order to obtain the above packet data, each of the encoded video information, the audio information and the meta information is packetized, An electronic device executing said instructions to control said communication device to wirelessly transmit said packet data to a display device.

2. In paragraph 1, At least one processor of the above, An electronic device executing an instruction that performs an encoding operation by using the buffered video information in the preset line units when video information in the preset line units among a plurality of video information constituting one frame in the digital image content is buffered.

3. In paragraph 2, At least one processor of the above, When encoding of one frame in the digital image content is completed, information about the frame is stored in at least one memory, An electronic device executing the instruction, which performs an encoding operation by using the video information of the preset line unit and the frame information corresponding to the preset line unit of the stored previous frame.

4. In paragraph 2, At least one processor of the above, An electronic device comprising a plurality of encoding cores that perform encoding in preset block units for a plurality of different lines.

5. In paragraph 2, At least one processor of the above, An electronic device executing an instruction that performs encoding at a bit rate based on the complexity of input video information during a process of performing encoding of preset block units.

6. In paragraph 1, At least one processor of the above, Separate data other than a preset section within a signal received from the above input / output interface into image information, An electronic device executing the instruction to separate meta information and audio information by checking the packet type of a signal within the above-described preset section.

7. In paragraph 1, At least one processor of the above, Based on the synchronization signal in the signal received from the above input / output interface, a frame index is obtained, An electronic device executing the instructions, wherein the instructions add a header including the frame index to each of the encoded video information, the audio information and the meta information.

8. In paragraph 1, At least one processor of the above, An electronic device executing the instructions, wherein the instructions merge a predetermined number of audio samples to obtain merged audio samples and an audio payload based on audio information including audio samples and headers corresponding to the audio samples, obtain a header corresponding to the audio payload using the header of each of the merged audio samples, and obtain a wireless audio packet by combining the audio payload and the header.

9. In paragraph 8, At least one processor of the above, An electronic device executing the instruction to obtain a header corresponding to the audio payload, the header including frame index information corresponding to a location of the audio information.

10. In paragraph 1, At least one processor of the above, An electronic device executing the instructions to merge the meta information in units of frames to obtain a meta payload, obtain a meta header including frame index information corresponding to the meta information, and combine the meta header and the meta payload to obtain a wireless meta packet.

11. In paragraph 1, At least one processor of the above, An electronic device executing the instruction to obtain packet data including one of the encoded video information, the audio information, and the meta information and classification information indicating the type of data included in the data area.

12. In paragraph 1, At least one processor of the above, An electronic device executing the instruction that controls the communication device to transmit packet data corresponding to meta information among packet data with the highest priority.

13. A method for controlling an electronic device using at least one processor, A step of separating video information, audio information, and meta information within uncompressed digital image content; A step of performing encoding of a preset block unit by using preset line unit video information among the video information to obtain encoded video information; To obtain packet data, a step of packetizing each of the encoded video information, the audio information and the meta information; and A control method comprising: a step of controlling a communication device to wirelessly transmit the packet data to a display device.

14. In paragraph 13, The steps for performing the above encoding are: A control method for performing an encoding operation by using the video information of the preset line units buffered among the plurality of video information constituting one frame within the digital image content.

15. A non-transitory computer-readable recording medium storing a program for executing a method of controlling an electronic device on at least one processor, The above control method is, A step of separating video information, audio information, and meta information within uncompressed digital image content; A step of performing encoding of a preset block unit by using preset line unit video information among the video information to obtain encoded video information; To obtain packet data, a step of packetizing each of the encoded video information, the audio information and the meta information; and A computer-readable recording medium comprising: a step of controlling a communication device to wirelessly transmit the packet data to a display device.

Images