Display device, and display method thereof

Abstract

This display device comprises a display and a processor, wherein the processor: displays an extracted additional information area; and controls the display to update, at each preset update time point, at least a part of the additional information area displayed on the display according to a change in a content screen displayed on an external display device, wherein the additional information area is included in the content screen displayed on the external display device and is extracted from the content on the basis of pixel data of the content screen.

Description

Display device and method of displaying the same

[0001] The present invention relates to a display device and a display method thereof for displaying at least a portion of a screen displayed on an external display device.

[0002] Thanks to advancements in electronic technology, various types of display devices are now used in everyday life. Among these display devices are sub-display devices, which are implemented in a form connected to an external display device.

[0003] Traditionally, users typically connected a sub-display device to the main display to view a larger screen. However, beyond expanding the screen size or displaying a duplicate of the same screen, there weren't many ways to utilize the sub-display device.

[0004] Therefore, the need for a method to increase the usability of display devices connected to external display devices has arisen.

[0005] According to at least one embodiment of the present disclosure, a display device includes a display, a processor, and the processor controls the display to display the extracted additional information area and update at least a portion of the additional information area displayed on the display according to a change in a content screen being displayed on an external display device at each preset update time, wherein the additional information area is included in the content screen being displayed on the external display device and is extracted from the content based on pixel data of the content screen.

[0006] In addition, according to at least one embodiment of the present disclosure, a display method of a display device including a display includes a step of displaying an additional information area extracted based on pixel data, and a step of updating at least a part of the additional information area displayed on the display according to a change in a content screen being displayed on an external display device at each preset update time, wherein the additional information area is included in the content screen being displayed on the external display device and is extracted from the content based on pixel data of the content screen.

[0007] In addition, according to at least one embodiment of the present disclosure, a non-transitory readable recording medium storing a program that, when executed by a display device, causes the display device to perform operations, the operations including: a step of displaying an additional information area extracted based on the pixel data; and a step of updating at least a portion of the additional information area displayed on the display according to a change in a content screen being displayed on an external display device at each preset update time, wherein the additional information area is included in the content screen being displayed on the external display device and is extracted from the content based on pixel data of the content screen.

[0008] In addition, according to at least one embodiment of the present disclosure, a display device includes a display, a communication interface, and a processor, wherein the processor obtains pixel data of a content screen being displayed on an external display device through the communication interface, extracts an additional information area included in the content screen based on the obtained pixel data, displays the extracted additional information area, and controls the display to update and display at least a part of the additional information area according to a change in the content screen being displayed on the external display device at each preset update time.

[0009] FIG. 1 is a drawing for explaining the operation of a display device according to at least one embodiment of the present disclosure.

[0010] FIG. 2 is a block diagram illustrating a configuration of a display device according to at least one embodiment of the present disclosure.

[0011] FIG. 3 is a drawing showing an example of a screen displayed by a display device according to at least one embodiment of the present disclosure.

[0012] FIG. 4 is a drawing for explaining a setting UI of a display device according to at least one embodiment of the present disclosure.

[0013] FIG. 5 is a diagram illustrating an example of displaying an additional information area in a display device according to at least one embodiment of the present disclosure.

[0014] FIG. 6 is a drawing showing an example in which the display of an additional information area is maintained in a display device according to at least one embodiment of the present disclosure.

[0015] FIG. 7 is a diagram illustrating an example in which a display device according to at least one embodiment of the present disclosure receives data from at least one server device.

[0016] FIG. 8 is a drawing for explaining an example of a display device according to at least one embodiment of the present disclosure displaying the location of an avatar within a map.

[0017] FIG. 9 is a drawing showing an example in which the display of an additional information area is maintained in a display device according to at least one embodiment of the present disclosure.

[0018] FIG. 10 is a flowchart for explaining a display method of a display device according to at least one embodiment of the present disclosure.

[0019] FIG. 11 is a diagram illustrating an example of displaying an additional information area on three or more display devices according to at least one embodiment of the present disclosure.

[0020] FIGS. 12 to 16 are drawings showing examples of locations for displaying additional information areas in display devices according to various embodiments of the present disclosure.

[0021] FIG. 17 is a diagram illustrating an example of displaying an additional information area in a display device according to at least one embodiment of the present disclosure.

[0022] FIG. 18 is a drawing showing an example of displaying an additional information area in a projector display device according to at least one embodiment of the present disclosure.

[0023] FIG. 19 and FIG. 20 are flowcharts for explaining a display method of a display device according to at least one embodiment of the present disclosure.

[0024] The terms used in the various embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should be defined based on the meaning of the terms and the overall content of this disclosure, rather than simply their names.

[0025] It should be understood that the various embodiments of the present disclosure and the terminology used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, but include various modifications, equivalents, or substitutes of the embodiments.

[0026] In connection with the description of the drawings, similar reference numerals may be used for similar or related components.

[0027] The singular form of a noun corresponding to an item may include one or more of said items, unless the relevant context clearly indicates otherwise.

[0028] In this disclosure, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0029] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0030] When a component (e.g., a first component) is referred to as being “coupled” or “connected” to another component (e.g., a second component), with or without the terms “functionally” or “communicatively,” it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0031] Terms such as "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the present disclosure, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0032] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0033] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0034] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0035] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor, excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0036] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0037] In this disclosure, the term user may refer to a person using an electronic device or a device used by the person.

[0038] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0039] FIG. 1 is a drawing for explaining the operation of a display device (100) according to at least one embodiment of the present disclosure.

[0040] In FIG. 1, the display device (100) is illustrated as a monitor, but the display device (100) may be implemented as various types of display devices such as a TV, a laptop, a monitor, a kiosk, a tablet PC, an electronic picture frame, a mobile phone, a large format display (LFD), a digital signage, a digital information display (DID), a video wall, a projector display, etc. However, the display device (100) is not limited thereto, and in some cases, it may be implemented as various devices such as an image processing device (e.g., a set-top box, one connected box) that is connected to the display device and provides an image. In addition, the display devices mentioned above are merely examples, and in addition to the display devices mentioned above, a device that is connected to another display device or a server and can perform the operations described below may be included in the display device according to various embodiments of the present disclosure.

[0041] In FIG. 1, the external display device (200) may be an independent device provided separately from the display device (100). In FIG. 1, the external display device (200) is illustrated as being the same type of device as the display device (100), but is not necessarily limited thereto, and may be implemented as a different type of device, or may be implemented as a single wide monitor, or may be implemented as a form in which three or more display devices are connected. For example, when a content screen and an additional information area are displayed on a single wide monitor, the content may be displayed in a designated area within a single screen, an additional information area may be extracted based on pixel data of the content screen, and the extracted additional information area may be displayed in another area.

[0042] For convenience of explanation, the present disclosure illustrates a display device (100) and an external display device (200) one by one, but three or more devices may be connected to each other and used. In addition, for convenience of explanation, the present disclosure illustrates a display device (100) and an external display device (200), but they may alternatively be described as a sub-display device (100) and a main display device (200).

[0043] Meanwhile, the various embodiments of the present disclosure described below are not necessarily applicable only to a state in which multiple display devices (100, 200) are connected to each other as illustrated in FIG. 1, but can also be applied to a display device equipped with a single large display, where the screen is divided into multiple areas. For example, within a wide screen, the screen can be divided into left and right halves, displaying content on the left side, and storing and analyzing pixel data related to the content to display an additional information area on the right side.

[0044] According to FIG. 1, the display device (100) can be directly connected to an external display device (200). Accordingly, the display device (100) can obtain pixel data of a content screen being displayed on the external display device (200).

[0045] A content screen can be a screen that provides various types of content, including game content, video content, content displaying documents, photos, images, and TV broadcast content. Depending on the content type, content screens can be categorized into various types, including game content screens, video content playback screens, still images, music content playback screens, and text content screens.

[0046] Meanwhile, the content screen may display various additional information related to the content. In the present disclosure, the area where such additional information is displayed is referred to as an additional information area. That is, the additional information area may be a portion of the content screen being displayed on the external display device (200). For example, if the screen being displayed is a playback screen of video or music content, the external display device (200) may display the degree to which the video or music content is being played in the form of a bookmark or progress bar. Alternatively, the external display device (200) may display a recommended content area indicating recommended content related to the video or music content, a text area indicating text related to the video or music content, etc. These additional information areas may be fixedly displayed on the content screen, but at least some of the additional information areas may be selectively displayed according to the user's selection.

[0047] For example, if the game content is one in which a user moves his or her avatar within a game environment and performs various missions, additional information such as maps, quests, and items may be displayed on the game screen according to the user's selection or by default. The area where such additional information is displayed is referred to as an additional information area. In the present disclosure, an avatar may be an image that moves according to the user's operation. An avatar may be described in various ways, such as a character or a player. An avatar may be created in the form of a human, but is not limited thereto, and may also be created in various forms, such as an animal, other objects, or a robot.

[0048] FIG. 1 illustrates a state in which video content is being played on an external display device (200), and a progress bar (212) or other additional information (211) is displayed on the screen.

[0049] Additional information may include various information depending on the type of content screen, and may be described as recommended information, related information, display information, screen information, content information, etc., but in this disclosure, it is described as additional information.

[0050] The display device (100) analyzes pixel data constituting the content screen, extracts additional information areas (211, 212) within the content screen, and displays the extracted additional information areas (111, 112). The displayed additional information areas (111, 112) may be displayed in the same position, size, shape, etc. as the additional information areas (211, 212) within the actual content screen, but are not necessarily limited thereto. For example, the display device (100) may display the additional information areas (111, 112) in various display states based on display option information for displaying the additional information areas. The display option information may be changed according to the user's settings, but is not limited thereto, and may be information set as default.

[0051] The additional information area may be displayed in various locations or devices depending on various conditions, such as the number of display devices, connection with external devices, and the characteristics of the additional information area. Specific descriptions of various embodiments related to the display of the additional information area are provided below in FIGS. 12 to 15.

[0052] The display device (100) can update and display at least a portion of the additional information area according to changes in the content screen being displayed on the external display device (200) at each preset update point. In the present disclosure, an update may be an operation of changing the display position, content, shape, size, etc. of the additional information area, and the update point may be a state in which a condition set to perform such an update is met.

[0053] For example, if the content screen being displayed on an external display device (200) is a video playback screen and the additional information area (212) is a progress bar, the progress bar moves to the right depending on the playback level of the video. The display device (100) can update and display the position of the moved progress bar at each preset update point.

[0054] A preset update point may include at least one of the following: when a preset cycle occurs and when a menu or function is used to display an additional information area within the content.

[0055] Here, the cycle may be set differently for each menu or function. As in the example described above, if the content displayed by the external display device (200) is a video playback screen and the additional information area (212) displayed by the display device (100) is a progress bar, the update cycle may be set short (specifically, the same cycle as the content playback speed) so that the movement of the progress bar is proportional to the actual playback speed of the content. As another example, if the content of the external display device (200) is a game screen and the additional information area (212) displayed by the display device (100) is a map screen that was temporarily displayed within the game screen, the display device (100) may change the display of the avatar's position within the map screen at regular intervals (for example, every 10 seconds).

[0056] Meanwhile, in the above-described section, it was described that the display device (100) receives pixel data from an external display device (200), analyzes it, and extracts an additional information area, but it is not necessarily limited to this.

[0057] For example, the external display device (200) may directly analyze pixel data to extract an additional information area. In this case, the display device (100) may receive data on the additional information area extracted from the external display device (200). In addition, the external display device (200) or another device other than the display device (100) (e.g., a server device, a terminal device, another display device, etc.) may detect an additional information area based on pixel data and then provide it to the display device (100).

[0058] Alternatively, the display device (100) may directly detect and display an additional information area as described above based on pixel data received from an external display device (200), or may display the received additional information area when the additional information area detected from the external display device (200) or other device is received as is.

[0059] Pixel data refers to data about the pixels that make up a single screen. Pixel data can include pixel coordinates and pixel values. When using the RGB (Red, Green, Blue) color model, the pixel value for each pixel coordinate contains R, G, and B data.

[0060] In Fig. 1, a display device (100) is connected to an external display device (200) via a wired cable, and the external display device (200) is also connected to another electronic device via a wired cable. The electronic device may be a variety of content source devices such as a PC main body, a game player, a set-top box, etc.

[0061] Although Fig. 1 illustrates a state in which the display device (100) and the external display device (200) are connected via a wired cable, the display device (100) and the external display device (200) or the electronic device may be connected via wireless communication.

[0062] Specifically, each device may be connected through a wide area network (WAN) such as the Internet, a local area network (LAN) formed around an access point (AP), or a short-range wireless network that does not use an access point (AP). Short-range wireless networks may include, but are not limited to, Bluetooth (IEEE 802.15.1), Zigbee (IEEE 802.15.4), Wi-Fi Direct, Near Field Communication (NFC), Z-Wave, etc.

[0063] The display device (100) may store various programs. For example, various programs may be stored, such as a web browser that can access an external server device via the Internet, an application that performs pixel data storage and analysis functions, etc. The display device (100) may perform various operations as the programs are executed.

[0064] Specifically, when an additional information area is displayed within a content screen on an external display device (200) based on a user's input or other manipulation, the display device (100) may receive pixel data regarding the displayed additional information area and input it into a stored data analysis program. The data analysis program may analyze the object corresponding to each pixel based on the input pixel data, and then extract the additional information area based on the analysis result. The display device (100) displays the extracted additional information area on its own. Accordingly, even if the user removes the additional information area displayed on the external display device (200), the display device (100) may still maintain the display of the additional information area. Accordingly, the additional information area can be easily checked while minimizing obstruction of the view of the external display device (200), thereby improving the user's immersion and convenience.

[0065] Meanwhile, the above description assumes that the additional information area is extracted based on pixel data, but it is not necessarily limited to pixel data, and the additional information area can also be extracted based on other graphic data in addition to pixel data.

[0066] Below, a method for displaying a settings UI, a method for changing and displaying an additional information area, etc., according to various embodiments of the present disclosure are specifically described.

[0067] FIG. 2 is a block diagram illustrating the configuration of a display device (100) according to at least one embodiment of the present disclosure.

[0068] The display device (100) may include a memory (110), a display (120), a communication interface (130), and a processor (140).

[0069] The memory (110) is a configuration for storing at least one instruction, an operating system (O / S), a program, pixel data, etc. regarding the display device (100). For example, a data analysis program such as a pre-trained artificial intelligence model can be stored to identify content being displayed on an external display device (200). Specifically, the artificial intelligence model can be implemented as a convolutional neural network (CNN), a long short-term memory (LSTM), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), etc., but is not limited to these examples. Such a learning model can learn using various data such as data provided by a game server and data provided by an analysis APP server, and can also learn in various ways such as an on-device learning method, an artificial intelligence learning method, and a user-manually designated learning method.

[0070] These artificial intelligence models are computing systems implemented based on the neural networks of human or animal brains, and may also be referred to as learning models, machine learning models, neural network models, or deep learning models.

[0071] The artificial intelligence model stored in the memory (110) can detect an additional information area by analyzing pixel data received from an external display device (200).

[0072] The memory (110) may be implemented in the form of memory embedded in the display device (100) or may be implemented in the form of memory detachable from the display device (100) depending on the purpose of data storage. For example, data for driving the display device (100) may be stored in the memory (110) embedded in the display device (100), and data for the expansion function of the display device (100) may be stored in the memory (110) detachable from the display device (100).

[0073] In the case of the memory (110) embedded in the display device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)).

[0074] The display (120) can display various screens under the control of the processor (140). The display (120) can be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. In addition, the display (120) can be implemented as an LFD display. For example, it can be implemented as a display in various forms such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The display (120) can also include a driving circuit, a backlight unit, etc., which can be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc.

[0075] For example, when a game screen is running on an external display device (200), the processor (140) may display an item area that is temporarily or permanently displayed within the game screen through the display (120). The specific details of displaying the additional information area in this manner are described in detail in FIG. 5.

[0076] The communication interface (130) includes a circuit and is configured to communicate with various external devices. Specifically, the communication interface (130) can communicate with an external display device (200), a server device, or an electronic device. The processor (140) can receive various data or information, particularly pixel data, from an external display device (200) connected via the communication interface (130).

[0077] The processor (140) may transmit pixel data or other various data or information to at least one server device via the interface (130). The operation of performing communication with at least one server device is described in detail in FIG. 7.

[0078] The communication interface (130) may include various wired interfaces such as a USB port, an HDMI port, a display port, an RGB port, a DVI (Digital Visual Interface) port, a Thunderbolt, and a component port, as well as various wireless interfaces such as a WiFi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB module (Ultra Wide Band).

[0079] Figure 1 shows a state in which an external display device (200) is connected via an HDMI interface. The processor (140) can receive pixel data from the external display device (200) via a communication interface (130).

[0080] The processor (140) controls the overall operation of the display device (100). Specifically, the processor (140) is connected to each component of the display device (100) and can control the overall operation of the display device (100). The processor (140) may be composed of one or more processors.

[0081] The processor (140) can perform operations of the display device (100) according to various embodiments by executing one or more commands stored in the memory (110).

[0082] The processor (140) may include one or more of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), a DSP (Digital Signal Processor), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. The processor (140) may control one or any combination of other components of the display device (100), and may perform operations related to communication or data processing. The processor (140) may execute one or more programs or instructions stored in the memory (110). For example, one or more processors (140) may execute one or more instructions stored in the memory (110), thereby performing a method according to one or more embodiments of the present disclosure.

[0083] In embodiments of the present disclosure, the processor (140) may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0084] For example, if the content being executed on the external display device (200) is video content, the processor (140) may display an additional information area including a progress bar indicating the playback time and a text area indicating text related to the content through the display (120).

[0085] When a user activates a text area in which a comment can be entered for a video while a video is being played on an external display device (200), the external display device (200) adds the text area to the video content screen and displays it.

[0086] The processor (140) receives pixel data of image frames constituting the video content screen, extracts a text area included in each image frame, and then displays only the text area through the display (120). In this case, when the user deactivates the text area, the video content screen that does not include the text area is displayed on the external display device (200), but the text area is displayed on the display device (100).

[0087] A detailed description of the video content playback screen will be described later in Fig. 9.

[0088] FIG. 3 is a drawing showing an example of a screen of a display device according to at least one embodiment of the present disclosure.

[0089] When the display device (100) is turned on, the processor (140) can control the display (120) to display a UI (310) including icons for various applications installed on the display device (100) based on a program stored in the memory (110).

[0090] If an application for performing the operation described in Fig. 1 is installed, the processor (140) may also display an icon for executing the application. Fig. 3 illustrates a state in which an icon named “AI Analysis APP” is displayed.

[0091] When the AI ​​analysis app is selected by the user, the processor (140) may analyze and process pixel data of the content screen displayed on the external display device (200) by applying an artificial intelligence model to display the above-described additional information area. At this time, the screen may be displayed in a form convenient for the user through histogram analysis based on the analyzed pixel data. For example, if the quest window is the most used in the game content screen, the quest window (113) among the additional information areas may be displayed on the left side of the display device (100) with the highest priority, as shown in FIG. 15. Histogram analysis is an analysis method that divides data into sections and then expresses the frequency of data belonging to each section as a bar graph. However, it is not necessarily limited thereto, and the additional information area may be provided in a form of a screen convenient for the user through various analysis methods.

[0092] However, in addition to AI models, data analysis methods can also be used, such as rule-based analysis models, which use predefined rules or heuristics to analyze data and make decisions. In this case, the icon in Figure 3 may be displayed under a different name than "AI Analysis App."

[0093] In addition, the software is not limited to this and may include an analysis model that combines a rule-based system with machine learning.

[0094] FIG. 4 is a drawing for explaining a setting UI of a display device (100) according to at least one embodiment of the present disclosure.

[0095] The processor (140) may display additional information areas, etc., that can be extracted from a content screen running on an external display device (200), on a single setting UI (410). If there are multiple extractable additional information areas, the processor (140) may display each of the multiple additional information areas in a selectable state on the setting UI (410). Alternatively, the processor (140) may display only some of the extractable additional information areas. For example, the processor (140) may receive and analyze data from at least one server device via the communication interface (130) and display a plurality of useful additional information areas related to the content for selection. For example, if the content is a game screen, the setting UI (410) may display the additional information areas so that they can be selected based on the display frequency of the quest area. That is, additional information areas that are not frequently displayed may not be displayed on the setting UI (410). In this way, the plurality of additional information areas displayed on the setting UI (410) may vary depending on the displayed content and the user.

[0096] When a user inputs an operation to select an additional information area, the processor (140) can store information about the set additional information area in the memory (110). The processor (140) can extract the set additional information area from the content screen based on the information stored in the memory (110). In the case of FIG. 4, if the user selects only Map, the processor (140) can detect and display only the Map portion from the content screen of the external display device (200).

[0097] An operation for selecting an additional information area can be input through a user input unit (500) or a remote control device (400) within the display device (100). For example, if the content screen is a game screen, the additional information area includes at least one of a game title area, a map area, an item area, a quest area, a participant information area, and a chat area, and an additional information area to be displayed on the display device (100) can be selected through a user operation.

[0098] The remote control device (400) may be implemented as a remote control capable of controlling the display device (100), as shown in FIG. 4, but this is only one embodiment and may be implemented as a variety of portable terminals such as a smart phone, tablet PC, etc.

[0099] When a remote control device (400) is used, the display device (100) may further include an RF signal module for receiving a remote control signal generated from the remote control device (400). Alternatively, when the remote control device (400) and the display device (100) are connected by a wireless communication method such as Bluetooth communication, the processor (140) may receive a control signal of the remote control device (400) through the communication interface (130).

[0100] The user input unit (500) is a configuration that can receive user commands other than the remote control device (400). The user input unit (500) may include a button, a lever, a switch, a touch interface, etc.

[0101] The processor (140) can control the operation of the display device (100) based on a signal received from the remote control device (400) or a user command input through the user input unit (500). For example, when a button for setting the UI of the remote control device (100) is selected, the processor (140) can display a setting UI (410) as shown in FIG. 4. However, the present invention is not limited thereto, and the processor (140) can extract an additional information area during a preset period of time after pixel data for the content screen is received, and then configure a setting UI (410) as shown in FIG. 4 using information about the additional information areas extracted up to that period of time. For example, if the title of the content is identified as AAA and there are three additional information areas, such as Map, Quest, and Item list, the processor (140) can configure a setting UI (410) including both the title and selectable additional information areas as shown in FIG. 4 and display it through the display (120).

[0102] In a similar manner, the processor (140) can also change the display options of each additional information area. Specifically, when a user operation for changing at least one of the display layout, color, display position, and size of the additional information area displayed by the display (120) is input through the remote control device (400) or the user input unit (500), the processor (140) can change information about the display options of the additional information area according to the user operation and store the changed information in the memory (110). Thereafter, when an event requiring the display of the additional information area occurs, the processor (140) can control the display (120) to display the additional information area in a changed display state based on the stored display option information.

[0103] FIG. 5 is a drawing showing an example of displaying an additional information area in a display device (100) according to at least one embodiment of the present disclosure.

[0104] If the content screen running on the external display device (200) connected to the display device (100) is a game screen, the additional information area may include at least one of a game title area, a map area, an item area, a quest area, a participant information area, and a chat area. The additional information area may vary depending on the type of game.

[0105] For example, if a game that provides a game screen including an avatar and a background screen that moves according to user operation is being run by an external display device (200) or an electronic device connected to the device, the user can enjoy the game while moving the avatar to a desired location. In this case, the user can press a button to check the mini-map (211) to check the location of his or her avatar within the game environment. In this case, as illustrated in FIG. 5, the external display device (200) can display the mini-map (211) by overlaying it on the game screen. Alternatively, if the user's avatar obtains a new item or a confirmation button for checking an existing item is pressed, an item (212) can be displayed by overlaying it on the game screen.

[0106] In this way, various additional information areas, such as a mini map (211), an item (212), a quest area (213), etc., can be displayed on the content screen of the external display device (200), i.e., within the game screen. Some of these additional information areas can be areas that are selectively displayed or disappeared depending on user operation, and some can be areas that are maintained fixedly.

[0107] The display device (100) can receive pixel data therefor and display the entire map (111), item (112), and quest area (113). The processor (140) can display each of the additional information areas (111, 112, 113) in the same size and shape at the location within the original content screen, but when display option information is set through the setting UI as described above, the display layout, color, display position, size, etc. can be changed based on the display option information. In FIG. 5, compared to the additional information areas (211, 212, 213) displayed within the original content screen, the names of the areas (i.e., Map, Item, Quest) are added to each of the additional information areas (111, 112, 113) and the size of some areas (e.g., 111) is shown to be larger.

[0108] After this, even if the additional information area disappears within the content screen displayed on the external display device (200), the processor (140) can control the display (110) to maintain the display of the additional information area based on pixel data. Accordingly, the user can view the content screen that is not obscured by the additional information on the external display device (200) that is primarily being viewed, and, if necessary, can immediately check the additional information on the surrounding display device (100).

[0109] Meanwhile, according to another embodiment of the present disclosure, the processor (140) may identify a change in the content screen based on a change in pixel data.

[0110] For example, in the case of FIG. 5, the content screen may display the user's avatar and the background screen together. Here, when the user moves the avatar to the left, the avatar takes a motion of moving to the left, while the background screen moves to the right. The processor (140) can identify changes in the game screen by tracking the position of pixels for each image frame within the pixel data. For example, the position of the avatar within the mini-map (111) can be moved to the left. The processor (140) can also calculate the speed of the avatar's position movement by analyzing the rate of change over time of the pixels corresponding to the background screen. The processor (140) can change the position of the avatar on the mini-map (111) by considering the scale ratio between the size of the entire map and the mini-map, and the movement speed. Thereafter, when the mini-map is displayed again on the external display device (200), the processor (140) can update the mini-map again based on the pixel data.

[0111] Not limited to the map, the processor (140) may change at least one of the game title area, map area, item area, quest area, participant information area, and chat area based on the identified change. For example, if a scene such as an avatar acquiring a new item is included within the game screen, the processor (140) may add item information corresponding to the shape of the new item to the additional information area (112).

[0112] According to this embodiment, even if the user does not re-activate the additional information area on the external display device (200), the processor (140) can display the additional information area in various changes.

[0113] FIG. 6 is a drawing showing an example in which the display of the additional information area is maintained in a display device (100) according to at least one embodiment of the present disclosure.

[0114] FIG. 6 illustrates changes in operation of the external display device (200) and the display device (100). As described above, even if some of the additional information areas disappear within the content screen of the external display device (200), the processor (140) can control the display (120) to maintain the additional information areas within the external display device (200). For example, if the content screen is a game screen, even if the quest area and item areas (212, 213) disappear from the external display device (200), the display of the quest area and item areas (112, 113) displayed on the display device (100) can be maintained.

[0115] At this time, as described above in FIG. 5, the external display device (200) may identify a change in the game screen and change and display the additional information area displayed on the display device (100).

[0116] Meanwhile, in order to identify an additional information area based on pixel data, the display device (100) may be linked with at least one server device.

[0117] FIG. 7 is a drawing for explaining an example in which a display device (100) according to at least one embodiment of the present disclosure receives data from at least one server device.

[0118] At least one server device (700) may include a communication unit (not shown) capable of communicating with another server or display device (100), at least one processor (not shown) capable of processing data received from another server or display device (100), and at least one memory (not shown) capable of storing a program for processing data or processed data. The server device (700) may be implemented as various computing devices such as a workstation, a cloud, a data drive, a data station, etc. The server device (700) may be implemented as one or more servers that are physically or logically separated based on function, detailed configuration of function, or data, and may transmit and receive data through communication between each server and process the transmitted and received data.

[0119] The processor (140) can transmit the stored pixel data to at least one server device (700) connected to the communication interface (130). When the at least one server device (700) receives the pixel data, it can check whether there is an additional information area among the pixel data based on the server device (700)'s own database or an external database. For this purpose, various additional information may be stored in the server device (700)'s own database or the external database. For example, in the case of game content, the creators of each game content or portal servers, etc., can provide various image data regarding maps, items, quests, etc. within the game screen. Alternatively, in the case of video content, the creators of the video content, etc., can provide various image data such as highlight videos or profile pictures of characters.

[0120] The server device (700) groups each pixel in the pixel data into pixel blocks containing multiple pixels, and then calculates a representative pixel value for each pixel block. The representative pixel value may be an average pixel value, but is not limited thereto, and the representative pixel value may be determined based on various criteria. When a plurality of pixel blocks with similar representative pixel values ​​are continuously positioned to form a closed curve, the server device (700) recognizes a line connecting the pixel blocks forming the closed curve as an object within the content screen. The server device (700) can identify the shape, color, size, etc. of the object based on the shape of the line recognized as an object, the number of pixel blocks within the line, its representative pixel value, etc. The server device (700) compares the identified information with additional information stored in its own database or an external device to determine whether matching additional information exists.

[0121] As a result of the judgment, if there is matching additional information, the server device (700) can extract pixels corresponding to the corresponding part within the content screen as an additional information area.

[0122] However, the extraction of the additional information area is not limited to this method, and the server device (700) may extract the additional information area using an artificial intelligence model as described above, or may extract the additional information area by combining a rule base and an artificial intelligence model.

[0123] As described above, when the server device (700) extracts data for the additional information area from the content screen, the processor (140) can receive the data and display a setting UI (410) that sets the additional information area on the display device (100).

[0124] However, even if data isn't received from at least one server device, it's possible to extract and display data on-device. On-device refers to a device that processes all data directly on the user's device, such as a smartphone, tablet, or computer, without transmitting it to the cloud or an external server. When implemented on-device, additional information areas can be displayed even without an internet connection, enabling rapid processing without network delays.

[0125] For example, data on various additional information may be stored in advance in the memory (110).

[0126] The processor (140) can compare the stored pixel data with data for a plurality of stored additional information and extract an area matching the stored data as an additional information area.

[0127] FIG. 8 is a drawing for explaining an example in which a display device (100) according to at least one embodiment of the present disclosure displays the location of an avatar within a map.

[0128] If the content screen is a game screen including an avatar and a background screen that move according to user operation, and the additional information area is a map of the space where the avatar is located, the processor (140) can identify the movement direction and movement distance of the avatar based on changes in pixel data corresponding to the background screen among the pixel data. In addition, the position of the avatar within the map can be changed and displayed based on the identified movement direction and movement distance.

[0129] Referring to Fig. 8, when an avatar moves to the left within the background, the direction and distance of movement of the avatar can be identified through pixel data corresponding to the background. That is, when the position of an object (81, 82, 83) within the background that was on the left of the avatar is changed to the right of the avatar and displayed due to the movement of the avatar, the position of the avatar can be identified through the changed pixel data and the position of the avatar within the map can be changed and displayed from the right (t1) to the left (t2).

[0130] FIG. 9 is a drawing showing an example in which the display of the additional information area is maintained in a display device (100) according to at least one embodiment of the present disclosure.

[0131] When the content screen being displayed on the external display device (200) is a playback screen of video content, the display device (100) can display an additional information area for the video content playback screen.

[0132] For example, when video content is played and a progress bar (911) indicating the playback time is displayed on the playback screen of an external display device (200) by a user's operation, pixel data for this may be received to display a progress bar (912) indicating the playback time on the display (120). At this time, even if the progress bar indicating the playback time on the external display device (200) disappears, the display (913) of the progress bar displayed on the display device (100) may be maintained. In addition, when a text area (921) for leaving a comment on the video content is displayed, the text area (922) may be displayed together with the progress bar (914) that is currently being displayed.

[0133] However, this is not limited to this, and in addition to the progress bar, a recommended content area related to the video or a text area indicating text related to the content may also be displayed. This will be described in detail later in Figure 17.

[0134] FIG. 10 is a flowchart for explaining a display method of a display device according to at least one embodiment of the present disclosure.

[0135] Referring to FIG. 10, pixel data of a content screen being displayed on an external display device can be received (S1010). Specifically, pixel data can be received in a form where R, G, and B data are each expressed as numbers.

[0136] In addition, an additional information area included in a content screen can be extracted and displayed based on pixel data (S1020). For example, as described above, an additional information area included in a content screen displayed on an external display device can be extracted and displayed based on pixel data. The displayed additional information area can be maintained even if the display status of the external display device changes.

[0137] Additionally, at each preset update point, at least a portion of the additional information area may be updated and displayed according to changes in the content screen being displayed on the external display device (S1030). For example, if a map screen is temporarily displayed within the game screen, the display device (100) may change and display the avatar's location within the map screen at regular intervals (e.g., every 10 seconds).

[0138] Extraction and display of the additional information area have been specifically described in the above section, so a duplicate explanation will be omitted.

[0139] Additionally, as described above, the display device can provide various types of settings UIs. Through the settings UI, users can selectively display only the desired additional information areas among multiple additional information areas. Alternatively, the display options for the additional information areas displayed on the display device can also be varied.

[0140] The display method described in FIG. 10 can be performed by a display device (100) having the configuration shown in FIG. 2, but is not necessarily limited thereto, and can also be performed by a display device (100) having a configuration different from that of FIG. 2.

[0141] Additionally, it may be performed by a user terminal device (e.g., a set-top box, a PC body, etc.) or a projector device that is connected to an external display rather than a display device equipped with a display.

[0142] In addition to the case where a plurality of different display devices are connected to each other and used as described above, at least one embodiment of the present disclosure may be implemented in a display device having a single wide display.

[0143] For example, when a content screen is displayed in one area within a display, the processor may analyze the pixel values ​​of the content screen and separately display an additional information area within the content screen in another area within the same display.

[0144] Meanwhile, the various embodiments described above have been described assuming that one display device (100) is connected to an external display device (200), but a user may also use three or more display devices connected to each other.

[0145] FIG. 11 illustrates a state in which three display devices (200, 100-1, 100-2) are connected. Specifically, FIG. 11 assumes that two display devices (100-1, 100-2) are connected on both sides of an external display device (200), but the connection relationship and the number of devices may be varied. For convenience of explanation, the display device (200) that displays content is referred to as the main display device, and the two display devices (100-1, 100-2) are referred to as the first and second display devices (100-1), respectively.

[0146] According to FIG. 11, while displaying content including a plurality of additional information areas (211, 212, 213) on the main display device, the first and second display devices (100-1, 100-2) can display the additional information areas (111, 112, 113) separately from each other.

[0147] The analysis subject that detects the additional information area based on pixel data or the decision subject that determines the device to display the detected additional information area may be determined in various ways depending on the embodiment.

[0148] For example, the analysis subject may be the first display device (100-1) that receives pixel data from the main display device (200), or each of the first and second display devices (100-1, 100-2). Alternatively, the main display device (200) itself may be the analysis subject, or another external device (e.g., a server device, a PC, etc.) may be the analysis subject.

[0149] Alternatively, the analysis subject may be determined differently depending on the specifications, connection relationship, connection order, etc. of each device. For example, each device (200, 100-1, 100-2) may have different specifications such as resolution, display size, memory capacity, CPU performance, frame rate, brightness, etc. Each device (200, 100-1, 100-2) may transmit and receive information about these specifications and share them during the connection process. At least one of each device (200, 100-1, 100-2) may compare the shared information and determine the device with the best specifications as the analysis subject.

[0150] For example, if the first display device (100-1) determines that its specification information is the best, it may perform an operation of detecting an additional information area from pixel data received from the main display device (200). When determining the analysis subject, specification information may be compared among devices other than the main display device (200), but is not limited thereto, and the specification information of the main display device (200) may also be considered when making the decision.

[0151] The timing of selecting the analysis subject can vary, including but not limited to: content execution, device power-on, device connection, and device disconnection. For example, when a device is powered on, the specifications of each device can be compared and the device with the best specifications can be selected as the analysis subject. Furthermore, if a device is disconnected among multiple devices, the specifications of each device that remains connected can be compared when the device is disconnected, and the device with the best specifications can be selected as the analysis subject.

[0152] A device determined as an analysis subject (e.g., the first display device (100-1)) can detect an additional information area in the same manner as described above.

[0153] The additional information areas (211, 212, 213) detected in the above manner can be displayed in a distributed manner among the remaining display devices (100-1, 100-2) excluding the main display device (200). Whether to display which additional information area on which device can be determined by the above-described decision entity. In one example, the above-described analysis entity can be the decision entity, but is not necessarily limited thereto, and the decision entity can also be determined in various ways depending on the embodiment. For example, among the above-described specifications, a device with superior memory capacity or CPU performance can be determined as the decision entity, a device on which the above-described AI analysis APP is executed can be the decision entity, or a device that was most recently connected or first connected to the main display device (200) can be the decision entity.

[0154] In the case where the first display device (100-1) is determined as the decision subject in FIG. 11, the processor of the first display device (100-1) can control the display to transmit at least one of the plurality of additional information areas to another display device, i.e., the second display device (100-2), and display the remaining additional information areas.

[0155] FIG. 11 is a diagram illustrating an example of displaying an additional information area on three or more display devices according to at least one embodiment of the present disclosure.

[0156] If at least one other display device (100-1) is connected to an external display device (200) or a display device (100-2), and there are multiple extracted additional information areas (211, 212, 213), at least one of the multiple additional information areas can be transmitted to the other display device (100-1), and the remaining additional information areas can be displayed. At this time, the performance of the other display device (100-1) and the performance of the display device (100-2) can be compared, and an additional information area to be displayed on the other display device (100-1) among the multiple additional information areas can be determined based on the comparison result. At this time, the performance of the display device includes, but is not limited to, the size, resolution, refresh rate, etc. of the display device.

[0157] For example, if the additional information areas extracted from the external display device (200) include a map area (211), an item area (212), and a quest area (213), and a display device (100-2) and another display device (100-1) are connected to the external display device, the map area (111) can be displayed on the other display device (100-1) having a larger display screen size, and the item area (112) and the quest area (113) can be displayed on the display device (100-2) having a smaller display screen size. That is, by displaying the map area (111) on the other display device (100-1) having a larger screen size, the user can conveniently enjoy the game content.

[0158] At this time, the number of other display devices (100-1) connected to the external display device (200) or the display device (100-2) may be identified, and based on the number of other display devices (100-1) and the number of additional information areas, the additional information areas to be displayed on the display device (100-2) and the other display device (100-1) may be distributed. At this time, as described above, the performance of the display devices may be compared to distribute the additional information areas.

[0159] FIGS. 12 to 16 are diagrams showing examples of locations for displaying additional information areas in a display device according to at least one embodiment of the present disclosure.

[0160] The processor (140) can control the display (120) to display the extracted additional information area at a location corresponding to the display location of the additional information area within the content screen. Referring to FIG. 12, when the quest area (213) and the item area (212) are displayed on the external display device (200), the quest area (113) and the item area (112) can be displayed at the location corresponding to the displayed location among the locations of the display device (100).

[0161] On the other hand, the processor (140) may control the display (120) to display the additional information area with a different layout depending on the type of menu or function for displaying the additional information area within the content. Referring to FIG. 12, the display positions of the quest area (113) and the item area (112) displayed on the display device (100) may be changed, or the colors, sizes, etc. may be changed to display them. In this case, the display may be controlled to display the additional information area with a different layout depending on the properties of the content.

[0162] In addition, if there are multiple extracted additional information areas, the processor (140) can control the display (120) to sequentially arrange and display the extracted additional information areas according to the display order of each additional information area within the content screen. Referring to FIG. 13, when the quest area (213) is displayed on the external display device (200) and then the item area (212) is displayed, the quest area (113) and the item area (112) can be sequentially displayed on the screen of the display device (100).

[0163] Meanwhile, as shown in FIG. 12, the additional information area displayed on the external display device (200) may be displayed as is on the display device (100), or as shown in FIG. 13, the display of the quest area (213) may disappear on the screen of the external display device (200), but may be displayed as is on the screen of the display device (100), so that the screen displays of the external display device (200) and the display device (100) may be different. In addition, depending on the properties of the menu or function of the additional information area, all or part of the additional information area may be displayed on the external display device (200) or the display device (100). In addition, when a quest window is clicked in the game content, the convenience of the content user may be improved by displaying it only on the display device (100) and not on the external display device (200). Specific details regarding this will be described later in FIG. 16. If there are multiple extracted additional information areas, the processor (140) can control the display (120) to sequentially stack and display the multiple extracted additional information areas at the same location according to the display order of each additional information area within the content screen. Referring to FIG. 14, when the quest area (213) is displayed on the external display device (200) and then the item area (212) is displayed, the quest area (113) may be displayed on the display device (100) and then the item area (112) may be stacked and displayed at the same location. At this time, the additional information areas displayed in a stacked manner may be stacked and displayed in an opaque state or may be stacked and displayed in a semi-transparent state.

[0164] If there are multiple extracted additional information areas, the processor (140) can control the display (120) to sequentially display the extracted additional information areas according to the frequency of use of each additional information area within the content screen. Referring to FIG. 15, if the user uses the quest area (113) the most and the map area (211) the least among the quest area (213), item area (212), and map area (211) displayed on the external display device (200), the display device (100) can display the quest area (113), item area (112), and map area (111) in that order from the left. At this time, the frequency of use of the additional information area may be a value calculated based on at least one of the on-off number or usage time of the corresponding additional information area.

[0165] FIG. 16 is a drawing showing an example of a screen of a display device (100) in which an additional information area is displayed when an additional information window is clicked on an external display device (200).

[0166] Specifically, when the quest window (214) is clicked, the quest area (213) is not displayed on the screen of the external display device (200), but is displayed only on the screen of the display device (100), so that the screen displays of the external display device (200) and the display device (100) may be different. However, this is not limited to this, and depending on the menu for displaying the additional information area or the performance of the display device, the additional information area may be displayed only on the screen of the external display device (200).

[0167] Additionally, although not illustrated in FIG. 16, when displaying an additional information area on three or more display devices, the additional information area may be selectively displayed only on the screens of some display devices. For example, referring to FIG. 11, when clicking on a map window, the map area may be displayed only on the display device (100-1) with the larger screen among the display devices.

[0168] FIG. 17 is a diagram illustrating an example of displaying an additional information area in a display device according to at least one embodiment of the present disclosure.

[0169] Referring to FIG. 17, when a video content screen is displayed on an external display device (200), various additional information related to the video being played can be displayed on the display device (100). For example, when the video content is a video about a specific drama, a recommended video (114) that recommends a drama that has a similar storyline to the currently playing drama or at least one main character can be displayed on the display device (100). In addition, character information about the characters and information about fashion products such as accessories and clothing worn by the main characters can be displayed in the form of content information (115). However, the present invention is not limited thereto, and the recommended video can display not only dramas and movies, but also various videos in which the main characters appeared.

[0170] FIG. 18 is a drawing showing an example of displaying an additional information area in a projector display device according to at least one embodiment of the present disclosure.

[0171] Referring to FIG. 18, some of the plurality of display devices may be implemented as projector display devices (100-3). Specifically, when a plurality of additional information areas (211, 212, 213) are displayed on an external display device (200), only the quest area (113) and the item area (112) may be displayed on the display device (100-2), and the map area (116) may be displayed by the projector display device (100-3). However, this is not limited thereto, and the additional information areas may be distributed and displayed depending on the performance of the display devices.

[0172] FIG. 19 and FIG. 20 are flowcharts for explaining a display method of a display device according to at least one embodiment of the present disclosure.

[0173] Referring to FIG. 19, the display device can receive pixel data of a content screen being displayed on an external display device (S1910). Specifically, the display device can receive pixel data from the external display device in the form of R, G, and B data each expressed as a number.

[0174] In addition, the display device can extract an additional information area included in the content screen based on pixel data (S1920). For example, the display device can extract and display an additional information area included in the content screen displayed on an external display device based on pixel data as described above. The displayed additional information area can be maintained even if the display status of the external display device changes.

[0175] Additionally, the display device can identify whether other display devices exist (S1930), and if so, the number of other display devices can be identified (S1940). At this time, the display device can transmit additional information areas based on the number of identified other display devices and the number of additional information areas (S1950), and can display some of the additional information areas (S1960). Additionally, if the display device identifies that no other display devices exist, the display device can display the entire additional information area (S1941).

[0176] Referring to FIG. 20, the display device can receive pixel data of a content screen being displayed on an external display device (S2010). Specifically, the display device can receive pixel data from the external display device in the form of R, G, and B data each expressed as a number.

[0177] In addition, the display device can extract an additional information area included in the content screen based on pixel data (S2020). For example, the display device can extract and display an additional information area included in the content screen displayed on an external display device based on pixel data as described above. The displayed additional information area can be maintained even if the display status of the external display device changes.

[0178] Additionally, the display device can compare the performance of the display device (S2030). At this time, the performance of the display device may include the size, resolution, refresh rate, etc. of the display device.

[0179] In addition, the display device can identify whether the performance of another display device is superior (S2040), and if the performance of the other display device is identified as superior, the display device can transmit the additional information area to the other display device (S2050). On the other hand, if the performance of the other display device is identified as not superior, the display device can display the additional information area (S2051). At this time, if the additional information areas being transmitted or displayed are multiple additional information areas, all of the multiple additional information areas can be transmitted or displayed, or only a portion of the additional information areas can be transmitted or displayed.

[0180] The display method described in FIG. 10, FIG. 19 and FIG. 20 can be performed by a device having various configurations such as the above-described FIG. 2, but is not necessarily limited thereto, and can also be performed by a device having various configurations.

[0181] The various embodiments described above may be implemented as a single embodiment, or at least one embodiment may be combined with each other in whole or in part and implemented together in one device.

[0182] According to the various embodiments described above, by utilizing graphics output from an external display device to display additional information areas on the display device, the obstruction of the external display device's view can be minimized. Ultimately, this can enhance the user's immersion and convenience in viewing content.

[0183] Various embodiments of the present disclosure may be implemented as software stored in a machine-readable storage media that can be installed or connected to a smartphone, a user terminal device, or other various electronic devices (e.g., a computer).

[0184] Specifically, a non-transitory readable storage medium may be provided that stores software for sequentially performing a step of receiving pixel data of a content screen being displayed on an external display device and a step of extracting and displaying an additional information area included in the content screen based on the pixel data.

[0185] A device equipped with such a non-transitory readable medium can perform various operations, such as acquiring pixel data, identifying the direction and distance of movement of an avatar, extracting and displaying an additional information area, and changing the display of an additional information area, as described in the various embodiments described above.

[0186] In the context of non-transitory readable storage media, 'non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily on the storage medium.

[0187] Alternatively, a program for performing the method according to the various embodiments described above may be distributed online through an application store. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated on a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0188] Each component (e.g., a module or a program) according to various embodiments may be composed of one or more entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., a module or a program) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0189] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In the display device, display; Processor; including; The above processor, Display the extracted additional information area, Controlling the display to update at least a portion of the additional information area displayed on the display according to changes in the content screen being displayed on the external display device at each preset update point; The above additional information area is: A display device, which is included in the content screen being displayed on the external display device, and is extracted from the content based on pixel data of the content screen.

2. In paragraph 1, The above preset update time is, At least one of the time when the preset cycle arrives and the time when a menu or function for displaying the additional information area within the content is used, The above preset cycle is a display device that is set differently for each menu or function.

3. In paragraph 1, further comprising a communication interface; If the external display device or at least one other display device is connected to the display device, and the extracted additional information area is more than one additional information area, The above processor, Display at least one additional information area among the additional information areas exceeding the above one, A display device that transmits at least one additional information area among the additional information areas exceeding the above one to each display device among the at least one other display device through the communication interface, and controls the display to display the remaining additional information areas.

4. In paragraph 1, The above additional information area includes more than one additional information area, The above processor, Identify the number of said external display devices or other display devices connected to said display device, A display device that distributes more than one additional information area to be displayed on each of the display device and the other display devices based on the number of the other display devices and the number of the additional information areas.

5. In paragraph 1, If the external display device or at least one other display device is connected to the display device, The above processor, A display device that compares the performance of the display device with the performance of at least one other display device, and determines a display device to display the additional information area based on the comparison result.

6. In paragraph 1, The above processor, A display device that controls the display to display the additional information area with a different layout from the content screen depending on the type of menu or function for displaying the additional information area.

7. In paragraph 1, The above processor, A display device that controls the display so that, if the extracted additional information area exceeds one, the extracted additional information areas are sequentially arranged and displayed according to the display order of each additional information area within the content screen.

8. In paragraph 1, The above processor, A display device that controls the display so that, if the extracted additional information area exceeds one, the extracted additional information areas exceeding one are sequentially stacked and displayed at the same location according to the display order of each additional information area within the content screen.

9. In the first paragraph, the processor A display device that controls the display to sequentially display the extracted additional information areas exceeding one, depending on the frequency of use of each additional information area within the content screen, if the extracted additional information area exceeds one.

10. In paragraph 1, The above content screen includes more than one additional information area, The above processor, A display device that controls the display to display a setting UI for setting at least one additional information area among the additional information areas exceeding the above one to the extracted additional area information.

11. In paragraph 1, The above processor, Controlling the display to display the additional information area based on setting information for at least one of the display layout, color, display position, and size of the additional information area; A display device that controls the display so as to maintain the display of the additional information area even if the additional information area disappears within the content screen being displayed on the external display device.

12. In paragraph 1, If the above content screen is a game screen, the additional information area includes at least one of a game title area, a map area, an item area, a quest area, a participant information area, and a chat area. The above processor, A display device that, when a change in the game screen is identified based on the pixel data, changes at least one of the game title area, map area, item area, quest area, participant information area, and chat area according to the identified change.

13. In paragraph 1, The above content screen is a playback screen for video or music content, If the area selectively displayed on the content screen according to the user operation is at least one area among a progress bar indicating a playback point within the playback screen, a recommended content area indicating at least one recommended content related to the video or music content, and a text area indicating text related to the video or music content, A display device wherein the processor identifies the optionally displayed area as the extracted additional information area.

14. In a display method of a display device including a display, A step of displaying an extracted additional information area based on pixel data; and A step of updating at least a part of the additional information area displayed on the display according to a change in the content screen being displayed on the external display device at each preset update point; The above additional information area is: A display method, which is included in the content screen displayed on the external display device and extracted from the content based on pixel data of the content screen.

15. In the display device, display; communication interface; and Processor; including; The above processor, Obtain pixel data of the content screen being displayed on an external display device through the above communication interface, Extracting an additional information area included in the content screen based on the stored pixel data, Display the extracted additional information area, A display device that controls the display to update and display at least a portion of the additional information area according to changes in the content screen being displayed on the external display device at each preset update point.

Images