Display device and operating method thereof

Abstract

A display device according to one embodiment comprises: a display; at least one camera acquiring line-of-sight information of a user; a memory storing one or more instructions; and at least one processor including a processing circuit. When the one or more instructions are individually or collectively executed by the at least one processor, the display device may: display an image corresponding to image content on the display; identify an information area in the image on the basis of image content information; identify a gaze area corresponding to the line-of-sight information of the user in the image; and output a sub-image corresponding to the information area on the basis of matching of the information area and the gaze area.

Description

Display device and method of operation thereof

[0001] Various embodiments relate to a display device and a method of operation thereof that identify specific information included in a displayed image and provide an image corresponding to the identified specific information.

[0002] Due to technological advancements in display devices, display devices capable of implementing various functions are being developed. A representative example of a display device is the television. Conventional TVs simply received broadcast signals and performed the function of playing broadcast content corresponding to the received signals (e.g., news, dramas, entertainment programs, etc.). Recently, however, due to technological advancements in display devices, various applications or programs can be stored and installed on the TV to perform diverse functions. Through these installed applications, the TV can provide various functions and services in addition to playing broadcast content. For example, a game application can be stored and installed on the TV, allowing game content to be executed through the application.

[0003] In addition, as communication technology or communication connection functions of the display device are developed, the display device can provide various functions or services through wired or wireless communication with an external device. For example, the display device is connected to an external device (e.g., a source device, a game console device) via wired or wireless communication, and can receive content (e.g., game content) from the external device (e.g., a source device, a game console device) through the aforementioned wired or wireless communication.

[0004] When providing game content through a display device, the device needs to output the content in a way that enhances user satisfaction and convenience. For example, the display device can perform functions such as magnifying and displaying specific information included in the game video or sharing it with an external device. To improve usability when magnifying or sharing specific information included in the game video, a method is required to automatically recognize specific information without user input.

[0005] A display device according to one embodiment may include a display, at least one camera for acquiring user gaze information, a memory for storing one or more instructions, and at least one processor including a processing circuit.

[0006] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can display an image corresponding to the image content on the display.

[0007] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can identify an information region in the image based on the image content information.

[0008] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can identify a gaze area corresponding to the user's gaze information in the image.

[0009] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can output a sub-image corresponding to the information area based on the fact that the information area and the gaze area coincide.

[0010] A method of operating a display device according to one embodiment may include the step of displaying an image corresponding to image content.

[0011] A method of operating a display device according to one embodiment may include the step of identifying an information region in the image based on the image content information.

[0012] A method of operating a display device according to one embodiment may include the step of identifying a gaze area corresponding to user gaze information in the image.

[0013] A method of operation of a display device according to one embodiment may include the step of outputting a sub-image corresponding to the information area based on the fact that the information area and the gaze area coincide.

[0014] FIG. 1 is a drawing showing a display device and a source device according to one embodiment.

[0015] FIG. 2 shows game images corresponding to various game contents according to one embodiment.

[0016] FIGS. 3a and FIGS. 3b show game images having various aspect ratios according to one embodiment.

[0017] FIG. 4 is a diagram illustrating the operation of a display device according to one embodiment to enlarge or share a portion of an image.

[0018] FIG. 5 is a drawing showing a user interface screen for setting a game mode of a display device according to one embodiment.

[0019] FIG. 6 is a flowchart illustrating the operation method of a display device according to one embodiment.

[0020] FIGS. 7 and 8 are drawings illustrating the operation of a display device according to one embodiment identifying an information region in an image.

[0021] FIG. 9 is a diagram showing the operation of a display device according to one embodiment identifying a gaze area corresponding to the user's gaze information.

[0022] FIG. 10 is a diagram illustrating an operation for determining whether an information area identified in an image according to one embodiment matches a user's gaze area.

[0023] FIG. 11 is a diagram showing the operation of a display device displaying a confirmation message according to one embodiment.

[0024] FIG. 12 is a diagram showing the operation of a display device according to one embodiment displaying a menu for selecting to expand or share an information area.

[0025] FIG. 13 is a diagram showing the operation of a display device according to one embodiment.

[0026] FIG. 14 is a block diagram showing the configuration of a source device according to one embodiment.

[0027] FIG. 15 is a block diagram showing the configuration of a display device according to one embodiment.

[0028] FIG. 16 is a block diagram showing the configuration of a display device according to one embodiment.

[0029] The terms used in this specification will be briefly explained, and the invention will be described in detail.

[0030] The terms used in this invention have been selected based on currently widely used general terms, taking into account their functions within the invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0031] When a part of a specification is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "...part" or "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0032] The following describes embodiments with reference to the attached drawings so that those skilled in the art can easily implement the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0033] In the embodiments of this specification, the term "user" refers to a person who controls a system, function, or operation, and may include a developer, administrator, or installer.

[0034] Additionally, in the embodiments of this specification, 'image' or 'picture' may represent a still image, a video composed of a plurality of consecutive still images (or frames), or a video.

[0035] FIG. 1 is a drawing showing a display device and a source device according to one embodiment.

[0036] Referring to FIG. 1, a display device (100) according to one embodiment may be connected to a source device (200) via wired or wireless communication.

[0037] A source device (200) according to one embodiment may provide content such as video or audio to a display device (100). For example, the source device (200) may be a video game console. However, it is not limited thereto, and the source device (200) may include various types of electronic devices capable of providing content to the display device (100), such as a set-top box, DVD player, Blu-ray disc, PC, game console, etc. The source device (200) is a device that provides content and may be referred to as a source device, but may also be referred to as a host device, content providing device, electronic device, storage device, computing device, server device, server, etc.

[0038] A display device (100) according to one embodiment can output or display content received from a source device (200). The display device (100) may include various types of electronic devices capable of receiving and outputting content, such as, for example, a network TV, a smart TV, an internet TV, a web TV, an IPTV, a PC, etc. The display device (100) is a device that receives and displays content and may be referred to as a content receiving device, a sink device, an electronic device, or a computing device.

[0039] The source device (200) and the display device (100) can transmit and receive content via wired communication. For example, the source device (200) and the display device (100) can be connected via a cable, and each of the source device (200) and the display device (100) may include one or more ports for cable connection. The one or more ports may include digital input interfaces such as, for example, an HDMI port, a DisplayPort, or Type-C.

[0040] Alternatively, the source device (200) and the display device (100) may transmit and receive content via wireless communication. For example, the source device (200) and the display device (100) may include at least one communication module that performs communication according to communication standards such as wireless HDMI communication, Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC / RFID, Wi-Fi Direct, UWB, ZIGBEE, the Internet, 3G, 4G, 5G and / or 6G, and may be connected through the communication module to transmit and receive content.

[0041] For convenience of explanation, the following description will be given as an example in which a source device (200) and a display device (100) are connected via an HDMI cable to perform wired HDMI communication.

[0042] For example, the source device (200) and the display device (100) each include an HDMI port and can communicate through an HDMI cable connected to the port. When the source device (200) provides content to the display device (100), the source device (200) first receives an EDID (Extended Display Identification Data) provided by the display device (100), and can generate content in a format corresponding to the received EDID and provide it to the display device (100).

[0043] The EDID may include data or information defining display performance or display specifications that enable the display device (100) to display content. For example, the EDID may include at least one of information about the manufacturer of the display device, product information, EDID version information, timing information, information about screen size, information about brightness, and information about pixels. Here, the information about screen size may include resolution, aspect ratio, etc., that the display device can provide.

[0044] The source device (200) can acquire an EDID provided by the display device (100), generate a format of content to be transmitted to the display device (100) based on the acquired EDID, and provide it to the display device (100). For example, if the aspect ratio identified according to the acquired EDID (e.g., the value of the identified aspect ratio) is 32:9, the source device (200) can generate an image having the identified aspect ratio value of 32:9 and transmit it to the display device (100). That is, the source device (200) that receives the EDID from the display device (100) can provide content in a format suitable for the display performance of the display device (100) to the display device (100).

[0045] Meanwhile, a display device (100) according to one embodiment may support output of various aspect ratios. Here, the aspect ratio represents the ratio of the vertical length to the horizontal length of the screen and may be expressed as a value such as 4:3, 16:9, 16:10, 21:9, 32:9, etc. Additionally, the aspect ratio represents the ratio of the horizontal and vertical lengths of the output screen or image and may be referred to as 'screen ratio', 'image ratio', etc. The aspect ratio of the screen may be determined according to the type of input image.

[0046] A display device (100) according to one embodiment can receive and display various content from a source device (200). For example, the display device (100) can receive various types of game content from the source device (200) and display a game screen or game video. Game videos corresponding to various game content will be described with reference to FIG. 2.

[0047] Additionally, the display device (100) can receive and display images of various aspect ratios even for the same content. For example, the display device (100) can receive and display game screens or game images having different aspect ratios corresponding to the same game content from the source device (200). Game images having various aspect ratios will be described with reference to FIGS. 3a and 3b.

[0048] A display device (100) according to one embodiment can identify specific information (e.g., a sub-window) included in a game image based on title information of game content and the aspect ratio of a game image, and output an image corresponding to the identified specific information. This will be explained in detail below with reference to the drawings.

[0049] FIG. 2 shows game images corresponding to various game contents according to one embodiment.

[0050] Referring to FIG. 2, a display device (100) according to one embodiment can receive various game content from a source device (200). For example, the game content may be received in the form of a stream, packet, etc., including audio and / or video data. Additionally, a stream or packet corresponding to the game content may include a plurality of images. Here, the images may include images displayed for the playback of the game content, such as a playback screen or a menu screen. For example, the images may be in the form of an image frame, a frame, a group of pictures (GOP), etc.

[0051] The display device (100) can display game images corresponding to various game content received from the source device (200).

[0052] Additionally, the game content provided by the source device (200) may include 3D game content. A 3D game may refer to a game that allows a user to enjoy the game within a 3D space (e.g., a 3D space that represents a virtual three-dimensional space and shape). For example, a user playing a 3D game may proceed with the game by engaging in competition, racing, combat, fighting, or war with virtual characters, etc., within a predetermined point within the 3D space. Alternatively, the game content may include general games (2D games) that are played within a 2D space, in addition to content that is played within a 3D space such as a 3D game.

[0053] In the case of 3D games or general games, the game image may include a separate sub-window distinct from the game playback image. For example, the sub-window may be displayed in a partial area of ​​the game playback screen, and the area where the sub-window is displayed may be referred to as an information area. However, it is not limited thereto.

[0054] Sub-windows may include information related to game progress and services for game participants (e.g., chat windows). Sub-windows may include a mini map, skill window, item-related window, character-related window, chat window, control-related window, etc. Additionally, the types of sub-windows included in the game video may vary depending on the type of game.

[0055] Specifically, the minimap may include a small image displaying a map that represents the entire space where the game is played. The skill window may be a status window that displays the cooldown time of skills. The item-related window may be a window that provides information about items used in the game. For example, the item-related window may include an item inventory window. The item inventory window may display the types of items the user possesses and / or their reuse times, etc. The character-related window may be a window that provides information about the character performing the game. For example, the character-related window may include a character status window. In games where there are allies and enemies, such as battle games, the character status window may be a window that provides the HP (Health Point) and / or MP (Mana Point), respawn time, etc., of each ally and / or enemy. The chat window may be a window that displays the chat content of multiple people participating in or spectating the game. Alternatively, the chat window may be a window that displays messages related to the game.

[0056] A control-related window may include a window for providing information related to the cockpit or driver's seat in a game where the game user controls an object used in the game (e.g., a flying object, a car, etc.). For example, the control-related window may include a flight simulation cockpit window for providing information necessary or provided during flight or operation. The flight simulation cockpit window may display air speed, engine status, fuel, flaps, trim, altitude status, etc.

[0057] A sub-window according to one embodiment may be displayed in a separately partitioned area on the game image. Alternatively, the sub-window may be displayed in an opaque or translucent form on a partial area of ​​the game image.

[0058] A display device (100) according to one embodiment may provide a function to display an enlarged image of an information area where a sub-window is displayed in a game video, or to share it with an external device. For example, the display device (100) may display an enlarged image of an area where a mini-map is displayed on a display, or to share it with an external device.

[0059] Referring to FIG. 2, depending on the type of game content (e.g., game title), the size and display location of a sub-window (e.g., mini-map) in the game image may differ.

[0060] For example, a display device (100) may receive a first game content from a source device (200) and display a first game image (210) corresponding to the first game content. At this time, in the first game image (210), a first mini-map (215) may be displayed at a first position with a first size.

[0061] The display device (100) can set the first mini map (215) as an enlarged or shared area based on user input. At this time, a specific object (e.g., a box border, etc.) may be displayed in the area set as the enlarged or shared area. However, it is not limited thereto.

[0062] When the first mini map (215) is set as an enlarged or shared area, the display device (100) can display an enlarged image of the first mini map (215) or share it with an external device.

[0063] When the game content received from the source device (200) changes from the first game content to the second game content, the display device (100) can display a second game image (220) corresponding to the second game content. At this time, the second mini-map (225) in the second game image can be displayed at a second location with a second size, and may differ from the size and location of the first mini-map (215).

[0064] In order to reconfigure the second mini-map (225) into an enlarged or shared area, it is necessary to move a specific object displayed at the location of the first mini-map (215) to the location of the second mini-map (225). Accordingly, the user has the inconvenience of having to reconfigure the enlarged or shared area whenever the game content changes, which may interfere with the game.

[0065] FIGS. 3a and FIGS. 3b show game images having various aspect ratios according to one embodiment.

[0066] Referring to FIG. 3a, a display device (100) according to one embodiment can display the same game content as game images having various aspect ratios. When the aspect ratios of the game images are different, different field of view (FOV) can be provided. For example, as the aspect ratio of the game images increases in the direction of expansion to the left and right, a wider field of view can be provided. For example, a second game image (320) having an aspect ratio of 21:9 can provide a wider field of view than a first game image (310) having an aspect ratio of 16:9. The second game image (320) may include additional images of the left area (321) and the right area (322) compared to the first game image (310).

[0067] Additionally, the third game video (330) having a 32:9 aspect ratio can provide a wider field of view than the second game video (320) having a 21:9 aspect ratio. The third game video (330) may include more images of the left area (331) and the right area (332) compared to the second game video (320). Therefore, the third game video (330) having a 32:9 aspect ratio can provide more information to the game content user than the second game video (320) having a 21:9 aspect ratio or the first game video (310) having a 16:9 aspect ratio. When playing a game while watching a game video that provides a wider field of view, one can enjoy a more vivid and immersive game.

[0068] Meanwhile, referring to FIG. 3a, even for the same game content, the size and display position of a sub-window (e.g., a mini-map) may differ depending on the aspect ratio of the game image. In one embodiment, the size and display position of the mini-map may be determined according to the mini-map reference aspect ratio. In one embodiment, the mini-map reference aspect ratio may be the same as the aspect ratio of the game image.

[0069] For example, when a display device (100) displays a first game image (310) having an aspect ratio of 16:9, the first mini map (315) may be displayed based on an aspect ratio of 16:9 that is the same as the aspect ratio of the first game image (310). As shown in FIG. 3a, the first mini map (315) may be displayed at a first position with a first size.

[0070] The display device (100) can set the first mini map (315) as an enlarged or shared area based on user input. At this time, a specific object (e.g., a box border, etc.) may be displayed in the area set as the enlarged or shared area. However, it is not limited thereto. When the first mini map (315) is set as an enlarged or shared area, the display device (100) can display an enlarged image of the first mini map (315) or share it with an external device.

[0071] When the aspect ratio of the game image displayed on the display changes, for example, when changing from a first game image (310) of 16:9 to a second game image (320) of 21:9, the display device (100) can display the second game image (320) having an aspect ratio of 21:9. The second mini-map (325) can be displayed based on an aspect ratio of 21:9, which is the same as the aspect ratio of the second game image (320). As shown in FIG. 3a, the second mini-map (325) has a second size and can be displayed at a second location. At this time, the size and location of the second mini-map (325) may differ from the size and location of the first mini-map (315).

[0072] In order to enlarge or reset the second mini-map (325) to a shared area, it is necessary to move a specific object displayed at the location of the first mini-map (315) to the location of the second mini-map (325).

[0073] Alternatively, when changing from a second game video (320) of 21:9 to a third game video (330) of 32:9, the display device (100) may display the third game video (330) having an aspect ratio of 32:9. The third mini-map (335) may be displayed based on an aspect ratio of 32:9, which is the same as the aspect ratio of the third game video (330). As shown in FIG. 3a, the third mini-map (335) may have a third size and may be displayed at a third location. At this time, the size and location of the third mini-map (335) may differ from the size and location of the second mini-map (325).

[0074] In order to reconfigure the third mini-map (335) into an enlarged or shared area, it is necessary to move a specific object displayed at the location of the second mini-map (325) to the location of the third mini-map (335). Accordingly, the user has the inconvenience of having to reconfigure the enlarged or shared area whenever the aspect ratio of the game video changes, which may interfere with the game.

[0075] In addition, referring to Fig. 3b, even if the aspect ratio of the game video is the same, the size and display position of the minimap may differ depending on the reference aspect ratio of the sub-window (e.g., minimap).

[0076] For example, if the aspect ratio of the fourth game video (340) is 32:9 and the standard aspect ratio of the minimap is 16:9, the display device (100) can display the fourth minimap (345) in the fourth game video (340) based on the aspect ratio of 16:9. As shown in FIG. 3b, the fourth minimap (345) has a fourth size and can be displayed at a fourth position.

[0077] Additionally, if the aspect ratio of the fifth game video (350) is 32:9 and the standard aspect ratio of the mini map is 21:9, the display device (100) can display the fifth mini map (355) in the fifth game video (350) based on the aspect ratio of 21:9. As shown in FIG. 3b, the fifth mini map (355) has a fifth size and can be displayed at a fifth position.

[0078] Additionally, if the aspect ratio of the sixth game video (360) is 32:9 and the standard aspect ratio of the mini map is 32:9, the display device (100) can display the sixth mini map (365) in the sixth game video (360) based on the aspect ratio of 32:9. As shown in FIG. 3b, the sixth mini map (365) has a sixth size and can be displayed at a sixth position.

[0079] Even if the aspect ratio of the game video is the same, if the standard aspect ratio of the minimap is different, the position and size of the minimap may change. When the minimap is set to a zoomed-in or shared area, users face the inconvenience of having to reset the zoom or shared area whenever the standard aspect ratio of the minimap changes, which can interfere with gameplay.

[0080] A display device (100) according to one embodiment can automatically identify an area where specific information, such as a sub-window, is displayed in an image corresponding to the changed content or in an image having a changed aspect ratio when the received content is changed, the aspect ratio of the game image is changed, or the reference aspect ratio of an area where specific information is displayed, such as a sub-window, is changed. For example, when the game content is changed, the display device (100) according to one embodiment can automatically identify a mini-map in an image corresponding to the changed content. Alternatively, when the aspect ratio of the game image is changed, the display device (100) according to one embodiment can automatically identify a mini-map in an image having a changed aspect ratio. Alternatively, when the reference aspect ratio of an area where specific information is displayed (information area) is changed, the display device (100) according to one embodiment can automatically identify a mini-map in a game image.

[0081] FIG. 4 is a diagram illustrating the operation of a display device according to one embodiment to enlarge or share a portion of an image.

[0082] Referring to FIG. 4, a display device (100) according to one embodiment can receive game content from a source device (200) and display a game image (410) corresponding to the game content.

[0083] The display device (100) can display a game-related menu on the game video (410). For example, the display device (100) can receive user input requesting the display of a game-related menu during the playback of the game video (410). The display device (100) can display a menu including a first item (420) indicating a function to enlarge a specific area included in the game video (410) and a second item (430) indicating a function to share a specific area included in the game video (410).

[0084] Based on user input selecting the first item (420) or the second item (430), the display device (100) can enlarge or share a portion of the game image (410).

[0085] A display device (100) according to one embodiment may set a portion of a game image (410) as an enlarged or shared area based on user input. For example, the display device (100) may display a specific object (440, e.g., a border) on the game image (410). The display device (100) may set an enlarged or shared area based on user input to move the specific object (440) or to adjust the size of the specific object (440). For example, the display device (100) may set an area where a mini-map is displayed as an enlarged or shared area.

[0086] Alternatively, a display device (100) according to one embodiment may obtain title information of game content by analyzing a game image (410), and identify an information area such as a mini-map based on the title information. The display device (100) may set the identified area as an enlarged or shared area.

[0087] Alternatively, a display device (100) according to one embodiment may detect at least one object included in the game image (410) by analyzing the game image (410), and identify an information area such as a mini-map based on the characteristics of the detected object. The display device (100) may set the identified area as an enlarged or shared area. However, it is not limited thereto.

[0088] When the display device (100) magnifies the area (440) where the mini map is displayed, the image corresponding to the area (440) where the mini map is displayed can be magnified and displayed in the magnification window (450). At this time, the magnification window (450) may be displayed in an area separate from the game image (410) and may be displayed in any one of the left, right, top, or bottom areas of the game image (410).

[0089] Additionally, if the display device (100) shares the area (440) where the mini-map is displayed with an external device, it can transmit an image corresponding to the area (440) where the mini-map is displayed to the external device (460). The external device (460) can display the image received from the display device (100) on the display of the external device (460).

[0090] FIG. 5 is a drawing showing a user interface screen for setting a game mode of a display device according to one embodiment.

[0091] Referring to FIG. 5, a display device (100) according to one embodiment may display a setting menu screen (510) for setting various functions provided in game mode. The setting menu screen (510) may include a menu (520) for determining whether to fix or expand the screen display and a menu (530) for setting the screen ratio (or aspect ratio). Additionally, the setting menu screen (510) may include a menu (540) for setting the display position of the zoom screen. Here, the zoom screen may refer to a window in which an image of a magnified portion is displayed when performing a zoom function to magnify a portion of the game image. For example, the zoom screen may be displayed in any one of the top, bottom, left, or right regions of the game image. However, it is not limited thereto.

[0092] Additionally, the settings menu screen (510) may include a menu (550) for setting an external device to share a portion of the area when performing the function of sharing a portion of the game video.

[0093] Additionally, the settings menu screen (510) may include a menu (560) for setting the On / Off status of the information area automatic detection function. The information area automatic detection function may include a function for automatically detecting information areas, such as mini maps, in an image displayed on the display device (100). Alternatively, it may include a function for automatically detecting changed information areas when the location or size of information areas, such as mini maps identified within the image, changes, such as when the image content changes or the aspect ratio of the image changes.

[0094] Additionally, the settings menu screen (510) may include a menu (570) for setting the On / Off status of the eye tracking function for the information area. The eye tracking function for the information area may include a function to acquire the user's gaze information and, based on the user's gaze information, estimate the area the user is looking at as an information area such as a mini-map.

[0095] A display device (100) according to one embodiment can enlarge and display an information area or share it with an external device when both the information area automatic detection function and the eye tracking function are set to On, and the information area automatically detected in the image displayed on the display matches the gaze area obtained based on the user's gaze information. This will be explained in detail below with reference to the drawings.

[0096] FIG. 6 is a flowchart illustrating the operation method of a display device according to one embodiment.

[0097] Referring to FIG. 6, a display device (100) according to one embodiment can display an image corresponding to image content (S610).

[0098] For example, a display device (100) can receive video content from a source device (200). According to one embodiment, the video content may be game content, movie content, music content, home training service content, broadcast content, etc. Hereinafter, the video content will be described as game content.

[0099] The display device (100) can display a game video corresponding to the game content.

[0100] A display device (100) according to one embodiment can identify an information area in an image (S620).

[0101] According to one embodiment, the information area may be an area where at least one sub-window included in the game image is displayed. The display device (100) may identify an area where at least one of a mini-map, skill window, item-related window, character-related window, chat window, and control-related window included in the game image is displayed as the information area.

[0102] For example, the display device (100) can store information about the standard aspect ratio, location, size, shape, etc. of an information area (e.g., mini map) according to game content genre information, title information, aspect ratio information, etc. in a database.

[0103] The display device (100) can obtain title information of game content by analyzing game video. For example, the display device (100) can detect at least one object included in the game video and obtain title information of game content based on the characteristics of the detected object. Alternatively, the display device (100) can obtain title information of game content by using artificial intelligence technology based on a neural network.

[0104] The display device (100) can obtain the reference aspect ratio, location, size, shape, etc. of an information area (e.g., mini map) according to the title information of the game content and the aspect ratio information of the game video by using a database.

[0105] Alternatively, the display device (100) can identify information areas, such as a mini-map having a specific shape, by analyzing the game image. The display device (100) can identify areas (or objects) representing a rectangular shape, areas (or objects) representing a circular shape, etc., in the game image as mini-map areas by using an edge detection algorithm or a circle detection algorithm, etc.

[0106] A display device (100) according to one embodiment can identify a gaze area corresponding to the user's gaze information in an image (S630).

[0107] The display device (100) can track the user's gaze. For example, the display device (100) can track the user's gaze using at least one camera. The method by which the display device (100) tracks the user's gaze will be described in detail with reference to FIG. 9.

[0108] The display device (100) can acquire the user's gaze direction and the gaze point where the user's gaze direction meets the display screen. When the gaze point is acquired, the display device (100) can identify a gaze area corresponding to the gaze point. The display device (100) can identify a gaze area centered on the gaze point based on the size and shape of the information area. For example, the display device (100) can identify an area centered on the gaze point and having the same size and shape as the mini-map area as the gaze area.

[0109] A display device (100) according to one embodiment can determine whether the information area identified in step 620 (S620) and the gaze area identified in step 630 (S630) match (S640).

[0110] The display device (100) may determine that the information area and the center area coincide if the distance between the center point of the information area and the center point of the gaze area is less than or equal to a preset threshold. Alternatively, the display device (100) may determine that the information area and the center area coincide if the user's gaze point or the center point of the gaze area is located within the information area. However, it is not limited thereto, and the display device (100) may determine whether the information area and the gaze area coincide based on various criteria.

[0111] When the information area and the viewing area match, the display device (100) can output a sub-image corresponding to the information area (S650).

[0112] For example, the display device (100) can display an image with an enlarged information area. Additionally, the display device (100) can transmit an image corresponding to the information area to an external device in order to share the information area with an external device.

[0113] FIGS. 7 and 8 are drawings illustrating the operation of a display device according to one embodiment identifying an information region in an image.

[0114] Referring to FIG. 7, a display device (100) according to one embodiment can display an image corresponding to video content. For example, the display device (100) can display a game image (710) corresponding to game content.

[0115] The display device (100) can identify an information area in the game image (710). For example, the display device (100) can identify a mini-map area (720) in the game image (710). The display device (100) can obtain title information of the game content. For example, the display device (100) can analyze the game image (710) to extract at least one object included in the game image, and obtain title information of the game content based on the characteristics of the extracted object. The display device (100) can obtain information regarding the standard aspect ratio, location, shape, size, etc. of the mini-map using a database.

[0116] For example, referring to FIG. 8, the display device (100) can store information regarding the standard aspect ratio, location, shape, and size of the minimap according to game title information and aspect ratio information of the game image in a database (810). Even for game contents having the same game title, the standard aspect ratio, location, size, and shape of the minimap may differ depending on the aspect ratio of the game image, as described in FIG. 3.

[0117] The display device (100) can obtain information regarding the standard aspect ratio, location, shape, and size of the mini map based on the acquired title information and the aspect ratio of the game video.

[0118] For example, if, as a result of analyzing the game video, the title of the game content is "Game 1" and the aspect ratio of the game video is 16:9, the display device (100) can determine that the standard aspect ratio of the mini map is 16:9, the mini map is located at the top-left corner of the game video, has a rectangular shape, and has a size of W1*H1. Based on information regarding the standard aspect ratio, location, shape, and size of the mini map, the display device (100) can identify the mini map area in the game video. For example, the display device (100) can detect a rectangular object having a size of W1*H1 in the top-left corner of the game video and identify the detected object as the mini map area. However, it is not limited thereto.

[0119] Alternatively, a display device (100) according to one embodiment can identify a mini-map area in a game image using artificial intelligence technology based on a neural network.

[0120] Alternatively, a display device (100) according to one embodiment may detect at least one area (or object) having a specific shape or size included in a game image, and identify the detected area (or object) as a mini-map area based on the characteristics of the detected area (or object).

[0121] FIG. 9 is a diagram showing the operation of a display device according to one embodiment identifying a gaze area corresponding to the user's gaze information.

[0122] Referring to FIG. 9, a display device (100) according to one embodiment can track the gaze (910) of a user (901). Generally, 'gaze' refers to the direction of looking, and 'eye tracking' refers to the process of measuring the user's gaze (e.g., the point (920) that the user is looking at), and can be performed by tracking the position and movement of both eyes.

[0123] A display device (100) according to one embodiment may include at least one camera (930) to track the gaze (910) of a user (901). The display device (100) may capture the user's face and eyes in real time using at least one camera (930). The display device (100) may recognize the face and eyes in the captured image and analyze the user's eye movements by recognizing the pupil and iris in the eyes. The display device (100) may estimate the user's gaze (910) based on the eye movements.

[0124] For example, as illustrated in FIG. 9, the display device (100) can obtain the user's gaze direction (910) and the gaze point (920) where the user's gaze direction (910) meets the screen.

[0125] A display device (100) according to one embodiment can identify a gaze area (940) corresponding to a gaze point (920) when a gaze point (920) is acquired. The display device (100) can identify a gaze area (940) centered on the gaze point (920) based on the size and shape of the information area. For example, the display device (100) can identify an area centered on the gaze point and having the same size and shape as a mini-map area as the gaze area (940). However, it is not limited thereto.

[0126] According to one embodiment, the information area, gaze point, and gaze area may be represented as coordinate information.

[0127] FIG. 10 is a diagram illustrating an operation for determining whether an information area identified in an image according to one embodiment matches a user's gaze area.

[0128] Referring to FIG. 10, a display device (100) according to one embodiment can determine whether an information area (1020) identified in a game image (1010) matches a gaze area (1030) corresponding to the user's gaze information. For example, since the method of identifying the information area (1020) in the game image (1010) has been described in detail in FIG. 7 and 8, a detailed description will be omitted. Also, since the method of identifying the gaze area (1030) corresponding to the user's gaze information has been described in detail in FIG. 9, a detailed description will be omitted.

[0129] The display device (100) can determine that the information area (1020) and the gaze area (1030) are in alignment if the distance between the center point (1025) of the information area (1020) and the center point (1035) of the gaze area (1030) is less than or equal to a preset value.

[0130] Alternatively, the display device (100) may determine that the information area (1020) and the gaze area (1030) match if at least a portion of the information area (1020) overlaps with at least a portion of the gaze area (1030).

[0131] Alternatively, the display device (100) may determine that the information area (1020) and the gaze area (1030) coincide when the center point (1035) of the gaze point or gaze area is located within the information area (1020). However, the display device (100) is not limited thereto and may determine whether the information area (1020) and the gaze area (1030) coincide based on various criteria.

[0132] A display device (100) according to one embodiment may enlarge and display the information area (1020) when the information area (1020) and the viewing area (1030) coincide. For example, the display device (100) may enlarge an image corresponding to a mini-map area and display it in an enlargement window (1040). At this time, the enlargement window (1040) may be displayed in an area separate from the game image (1010). Although FIG. 10 illustrates the enlargement window (1040) being displayed to the right of the game image (1010), it is not limited thereto and may be displayed in any one of the left, right, top, or bottom areas of the game image (1010).

[0133] Alternatively, a display device (100) according to one embodiment may share an image corresponding to an information area (1020) with an external device when the information area (1020) and the viewing area (1030) coincide. For example, the display device (100) may transmit an image corresponding to a mini-map area to an external device. The external device may display the image received from the display device (100) on the display of the external device.

[0134] FIG. 11 is a diagram showing the operation of a display device displaying a confirmation message according to one embodiment.

[0135] Referring to FIG. 11, a display device (100) according to one embodiment can identify an information area (e.g., a mini-map area, 1110) in a game image and identify a gaze area corresponding to the user's gaze information.

[0136] When the mini map area (1110) identified in the game image matches the gaze area corresponding to the user's gaze information, the display device (100) may display a confirmation message (1120) for the execution of the mini map zoom function before the execution of the mini map zoom function.

[0137] The display device (100) can execute a mini-map zoom function based on user input selecting "Yes". For example, the display device (100) can zoom in on an image corresponding to a mini-map area (1110) and display it in a zoom window. At this time, the zoom window may be displayed in an area separate from the game image.

[0138] FIG. 12 is a diagram showing the operation of a display device according to one embodiment displaying a menu for selecting to expand or share an information area.

[0139] Referring to FIG. 12, a display device (100) according to one embodiment can identify an information area (e.g., a mini-map area, 1210) in a game image and identify a gaze area corresponding to the user's gaze information.

[0140] When the mini map area (1210) identified in the game video matches the gaze area corresponding to the user's gaze information, the display device (100) may display a menu (1220) that allows selection of whether to enlarge and display the image corresponding to the mini map area (1210) or to share it with an external device.

[0141] The display device (100) can execute a mini-map zoom function based on user input selecting a zoom function. For example, the display device (100) can zoom in on an image corresponding to a mini-map area (1210) and display it in a zoom window. At this time, the zoom window may be displayed in an area separate from the game image.

[0142] Alternatively, the display device (100) may execute a mini-map sharing function based on user input selecting a sharing function. For example, the display device (100) may transmit an image corresponding to a mini-map area (1210) to an external device. The external device may display the image received from the display device (100) on the display of the external device.

[0143] FIG. 13 is a diagram showing the operation of a display device according to one embodiment.

[0144] Referring to FIG. 13, a display device (100) according to one embodiment can analyze a game image (1310) to identify a first mini-map area (1320). For example, the display device (100) can detect at least one area (or object) having a specific shape or size included in the game image (1310), and based on the characteristics of the detected area (or object), identify the detected area (or object) as the first mini-map area (1320).

[0145] Additionally, a display device (100) according to one embodiment can obtain title information of game content by analyzing a game image (1310). For example, the display device (100) can obtain title information of game content by analyzing the game image (1310), extracting at least one object included in the game image (1310), and based on the characteristics of the extracted object. When the title information of game content is obtained, the display device (100) can obtain information regarding the standard aspect ratio, location, shape, size, etc. of the mini-map by using a database containing information regarding the mini-map area according to the title information of the game content. Based on the information regarding the standard aspect ratio, location, shape, size, etc. of the mini-map, the display device (100) can identify a second mini-map area (1330) in the game image (1310).

[0146] The display device (100) can determine whether the first mini-map area (1320) and the second mini-map area (1330) are in alignment. For example, the display device (100) can determine that the first mini-map area (1320) and the second mini-map area (1330) are in alignment if the center point of the first mini-map area (1320) and the center point of the second mini-map area (1330) are less than or equal to a preset distance. Alternatively, the display device can determine that the first mini-map area (1320) and the second mini-map area (1330) are in alignment if at least a portion of the first mini-map area (1320) and at least a portion of the second mini-map area (1330) overlap. However, the display device (100) is not limited thereto, and can determine whether the first mini-map area (1320) and the second mini-map area (1330) are in alignment through various methods.

[0147] When the first mini-map area (1320) and the second mini-map area (1330) coincide, the display device (100) can display an enlarged image (1340) of either the first mini-map area (1320) or the second mini-map area (1330), as shown in FIG. 13. Alternatively, the display device (100) can share either the first mini-map area (1320) or the second mini-map area (1330) with an external device.

[0148] Alternatively, if the first mini-map area (1320) and the second mini-map area (1330) do not match, the display device (100) may display a guide for magnification on either the first mini-map area (1320) or the second mini-map area (1330) selected based on user input.

[0149] FIG. 14 is a block diagram showing the configuration of a source device according to one embodiment.

[0150] Referring to FIG. 14, a source device (200) according to one embodiment may include a communication unit (230), a processor (240), and a memory (250).

[0151] A communication unit (230) according to one embodiment can receive or transmit content by performing a communication connection between a source device (200) and an external device (e.g., a display device (100)). For example, the communication unit (230) can perform communication with the external device through a wired or wireless communication network. The communication unit (230) may include at least one port for connecting to the external device via a wired cable in order to communicate with the external device via a wired connection. For example, the communication unit (230) may include at least one of an HDMI (High-Definition Multimedia Interface) port, an MHL (Mobile High-Definition Link) port, a USB (Universal Serial Bus) port, a DP (Display Port), a Thunderbolt port, a VGA (Video Graphics Array) port, an RGB port, a D-SUB (D-subminiature), a DVI (Digital Visual Interface), a component jack, and a PC port. The communication unit (230) can communicate with an external device connected via a wire through at least one port.

[0152] Additionally, the communication unit (230) may include at least one wireless communication module, wireless communication circuit, or wireless communication device that performs wireless communication with an external device. For example, the communication unit (230) may include at least one communication module that performs communication according to communication standards such as Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC / RFID, Wi-Fi Direct, UWB, ZIGBEE, the Internet, 3G, 4G, 5G, and / or 6G. However, it is not limited thereto, and the configuration and operation of the communication unit (230) may be implemented in various ways according to embodiments of the present invention.

[0153] For example, the communication unit (230) is connected to the display device (100) and can receive EDID information of the display device (100) from the display device (100) and output content to the display device (100).

[0154] A processor (240) according to one embodiment can control the overall operation of a source device (200). Specifically, the processor (240) can control the process of outputting a video of content being played on the source device (200) to a display device (100).

[0155] The processor (240) can execute one or more programs stored in memory (250). In one embodiment, memory (250) can store various data, programs, or applications for driving and controlling the source device (200).

[0156] Additionally, the program stored in memory (250) may include one or more instructions. The program (one or more instructions) or application stored in memory (250) may be executed by the processor (240).

[0157] A source device (200) according to one embodiment can receive content transmitted from an external device or an external server. The source device (200) can receive content from an external recording medium such as a USB memory or an HDD. Alternatively, it can read content recorded on an optical disc such as a DVD or a Blu-ray disc.

[0158] A processor (240) according to one embodiment can play content by executing one or more instructions stored in memory (250), and can be implemented in various forms depending on the embodiment. For example, the processor (240) can perform processing on content data received by the source device (200).

[0159] FIG. 15 is a block diagram showing the configuration of a display device according to one embodiment.

[0160] Referring to FIG. 15, a display device (100) according to one embodiment may include a communication unit (110), a processor (120), a memory (130), a display (140), and a camera (150).

[0161] A communication unit (110) according to one embodiment can transmit and receive data or signals with an external device (e.g., a source device (200)).

[0162] A communication unit (110) according to one embodiment can transmit or receive data or signals to and from an external device (e.g., a source device (200)) or a server. For example, the communication unit (110) can communicate with an external device through a wired or wireless communication network. To communicate with an external device via a wired connection, the communication unit (110) may include at least one port for being connected via a wired cable. For example, the communication unit (110) may include at least one of an HDMI (High-Definition Multimedia Interface) port, an MHL (Mobile High-Definition Link) port, a USB (Universal Serial Bus) port, a DP (Display Port), a Thunderbolt port, a VGA (Video Graphics Array) port, an RGB port, a D-SUB (D-subminiature), a DVI (Digital Visual Interface), a component jack, or a PC port. The communication unit (110) can communicate with an external device connected via a wired connection through at least one port.

[0163] Additionally, the communication unit (110) may include a Wi-Fi module, a Bluetooth module, an infrared communication module and a wireless communication module, a LAN module, an Ethernet module, etc. At this time, each communication module may be implemented in the form of at least one hardware chip.

[0164] The Wi-Fi module and the Bluetooth module perform communication using the Wi-Fi method and the Bluetooth method, respectively. When using the Wi-Fi module or the Bluetooth module, various connection information such as SSID and session key is transmitted and received first, and then various information can be transmitted and received after establishing a communication connection using this information. The wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0165] A communication unit (110) according to one embodiment may request content (e.g., game content) from a source device (200). Additionally, the communication unit (110) may provide an EDID to the source device (200).

[0166] The communication unit (110) can receive content from the source device (200). Even for the same content, it can receive images having various aspect ratios.

[0167] A communication unit (110) according to one embodiment can receive a control signal or control command, etc., corresponding to user input (e.g., key or button input of the control device) from a control device.

[0168] A processor (120) according to one embodiment controls the overall operation of the display device (100) and the signal flow between the internal components of the electronic device (100), and performs the function of processing data.

[0169] The processor (120) may include a single core, dual core, triple core, quad core, and multiples thereof. Additionally, the processor (120) may include multiple processors. For example, the processor (120) may be implemented as a main processor (not shown) and a sub processor (not shown).

[0170] Additionally, the processor (120) may include at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and a VPU (Video Processing Unit). Alternatively, depending on the embodiment, it may be implemented in the form of a System On Chip (SoC) integrating at least one of a CPU, a GPU, and a VPU. Alternatively, the processor (120) may further include a Neural Processing Unit (NPU).

[0171] A memory (130) according to one embodiment can store various data, programs, or applications for driving and controlling a display device (100).

[0172] A memory (130) according to one embodiment may include a database (810) illustrated and described in FIG. 8.

[0173] Additionally, the program stored in memory (130) may include one or more instructions. The program (one or more instructions) or application stored in memory (130) may be executed by the processor (120).

[0174] A processor (120) according to one embodiment can display an image corresponding to video content by executing one or more instructions stored in the memory (130). The video content according to one embodiment may be game content, movie content, music content, home training service content, broadcast content, etc. For example, the processor (120) can display a game image corresponding to game content.

[0175] A processor (120) according to one embodiment can identify an information area in an image by executing one or more instructions stored in the memory (130). An information area according to one embodiment may be an area where at least one sub-window included in the game image is displayed. For example, the processor (120) may identify an area where at least one of a mini-map, skill window, item-related window, character-related window, chat window, or control-related window included in the game image is displayed as an information area.

[0176] A processor (120) according to one embodiment can store information regarding the reference aspect ratio, location, size, shape, etc. of an information area (e.g., a mini map) according to game content genre information, title information, aspect ratio information, etc. in a database by executing one or more instructions stored in the memory (130).

[0177] A processor (120) according to one embodiment can obtain title information of game content by analyzing a game image by executing one or more instructions stored in the memory (130). For example, the processor (120) can detect at least one object included in the game image and obtain title information of game content based on the characteristics of the detected object. Alternatively, the processor (120) can obtain title information of game content by using artificial intelligence technology based on a neural network.

[0178] A processor (120) according to one embodiment can obtain the reference aspect ratio, location, size, shape, etc. of an information area (e.g., mini map) according to the title information of the game content and the aspect ratio information of the game image by executing one or more instructions stored in the memory (130) using a database.

[0179] Alternatively, the processor (120) can identify information areas, such as a mini-map having a specific shape, by analyzing the game image. The display device (100) can identify areas (or objects) representing a rectangular shape, areas (or objects) representing a circular shape, etc., in the game image as mini-map areas by using an edge detection algorithm or a circle detection algorithm.

[0180] A processor (120) according to one embodiment can identify a gaze area corresponding to the user's gaze information in an image by executing one or more instructions stored in the memory (130).

[0181] A processor (120) according to one embodiment can track a user's gaze by executing one or more instructions stored in the memory (130). For example, the processor (120) can track a user's gaze by using at least one camera (150).

[0182] A camera (150) according to one embodiment can acquire image frames, such as still images or video. For example, the camera (150) can capture images outside the display device (100). At this time, the images captured through the camera (150) can be processed through a processor (120) or a separate image processor.

[0183] For example, the camera (150) can acquire images (e.g., consecutive frames) corresponding to the user's face within the camera recognition range. A processor (120) according to one embodiment can perform eye tracking of the user based on images of the user. Since the method of tracking the user's gaze has been described in detail with reference to FIG. 9, a detailed description will be omitted.

[0184] A processor (120) according to one embodiment can obtain a user's gaze direction and a gaze point where the user's gaze direction meets a display screen. When a gaze point is obtained, the processor (120) can identify a gaze area corresponding to the gaze point.

[0185] A processor (120) according to one embodiment can identify a gaze area centered on a gaze point based on the size and shape of an information area by executing one or more instructions stored in the memory (130). For example, the processor (120) can identify a gaze area centered on a gaze point and having the same size and shape as a mini-map area.

[0186] A processor (120) according to one embodiment can determine whether an identified information area and an identified gaze area match by executing one or more instructions stored in the memory (130). For example, the processor (120) can determine that the information area and the center area match if the distance between the center point of the information area and the center point of the gaze area is less than or equal to a preset threshold. Alternatively, the processor (120) can determine that the information area and the center area match if the user's gaze point or the center point of the gaze area is located within the information area. However, it is not limited thereto, and the processor (120) can determine whether the information area and the gaze area match based on various criteria.

[0187] A processor (120) according to one embodiment can output a sub-image corresponding to an information area when an information area and a gaze area match by executing one or more instructions stored in the memory (130).

[0188] For example, the processor (120) can display an enlarged image of the information area. Additionally, the processor (120) can transmit an image corresponding to the information area to an external device in order to share the information area with an external device.

[0189] A display (140) according to one embodiment converts a video signal, data signal, OSD signal, control signal, etc. processed by a processor (120) to generate a driving signal. The display (240) can be implemented as a PDP, LCD, OLED, flexible display, etc., and can also be implemented as a 3D display. Additionally, the display (140) can be configured as a touch screen and used as an input device in addition to an output device.

[0190] A display (140) according to one embodiment may display a game image, an image of an information area identified in the game image that is enlarged, a user interface screen for setting a game mode, etc. However, it is not limited thereto.

[0191] FIG. 16 is a block diagram showing the configuration of a display device according to one embodiment.

[0192] The display device (2000) of FIG. 16 may be an embodiment of the display device (100) described with reference to FIG. 1 to 13 and FIG. 15.

[0193] Referring to FIG. 16, a display device (1600) according to one embodiment may include a tuner unit (1640), a processor (1610), a display unit (1620), a communication unit (1650), a detection unit (1630), an input / output unit (1670), a video processing unit (1680), an audio processing unit (1685), an audio output unit (1660), a memory (1690), and a power supply unit (1695).

[0194] The communication unit (1650) of FIG. 16 may correspond to the communication unit (110) of FIG. 15, the processor (1610) of FIG. 16 may correspond to the processor (120) of FIG. 15, the memory (1690) of FIG. 16 may correspond to the memory (130) of FIG. 15, the display unit (1620) of FIG. 16 may correspond to the display (140) of FIG. 15, and the camera unit (1632) of FIG. 16 may correspond to the camera (150) of FIG. 15. Accordingly, content identical to that described above will be omitted.

[0195] A tuner unit (1640) according to one embodiment can select only the frequency of the channel to be received by the display device (1600) among many radio wave components by tuning through amplification, mixing, resonance, etc. of a broadcast signal received via wired or wireless means. The broadcast signal includes audio, video, and additional information (e.g., EPG (Electronic Program Guide)).

[0196] The tuner unit (1640) can receive broadcast signals from various sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, internet broadcasting, etc. The tuner unit (1640) can also receive broadcast signals from sources such as analog broadcasting or digital broadcasting.

[0197] A communication unit (1650) according to one embodiment may receive a control signal or a control command, etc. from an external control device. For example, the communication unit (1650) may include an IR module capable of transmitting and receiving signals with an external control device according to an IR communication standard. Specifically, the communication unit (1650) may receive a control signal or a control command, etc. corresponding to a user input (e.g., a key or button input of the control device) from the control device.

[0198] A detection unit (1630) according to one embodiment detects a user's voice, a user's image, or a user's interaction, and may include a microphone (1631), a camera unit (1632), and an optical receiver (1633).

[0199] The microphone (1631) receives the uttered voice of the user. The microphone (1631) can convert the received voice into an electrical signal and output it to the processor (1610). The user voice may include, for example, a voice corresponding to a menu or function of the display device (1600). For example, the microphone (1631) can receive a user voice corresponding to a rotation command of the display and convert the received voice into an electrical signal and output it to the processor (1610).

[0200] The camera unit (1632) can receive images (e.g., consecutive frames) corresponding to a user's motion including a gesture within the camera recognition range. The processor (1610) can select a menu displayed on the display device (1600) or perform control corresponding to the motion recognition result using the received motion recognition result. For example, the processor (2010) can receive images from the camera unit (1632), recognize a user's motion corresponding to the rotation of the display from the received images, and rotate the display accordingly.

[0201] The light receiver (1633) receives a light signal (including a control signal) received from an external control device through a light window (not shown) in the bezel of the display unit (1620). The light receiver (1633) can receive a light signal corresponding to user input (e.g., touch, press, touch gesture, voice, or motion) from the control device. A control signal can be extracted from the received light signal by the control of the processor (1610).

[0202] An input / output unit (1670) according to one embodiment can receive video (e.g., video, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.) from outside the display device (1600). The input / output unit (1670) may include any one of HDMI (High-Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), DVI (Digital Visual Interface), component jack, and PC port.

[0203] The processor (1610) controls the overall operation of the display device (1600) and the signal flow between the internal components of the display device (2000), and performs the function of processing data. The processor (1610) can execute an OS (Operation System) and various applications stored in memory (1690) when there is user input or when a pre-set and stored condition is satisfied.

[0204] The processor (1610) may include RAM used as a storage area for storing signals or data input from outside the display device (1600) or for various tasks performed on the display device (1600), ROM stored as a control program for controlling the display device (1600), and a processor.

[0205] The video processing unit (1680) performs processing on video data received by the display device (1600). The video processing unit (1680) can perform various image processing on the video data, such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion.

[0206] The audio processing unit (1685) performs processing on audio data. Various processing such as decoding, amplification, and noise filtering on audio data can be performed in the audio processing unit (1685). Meanwhile, the audio processing unit (1685) may be equipped with multiple audio processing modules to process audio corresponding to multiple contents.

[0207] The audio output unit (1660) outputs audio included in a broadcast signal received through the tuner unit (1640) under the control of the processor (1610). The audio output unit (1660) can output audio (e.g., voice, sound) input through the communication unit (1650) or the input / output unit (1670). Additionally, the audio output unit (1660) can output audio stored in the memory (1690) under the control of the processor (1610). The audio output unit (1660) may include at least one of a speaker, a headphone output terminal, or an S / PDIF (Sony / Philips Digital Interface) output terminal.

[0208] The power supply unit (1695) supplies power input from an external power source to the components inside the display device (1600) under the control of the processor (1610). Additionally, the power supply unit (1695) can supply power output from one or more batteries (not shown) located inside the display device (1600) to the internal components under the control of the processor (1610).

[0209] The memory (1690) may store various data, programs, or applications for driving and controlling the display device (1600) under the control of the processor (1610). The memory (1690) may include a broadcast receiving module, a channel control module, a volume control module, a communication control module, a voice recognition module, a motion recognition module, an optical receiving module, a display control module, an audio control module, an external input control module, a power control module, a power control module for an external device connected wirelessly (e.g., Bluetooth), a voice database (DB), or a motion database (DB), which are not illustrated. The modules not illustrated and the database of the memory (1690) may be implemented in software form to perform broadcast reception control functions, channel control functions, volume control functions, communication control functions, voice recognition functions, motion recognition functions, optical reception control functions, display control functions, audio control functions, external input control functions, power control functions, or power control functions for an external device connected wirelessly (e.g., Bluetooth) in the display device (1600). The processor (1610) can perform each of these functions using the software stored in memory (1690).

[0210] Meanwhile, the block diagrams of the display devices (100, 1600) illustrated in FIGS. 15 and 16 are block diagrams for one embodiment. Each component of the block diagrams may be integrated, added, or omitted according to the specifications of the actual implemented display device (100, 1600). That is, as needed, two or more components may be combined into a single component, or a single component may be subdivided into two or more components. Furthermore, the functions performed in each block are intended to explain the embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

[0211] A display device according to one embodiment may include a display, at least one camera for acquiring user gaze information, a memory for storing one or more instructions, and at least one processor including a processing circuit.

[0212] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can display an image corresponding to the image content on the display.

[0213] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can identify an information region in the image based on the image content information.

[0214] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can identify a gaze area corresponding to the user's gaze information in the image.

[0215] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can output a sub-image corresponding to the information area based on the fact that the information area and the gaze area coincide.

[0216] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can acquire title information of the video content.

[0217] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can obtain at least one of the reference aspect ratio, position, size, and shape of the information area based on the title information of the image content and the aspect ratio information of the image.

[0218] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can identify the information region in the image based on at least one of the reference aspect ratio information, location, size, and shape of the information region.

[0219] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can obtain title information of the image content by recognizing at least one object included in the image.

[0220] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can determine whether the information area and the gaze area coincide based on the distance between the point where the user's gaze is directed and the center point of the information area in the image.

[0221] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can set the on / off of a function for automatically detecting the information area and the on / off of a user eye tracking function based on user input.

[0222] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can identify the information area and the gaze area based on the function of automatically detecting the information area and the user eye tracking function being set to On.

[0223] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device may display a message asking whether to output a sub-image corresponding to the information area based on the fact that the information area and the gaze area match.

[0224] Video content according to one embodiment may include game content.

[0225] An image according to one embodiment may include a game image corresponding to the game content.

[0226] An information area according to one embodiment may include at least one of a mini-map, a skill window, an item-related window, a character-related window, a chat window, and a control-related window included in the game image.

[0227] A sub-image according to one embodiment may include an image that is an enlarged version of the information area.

[0228] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can display the sub-image on the display.

[0229] When the one or more instructions are executed individually or collectively by at least one processor according to one embodiment, the display device can transmit the sub-image to an external device.

[0230] A method of operating a display device according to one embodiment may include the step of displaying an image corresponding to image content.

[0231] A method of operating a display device according to one embodiment may include the step of identifying an information region in the image based on the image content information.

[0232] A method of operating a display device according to one embodiment may include the step of identifying a gaze area corresponding to user gaze information in the image.

[0233] A method of operation of a display device according to one embodiment may include the step of outputting a sub-image corresponding to the information area based on the fact that the information area and the gaze area coincide.

[0234] The step of identifying an information region in the above image may include the step of obtaining title information of the image content.

[0235] The step of identifying an information region in the above image may include obtaining at least one of the reference aspect ratio information, location, size, and shape of the information region based on the title information of the image content and the aspect ratio information of the image.

[0236] The step of identifying an information region in the above image may include identifying the information region in the above image based on at least one of the reference aspect ratio information, location, size, and shape of the information region.

[0237] The step of obtaining title information of the above video content may include the step of obtaining title information of the above video content by recognizing at least one object included in the above video.

[0238] The above operation method may further include a step of determining whether the information area and the gaze area coincide based on the distance between the point where the user's gaze is directed and the center point of the information area in the image.

[0239] The above operation method may further include the step of setting the function for automatically detecting the information area based on user input to On and setting the user eye tracking function to On.

[0240] The above operation method may further include the step of displaying a message asking whether to output a sub-image corresponding to the information area based on the fact that the information area and the gaze area match.

[0241] The step of outputting the above sub-image may include the step of displaying a sub-image with the information area enlarged.

[0242] The step of outputting the sub-image may include the step of transmitting the sub-image including the information region to an external device.

[0243] A display device according to one embodiment can provide an automatic recognition function for specific information included in video content (e.g., game content), thereby providing various viewing environments.

[0244] A display device according to one embodiment can execute an enlargement function or a sharing function for a specific information area based on the user's gaze information, thereby enhancing immersion in the game and improving usability.

[0245] A display device according to one embodiment can automatically recognize (detect) or identify specific information included in an image corresponding to image content, and can reconfirm the identified specific information based on the user's gaze information, thereby increasing the accuracy of the recognition (detection) of specific information.

[0246] A method of operation of a display device according to one embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0247] In addition, the method of operation of the display device according to the disclosed embodiments may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product.

[0248] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product may include a product in the form of a software program (e.g., a downloadable app) that is electronically distributed through a manufacturer of an electronic device or an electronic market (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily created. In this case, the storage medium may be a server of the manufacturer, a server of the electronic market, or a storage medium of a relay server that temporarily stores the software program.

[0249] A computer program product may include a storage medium of a server or a storage medium of a client device in a system composed of a server and a client device. Alternatively, if a third device (e.g., a smartphone) is communicationally connected to the server or client device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include the S / W program itself, which is transmitted from the server to the client device or the third device, or from the third device to the client device.

[0250] In this case, one of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the client device, and the third device may execute the computer program product to perform the method according to the disclosed embodiments in a distributed manner.

[0251] For example, a server (e.g., a cloud server or an artificial intelligence server, etc.) can execute a computer program product stored on the server to control a client device connected to the server in communication to perform the method according to the disclosed embodiments.

[0252] Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims also fall within the scope of the present invention.

Claims

1. In a display device (100), Display (140); At least one camera (150) for acquiring user's gaze information; Memory (130) for storing one or more instructions; and It includes at least one processor (120) including a processing circuit, and When the one or more instructions are executed individually or collectively by the at least one processor (120), the display device (100) is, Displays a video corresponding to the video content on the above display, and Based on the above video content information, an information region is identified in the above video, and In the above image, a gaze area corresponding to the user's gaze information is identified, and A display device that outputs a sub-image corresponding to the information area based on the fact that the information area and the gaze area match.

2. In Paragraph 1, When the one or more instructions are executed individually or collectively by the at least one processor (120), the display device, Obtain title information for the above video content, and Based on the title information of the video content and the aspect ratio information of the video, at least one of the reference aspect ratio information, location, size, and shape of the information area is obtained, and A display device that identifies an information region in an image based on at least one of reference aspect ratio information, location, size, and shape of the information region.

3. In Paragraph 2, When the one or more instructions are executed individually or collectively by the at least one processor (120), the display device, A display device that obtains title information of the image content by recognizing at least one object included in the image.

4. In any one of paragraphs 1 through 3, When the one or more instructions are executed individually or collectively by the at least one processor (120), the display device, A display device that determines whether the information area and the gaze area coincide based on the distance between the point where the user's gaze is directed and the center point of the information area in the above image.

5. In any one of paragraphs 1 through 4, When the one or more instructions are executed individually or collectively by the at least one processor (120), the display device, A display device that sets the On / Off status of a function for automatically detecting the information area and the On / Off status of a user eye tracking function based on user input.

6. In Paragraph 5, When the one or more instructions are executed individually or collectively by the at least one processor (120), the display device, A display device that identifies the information area and the gaze area based on the fact that the function of automatically detecting the information area and the user gaze tracking function are set to On.

7. In any one of paragraphs 1 through 6, A display device that displays a message asking whether to output a sub-image corresponding to the information area based on the fact that the information area and the gaze area match.

8. In any one of paragraphs 1 through 7, The above video content includes game content, and The above video includes game video corresponding to the above game content, and A display device wherein the information area includes at least one of a minimap, a skill window, an item-related window, a character-related window, a chat window, and a control-related window included in the game image.

9. In any one of paragraphs 1 through 8, The above sub-image includes an image that is an enlarged version of the information area, and When the one or more instructions are executed individually or collectively by the at least one processor (120), the display device, A display device that displays the above sub-image on the above display.

10. In any one of paragraphs 1 through 9, When the one or more instructions are executed individually or collectively by the at least one processor (120), the display device, A display device that transmits the above sub-image to an external device.

11. In a method of operating a display device, Step of displaying a video corresponding to the video content (S610); A step (S620) of identifying an information region in the image based on the above image content information; In the above image, a step of identifying a gaze area corresponding to the user's gaze information (S630); and A method of operating a display device, comprising the step (S650) of outputting a sub-image corresponding to the information area based on the fact that the information area and the gaze area match.

12. In Paragraph 11, The step of identifying information regions in the above image is, A step of obtaining title information of the above video content; A step of obtaining at least one of the reference aspect ratio information, location, size, and shape of the information area based on the title information of the video content and the aspect ratio information of the video; and A method of operating a display device comprising the step of identifying an information region in an image based on at least one of reference aspect ratio information, location, size, and shape of the information region.

13. In Paragraph 12, The step of obtaining title information for the above video content is, A method of operating a display device comprising the step of obtaining title information of the image content by recognizing at least one object included in the image.

14. In any one of paragraphs 11 through 13, The above method of operation is, A method of operating a display device, further comprising the step of determining whether the information area and the gaze area coincide based on the distance between the point where the user's gaze is directed and the center point of the information area in the above image.

15. One or more computer-readable recording media storing a program that enables the method of any one of paragraphs 11 through 14.

Images