Eye Gaze-Based Media Display Device Control

Abstract

An electronic device is provided that captures a first image of a user. The electronic device determines, based on the first image, that the position at which the user's eyes are gazing is on a first UI element among a plurality of UI elements displayed on a first display screen of the electronic device. The electronic device selects the first UI element based on determining that the position of the eye gaze is on the first UI element for a period of time. The electronic device sends a first control signal to a media display device based on the selection of the first UI element. The electronic device receives a feedback image from the media display device. The feedback image indicates the control of the media display device based on the first control signal. The electronic device controls the first display screen to display the feedback image together with the plurality of UI elements.

Description

[0001] Cross - Reference to Related Applications / Incorporation by Reference

[0002] This application claims priority to U.S. Patent Application Serial No. 17 / 175,881, filed on February 15, 2021, with the United States Patent and Trademark Office. Each of the above - cited applications is incorporated herein by reference in its entirety. Field of the Invention

[0003] Various embodiments of the present disclosure relate to controlling a media display device based on a user's eye gaze. More specifically, various embodiments of the present disclosure relate to apparatuses and methods for controlling a media display device based on eye gaze on an electronic device coupled to the media display device. Background Art

[0004] Recent advancements in the field of media content reproduction have led to the development of various techniques for controlling a media display device (e.g., a television). Generally, a user can utilize a remote controller to control functions of the media display device, such as switching channels on the media display device, controlling the volume of the media display device, and so on. In some cases, users with physical disabilities (such as neurodevelopmental disorders, e.g., motor disorders) may access the media display device. For example, a user may have difficulty in controlling the functions of the media display device via a remote controller. Thus, the user may require assistance to control the media display device.

[0005] More limitations and disadvantages of conventional and traditional methods will become apparent to those skilled in the art by comparison with the systems described and some aspects of the present disclosure as set forth in the remainder of the present application and with reference to the drawings. Summary of the Invention

[0006] Substantially, as more fully set forth in the claims and as illustrated and / or described in at least one of the figures, there is provided an electronic device and a method for controlling a media display device based on eye gaze via the electronic device.

[0007] These or other features and advantages of the present disclosure will be understood from a review of the following detailed description of the present disclosure in conjunction with the drawings, in which like reference numerals always refer to like parts. Brief Description of the Drawings

[0008] Figure 1 is a block diagram illustrating an exemplary network environment for controlling a media display device based on eye gaze as user input according to an embodiment of the present disclosure.

[0009] Figure 2 is illustrative of an embodiment of the present disclosure Figure 1Block diagram of an exemplary electronic device in.

[0010] Figure 3A and Figure 3B are jointly diagrams illustrating exemplary scenarios for controlling Figure 1 the media display device in accordance with embodiments of the present disclosure.

[0011] Figure 4A and Figure 4B are jointly diagrams illustrating other exemplary scenarios for controlling Figure 1 the media display device in accordance with embodiments of the present disclosure.

[0012] Figure 5A and Figure 5B are jointly diagrams illustrating other exemplary scenarios for controlling Figure 1 the media display device in accordance with embodiments of the present disclosure.

[0013] Figure 6 is a flowchart illustrating an exemplary method for controlling a media display device based on eye gaze as user input in accordance with embodiments of the present disclosure. Detailed Description

[0014] The implementation described below can be found in the disclosed electronic devices and methods for controlling a media display device based on a user's eye gaze as user input. Exemplary aspects of the present disclosure provide an electronic device (e.g., an electronic tablet or a smart phone) accessible to a user (e.g., a user with or without a physical disability). The electronic device may include a first display screen for displaying a plurality of user interface (UI) elements (e.g., but not limited to, control icons such as power control icons and volume control icons, navigation icons, or numeric icons). The electronic device may control a media display device communicatively coupled to the electronic device based on a user's selection of each of the plurality of UI elements. The electronic device may be configured to control an image capture device (e.g., a built-in front camera of an electronic tablet) to capture a first image of the user of the electronic device. The first image of the user may include an image of at least the face of the first user. The electronic device may be configured to determine, based on the captured first image of the user, that the position of the user's eye gaze is on a first UI element among the plurality of displayed UI elements. The first UI element may be, for example, a control icon such as an "increase volume icon", and when the increase volume icon is selected, the volume of the media content currently presented on the media display device communicatively coupled to the electronic device may be increased. The electronic device may also be configured to select the first UI element among the plurality of UI elements based on determining that the position of the eye gaze is on the first UI element for a period of time. For example, the period of time may be a few seconds (such as equal to or more than three seconds). In one example, the selection of the first UI element among the plurality of UI elements may be based on the eye gaze being at the first UI element for a period of 3 seconds.

[0015] The electronic device may also be configured to send a first control signal to the media display device based on the selection of the first UI element among the plurality of UI elements. The first control signal may correspond to the activation of a function associated with the media display device (such as volume increase) when the first UI element among the plurality of UI elements is selected. In one example, when the first UI element such as an "increase volume icon" is selected, the first control signal may correspond to a short-range radio wave signal indicating an increase in the volume of the media content presented on the media display device. The electronic device may also be configured to receive a feedback image from the media display device based on the sent first control signal. The received feedback image may indicate the control of the media display device based on the sent first control signal. According to one embodiment, the received feedback image may correspond to the currently presented media content and a UI icon indicating an increase in the volume of the presented media content displayed on the media display device. The electronic device may also be configured to control the first display screen to display the received feedback image together with the plurality of displayed UI elements.

[0016] In an exemplary implementation, the electronic device can be positioned such that the user can gaze at multiple UI elements of the electronic device to control the position of the media display device. The electronic device can allow the user to select different UI elements among the multiple UI elements on the electronic device to control different functions associated with the media display device. The electronic device of the present disclosure can provide the user (e.g., a user with or without a physical disability) with the ability to control the media display device based on the user's gaze as user input. The electronic device thus allows the user to control the media display device based on eye gaze and thus allows the user to control the media display device without assistance from other users.

[0017] In addition, the electronic device can simultaneously display multiple UI elements and a received feedback image that can include a portion of the media content. In some embodiments, the feedback image can also include a UI icon corresponding to the selected first UI element. Simultaneously displaying the feedback image and multiple UI elements on the electronic device can allow the user to focus the eye gaze on the selected first UI element while viewing the media content and / or the UI icon, thereby eliminating the need to switch the position of the eye gaze between the electronic device and the media display device while controlling the media display device via the electronic device. The electronic device of the present disclosure thus allows the user (e.g., a user with a physical disability) to control the media display device while continuing to view the media content on the electronic device by simultaneously displaying the feedback image (corresponding to the current presented portion of the media content) and multiple UI elements on the electronic device.

[0018] In addition, the electronic device of the present disclosure can control the media display device using short-range radio wave signals or using a wireless local area network, thereby allowing the user to control the media display device without the need to connect the electronic device to an external network (e.g., the Internet).

[0019] Figure 1 is a block diagram illustrating an exemplary network environment for controlling a media display device based on eye gaze as user input according to an embodiment of the present disclosure. Refer to Figure 1, a network environment 100 is shown here. The network environment 100 may include an electronic device 102. The electronic device 102 may include a first display screen 104 and an image capture device 108. The first display screen 104 may display a plurality of UI elements 106 and a feedback image 110. The plurality of UI elements 106 may include a first UI element 106A, a second UI element 106B, and an Nth UI element 106N. The network environment 100 may further include a media display device 112. The media display device 112 may include a second display screen 114. The second display screen 114 may present media content 116. The network environment 100 may further include a user 118, a server 120, and a communication network 122. The electronic device 102, the media display device 112, and the server 120 may be communicatively coupled to each other via the communication network 122. In some scenarios, the user 118 may be, for example, a user with a physical disability, such as a user with a neurodevelopmental disorder (e.g., a movement disorder), who may require a wheelchair for physical movement. In such a scenario, the electronic device 102 may be mounted on the armrest of the wheelchair and / or on furniture within the field of view of the user 118 to control the media display device 112 via the electronic device 102.

[0020] The electronic device 102 may include suitable logic, circuitry, and interfaces that may be configured to control the media display device 112. The electronic device 102 may also be configured to control the media display device 112 by using the user 118's eye gaze as user input. Examples of the electronic device 102 may include, but are not limited to, electronic tablets, smart phones, computing devices, cellular networks, mobile phones, gaming devices, hosts, servers, computer workstations, and / or consumer electronics (CE) devices. It will be noted that the shape and arrangement of the electronic device 102 shown in Figure 1 are shown only as examples. Without departing from the scope of the present disclosure, the present disclosure is also applicable to other shapes and arrangements of the electronic device 102.

[0021] The first display screen 104 may include suitable logic, circuitry, and interfaces that may be configured to display a plurality of UI elements 106 and feedback images 110. The first display screen 104 of the electronic device 102 may be configured to receive user input in the form of the user 118's eye gaze. Additionally, the first display screen 104 may be a touch screen that enables the user 118 to provide user input in the form of touch input via the first display screen 104. The touch screen may be at least one of a resistive touch screen, a capacitive touch screen, or a thermal touch screen. The first display screen 104 may be implemented by several known technologies, such as, but not limited to, at least one of a liquid crystal display (LCD) monitor, a light emitting diode display (LED) monitor, a plasma display, an organic LED (OLED) display technology, or other display devices. According to one embodiment, the first display screen 104 may refer to the display screen of a head-mounted device (FIMD), a smart glass device, a see-through display, a projection-based display, an electrochromic display, or a transparent display.

[0022] The plurality of UI elements 106 may correspond to control icons (e.g., user-selectable options) for controlling the media display device 112. For example, each of the plurality of UI elements 106 may represent a user-selectable icon, such as a control icon of a remote controller that may be used to control a media display device 112 (such as a television). The plurality of UI elements 106 may be configured to allow the user 118 to control different functions associated with the media display device 112 via selection of the plurality of UI elements 116 based on the user 118's eye gaze. Examples of the plurality of UI elements 116 may include, but are not limited to, control icons corresponding to the power settings of the media display device 112, home screen settings on the media display device 112, audio settings (such as volume control settings) of the media content presented on the media display device 112, video display settings (such as display characteristic adjustment, video mode selection) of the media content presented on the media display device 112, network settings of the media display device 112, security settings of the media display device 112, or accessibility settings of the media display device 112. According to one embodiment, the home screen settings on the media display device 112 may correspond to preferences related to the arrangement of menus or icons on the home screen of the media display device 112. Additionally, the security settings may include, but are not limited to, user privacy settings, parental control settings, and user account settings of the media display device 112. Additionally, the accessibility settings may include, but are not limited to, voice guidance settings, voice-based scene description settings, closed caption settings, video description settings, and color inversion settings of the media display device 112.

[0023] The image capture device 108 may include suitable logic, circuitry, interfaces, and / or code that may be configured to capture one or more images of a user, such as user 118. According to one embodiment, the image capture device 108 may capture one or more images that may include at least the face of user 118 to determine that the position of the user 118's eye gaze is on a first UI element among a plurality of UI elements 116. Examples of the image capture device 108 may include, but are not limited to, an image sensor, a wide-angle camera, a 360-degree camera, an action camera, a closed-circuit television (CCTV) camera, a camcorder, a digital camera, a camera phone, a time-of-flight camera (ToF camera), a night vision camera, and / or other image capture devices. Although in Figure 1 FIG., the image capture device 108 is shown integrated with the electronic device 102, the present disclosure is not limited thereto. In some embodiments, the image capture device 108 may be separated from the electronic device without departing from the scope of the present disclosure.

[0024] The media display device 112 may include suitable logic, circuitry, interfaces, and / or code that may be configured to present media content according to each viewer's preference (e.g., the preference of user 118). In one example, the media display device 112 may be configured to communicate with the server 120 to receive media content. In another example, the media display device 112 may be configured to receive broadcast content or streaming content as media content. According to one embodiment, the media display device 112 may also be configured to store the media content received from the server 120. The media display device 112 may also be configured to receive a first control signal via the electronic device 102 for controlling the media display device 112 based on a plurality of UI elements 106, wherein the selection of the plurality of UI elements is based on the eye gaze of user 118 as user input. Examples of the media display device 112 may include, but are not limited to, a television, a laptop computer, a personal computer, a mobile phone, a smart phone, a tablet computing device, a game controller, a media player, a sound system with display capabilities, a home theater system, a digital camera, a self-contained electronic device, an electronic musical instrument, or other consumer electronic devices with media presentation capabilities. It should be noted that, as shown in Figure 1 FIG., the shape and arrangement of the media display device 112 are shown only as examples. The present disclosure is also applicable to other shapes and arrangements of the media display device 112 without departing from the scope of the present disclosure.

[0025] The second display screen 114 may include suitable logic, circuitry, and interfaces that may be configured to display media content. The second display screen 114 may be implemented by several known technologies, such as, but not limited to, at least one of an LCD display, an LED display, a plasma display, an OLED display technology, or other display devices. According to one embodiment, the second display screen 114 may refer to a display screen of a smart glass device, a see-through display, a projection-based display, an electrochromic display, or a transparent display. According to one embodiment, the second display screen 114 may be a touch screen that enables a user 118 to provide user input in the form of touch via the second display screen 114. The touch screen may be at least one of a resistive touch screen, a capacitive touch screen, or a thermal touch screen.

[0026] The presented media content 116 may correspond to the content currently presented on the second display screen 114 of the media display device 112. Examples of the media content 116 may include, but are not limited to, the home screen of the media display device 112, the home screen of an application presented on the media display device 112, image content, video content presented on the media display device 112 (e.g., video recordings, broadcast content, streaming content), a game application interface, or a device settings interface of the media display device.

[0027] The feedback image 110 of the electronic device 102 may correspond to a duplicate image of a portion of the media content 116 currently presented on the media display device 112. In some embodiments, the feedback image 110 may correspond to an image of a UI icon displayed on the second display screen 114 that may indicate a selection of any one of the plurality of UI elements 106. In one embodiment, the feedback image 110 may be a set of images (e.g., a video stream) of the media content 116 received from the media display device 112. In an exemplary scenario, the feedback image 110 may correspond to a movie currently being presented on the second display screen 114 of the media display device 112 and a UI icon that may indicate an increase in the volume of the presented media content (e.g., the movie) displayed on the media display device 112.

[0028] The server 120 may include suitable logic, circuitry, interfaces, and / or code that may be configured to communicate with the media display device 112 via the communication network 122. The server 120 may be configured to store media content for distribution to the media display device 112. The server 120 may receive a media request from the media display device 112 and may provide the stored media content to the media display device 112 based on the received media request. Examples of the server 120 may include, but are not limited to, an application server, a media content server, a cloud server, a host server, a database server, a file server, a network server, or other types of servers.

[0029] The communication network 122 may include a wireless communication channel through which the media display device 112, the electronic device 102, and the server 120 can communicate with each other wirelessly. Examples of the communication network 122 include, but are not limited to, short-range communication technologies such as a Wi-Fi network, a near-field communication (NFC) channel, or a Bluetooth (BT) communication network. Various entities in the network environment 100, such as the electronic device 102, may be configured to connect to the communication network 122 according to various wireless communication protocols. Examples of such communication protocols may include, but are not limited to, at least one of a wireless device-to-device communication protocol or a Bluetooth communication protocol. In some embodiments, the communication network 122 may include a communication medium through which the media display device 112 and the server 120 can communicate with each other. Examples of the wired and wireless communication protocols of the communication network 122 through which the media display device 112 and the server 120 can communicate with each other include, but are not limited to, at least one of the Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Li-Fi, 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, and a wireless access point (AP).

[0030] In operation, the electronic device 102 may be configured to display a plurality of user interface (UI) elements for controlling different functions of the media display device 112. The electronic device 102 may be configured to control the image capture device 108 to capture a first image of the user 118 of the electronic device 102. The first image may at least include an image of the face of the user 118 to track the eye gaze of the user 118. In some embodiments, the electronic device 102 may be mounted on a wheelchair of a user (such as the user 118 with a physical disability), and may control the image capture device 108 to capture the first image of the user 118. The electronic device 102 may also be configured to determine that the position of the eye gaze of the user 118 is on the first UI element 106A among the plurality of displayed UI elements 106 based on the captured first image of the user 118. For example, in Figure 3A and Figure 3BDetails of determining that the position of the eye gaze of user 118 is on the first UI element 106A among the plurality of UI elements 106 displayed are further described. According to one embodiment, each UI element among the plurality of UI elements 106 may be associated with a different function associated with the media display device 112. The electronic device 102 may also be configured to select the first UI element 106A of the plurality of UI elements 106 based on determining that the position of the eye gaze is on the first UI element 106A for a period of time. The period of time may be preset, for example, within a range of several seconds (such as 3 to 5 seconds), or may be set based on user input. In an exemplary embodiment, the selection of the first UI element 106A may be based on other eye gestures, such as the eye blink of user 118 or the eye gaze after the blink. Thus, the electronic device 102 may determine the eye blink of user 118 at the position of the determined eye gaze on the first UI element 106A as user input to select the first UI element 106A. In one example, the first UI element 106A may correspond to an "upward movement" navigation icon that may be used to navigate among different icons in the home screen of the media display device 112.

[0031] The electronic device 102 may also be configured to send a first control signal to the media display device 112 based on the selection of the first UI element 106A among the plurality of UI elements 106. For example, the first control signal sent enables navigation between UI icons. For example, in Figure 3BDetails of navigating between UI icons based on the selection of a first UI element 106A are provided herein. The electronic device 102 may also be configured to receive a feedback image 110 from the media display content 102 based on the first control signal transmitted. In an exemplary embodiment, the feedback image 110 may be an image in the home screen of the media display device 112, where the media display device 112 shows the selection of a UI icon in the home screen of the media display device 112. The electronic device 102 may also be configured to control the first display screen 104 to display the received feedback image 110 with the plurality of UI elements 106 shown thereon. Thus, the electronic device 102 may allow the user 118 to control the media display device 112 via eye gaze as user input provided to the electronic device 102. Controlling the media display device 112 via the eye gaze of the user 118 allows the user 118 to view and control different functions of the media display device 112 without the assistance of other users. The electronic device 102 may also be configured to control the first display screen 104 to display at least a portion of the media content 116 being played on the media display device 112 as part of the received feedback image 110. Thus, the electronic device 102 may allow the user 118 to control the media display device 112 while being able to continuously view the media content 116. As a result, the user 118 does not miss a frame or a portion of the media content 116 being played on the media display device 112 (such as a television) during the provision of user input to control the media display device 112. Thus, a seamless viewing experience may be achieved for the user 118.

[0032] According to one embodiment, the electronic device 102 may be configured to control the movement of a cursor on the plurality of UI elements 106 in the media display device 102, the movement of the cursor corresponding to a change in the position of the eye gaze of the user 118 on the plurality of UI elements 106. For example, the electronic device 102 may move the cursor to the first UI element 106A when determining that the position of the eye gaze of the user 118 is on the first UI element 106A, and may move the cursor to the second UI element 106B when determining that the position of the eye gaze of the user 118 is on the second UI element 106B. In some embodiments, the electronic device 102 may be configured to highlight the first UI element 106A based on the selection of the first UI element 106A by the user 118 among the plurality of UI elements 106. For example, details of the movement of the cursor and the highlighting of the first UI element 106A are further described in Figure 3B wherein.

[0033] According to one embodiment, the feedback image 110 received from the media display device 112 may correspond to a UI icon for indicating control of the media display device 112 based on the first control signal transmitted. Examples of UI icons may include, but are not limited to, a "volume up" icon, a "volume down" icon, a "brightness control" icon, or a "mode selection" icon. For example, details of the reception of the UI icon as the feedback image 110 are further described in Figure 3A , 3B , 4A, 4B, 5A, and 5B.

[0034] According to one embodiment, the electronic device 102 may also be configured to control the first display screen 104 to display a plurality of UI elements 106 on a first portion of the first display screen 104 and display the received feedback image 110 on a second portion of the first display screen 104. For example, details of displaying the plurality of UI elements 106 on the first portion and displaying the received feedback image 110 on the second portion of the first display screen 104 are further described in Figure 3A , 3B , 4A, 4B, 5A, and 5B.

[0035] According to one embodiment, the feedback image 110 received from the media display device 112 may correspond to media content 116 displayed on a second display screen 114 of the media display device 112 based on the first control signal transmitted. For example, details of receiving the media content 116 (or a portion of the media content 116) as the feedback image 110 are further described in Figure 3A , 3B , 4A, 4B, 5A, and 5B.

[0036] According to one embodiment, the electronic device 102 may also be configured to control the overlay of the plurality of displayed UI elements 106 on the displayed feedback image 110, where the feedback image 110 corresponds to media content displayed on the second display screen 114 of the media display device 112. In addition, the electronic device 102 may be configured to control the transparency level of the plurality of displayed UI elements 106 overlaid on the displayed feedback image 110 such that the user 118 can easily view the media content 116 presented during the control of the media display device 112 on the electronic device 102. For example, details of the plurality of UI elements 106 overlaid on the displayed feedback image 110 and the control of the transparency level of the plurality of displayed UI elements 106 are further described in Figure 4A .

[0037] According to one embodiment, the electronic device 102 may also be configured to control the first display screen 104 to display information related to the media content 116 displayed on the second display screen 114 of the media display device 112 based on the selection of one or more UI elements in the plurality of UI elements 106. For example, the displayed information may correspond to a short description of the media content 116 being presented on the media display device 112. Figure 4B Details of controlling the first display screen 104 to display information related to the media content 106 are further described in.

[0038] According to one embodiment, the electronic device 102 may also be configured to present guidance information to the user 118 to control one or more configuration settings of the media display device 112 through one or more UI elements in the plurality of UI elements 106 displayed on the first display screen 104. In an exemplary embodiment, the guidance information to the user 118 may be, for example, information about operations associated with one or more UI elements in the plurality of UI elements 106. For example, in Figure 5A Details of the guidance information for user 118 are further described in .

[0039] According to one embodiment, the electronic device 102 may also be configured to control the first display screen 104 based on the received feedback image 110 to display a group of UI elements in the plurality of UI elements 106 on the first display screen 104. In one example, the group of UI elements in the plurality of UI elements 106 may be associated with control settings of the media content 116 in the media display device 112. For example, in Figure 5B Details of the display of this set of UI elements are further described in .

[0040] Figure 2 is a diagram illustrating an embodiment according to the present disclosure Figure 1 A block diagram of an exemplary electronic device in FIG. Figure 2 , a block diagram 200 of an electronic device 102 is shown. The electronic device 102 may include a circuit 202, a memory 204, and a network interface 206. The electronic device 102 may also include a first display screen 104. The first display screen 104 may display a plurality of UI elements 106. The electronic device 102 may also include an image capture device 108. Figure 1 , the first display screen 104, the image capture device 108, and the communication network 122 have been described in detail, and these detailed descriptions have been omitted here for the sake of brevity.

[0041] Circuit 202 may include suitable logic, circuitry, and / or interfaces configured to execute program instructions associated with different operations to be performed by electronic device 102. For example, some of these operations may include determining that the position of user 118's eye gaze is on first UI element 106A, selecting first UI element 106A based on determining the position of user 118's eye gaze, sending a first control signal to media display device 112, receiving a feedback image from media display device 112, and controlling first display screen 104 to display feedback image 110 and multiple UI elements 106. Circuit 202 may include one or more dedicated processing units that may be implemented as separate processors. In one embodiment, the one or more dedicated processing units may be implemented as an integrated processor or a processor cluster that jointly executes the functions of the one or more dedicated processing units. Circuit 202 may be implemented based on several processor technologies known in the art. Examples of implementations of circuit 202 may be an X-86 based processor, a graphics processing unit (GPU), a reduced instruction set computing (RISC) processor, an application specific integrated circuit (ASIC) processor, a complex instruction set computing (CISC) processor, a microcontroller, a central processing unit (CPU), and / or other control circuitry.

[0042] Memory 204 may include suitable logic, circuitry, interfaces, and / or code that may be configured to store one or more instructions for circuit 202 to execute. Memory 204 may be configured to store information corresponding to the functions associated with each of the UI elements among multiple UI elements 106. In some embodiments, memory 204 may be configured to store user input associated with user 118. Examples of ways memory 204 may be implemented may include, but are not limited to, random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), hard disk drive (HDD), solid state drive (SSD), CPU cache, and / or secure digital (SD) card.

[0043] Network interface 206 may include suitable logic, circuitry, and / or interfaces configured to facilitate communication between media display device 112 and electronic device 102 via communication network 122. Network interface 206 may be implemented through the use of various known technologies to support wired and wireless communication in electronic device 102 and communication network 122. Network interface 206 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a decoder-encoder (CODEC) chipset, a subscriber identity module (SIM) card, or local buffer circuitry.

[0044] Those of ordinary skill in the art will understand that, in addition to the components or systems illustrated herein for describing and explaining the functions and operations of the present disclosure, Figure 2 the electronic device 102 in Figure 3A may also include other suitable components or systems. For the sake of brevity, the present disclosure has omitted a detailed description of other components and systems in the electronic device 102. For example, the operation of the circuit 202 is further described in 3B ,

[0045] Figure 3A and 3B are diagrams that commonly illustrate exemplary scenarios for controlling the media display device 112 in Figure 1 in accordance with embodiments of the present disclosure. Figure 3A and 3B are explained in conjunction with the elements of Figure 1 and 2 . Referring to Figure 3A , an exemplary scenario 300A is shown here that includes the electronic device 102 (e.g., an electronic tablet or a smart phone) in Figure 1 , a first display screen 104, a plurality of UI elements 106, an image capture device 108 (e.g., a built-in front camera), a media display device 112 (e.g., a television), a second display screen 114, and a user 118. Additionally, in this exemplary scenario 300A, a first UI element 306 (such as a "volume up" element) among the plurality of UI elements 106 (e.g., "volume control" UI element, "power" UI element, "source" UI element, "playback control" UI element, "navigation" UI element, "home" UI element, "number" UI element, "information" UI element, "return" UI element, and "exit" UI element) is also shown here. In this exemplary scenario 300A, a received feedback image 304, a cursor 308, a first portion 310 of the first display screen 104, and a second portion 312 of the first display screen 104 are also shown here. Further, media content 302 and a "volume up" UI icon 302a on the second display screen 114 of the media display device 112 are also shown here.

[0046] Referring to Figure 3A, the media display device 112 can be configured to present media content 302, such as a movie on the second display screen 114. In an exemplary embodiment, the user 118 may need to increase the volume of the media content 302 presented on the media display device 112. In such a case, the user 118 can gaze at a plurality of UI elements 106 of the electronic device 102 to select a first UI element 306 (such as a "volume up" icon) to increase the volume of the media content 302 presented on the media display device 112. The circuit of the electronic device 102 can be configured to control the image capture device 108 to capture a first image of the user 118. The image capture device 108 can capture a first image of the user 118 based on the field of view (FOV) of the image capture device 108. The FOV of the image capture device 108 can be predefined to accurately capture the first image of the user 118 (or an image of the face of the user 118). The electronic device 102 can be positioned in such a way that the image capture device 108 can clearly capture the first image of the face of the user 118. For example, the electronic device 102 can be mounted near the user 118 or can be in the user 118's hand. In another example, the electronic device 102 can be mounted on the user 118's wheelchair (such as on the armrest or the backrest of the wheelchair). It should be noted that, as Figure 3A and 3B shown, the mounting location or position of the electronic device 102 on which the image capture device 108 is placed is only presented as an example. Without departing from the scope of the present disclosure, the image capture device 108 can be placed at other positions, locations or surfaces of the electronic device 102 such that the FOV of the image capture device 108 can clearly capture the first image of the user 118. For the sake of brevity, the present disclosure omits other positions, locations or surfaces of the electronic device 102 on which the image capture device 108 can be placed.

[0047] The circuit 202 of the electronic device 102 can also be configured to extract an image portion from the captured first image of the face of the user 118. This image portion can indicate the eyes of the face of the user 118. The circuit 202 can extract the image portion (such as a sub-image or pixels that may include the eyes) from the captured first image based on different image processing techniques known in the art. The circuit 202 can also be configured to track the movement of the eyes of the user 118 based on the extracted image portion. In some embodiments, the circuit 202 can control the image capture device 108 for a period of time to capture a plurality of images of the face of the user 118 and can also track the movement of the eyes of the user 118 based on the analysis of the image portion in each of the captured plurality of images. The movement of the eyes can indicate where the user 118 may be gazing (e.g., towards the electronic device 102 or elsewhere).

[0048] The circuitry of the electronic device 102 may also be configured to determine that the position of the user 118's eye gaze is on a first UI element 306 (such as a "volume up" UI element) among the plurality of UI elements 106 in the electronic device 102 based on the tracked movement of the user 118's eyes. In some embodiments, the circuitry 202 may determine that the position of the eye gaze is on any of the plurality of UI elements 106 of the electronic device 102 based on the captured first image.

[0049] According to one embodiment, the circuitry 202 may also control the movement of the cursor 308 on the plurality of UI elements 106 in correspondence with a change in the position of the user 118's eye gaze on the plurality of UI elements 106. When the user 118's eye gaze moves to different positions corresponding to different UI elements among the plurality of UI elements 106 in a first portion 310 of the first display screen 104, the circuitry may control the movement of the cursor 308 to the UI element that the user 118 is currently looking at. For example, when the circuitry 202 determines that the user 118's eye gaze is on an "upward movement" navigation UI element, the circuitry 202 may position the cursor 308 at the "upward movement" navigation UI element. Additionally, when the circuitry 202 determines that the user 118's eye gaze is on the first UI element 306 (such as a "volume up" UI element), the circuitry 202 may move the cursor 308 to the "volume up" UI element 306. The movement of the cursor 308 on the plurality of UI elements 106 may assist the user 118 in visually confirming the position of the user 118's eye gaze on the electronic device 102.

[0050] The circuitry 202 in the electronic device 102 may also select the first UI element 306 (such as a "volume up" UI element) when it determines that the user 118's eye gaze has been on the first UI element 306 (such as a "volume up" UI element) for a certain period of time (such as several seconds). In an exemplary embodiment, the circuitry 202 may select the first UI element 306 (such as a "volume up" UI element) among the plurality of UI elements 106 based on other eye postures. For example, the circuitry 202 may be configured to select the "volume up" UI element 306 based on the detection of a "blink" of the user 118's eyes while the eye gaze is determined to be on the first UI element 306 (such as a "volume up" UI element). In some embodiments, the circuitry 202 may be configured to determine at least one of the user's facial expression (e.g., smile) or the posture associated with the user 118's head (e.g., the tilt of the user 118's head in one direction) as a user input for selecting the first UI element 306 (such as a "volume up" UI element) among the plurality of UI elements 106 displayed on the first display screen 104.

[0051] According to one embodiment, asFigure 3A The circuit 202 shown in may also highlight a first UI element 306 (such as the "volume up" UI element) as confirmation of the selection of the first UI element 306 among the plurality of UI elements 106 (e.g., the "volume up" UI element). The highlighting of the first UI element 306 (i.e., the "volume up" UI element) can help the user 118 identify the selected first UI element 306. In one or more embodiments, the circuit 202 may be configured to control (e.g., increase) the size of the selected first UI element 306 to facilitate viewing of the selected UI icon by the user 118. In one example, the circuit 202 may also be configured to display the selected first UI element 306 as a "pop-up" UI element on the first display screen 104.

[0052] The circuit 202 may also be configured to send a first control signal to the media display device 112 based on the user 118's selection of the first UI element 306 (i.e., the "volume up" UI element). The circuit 202 may also be configured to send the first control signal using a short-range radio wave signal (e.g., Bluetooth) or using a wireless local area network (e.g., Wi-Fi Direct). When the media display device 112 receives the first control signal (i.e., related to the "volume up" UI element) from the electronic device 102, the first control signal may cause the media display device 112 to increase the volume of the media content 302 presented on the second display screen 114 of the media display device 112. Thus, the circuit 202 may allow the user 118 to control the volume of the media content 302 presented on the media display device 112 via the electronic device 102 by utilizing the user 118's eye gaze.

[0053] As Figure 3A shown, the media display device 112 may display a "volume up" UI icon 302a on the second display screen 114 based on a user input, and the "volume up" UI icon 302a may indicate an increase in the volume of the media content 302 presented on the media display device 112. It should be noted that the position of the "volume up" UI icon 302a shown in Figure 3A is shown only as an example. The "volume up" UI icon 302a may be located in any part of the second display screen 114 without departing from the scope of the present disclosure.

[0054] The circuitry of the electronic device 102 can also be configured to receive a feedback image 304 from the media display device 112 based on the first control signal transmitted. The feedback image 302 can correspond to an image of the "volume up" UI icon 302a as displayed on the second display screen 114. For example, when the user 118 is looking at the electronic device 102 to control the volume of the media content 302, the user may need to switch the gaze of the eyes between the electronic device 102 and the media display device 112 to confirm the change in volume indicated by the "volume up" UI icon 302a on the media display device 112. Thus, the circuitry 202 of the electronic device 102 can display the "volume up" UI icon 302a as the feedback image 304 on the electronic device 102, such that the user can confirm the volume increase through the feedback image 304 without having to switch the gaze of the eyes back and forth between the electronic device 102 and the media display device 112 to confirm the volume increase.

[0055] The circuitry 202 can also be configured to control the first display screen 104 to display the received feedback image 304 and the plurality of UI elements 106 displayed. According to one embodiment, the circuitry 202 can also be configured to control the first display screen 104 to display the plurality of UI elements 306 on a first portion 310 of the first display screen 104, and display the received feedback image 304 on a second portion 312 of the first display screen 104. Displaying the plurality of UI elements 106 and the feedback image 304 on different portions (such as the first portion 310 and the second portion 302, respectively) can allow the user 118 to view the image of the UI icon, such as the image of the "volume up" UI icon 302a, while controlling the volume of the media content 302 presented on the media display device 112. It should be noted here that Figure 3A the position of the feedback image 304 on the second portion 312 of the first display screen 104 shown is only for example. Without departing from the scope of the present disclosure, the feedback image 304 can appear at different positions on the second portion 312 of the first display screen 104. Moreover, as Figure 3AThe sizes and positions of the first portion 310 and the second portion 312 in the first display screen 104 shown are only presented as examples. Without departing from the scope of the present disclosure, the sizes and positions of the first portion 310 and the second portion 312 in the first display screen 104 can be represented in different ways. For example, on the first display screen 104, the first portion 310 can appear below the second portion 312. The electronic device 102 of the present disclosure can thus enable the user 118 to effectively control the media display device 112 based on eye gaze as user input without assistance from other users. The electronic device 102 of the present disclosure can enable the user 118 to effectively control the media display device 112 based on eye gaze as user input without the need to connect the electronic device 102 to an external network (e.g., the Internet).

[0056] Reference Figure 3B , an exemplary scenario 300B is shown here. In this exemplary scenario 300B, media content 314, a feedback image 316, a "move left" navigation UI element 318, a "program guide" UI element 320, and an image 322 corresponding to the "program guide" UI icon 320 are shown here.

[0057] The media display device 112 can display the home screen of the media display device 112 as media content 314 on the second display screen 114 based on the user input of the user 118. The user 118 may need to navigate among a set of UI icons that appear on the home screen in the media display device 112. This set of UI icons can include, for example, an "app" icon, a "game" UI icon, a "playlist" UI icon, a "settings" UI icon, a "search" UI icon, an "information" icon, and so on. Thus, the circuit 202 of the electronic device 102 can determine that the position of the user 118's eye gaze is on a navigation UI element among the plurality of UI elements 106. The circuit 202 can determine that the position of the user 118's eye gaze is on a first UI element (i.e., the "move left" navigation UI element 318) among the plurality of UI elements 106. The circuit 202 can select the "move left" navigation UI element 318 based on determining that the position of the user 118's eye gaze is on the "move left" navigation UI element 318.

[0058] The circuit 202 can also send a first control signal to the media display device 112 to navigate to a UI icon (on the second display screen 114) that can appear to the left of the currently selected UI icon among this set of UI icons. For example, the first control signal can enable the selection of the "program guide" UI icon 320 by navigating from the "app" UI icon among this set of UI icons that appears on the right side of the home screen to the "program guide" UI icon 320 that appears on the left side of the home screen.

[0059] The circuit 202 can also be configured to receive a feedback image 316 based on the first control signal transmitted. The received feedback image 316 can correspond to an image associated with the media content 314 of the media display device 112 (e.g., the home screen image shown in Figure 3B ). The feedback image 316 can depict the selection of the "program guide" UI icon 320 by highlighting the image 322 corresponding to the "program guide" UI icon 320. Thus, the electronic device 102 can enable the user 118 to effectively navigate this set of UI elements of the media content 304 without the need to switch the eye gaze back and forth between the electronic device 102 and the media display device 112. Therefore, the electronic device 102 can allow the user to control the media display device 112 while being able to continuously view the media content 314. As a result, during the provision of user input to control the media display device 112, the user 118 will not miss a frame or a part of the media content 314 being played on the media display device 112 (such as a television). Therefore, a seamless viewing experience can be achieved for the user 118.

[0060] Figure 4A and 4B are diagrams that jointly illustrate other exemplary scenarios for controlling the Figure 1 media display device 112 in the present disclosure. Figure 4A and 4B are explained in conjunction with the elements of Figure 1 and 2 . Referring to Figure 4A , an exemplary scenario 400A is shown here. In this exemplary scenario 400A, media content 402, a feedback image 404, and a first UI element 406 (such as a "volume up" UI element) are shown here. It should be noted that the function of the first UI element 406 (such as a "volume up" UI element) can be the same as the function of the "volume up" UI element (shown as the first UI element 306) described in, for example, Figure 3A .

[0061] The circuit 202 can determine that the position of the user 118's eye gaze is on the first UI element 406 (such as a "volume up" UI element) among the plurality of UI elements 106. The circuit 202 can also transmit a first control signal for controlling the volume of the media content 402 based on determining that the position of the user 118's eye gaze is on the first UI element 406 (such as a "volume up" UI element). Moreover, the circuit 202 can receive the feedback image 404 from the media display device 112 so that the feedback image 404 can include an image of the "volume up" UI icon 402a superimposed on the media content 402, as shown in Figure 4A .

[0062] According to one embodiment, the circuitry 202 of the electronic device 102 may control the overlay of the plurality of displayed UI elements 106 on the displayed feedback image 404. The circuitry 202 may control the first display screen 104 to display the plurality of UI elements 106 on top of the feedback image 404 in the same portion of the first display screen 104 of the electronic device 102, as Figure 4A shown.

[0063] In one or more embodiments, the circuitry 202 may control the transparency level of the plurality of displayed UI elements 106 overlayed on the displayed feedback image 404. The circuitry 202 may, for example, reduce the transparency level of the plurality of displayed UI elements 106 by a certain amount so that the media content 402 of the received feedback image 404 can be easily viewed through the plurality of displayed UI elements 106. For example, the transparency level of the plurality of displayed UI elements 106 may be reduced to a level of 40% transparency so that the display of the media content 402 of the feedback image 404 can be visible to the user 118. Such an arrangement of overlaying the plurality of UI elements 106 with a reduced transparency level on the feedback image 404 may allow the user to view both the feedback image 404 and the plurality of UI elements 106 while controlling the media display device 112 via the user 118's eye gaze. Moreover, such an arrangement may provide the user 118 with the ability to view the plurality of UI elements 106 and the feedback image 404 over the full screen (or a specific portion) of the first display screen 104. In an alternative embodiment, the plurality of UI elements 106 may be overlayed on the feedback image 404 such that the transparency level of the feedback image 404 is controlled (e.g., reduced), rather than controlling the transparency level of the plurality of UI elements 106. Since the feedback image 404 and the plurality of UI elements 106 are simultaneously displayed on the electronic device 102, the electronic device 102 may thus allow the user to control the media display device 112 while continuously viewing the media content 402 on the electronic device 102.

[0064] Referring Figure 4B , an exemplary scenario 400B is shown here. In this exemplary scenario 400B, the media content 404, the "information" UI element 410 (such as the first UI element) among the plurality of UI elements 106, the feedback image 412, the first information 414, and the third portion 416 of the first display screen 104 of the electronic device 102 are shown here.

[0065] In some embodiments, a user may need information about media content (e.g., media content 408 presented on media display device 112). For example, media display device 112 may present a documentary as media content 408. User 118 may need information about the content of the documentary, such as the title or abstract of the documentary, and move the position of their eye gaze to the "Info" UI element 410 (i.e., the first UI element) among the plurality of UI elements 106. In such a case, circuit 202 may determine that the position of user 118's eye gaze is on the "Info" UI element 410.

[0066] Circuit 202 may also select the "Info" UI element 410 among the plurality of UI elements 106 based on determining that the position of user 118's eye gaze is on the "Info" UI element 410. As a result, circuit 202 may send a first control signal to media display device 112 to receive information, such as first information 414 related to media content 408 displayed on media display device 112. In one example, first information 414 may be information about the documentary presented on media display device 112. In one embodiment, circuit 202 may also receive a feedback image, such as feedback image 412 corresponding to media content 408 with first information 414 from media display device 112. In some embodiments, circuit 202 may receive first information 414 related to media content 408 from server 120 (shown in Figure 1 ).

[0067] According to one embodiment, circuit 202 may be configured to control first display screen 104 to display information, such as first information 414 related to media content 408 displayed on second display screen 114 of media display device 112. Circuit 202 may control first display screen 104 to display first information 414 based on selecting the "Info" UI element 410 among the plurality of UI elements 106 through user 118's eye gaze. In one embodiment, circuit 202 may display first information 414 on the third portion 416 of first display screen 104 (i.e., different from the first portion 310 and the second portion 312 shown in Figure 3A ). Additionally, circuit 202 may also control first display screen 104 to display feedback image 412 on first display screen 104 of electronic device 102. It should be noted here that the position of the third portion 416 shown in Figure 4A and Figure 4B on first display screen 104 is shown only as an example. Without departing from the scope of the present disclosure, the present disclosure is also applicable to other positions of the third portion 416 on first display screen 104.

[0068] In one or more embodiments, a plurality of UI elements 106, a feedback image 412, and first information 414 may be displayed on three different portions of a first display screen 104. In an exemplary embodiment, the display positions of the plurality of UI elements 106, the feedback image 412, and the first information 414 may be preset. In another exemplary embodiment, the display positions of the plurality of UI elements 106, the feedback image 412, and the first information 414 may be set based on user input. In an exemplary embodiment, the plurality of UI elements 106 may be superimposed on the feedback image 412 and the first information 414 such that the plurality of UI elements 106 may be displayed on top of the feedback image 412 and the first information 414 at a controlled transparency level, such as a reduced transparency level, as referenced Figure 4A as described.

[0069] Figure 5A and 5B are jointly illustrative of other exemplary scenarios for controlling Figure 1 the media display device 112 in accordance with embodiments of the present disclosure. Figure 5A and 5B in conjunction with the elements in Figure 1 and 2 are explained. Referencing Figure 5A , an exemplary scenario 500A is shown herein. In this exemplary scenario 500A, a "Home" UI element 502 (i.e., a first UI element) among the plurality of UI elements 106, guidance information 504, and media content 506 are shown herein. As shown in Figure 5A , the media content 506 may be related to content related to news presented on the media display device 112.

[0070] According to one embodiment, the media display device 112 may present media content 506 (e.g., news content). The user 118 may need to view the home screen of the media device 112 to switch to other media content on the media display device 112 and may also gaze at the "Home" UI element 502. The circuitry 202 in the electronic device 102 may be configured to present guidance information 504 (e.g., information about the operation of UI elements such as the "Home" UI element 502) for the user 118 to view the home screen (e.g., the main page) of the media display device 112 based on determining the position of the user 118's eye gaze on the "Home" UI element 502 among the plurality of UI elements 106. Thus, the circuitry 202 may select the "Home" UI element 502 to display the home screen on the second display screen 114 of the media display device 112 based on the user 118's eye gaze on the "Home" UI element 502 (in Figure 5A(not shown in the figure). Additionally, guidance information 504 may be presented to control one or more configuration settings of the media display device 112 through one or more of the plurality of UI elements 106 displayed on the first display screen 104. For example, the guidance information 504 may provide the user 118 with information for configuring specific settings of the media display device 112 (e.g., but not limited to, adjustment of color, adjustment of aspect ratio). In one embodiment, a plurality of UI elements 106 and guidance information 504 may be provided in different parts of the first display screen 104 of the electronic device 102. It should be noted here that the position of the guidance information 504 shown in Figure 5A on the first display screen 104 is only shown as an example. Without departing from the scope of the present disclosure, the present disclosure is also applicable to other positions of the guidance information 504 on the first display screen 104.

[0071] In some embodiments, the circuit 202 may be configured to present the guidance information 504 to the user 118 by determining that the user's 118 eye gaze is on the "Information" UI element 410 (shown in Figure 4B ). In one exemplary embodiment, Figure 4B the first information 414 may be information about the operation of each of the plurality of UI elements 106 or information about the media content 506 currently presented on the media display device 112. Thus, the guidance information 504 may assist the user 118 in selecting appropriate UI elements to control one or more configuration settings or different functions of the media display device 112.

[0072] Referring to Figure 5B , an exemplary scenario 500B is shown here. In this exemplary scenario 500B, media content 508, a feedback image 510, and a "Playback Control" UI element (e.g., "Rewind" UI element 512) among the plurality of UI elements 106 are shown here. The media content may include a movie presented on the media display device 112.

[0073] According to one embodiment, the media display device 112 may present media content 508, e.g., a movie. The circuit 202 of the electronic device 102 may be configured to control the first display screen 104 to display a set of UI elements 514 (i.e., it may be Figure 5A a subset of the plurality of UI elements on the first display screen) based on the received feedback image 510. For example, as shown in Figure 5BThe set of UI elements 514 shown in FIG. may include a "Power" UI element, a "Home" UI element, an "Info" UI element, a "Volume Up" UI element, a "Volume Down" UI element, a "Rewind" UI element, a "Pause" UI element, a "Play" UI element, and a "Forward" UI element. The set of UI elements 514 may be displayed based on the type of media content 508 displayed on the media display device 112 and the feedback image 510 received by the electronic device 102. The circuit 202 may also hide or omit unimportant UI elements among the plurality of UI elements 106 based on the presented media content 508 so that the screen space of the first display screen 104 in Figure 5A is relatively less cluttered compared to the first display screen 104 in Figure 5B As a result, the circuit 202 can effectively determine that the position of the user 118's eye gaze is on any particular UI element (i.e., the first UI element) in the set of UI elements 514. In one example, the circuit 202 may hide or ignore elements such as "Navigation" UI elements, "Number" UI elements, etc. during movie playback. As shown in Figure 5B FIG., the circuit 202 may control the first display screen 104 to display the set of UI elements 514, which are the UI elements that the user needs to control the playback of media content 508 (e.g., a movie) on the media display device 112.

[0074] In an exemplary embodiment, the user 118 may need to rewind the media content 508 presented on the media display device 112. The circuit 202 may also determine that the position of the user 118's eye gaze is on the "Rewind" UI element 512 (i.e., the first UI element) in the set of UI elements 514 for a certain period of time (e.g., several seconds). The circuit 202 may select the "Rewind" UI element 512 based on determining that the user 118's eye gaze is on the "Rewind" UI element 512. The circuit 202 may also send a first control signal to the media display device 112 based on the eye gaze as user input to rewind the media content 508 on the media display device 112. In addition, the circuit 202 may receive a feedback image 510 including an image of the media content 508 from the media display device 112 based on the sent first control signal. The set of UI elements 514 and the feedback image 510 may be displayed on different parts of the first display screen 104. In some embodiments, the set of UI elements 514 may be superimposed on the feedback image 510 on the first display screen 104. Therefore, the circuit 202 can allow the user to effectively control the presentation of the media content 508 on the media display device 112 by using the set of UI elements 512. It should be noted that in Figure 5BThe set of UI elements 514 shown is only presented as an example. Without departing from the scope of the present disclosure, the present disclosure is also applicable to other sets of UI elements on the first display screen 104.

[0075] Figure 6 is a flowchart illustrating an exemplary method for controlling a media display device via an electronic device according to an embodiment of the present disclosure. Figure 6 associated with Figure 1 、 2 、3A, 3B, 4A, 4B, 5A, and 5B are described in conjunction with the elements. Referring to Figure 6 , flowchart 600 is shown herein. The operations of flowchart 600 can be performed by a computing system such as electronic device 102 or circuit 202. The operations begin at 602 and end at 604.

[0076] At 604, the image capture device 108 can be controlled to capture a first image of a user of the electronic device 102. In one or more embodiments, for example, the circuit 202 of the electronic device 102 can be configured to control the image capture device 108 of the electronic device 102 to capture a first image of the user 118 of the electronic device 102, as described, for example, in Figure 3A .

[0077] At 606, it can be determined that the position of the user 118's eye gaze is on a first UI element among the plurality of displayed UI elements based on the captured first image of the user 118. In one or more embodiments, for example, the circuit 202 of the electronic device 102 can be configured to determine that the position of the user 118's eye gaze is on a first UI element 106A among the plurality of displayed UI elements 106 based on the captured first image of the user 118, as described, for example, in Figure 3A .

[0078] At 608, the first UI element 106A among the plurality of UI elements 106 can be selected based on determining that the position of the eye gaze is on the first UI element 106A for a period of time. In one or more embodiments, for example, the circuit 202 of the electronic device 102 can be configured to select the first UI element 106A among the plurality of UI elements 106 based on determining that the position of the eye gaze is on the first UI element 106A for a period of time, as described, for example, in Figure 3A . In some embodiments, the period of time can be several seconds (such as at least 3 seconds).

[0079] At 610, based on the selection of the first UI element 106A among the multiple UI elements 106, a first control signal can be sent to the media display device 112. In one or more embodiments, for example, the circuit 202 of the electronic device 102 can send the first control signal to the media display device 112 based on the selection of the first UI element 106A among the multiple UI elements 106, as described, for example, in Figure 3A as described.

[0080] At 612, a feedback image can be received from the media display device 112 based on the sent first control signal. In one or more embodiments, for example, the circuit 202 of the electronic device 102 can receive the feedback image 304 from the media display device 112 based on the sent first control signal, as described, for example, in Figure 3A as described. The feedback image 304 can indicate the control of the media display device 112 based on the sent first control signal.

[0081] At 614, the first display screen can be controlled to display the feedback image 304 together with the multiple displayed UI elements 106. In one or more embodiments, for example, the circuit 202 of the electronic device 102 can be configured to control the first display screen 104 to display the received feedback image 304 together with the multiple displayed UI elements 106, as described, for example, in Figure 3A as described. This control can go to the end.

[0082] Although the flowcharts are illustrated as discrete operations, such as 604, 606, 608, 610, 612, and 614, the present disclosure is not limited thereto. Accordingly, in certain embodiments, without detracting from the essence of this embodiment, such discrete operations can also be divided into additional operations, combined into fewer operations, or eliminated depending on the specific application.

[0083] Various embodiments of the present disclosure may provide a non-transitory computer-readable medium having computer-readable instructions stored thereon that, when executed by an electronic device (electronic device 102) including an image capture device (such as image capture device 108) and a first display screen (such as first display screen 104), cause the electronic device 102 to perform operations. These operations may include controlling the first display screen to display a plurality of user interface (UI) elements. Selection control of each of the plurality of UI elements controls a media display device (such as media display device 112) communicatively coupled to the electronic device. These operations also include controlling the image capture device to capture a first image of a user of the electronic device. The operations also include determining, based on the captured first image of the user, that a position of the user's eye gaze is on a first UI element among the displayed UI elements. These operations also include selecting the first UI element among the plurality of UI elements based on determining that the eye gaze position is on the first UI element for a period of time. These operations also include sending a first control signal to the media display device based on the selection of the first UI element among the plurality of UI elements. These operations also include receiving a feedback image from the media display device based on the sent first control signal. The received feedback image indicates control of the media display device based on the sent first control signal. These operations also include controlling the first display screen to display the received feedback image together with the plurality of displayed UI elements.

[0084] Exemplary aspects of the present disclosure may include an electronic device (such as electronic device 102). The electronic device 102 may include an image capture device (such as image capture device 108) and a first display screen (such as first display screen 104) that may be configured to display a plurality of user interface (UI) elements (such as a plurality of UI elements 106). Selection of each of the plurality of UI elements 106 may control a media display device (such as media display device 112) communicatively coupled to the electronic device 102. The electronic device 102 may also include circuitry (such as circuitry 202) coupled to the image capture device 108 and the first display screen 104. The circuitry 202 may be configured to control the image capture device 108 to capture a first image of a user (e.g., user 118) of the electronic device 102. The circuitry 202 may also be configured to determine, based on the captured first image of the user 118, that the position of the user 118's eye gaze is on a first UI element (e.g., first UI element 106A) of the plurality of UI elements 106 being displayed. The circuitry 202 may also be configured to select the first UI element 106A of the plurality of UI elements 106 based on determining that the position of the eye gaze has been on the first UI element 106A for a period of time. The circuitry 202 may also be configured to send a first control signal to the media display device 112 based on the selection of the first UI element 106A of the plurality of UI elements 106. The circuitry 202 may also be configured to receive a feedback image (e.g., feedback image 110) from the media display device 112 based on the sent first control signal. The received feedback image 110 may indicate the control of the media display device 112 based on the sent first control signal. The circuitry 202 may also be configured to control the first display screen 104 to display the received feedback image 110 together with the plurality of UI elements 106 being displayed.

[0085] According to one embodiment, each UI element of the plurality of UI elements 106 may be associated with a different function associated with the media display device 112.

[0086] According to one embodiment, the circuitry 202 may also be configured to control the movement of a cursor (such as cursor 308) on the plurality of UI elements 106 in correspondence with a change in the position of the user 118's eye gaze on the plurality of UI elements 106.

[0087] According to one embodiment, the circuitry 202 may also be configured to highlight the first UI element 106A based on the selection of the first UI element 106A of the plurality of UI elements 106.

[0088] According to one embodiment, the feedback image 304 received from the media display device 112 may correspond to a UI icon (such as UI icon 302a) for indicating the control of the media display device 112 based on the sent first control signal.

[0089] According to one embodiment, the feedback image 316 received from the media display device 112 may correspond to media content (such as media content 314) displayed on a second display screen (such as the second display screen 114) of the media display device 112 based on the first control signal transmitted.

[0090] According to one embodiment, the circuit may also be configured to control the overlay of a plurality of displayed UI elements 106 on the displayed feedback image 404, and the feedback image 404 may correspond to media content 402 displayed on the second display screen 114 of the media display device 112.

[0091] According to one embodiment, the circuit 202 may also be configured to control the transparency level of the plurality of displayed UI elements 106 overlaid on the displayed feedback image 404.

[0092] According to one embodiment, the circuit 202 may also be configured to control the first display screen 104 to display information (such as first information 414) related to the media content 408 displayed on the second display screen of the media display device 112 based on the selection of one or more of the plurality of UI elements 106.

[0093] According to one embodiment, the circuit 202 may also be configured to control the first display screen 104 to display the plurality of UI elements 106 on a first portion (such as the first portion 310) of the first display screen 104, and display the received feedback image 304 on a second portion (such as the second portion 312) of the first display screen 104. The first portion 310 may be different from the second portion 312.

[0094] According to one embodiment, the circuit 202 may also be configured to determine at least one of a gesture associated with the facial expression of the user 118 or a gesture associated with the head of the user 118 to select a first UI element 106A among the plurality of UI elements 106 displayed on the first display screen 104.

[0095] According to one embodiment, the circuit 202 may also be configured to present guidance information (e.g., guidance information 504) to the user 118 to control one or more configuration settings of the media display device 112 through one or more of the plurality of UI elements 106 displayed on the first display screen 104.

[0096] According to one embodiment, the circuit 202 may also be configured to control the first display screen 104 to display a set of UI elements among the plurality of UI elements 106 on the first display screen 104 based on the received feedback image 510.

[0097] The present disclosure can be implemented in hardware or in a combination of hardware and software. The present disclosure can be implemented in a centralized manner, in at least one computer system, or in a distributed manner in which different elements can be distributed across several interconnected computer systems. A computer system or other components suitable for performing the methods described herein may be appropriate. The combination of hardware and software may be a general-purpose computer system with a computer program that, when loaded and executed, can control the computer system to perform the methods described herein. The present disclosure can be implemented in hardware as part of an integrated circuit that also performs other functions.

[0098] The present disclosure can also be embedded in a computer program product that includes all the features enabling the implementation of the methods described herein and that, when loaded into a computer system, is capable of executing these methods. In this context, a computer program means any expression of a set of instructions represented in any language, code, or notation, which instructions are intended to cause a system with information processing capabilities to perform a particular function either directly or after one or both of the following: a) conversion to another language, code, or notation; b) reproduction in a different physical form.

[0099] Although the present disclosure has been described with reference to certain embodiments, those skilled in the art will understand that various changes and substitutions of equivalents can be made without departing from the scope of the present disclosure. In addition, many modifications can be made to adapt a particular situation or material to the teachings of the present disclosure without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the particular embodiments disclosed, but the present disclosure will include all embodiments falling within the scope of the appended claims.

Claims

1. An electronic device for controlling a media display device, comprising: an image capture device; a first display screen configured to display a plurality of UI (user interface) elements, wherein the selection control of each of the plurality of UI elements is communicatively coupled to the media display device of the electronic device; and a circuit coupled to the image capture device and the first display screen, wherein the circuit is configured to: control the image capture device to capture a first image of a user of the electronic device; determine, based on the captured first image of the user, that the position of the user's eye gaze is on a first UI element among the plurality of displayed UI elements; select the first UI element among the plurality of UI elements based on determining that the position of the eye gaze is on the first UI element for a period of time; send a first control signal to the media display device based on the selection of the first UI element among the plurality of UI elements; receive a feedback image from the media display device based on the sent first control signal, wherein the received feedback image indicates the control of the media display device based on the sent first control signal; and control the first display screen to display the received feedback image together with the plurality of displayed UI elements, wherein the circuit is further configured to: display a subset of the plurality of UI elements on the first display screen, the subset being based on the received feedback image and the type of media content displayed on a second display screen of the media display device; determine that the position of the user's eye gaze is on a second UI element among the displayed subset of UI elements; select the second UI element from the subset based on determining that the position of the eye gaze is on the second UI element for a period of time; and allow the user to effectively control the presentation of media content on the second display screen by using the second UI element selected from the subset.

2. The electronic device according to claim 1, wherein each UI element among the plurality of UI elements is associated with a different function associated with the media display device.

3. The electronic device according to claim 1, wherein the circuit is further configured to control the movement of a cursor on the plurality of UI elements corresponding to a change in the position of the user's eye gaze on the plurality of UI elements.

4. The electronic device according to claim 1, wherein the circuit is further configured to highlight the first UI element based on the selection of the first UI element among the plurality of UI elements.

5. The electronic device according to claim 1, wherein the received feedback image from the media display device corresponds to a UI icon for indicating the control of the media display device based on the sent first control signal.

6. The electronic device according to claim 1, wherein the received feedback image from the media display device corresponds to the media content displayed on the second display screen of the media display device based on the sent first control signal.

7. The electronic device according to claim 6, wherein the circuit is further configured to control an overlay of the plurality of displayed UI elements on a displayed feedback image corresponding to the media content displayed on a second display screen of the media display device.

8. The electronic device according to claim 7, wherein the circuit is further configured to control a transparency level of the plurality of displayed UI elements overlaid on the displayed feedback image.

9. The electronic device according to claim 6, wherein the circuit is further configured to control the first display screen to display information related to the media content displayed on the second display screen of the media display device based on a selection of one or more of the plurality of UI elements.

10. The electronic device according to claim 1, wherein the circuit is further configured to control the first display screen to display the plurality of UI elements on a first portion of the first display screen and to display the received feedback image on a second portion of the first display screen, and wherein the first portion is different from the second portion.

11. The electronic device according to claim 1, wherein the circuit is further configured to determine at least one of a gesture associated with the user's facial expression or a gesture associated with the user's head to select a first UI element among the plurality of UI elements displayed on the first display screen.

12. The electronic device according to claim 1, wherein the circuit is further configured to present guidance information to the user to control one or more configuration settings of the media display device through one or more of the plurality of UI elements displayed on the first display screen.

13. A method for controlling a media display device, comprising: In an electronic device including an image capture device and a first display screen configured to display a plurality of UI (user interface) elements, wherein a selection of each of the plurality of UI elements controls a media display device communicatively coupled to the electronic device; Controlling the image capture device to capture a first image of a user of the electronic device; Based on the captured first image of the user, determining a position of the user's eye gaze on a first UI element among the plurality of displayed UI elements; Selecting the first UI element among the plurality of UI elements based on determining that the position of the eye gaze is on the first UI element for a period of time; Sending a first control signal to the media display device based on the selection of the first UI element among the plurality of UI elements; Receiving a feedback image from the media display device based on the sent first control signal, wherein the received feedback image indicates control of the media display device based on the sent first control signal; And Controlling the first display screen to display the received feedback image together with the plurality of displayed UI elements, The method further includes: Displaying a subset of the plurality of UI elements on the first display screen, the subset being based on the received feedback image and a type of media content displayed on a second display screen of the media display device. Determine that the position where the user's eye gaze is located is on a second UI element in a subset of the displayed UI elements; Select the second UI element from the subset based on determining that the eye gaze position is on the second UI element for a period of time; and Allow the user to effectively control the presentation of media content on the second display screen by using the second UI element selected from the subset.

14. The method according to claim 13, further comprising controlling the movement of a cursor on the plurality of UI elements corresponding to a change in the position of the user's eye gaze on the plurality of UI elements.

15. The method according to claim 13, wherein the feedback image received from the media display device corresponds to the media content displayed on the second display screen of the media display device based on the first control signal sent.

16. The method according to claim 15, further comprising controlling the overlay of the plurality of displayed UI elements on the displayed feedback image corresponding to the media content displayed on the second display screen of the media display device.

17. The method according to claim 16, further comprising controlling the transparency level of the plurality of displayed UI elements overlaid on the displayed feedback image.

18. The method according to claim 13, further comprising controlling the first display screen to display the plurality of UI elements on a first part of the first display screen and display the received feedback image on a second part of the first display screen, and wherein the first part is different from the second part.

19. A non-transitory computer-readable medium storing computer-executable instructions, the computer-executable instructions, when executed by an electronic device including an image capture device and a first display screen, cause the electronic device to perform operations, the operations including: Controlling the first display screen to display a plurality of UI (user interface) elements, wherein the selection control of each of the plurality of UI elements is communicatively coupled to a media display device communicatively coupled to the electronic device; Controlling the image capture device to capture a first image of the user of the electronic device; Determining, based on the captured first image of the user, the position where the user's eye gaze is located on a first UI element among the plurality of displayed UI elements; Selecting the first UI element among the plurality of UI elements based on determining that the eye gaze position is on the first UI element for a period of time; Sending a first control signal to the media display device based on the selection of the first UI element among the plurality of UI elements; Receiving a feedback image from the media display device based on the sent first control signal, wherein the received feedback image indicates the control of the media display device based on the sent first control signal; And Controlling the first display screen to display the received feedback image together with the plurality of displayed UI elements, The operations further include: Displaying a subset of the plurality of UI elements on the first display screen, the subset being based on the received feedback image and the type of media content displayed on the second display screen of the media display device; Determine that the position of the user's eye gaze is on a second UI element in a subset of the displayed UI elements; Select the second UI element from the subset based on determining that the eye gaze position has been on the second UI element for a period of time; and Allow the user to effectively control the rendering of media content on a second display screen by using the second UI element selected from the subset.

Images