Eye-gaze-based control of media display devices

The electronic device facilitates gaze-based control of media display devices, addressing the challenge of user interaction for individuals with disabilities by allowing independent operation through gaze inputs and simultaneous feedback display, ensuring a seamless viewing experience.

JP7595847B2Active Publication Date: 2024-12-09SONY GROUP CORP
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
JP2023547247
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-02-15
Filing Date
2022-02-12
Publication Date
2024-12-09
Estimated Expiration
2042-02-12

AI Technical Summary

Technical Problem

Users with physical disabilities, such as neurodevelopmental disorders, face challenges in controlling media display devices like televisions due to difficulties with traditional remote controls, necessitating assistance for device operation.

Method used

An electronic device equipped with an image capture system and display screen that allows users to control media display devices via gaze-based inputs, enabling selection of UI elements and sending control signals to the media display device using short-range radio signals, allowing simultaneous display of feedback images and UI elements.

Benefits of technology

Enables users with disabilities to control media display devices independently by gaze, providing a seamless viewing experience without the need for physical interaction with multiple devices or external networks, enhancing user independence and convenience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007595847000001
    Figure 0007595847000001
  • Figure 0007595847000002
    Figure 0007595847000002
  • Figure 0007595847000003
    Figure 0007595847000003
Patent Text Reader

Abstract

An electronic device is provided that captures a first image of a user. The electronic device determines, based on the first image, a position of the user's gaze on a first UI element of a plurality of UI elements displayed on a first display screen of the electronic device. The electronic device selects the first UI element based on determining the position of the gaze on the first UI element over a time interval. The electronic device sends a first control signal to a media display device based on the selection of the first UI element. The electronic device receives a feedback image from the media display device. The feedback image indicates control of the media display device based on the first control signal. The electronic device controls the first display screen to display the feedback image with the plurality of UI elements.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS REFERENCE TO RELATED APPLICATIONS / INCORPORATION BY REFERENCE none

[0002] Various embodiments of the present disclosure relate to controlling a media display device based on a user's gaze. In particular, various embodiments of the present disclosure relate to an apparatus and method for controlling a media display device based on a user's gaze on an electronic device coupled to the media display device. [Background technology]

[0003] Recent advances in the field of media content playback have led to the development of various techniques for controlling a media display device (e.g., a television). Typically, a user may utilize a remote control device to control the functions of the media display device, such as changing channels on the media display device and controlling the volume of the media display device. In some situations, the media display device may be accessed by a user with a physical disability, such as a neurodevelopmental disability, e.g., a movement disorder. The user may have difficulty controlling the functions of the media display device, e.g., via a remote control. Thus, the user may require assistance to control the media display device. Summary of the Invention [Problem to be solved by the invention]

[0004] Further limitations and disadvantages of the conventional approach will become apparent to those skilled in the art upon comparison of the described system with certain aspects of the present disclosure illustrated in the remaining portions of this application and with reference to the drawings. [Means for solving the problem]

[0005] Provided is an electronic device for controlling a media display device based on line of sight, and a method for controlling a media display device via such an electronic device, substantially as illustrated and / or described in connection with at least one of the drawings and more fully set forth in the claims.

[0006] These and other features and advantages of the present disclosure will become apparent from a consideration of the following detailed description of the disclosure when taken in conjunction with the accompanying drawings, in which like reference characters refer to like elements throughout. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram illustrating an exemplary network environment for control of a media display device based on gaze as user input, according to an embodiment of the present disclosure. [Diagram 2] 2 is a block diagram illustrating the example electronic device of FIG. 1 in accordance with an embodiment of the present disclosure. [Figure 3A] 3A and 3B together illustrate an example scenario for control of the media presentation device of FIG. 1 according to an embodiment of the present disclosure. [Figure 3B] 3A and 3B illustrate an example scenario for control of the media presentation device of FIG. 1 according to an embodiment of the present disclosure. [Figure 4A] 4B, together with FIG. 4C, illustrate another example scenario for control of the media presentation device of FIG. 1 according to an embodiment of the present disclosure. [Figure 4B] 4A and 4B illustrate another example scenario for control of the media presentation device of FIG. 1 according to an embodiment of the present disclosure. [Figure 5A] 5A and 5B illustrate another example scenario for control of the media presentation device of FIG. 1 according to an embodiment of the present disclosure. [Figure 5B] 5A and 5B illustrate another example scenario for control of the media presentation device of FIG. 1 according to an embodiment of the present disclosure. [Figure 6]1 is a flowchart illustrating an exemplary method for control of a media display device based on gaze as user input, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The electronic device and method of the present disclosure for controlling a media display device based on a user's gaze as a user input may find implementation as described below. An exemplary aspect of the present disclosure provides an electronic device (e.g., an electronic tablet or a smartphone) accessible to a user (e.g., a user with or without a physical disability). The electronic device may include a first display screen that displays a plurality of user interface (UI) elements (e.g., control icons such as, but not limited to, a power control icon and a volume control icon, a navigation icon, or a numeric icon). The electronic device may control a media display device (e.g., a television) communicatively coupled to the electronic device based on a selection of each of the plurality of UI elements by the user. The electronic device may be configured to control an image capture device (e.g., a built-in front camera of an electronic tablet) to capture a first image of a user of the electronic device. The first image of the user may include an image of at least a face of the first user. The electronic device may be configured to determine a position of the user's gaze on a first UI element of the plurality of displayed UI elements based on the captured first image of the user. For example, the first UI element can be a control icon, such as a "volume increase icon," that upon selection can increase the volume of media content currently being rendered on a media display device communicatively coupled to the electronic device. The electronic device can be further configured to select the first UI element of the plurality of UI elements based on determining a gaze position on the first UI element over a time interval. For example, the time interval can be several seconds (such as 3 seconds or more). In one example, the selection of the first UI element of the plurality of UI elements can be based on a gaze at the first UI element over a 3 second time interval.

[0009] The electronic device may be further configured to transmit a first control signal to the media display device based on a selection of a first UI element of the plurality of UI elements. The first control signal may correspond to activating a function (such as a volume increase) that may be associated with the media display device upon selection of the first UI element of the plurality of UI elements. In one example, the first control signal may correspond to a short-range radio signal indicating an increase in the volume of media content being rendered on the media display device upon selection of the first UI element, such as a "volume increase icon." The electronic device may be further configured to receive a feedback image from the media display device based on the transmitted first control signal. The received feedback image may indicate control of the media display device based on the transmitted first control signal. According to an embodiment, the received feedback image may correspond to the currently rendered media content and a UI icon indicating an increase in the volume of the rendering media content being displayed on the media display device. The electronic device may be further configured to control the first display screen to display the received feedback image together with the displayed plurality of UI elements.

[0010] In an exemplary implementation, the electronic device can be positioned in a position where a user can gaze at a plurality of UI elements of the electronic device to control the media display device. The electronic device can enable the user to select different UI elements of the plurality of UI elements on the electronic device to control different functions related to the media display device. The electronic device of the present disclosure can provide a user (e.g., a user with or without a physical disability) with the ability to control the media display device based on gaze as user input. Thus, the electronic device enables a user to control the media display device based on gaze, thus enabling the user to control the media display device without assistance from another user.

[0011] Additionally, the electronic device can simultaneously display multiple UI elements and the received feedback image, which can include a portion of the media content. In some embodiments, the feedback image can also include a UI icon corresponding to the selected first UI element. By simultaneously displaying the feedback image and multiple UI elements on the electronic device, the user can focus their gaze on the selected first UI element while viewing the media content and / or the UI icon, thereby eliminating the need to switch gaze positions between the electronic device and the media display device while controlling the media display device via the electronic device. Thus, the disclosed electronic device enables a user (e.g., with a physical disability) to control the media display device while continuing to view the media content on the electronic device by simultaneously displaying a feedback image (corresponding to a portion of the currently rendered media content) and multiple UI elements on the electronic device.

[0012] Additionally, the disclosed electronic device can also control the media display device using short-range radio signals or a wireless local network, allowing a user to control the media display device without having to connect the electronic device to an external network (e.g., the Internet).

[0013] FIG. 1 is a block diagram illustrating an exemplary network environment for control of a media display device based on gaze as a user input, according to an embodiment of the present disclosure. FIG. 1 illustrates a network environment 100. The network environment 100 may include an electronic device 102. The electronic device 102 may include a first display screen 104 and an image capture device 108. The first display screen 104 may display a plurality of UI elements 106 and a feedback image 110. The plurality of UI elements 106 may include a first UI element 106A, a second UI element 106B, and an Nth UI element 106N. The network environment 100 may further include a media display device 112. The media display device 112 may include a second display screen 114. The second display screen 114 may render media content 116. The network environment 100 may further include a user 118, a server 120, and a communication network 122. The electronic device 102, the media presentation device 112, and the server 120 may be communicatively coupled to each other via a communication network 122. In some scenarios, the user 118 may be a user with a physical disability, such as a neurodevelopmental disability (e.g., movement disorder), who may require a wheelchair for physical mobility. In such a scenario, the electronic device 102 may be attached to a wheelchair handrail and / or a piece of furniture within the field of view of the user 118 in order to control the media presentation device 112 via the electronic device 102.

[0014] The electronic device 102 may include suitable logic, circuitry, and interfaces that may be configured to control the media presentation device 112. The electronic device 102 may be further configured to control the media presentation device 112 by utilizing the gaze of the user 118 as user input. Examples of the electronic device 102 may include, but are not limited to, electronic tablets, smart phones, computing devices, cellular phones, mobile phones, gaming devices, mainframe machines, servers, computer workstations, and / or consumer electronics (CE) devices. It should be noted that the shape and configuration of the electronic device 102 as shown in FIG. 1 is presented as an example only. The present disclosure may be applied to other shapes and configurations of the electronic device 102 without departing from its scope.

[0015] The first display screen 104 may include suitable logic, circuits, and interfaces that may be configured to display a plurality of UI elements 106 and a feedback image 110. The first display screen 104 of the electronic device 102 may be configured to receive user input in the form of a gaze of the user 118. Furthermore, the first display screen 104 may be a touch screen that allows the user 118 to provide user input in the form of touch input via the first display screen 104. The touch screen may be at least one of a resistive touch screen, a capacitive touch screen, or a thermal touch screen. The first display screen 104 may be realized through a number of known technologies, such as, but not limited to, at least one of a liquid crystal display (LCD) display, a light emitting diode (LED) display, a plasma display, or an organic LED (OLED) display technology, or other display devices. According to an embodiment, the first display screen 104 may refer to a display screen of a head mounted device (HMD), a smart glasses device, a see-through display, a projection display, an electrochromic display, or a transparent display.

[0016] The UI elements 106 may correspond to control icons (e.g., user selectable options) for controlling the media presentation device 112. For example, each of the UI elements 106 may represent a user selectable icon, such as a control icon of a remote control device that may be used to control the media presentation device 112 (e.g., a television). The UI elements 106 may be configured to allow the user 118 to control different functions associated with the media presentation device 112 through selection of the UI elements 106 based on the user's gaze. Examples of the UI elements 106 may include, but are not limited to, control icons corresponding to power settings of the media presentation device 112, home screen settings of the media presentation device 112, audio settings (e.g., volume control settings) of media content being rendered on the media presentation device 112, video display settings (e.g., display characteristic adjustments, video mode selections) of media content being rendered on the media presentation device 112, network settings of the media presentation device 112, security settings of the media presentation device 112, or accessibility settings of the media presentation device 112. According to an embodiment, home screen settings of the media display device 112 may correspond to preferences related to the placement of menus or icons on the home screen of the media display device 112. Additionally, security settings may include, but are not limited to, user privacy settings, parental control settings, and user account settings of the media display device 112. Additionally, accessibility settings may include, but are not limited to, audio description settings, audio-based scene description settings, closed caption settings, video description settings, and color inversion settings of the media display device 112.

[0017] Image capture device 108 may include suitable logic, circuitry, interfaces, and / or code that may be configured to capture one or more images of a user, such as user 118. According to an embodiment, image capture device 108 may capture one or more images that may include at least the face of user 118 to determine the position of the user's gaze on a first UI element of the plurality of UI elements 106. Examples of image capture device 108 may include, but are not limited to, an image sensor, a wide-angle camera, a 360-degree camera, an action camera, a closed circuit television (CCTV) camera, a camcorder, a digital camera, a camera phone, a time-of-flight camera (ToF camera), a night vision camera, and / or other image capture devices. Although FIG. 1 illustrates image capture device 108 integrated with electronic device 102, the disclosure is not so limited. In some embodiments, image capture device 108 may be separate from electronic device 102 without departing from the scope of the disclosure.

[0018] The media display device 112 may include suitable logic, circuitry, interfaces, and / or code that may be configured to render media content according to viewer preferences (e.g., preferences of the user 118). In one example, the media display device 112 may be configured to communicate with the server 120 to receive media content. In another example, the media display device 112 may be configured to receive broadcast or streaming content as media content. According to an embodiment, the media display device 112 may be further configured to store the media content received from the server 120. The media display device 112 may be further configured to receive, via the electronic device 102, a first control signal for controlling the media display device 112 based on a plurality of UI elements 106 selected based on the gaze of the user 118 as user input. Examples of the media display device 112 may include, but are not limited to, a television, a laptop, a personal computer, a mobile phone, a smart phone, a tablet computing device, a gaming console, a media player, a sound system with display capabilities, a home theater system, a digital camera, an automotive electronic device, an electronic musical instrument, or other consumer electronic device with media rendering capabilities. It should be noted that the shape and arrangement of the media display device 112 as shown in Figure 1 is provided by way of example only, and that the present disclosure is applicable to other shapes and arrangements of media display devices 112 without departing from the scope thereof.

[0019] The second display screen 114 may include suitable logic, circuitry, and interfaces that may be configured to display media content. The second display screen 114 may be realized through a number of known technologies, such as, but not limited to, at least one of LCD display, LED display, plasma display, or OLED display technologies, or other display devices. According to an embodiment, the second display screen 114 may refer to a display screen of a smart glasses device, a see-through display, a projection display, an electrochromic display, or a transparent display. According to an embodiment, the second display screen 114 may be a touch screen that allows a user 118 to provide a user input in the form of touch via the second display screen 114. The touch screen may be at least one of a resistive touch screen, a capacitive touch screen, or a thermal touch screen.

[0020] The rendered media content 116 may correspond to content currently being rendered on the second display screen 114 of the media presentation device 112. Examples of media content 116 may include, but are not limited to, a home screen of the media presentation device 112, a home screen of an application being rendered on the media presentation device 112, image content being rendered on the media presentation device 112, video content (e.g., video recording, broadcast content, streaming content), a gaming application interface, or a device settings interface of the media presentation device 112.

[0021] The feedback image 110 of the electronic device 102 may correspond to a replicated image of a portion of the media content 116 currently being rendered on the media display device 112. In an embodiment, the feedback image 110 may correspond to an image of a UI icon displayed on the second display screen 114 that may indicate a selection of one of the plurality of UI elements 106. In one embodiment, the feedback image 110 may be a set of images (e.g., a video stream) of the media content 116 received from the media display device 112. In an exemplary scenario, the feedback image 110 may correspond to a movie currently being rendered on the second display screen 114 of the media display device 112 and a UI icon that may indicate an increase in the volume of the rendering media content (e.g., movie) displayed on the media display device 112.

[0022] The server 120 may include suitable logic, circuitry, interfaces and / or code that may be configured to communicate with the media presentation device 112 via the communications network 122. The server 120 may be configured to store media content to be delivered to the media presentation device 112. The server 120 may receive content requests from the media presentation device 112 and provide the stored media content to the media presentation device 112 based on the received content requests. Examples of the server 120 may include, but are not limited to, an application server, a media content server, a cloud server, a mainframe server, a database server, a file server, a web server, or other types of servers.

[0023] The communication network 122 may include wireless communication channels that enable the media presentation device 112, the electronic device 102, and the server 120 to communicate with each other wirelessly. Examples of the communication network 122 may include, but are not limited to, short-range communication technologies such as a Wireless Fidelity (Wi-Fi) network, a Near Field Communication (NFC) channel, or a Bluetooth® (BT) communication network. Various entities, such as the electronic device 102 in the network environment 100, may be configured to connect to the communication network 122 according to various wireless communication protocols. Examples of such wireless communication protocols may include, but are not limited to, at least one of a wireless device-to-device communication protocol or a Bluetooth® communication protocol. In some embodiments, the communication network 122 may include a communication medium that enables the media presentation device 112 and the server 120 to communicate with each other. Examples of wired and wireless communication protocols of the communications network 122 that enable the media display device 112 and the server 120 to communicate include, but are not limited to, Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communications, and wireless access points (APs).

[0024] In operation, the electronic device 102 can be configured to display a number of user interface (UI) elements that control different functions of the media presentation device 112. The electronic device 102 can be configured to control the image capture device 108 to capture a first image of a user 118 of the electronic device 102. The first image can include at least an image of the face of the user 118 to track the gaze of the user 118. In some embodiments, the electronic device 102 can be mounted on a wheelchair of a user (such as a physically disabled user 118) and control the image capture device 108 to capture a first image of the user 118. The electronic device 102 can be further configured to determine a position of the gaze of the user 118 on a first UI element 106A of the displayed number of UI elements 106 based on the captured first image of the user 118. Details of determining a position of the gaze of the user 118 on a first UI element 106A of the displayed number of UI elements 106 are further described, for example, in FIG. 3A. According to an embodiment, each UI element of the plurality of UI elements 106 can be associated with a different function related to the media presentation device 112. The electronic device 102 can be further configured to select the first UI element 106A of the plurality of UI elements 106 based on determining a gaze position on the first UI element 106A over a certain time interval. The time interval can be preset, for example, within a range of a few seconds (such as 3-5 seconds), or can be set based on a user input. In an exemplary embodiment, the selection of the first UI element 106A can be based on, for example, a blink of the user's 118 eyes or other eye gestures, such as a blink of the eye after gaze. Thus, the electronic device 102 can determine a blink of the user's 118 eyes at the determined gaze position on the first UI element 106A as a user input for selecting the first UI element 106A. In one example, the first UI element 106A may correspond to a “move up” navigation icon that can be used to navigate between different icons (e.g., a settings icon, an application icon, a game icon) on a home screen of the media presentation device 112.

[0025] The electronic device 102 may be further configured to send a first control signal to the media presentation device 112 based on a selection of the first UI element 106A of the plurality of UI elements 106. For example, the sent first control signal may enable navigation between UI icons. Details of the navigation between UI icons based on the selection of the first UI element 106A are shown, for example, in FIG. 3B. The electronic device 102 may be further configured to receive a feedback image 110 from the media presentation device 112 based on the sent first control signal. In an exemplary embodiment, the feedback image 110 may be an image indicating a selection of a UI icon of a home screen of the media presentation device 112. The electronic device 102 may be further configured to control the first display screen 104 to display the received feedback image 110 together with the displayed plurality of UI elements 106. Thus, the electronic device 102 may enable the user 118 to control the media presentation device 112 via a gaze as a user input provided to the electronic device 102. Controlling the media display device 112 via the user's 118 line of sight may enable the user 118 to see and control different functions of the media display device 112 without assistance from other users. The electronic device 102 may be further configured to control the first display screen 104 to display at least a portion of the media content 116 being played on the media display device 112 as part of the received feedback image 110. Thus, the electronic device 102 may enable the user 118 to control the media display device 112 while continuing to watch the media content 116. As a result, the user 118 does not miss a frame or a portion of the media content 116 being played on the media display device 112 (such as a television) while providing user input to control the media display device 112. Thus, a seamless viewing experience may be achieved for the user 118.

[0026] According to an embodiment, the electronic device 102 can be configured to control a movement of a cursor on the plurality of UI elements 106 of the electronic device 102 in response to a change in a position of the gaze of the user 118 on the plurality of UI elements 106. For example, the electronic device 102 can move the cursor to the first UI element 106A upon determining a position of the gaze of the user 118 on the first UI element 106A, and further move the cursor to the second UI element 106B upon determining a position of the gaze of the user 118 on the second UI element 106B. In some embodiments, the electronic device 102 can be configured to highlight the first UI element 106A based on a selection of the first UI element 106A of the plurality of UI elements 106 by the user 118. Details of the movement of the cursor and the highlighting of the first UI element 106A are further described, for example, in FIG. 3B.

[0027] According to one embodiment, the feedback image 110 received from the media display device 112 may correspond to a UI icon indicating control of the media display device 112 based on the transmitted first control signal. Examples of UI icons may include, but are not limited to, a "volume up" icon, a "volume down" icon, a "brightness control" icon, or a "mode selection" icon. Details of receiving UI icons as feedback images 110 are further described in, for example, Figures 3A, 3B, 4A, 4B, 5A, and 5B.

[0028] According to an embodiment, the electronic device 102 may be further configured to control the first display screen 104 to display the plurality of UI elements 106 on a first portion of the first display screen 104 and to display the received feedback image 110 on a second portion of the first display screen 104. Details of displaying the plurality of UI elements 106 on a first portion of the first display screen 104 and displaying the received feedback image 110 on a second portion are further described in, for example, Figures 3A, 3B, 4A, 4B, 5A and 5B.

[0029] According to one embodiment, the feedback image 110 received from the media presentation device 112 may correspond to the media content 116 displayed on the second display screen 114 of the media presentation device 112 based on the transmitted first control signal. Details of receiving the media content 116 (or a portion of the media content 116) as the feedback image 110 are further described in, e.g., Figures 3A, 3B, 4A, 4B, 5A and 5B.

[0030] According to an embodiment, the electronic device 102 may be further configured to control an overlay of the UI elements 106 displayed on the displayed feedback image 110, which may correspond to the media content 116 being displayed on the second display screen 114 of the media presentation device 112. Additionally, the electronic device 102 may be configured to control an opacity of the UI elements 106 overlaid on the displayed feedback image 110 such that the user 118 can easily view the media content 116 being rendered on the electronic device 102 while controlling the media presentation device 112. Details of the UI elements 106 overlaid on the displayed feedback image 110 and controlling the opacity of the displayed UI elements 106 are further described, for example, in FIG.

[0031] According to an embodiment, the electronic device 102 may be further configured to control the first display screen 104 to display information related to the media content 116 being displayed on the second display screen 114 of the media presentation device 112 based on a selection of one or more UI elements of the plurality of UI elements 106. For example, the display information may correspond to a short description of the media content 116 being rendered on the media presentation device 112. Details of controlling the first display screen 104 to display information related to the media content 116 are further described, e.g., in FIG. 4B .

[0032] According to an embodiment, the electronic device 102 may be further configured to render guidance information for the user 118 for controlling one or more configuration settings of the media presentation device 112 through one or more UI elements of the plurality of UI elements 106 displayed on the first display screen 104. In an example embodiment, the guidance information for the user 118 may be, for example, information regarding an action associated with one or more UI elements of the plurality of UI elements 106. Details of the guidance information for the user 118 are further described, for example, in FIG. 5A.

[0033] According to an embodiment, the electronic device 102 may be further configured to control the first display screen 104 to display a sequence of UI elements of the plurality of UI elements 106 on the first display screen 104 based on the received feedback image 110. In one example, the sequence of UI elements of the plurality of UI elements 106 may be associated with a control setting of the media content 116 of the media presentation device 112. Details of displaying the sequence of UI elements are further described, for example, in FIG. 5B .

[0034] FIG. 2 is a block diagram illustrating the example electronic device of FIG. 1, in accordance with an embodiment of the present disclosure. FIG. 2 illustrates a block diagram 200 of an electronic device 102. The electronic device 102 may include a circuit 202, a memory 204, and a network interface 206. The electronic device 102 may further include a first display screen 104. The first display screen 104 may display a number of UI elements 106. The electronic device 102 may further include an image capture device 108. A detailed description of the first display screen 104, the image capture device 108, and the communication network 122 has been provided with reference to FIG. 1 and is therefore omitted here for brevity.

[0035] The circuitry 202 may include suitable logic, circuits, and / or interfaces that may be configured to execute program instructions associated with different operations performed by the electronic device 102. For example, some of these operations may include determining a position of the gaze of the user 118 on the first UI element 106A, selecting the first UI element 106A based on the determination of the gaze position of the user 118, sending a first control signal to the media presentation device 112, receiving a feedback image from the media presentation device 112, and controlling the first display screen 104 to display the feedback image 110 with the multiple UI elements 106. The circuitry 202 may include one or more specialized processing units that may be implemented as independent processors. In some embodiments, the one or more specialized processing units may be implemented as an integrated processor or processors that collectively perform the functions of the one or more specialized processing units. The circuitry 202 may be implemented based on multiple processor technologies known in the art. An example implementation of the circuit 202 may be an X86-based processor, a graphics processing unit (GPU), a reduced instruction set computing (RISC) processor, an application specific integrated circuit (ASIC) processor, a complex instruction set computing (CISC) processor, a microcontroller, a central processing unit (CPU), and / or other control circuitry.

[0036] The memory 204 may include suitable logic, circuitry, interfaces, and / or code that may be configured to store one or more instructions executed by the circuitry 202. The memory 204 may be configured to store information corresponding to a function associated with each UI element of the plurality of UI elements 106. In some embodiments, the memory 204 may be configured to store user input associated with the user 118. Example implementations of the memory 204 may include, but are not limited to, a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a hard disk drive (HDD), a solid state drive (SSD), a CPU cache, and / or a secure digital (SD) card.

[0037] The network interface 206 may include suitable logic, circuitry, and / or interfaces that may be configured to facilitate communications between the media presentation device 112 and the electronic device 102 over the communications network 122. The network interface 206 may be implemented using various known technologies that support wired or wireless communications between the electronic device 102 and the communications network 122. The network interface 206 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, a subscriber identity module (SIM) card, or a local buffer circuit.

[0038] Those skilled in the art will appreciate that electronic device 102 of Figure 2 may include other suitable components or systems in addition to those illustrated herein for purposes of illustrating and explaining the functionality and operation of the present disclosure. Detailed descriptions of other components or systems of electronic device 102 are omitted from this disclosure for the sake of brevity. Operation of circuit 202 is further described, for example, in Figures 3A, 3B, 4A, 4B, 5A, and 5B.

[0039] 3A and 3B collectively illustrate exemplary scenarios for control of the media display device 112 of FIG. 1, in accordance with an embodiment of the present disclosure. The descriptions of FIGS. 3A and 3B are provided in conjunction with the elements of FIGS. 1 and 2. FIG. 3A illustrates an exemplary scenario 300A including the electronic device 102 (e.g., an electronic tablet or smartphone), the first display screen 104, a number of UI elements 106, an image capture device 108 (e.g., a built-in front camera), the media display device 112 (e.g., a television), the second display screen 114, and a user 118 of FIG. 1. Additionally, the exemplary scenario 300A also illustrates a first UI element 306 (e.g., a "volume up" UI element) of the number of UI elements 106 (e.g., a "volume control" UI element, a "power" UI element, a "source" UI element, a "playback control" UI element, a "navigation" UI element, a "home" UI element, a "numeric" UI element, an "information" UI element, a "back" UI element, and an "end" UI element). The exemplary scenario 300A further illustrates a received feedback image 304, a cursor 308, a first portion 310 of the first display screen 104, and a second portion 312 of the first display screen 104. Also illustrated is media content 302 and a "volume up" UI icon 302a on the second display screen 114 of the media presentation device 112.

[0040] With reference to FIG. 3A , the media presentation device 112 may be configured to render media content 302, such as a movie, on the second display screen 114. In an exemplary embodiment, the user 118 may need to turn up the volume of the media content 302 being rendered on the media presentation device 112. In such a case, the user 118 may gaze towards the UI elements 106 of the electronic device 102 and select a first UI element 306 (such as a “volume up” UI element) to turn up the volume of the media content 302 being rendered on the media presentation device 112. The circuitry 202 of the electronic device 102 may be configured to control the image capture device 108 to capture a first image of the user 118. The image capture device 108 may capture the first image of the user 118 based on a field of view (FOV) of the image capture device 108. The FOV of the image capture device 108 may be predetermined to accurately capture the first image of the user 118 (or an image of the face of the user 118). The electronic device 102 may be positioned such that the image capture device 108 can clearly capture a first image of the face of the user 118. For example, the electronic device 102 may be mounted near the user 118 or may be in the hand of the user 118. In another example, the electronic device 102 may be mounted to the wheelchair of the user 118 (e.g., on the handrail or armrest of the wheelchair). It is noted that the mounting location or position of the electronic device 102 where the image capture device 108 is positioned as shown in FIG. 3A is provided by way of example only. The image capture device 108 may be positioned in other positions, locations, or surfaces of the electronic device 102 such that the FOV of the image capture device 108 can clearly capture a first image of the user 118 without departing from the scope of this disclosure. Other positions, locations, or surfaces of the electronic device 102 where the image capture device 108 may be positioned are omitted from this disclosure for the sake of brevity.

[0041] The circuit 202 of the electronic device 102 may be further configured to extract image portions from a captured first image of the face of the user 118. The image portions may be indicative of the eyes of the face of the user 118. The circuit 202 may extract image portions (e.g., may include sub-images or pixels of the eyes) from the captured first image based on different image processing techniques known in the art. The circuit 202 may be further configured to track eye movements of the user 118 based on the extracted image portions. In some embodiments, the circuit 202 may control the image capture device 108 to capture multiple images of the face of the user 118 and further track eye movements of the user 118 based on an analysis of the image portions in each of the multiple captured images over a period of time. The eye movements may indicate where the user 118 is believed to be gazing (e.g., toward the electronic device 102 or elsewhere).

[0042] The circuitry 202 of the electronic device 102 may be further configured to determine a gaze position of the user 118 on a first UI element 306 (such as a “volume up” UI element) of the plurality of UI elements 106 of the electronic device 102 based on the tracked eye movements of the user 118. In some embodiments, the circuitry 202 may determine a gaze position on any UI element of the plurality of UI elements 106 of the electronic device 102 based on the captured first image.

[0043] According to an embodiment, the circuit 202 can further control the movement of the cursor 308 on the plurality of UI elements 106 in response to a change in the position of the user's 118's gaze on the plurality of UI elements 106. The circuit can control the movement of the cursor 308 to a UI element that the user 118 is currently considered to be looking at when the user's gaze moves to different positions on the first portion 310 of the first display screen 104 corresponding to different UI elements of the plurality of UI elements 106. For example, the circuit 202 can place the cursor 308 on a "move up" navigation UI element when the circuit 202 determines that the user's gaze is on a "move up" navigation UI element. Furthermore, the circuit 202 can move the cursor 308 to the "volume up" UI element 306 when the circuit 202 determines that the user's gaze is on the first UI element 306 (such as a "volume up" UI element). The movement of the cursor 308 on the plurality of UI elements 106 can help the user 118 visually confirm the position of the user's gaze on the electronic device 102.

[0044] The circuit 202 of the electronic device 102 may further select the first UI element 306 (such as a “volume up” UI element) upon determining a gaze of the user 118 on the first UI element 306 (such as a “volume up” UI element) for a certain time interval (such as a few seconds). In an exemplary implementation, the circuit 202 may also select the first UI element 306 (such as a “volume up” UI element) of the plurality of UI elements 106 based on other eye gestures. For example, the circuit 202 may be configured to select the “volume up” UI element 306 based on detecting an eye “blink” by the user 118 while the gaze is determined to be on the first UI element 306 (such as a “volume up” UI element). In some embodiments, the circuit 202 can be configured to determine at least one of a facial expression of the user 118 (e.g., a smile) or a head-related gesture of the user 118 (e.g., tilting the head of the user 118 in a certain direction) as a user input for selecting a first UI element 306 (such as a “volume up” UI element) of the multiple UI elements 106 displayed on the first display screen 104.

[0045] According to an embodiment, the circuit 202 may further highlight the first UI element 306 (e.g., the “volume up” UI element) as shown in FIG. 3A as confirmation of the selection of the first UI element 306 (i.e., the “volume up” UI element) from the plurality of UI elements 106. The highlighting of the first UI element 306 (i.e., the “volume up” UI element) may help the user 118 identify the selected first UI element 306. In one or more embodiments, the circuit 202 may be configured to control (e.g., increase) a size of the selected first UI element 306 (i.e., the “volume up” UI element) to make the selected UI icon more visible to the user 118. In an example, the circuit 202 may be further configured to display the selected first UI element 306 as a “pop-up” UI element on the first display screen 104.

[0046] The circuit 202 may be further configured to transmit a first control signal to the media presentation device 112 based on a selection of the first UI element 306 (i.e., a "volume up" UI element) by the user 118. The circuit 202 may be further configured to transmit the first control signal using a short-range radio signal (e.g., Bluetooth) or a wireless local network (e.g., Wi-Fi Direct). The first control signal may cause the media presentation device 112 to increase the volume of the media content 302 being rendered on the second display screen 114 of the media presentation device 112 when the media presentation device 112 receives the first control signal (i.e., the control signal associated with the "volume up" UI element) from the electronic device 102. Thus, the circuit 202 may enable the user 118 to control the volume of the media content 302 being rendered on the media presentation device 112 via the electronic device 102 by utilizing his / her line of sight.

[0047] As shown in Figure 3A, the media display device 112 may display a "volume up" UI icon 302a on the second display screen 114 based on user input, which may indicate an increase in the volume of the media content 302 being rendered on the media display device 112. It should be noted that the location of the "volume up" UI icon 302a as shown in Figure 3A is provided by way of example only. The "volume up" UI icon 302a may be located on any portion of the second display screen 114 without departing from the scope of the present disclosure.

[0048] The circuitry 202 of the electronic device 102 can be further configured to receive a feedback image 304 from the media display device 112 based on the transmitted first control signal. The feedback image 304 can correspond to an image of a “volume up” UI icon 302a displayed on the second display screen 114. For example, when the user 118 is gazing at the electronic device 102 to control the volume of the media content 302, the user 118 may need to switch his / her gaze between the electronic device 102 and the media display device 112 to confirm the change in volume indicated by the “volume up” UI icon 302a on the media display device 112. Thus, the circuitry 202 of the electronic device 102 can display the “volume up” UI icon 302a on the electronic device 102 as the feedback image 304 such that the user 118 can confirm the increase in volume by the feedback image 304 without having to gaze back and forth between the electronic device 102 and the media display device 112 to confirm the increase in volume.

[0049] The circuitry 202 may be further configured to control the first display screen 104 to display the received feedback image 304 along with the displayed UI elements 106. According to an embodiment, the circuitry 202 may be further configured to control the first display screen 104 to display the UI elements 106 on a first portion 310 of the first display screen 104 and to display the received feedback image 304 on a second portion 312 of the first display screen 104. By displaying the UI elements 106 and the feedback image 304 on different portions, such as the first portion 310 and the second portion 312, respectively, the user 118 may be able to view an image of a UI icon, such as an image of a "volume up" UI icon 302a, while controlling the volume of the media content 302 rendered on the media presentation device 112. It should be noted that the location of the feedback image 304 on the second portion 312 of the first display screen 104 as shown in FIG. 3A is provided by way of example only. The feedback image 304 may be present at a different location on the second portion 312 of the first display screen 104 without departing from the scope of the present disclosure. Additionally, the size and location of the first portion 310 and the second portion 312 of the first display screen 104 as shown in FIG. 3A are also presented as an example only. The size and location of the first portion 310 and the second portion 312 of the first display screen 104 may be presented differently without departing from the scope of the present disclosure. For example, the first portion 310 may be present below the second portion 312 on the first display screen 104. Thus, the disclosed electronic device 102 may enable the user 118 to effectively control the media display device 112 based on eye gaze as user input without assistance from another user. The disclosed electronic device 102 may also enable the user 118 to effectively control the media display device 112 based on eye gaze as user input without the need to connect the electronic device 102 to an external network (e.g., the Internet).

[0050] 3B illustrates an example scenario 300B showing media content 314, a feedback image 316, a "move left" navigation UI element 318, a "Program Guide" UI icon 320, and a corresponding image 322 of the "Program Guide" UI icon 320.

[0051] The media presentation device 112 may display a home screen of the media presentation device 112 as media content 314 on the second display screen 114 based on a user input of the user 118. The user 118 may need to navigate a series of UI icons present on the home screen of the media presentation device 112. The series of UI icons may include, for example, an "Applications" UI icon, a "Games" UI icon, a "Playlist" UI icon, a "Settings" UI icon, a "Search" bar, and an "Information" icon. Thus, the circuit 202 of the electronic device 102 may determine a position of the user's 118 gaze on a navigation UI element of the plurality of UI elements 106. The circuit 202 may determine a position of the user's 118 gaze on a first UI element (i.e., the "Move Left" navigation UI element 318) of the plurality of UI elements 106. The circuit 202 may select the "Move Left" navigation UI element 318 based on the determination of the position of the user's 118 gaze on the "Move Left" navigation UI element 318.

[0052] Additionally, the circuit 202 may send a first control signal to the media presentation device 112 to navigate to a UI icon (on the second display screen 114) that may be to the left of a currently selected UI icon in the series of UI icons. For example, the first control signal may enable selection of the “Program Guide” UI icon 320 by navigation from an “Applications” UI icon in the series of UI icons on the right side of the home screen to the “Program Guide” UI icon 320 on the left side of the home screen.

[0053] The circuitry 202 may be further configured to receive a feedback image 316 based on the transmitted first control signal. The received feedback image 316 may correspond to an image associated with the media content 314 of the media display device 112 (e.g., an image of a home screen shown in FIG. 3B). The feedback image 316 may indicate a selection of the "Program Guide" UI icon 320 by highlighting a corresponding image 322 of the "Program Guide" UI icon 320. Thus, the electronic device 102 may enable the user 118 to efficiently navigate a series of UI icons (e.g., a home screen) of the media content 314 without having to move his / her gaze back and forth between the electronic device 102 and the media display device 112. Thus, the electronic device 102 may enable the user 118 to control the media display device 112 while continuing to view the media content 314. As a result, the user 118 does not miss a frame or portion of the media content 314 being played on the media display device 112 (e.g., a television) while providing user input to control the media display device 112. Thus, a seamless viewing experience for the user 118 can be achieved.

[0054] Figures 4A and 4B collectively illustrate other exemplary scenarios for control of the media presentation device 112 of Figure 1, in accordance with an embodiment of the present disclosure. Figures 4A and 4B are described with reference to elements of Figures 1 and 2. Figure 4A illustrates an exemplary scenario 400A. Exemplary scenario 400A illustrates media content 402, a feedback image 404, and a first UI element 406 (e.g., a "volume up" UI element). It should be noted that the functionality of first UI element 406 (e.g., a "volume up" UI element) can be the same as the functionality of the "volume up" UI element (shown as first UI element 306) described, for example, in Figure 3A.

[0055] The circuit 202 can determine a position of the user's 118 gaze on a first UI element 406 (e.g., a “volume up” UI element) of the plurality of UI elements 106. Additionally, the circuit 202 can transmit a first control signal to control a volume of the media content 402 based on a determination of the position of the user's 118 gaze on the first UI element 406 (e.g., a “volume up” UI element). Additionally, the circuit 202 can receive a feedback image 404 from the media presentation device 112, such as may include an image of a “volume up” UI icon 402a overlaid on the media content 402 as shown in FIG. 4A.

[0056] According to an embodiment, the circuitry 202 of the electronic device 102 can control the displayed UI elements 106 to be overlaid on the displayed feedback image 404. As shown in FIG. 4A, the circuitry 202 can control the first display screen 104 of the electronic device 102 to display the UI elements 106 over the feedback image 404 in the same portion of the first display screen 104.

[0057] In one or more embodiments, the circuitry 202 can control the opacity of the displayed UI elements 106 overlaid on the displayed feedback image 404. For example, the circuitry 202 can reduce the amount of opacity of the displayed UI elements 106 such that the media content 402 of the received feedback image 404 is easily visible through the displayed UI elements 106. For example, the opacity of the displayed UI elements 106 can be reduced to 40% opacity such that the display of the media content 402 of the feedback image 404 is visible to the user 118. This arrangement of the low opacity UI elements 106 overlaid on the feedback image 404 allows the user 118 to simultaneously view the feedback image 404 and the UI elements 106 while controlling the media display device 112 via his or her line of sight. Furthermore, this arrangement allows the user 118 to view the UI elements 106 and the feedback image 404 on the entire screen (or a specific portion) of the first display screen 104. In another embodiment, the multiple UI elements 106 may be overlaid on the feedback image 404 such that the opacity of the feedback image 404 is controlled (e.g., reduced) instead of controlling the opacity of the multiple UI elements 106. Thus, the electronic device 102 may allow the user to control the media presentation device 112 while continuing to view the media content 402 on the electronic device 102 by simultaneously displaying the feedback image 404 and the multiple UI elements 106 on the electronic device 102.

[0058] 4B illustrates an example scenario 400B showing media content 408, an “information” UI element 410 (e.g., a first UI element) of the plurality of UI elements 106, a feedback image 412, a first piece of information 414, and a third portion 416 of a first display screen 104 of an electronic device 102.

[0059] In some embodiments, a user 118 may require information regarding media content, such as media content 408 being rendered on the media display 112. For example, the media display 112 may render a documentary as the media content 408. The user 118 requires information regarding the content of the documentary, such as the name or synopsis of the documentary, and may therefore move a gaze position onto an “information” UI element 410 (i.e., a first UI element) of the plurality of UI elements 106. In such a case, the circuit 202 may determine a gaze position of the user 118 on the “information” UI element 410.

[0060] Additionally, the circuit 202 can select the “information” UI element 410 from the plurality of UI elements 106 based on determining the position of the user's 118 gaze on the “information” UI element 410. As a result, the circuit 202 can send a first control signal to the media display device 112 to receive information, such as first information 414, regarding the media content 408 being displayed on the media display device 112. In one example, the first information 414 can be information regarding a documentary being rendered on the media display device 112. In one embodiment, the circuit 202 can receive a feedback image, such as the feedback image 412 corresponding to the media content 408, and the first information 414 from the media display device 112. In some embodiments, the circuit 202 can receive the first information 414 regarding the media content 408 from the server 120 (shown in FIG. 1 ).

[0061] According to an embodiment, the circuitry 202 can be configured to control the first display screen 104 to display information, such as first information 414, regarding the media content 408 being displayed on the second display screen 114 of the media presentation device 112. The circuitry 202 can control the first display screen 104 to display the first information 414 based on a selection of an “information” UI element 410 among the plurality of UI elements 106 by a gaze of the user 118. In an embodiment, the circuitry 202 can display the first information 414 on a third portion 416 of the first display screen 104 (i.e., a portion different from the first portion 310 and the second portion 312 shown in FIG. 3A). Furthermore, the circuitry 202 can control the first display screen 104 of the electronic device 102 to display a feedback image 412 on the first display screen 104. It is noted that the location of the third portion 416 on the first display screen 104 shown in FIG. 4B is provided by way of example only. The present disclosure is applicable to other locations of the third portion 416 on the first display screen 104 without departing from the scope of the present disclosure.

[0062] In one or more embodiments, the UI elements 106, the feedback image 412, and the first information 414 may be displayed in three different portions of the first display screen 104. In an exemplary embodiment, the display positions of the UI elements 106, the feedback image 412, and the first information 414 may be preset. In another exemplary embodiment, the display positions of the UI elements 106, the feedback image 412, and the first information 414 may be set based on user input. In an exemplary embodiment, the UI elements 106 may be overlaid on the feedback image 412 and the first information 414 with a controlled opacity, such as a low opacity as described with reference to FIG. 4A.

[0063] Figures 5A and 5B collectively illustrate other exemplary scenarios for control of the media presentation device 112 of Figure 1, in accordance with an embodiment of the present disclosure. Figures 5A and 5B are described with reference to elements of Figures 1 and 2. Figure 5A illustrates an exemplary scenario 500A. The exemplary scenario 500A illustrates a "home" UI element 502 (i.e., a first UI element) of the plurality of UI elements 106, guidance information 504, and media content 506. As shown in Figure 5A, the media content 506 may relate to news-related content being rendered on the media presentation device 112.

[0064] According to an embodiment, the media display 112 can render media content 506 (e.g., news content). The user 118 needs to look at a home screen of the media display 112 to switch to another media content on the media display 112, and thus can further gaze at the "Home" UI element 502. The circuit 202 of the electronic device 102 can be configured to render guidance information 504 (e.g., information regarding the operation of UI elements such as the "Home" UI element 502) for the user 118 to view a home screen (e.g., a main page) of the media display 112 based on a determination of a position of the user's 118 gaze on the "Home" UI element 502 among the plurality of UI elements 106. Thus, the circuit 202 can select the "Home" UI element 502 based on the user's 118 gaze on the "Home" UI element 502 to display the home screen (not shown in FIG. 5A ) on the second display screen 114 of the media display 112. Additionally, guidance information 504 may be rendered to control one or more configuration settings of the media presentation device 112 through one or more UI elements of the plurality of UI elements 106 displayed on the first display screen 104. For example, the guidance information 504 may provide information to the user 118 to configure a particular setting of the media presentation device 112 (e.g., but not limited to, adjusting color, adjusting aspect ratio). In an embodiment, the plurality of UI elements 106 and the guidance information 504 may be provided on different portions of the first display screen 104 of the electronic device 102. It should be noted that the location of the guidance information 504 on the first display screen 104 shown in FIG. 5A is provided by way of example only. The present disclosure may be applicable to other locations of the guidance information 504 on the first display screen 104 without departing from the scope of the present disclosure.

[0065] In some embodiments, the circuitry 202 can be configured to render guidance information 504 for the user 118 by determining the user's 118's gaze on an "information" UI element 410 (shown in FIG. 4B ). In an example embodiment, the first information 414 in FIG. 4B can be information regarding the operation of each UI element of the plurality of UI elements 106, or information regarding the media content 506 currently being rendered on the media presentation device 112. Thus, the guidance information 504 can assist the user 118 in selecting an appropriate UI element for controlling one or more configuration settings or different functions of the media presentation device 112.

[0066] 5B illustrates an example scenario 500B showing media content 508, a feedback image 510, and a "playback control" UI element (e.g., a "rewind" UI element 512) among the UI elements 106. The media content 508 may include a movie being rendered on the media display device 112.

[0067] According to an embodiment, the media presentation device 112 can render media content 508, such as a movie. The circuitry 202 of the electronic device 102 can be configured to control the first display screen 104 to display a set of UI elements 514 (i.e., a set of UI elements that can be a subset of the UI elements 106 on the first display screen 104 of FIG. 5A) based on the received feedback image 510. For example, the set of UI elements 514 can include a "power" UI element, a "home" UI element, an "information" UI element, a "volume up" UI element, a "volume down" UI element, a "rewind" UI element 512, a "pause" UI element, a "play" UI element, and a "fast forward" UI element, etc., as shown in FIG. 5B. The set of UI elements 514 can be displayed based on the type of media content 508 being displayed on the media presentation device 112 and the feedback image 510 received by the electronic device 102. Additionally, the circuit 202 may also hide or omit unimportant UI elements from the plurality of UI elements 106 based on the media content 508 being rendered such that the screen real estate of the first display screen 104 of FIG. 5B is relatively less cluttered compared to the first display screen 104 of FIG. 5A. As a result, the circuit 202 may efficiently determine the position of the user 118's gaze on any particular UI element (i.e., the first UI element) of the set of UI elements 514. In one example, the circuit 202 may hide or omit UI elements such as "navigation" and "numeric" UI elements during the playback of a movie. As shown in FIG. 5B, the circuit 202 may control the first display screen 104 to display a set of UI elements 514 that are deemed necessary for the user 118 to control the playback of the media content 508 (e.g., a movie) on the media presentation device 112.

[0068] In an exemplary embodiment, a user 118 may need to rewind the media content 508 being rendered on the media display device 112. The circuit 202 may further determine a position of the user's 118 gaze over a certain time interval (e.g., several seconds) on a "rewind" UI element 512 (i.e., a first UI element) of the series of UI elements 514. The circuit 202 may select the "rewind" UI element 512 based on the determination of the user's 118 gaze on the "rewind" UI element 512. Further, the circuit 202 may send a first control signal to the media display device 112 to rewind the media content 508 on the media display device 112 based on the gaze as a user input. Further, the circuit 202 may receive a feedback image 510 including an image of the media content 508 from the media display device 112 based on the sent first control signal. The series of UI elements 514 and the feedback image 510 may be displayed on different portions of the first display screen 104. In some embodiments, a series of UI elements 514 may be overlaid on the feedback image 510 on the first display screen 104. Thus, the circuit 202 may enable the user 118 to efficiently control the rendering of the media content 508 on the media presentation device 112 by using the series of UI elements 514. The series of UI elements 514 shown in FIG. 5B is presented by way of example only. The present disclosure is applicable to other series of UI elements on the first display screen 104 without departing from the scope of the present disclosure.

[0069] Figure 6 is a flow chart illustrating an exemplary method for control of a media presentation device via an electronic device, according to an embodiment of the disclosure. Figure 6 is described with reference to elements of Figures 1, 2, 3A, 3B, 4A, 4B, 5A, and 5B. Figure 6 illustrates a flow chart 600. The operations of flow chart 600 may be performed by a computer system, such as electronic device 102 or circuit 202. Operations may begin at 602 and proceed to 604.

[0070] At 604, image capture device 108 can be controlled to capture a first image of a user of electronic device 102. In one or more embodiments, circuitry 202 of electronic device 102 can be configured to control image capture device 108 of electronic device 102 to capture a first image of user 118 of electronic device 102, for example, as described in FIG.

[0071] At 606, a gaze position of the user 118 on a first UI element of the displayed plurality of UI elements can be determined based on the first captured image of the user 118. In one or more embodiments, the circuitry 202 of the electronic device 102 can be configured to determine a gaze position of the user 118 on a first UI element 106A of the displayed plurality of UI elements 106 based on the first captured image of the user 118, for example, as described in FIG.

[0072] At 608, a first UI element 106A of the plurality of UI elements 106 may be selected based on determining a position of the gaze on the first UI element 106A over a period of time. In one or more embodiments, the circuitry 202 of the electronic device 102 may be configured to select the first UI element 106A of the plurality of UI elements 106 based on determining a position of the gaze on the first UI element 106A over a period of time, for example as described in FIG. 3A. In some embodiments, the period of time may be several seconds (e.g., at least 3 seconds).

[0073] At 610, a first control signal may be sent to the media presentation device 112 based on the selection of the first UI element 106A of the plurality of UI elements 106. In one or more embodiments, the circuitry 202 of the electronic device 102 may be configured to send the first control signal to the media presentation device 112 based on the selection of the first UI element 106A of the plurality of UI elements 106, for example, as described in FIG.

[0074] At 612, a feedback image may be received from the media display device 112 based on the transmitted first control signal. In one or more embodiments, the circuitry 202 of the electronic device 102 may be configured to receive a feedback image 304 from the media display device 112 based on the transmitted first control signal, for example, as described in FIG. 3A. The received feedback image 304 may indicate control of the media display device 112 based on the transmitted first control signal.

[0075] At 614, the first display screen may be controlled to display the received feedback image 304 along with the displayed plurality of UI elements 106. In one or more embodiments, the circuitry 202 of the electronic device 102 may be configured to control the first display screen 104 to display the received feedback image 304 along with the displayed plurality of UI elements 106, for example, as described in FIG. 3A. Control may proceed to an end.

[0076] Although flowchart 600 depicts discrete operations, such as 604, 606, 608, 610, 612, and 614, the disclosure is not so limited. Thus, in some embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without departing from the essence of the disclosed embodiments.

[0077] Various embodiments of the present disclosure may provide a non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by an electronic device (electronic device 102) including an image capture device (such as image capture device 108) and a first display screen (such as first display screen 104), cause the electronic device 102 to perform operations. The operations may include controlling the first display screen to display a plurality of user interface (UI) elements. Selection of each of the plurality of UI elements controls a media display device (such as media display device 112) communicatively coupled to the electronic device. The operations further include controlling the image capture device to capture a first image of a user of the electronic device. The operations further include determining a position of the user's gaze on a first UI element of the displayed plurality of UI elements based on the captured first image of the user. The operations further include selecting a first UI element of the plurality of UI elements based on determining a position of the gaze on the first UI element over a period of time. The operations further include transmitting a first control signal to the media display device based on the selection of the first UI element of the plurality of UI elements. The operations further include receiving a feedback image from the media display device based on the transmitted first control signal, the received feedback image indicating control of the media display device based on the transmitted first control signal, and controlling the first display screen to display the received feedback image along with the displayed plurality of UI elements.

[0078] An exemplary aspect of the disclosure may include an electronic device (e.g., electronic device 102). Electronic device 102 may include an image capture device (e.g., image capture device 108) and a first display screen (e.g., first display screen 104) that may be configured to display a plurality of user interface (UI) elements (e.g., a plurality of UI elements 106). Selection of each of the plurality of UI elements 106 may control a media display device (e.g., media display device 112) communicatively coupled to electronic device 102. Electronic device 102 may further include a circuit (e.g., circuit 202) coupled to image capture device 108 and first display screen 104. Circuit 202 may be configured to control image capture device 108 to capture a first image of a user (e.g., user 118) of electronic device 102. The circuit 202 may be further configured to determine a gaze position of the user 118 on a first UI element (e.g., the first UI element 106A) of the displayed plurality of UI elements 106 based on the captured first image of the user 118. The circuit 202 may be further configured to select the first UI element 106A of the plurality of UI elements 106 based on the determination of the gaze position on the first UI element 106A over a period of time. The circuit 202 may be further configured to transmit a first control signal to the media display device 112 based on the selection of the first UI element 106A of the plurality of UI elements 106. The circuit 202 may be further configured to receive a feedback image (e.g., the feedback image 110) from the media display device 112 based on the transmitted first control signal. The received feedback image 110 may indicate control of the media display device 112 based on the transmitted first control signal. The circuitry 202 may be further configured to control the first display screen 104 to display the received feedback image 110 along with the displayed plurality of UI elements 106 .

[0079] According to one embodiment, each UI element of the plurality of UI elements 106 may be associated with a different function related to the media presentation device 112 .

[0080] According to an embodiment, the circuitry 202 may be further configured to control movement of a cursor (such as cursor 308) on the plurality of UI elements 106 in response to changes in the position of the user's 118 gaze on the plurality of UI elements 106.

[0081] According to an embodiment, the circuit 202 may be further configured to highlight the first UI element 106A based on a selection of the first UI element 106A of the plurality of UI elements 106.

[0082] According to one embodiment, the feedback image 304 received from the media display device 112 may correspond to a UI icon (such as UI icon 302a) that indicates control of the media display device 112 based on the transmitted first control signal.

[0083] According to one embodiment, the feedback image 316 received from the media display device 112 may correspond to media content (e.g., media content 314) being displayed on a second display screen (e.g., second display screen 114) of the media display device 112 based on the transmitted first control signal.

[0084] According to an embodiment, the circuitry 202 may be further configured to control the overlay of a displayed plurality of UI elements 106 on a displayed feedback image 404 that may correspond to media content 402 displayed on the second display screen 114 of the media presentation device 112.

[0085] According to an embodiment, the circuitry 202 may be further configured to control the opacity of the plurality of UI elements 106 overlaid on the displayed feedback image 404 .

[0086] According to an embodiment, the circuitry 202 may be further configured to control the first display screen 104 to display information (such as the first information 414) related to the media content 408 being displayed on the second display screen 114 of the media presentation device 112 based on a selection of one or more UI elements of the plurality of UI elements 106.

[0087] According to an embodiment, the circuitry 202 may be further configured to control the first display screen 104 to display the plurality of UI elements 106 on a first portion (e.g., first portion 310) of the first display screen 104 and to display the received feedback image 304 on a second portion (e.g., second portion 312) of the first display screen 104. The first portion 310 may be different from the second portion 312.

[0088] According to one embodiment, the circuit 202 may be further configured to determine at least one of a gesture associated with a facial expression of the user 118 or a gesture associated with the head of the user 118 to select a first UI element 106A of the plurality of UI elements 106 displayed on the first display screen 104.

[0089] According to an embodiment, the circuitry 202 may be further configured to render guidance information (e.g., guidance information 504) for the user 118 to control one or more configuration settings of the media presentation device 112 through one or more UI elements of the plurality of UI elements 106 displayed on the first display screen 104.

[0090] According to an embodiment, the circuitry 202 may be further configured to control the first display screen 104 to display a sequence of UI elements 514 of the plurality of UI elements 106 on the first display screen 104 based on the received feedback image 510.

[0091] The present disclosure can be realized in hardware or in a combination of hardware and software. The present disclosure can be realized in a centralized manner in at least one computer system, or in a distributed manner where different elements can be distributed across several interconnected computer systems. Any computer system or other device adapted to perform the methods described herein can be suitable. The combination of hardware and software can be a general-purpose computer system that includes a computer program that, when loaded and executed, can control a computer system to perform the methods described herein. The present disclosure can be realized in hardware, including parts of an integrated circuit that also perform other functions.

[0092] The present disclosure may also be embodied in a computer program product, which includes all features enabling the implementation of the methods described herein and which is capable of executing these methods when loaded into a computer system. A computer program in this context means any expression, in any language, code or notation, of a set of instructions intended to cause a system capable of processing information to execute a particular function, either directly, or after a) conversion into another language, code or notation, b) reproduction in a different content form, or both.

[0093] While the present disclosure has been described with reference to several embodiments, those skilled in the art will recognize that various modifications can be made and equivalents substituted without departing from the scope of the present disclosure. Many modifications can also be made to adapt a particular situation or material to the teachings of the present disclosure without departing from the scope of the present disclosure. Therefore, it is not intended that the present disclosure be limited to the particular embodiments disclosed, but rather, it is intended to include all embodiments falling within the scope of the appended claims. [Explanation of symbols]

[0094] 102 Electronic equipment 104 First Display Screen 106 Multiple UI Elements 108 Image capture device 112 Media display device 114 Second Display Screen 118 users 400A Scenario 402 Media Content 402A "Volume Up" UI Icon 404 Feedback 406 First UI Element

Claims

1. 1. An electronic device comprising: An image capture device; a first display screen configured to display a plurality of user interface (UI) elements; circuitry coupled to the image capture device and the first display screen; wherein selection of each of the plurality of UI elements controls a media display device communicatively coupled to the electronic device, and the circuitry further comprises: controlling the image capture device to capture a first image of a user of the electronic device; determining a gaze position of the user on a first UI element of the displayed plurality of UI elements based on the captured first image of the user; selecting the first UI element of the plurality of UI elements based on determining a position of the gaze on the first UI element over a time interval; sending a first control signal to the media presentation device based on the selection of the first UI element of the plurality of UI elements; receiving, based on the transmitted first control signal, a feedback image from the media display device indicative of control of the media display device based on the transmitted first control signal; controlling the first display screen to display the received feedback image along with the displayed plurality of UI elements; It is configured as follows:

1. An electronic device comprising:

2. each UI element of the plurality of UI elements associated with a different function associated with the media presentation device; 2. The electronic device of claim 1.

3. the circuitry is further configured to control movement of a cursor on the plurality of UI elements in response to a change in a position of the user's gaze on the plurality of UI elements.

2. The electronic device of claim 1.

4. the circuitry is further configured to highlight the first UI element of the plurality of UI elements based on the selection of the first UI element.

2. The electronic device of claim 1.

5. the feedback image received from the media display device corresponds to a UI icon indicative of control of the media display device based on the transmitted first control signal.

2. The electronic device of claim 1.

6. the feedback image received from the media display device corresponds to media content being displayed on a second display screen of the media display device based on the transmitted first control signal.

2. The electronic device of claim 1.

7. the circuitry is further configured to control an overlay of the displayed plurality of UI elements on the displayed feedback image corresponding to media content being displayed on the second display screen of the media presentation device.

7. The electronic device of claim 6.

8. the circuitry is further configured to control an opacity of the displayed UI elements overlaid on the displayed feedback image.

8. The electronic device of claim 7.

9. the circuitry is further configured to control the first display screen to display information related to media content being displayed on the second display screen of the media presentation device based on a selection of one or more UI elements of the plurality of UI elements.

7. The electronic device of claim 6.

10. the circuitry is further configured to control the first display screen to display the plurality of UI elements on a first portion of the first display screen and to display the received feedback image on a second portion of the first display screen different from the first portion.

2. The electronic device of claim 1.

11. the circuitry is further configured to determine at least one of a gesture associated with a facial expression of the user or a gesture associated with a head of the user to select the first UI element of the plurality of UI elements displayed on the first display screen.

2. The electronic device of claim 1.

12. the circuitry is further configured to render guidance information for the user for controlling one or more configuration settings of the media presentation device through one or more UI elements of the plurality of UI elements displayed on the first display screen.

2. The electronic device of claim 1.

13. the circuitry is further configured to control the first display screen to display a sequence of UI elements of the plurality of UI elements on the first display screen based on the received feedback image.

2. The electronic device of claim 1.

14. 1. An electronic device comprising an image capture device and a first display screen configured to display a plurality of user interface (UI) elements, wherein selection of each of the plurality of UI elements controls a media display device communicatively coupled to the electronic device, controlling the image capture device to capture a first image of a user of the electronic device; determining a gaze position of the user on a first UI element of the displayed plurality of UI elements based on the captured first image of the user; selecting the first UI element of the plurality of UI elements based on determining a position of the gaze on the first UI element over a time interval; sending a first control signal to the media presentation device based on the selection of the first UI element of the plurality of UI elements; receiving, based on the transmitted first control signal, a feedback image from the media display device indicative of control of the media display device based on the transmitted first control signal; controlling the first display screen to display the received feedback image along with the displayed plurality of UI elements; The method according to claim 1, further comprising:

15. controlling a movement of a cursor on the plurality of UI elements in response to a change in a position of the user's gaze on the plurality of UI elements. The method of claim 14.

16. the feedback image received from the media display device corresponds to media content being displayed on a second display screen of the media display device based on the transmitted first control signal. The method of claim 14.

17. and controlling an overlay of the displayed plurality of UI elements on the displayed feedback image corresponding to the media content being displayed on the second display screen of the media presentation device.

17. The method of claim 16.

18. and controlling an opacity of the displayed UI elements overlaid on the displayed feedback image.

20. The method of claim 17.

19. controlling the first display screen to display the plurality of UI elements on a first portion of the first display screen and to display the received feedback image on a second portion of the first display screen different from the first portion. The method of claim 14.

20. A non-transitory computer readable medium having computer executable instructions stored thereon, the computer executable instructions, when executed by an electronic device including an image capture device and a first display screen, causing the electronic device to: controlling the first display screen to display a plurality of user interface (UI) elements, selection of each of the plurality of UI elements controlling a media display device communicatively coupled to the electronic device; controlling the image capture device to capture a first image of a user of the electronic device; determining a gaze position of the user on a first UI element of the displayed plurality of UI elements based on the captured first image of the user; selecting the first UI element of the plurality of UI elements based on determining a position of the gaze on the first UI element over a time interval; sending a first control signal to the media presentation device based on the selection of the first UI element of the plurality of UI elements; receiving, based on the transmitted first control signal, a feedback image from the media display device indicative of control of the media display device based on the transmitted first control signal; controlling the first display screen to display the received feedback image along with the displayed plurality of UI elements; A non-transitory computer-readable medium for causing a computer to perform operations including:

Citation Information

Patent Citations

  • Method and device for providing option menus using graphic user interface

    US20070226654A1

  • Portable Multifunction Device, Method, and Graphical User Interface Supporting User Navigations of Graphical Objects on a Touch Screen Display

    US20080165153A1

  • Image display apparatus and method for operating the same

    US20110267291A1

  • Method and apparatus for providing remote control via a touchable display

    US20120081299A1

  • Image display apparatus and method of operating the image display apparatus

    US20150009334A1