Head-mounted electronic device with adjustable frame rate

Abstract

A head-mounted device includes one or more displays configured to present media content. The media content may be presented in a user interface window. The head-mounted device may include a display controller configured to adjust a frame rate of the one or more displays, and may include a frame rate management circuit configured to manage the frame rate of the one or more displays. The frame rate management circuitry is configured to manage a frame rate based on a type of media content being presented in the user interface window and based on a preferred frame rate, such as associated with the media content, a size of the user interface window relative to a total display area of the one or more displays, gaze point data, gesture data, head pose data, and the like. Data associated with other body parts, audio information, and additional information of other data determine whether to adjust the frame rate of the one or more displays.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 349,411 filed on July 10, 2023 and U.S. Provisional Patent Application No. 63 / 405,685 filed on September 12, 2022, which are hereby incorporated by reference in their entirety. Technical Field

[0002] The present disclosure relates generally to electronic devices, and more particularly to electronic devices such as head mounted devices. Background Art

[0003] An electronic device, such as a head mounted device, may have a display for displaying images. The display may have a default frame rate. The display may be used to present film (movie) content that is typically recorded at 24 frames per second (fps). Such 24fps movie content does not always divide evenly into the default frame rate of the display. This may cause motion that appears on the display to have an uneven pace, a phenomenon sometimes referred to as jitter. Summary of the invention

[0004] The head mounted device may include one or more displays configured to present different types of content to the user. Depending on the type of content or where the user is currently focused, the frame rate manager may determine whether to adjust the display frame (refresh) rate. For example, if the frame rate manager detects that the user is currently focused on movie content encoded at 24fps, the frame rate manager may adjust the display frame rate to a frame rate that is a multiple of 24 to reduce jitter. If the frame rate manager detects that the user is no longer focused on the movie content, the frame rate manager may reduce the display frame rate to a default (native) display frame rate that is not a multiple of 24 to save power.

[0005] The frame rate manager may consider a variety of information to determine whether to adjust the display frame rate. The decision whether to adjust the display frame rate at the frame rate manager may be based on the user's focus position within a two-dimensional or three-dimensional environment / scene, a preferred frame rate associated with the content type, the size of the graphical user interface window in which the content is currently presented, gaze point information, gesture information, head posture information, information about whether the user is currently speaking, information associated with other body parts, information reflecting whether the system is oversubscribed or overloaded, thermal (heat) information associated with one or more hardware components within the head mounted device, battery charge information, power setting information reflecting whether the head mounted device is operating in a low power mode or a power saving mode, audio information, combinations of these inputs, and / or other information.

[0006] At least some of the information received at the frame rate manager can be used as a proxy for determining whether the user is currently focused on the content of interest or on a specific location or area of ​​interest in a two-dimensional or three-dimensional environment / scene. In one embodiment, when the media playback window in which the movie content is being presented expands to full screen or occupies a significant portion of the entire display area, the frame rate manager can infer that the user is focused on the movie content. If the user's focus position or gaze point is within the media playback window for a period of time, the frame rate manager can infer that the user is focused on the movie content. When it is detected that the user's hands and / or head have been relatively idle for a period of time, the frame rate manager can conclude that the user is ready to watch the movie content. The frame rate manager can only allow switching to a higher frame rate when: the system is not overbooked; thermal information indicates that the system is not overheated; battery charge information indicates that the current battery charge exceeds a predetermined threshold; and / or the device is not currently operating in a low power mode or power saving mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a top view of an illustrative head mounted device according to some embodiments.

[0008] Figure 2 is a schematic diagram of an exemplary head-mounted device according to some embodiments.

[0009] Figure 3 is a state diagram showing how a display in a head mounted device according to some embodiments can operate in multiple display modes with different frame rates.

[0010] Figure 4 is an illustration of an illustrative head mounted device having a frame rate management circuit configured to adjust a display frame rate based on different types of information according to some embodiments.

[0011] Figure 5 is a view of a display having a window occupying a portion of the display area according to some embodiments.

[0012] Figure 6 is a view of a display with a window expanding to fill the entire display area according to some embodiments.

[0013] Figure 7 is a view of a display and a user's gaze points at various locations in a display area according to some embodiments.

[0014] Figure 8 According to some embodiments, a method for operating a combination of Figures 1 to 7 A flowchart of illustrative steps for use with a head-mounted electronic device of the type shown. DETAILED DESCRIPTION

[0015] An electronic device, such as a head mounted device, may have a front face facing away from a user's head, and may have an opposite back face facing the user's head. One or more displays may be used to provide images to the user's eyes. The display may be capable of operating at a plurality of different frame rates. The display may be used to display a three-dimensional environment or scene. When a user is viewing a first type of content on the display, the display may be operated using a first frame rate. When a user is viewing a second type of content on the display, the display may be operated using a second frame rate different from the first frame rate that is more suitable for the second type of content. Using the second frame rate to display the second type of content may help reduce jitter that would otherwise be noticeable if the first frame rate was used.

[0016] The determination of whether to switch between a first frame rate and a second frame rate and between other frame rates may be based on various information, including, but not limited to, a preferred frame rate obtained from the content to be displayed, a window size associated with the content of interest, gaze point information, gesture information, head posture information, other information associated with one or more body parts of the user, and information associated with other hardware components and / or software applications on the head mounted device.

[0017] Figure 1 A top view of an exemplary head mounted device is shown in FIG. Figure 1 As shown in , a head-mounted device such as electronic device 10 may have a head-mounted support structure such as housing 12. Housing 12 may include a portion (e.g., head-mounted support structure 12T) for allowing device 10 to be worn on the user's head. Support structure 12T may be formed of fabric, polymer, metal, and / or other materials. Support structure 12T may form a strap or other head-mounted support structure to help support device 10 on the user's head. The main support structure of housing 12 (e.g., a head-mounted housing such as main housing portion 12M) may support electronic components such as display 14.

[0018] The main housing portion 12M may include a housing structure formed of metal, polymer, glass, ceramic and / or other materials. For example, the housing portion 12M may have a housing wall on the front F and a housing wall on the adjacent top, bottom, left and right sides formed of a rigid polymer or other rigid support structure, and these rigid walls may be optionally covered with electronic components, fabrics, leather or other soft materials, etc. The housing portion 12M may also have an internal support structure, such as a frame (chassis) and / or a structure that performs multiple functions (such as controlling airflow and heat dissipation while providing structural support). The walls of the housing portion 12M may enclose the internal components 38 in the internal area 34 of the device 10, and the internal area 34 may be separated from the environment (external area 36) surrounding the device 10. The internal components 38 may include integrated circuits, actuators, batteries, sensors and / or other circuits and structures for the device 10. The housing 12 may be configured to be worn on the user's head and may form glasses, frame glasses, hats, masks, helmets, goggles and / or other head-mounted devices. Configurations in which housing 12 forms goggles are sometimes described herein as an example.

[0019] The front side F of the housing 12 may face outwardly, away from the user's head and face. The opposite rear side R of the housing 12 may face the user. The portion of the housing 12 located on the rear side R (e.g., the portion of the main housing 12M) may form a cover, such as cover 12C (sometimes referred to as a blind). The presence of cover 12C on the rear side R may help hide the internal housing structure, the interior components 38, and other structures in the interior area 34 from being seen by the user.

[0020] Device 10 may have one or more cameras, such as Figure 1 Camera 46. Camera 46 mounted on front face F and facing outward (toward the front of device 10 and away from the user) may sometimes be referred to herein as a forward-pointing or front-facing camera. Camera 46 may capture visual odometry information, image information processed to locate objects in the user's field of view (e.g., so that virtual content can be properly aligned with respect to real-world objects), image content displayed in real time to the user of device 10, and / or other suitable image data. For example, a forward-pointing (positive) camera may allow device 10 to monitor movement of device 10 relative to the environment surrounding device 10 (e.g., the camera may be used to form part of a visual odometry system or a visual-inertial odometry system). The forward-pointing camera may also be used to capture images of the environment displayed to the user of device 10. If desired, images from multiple forward-pointing cameras may be merged with each other and / or the forward-pointing camera content may be merged with computer-generated content for the user.

[0021] Device 10 may have any suitable number of cameras 46. For example, device 10 may have K cameras, where the value of K is at least one, at least two, at least four, at least six, at least eight, at least ten, at least 12, less than 20, less than 14, less than 12, less than 10, 4-10, or other suitable value. Cameras 46 may be sensitive at infrared wavelengths (e.g., camera 46 may be an infrared camera), may be sensitive at visible wavelengths (e.g., camera 46 may be a visible camera), and / or cameras 46 may be sensitive at other wavelengths. If desired, cameras 46 may be sensitive at both visible and infrared wavelengths.

[0022] The device 10 may have a left optical module and a right optical module 40. The optical module 40 supports electronic components and optical components such as light-emitting components and lenses, and therefore may sometimes be referred to as an optical assembly, an optical system, an optical component support structure, a lens and display support structure, an electronic component support structure, or a housing structure. Each optical module may include a corresponding display 14, a lens 30, and a support structure such as a support structure 32. The support structure 32, which may sometimes be referred to as a lens support structure, an optical component support structure, an optical module support structure, or an optical module portion, or a lens barrel, may include a hollow cylindrical structure with an open end or other support structures for accommodating the display 14 and the lens 30. The support structure 32 may, for example, include a left lens barrel that supports the left display 14 and the left lens 30 and a right lens barrel that supports the right display 14 and the right lens 30.

[0023] Display 14 may include an array of pixels or other display devices to produce images. Display 14 may include, for example, organic light emitting diode pixels formed on a substrate having thin film circuits and / or formed on a semiconductor substrate, pixels formed from a crystalline semiconductor die, liquid crystal display pixels, scanning display devices, and / or other display devices for producing images.

[0024] Lens 30 may include one or more lens elements for providing image light from display 14 to corresponding eye zone 13. The lens may be implemented using a refractive glass lens element, using a mirror lens structure (a catadioptric lens), using a Fresnel lens, using a holographic lens, and / or other lens systems.

[0025] When the user's eyes are located in the eye zone 13, the display (display panel) 14 operates together to form the display of the device 10 (e.g., the user's eyes can view the images provided by the corresponding left and right optical modules 40 in the eye zone 13, so that a stereoscopic image is created for the user). When the user views the display, the left image from the left optical module is fused with the right image from the right optical module.

[0026] It may be desirable to monitor the user's eyes when the user's eyes are located in the eye-adaptive zone 13. For example, it may be desirable to use a camera to capture an image of the user's iris (or other parts of the user's eyes) for user authentication. It may also be desirable to monitor the direction of the user's gaze. Gaze tracking information can be used as a form of user input and / or can be used to determine where the image content resolution should be locally enhanced within the image in the fovea imaging system. In order to ensure that the device 10 can capture a satisfactory eye image when the user's eyes are located in the eye-adaptive zone 13, each optical module 40 may be provided with a camera (such as camera 42) and one or more light sources (such as light emitting diodes 44) or other light emitting devices (such as lasers, lamps, etc.). The camera 42 and the light emitting diode 44 can operate at any suitable wavelength (visible light, infrared light, and / or ultraviolet light). For example, the diode 44 can emit infrared light that is invisible (or almost invisible) to the user. This allows the eye monitoring operation to be performed continuously without interfering with the user's ability to view the image on the display 14.

[0027] Figure 2 A schematic diagram of an illustrative electronic device (such as a head-mounted device or other wearable device) is shown in FIG. Figure 2 Device 10 may operate as a standalone device and / or resources of device 10 may be used to communicate with external electronic equipment. For example, communication circuitry in device 10 may be used to send user input information, sensor information, and / or other information to an external electronic device (e.g., wirelessly or via a wired connection). Each of these external devices may include Figure 2 Components of the type shown in device 10.

[0028] like Figure 2 As shown in , a head mounted device such as device 10 may include a control circuit 20. The control circuit 20 may include storage and processing circuits for supporting the operation of the device 10. The storage and processing circuits may include storage devices such as non-volatile memory (e.g., flash memory or other electrically programmable read-only memory configured to form a solid state drive), volatile memory (e.g., static or dynamic random access memory), etc. The processing circuits in the control circuit 20 may be used to collect inputs from sensors and other input devices, and may be used to control output devices. The processing circuits may be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors and other wireless communication circuits, power management units, audio chips, application specific integrated circuits, etc. During operation, the control circuit 20 may use the display 14 and other output devices to provide visual output and other outputs to the user.

[0029] To support communication between device 10 and external equipment, control circuit 20 may use communication circuit 22 for communication. Circuit 22 may include an antenna, a radio frequency transceiver circuit, and other wireless communication circuits and / or wired communication circuits. Circuit 22 (which may sometimes be referred to as control circuits and / or control and communication circuits) may support two-way wireless communication via a wireless link between device 10 and external equipment (e.g., a companion device such as a computer, a cellular phone, or other electronic device, an accessory such as a pointing device, a controller, a computer stylus or other input device, a speaker or other output device, etc.). For example, circuit 22 may include a radio frequency transceiver circuit, such as a wireless local area network transceiver circuit configured to support communication via a wireless local area network link, a near field communication transceiver circuit configured to support communication via a near field communication link, a cellular phone transceiver circuit configured to support communication via a cellular phone link, or a transceiver circuit configured to support communication via any other appropriate wired or wireless communication link. For example, communication may be performed via a wireless local area network link. link, Wireless communications may be supported by a wireless link, a wireless link operating at frequencies between 10 GHz and 400 GHz, a 60 GHz link or other millimeter wave link, a cellular telephone link, or other wireless communications link. Device 10 may include power circuits for sending and / or receiving wired and / or wireless power, if desired, and may include a battery or other energy storage device. For example, device 10 may include a coil and a rectifier to receive wireless power provided to circuits in device 10.

[0030] Device 10 may include input-output devices, such as device 24. Input-output devices 24 may be used to collect user input, to collect information about the user's surroundings, and / or to provide output to the user. Device 24 may include one or more displays, such as display 14. Display 14 may include one or more display devices, such as an organic light emitting diode display panel (a panel having organic light emitting diode pixels formed on a polymer substrate or silicon substrate containing pixel control circuitry), a liquid crystal display panel, a micro-electromechanical system display (e.g., a two-dimensional mirror array or scanning mirror display device), a display panel having an array of pixels formed from crystalline semiconductor light emitting diode dies (sometimes referred to as micro-LEDs), and / or other display devices.

[0031] Sensors 16 in input-output devices 24 may include force sensors (e.g., strain gauges, capacitive force sensors, resistive force sensors, etc.), audio sensors (such as microphones), touch and / or proximity sensors (such as capacitive sensors, such as touch sensors forming buttons, trackpads, or other input devices), and other sensors. If desired, sensors 16 may include optical sensors (such as optical sensors that emit and detect light), ultrasonic sensors, optical touch sensors, optical proximity sensors, and / or other touch sensors and / or proximity sensors, monochrome and color ambient light sensors, image sensors (e.g., cameras), fingerprint sensors, iris scan sensors, retina scan sensors, and other biometric sensors, temperature sensors, sensors for measuring three-dimensional contactless gestures (“air gestures”), pressure sensors, sensors for detecting the position, orientation, and / or movement of device 10 and / or information about the user's head posture (e.g., accelerometers, magnetic sensors such as compasses) Sensors, gyroscopes, and / or inertial measurement units that include some or all of these sensors), health sensors such as blood oxygen sensors, heart rate sensors, blood flow sensors, and / or other health sensors, radio frequency sensors, three-dimensional camera systems such as depth sensors (e.g., structured light sensors and / or depth sensors based on stereo imaging devices that capture three-dimensional images), and / or optical sensors such as self-mixing sensors and light detection and ranging (laser) sensors that collect time-of-flight measurements (e.g., time-of-flight cameras), humidity sensors, moisture sensors, gaze tracking sensors, electromyography sensors that sense muscle activation, facial sensors, and / or other sensors. In some arrangements, device 10 may use sensor 16 and / or other input-output devices to collect user input. For example, buttons may be used to collect button press inputs, touch sensors that overlap with a display may be used to collect user touch screen inputs, a touchpad may be used to collect touch inputs, a microphone may be used to collect audio inputs (e.g., voice commands), an accelerometer may be used to monitor when a finger contacts an input surface and may therefore be used to collect finger press inputs, etc.

[0032] If desired, the electronic device 10 may include additional components (see, for example, other devices 18 in input-output devices 24). Additional components may include tactile output devices, actuators for moving the movable housing structure, audio output devices such as speakers, light emitting diodes for status indicators, light sources such as light emitting diodes for illuminating portions of the housing and / or display structure, other optical output devices, and / or other circuits for collecting input and / or providing output. The device 10 may also include a battery or other energy storage device, a connector port for supporting wired communication with auxiliary equipment and for receiving wired power, and other circuits.

[0033] Display 14 can be used to present various contents to the user's eyes. When viewed through eye zone 13, the left and right displays 14 for presenting a synthetic stereoscopic image to the user's eyes may sometimes be collectively referred to as displays 14. In one scenario, the user may be reading static content in a web browser on display 14. In another scenario, the user may be watching dynamic content, such as movie content, in a web browser or media player on display 14. In another scenario, the user may be watching video game (playing game) content on display 14. In another scenario, the user may be watching a live feed of the environment around the device 10 captured using one or more front cameras 46. If desired, computer-generated (virtual) content may be overlaid on one or more parts of the live feed presented on display 14. In another scenario, the user may be watching a live event recorded elsewhere on display 14 (e.g., at a location different from the user's location). In another scenario, the user may be using device 10 for video conferencing (live conferencing) while watching participants and / or any shared conference content of display 14. These examples are merely illustrative. In general, display 14 may be used to output any type of image or video content.

[0034] The display 14 can operate at one or more frame rates. The frame rate of the display is sometimes also referred to as its refresh rate. The frame (refresh) rate of the display 14 may not always be suitable for each type of content output by the display. If you are not careful, the mismatch between the frame rate of the display and the optimal frame rate of the specific media content may cause the motion appearing on the display 14 to have an uneven pace, which is sometimes referred to as jitter. For example, consider a scene where the default frame rate of the display is 60Hz. However, most movie (film) content is recorded or encoded at 24 frames per second (fps). Ideally, such movie content will be displayed using a display frame rate (e.g., at 48Hz, 72Hz, 96Hz, 120Hz, etc.) of a multiple of 24fps. Using a 60Hz display to display a movie encoded at 24fps will produce jitter, which may be uncomfortable for the user when using the device 10 for a long time and the movie content presented is not what the movie maker created or wanted.

[0035] According to one embodiment, device 10 may be configured to detect the type of content that a user is currently focusing on, determine an optimal display frame rate suitable for that particular type of content, and dynamically adjust the display frame rate accordingly. Figure 3 1 is a state diagram showing how display 14 can operate in multiple display modes. Display 14 can be used to display a two-dimensional environment (scene) or a three-dimensional environment (scene). Figure 3As shown, the display 14 can operate in at least a first display mode (A) 50 and a second display mode (B) 52. During operation in the first display mode 50, the display 14 can have a first frame rate equal to FR1. During operation in the second display mode 52, the display 14 can have a second frame rate different from FR1 and equal to FR2. The display 14 can be set to mode 50 by default (as an example). The frame rate FR1 can be set to be equal to 60 Hz, 90 Hz, less than 60 Hz, greater than 60 Hz, greater than 90 Hz, 60 Hz-100 Hz, greater than 120 Hz, 150 Hz, 180 Hz, 210 Hz, greater than 200 Hz, etc. The frame rate FR2 can be set to be equal to 48 Hz, 72 Hz, 96 Hz, 120 Hz, a value greater than 45 Hz, or other multiples of 24. In some embodiments, FR1 can be a multiple of 30, and FR2 can be a multiple of 24.

[0036] Configured in this way, the second display mode 52 may be more suitable for displaying movie (film) content recorded / encoded at 24fps to prevent jitter. Display mode 52 may therefore sometimes be referred to as a movie mode, while mode 50 and other display modes may be referred to as non-movie modes. Upon detecting that one or more portions of display 14 are presenting movie content, device 10 may automatically switch from non-movie mode 50 to movie mode 52. Device 10 may also check whether the user is actually focused on the movie content. For example, consider a scenario in which a user opens a media playback window to play a movie. The media playback window may be minimized or tiled to one side of display 14. In such scenarios, the user may not be fully focused on the movie, but may be focused on other content on the display, so display 14 may operate in mode 50.

[0037] However, if the user expands the media playback window to full screen (e.g., to fill the entire display area), the user may be focused on the movie content, so the display 14 should be operated in mode 52. Therefore, the head mounted device 10 may automatically adjust the display 14 to switch from mode 50 to mode 52 (see transition 54) in response to detecting that the user is currently focused on media content that is better displayed using frame rate FR2 (e.g., where the media content matches the display frame rate to reduce jitter), and may automatically adjust the display 14 to switch from mode 52 to mode 50 (see transition 56) in response to detecting that the user is currently focused on media content that is better displayed using frame rate FR1. The device 10 may also automatically adjust the display 14 to switch from mode 52 to mode 50 when it is detected that the user is currently focused on media content that does not need to be displayed using frame rate FR2 (e.g., switching back to FR1 will not cause jitter), because operating in FR2 may consume more power than operating in FR1.

[0038] Figure 3 The examples showing two different display modes 50 and 52 are illustrative. In general, display 14 may operate in three or more display modes, each at a different frame rate, four or more display modes, each at a different frame rate, or any number of display modes with different frame rates that match the different types of content that may be presented on display 14.

[0039] The determination of whether to switch between different display modes may be based on a variety of factors. Figure 4 is a diagram showing how head mounted device 10 may include frame rate management circuitry such as frame rate manager 60 that is configured to adjust the frame rate of display 14 based on different types of information. Figure 4 As shown, frame rate manager 60 may receive a preferred frame rate from media system 64. A separate frame queue 66 may be used to store images, videos, or other content to be presented on display 14. In some cases, the preferred frame rate may be encoded directly in the content to be displayed. If desired, optional content management circuitry such as content manager 67 may be used to obtain the preferred frame rate from the content, or the preferred frame rate may be otherwise calculated / measured by analyzing the content as it is read out of frame queue 66. If the preferred frame rate is equal to or more appropriate for display using refresh rate FR1 (e.g., if FR1 is some multiple of the preferred frame rate, or if FR1 can be used to display the current content without causing jitter), frame rate manager 60 will output a command instructing a display controller such as display controller 62 to operate display 14 in first mode 50. On the other hand, if the preferred frame rate is equal to or more suitable for display using refresh rate FR2 (e.g., if FR2 is some multiple of the preferred frame rate, or if the current content can be displayed using FR2 without causing jitter), frame rate manager 60 will output a command instructing display controller 62 to operate the display in second mode 52. After playback of the movie is complete, media system 64 may notify frame rate manager 60, which may then instruct frame rate manager 60 to transmit a command to revert display 14 to the previous (lower) frame rate.

[0040] The frame rate manager 60 may also determine whether to adjust the frame rate based on the window size information. Generally speaking, a user may open one or more windows to view different types of content. For example, a movie player application in a movie playback window may be used to present a movie (movie content) on the display 14. As another example, a web browsing application in a web browser window may be used to present web content on the display 14. As another example, a game application in a game window may be used to present game content on the display 14. As another example, a text editing application in a text editor window may be used to present editable text on the display 14. As another example, a photo / video editing application in a media editor window may be used to present an image or video on the display 14. As another example, a teleconference application in a teleconference window may be used to present an image or video on the display 14. Windows are sometimes referred to as user interface (UI) elements, user interface windows, graphical user interface windows, panes, or window panes.

[0041] The window may have any size up to the active display area of ​​display 14. The size of a window is sometimes referred to as the window size. Figure 5 is a diagram illustrating a graphical user interface (GUI) window 82 displayed on the active display area 80. The size of the window 82 can be used as a proxy for determining whether the content within the window 82 is focused by the user.

[0042] For example, consider a scenario in which window 82 is a movie playback window presenting a movie encoded at 24 fps. If the size of window 82 is small relative to the total display area 80 (e.g., if window 82 occupies less than 60% of the total display area, less than 70% of the total display area, less than 80% of the total display area, less than 90% of the total display area, or less than 100% of the total display area), frame rate manager 60 may determine that the user is not fully focused on the movie content in window 82 and then instruct display controller 62 to operate the display in first mode 50 (e.g., a non-movie mode running in FR1 that may cause jitter when playing movies encoded at 24 fps). If the size of window 82 is large relative to the total display area 80 (e.g., if window 82 occupies more than 60% of the total display area, more than 70% of the total display area, more than 80% of the total display area, more than 90% of the total display area, or 100% of the total display area), frame rate manager 60 may determine that the user is not fully focused on the movie content in window 82 and then instruct display controller 62 to operate the display in first mode 50 (e.g., a non-movie mode running in FR1 that may cause jitter when playing movies encoded at 24 fps). Figure 6 As shown, where window 82' is expanded to full screen), the frame rate manager 60 may determine that the user is focused on the movie content in window 82 and then instruct the display controller 62 to operate the display in the second mode 52 (e.g., a movie mode running in FR2 that does not produce jitter when playing movies encoded at 24fps).

[0043] This example in which frame rate manager 60 decides whether to switch between modes 50 and 52 by considering the window size of the movie playback window is illustrative only. In general, frame rate manager 60 may consider the type of content within any open windows and the size of any open windows to determine what content the user is currently focused on and to determine the optimal display frame rate for presenting the content the user is currently focused on.

[0044] As another example, consider a scenario where three windows are simultaneously opened on the display area 80: a web browser window, a telephone conference window, and a movie playback window. The frame rate manager 60 may receive window size information for all of these open windows and determine the optimal display mode based on the largest window (e.g., the window that occupies the largest area on the display area 80). If the web browser window is the largest window, the frame rate manager 60 may instruct the controller 62 to operate the display using a frame rate that is most suitable for presenting web content. If the telephone conference window is the largest window, the frame rate manager 60 may instruct the controller 62 to operate the display using a frame rate that is most suitable for presenting images or video content shared during the meeting. If the movie playback window is the largest window, the frame rate manager 60 may instruct the controller 62 to operate the display using a frame rate that is most suitable for presenting movie (film) content. Other heuristics based on window size information may also be used if desired.

[0045] The frame rate manager 60 may also determine whether to adjust the frame rate based on the gaze information. As a specific example, the device 10 may include a gaze tracker 70 (sometimes referred to as a gaze tracking system, a gaze tracking sensor, etc.) configured to collect gaze information or gaze point information. The gaze tracker 70 may be one of the sensors 16 (see Figure 1 ). The gaze tracker 70 may include a camera and / or other gaze tracking system components (see, e.g. Figure 1The eye-facing components 42 and 44 in the gaze tracking system and / or other light sources that emit light beams so that reflections of light beams from the user's eyes can be detected) to monitor the user's eyes. One or more gaze tracking sensors can face the user's eyes and can track the user's gaze. The camera in the gaze tracking system can determine the position of the user's eyes (e.g., the center of the user's pupil), can determine the orientation direction of the user's eyes (the direction of the user's gaze), can determine the user's pupil size (e.g., so that light modulation and / or other optical parameters, and / or sequential amounts used to adjust one or more of these parameters spatially, and / or areas in which one or more of these optical parameters are adjusted, these areas are adjusted based on pupil size), can be used to monitor the current focus of the lens in the user's eyes (e.g., whether the user is focusing on the near field or the far field, which can be used to assess whether the user is daydreaming or thinking strategically or tactically) and / or other gaze information. The camera in the gaze tracker 70 may sometimes be referred to as an inward-facing camera, a gaze detection camera, an eye tracking camera, a gaze tracking camera, or an eye monitoring camera. If desired, other types of image sensors (eg, infrared and / or visible light emitting diodes and photodetectors, etc.) may also be used to monitor the user's gaze.

[0046] The user's gaze point collected using the gaze tracker 70 can be used to provide user input to the electronic device. For example, the user's gaze point can act as a cursor to select an area of ​​interest on the display 14. In an extended reality system where the display presents images close to the user's eyes (so touch input is not practical), the gaze point is a useful user input technique. The user's gaze point can also be used as an input to the frame rate manager 60 to determine whether to adjust the display frame rate.

[0047] Figure 782 is a diagram showing a window 82 occupying a portion of the total display area 80 and how the user's gaze point may be at various locations within the display area 80 or at various locations within a two-dimensional or three-dimensional environment / scene. For example, consider a scenario in which window 82 is a media playback window for displaying movie content encoded at 24fps. If gaze tracker 70 determines that the user's gaze point is at location 86 or 88 that falls outside of window 82 (i.e., if the user's gaze point does not overlap with window 82), frame rate manager 60 may determine that the user is not currently focused on the movie content in window 82 and instruct display controller 62 to select a frame rate suitable for displaying non-movie content (e.g., assuming the display is not already operating in non-movie mode 50 and switches to mode 50). If the gaze tracker 70 determines that the user's gaze point is at position 84 that falls within the window 82 (i.e., if the user's gaze point overlaps with the window 82), the frame rate manager 60 may determine that the user is currently focused on movie content and instruct the display controller 62 to select a frame rate that is most suitable for displaying the movie content (e.g., assuming that the display is not already operating in the movie mode 52 and switches to a mode 52 that prevents jitter when displaying movie content).

[0048] This example of the frame rate manager 60 deciding whether to adjust the display refresh rate by considering whether the user's gaze point falls within the movie playback window is merely illustrative. In general, the frame rate manager 60 may consider the type of content within any open window and whether the detected gaze point overlaps or intersects with a particular type of content or whether the user is focused on a particular location in a two-dimensional or three-dimensional environment to determine what content the user is currently focused on and determine the optimal display frame rate for presenting the content the user is currently focused on.

[0049] Frame rate manager 60 may also determine whether to adjust the frame rate based on information associated with the user's hand. For example, device 10 may include a hand tracker 72 (sometimes referred to as a hand tracking system, hand tracking sensor, gesture tracking system, gesture tracking sensor, etc.) configured to collect gesture information. Hand tracker 72 may be one of sensors 16 (see Figure 1). The hand tracker 72 may include a camera and / or other gesture tracking system components (e.g., an outward-facing component and / or a light source that emits a light beam so that reflections of the light beam from the user's hand can be detected) to monitor the user's hand. One or more hand tracking sensors may be oriented toward the user's hand and may track motion associated with the user's hand, may determine whether the user is performing a swiping motion with his / her hand, may determine whether the user is performing a contactless button press or object selection operation with his / her hand, may determine whether the user is performing a grasping or gripping motion with his / her hand, may determine whether the user is using his / her hand or fingers to point to a given object presented on the display 14, may determine whether the user is performing a waving or striking motion with his / her hand, or may generally measure / monitor three-dimensional contactless gestures ("air gestures") associated with the user's hand.

[0050] The gesture information collected using the hand tracker 72 can be used to provide user input to the electronic device. For example, the user's hand or finger can act as a cursor to select an area of ​​interest on the display 14. In an extended reality system with a display that presents images close to the user's eyes (and direct contact touch input is therefore not practical), non-contact air gesture information is a useful user input technology. If desired, in the case where the user holds a controller to control the operation of the device 10, the hand tracker 72 can also track the movement of such a controller.

[0051] The user's gesture information may also be used as an input to the frame rate manager 60 to determine whether to adjust the display frame rate. For example, consider a scenario in which a user opens a media playback window to play movie content in the window on the display 14 but continues to perform gestures as detected by the hand tracker 72. The continued presence of the gesture may be used as a proxy for indicating that the user is not yet ready to focus on the movie content. In this scenario, the frame rate manager 60 may delay switching to the movie mode 52 until the user stops performing the gesture for some predetermined period of time. For example, the frame rate manager 60 may adjust the frame rate to FR2 to prevent jitter after determining that the user has stopped performing the gesture for at least 3 seconds, at least 5 seconds, 3 to 10 seconds, at least 10 seconds, at least 20 seconds, or other durations.

[0052] This example of frame rate manager 60 deciding whether to adjust the display refresh rate by considering the user's hand motion when a movie playback window is open is illustrative only. In general, frame rate manager 60 may consider the type of content within any open window and whether the user's hand is actively performing hand input gestures when the window is open to determine whether the user is ready to focus on any particular content and to determine the optimal display frame rate for presenting the content that the user is currently focused on.

[0053] The frame rate manager 60 may also determine whether to adjust the frame rate based on information associated with the user's head. Figure 1 As mentioned, the head mounted device 10 may include one or more outward facing cameras 46 and one or more motion sensors (e.g., gyroscopes, accelerometers, and other inertial measurement units as part of the sensors 16) that are used to form a visual inertial odometry (VIO) system for tracking the orientation and position of the device when the device 10 is worn on the user's head. These position and motion sensors may assume that the head mounted device 10 is mounted on the user's head. Therefore, in this document, references to head posture, head movement, yaw of the user's head (e.g., rotation around a vertical axis), pitch of the user's head (e.g., rotation around a left-right axis), roll of the user's head (e.g., rotation around a front-to-back axis), etc. may be considered interchangeable with references to device posture, device movement, yaw of the device, pitch of the device, roll of the device, etc. These components for tracking the orientation and / or position of the user's head relative to the surrounding environment may be collectively referred to as a head tracker 74, a head tracking system, a head tracking sensor, etc. The head tracker 74 may directly determine the posture, movement, yaw, pitch, roll, etc. of the head mounted device 10. The yaw, roll and pitch of the user's head can collectively define the user's head posture. The detected changes in head posture can be used as user input to the head mounted device 10.

[0054] The user's head pose information may also be used as an input to the frame rate manager 60 to determine whether to adjust the display frame rate. For example, consider a scenario in which a user opens a media playback window to play movie content in the window on the display 14 but continues to move his head back and forth as detected by the head tracker 74. The continuous change in head pose may be used as a proxy for indicating that the user is not yet ready to focus on the movie content. In this scenario, the frame rate manager 60 may delay switching to the movie mode 52 until the user stops moving his head for a predetermined period of time. For example, the frame rate manager 60 may adjust the frame rate to FR2 to prevent jitter after determining that the user has stopped changing head pose for at least 3 seconds, at least 5 seconds, 3 to 10 seconds, at least 10 seconds, at least 20 seconds, or other durations.

[0055] This example of frame rate manager 60 deciding whether to adjust the display refresh rate by considering the user's head posture when a movie playback window is open is illustrative only. In general, frame rate manager 60 may consider the type of content within any open window and whether the user's head is relaxed (static) or continuously moving when the window is open to determine whether the user is ready to focus on any particular content and to determine the optimal display frame rate for presenting the content that the user is currently focused on.

[0056] The examples of using gaze tracker 70 to monitor the user's gaze point (to obtain gaze point data), using hand tracker 72 to monitor the user's hand movements / gestures (to obtain gesture data), and using head tracker 74 to monitor the user's head posture (to obtain head posture data) are illustrative and are not intended to limit the scope of the embodiments of the present invention. If desired, device 10 may include other trackers or sensors for collecting information / data about the user's elbows, hips, feet, knees, toes and / or other body parts. In general, any information related to one or more user body parts may be used as input to frame rate manager 60 to determine whether to adjust the frame rate of display 14 (e.g., to ensure that the display is being operated using a frame rate that best suits or best matches the specific content that the user is currently focusing on or is about to focus on).

[0057] The frame rate manager 60 may also determine whether to adjust the frame rate based on audio information collected using a microphone. The device 10 may include an input device, such as one or more microphones 78 configured to collect audio information from a user. The microphone 78 may detect whether the user is currently speaking. In some cases, such as during an audio-only conference call, the detection of the user speaking may be used as an input to the frame rate manager 60 to determine whether to adjust the frame rate of the display 14. For example, the detection of the user speaking may be used as a proxy to indicate that the user is no longer focused on the movie content, and in response, the frame rate manager 60 may reduce the display frame rate to save power.

[0058] Frame rate manager 60 may also determine whether to adjust the frame rate based on information associated with the state of one or more software or hardware components in device 10. Device 10 may include system monitoring circuitry, such as system monitor 76 configured to monitor one or more system parameters associated with the operation of device 10. For example, system monitor 76 may monitor current system capabilities to determine whether the system can handle adjustment of the display frame rate. If system memory is overloaded, if too many applications are running on the system simultaneously, or if the system is oversubscribed, frame rate manager 60 may decide to delay or otherwise prevent adjustment of display 14 to a higher frame rate, even though doing so may risk presenting jitter to the user.

[0059] As another example, system monitor 76 may monitor system thermal information to determine whether the system can handle adjusting to a higher display frame rate. Device 10 may include one or more thermal (temperature) sensors located near the processor, display, and other control circuits within device 10 to determine whether any of the hardware components are overheating. If the temperature sensor indicates that one or more hardware components are excessively overheated (e.g., if the measured temperature level is greater than a predetermined thermal threshold), frame rate manager 60 may decide to delay or otherwise prevent the display 14 from adjusting to a higher frame rate, even though doing so may risk presenting jitter to the user. If the measured temperature level is less than the predetermined thermal threshold, frame rate manager 60 may be allowed to adjust the display refresh rate if doing so will help match the frame rate to the type of content that the user is currently focused on (e.g., to prevent jitter by switching display modes).

[0060] As another example, system monitor 76 may monitor the battery charge of device 10 to determine whether the system can handle adjusting to a higher frame rate. Device 10 may include a power management unit that monitors the current battery charge of device 10 (the power management unit may be used as a Figure 4 A portion of the system monitor 76 in Figure 2 The battery charge of device 10 is sometimes referred to as the charge state. If the measured battery charge of device 10 is less than a given battery charge threshold, frame rate manager 60 may decide to delay or otherwise prevent adjustment of display 14 to a higher frame rate, even though doing so may risk presenting judder to the user. If the measured battery charge is less than a given battery charge threshold, frame rate manager 60 may be permitted to adjust the display refresh rate if doing so will help match the frame rate to the type of content that the user is currently focusing on (e.g., to prevent judder by switching display modes).

[0061] As another example, frame rate manager 60 may also take into account the current power setting of the system. Device 10 may operate in a low power mode or power saving mode when indicated by the user or when the battery charge drops below a given threshold. If device 10 is operating in a low power mode or power saving mode, frame rate manager 60 may decide to delay or otherwise prevent display 14 from adjusting to a higher frame rate, even though doing so may risk presenting jitter to the user. If device 10 is not operating in a low power mode or power saving mode, frame rate manager 60 may be allowed to adjust the display refresh rate if doing so will help match the frame rate to the type of content that the user is currently focusing on (e.g., to prevent jitter by switching display modes).

[0062] Frame rate manager 60 controls the frame rate of display 14 based on one or more inputs. Figure 4The examples are illustrative only and are not intended to limit the scope of embodiments of the present invention, and one or more inputs such as a preferred frame rate obtained using media system 64 (and optional content manager 66), window size information obtained using window manager 68, gaze point information obtained using gaze tracking system 70, gesture information obtained using hand tracking system 72, head posture information obtained using head tracking system 74, and system parameters including system capabilities, thermal information, battery charge information, and power mode information obtained using system monitor 76. Frame rate manager 60 may consider other system parameters or inputs from additional sensors or hardware components when making decisions about whether to adjust the frame rate of display 14.

[0063] In some embodiments, when making a decision whether to adjust the display frame rate, frame rate manager 60 may take into account Figure 4 Two or more of the inputs shown. For example, in response to determining that the preferred frame rate is a multiple of 24 and the window size of the movie playback window is currently extended to the full screen, the frame rate manager 60 may instruct the controller 62 to switch to the movie mode 52. For another example, in response to determining that the preferred frame rate is a multiple of 24, the window size of the movie playback window is currently greater than 60% of the total display area, and the user's gaze point is within the movie playback window for a predetermined duration, the frame rate manager 60 may instruct the controller 62 to switch to the movie mode 52. For another example, in response to determining that the preferred frame rate is a multiple of 24, the window size of the movie playback window is currently greater than 70% of the total display area, the user's gaze point is within the movie playback window for a predetermined duration, and the user's hands have been idle for a given period of time, the frame rate manager 60 may instruct the controller 62 to switch to the movie mode 52.

[0064] As another example, in response to determining that the preferred frame rate is a multiple of 24, the window size of the movie playback window is currently greater than 70% of the total display area, the user's gaze point has been within the movie playback window for a predetermined duration, and the user's head has been idle for a given period of time, the frame rate manager 60 may instruct the controller 62 to switch to the movie mode 52. As another example, in response to determining that the preferred frame rate is a multiple of 24, the window size of the movie playback window is currently greater than 80% of the total display area, the user's gaze point has been within the movie playback window for a predetermined duration, and the system is able to handle the required frame rate (e.g., to ensure that the system is not overloaded or oversubscribed, to ensure that the system is not overheated, to ensure that the battery power is above a battery power threshold, to ensure that the device is not in a low power mode or power saving mode, etc.), the frame rate manager 60 may instruct the controller 62 to switch to the movie mode 52.

[0065] Figure 8 It is used to operate the combination Figures 1 to 7Flowchart of illustrative steps of the head mounted electronic device 10 of the type described. During the operation of block 90, the frame rate manager 60 may receive information from various sources. Figure 4 As shown in the example of , the frame rate manager 60 may receive a preferred frame rate from the media system (e.g., the preferred frame rate may be encoded in the content itself or may be calculated / measured using a separate content manager or may be declared or presented as part of the content description in a media playlist such as an HLS (HTTP Live Streaming) playlist), window size information from a window manager (e.g., to determine whether the movie playback window is currently extended to the full screen or occupies a certain percentage of the full screen), gaze point information from a gaze tracker (e.g., to determine whether the user's gaze point intersects or overlaps the movie playback window), gesture information from a hand tracker (e.g., to determine whether the user is actively engaging with the device 10 or is idle and, therefore, ready to focus on the movie content), head posture information from a head tracker (e.g., to determine whether the user is actively moving around or is idle and, therefore, ready to focus on the movie content), and system information from a system monitor (e.g., to determine whether the operating system is currently oversubscribed, whether system memory is near capacity, whether a hardware component on the device 10 is overheating, whether the battery on the device 10 is low, whether the device 10 is currently in a low power mode or power saving mode, etc.).

[0066] During operation at block 92, frame rate manager 60 may determine whether to adjust the display frame (refresh) rate based on at least some of the information received during operation at block 90. ​​Frame rate manager 60 may decide whether to switch display modes based on only one of the inputs received at block 90, based on two of the inputs received at block 90, based on three of the inputs received at block 90, based on four of the inputs received at block 90, based on five of the inputs received at block 90, or based on all of the inputs received at block 90. ​​For example, based on determining that the preferred frame rate is a multiple of 24 and that the window size of the movie playback window is currently extended to full screen, frame rate manager 60 may instruct controller 62 to switch to movie mode 52. For example, based on determining that the preferred frame rate is a multiple of 24, the window size of the movie playback window is currently larger than 60% of the total display area, and the user's gaze point remains within the movie playback window for a predetermined duration (for example, by checking whether the user's gaze point remains within the movie playback window for at least 2 seconds, 3 seconds, 4 seconds, 5 seconds, 5 seconds to 10 seconds, greater than 10 seconds, etc.), the frame rate manager 60 may instruct the controller 62 to switch to movie mode 52.

[0067] During the operation of block 94, frame rate manager 60 may then instruct display controller 62 to adjust the display frame rate to a frame rate that best matches or complements the type of content that the user is currently focused on. For example, if frame rate manager 60 determines that the user is currently focused on movie content (as determined using the operations of blocks 90 and 92), frame rate manager 60 may transmit a command instructing display 14 to operate in movie mode 52. As another example, if frame rate manager 60 determines that the user is no longer currently focused on movie content (as determined using the operations of blocks 90 and 92), frame rate manager 60 may transmit a command instructing display 14 to operate in non-movie mode 50. As another example, if frame rate manager 60 determines that the user is currently focused on a particular type of content other than movie content (as determined using the operations of blocks 90 and 92), frame rate manager 60 may transmit a command instructing display 14 to operate in yet another display mode having a frame rate that best matches or complements the particular type of content that the user is currently focused on while minimizing jitter.

[0068] Figure 8 The operations may be performed periodically in response to user input (e.g., in response to the user opening a window, launching an application, playing a video, stopping a video, making a gesture, moving their head, adjusting system power settings, etc.), or may be performed continuously as a background process while device 10 is powered on.

[0069] A physical environment refers to the physical world that people can sense and / or interact with without the help of electronic devices. In contrast, an extended reality (XR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic devices. For example, an XR environment may include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, etc. In the case of an XR system, a subset of a person's physical movements, or a representation thereof, is tracked, and in response, one or more characteristics of one or more virtual objects simulated in the XR environment are adjusted in a manner that complies with at least one law of physics.

[0070] There are many different types of electronic systems that can enable a person to sense and / or interact with various XR environments. Examples include head-mountable systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed over a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smartphones, tablet computers, and desktop / laptop computers.

[0071] According to one embodiment, a method for operating an electronic device is provided, the method comprising: presenting content in a graphical user interface window using a display; determining whether to adjust the frame rate of the display based on the type of content being presented and based on the size of the graphical user interface window in which the content is being presented; and in response to determining that the frame rate of the display will be adjusted, adjusting the frame rate of the display from a first frame rate to a second frame rate different from the first frame rate.

[0072] According to another embodiment, determining whether to adjust the frame rate of the display includes determining whether to adjust the frame rate based on a preferred frame rate associated with the type of content being presented within the graphical user interface window.

[0073] According to another embodiment, the method includes storing the content in a frame queue, preferably the frame rate is encoded as part of the content stored in the frame queue.

[0074] According to another embodiment, the method includes presenting the preferred frame rate as part of the content description in the media playlist.

[0075] According to another embodiment, the method includes storing content in a frame queue and calculating a preferred frame rate based on the stored content in the frame queue.

[0076] According to another embodiment, determining whether to adjust the frame rate of the display based on the size of the graphical user interface window includes determining an amount of the total viewing area of ​​the display that the graphical user interface window occupies.

[0077] According to another embodiment, the method includes adjusting a frame rate of the display from a first frame rate to a second frame rate in response to determining that the graphical user interface window occupies at least 60% of a total viewing area of ​​the display.

[0078] According to another embodiment, the method includes adjusting a frame rate of the display from a first frame rate to a second frame rate in response to determining that the graphical user interface window expands to fill a total viewing area of ​​the display.

[0079] According to another embodiment, the first frame rate is a multiple of 30 and the second frame rate is a multiple of 24.

[0080] According to another embodiment, the method includes determining whether to adjust the frame rate of the display based on additional information selected from gaze point information, gesture information, head posture information, system capability information, thermal information, battery power information, power mode information and audio information.

[0081] According to one embodiment, a method for operating an electronic device is provided, the method comprising: displaying corresponding content in multiple areas on a display; determining whether a user is focused on any one of the multiple areas on the display; and in response to determining that the user is focused on a given area of ​​the multiple areas on the display, adjusting the frame rate of the display based on the content in the given area on which the user is focused.

[0082] According to another embodiment, determining whether the user is focused on any of a plurality of areas on the display includes obtaining a gaze point.

[0083] According to another embodiment, the method includes adjusting a frame rate of a display in response to determining that the gaze point is within a given area.

[0084] According to another embodiment, the method includes adjusting a frame rate of a display in response to determining that a gaze point is within a given area for a duration exceeding a timing threshold.

[0085] According to another embodiment, determining whether the user is focused on any one of a plurality of areas on the display includes obtaining gesture information or head posture information.

[0086] According to another embodiment, the method includes displaying a three-dimensional environment in at least one area of ​​a plurality of areas, and determining whether a user is focused on a given area includes determining a focus position of the user in the three-dimensional environment.

[0087] According to another embodiment, the method includes determining whether to adjust the frame rate of the display based on additional information selected from a preferred frame rate associated with the content, system capability information, thermal information, battery charge information, power mode information, and audio information.

[0088] According to one embodiment, a method for operating an electronic device is provided, the method comprising: presenting content in a graphical user interface window using a display; determining whether to adjust the frame rate of the display based on the type of content being presented and based on information selected from system capabilities, thermal information, battery charge information, power mode information, and audio information; and in response to determining that the frame rate of the display will be adjusted, adjusting the frame rate of the display from a first frame rate to a second frame rate different from the first frame rate.

[0089] According to another embodiment, the method includes determining whether to adjust a frame rate of the display based on a preferred frame rate associated with content being presented on the display.

[0090] According to another embodiment, the method includes determining whether to adjust the frame rate of the display based on gaze data, gesture data, or head posture data.

[0091] The foregoing is merely illustrative and various modifications may be made to the embodiments described. The foregoing embodiments may be implemented individually or in any combination.

Claims

1. A method for operating an electronic device, the method comprising: using a display to present content in a graphical user interface window; determining whether to adjust a frame rate of the display based on a type of the content being presented and based on a size of the graphical user interface window in which the content is being presented; as well as In response to determining that the frame rate of the display is to be adjusted, the frame rate of the display is adjusted from a first frame rate to a second frame rate different from the first frame rate.

2. The method of claim 1 , wherein determining whether to adjust the frame rate of the display comprises determining whether to adjust the frame rate based on a preferred frame rate associated with the type of content being presented within the graphical user interface window.

3. The method according to claim 2, further comprising: The content is stored in a frame queue, the preferred frame rate being encoded as a portion of the content stored in the frame queue.

4. The method according to claim 2, further comprising: The preferred frame rate is presented as part of the content description in the media playlist.

5. The method according to claim 2, further comprising: storing the content in a frame queue; as well as The preferred frame rate is calculated based on the stored contents in the frame queue.

6. The method of claim 1, wherein determining whether to adjust the frame rate of the display based on the size of the graphical user interface window comprises determining an amount of a total viewing area of ​​the display that the graphical user interface window occupies.

7. The method according to claim 6, further comprising: The frame rate of the display is adjusted from the first frame rate to the second frame rate in response to determining that the graphical user interface window occupies at least 60% of the total viewing area of ​​the display.

8. The method according to claim 6, further comprising: The frame rate of the display is adjusted from the first frame rate to the second frame rate in response to determining that the graphical user interface window expands to fill the total viewing area of ​​the display.

9. The method of claim 1, wherein the first frame rate is a multiple of 30 and the second frame rate is a multiple of 24.

10. The method according to claim 1, further comprising: Determining whether to adjust the frame rate of the display is based on additional information selected from gaze point information, gesture information, head posture information, system capability information, thermal information, battery power information, power mode information and audio information.

11. A method of operating an electronic device, the method comprising: displaying the corresponding content in a plurality of areas on the display; determining whether a user is focused on any one of the plurality of areas on the display; as well as In response to determining that the user is focused on a given area of ​​the plurality of areas on the display, a frame rate of the display is adjusted based on the content in the given area on which the user is focused. 12 . The method of claim 11 , wherein determining whether the user is focused on any one of the plurality of areas on the display comprises obtaining a gaze point.

13. The method according to claim 12, further comprising: The frame rate of the display is adjusted in response to determining that the gaze point is within the given area.

14. The method according to claim 12, further comprising: The frame rate of the display is adjusted in response to determining that the gaze point is within the given area for a duration exceeding a timing threshold. 15 . The method of claim 11 , wherein determining whether the user is focused on any one of the plurality of regions on the display comprises obtaining hand gesture information or head posture information.

16. The method according to claim 11, further comprising: A three-dimensional environment is displayed in at least one area of ​​the plurality of areas, wherein determining whether the user is focused on the given area comprises determining a focus position of the user in the three-dimensional environment.

17. The method according to claim 11, further comprising: A determination is made whether to adjust the frame rate of the display based on additional information selected from a preferred frame rate associated with the content, system capability information, thermal information, battery charge information, power mode information, and audio information.

18. A method of operating an electronic device, the method comprising: using a display to present content in a graphical user interface window; determining whether to adjust a frame rate of the display based on a type of the content being presented and based on information selected from the group consisting of system capabilities, thermal information, battery charge information, power mode information, and audio information; as well as In response to determining that the frame rate of the display is to be adjusted, the frame rate of the display is adjusted from a first frame rate to a second frame rate different from the first frame rate.

19. The method according to claim 18, further comprising: A determination is made whether to adjust the frame rate of the display based on a preferred frame rate associated with the content being presented on the display.

20. The method according to claim 18, further comprising: A determination is made whether to adjust the frame rate of the display based on gaze data, gesture data, or head pose data.

Images