Display apparatus, method of display, and computer-program product
The display apparatus addresses the challenge of balancing high image resolution and power consumption in virtual reality head-mount displays by dynamically adjusting display settings based on scene complexity and system performance, optimizing power usage and extending battery life.
Patent Information
- Application Number
- PCT/CN2023/120921
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-09-25
- Publication Date
- 2025-04-03
AI Technical Summary
Virtual reality head-mount display apparatuses face challenges in balancing high image display resolution with power consumption and battery life, leading to increased device size and weight.
A display apparatus with a memory and one or more processors that recognizes image display scene information, performs scene complexity detection, and adjusts the display resolution and frame rate based on scene complexity and system performance to optimize power usage.
The solution dynamically adjusts display settings to maintain a balance between image quality and power consumption, reducing system load and extending battery life while maintaining a lightweight and compact form factor.
Smart Images

Figure CN2023120921_03042025_PF_FP_ABST
Abstract
Description
DISPLAY APPARATUS, METHOD OF DISPLAY, AND COMPUTER-PROGRAM PRODUCTTECHNICAL FIELD
[0001] The present invention relates to display technology, more particularly, to a display apparatus, a method of display, and a computer-program product.BACKGROUND
[0002] Virtual reality and augmented reality are two computer technologies for viewing synthetic or partially synthetic environments. Virtual reality typically involves computer generated representations of various real-world or fictitious environments. Augmented reality typically involves various types of computer-assisted representations of the real world.SUMMARY
[0003] In one aspect, the present disclosure provides a display apparatus, comprising a memory; and one or more processors; wherein the memory and the one or more processors are connected with each other; and the memory stores computer-executable instructions for controlling the one or more processors to recognize an image display scene information; perform a scene complexity detection based on the image display scene information; group a scene complexity of an image display scene into multiple levels based on a type of the image display scene; render an image at a first resolution range upon determination that the scene complexity is at a first level; and render an image at a second resolution range upon determination that the scene complexity is at a second level; wherein the second resolution range is lower than the first resolution range; and the scene complexity at the second level is higher than the scene complexity at the first level.
[0004] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to render a first sub-image and a second sub-image; wherein the first sub-image is in a region intersecting with a gaze direction of a user; the second sub-image is in a region non-intersecting with the gaze direction; the first sub-image has a resolution higher than a resolution of the second sub-image; and the first sub-image is rendered at the first resolution range upon determination that the scene complexity is at a first level and at the second resolution range upon determination that the scene complexity is at the second level.
[0005] Optionally, when the image is rendered at the first resolution range, the memory further stores computer-executable instructions for controlling the one or more processors to monitor in real time rendering frame rate to determine if a current system performance meets rendering requirements; and upon determination that the current system performance fails to meet rendering requirements, render an image at the second resolution range.
[0006] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to render an image at a third resolution range upon determination that the scene complexity is at a third level; wherein the third resolution range is lower than the second resolution range; and the scene complexity at the third level is higher than the scene complexity at the second level.
[0007] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to control a frame rate above a threshold value whether the image is rendered at the first resolution range, the second resolution range, or the third resolution range.
[0008] Optionally, when the image is rendered at the second resolution range, the memory further stores computer-executable instructions for controlling the one or more processors to monitor in real time rendering frame rate to determine if a current system performance meets rendering requirements; and upon determination that the current system performance fails to meet rendering requirements, render an image at a third resolution range; wherein the third resolution range is lower than the second resolution range.
[0009] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to start one or more applications; initialize image display scene information; and recognize the image display scene information.
[0010] Optionally, the display apparatus further comprises a gaze point sensor configured to detect a gaze of a user; wherein the memory further stores computer-executable instructions for controlling the one or more processors to turn off the gaze point sensor upon determination that an image display scene is a first scene.
[0011] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to monitor a state of charge of a power supply of the display apparatus.
[0012] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to render an image at a fourth resolution range upon determination that the state of charge of the power supply is greater than a threshold state of charge; and render an image at a fifth resolution range upon determination that the state of charge of the power supply is equal to or less than the threshold state of charge; wherein the fourth resolution range is greater than the fifth resolution range.
[0013] Optionally, the display apparatus further comprises an acceleration sensor configured to detect a head movement of a user; wherein the memory further stores computer-executable instructions for controlling the one or more processors to turn on an acceleration sensor to monitor head movement of the user upon determination that the image display scene is a second scene.
[0014] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to turn off a gaze point sensor and render an image at a sixth resolution range, upon determination that a speed of the head movement is greater than a first threshold speed; and turn on the gaze point sensor; and monitor a speed of eye movement, upon determination that a speed of the head movement is equal to or less than the first threshold speed.
[0015] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to render an image at a seventh resolution range and control the gaze point sensor to monitor a speed of the eye movement in a first-speed exposure mode, upon determination that the speed of the eye movement is greater than a second threshold speed; and control the gaze point sensor to monitor a speed of the eye movement in a second-speed exposure mode, upon determination that the speed of the eye movement is equal to or less than a second threshold speed; wherein, in the first-speed exposure mode, the gaze point sensor is configured to monitor the speed of the eye movement at a first frequency; in the second-speed exposure mode, the gaze point sensor is configured to monitor the speed of the eye movement at a second frequency; and the second frequency is greater than the first frequency.
[0016] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to perform the scene complexity detection, upon determination that the speed of the eye movement is equal to or less than a second threshold speed.
[0017] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to generate a synchronization signal to synchronize operations of applications, fundamental-level components, input hardware, and output hardware of the display apparatus, wherein the fundamental-level components comprise a fundamental-level interface module, a display driving module configured to drive a display panel; a sensor driving module configured to drive a camera sensor; and a light emitting element driving module configured to drive a fill light driving circuit; transmit the synchronization signal to a synchronization signal distribution thread; and distribute, by the synchronization signal distribution thread, the synchronization signal to one or more applications.
[0018] Optionally, the memory further stores computer-executable instructions for controlling the one or more processors to transmit, by the one or more applications, the synchronization signal to a fundamental-level interface module; and transmit, by the fundamental-level interface module, the synchronization signal to fundamental-level component driving modules of fundamental-level components.
[0019] Optionally, the display panel further comprises the camera sensor, the fill light driving circuit, and the display panel; wherein the memory further stores computer-executable instructions for controlling the one or more processors to distribute, by the synchronization signal distribution thread, the synchronization signal to a sensor synchronization control thread, a light emitting element synchronization control thread, and a display panel synchronization control thread; transmit, by the sensor synchronization control thread, the light emitting element synchronization control thread, and the display panel synchronization control thread, the synchronization signal to a fundamental-level interface module; and transmit, by the fundamental-level interface module, the synchronization signal to a display driving module configured to drive the display panel; a sensor driving module configured to drive the camera sensor; and a light emitting element driving module configured to drive the fill light driving circuit.
[0020] Optionally, the display apparatus comprises a clock management module, an image rendering module configured to render an image, and a synchronization signal distribution thread; wherein, upon the image being rendered, the image rendering module is configured to transmit a signal to a clock management module; and upon receiving the signal, the clock management module is configured to generate a synchronization signal, and configured to transmit the synchronization signal to the synchronization signal distribution thread.
[0021] Optionally, the display apparatus further comprises a fill light driving circuit and a variable frequency drive module configured to modulate a pulse width modulation timing of the fill light driving circuit; wherein the variable frequency drive module is configured to modulate pulses for the fill light driving circuit to have a first pulse frequency in a first driving mode and a second pulse frequency in a second driving mode; the second pulse frequency is greater than the first pulse frequency; the first driving mode corresponds to a first-speed exposure mode of a gaze point sensor configured to monitor a speed of the eye movement; the second driving mode corresponds to a second-speed exposure mode of the gaze point sensor; and in the first-speed exposure mode, the gaze point sensor is configured to monitor the speed of the eye movement at a first frequency; in the second-speed exposure mode, the gaze point sensor is configured to monitor the speed of the eye movement at a second frequency; and the second frequency is greater than the first frequency.
[0022] In another aspect, the present disclosure provides a method of display, comprising recognizing an image display scene information; performing a scene complexity detection based on the image display scene information; grouping a scene complexity of an image display scene into multiple levels based on a type of the image display scene; rendering an image at a first resolution range upon determination that the scene complexity is at a first level; and rendering an image at a second resolution range upon determination that the scene complexity is at a second level; wherein the second resolution range is lower than the first resolution range; and the scene complexity at the second level is higher than the scene complexity at the first level.
[0023] In another aspect, the present disclosure provides a computer-program product, comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon, the computer-readable instructions being executable by a processor to cause the processor to perform recognizing an image display scene information; performing a scene complexity detection based on the image display scene information; grouping a scene complexity of an image display scene into multiple levels based on a type of the image display scene; rendering an image at a first resolution range upon determination that the scene complexity is at a first level; and rendering an image at a second resolution range upon determination that the scene complexity is at a second level; wherein the second resolution range is lower than the first resolution range; and the scene complexity at the second level is higher than the scene complexity at the first level.
[0024] BRIEF DESCRIPTION OF THE FIGURES
[0025] The following drawings are merely examples for illustrative purposes according to various disclosed embodiments and are not intended to limit the scope of the present invention.
[0026] FIG. 1 is a schematic diagram illustrating the structure of a display apparatus in some embodiments according to the present disclosure.
[0027] FIG. 2 is a schematic diagram illustrating the structure of a control module of a display apparatus in some embodiments according to the present disclosure.
[0028] FIG. 3 is a schematic diagram illustrating functionalities of a control module of a display apparatus in some embodiments according to the present disclosure.
[0029] FIG. 4 is a schematic diagram illustrating functionalities of a control module of a display apparatus in some embodiments according to the present disclosure.
[0030] FIG. 5 illustrates interaction among applications, fundamental-level components, input hardware, and output hardware in a display apparatus in some embodiments according to the present disclosure.
[0031] FIG. 6 illustrates synchronized operation by a synchronization signal in a display apparatus in some embodiments according to the present disclosure.
[0032] FIG. 7 illustrates a process of information output in a display apparatus in some embodiments according to the present disclosure.
[0033] FIG. 8 illustrates a process of determining a first-speed exposure mode or a second-speed exposure mode in some embodiments according to the present disclosure.
[0034] FIG. 9 is a schematic diagram illustrating a display apparatus in some embodiments according to the present disclosure.
[0035] FIG. 10 is a flow chart illustrating a method of display in some embodiments according to the present disclosure.
[0036] FIG. 11 is a flow chart illustrating a method of display in some embodiments according to the present disclosure.DETAILED DESCRIPTION
[0037] The disclosure will now be described more specifically with reference to the following embodiments. It is to be noted that the following descriptions of some embodiments are presented herein for purpose of illustration and description only. It is not intended to be exhaustive or to be limited to the precise form disclosed.
[0038] Related virtual reality head-mount display (HMD) apparatuses typically integrate all its components into a headgear. An increased image display resolution in the related virtual reality head-mount display apparatuses enhances the virtual reality experience but it also increases the overall system load and power consumption. In virtual reality display, power consumption has always been a significant concern, and the enhancement of display quality with 4k or 8k resolution exacerbates this issue, further impacting the already limited battery life of these products. Approaches such as increasing battery capacity or adding plug-in power modules often result in increased device size and weight, which is counterproductive to the goal of making VR products more lightweight and compact.
[0039] Accordingly, the present disclosure provides, inter alia, a display apparatus, a method of display, and a computer-program product that substantially obviate one or more of the problems due to limitations and disadvantages of the related art. In one aspect, the present disclosure provides a display apparatus. In some embodiments, the display apparatus includes a memory; and one or more processors. The memory and the one or more processors are connected with each other. The memory stores computer-executable instructions for controlling the one or more processors to recognize an image display scene information; perform a scene complexity detection based on the image display scene information; group a scene complexity of an image display scene into multiple levels based on a type of the image display scene; render an image at a first resolution range upon determination that the scene complexity is at a first level; and render an image at a second resolution range upon determination that the scene complexity is at a second level. Optionally, the second resolution range is lower than the first resolution range. Optionally, the scene complexity at the second level is higher than the scene complexity at the first level.
[0040] FIG. 1 is a schematic diagram illustrating the structure of a display apparatus in some embodiments according to the present disclosure. Referring to FIG. 1, the display apparatus in some embodiments includes a control module, a power supply module, and a display module. In the present display apparatus, the components of the display apparatus are configured to be worn by different human body parts, including head and neck. The display module is configured to be worn by a user’s head for viewing an image. The power supply module and the control module are configured to be worn by the user’s neck. The structure of the display apparatus results in a reduced weight on the head. In some embodiments, the display module includes a display panel and power supply circuits, an acceleration sensor and its peripheral circuits, camera and its peripheral circuits. In some embodiments, the power supply module includes a battery, a charger and its peripheral circuits.
[0041] FIG. 2 is a schematic diagram illustrating the structure of a control module of a display apparatus in some embodiments according to the present disclosure. Referring to FIG. 2, the control module in some embodiments includes one or more processors and its peripheral circuits, interfaces and peripheral circuits for connecting the display module, control circuits and interfaces for peripherals (such as WiFi / Bluetooth, camera, buttons, touch controls, fans, USB, audio, etc. ) . For example, the control module includes a USB2.0 interface, a USB3.0 interface, a Type-C interface, a power interface, one or more interfaces (e.g., Gpio and I2C) for touch control, an I2C interface for a P sensor, an I2S interface for Audio IC, which is connected to an audio system (e.g., a microphone MIC and a speaker SPK) . The control module may further include a PCIE2.0 SDIO interface for a wireless network integrated circuit WIFI IC and a Bluetooth integrated circuit BT IC, an eMMC interface for an embedded MultiMediaCard, a DDR interface for a random-access memory, a MIPI CSI interface for a camera, interfaces MIPI DS, MIPI DSI for a display panel, and a Serial Peripheral Interface SPI.
[0042] In some embodiments, the present display apparatus is configured to provide an intelligent modulation mechanism, which takes into account both user demands and usage scenario requirements, encompassing aspects such as human vision, hand movements, head movements, and different activities like movie watching, gaming, and teaching.
[0043] In some embodiments, the control module (e.g., one or more processors in the control module) is configured to adjust (e.g., automatically adjust) the display resolution based on requirements to balance image quality and performance (display resolution modulation) . In some embodiments, the control module is configured to adjust a frame rate of the display apparatus as needed to ensure a balance between smoothness and performance (frame rate modulation) . In some embodiments, the control module is configured to adjust the display resolution and / or the frame rate depending on the display apparatus’ real-time performance, including CPU and GPU performance (system performance modulation) . When the system's performance can fully meet the current resolution and frame rate requirements, the resolution and / or the frame rate may be adjusted (e.g., increased) to achieve a better user experience. Conversely, when the system's performance falls short of requirements, the display resolution may be reduced to maintain the frame rate. The display resolution modulation, the frame rate modulation, and the system performance modulation are performed cooperatively with respect to each other.
[0044] In some embodiments, the display resolution modulation, the frame rate modulation, and the system performance modulation are performed interactively with peripheral devices, including sensors such as a camera sensor, an acceleration sensor, and an infrared sensor. The primary goal is to save power, achieved by reducing the system's workload, including CPU and GPU loads. By having this mechanism, the control module is configured to establish a multidimensional intelligent detection and determination mechanism, which makes comprehensive determination bases on real-time system performance, data captured by sensors, application scenarios, and more. It dynamically adjusts the system's operating strategy. Sensor data includes acceleration sensor data for head movements and camera sensor data for eye movements.
[0045] For instance, during gaming applications, where high frame rates and resolution are required, the display apparatus is configured to display an image based on the user’s gaze direction, to reduce GPU and CPU power consumption. For example, the display apparatus is configured to render and process the image based on the user’s gaze direction (e.g., gaze point display) . In some embodiments, the display apparatus (e.g., one or more processors in the control module) is configured to track and monitor the user’s eye movements and gaze direction to determine where they are looking on a screen or in their field of view. This information is then used to control and interact with digital interfaces, applications, or devices. Gaze point display is often used for various purposes, including user interface navigation, gaming, accessibility, and research. When a user is looking at a specific area of the screen, gaze point display can prioritize rendering and processing resources for that particular region. This means that areas of the screen that are outside the user's gaze, known as peripheral or non-focused areas, can be rendered at a lower resolution or with fewer details.
[0046] In some embodiments, when head movement speeds exceed a certain threshold, the display apparatus is configured to display an image in a lower-resolution mode to further reduce the system load. Accordingly, the present display apparatus provides a multi-dimensional intelligent detection and determination mechanism to make comprehensive determinations based on the real-time operational state of the display apparatus’ performance, data captured by sensors, and application scenario information, among other factors. These determinations are used to dynamically adjust the system's operating strategy. The sensor data includes acceleration sensor data for capturing head movements and camera sensor data for capturing eye movements.
[0047] In some embodiments, the intelligent detection and determination mechanism monitors the type of application initiated by the user, for example, a gaming application or a video viewing application. For gaming applications, both frame rate and resolution requirements are relatively high, necessitating higher performance. However, higher performance also increases power consumption. Therefore, the intelligent detection and determination mechanism continues to modulate based on other mechanisms (e.g., head movements) to reduce power consumption. For example, when the speed of head movement is less than 30 degrees per second, the display apparatus is configured to render and process the image based on the user’s gaze direction (e.g., a gaze point display mode) , to reduce GPU and CPU power consumption. In another example, when the speed of head movement exceeds 30 degrees per second, the display apparatus is configured to display an image in a lower-resolution mode to further reduce the system load.
[0048] In some embodiments, when the speed of head movement is less than 30 degrees per second, the intelligent detection and determination mechanism further checks to achieve a better match between system performance and application operation. In one example, the intelligent detection and determination mechanism first checks the complexity of the scene, categorizing it into multiple levels (for example, a simple level: 1, a moderately complex level: 2, and a heavily loaded level: 3) , each with corresponding threshold values. These thresholds correspond to different resolutions. When the complexity is low, a slightly higher resolution mode can be used to enhance the experience. When the complexity is high, in order to save power and avoid a significant increase in power consumption due to system overload, a lower-resolution mode can be switched to achieve an overall performance balance.
[0049] In some embodiments, the scene complexity is detected based on detection of a complexity of a scene’s model. In some embodiments, the complexity of the scene's model can be understood as the number of vertices and polygons used when creating the scene. The more vertices and polygons a scene contains, the higher its model complexity and the scene complexity, which can result in increased rendering workload for the GPU. For example, in a gaming application or a 360 degrees panoramic application, scenes can vary in terms of model complexity and scene complexity. The entire application may include both complex scenes (such as buildings and dynamic characters) and low-complexity scenes (such as backgrounds and natural landscapes) . Adjusting scene complexity is necessary based on different processor platforms.
[0050] In the context of computer graphics and image rendering, "scene complexity" refers to the level of intricacy, level of detail, and computational demands of a particular scene or environment that needs to be rendered on a display or in a virtual world. It encompasses various factors such as the number of objects, their shapes, textures, lighting conditions, and the interactions between them. The term “model complexity” refers to the level of intricacy, level of detail, and sophistication of a model (e.g., a 3D model) that represents a scene in a digital environment. Model complexity encompasses various aspects that define the richness and realism of the model. Some key factors contributing to model complexity include geometric complexity, texture complexity, lighting complexity, physics and dynamic complexity, level of detail, transparency and overlapping objects, shadow complexity, and rendering techniques. Some key factors contributing to model complexity include polygon count, vertex count, texture maps, normal maps, rigging and animation, level of detail, material complexity, physics simulation, facial expressions, and interactivity.
[0051] In some embodiments, the intelligent detection and determination mechanism according to the present disclosure utilizes multiple threshold levels (e.g., threshold levels 1, 2, and 3) to determine the complexity of a scene. Each threshold level corresponds to a specific range of vertex and polygon counts. When the scene complexity is below the threshold 1. it is categorized as a simple scene. When the scene complexity is between 1 and 2, it is considered a moderately complex scene. When the scene complexity exceeds 3, it is categorized as a heavily loaded scene. Applications can then adjust their own resolution based on these threshold levels. Lowering the rendering resolution directly reduces the number of vertices and polygons in the scene, thereby reducing scene complexity.
[0052] In some embodiments, the scene complexity is detected based on detection of a model complexity of a scene. In some embodiments, the intelligent detection and determination mechanism utilizes a ray collision detection mechanism to determine model complexity, particularly in product applications where developers may not have prior knowledge of an application's model complexities. During GPU rendering, the ray collision detection mechanism simulates the interaction of light rays with models, evaluating the number of vertices and polygons by examining the reflection of light beams from the models.
[0053] In some embodiments, the intelligent detection and determination mechanism is configured to perform an auxiliary performance check in combination with the scene complexity detection. The auxiliary performance check involves real-time monitoring of the GPU rendering frame rate to determine if the current system performance meets the rendering requirements. If the performance falls short, even if the scene complexity detection result is 1, the system will further reduce the resolution to lower scene complexity until it meets the required system performance.
[0054] FIG. 3 is a schematic diagram illustrating functionalities of a control module of a display apparatus in some embodiments according to the present disclosure. Referring to FIG. 3, the control module (e.g., one or more processors and peripheral circuits connected thereto) is configured to perform an image display scene rendering. The image display scene rendering may be performed with respect to various appropriate applications, for example, a gaming application and a video viewing application. In one particular example, the image display scene rendering is a game scene rendering in a gaming application.
[0055] In some embodiments, the control module is configured to perform a scene complexity detection. In some embodiments, the control module is configured to group a scene complexity of the image display scene into multiple levels, including a first level ( “Level 1” ), a second level ( “Level 2) , and a third level ( “Level 3” ) . In some embodiments, the third level indicates a complexity level higher than the second level, which indicates a complexity level higher than the first level.
[0056] In some embodiments, the scene complexity is detected based on detection of a model complexity of a scene. In some embodiments, to perform the scene complexity detection, the control module is configured to perform a ray collision detection. Optionally, performing the ray collision detection includes simulating interaction of light rays with a model. As used herein, the term “model” refers to a three-dimensional representation of objects, scenes, or environments created within a computer graphics or simulation system. These models are used in various applications, such as video games, computer-aided design (CAD) , architectural visualization, virtual reality (VR) , and more. A complex model may have a high number of vertices, polygons, or texture maps, which can increase the computational resources required to render it realistically. Evaluating the model complexity helps determine how resource-intensive it is to render the scene or object, and this information is crucial for optimizing performance in real-time applications, especially in computer graphics and gaming. Optionally, performing the ray collision detection further includes evaluating a number of vertices and polygons by examining reflection of simulated light beams from the model.
[0057] In some embodiments, the control module is further configured to transmit complexity level information to one or more applications. As used herein, the term “application” refers to software programs or pieces of software designed for specific tasks or functions. These applications are executed on a computer system, mobile device, or other computing platforms to perform various functions or provide specific services to users. Examples of applications include gaming applications, video viewing applications, mobile applications, and web applications.
[0058] In some embodiments, the control module is further configured to render an image at a first resolution range ( “First resolution rendering” ) upon determination that the scene complexity is at the first level. Optionally, the first resolution range is a high-resolution range. In some embodiments, the control module is configured to render the image at the first resolution range and a high frame rate range. Optionally, the first frame rate range is a high frame rate range.
[0059] In some embodiments, when the control module is configured to render the image at the first resolution range, the control module is further configured to control a display panel to display an image based on the user’s gaze direction (e.g., in a gaze point display mode) . Optionally, the control module is configured to track the user’s eye movements and gaze direction. Optionally, the control module is further configured to render a first sub-image and a second sub-image, wherein the first sub-image is in a region intersecting with the gaze direction, and the second sub-image is in a region non-intersecting with the gaze direction. Optionally, the first sub-image has a resolution higher than a resolution of the second sub-image.
[0060] In some embodiments, the control module is further configured to render an image at a second resolution range ( “Second resolution rendering” ) upon determination that the scene complexity is at the second level. Optionally, the second resolution range is lower than the first resolution range. Optionally, the scene complexity at the second level is higher than the scene complexity at the first level.
[0061] In some embodiments, the control module is further configured to render an image at a third resolution range ( “Third resolution rendering” ) upon determination that the scene complexity is at the third level. Optionally, the third resolution range is lower than the second resolution range, which is lower than the first resolution range. Optionally, the scene complexity at the third level is higher than the scene complexity at the second level, which is higher than the scene complexity at the first level.
[0062] In some embodiments, when the image is rendered at the first resolution range, the second resolution range, or the third resolution range, the display apparatus is configured to display the image at a frame rate above a threshold value. In one particular example, the threshold value is 90 frame per second.
[0063] In some embodiments, the control module is further configured to transmit a resolution-switching command to a fundamental-level component, and drive the fundamental-level component to switch to a corresponding resolution output. As used herein, the term “fundamental-level component” refers to the fundamental or foundational elements of a system that provide essential functionality and interact closely with the underlying hardware. These components often operate at a fundamental-level of abstraction and are responsible for tasks such as hardware control, resource management, and interfacing with hardware devices. Examples of fundamental-level components include device drivers, kernels, firmware, hardware abstraction layers, and so on.
[0064] In some embodiments, the control module is further configured to monitor in real time rendering frame rate to determine if a current system performance meets rendering requirements ( “auxiliary performance check” ) . In some embodiments, upon determination that the current system performance fails to meet rendering requirements, the control module is further configured to render the image at a lower resolution range until the current system performance can meet rendering requirements.
[0065] FIG. 4 is a schematic diagram illustrating functionalities of a control module of a display apparatus in some embodiments according to the present disclosure. Referring to FIG. 4, the control module (e.g., one or more processors and peripheral circuits connected thereto) is configured to start one or more applications and initialize image display scene information. Subsequently, the control module is further configured to recognize the image display scene information. In some embodiments, the image display scenes include multiple different display scenes, includes a first scene and a second scene. In one example, the first scene is a video viewing scene for a user to view a video or an image using a video viewing application. In another example, the second scene is a gaming scene for a user to play a video game using a gaming application.
[0066] In some embodiments, the control module is configured to prompt a user to select a type of application during installation of the application. For example, when a user installs the application, the control module is configured to prompt the user to select from a list of image display scenes such as gaming scene or video viewing scene. Upon installation of the application, the control module appends a symbolic name of the application, indicating a type of the image display scene. When a user launches the application, the control module is configured to initialize image display scene information, during which the control module is configured to recognize the symbolic name.
[0067] In some embodiments, upon determination that the image display scene is a first scene, the control module is configured to turn off a gaze point sensor. In some embodiments, the control module is further configured to monitor a state of charge of a power supply (e.g., a battery) of the display apparatus. As used herein, the term “state of charge” refers to how much the power supply’s capacity is currently available. For example, a 50%state of charge indicates a half-charged state.
[0068] In some embodiments, upon determination that the state of charge of the power supply is greater than a threshold state of charge, the control module is configured to render an image at a fourth resolution range. In some embodiments, upon determination that the state of charge of the power supply is equal to or less than a threshold state of charge, the control module is configured to render an image at a fifth resolution range. In one example, the threshold state of charge is 50%of a power supply’s full capacity. In another example, the fourth resolution range is a resolution of 4K *4K. In another example, the fourth resolution range is a resolution of 4K *2K. In another example, the fifth resolution range is a resolution of 2K *2K.
[0069] In some embodiments, upon determination that the image display scene is a second scene, the control module is configured to turn on an acceleration sensor to monitor head movement of the user. In some embodiments, upon determination that a speed of the head movement is greater than a first threshold speed, the control module is configured to turn off the gaze point sensor; and render an image at a sixth resolution range. In one particular example, the first threshold speed is 30 degrees per second. In one example, the sixth resolution range is a resolution of 2K *2K.
[0070] In some embodiments, upon determination that a speed of the head movement is equal to or less than the first threshold speed, the control module is configured to turn on the gaze point sensor; and monitor a speed of eye movement. In some embodiments, upon determination that the speed of the eye movement is greater than a second threshold speed, the control module is configured to render an image at a second resolution range. Optionally, upon determination that the speed of eye movement is greater than a second threshold speed, the control module is further configured to control the gaze point sensor to monitor the speed of the eye movement in a first-speed exposure mode. In one example, the second threshold speed is 30 degrees per second.
[0071] In some embodiments, upon determination that the speed of the eye movement is equal to or less than the second threshold speed, the control module is configured to control the gaze point sensor to monitor the speed of the eye movement in a second-speed exposure mode. In the first-speed exposure mode, the sensor monitors the speed of the eye movement at a first frequency; in the second-speed exposure mode, the sensor monitors the speed of the eye movement at a second frequency; the second frequency is greater than the first frequency.
[0072] In some embodiments, upon determination that the speed of the eye movement is equal to or less than the second threshold speed, the control module is configured to perform a scene complexity detection. In some embodiments, the control module is configured to group a scene complexity of the image display scene into multiple levels, including a first level ( “Level 1” ) , a second level ( “Level 2) , and a third level ( “Level 3” ) . In some embodiments, the third level indicates a complexity level higher than the second level, which indicates a complexity level higher than the first level.
[0073] In some embodiments, the scene complexity is detected based on detection of a model complexity of a scene. In some embodiments, to perform the model complexity detection, the control module is configured to perform a ray collision detection. Optionally, performing the ray collision detection includes determining a number of vertices and polygons of a model. In some embodiments, the control module is configured to group a scene complexity of the image display scene into multiple levels, including a first level ( “Level 1” ) , a second level ( “Level 2) , and a third level ( “Level 3” ) . In some embodiments, the third level indicates a complexity level higher than the second level, which indicates a complexity level higher than the first level.
[0074] In some embodiments, the control module is further configured to render an image at a first resolution range upon determination that the scene complexity is at the first level. Optionally, the first resolution range is a high-resolution range. In some embodiments, the control module is configured to render the image at the first resolution range and a high frame rate range. Optionally, the first frame rate range is a high frame rate range. In one example, the first resolution range is a resolution of 2K *2K.
[0075] In some embodiments, the control module is further configured to render a first sub-image and a second sub-image, wherein the first sub-image is in a region intersecting with the gaze direction, and the second sub-image is in a region non-intersecting with the gaze direction. Optionally, the first sub-image has a resolution higher than a resolution of the second sub-image.
[0076] In some embodiments, the control module is configured to render the first sub-image at the first resolution range upon determination that the scene complexity is at the first level.
[0077] In some embodiments, the control module is further configured to monitor in real time rendering frame rate to determine if a current system performance meets rendering requirements ( “auxiliary performance check” ) . In some embodiments, upon determination that the current system performance fails to meet rendering requirements, the control module is further configured to render an image at a second resolution range. In one example, the second resolution range is a resolution of 1.5K *1.5K. In some embodiments, the control module is configured to render the first sub-image at the second resolution range upon determination that the current system performance fails to meet rendering requirements.
[0078] In some embodiments, the control module is further configured to render an image at a second resolution range upon determination that the scene complexity is at the second level. Optionally, the second resolution range is lower than the first resolution range. Optionally, the scene complexity at the second level is higher than the scene complexity at the first level. In one example, the second resolution range is a resolution of 1.5K *1.5K.
[0079] In some embodiments, the control module is configured to render the first sub-image at the second resolution range upon determination that the scene complexity is at the second level.
[0080] In some embodiments, the control module is further configured to monitor in real time rendering frame rate to determine if a current system performance meets rendering requirements ( “auxiliary performance check” ) . In some embodiments, upon determination that the current system performance fails to meet rendering requirements, the control module is further configured to render an image at a third resolution range. In one example, the third resolution range is a resolution of 1.2K *1.2K. In some embodiments, the control module is configured to render the first sub-image at the third resolution range upon determination that the current system performance fails to meet rendering requirements.
[0081] In some embodiments, the control module is further configured to render an image at a third resolution range upon determination that the scene complexity is at the third level. Optionally, the third resolution range is lower than the second resolution range, which is lower than the first resolution range. Optionally, the scene complexity at the third level is higher than the scene complexity at the second level, which is higher than the scene complexity at the first level. In one example, the third resolution range is a resolution of 1.2K *1.2K.
[0082] In some embodiments, the control module is configured to render the first sub-image at the third resolution range upon determination that the scene complexity is at the third level.
[0083] FIG. 5 illustrates interaction among applications, fundamental-level components, input hardware, and output hardware in a display apparatus in some embodiments according to the present disclosure. Referring to FIG. 5, when an application in an application layer is triggered, a signal is transmitted to a clock management module. The clock management module is configured to transmit a synchronization signal to a synchronization signal distribution thread and to a fundamental-level clock synchronization signal to a fundamental-level interface module. The synchronization signal distribution thread is configured to distribute the synchronization signal to various applications, including a sensor synchronization control thread, a light emitting element synchronization control thread, and a display panel synchronization control thread.
[0084] In some embodiments, the sensor synchronization control thread, the light emitting element synchronization control thread, or the display panel synchronization control thread are configured to transmit the synchronization signal to the fundamental-level interface module. The fundamental-level interface module is configured to transmit the synchronization signal to various fundamental-level component driving modules of the fundamental-level components, including a display driving module, a sensor driving module, and a light emitting element driving module. The sensor driving module is configured to drive a camera sensor. The light emitting element driving module is configured to drive a fill light driving circuit. The display driving module is configured to drive a display panel.
[0085] The synchronization signal synchronizes operations of the applications, the fundamental-level components, the input hardware, and the output hardware. Upon the application in the application layer being triggered, the synchronization signal distribution thread is configured to distribute the synchronization signal to various applications and various fundamental-level components, and eventually synchronizing the input hardware and the output hardware.
[0086] FIG. 6 illustrates synchronized operation by a synchronization signal in a display apparatus in some embodiments according to the present disclosure. Referring to FIG. 6, the control module in some embodiments includes an image rendering module configured to render an image. For example, the image rendering module may be configured to render an image at a first resolution range, render an image at a second resolution range, render an image at a third resolution range, render an image at a fourth resolution range, render an image at a fifth resolution range, or render an image at a sixth resolution range. Optionally, the image rendering module may be configured to render a first sub-image and a second sub-image, wherein the first sub-image is in a region intersecting with the gaze direction, and the second sub-image is in a region non-intersecting with the gaze direction. Upon the image being rendered, the image rendering module is configured to transmit a signal to a clock management module. Upon receiving the signal, the clock management module is configured to generate a synchronization signal. In some embodiments, the clock management module is configured to transmit the synchronization signal to a synchronization signal distribution thread. Referring to FIG. 5, the synchronization signal distribution thread is configured to distribute the synchronization signal to various applications, including a sensor synchronization control thread, a light emitting element synchronization control thread, and a display panel synchronization control thread.
[0087] Referring to FIG. 6, the control module in some embodiments includes a determination module configured to determine a speed of eye movement, determine a speed of head movement, determine an image display scene, and determine system performance, as discussed previously in connection with FIG. 3 and FIG. 4. The determination module is configured to transmit the determination results to the image rendering module.
[0088] In some embodiments, the determination module is configured to obtain coordinates of the gaze point at two different moments along with the angular velocity of the line of sight. The eye movement speed is then calculated as angular velocity divided by the time difference between the two moments.
[0089] In some embodiments, the determination module is configured to determine the direction of the head movement, such as up-down, left-right, or diagonal. Upon determination of the direction of the head movement, the determination module is further configured to extract angular velocity data in a horizontal direction for the head movement and convert the angular velocity data into the speed of the head movement.
[0090] In some embodiments, the control module further includes a real-time information acquisition module configured to acquire information required for the determination module in real time. Examples of real-time information include position information, gaze point information, symbolic name of an application, a rendering frame rate, and a model complexity level or a scene complexity level.
[0091] In some embodiments, the control module further includes an eye movement speed and head movement speed computation module configured to calculate the eye movement speed based on gaze point image information transmitted by a camera sensor and calculate the head movement speed based on gesture information transmitted by an acceleration sensor. The eye movement speed and head movement speed computation module is further configured to transmit an eye movement speed and a head movement speed to the real-time information acquisition module.
[0092] In some embodiments, the control module further includes an initialization module configured to initialize image display scene information based on a scene content. Examples of scene contents include a moving scene, a static scene, a gaming scene, and a video viewing scene. The initialization module is further configured to transmit the image display scene information to the real-time information acquisition module.
[0093] In some embodiments, the control module further includes additional modules such as a software development rendering module and a graphic memory rendering module.
[0094] FIG. 7 illustrates a process of information output in a display apparatus in some embodiments according to the present disclosure. Referring to FIG. 7, the display apparatus in some embodiments is configured to pre-configure a list of supported display resolutions in the system. When an application triggers the resolution switching mechanism, it will call the resolution switching API interface and send the required resolution change information to the kernel driver layer. The kernel driver layer will traverse the resolution list, match it, and then reset the timing, recalculate the information buffer size, the required speed, and bandwidth for information transmission. The kernel driver layer will also trigger the screen controller to refresh a new initialization sequence to prepare for receiving information from the new resolution image. The system layer will also reset the framebuffer according to the new resolution to match the new display resolution.
[0095] As discussed previously, in some embodiments, upon determination that the speed of eye movement is greater than a second threshold speed, the control module is further configured to control the gaze point sensor to monitor the speed of the eye movement in a first-speed exposure mode. When the speed of eye movement is greater than the second threshold speed, the sensor recognizing eye movement data does not require precise positioning; it only needs to detect eye movement speed. Therefore, the sensor will adjust to first-speed exposure, and the fill light will also synchronize to low-frequency blinking, only illuminating when the sensor is exposed.
[0096] In some embodiments, upon determination that the speed of the eye movement is equal to or less than the second threshold speed, the control module is configured to control the gaze point sensor to monitor the speed of the eye movement in a second-speed exposure mode. When the speed of eye movement is equal to or less than the second threshold speed, precise positioning of the gaze point is needed, the sensor requires second-speed exposure, and the fill light can also use full illumination mode. Adopting this working mode can significantly reduce the energy consumption required for operation, effectively reducing the system's power consumption. FIG. 8 illustrates a process of determining a first-speed exposure mode or a second-speed exposure mode in some embodiments according to the present disclosure. As shown in FIG. 8, the operating mechanism is based on the synchronization signal to achieve synchronization between the sensor, fill light, and the system's operating clock.
[0097] In some embodiments, the display apparatus includes a variable frequency drive module configured to modulate a pulse width modulation timing of a fill light driving circuit. The variable frequency drive module is configured to modulate pulses for the fill light driving circuit to have a first pulse frequency in a first driving mode and a second pulse frequency in a second driving mode, wherein the first pulse frequency is less than the second pulse frequency. The first driving mode corresponds to a first-speed exposure mode of the gaze point sensor configured to monitor a speed of the eye movement, and the second driving mode corresponds to a second-speed exposure mode of the gaze point sensor. When the speed of eye movement is greater than the second threshold speed, the sensor recognizing eye movement data does not require precise positioning, and the pulses for the fill light driving circuit have a first pulse frequency in the first driving mode. When the speed of eye movement is equal to or less than the second threshold speed, precise positioning of the gaze point is needed, and the pulses for the fill light driving circuit have a second pulse frequency in the second driving mode.
[0098] FIG. 9 is a schematic diagram illustrating a display apparatus in some embodiments according to the present disclosure. Referring to FIG. 9, the display apparatus in some embodiments includes a processor 1002, a storage medium 1004, a display 1006, a communication module 1008, a database 1010, peripherals 1012, and a camera 1014. Certain devices may be omitted, and other devices may be included to better describe the relevant embodiments. The display apparatus may include any appropriate type of display panels, such as a plasma display panel, a liquid crystal display (LCD) panel, a touch screen display panel, a projection display panel, a non-smart display panel, a smart display panel, etc. The display apparatus may also include other computing systems, such as a personal computer (PC) , a tablet or mobile computer, or a smart phone, etc. In addition, the display apparatus may be any appropriate content-presentation device capable of presenting any appropriate content. Users may interact with the display apparatus to perform other activities of interest.
[0099] The processor 1002 may include any appropriate processor or processors. Further, the processor 1002 may include multiple cores for multi-thread or parallel processing. The processor 1002 may execute sequences of computer program instructions to perform various processes. The storage medium 1004 may include memory modules, such as ROM, RAM, flash memory modules, and mass storages, such as CD-ROM and hard disk, etc. The storage medium 1004 may store computer programs for implementing various processes when the computer programs are executed by the processor 1002. For example, the storage medium 1004 may store computer programs for implementing various algorithms when the computer programs are executed by the processor 1002.
[0100] Further, the communication module 1008 may include certain network interface devices for establishing connections through communication networks, such as TV cable network, wireless network, internet, etc. The database 1010 may include one or more databases for storing certain data and for performing certain operations on the stored data, such as database searching.
[0101] The display 1006 may provide information to users. The display 1006 may include any appropriate type of computer display device or electronic apparatus display such as LCD or OLED based devices. The peripherals 1012 may include various sensors and other I / O devices, such as keyboard and mouse.
[0102] Examples of appropriate display apparatuses include, but are not limited to, an electronic paper, a mobile phone, a tablet computer, a television, a monitor, a notebook computer, a digital album, a GPS, etc. Optionally, the display apparatus is an organic light emitting diode display apparatus. Optionally, the display apparatus is a micro light emitting diode display apparatus. Optionally, the display apparatus is a mini light emitting diode display apparatus.
[0103] In some embodiments, the apparatus includes a memory; and one or more processors. Optionally, the memory and the one or more processors are connected with each other. Optionally, the memory stores computer-executable instructions for controlling the one or more processors to execute various functions described herein.
[0104] In another aspect, the present disclosure provides a method of display. FIG. 10 is a flow chart illustrating a method of display in some embodiments according to the present disclosure. Referring to FIG. 10, in some embodiments, the method includes recognizing an image display scene information; performing a scene complexity detection based on the image display scene information. In some embodiments, performing the scene complexity detection includes grouping a scene complexity of the image display scene into multiple levels based on a type of the image display scene, including a first level, a second level, and a third level. In some embodiments, the third level indicates a complexity level higher than the second level, which indicates a complexity level higher than the first level.
[0105] In some embodiments, the scene complexity is detected based on detection of a model complexity of a scene. In some embodiments, performing the model complexity detection includes performing a ray collision detection. Optionally, performing the ray collision detection includes simulating interaction of light rays with a model. Optionally, performing the ray collision detection further includes evaluating a number of vertices and polygons by examining reflection of simulated light beams from the model.
[0106] In some embodiments, the method further includes transmitting complexity level information to one or more applications.
[0107] In some embodiments, the method further includes rendering an image at a first resolution range upon determination that the scene complexity is at the first level. Optionally, the first resolution range is a high-resolution range. In some embodiments, the control module is configured to render the image at the first resolution range and a high frame rate range. Optionally, the first frame rate range is a high frame rate range.
[0108] In some embodiments, the method further includes controlling a display panel to display an image based on the user’s gaze direction. Optionally, the method further includes tracking the user’s eye movements and gaze direction. Optionally, the method further includes rendering a first sub-image and a second sub-image, wherein the first sub-image is in a region intersecting with the gaze direction, and the second sub-image is in a region non-intersecting with the gaze direction. Optionally, the first sub-image has a resolution higher than a resolution of the second sub-image.
[0109] In some embodiments, the method further includes rendering an image at a second resolution range upon determination that the scene complexity is at the second level. Optionally, the second resolution range is lower than the first resolution range. Optionally, the scene complexity at the second level is higher than the scene complexity at the first level.
[0110] In some embodiments, the method further includes rendering an image at a third resolution range upon determination that the scene complexity is at the third level. Optionally, the third resolution range is lower than the second resolution range, which is lower than the first resolution range. Optionally, the scene complexity at the third level is higher than the scene complexity at the second level, which is higher than the scene complexity at the first level.
[0111] In some embodiments, the method further includes maintaining a frame rate of displaying the image above a threshold value. In one particular example, the threshold value is 90 frame per second.
[0112] In some embodiments, the method further includes transmitting a resolution-switching command to a fundamental-level component, and driving the fundamental-level component to switch to a corresponding resolution output.
[0113] In some embodiments, the method further includes monitoring in real time rendering frame rate to determine if a current system performance meets rendering requirements. In some embodiments, upon determination that the current system performance fails to meet rendering requirements, the method further includes rendering the image at a lower resolution range until the current system performance can meet rendering requirements.
[0114] FIG. 11 is a flow chart illustrating a method of display in some embodiments according to the present disclosure. Referring to FIG. 11, the method in some embodiments includes starting one or more applications and initializing image display scene information; and recognizing the image display scene information. In some embodiments, the image display scenes include multiple different display scenes, includes a first scene and a second scene. In one example, the first scene is a video viewing scene for a user to view a video or an image using a video viewing application. In another example, the second scene is a gaming scene for a user to play a video game using a gaming application.
[0115] In some embodiments, the method further includes, upon determination that the image display scene is a first scene, turning off a gaze point sensor. In some embodiments, the method further includes monitoring a state of charge of a power supply of a display apparatus; and determining whether the state of charge of the power supply is greater than a threshold state of charge. In some embodiments, the method further includes, upon determination that the state of charge of the power supply is greater than a threshold state of charge, rendering an image at a fourth resolution range. In some embodiments, the method further includes, upon determination that the state of charge of the power supply is equal to or less than a threshold state of charge, rendering an image at a fifth resolution range. In one example, the threshold state of charge is 50%of a power supply’s full capacity. In another example, the fourth resolution range is a resolution of 4K *4K. In another example, the fourth resolution range is a resolution of 4K *2K. In another example, the fifth resolution range is a resolution of 2K *2K.
[0116] In some embodiments, the method further includes, upon determination that the image display scene is a second scene, turning on an acceleration sensor to monitor head movement of the user. In some embodiments, the method further includes, upon determination that a speed of the head movement is greater than a first threshold speed, turning off the gaze point sensor; and rendering an image at a sixth resolution range. In one particular example, the first threshold speed is 30 degrees per second. In one example, the sixth resolution range is a resolution of 2K *2K.
[0117] In some embodiments, the method further includes, upon determination that a speed of the head movement is equal to or less than the first threshold speed, turning on the gaze point sensor; and monitoring a speed of eye movement. In some embodiments, the method further includes, upon determination that the speed of the eye movement is greater than a second threshold speed, rendering an image at a seventh resolution range. Optionally, the method further includes, upon determination that the speed of eye movement is greater than a second threshold speed, controlling the gaze point sensor to monitor the speed of the eye movement in a first-speed exposure mode. In one example, the second threshold speed is 30 degrees per second. In one example, the seventh resolution range is a resolution of 2K *2K.
[0118] In some embodiments, the method further includes, upon determination that the speed of the eye movement is equal to or less than the second threshold speed, controlling the gaze point sensor to monitor the speed of the eye movement in a second-speed exposure mode.
[0119] In some embodiments, the method further includes, upon determination that the speed of the eye movement is equal to or less than the second threshold speed, performing a scene complexity detection. In some embodiments, the method further includes grouping a scene complexity of the image display scene into multiple levels, including a first level, a second level, and a third level. In some embodiments, the third level indicates a complexity level higher than the second level, which indicates a complexity level higher than the first level.
[0120] In some embodiments, the scene complexity is detected based on detection of a model complexity of a scene. In some embodiments, performing the model complexity detection includes performing a ray collision detection. Optionally, performing the ray collision detection includes determining a number of vertices and polygons of a model. In some embodiments, the method further includes grouping a scene complexity of the image display scene into multiple levels, including a first level, a second level, and a third level. In some embodiments, the third level indicates a complexity level higher than the second level, which indicates a complexity level higher than the first level.
[0121] In some embodiments, the method further includes rendering an image at a first resolution range upon determination that the scene complexity is at the first level. Optionally, the first resolution range is a high-resolution range. In some embodiments, the method further includes rendering the image at the first resolution range and a high frame rate range. Optionally, the first frame rate range is a high frame rate range. In one example, the first resolution range is a resolution of 2K *2K.
[0122] In some embodiments, the method further includes rendering a first sub-image and a second sub-image, wherein the first sub-image is in a region intersecting with the gaze direction, and the second sub-image is in a region non-intersecting with the gaze direction. Optionally, the first sub-image has a resolution higher than a resolution of the second sub-image.
[0123] In some embodiments, the method further includes rendering the first sub-image at the first resolution range upon determination that the scene complexity is at the first level.
[0124] In some embodiments, the method further includes monitoring in real time rendering frame rate to determine if a current system performance meets rendering requirements. In some embodiments, the method further includes, upon determination that the current system performance fails to meet rendering requirements, rendering an image at a second resolution range. In one example, the second resolution range is a resolution of 1.5K *1.5K. In some embodiments, the method further includes rendering the first sub-image at the second resolution range upon determination that the current system performance fails to meet rendering requirements.
[0125] In some embodiments, the method further includes rendering an image at a second resolution range upon determination that the scene complexity is at the second level. Optionally, the second resolution range is lower than the first resolution range. Optionally, the scene complexity at the second level is higher than the scene complexity at the first level. In one example, the second resolution range is a resolution of 1.5K *1.5K.
[0126] In some embodiments, the method further includes rendering the first sub-image at the second resolution range upon determination that the scene complexity is at the second level.
[0127] In some embodiments, the method further includes monitoring in real time rendering frame rate to determine if a current system performance meets rendering requirements. In some embodiments, the method further includes, upon determination that the current system performance fails to meet rendering requirements, rendering an image at a third resolution range. In one example, the third resolution range is a resolution of 1.2K *1.2K. In some embodiments, the method further includes rendering the first sub-image at the third resolution range upon determination that the current system performance fails to meet rendering requirements.
[0128] In some embodiments, the method further includes rendering an image at a third resolution range upon determination that the scene complexity is at the third level. Optionally, the third resolution range is lower than the second resolution range, which is lower than the first resolution range. Optionally, the scene complexity at the third level is higher than the scene complexity at the second level, which is higher than the scene complexity at the first level. In one example, the third resolution range is a resolution of 1.2K *1.2K.
[0129] In some embodiments, the method further includes rendering the first sub-image at the third resolution range upon determination that the scene complexity is at the third level.
[0130] In some embodiments, the method further includes generating (e.g., by a clock management module) a synchronization signal; transmitting the synchronization signal to a synchronization signal distribution thread; and distributing (e.g., by the synchronization signal distribution thread) the synchronization signal to one or more applications. Examples of one or more applications include a sensor synchronization control thread, a light emitting element synchronization control thread, and a display panel synchronization control thread.
[0131] In some embodiments, the method further includes transmitting (e.g., by the one or more applications) the synchronization signal to a fundamental-level interface module; and transmitting (e.g., by the fundamental-level interface module) the synchronization signal to fundamental-level component driving modules of fundamental-level components. Examples of the fundamental-level component driving modules include a display driving module, a sensor driving module, and a light emitting element driving module. The sensor driving module is configured to drive a camera sensor. The light emitting element driving module is configured to drive a fill light driving circuit. The display driving module is configured to drive a display panel.
[0132] In some embodiments, the method further includes transmitting a fundamental-level clock synchronization signal to the fundamental-level interface module.
[0133] The synchronization signal synchronizes operations of the applications, the fundamental-level components, the input hardware, and the output hardware. Upon the application in the application layer being triggered, the synchronization signal distribution thread is configured to distribute the synchronization signal to various applications and various fundamental-level components, and eventually synchronizing the input hardware and the output hardware.
[0134] In another aspect, the present disclosure provides a computer-program product comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon. In some embodiments, the computer-readable instructions are executable by one or more processors to cause the one or more processors to perform recognizing an image display scene information; performing a scene complexity detection based on the image display scene information; grouping a scene complexity of an image display scene into multiple levels based on a type of the image display scene; rendering an image at a first resolution range upon determination that the scene complexity is at a first level; and rendering an image at a second resolution range upon determination that the scene complexity is at a second level. Optionally, the second resolution range is lower than the first resolution range. Optionally, the scene complexity at the second level is higher than the scene complexity at the first level.
[0135] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform performing a ray collision detection. Optionally, performing the ray collision detection includes simulating interaction of light rays with a model. Optionally, performing the ray collision detection further includes evaluating a number of vertices and polygons by examining reflection of simulated light beams from the model.
[0136] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform transmitting complexity level information to one or more applications.
[0137] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering an image at a first resolution range upon determination that the scene complexity is at the first level. Optionally, the first resolution range is a high-resolution range. In some embodiments, the control module is configured to render the image at the first resolution range and a high frame rate range. Optionally, the first frame rate range is a high frame rate range.
[0138] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform controlling a display panel to display an image based on the user’s gaze direction. Optionally, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform tracking the user’s eye movements and gaze direction. Optionally, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering a first sub-image and a second sub-image, wherein the first sub-image is in a region intersecting with the gaze direction, and the second sub-image is in a region non-intersecting with the gaze direction. Optionally, the first sub-image has a resolution higher than a resolution of the second sub-image.
[0139] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering an image at a second resolution range upon determination that the scene complexity is at the second level. Optionally, the second resolution range is lower than the first resolution range. Optionally, the scene complexity at the second level is higher than the scene complexity at the first level.
[0140] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering an image at a third resolution range upon determination that the scene complexity is at the third level. Optionally, the third resolution range is lower than the second resolution range, which is lower than the first resolution range. Optionally, the scene complexity at the third level is higher than the scene complexity at the second level, which is higher than the scene complexity at the first level.
[0141] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform maintaining a frame rate of displaying the image above a threshold value. In one particular example, the threshold value is 90 frame per second.
[0142] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform transmitting a resolution-switching command to a fundamental-level component, and driving the fundamental-level component to switch to a corresponding resolution output.
[0143] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform monitoring in real time rendering frame rate to determine if a current system performance meets rendering requirements. In some embodiments, upon determination that the current system performance fails to meet rendering requirements, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering the image at a lower resolution range until the current system performance can meet rendering requirements.
[0144] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform starting one or more applications and initializing image display scene information; and recognizing the image display scene information. In some embodiments, the image display scenes include multiple different display scenes, includes a first scene and a second scene. In one example, the first scene is a video viewing scene for a user to view a video or an image using a video viewing application. In another example, the second scene is a gaming scene for a user to play a video game using a gaming application.
[0145] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the image display scene is a first scene, turning off a gaze point sensor. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform monitoring a state of charge of a power supply of a display apparatus; and determining whether the state of charge of the power supply is greater than a threshold state of charge. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the state of charge of the power supply is greater than a threshold state of charge, rendering an image at a fourth resolution range. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the state of charge of the power supply is equal to or less than a threshold state of charge, rendering an image at a fifth resolution range. In one example, the threshold state of charge is 50%of a power supply’s full capacity. In another example, the fourth resolution range is a resolution of 4K *4K. In another example, the fourth resolution range is a resolution of 4K *2K. In another example, the fifth resolution range is a resolution of 2K *2K.
[0146] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the image display scene is a second scene, turning on an acceleration sensor to monitor head movement of the user. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that a speed of the head movement is greater than a first threshold speed, turning off the gaze point sensor; and rendering an image at a sixth resolution range. In one particular example, the first threshold speed is 30 degrees per second. In one example, the sixth resolution range is a resolution of 2K *2K.
[0147] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that a speed of the head movement is equal to or less than the first threshold speed, turning on the gaze point sensor; and monitoring a speed of eye movement. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the speed of the eye movement is greater than a second threshold speed, rendering an image at a seventh resolution range. Optionally, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the speed of eye movement is greater than a second threshold speed, controlling the gaze point sensor to monitor the speed of the eye movement in a first-speed exposure mode. In one example, the second threshold speed is 30 degrees per second. In one example, the seventh resolution range is a resolution of 2K *2K.
[0148] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the speed of the eye movement is equal to or less than the second threshold speed, controlling the gaze point sensor to monitor the speed of the eye movement in a second-speed exposure mode.
[0149] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the speed of the eye movement is equal to or less than the second threshold speed, performing a scene complexity detection. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform grouping a scene complexity of the image display scene into multiple levels, including a first level, a second level, and a third level. In some embodiments, the third level indicates a complexity level higher than the second level, which indicates a complexity level higher than the first level.
[0150] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform performing a ray collision detection. Optionally, performing the ray collision detection includes determining a number of vertices and polygons of a model. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform grouping a scene complexity of the image display scene into multiple levels, including a first level, a second level, and a third level. In some embodiments, the third level indicates a complexity level higher than the second level, which indicates a complexity level higher than the first level.
[0151] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering an image at a first resolution range upon determination that the scene complexity is at the first level. Optionally, the first resolution range is a high-resolution range. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering the image at the first resolution range and a high frame rate range. Optionally, the first frame rate range is a high frame rate range. In one example, the first resolution range is a resolution of 2K *2K.
[0152] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering a first sub-image and a second sub-image, wherein the first sub-image is in a region intersecting with the gaze direction, and the second sub-image is in a region non-intersecting with the gaze direction. Optionally, the first sub-image has a resolution higher than a resolution of the second sub-image.
[0153] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering the first sub-image at the first resolution range upon determination that the scene complexity is at the first level.
[0154] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform monitoring in real time rendering frame rate to determine if a current system performance meets rendering requirements. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the current system performance fails to meet rendering requirements, rendering an image at a second resolution range. In one example, the second resolution range is a resolution of 1.5K *1.5K. In some embodiments, the method further includes rendering the first sub-image at the second resolution range upon determination that the current system performance fails to meet rendering requirements.
[0155] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering an image at a second resolution range upon determination that the scene complexity is at the second level. Optionally, the second resolution range is lower than the first resolution range. Optionally, the scene complexity at the second level is higher than the scene complexity at the first level. In one example, the second resolution range is a resolution of 1.5K *1.5K.
[0156] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering the first sub-image at the second resolution range upon determination that the scene complexity is at the second level.
[0157] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform monitoring in real time rendering frame rate to determine if a current system performance meets rendering requirements. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform, upon determination that the current system performance fails to meet rendering requirements, rendering an image at a third resolution range. In one example, the third resolution range is a resolution of 1.2K *1.2K. In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering the first sub-image at the third resolution range upon determination that the current system performance fails to meet rendering requirements.
[0158] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering an image at a third resolution range upon determination that the scene complexity is at the third level. Optionally, the third resolution range is lower than the second resolution range, which is lower than the first resolution range. Optionally, the scene complexity at the third level is higher than the scene complexity at the second level, which is higher than the scene complexity at the first level. In one example, the third resolution range is a resolution of 1.2K *1.2K.
[0159] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform rendering the first sub-image at the third resolution range upon determination that the scene complexity is at the third level.
[0160] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform generating (e.g., by a clock management module) a synchronization signal; transmitting the synchronization signal to a synchronization signal distribution thread; and distributing (e.g., by the synchronization signal distribution thread) the synchronization signal to one or more applications. Examples of one or more applications include a sensor synchronization control thread, a light emitting element synchronization control thread, and a display panel synchronization control thread.
[0161] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform transmitting (e.g., by the one or more applications) the synchronization signal to a fundamental-level interface module; and transmitting (e.g., by the fundamental-level interface module) the synchronization signal to fundamental-level component driving modules of fundamental-level components. Examples of the fundamental-level component driving modules include a display driving module, a sensor driving module, and a light emitting element driving module. The sensor driving module is configured to drive a camera sensor. The light emitting element driving module is configured to drive a fill light driving circuit. The display driving module is configured to drive a display panel.
[0162] In some embodiments, the computer-readable instructions are executable by one or more processors to further cause the one or more processors to perform transmitting a fundamental-level clock synchronization signal to the fundamental-level interface module.
[0163] The synchronization signal synchronizes operations of the applications, the fundamental-level components, the input hardware, and the output hardware. Upon the application in the application layer being triggered, the synchronization signal distribution thread is configured to distribute the synchronization signal to various applications and various fundamental-level components, and eventually synchronizing the input hardware and the output hardware.
[0164] Various illustrative operations described in connection with the configurations disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. Such operations may be implemented or performed with a general purpose processor, a digital signal processor (DSP) , an ASIC or ASSP, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to produce the configuration as disclosed herein. For example, such a configuration may be implemented at least in part as a hard-wired circuit, as a circuit configuration fabricated into an application-specific integrated circuit, or as a firmware program loaded into non-volatile storage or a software program loaded from or into a data storage medium as machine-readable code, such code being instructions executable by an array of logic elements such as a general purpose processor or other digital signal processing unit. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. A software module may reside in a non-transitory storage medium such as RAM (random-access memory) , ROM (read-only memory) , nonvolatile RAM (NVRAM) such as flash RAM, erasable programmable ROM (EPROM) , electrically erasable programmable ROM (EEPROM) , registers, hard disk, a removable disk, or a CD-ROM; or in any other form of storage medium known in the art. An illustrative storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
[0165] The foregoing description of the embodiments of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form or to exemplary embodiments disclosed. Accordingly, the foregoing description should be regarded as illustrative rather than restrictive. Obviously, many modifications and variations will be apparent to practitioners skilled in this art. The embodiments are chosen and described in order to explain the principles of the invention and its best mode practical application, thereby to enable persons skilled in the art to understand the invention for various embodiments and with various modifications as are suited to the particular use or implementation contemplated. It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents in which all terms are meant in their broadest reasonable sense unless otherwise indicated. Therefore, the term “the invention” , “the present invention” or the like does not necessarily limit the claim scope to a specific embodiment, and the reference to exemplary embodiments of the invention does not imply a limitation on the invention, and no such limitation is to be inferred. The invention is limited only by the spirit and scope of the appended claims. Moreover, these claims may refer to use “first” , “second” , etc. following with noun or element. Such terms should be understood as a nomenclature and should not be construed as giving the limitation on the number of the elements modified by such nomenclature unless specific number has been given. Any advantages and benefits described may not apply to all embodiments of the invention. It should be appreciated that variations may be made in the embodiments described by persons skilled in the art without departing from the scope of the present invention as defined by the following claims. Moreover, no element and component in the present disclosure is intended to be dedicated to the public regardless of whether the element or component is explicitly recited in the following claims.
Claims
1.A display apparatus, comprising:a memory; andone or more processors;wherein the memory and the one or more processors are connected with each other; andthe memory stores computer-executable instructions for controlling the one or more processors to:recognize an image display scene information;perform a scene complexity detection based on the image display scene information;group a scene complexity of an image display scene into multiple levels based on a type of the image display scene;render an image at a first resolution range upon determination that the scene complexity is at a first level; andrender an image at a second resolution range upon determination that the scene complexity is at a second level;wherein the second resolution range is lower than the first resolution range; andthe scene complexity at the second level is higher than the scene complexity at the first level.2.The display apparatus of claim 1, wherein the memory further stores computer-executable instructions for controlling the one or more processors to render a first sub-image and a second sub-image;wherein the first sub-image is in a region intersecting with a gaze direction of a user;the second sub-image is in a region non-intersecting with the gaze direction;the first sub-image has a resolution higher than a resolution of the second sub-image; andthe first sub-image is rendered at the first resolution range upon determination that the scene complexity is at a first level and at the second resolution range upon determination that the scene complexity is at the second level.3.The display apparatus of claim 1, wherein, when the image is rendered at the first resolution range, the memory further stores computer-executable instructions for controlling the one or more processors to:monitor in real time rendering frame rate to determine if a current system performance meets rendering requirements; andupon determination that the current system performance fails to meet rendering requirements, render an image at the second resolution range.4.The display apparatus of any one of claims 1 to 3, wherein the memory further stores computer-executable instructions for controlling the one or more processors to render an image at a third resolution range upon determination that the scene complexity is at a third level;wherein the third resolution range is lower than the second resolution range; andthe scene complexity at the third level is higher than the scene complexity at the second level.5.The display apparatus of claim 4, wherein the memory further stores computer-executable instructions for controlling the one or more processors to control a frame rate above a threshold value whether the image is rendered at the first resolution range, the second resolution range, or the third resolution range.6.The display apparatus of claim 1, wherein, when the image is rendered at the second resolution range, the memory further stores computer-executable instructions for controlling the one or more processors to:monitor in real time rendering frame rate to determine if a current system performance meets rendering requirements; andupon determination that the current system performance fails to meet rendering requirements, render an image at a third resolution range;wherein the third resolution range is lower than the second resolution range.7.The display apparatus of any one of claims 1 to 6, wherein the memory further stores computer-executable instructions for controlling the one or more processors to:start one or more applications;initialize image display scene information; andrecognize the image display scene information.8.The display apparatus of any one of claims 1 to 7, further comprising a gaze point sensor configured to detect a gaze of a user;wherein the memory further stores computer-executable instructions for controlling the one or more processors to turn off the gaze point sensor upon determination that an image display scene is a first scene.9.The display apparatus of claim 8, wherein the memory further stores computer-executable instructions for controlling the one or more processors to monitor a state of charge of a power supply of the display apparatus.10.The display apparatus of claim 9, wherein the memory further stores computer-executable instructions for controlling the one or more processors to:render an image at a fourth resolution range upon determination that the state of charge of the power supply is greater than a threshold state of charge; andrender an image at a fifth resolution range upon determination that the state of charge of the power supply is equal to or less than the threshold state of charge;wherein the fourth resolution range is greater than the fifth resolution range.11.The display apparatus of any one of claims 1 to 7, further comprising an acceleration sensor configured to detect a head movement of a user;wherein the memory further stores computer-executable instructions for controlling the one or more processors to turn on an acceleration sensor to monitor head movement of the user upon determination that the image display scene is a second scene.12.The display apparatus of claim 11, wherein the memory further stores computer-executable instructions for controlling the one or more processors to:turn off a gaze point sensor and render an image at a sixth resolution range, upon determination that a speed of the head movement is greater than a first threshold speed; andturn on the gaze point sensor; and monitor a speed of eye movement, upon determination that a speed of the head movement is equal to or less than the first threshold speed.13.The display apparatus of claim 12, wherein the memory further stores computer-executable instructions for controlling the one or more processors to:render an image at a seventh resolution range and control the gaze point sensor to monitor a speed of the eye movement in a first-speed exposure mode, upon determination that the speed of the eye movement is greater than a second threshold speed; andcontrol the gaze point sensor to monitor a speed of the eye movement in a second-speed exposure mode, upon determination that the speed of the eye movement is equal to or less than a second threshold speed;wherein, in the first-speed exposure mode, the gaze point sensor is configured to monitor the speed of the eye movement at a first frequency; in the second-speed exposure mode, the gaze point sensor is configured to monitor the speed of the eye movement at a second frequency; and the second frequency is greater than the first frequency.14.The display apparatus of claim 12, wherein the memory further stores computer-executable instructions for controlling the one or more processors to perform the scene complexity detection, upon determination that the speed of the eye movement is equal to or less than a second threshold speed.15.The display apparatus of any one of claims 1 to 14, wherein the memory further stores computer-executable instructions for controlling the one or more processors to:generate a synchronization signal to synchronize operations of applications, fundamental-level components, input hardware, and output hardware of the display apparatus, wherein the fundamental-level components comprise a fundamental-level interface module, a display driving module configured to drive a display panel; a sensor driving module configured to drive a camera sensor; and a light emitting element driving module configured to drive a fill light driving circuit;transmit the synchronization signal to a synchronization signal distribution thread; anddistribute, by the synchronization signal distribution thread, the synchronization signal to one or more applications.16.The display apparatus of claim 15, wherein the memory further stores computer-executable instructions for controlling the one or more processors to:transmit, by the one or more applications, the synchronization signal to a fundamental-level interface module; andtransmit, by the fundamental-level interface module, the synchronization signal to fundamental-level component driving modules of fundamental-level components.17.The display apparatus of claim 15, further comprising the camera sensor, the fill light driving circuit, and the display panel;wherein the memory further stores computer-executable instructions for controlling the one or more processors to:distribute, by the synchronization signal distribution thread, the synchronization signal to a sensor synchronization control thread, a light emitting element synchronization control thread, and a display panel synchronization control thread;transmit, by the sensor synchronization control thread, the light emitting element synchronization control thread, and the display panel synchronization control thread, the synchronization signal to a fundamental-level interface module; andtransmit, by the fundamental-level interface module, the synchronization signal to a display driving module configured to drive the display panel; a sensor driving module configured to drive the camera sensor; and a light emitting element driving module configured to drive the fill light driving circuit.18.The display apparatus of claim 1, comprising a clock management module, an image rendering module configured to render an image, and a synchronization signal distribution thread;wherein, upon the image being rendered, the image rendering module is configured to transmit a signal to a clock management module; andupon receiving the signal, the clock management module is configured to generate a synchronization signal, and configured to transmit the synchronization signal to the synchronization signal distribution thread.19.The display apparatus of claim 1, further comprising a fill light driving circuit and a variable frequency drive module configured to modulate a pulse width modulation timing of the fill light driving circuit;wherein the variable frequency drive module is configured to modulate pulses for the fill light driving circuit to have a first pulse frequency in a first driving mode and a second pulse frequency in a second driving mode;the second pulse frequency is greater than the first pulse frequency;the first driving mode corresponds to a first-speed exposure mode of a gaze point sensor configured to monitor a speed of the eye movement;the second driving mode corresponds to a second-speed exposure mode of the gaze point sensor; andin the first-speed exposure mode, the gaze point sensor is configured to monitor the speed of the eye movement at a first frequency; in the second-speed exposure mode, the gaze point sensor is configured to monitor the speed of the eye movement at a second frequency; and the second frequency is greater than the first frequency.20.A method of display, comprising:recognizing an image display scene information;performing a scene complexity detection based on the image display scene information;grouping a scene complexity of an image display scene into multiple levels based on a type of the image display scene;rendering an image at a first resolution range upon determination that the scene complexity is at a first level; andrendering an image at a second resolution range upon determination that the scene complexity is at a second level;wherein the second resolution range is lower than the first resolution range; andthe scene complexity at the second level is higher than the scene complexity at the first level.21.A computer-program product, comprising a non-transitory tangible computer-readable medium having computer-readable instructions thereon, the computer-readable instructions being executable by a processor to cause the processor to perform:recognizing an image display scene information;performing a scene complexity detection based on the image display scene information;grouping a scene complexity of an image display scene into multiple levels based on a type of the image display scene;rendering an image at a first resolution range upon determination that the scene complexity is at a first level; andrendering an image at a second resolution range upon determination that the scene complexity is at a second level;wherein the second resolution range is lower than the first resolution range; andthe scene complexity at the second level is higher than the scene complexity at the first level.
Citation Information
Patent Citations
VR image processing method, device and system, VR equipment and storage medium
CN111711811A
Virtual scene display method and device, equipment and storage medium
CN112370784A
Virtual information rendering method and device, augmented reality device and storage medium
CN114549718A
Video recording and playback system and method
CN115623149A
Graphics rendering method and apparatus
WO2022228383A1