Head-mounted display device and display method

Abstract

The embodiment of the application discloses a head-mounted display device and a display method. The method comprises the following steps: the head-mounted display device acquires the first residual power of the head-mounted display device at the current time in response to the energy-saving instruction input by a user; the first buffer area and / or the first rendering sampling rate of the head-mounted display device are determined based on the first residual power, wherein the first buffer area is used for buffering the image to be rendered on the display screen; and the image rendering is performed based on the first buffer area and the first rendering sampling rate, so that the optical distortion and chromatic aberration of the head-mounted display device are reduced, the energy consumption of the head-mounted display device is reduced, the head-mounted display device is prevented from freezing or unexpected shutdown, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of display technology, and more particularly to a head-mounted display device and display method. Background Technology

[0002] With the development of Virtual Reality (VR) and Augmented Reality (AR) technologies, virtual reality headsets have emerged. A virtual reality headset consists of an integrated graphics system, optical system, and posture tracking system, providing users with an interactive and immersive experience.

[0003] Current virtual reality headsets' optical systems consist of one or more lenses. Optical distortion and chromatic aberration caused by these lenses can impair the user's experience when operating the headset. To address this technical issue, the industry's common approach is to not render the scene directly onto the display during rendering. Instead, an intermediate buffer is allocated, and the content to be displayed is rendered into the buffer. Then, based on the mapping relationship between pixel points and display pixels, the content in the buffer is texture-sampled at a certain sampling rate and drawn onto the display.

[0004] However, current virtual reality headsets consume a lot of power, have short battery life, and are prone to lag. Summary of the Invention

[0005] This application provides a display device and display method that reduces optical distortion and color difference in the display device while increasing its battery life and improving its smooth operation.

[0006] In a first aspect, embodiments of this application provide a head-mounted display device, comprising: a display screen configured to display an image rendered on the display screen; a user input interface configured to receive instructions input by a user; and a processor communicating with both the display screen and the user input interface, the processor being configured to:

[0007] In response to a user-inputted energy-saving command, the device obtains the first remaining power of the head-mounted display at the current moment. Based on the first remaining power, it determines the first buffer and / or the first rendering sampling rate of the head-mounted display and performs image rendering based on the first buffer and / or the first rendering sampling rate. The first buffer is used to buffer the image to be rendered onto the display screen.

[0008] Secondly, embodiments of this application provide a display method, including: responding to a user-inputted energy-saving command, obtaining a first remaining power of a head-mounted display device at the current moment; determining a first buffer and / or a first rendering sampling rate of the head-mounted display device based on the first remaining power, wherein the first buffer is used to buffer an image to be rendered onto the display screen; and performing image rendering based on the first buffer and / or the first rendering sampling rate.

[0009] Thirdly, embodiments of this application provide a display device that exists in the form of a chip. The device includes a processor and a memory. The memory is coupled to the processor to store necessary program instructions and data for the device. The processor executes the program instructions stored in the memory, causing the device to perform the display method described in any of the second aspects above.

[0010] Fourthly, embodiments of this application provide a head-mounted display device, including a processor, a memory, and a display screen; wherein the memory is used to store a computer program; the processor is used to execute the computer program to implement the display method described in any of the second aspects above; and the display screen is used to display a rendered image.

[0011] Fifthly, embodiments of this application provide a computer storage medium including computer instructions that, when executed by a computer, cause the computer to implement the display method as described in any of the second aspects.

[0012] In a sixth aspect, embodiments of this application provide a computer program product comprising a computer program stored in a readable storage medium, wherein at least one processor of a computer can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the computer to implement any of the display methods described in the second aspect.

[0013] The head-mounted display device and display method provided in this application embodiment involve the head-mounted display device responding to a user-inputted energy-saving command, acquiring the current remaining power of the head-mounted display device; determining a first buffer and / or a first rendering sampling rate based on the first remaining power, wherein the first buffer is used to buffer the image to be rendered onto the display screen; and performing image rendering based on the first buffer and the first rendering sampling rate. In other words, in this application embodiment, when the head-mounted display device responds to a user-inputted energy-saving command and adjusts the size of the buffer and / or the rendering sampling rate in real time according to the remaining power of the head-mounted display device, and performs image rendering using the adjusted buffer and / or rendering sampling rate, it reduces optical distortion and color difference in the head-mounted display device's imaging, lowers the power consumption of the head-mounted display device, prevents the head-mounted display device from lag or unexpectedly shutting down, and thus improves the user experience. Attached Figure Description

[0014] Figure 1 The diagram above exemplarily illustrates a structural schematic of a head-mounted display device in this embodiment;

[0015] Figure 2 The figure above exemplarily illustrates a hardware schematic diagram of a head-mounted display device in this embodiment;

[0016] Figure 3 The flowchart of the display method in this embodiment is shown as an example.

[0017] Figure 4 The diagram illustrates a user interface according to an embodiment of this application.

[0018] Figure 5 Another user interface diagram of an embodiment of this application is illustrated in the figure below;

[0019] Figure 6 The diagram below illustrates a process diagram of image rendering in this embodiment.

[0020] Figure 7 The diagram above illustrates the correspondence between pixels and image points in an embodiment of this application.

[0021] Figure 8 The flowchart of the display method in this embodiment is shown as an example.

[0022] Figure 9 The flowchart of the display method in this embodiment is shown as an example.

[0023] Figure 10 The diagram below exemplarily illustrates the structure of the display device in this embodiment;

[0024] Figure 11 The diagram below illustrates the structure of the head-mounted display device in this embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0026] To avoid optical distortion and chromatic aberration caused by one or more lenses, current head-mounted display devices employ the following approach: First, determine the mapping relationship between image points and screen pixels. During rendering, instead of directly rendering the scene onto the screen, allocate a buffer, render the content to be displayed into the buffer, and then, based on the mapping relationship between image points and display screen pixels, perform texture sampling on the image points in the buffer at a certain sampling rate and draw it onto the display screen.

[0027] Theoretically, the higher the image resolution of a head-mounted display, the better the distortion correction and chromatic aberration elimination, resulting in a better user experience. However, this also increases the computational load on the CPU and GPU, leading to higher power consumption. In some usage scenarios, users don't need high image quality but rather longer battery life. For example, when a user is playing a game and the battery is about to run out before completing a level, continuing to render the screen at a high resolution and sampling rate will cause the device to lag or unexpectedly shut down, undoubtedly reducing the user experience.

[0028] Based on the above considerations, in order to overcome the above-mentioned defects, this application discloses a display method that dynamically sets the size of the buffer and / or the rendering sampling rate according to the remaining power of the head-mounted display device, and performs image rendering with the adjusted buffer and / or rendering sampling rate. This can not only correct optical distortion and color difference, but also increase the battery life of the head-mounted display device, prevent the device from freezing or unexpectedly shutting down, and thus improve the user experience.

[0029] The concepts involved in this application will first be explained with reference to the accompanying drawings. It should be noted that the following explanation of each concept is only to make the content of this application easier to understand and does not imply any limitation on the scope of protection of this application.

[0030] It should be understood that the terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate, for example, to implement in a sequence other than those given in the illustrations or descriptions of the embodiments of this application. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. In the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0031] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0032] As used in this application, the term "module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0033] As used in this application, the term "gesture" refers to user behavior in which a user expresses an expected idea, action, purpose, and / or result through a change in hand shape or hand movement.

[0034] The term "hardware system" as used in the embodiments of this application can refer to a physical component with computing, control, storage, input, and output functions, composed of mechanical, optical, electrical, and magnetic devices such as integrated circuits (ICs) and printed circuit boards (PCBs). In the embodiments of this application, the hardware system is also commonly referred to as a motherboard or chip.

[0035] It is understood that the technical solutions provided in this application can be applied to scenarios such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). This application does not impose any special restrictions on the specific form of the head-mounted display device that implements the technical solution.

[0036] The head-mounted display device in this application embodiment can be a virtual reality helmet, virtual reality glasses, or it can be an augmented reality device, such as an augmented reality helmet.

[0037] Figure 1 The diagram above exemplifies the structure of a head-mounted display device in this embodiment. Figure 1 As shown, the head-mounted display device includes a body 201, a first distance sensor 202 and a second distance sensor 203 disposed on the body 201. The body 201 includes a first lens barrel 2011 and a second lens barrel 2012. A first lens 2013 is disposed at a first end of the first lens barrel 2011, and a second lens 2014 is disposed at a first end of the second lens barrel 2012. Optionally, the body 201 may also include a camera module, a motor, a display screen and a processor. In addition, the body 201 may also include other components for realizing the display function.

[0038] Both the first lens barrel 2011 and the second lens barrel 2012 include opposite ends. For the sake of distinction, the end of the head-mounted display device facing the user when it is normally worn is referred to as the first end. A first lens 2013 is fixed to the edge region of the first end of the first lens barrel 2011, and a second lens 2014 is fixed to the edge region of the first end of the second lens barrel 2012.

[0039] The first lens 2013 can be a positive focal length lens or a negative focal length lens. The first lens 2013 may include a first mirror surface and a second mirror surface. The first mirror surface faces the first end and can face the user's left eye when the head-mounted display device is worn normally. The first mirror surface can be convex, with the convex direction facing the first end. The second mirror surface is opposite to the first mirror surface and faces the inside of the head-mounted display device when it is worn normally. The second mirror surface can be concave, with the concave direction also facing the first end; alternatively, both the first and second mirror surfaces can be convex, or the first mirror surface can be concave and the second mirror surface can be convex.

[0040] The second lens 2014 is similar to the first lens 2013, that is, the second lens 2014 may include a third mirror and a fourth mirror. The third mirror faces the first end and can face the user's right eye when the head-mounted display device is worn normally. The third mirror can be convex, and the convex direction faces the first end. The fourth mirror is opposite to the third mirror and will face the inside of the head-mounted display device when it is worn normally. The fourth mirror can be concave, and the concave direction also faces the first end. Alternatively, both the first and second mirrors can be convex, or the first mirror can be concave and the second mirror can be convex.

[0041] The first distance sensor 202 and the second distance sensor 203 are symmetrically arranged on both sides of the longitudinal central axis of the main body 201. The longitudinal central axis refers to the central axis in the vertical direction of the head-mounted display device. The symmetrical arrangement means that the first distance sensor 202 and the second distance sensor 203 are respectively arranged on the left and right sides of the longitudinal central axis. The distance between the first distance sensor 202 and the longitudinal central axis and the distance between the second distance sensor 203 and the longitudinal central axis can be equal.

[0042] The first distance sensor 202 can be positioned on the left side of the longitudinal central axis to measure the distance between the left eyeball and the first lens 2013. The first distance sensor 202 can be a high-precision distance sensor, such as a laser distance sensor, infrared distance sensor, or microwave distance sensor. The second distance sensor 203 can be positioned on the right side of the longitudinal central axis to measure the distance between the right eyeball and the second lens 2014. The second distance sensor 203 can also be a high-precision distance sensor, such as a laser distance sensor, infrared distance sensor, or microwave distance sensor.

[0043] Figure 2 The diagram illustrates a hardware schematic of a head-mounted display device according to this embodiment. It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the head-mounted display device. In other embodiments of this application, the head-mounted display device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0044] like Figure 2As shown, the head-mounted display device includes two camera modules, a left camera module 150 and a right camera module 160, for capturing images of the real-world scene in front of the user. The left camera module 150 is located in front of the user's left eye, and the right camera module 160 is located in front of the user's right eye. Each pair of camera modules includes at least two cameras, of which at least one has a large field of view and at least one has a small field of view. Optionally, at least one camera with a large field of view (e.g., a field of view between 100 and 200 degrees, and a resolution between VGA and 720p); and at least one camera with a small field of view (e.g., a field of view between 20 and 60 degrees, and a resolution between 720p and 2Kp). The large field of view camera captures images with a wider field of view and lower resolution. The small field of view camera captures images with a narrower field of view and higher resolution.

[0045] In some embodiments, the head-mounted display device includes sensors for sensing the movement of the head-mounted display device, such as an inertial measurement unit (IMU) 170, used to measure the head-mounted display device's attitude information, such as three-axis attitude angles (or angular rates) and acceleration. Generally, the IMU consists of three accelerometers and three gyroscopes, mounted on mutually perpendicular measurement axes. Low-precision IMUs can also be corrected in other ways. For example, GPS can be used to correct long-term position drift, barometers can be used to correct altitude, and magnetometers can be used to correct attitude. In VR / AR / MR scenarios, the attitude information of the head-mounted display device obtained by the IMU, along with images of the real scene acquired by the camera module, can be used to locate and map the real scene in which the head-mounted display device is located.

[0046] The head-mounted display device is equipped with a left eyepiece and a right eyepiece corresponding to the left and right eyes, respectively. Inside the head-mounted display device, there are displays corresponding to the left and right eyepieces, respectively. It is understood that the field of view of the displays is not less than the field of view of the wide-angle camera in the camera module, so that the displays can fully display the image captured by the wide-angle camera. Specifically, the left eyepiece corresponds to the left display 210, and the right eyepiece corresponds to the right display 220. The left and right eyepieces help the user's eyes focus on the left and right displays, respectively. Thus, the user's left eye can view the image displayed on the left display 210 through the left eyepiece, and the user's right eye can view the image displayed on the right display 220 through the right eyepiece. It should be noted that the left display 210 and the right display 220 can be two independent displays, or different display areas within a single display; this embodiment does not limit either approach.

[0047] Generally, the cameras in the left camera module 150 and the right camera module 160 capture images simultaneously. Due to the positional difference between the left camera module 150 and the right camera module 160, there is a certain viewing angle difference in the images captured simultaneously by the two modules, which is the same as the visual difference of a human face. Therefore, when the user's eyes simultaneously view these two images on the left display 210 and the right display 220, a 3D viewing effect is produced.

[0048] The head-mounted display device may also include a processor 501, an external memory interface 502, an internal memory 503, a universal serial bus (USB) interface 504, a charging management module 505, a power management module 506, a battery 507, a wireless communication module 508, an audio module 509, a speaker 510, a receiver 511, a microphone 512, a headphone jack 513, buttons 514, a motor 515, an indicator 516, etc.

[0049] The processor 501 includes one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units can be independent devices or integrated into one or more processors.

[0050] The processor 501 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 501 is a cache memory. This memory can store instructions or data that the processor 501 has just used or that are used repeatedly. If the processor 501 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 501, and thus improves the efficiency of the system.

[0051] In some embodiments, the processor 501 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0052] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the head-mounted display device. In other embodiments of this application, the head-mounted display device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0053] The charging management module 505 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 505 receives charging input from the wired charger via a USB interface 504. In some wireless charging embodiments, the charging management module 505 receives wireless charging input via the wireless charging coil of the head-mounted display device. While charging the battery, the charging management module 505 can also supply power to the head-mounted display device via the power management module.

[0054] The power management module 506 connects the battery 507, the charging management module 505, and the processor 501. The power management module 506 receives input from the battery 507 and / or the charging management module 505, supplying power to the processor 501, internal memory, left display 210 and right display 220, camera module 150 and camera module 160, and wireless communication module, etc. The power management module 506 can also monitor parameters such as battery 507 capacity, battery 507 cycle count, and battery 507 health status (leakage current, impedance). In some other embodiments, the power management module 506 may also be located within the processor 501. In other embodiments, the power management module 506 and the charging management module 505 may be located in the same device.

[0055] The wireless communication module 508 can provide solutions for wireless communication applications in head-mounted display devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 508 can be one or more devices integrating at least one communication processing module. The wireless communication module 508 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to the processor 501. The wireless communication module 508 can also receive signals to be transmitted from the processor 501, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via the antenna.

[0056] The head-mounted display device implements display functions through a GPU, a left display 210 and a right display 220, and an application processor. The GPU is a microprocessor for image processing, connected to the left display 210, the right display 220, and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 501 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0057] In some embodiments of this application, the head-mounted display device can be an integrated helmet-mounted display (HMD), that is, the head-mounted display device can perform data processing on the acquired images and the posture information of the head-mounted display device through CPU, GPU, NPU, application processor, etc., for example: preprocessing low-resolution images captured by a wide field of view camera and high-resolution images captured by a narrow field of view camera, and synthesizing images to obtain images of real scenes; generating images containing virtual objects; and synthesizing images containing virtual objects with images of real scenes, etc.

[0058] In some embodiments of this application, the head-mounted display device can be a split-type HMD, meaning that the head-mounted display device can delegate all or part of the data processing, such as the acquired images and the head-mounted display device's posture information, to other electronic devices or apparatuses. For example, the head-mounted display device can send the acquired high-definition and low-definition images, as well as the acquired posture information, to another connected electronic device (e.g., a mobile phone, personal computer, tablet computer, etc.) via, for example, a USB interface 504. This electronic device processes the data and then returns the processing results to the head-mounted display device for display. In this example, the head-mounted display device may not include any one or more of the following hardware devices: GPU, NPU, application processor, etc.

[0059] The left display 210 is used to display images, videos, etc., corresponding to the left eye, and the right display 220 is used to display images, videos, etc., corresponding to the right eye. Both the left display 210 and the right display 220 can include display panels. The display panels can be liquid crystal displays (LCDs), organic light-emitting diodes (OLEDs), active-matrix organic light-emitting diodes (AMOLEDs), flexible light-emitting diodes (FLEDs), miniature LEDs, microLEDs, micro-OLEDs, quantum dot light-emitting diodes (QLEDs), etc.

[0060] The head-mounted display device can achieve shooting functions through an ISP, camera modules 150 and 160, video codecs, GPU, left display 210 and right display 220, and application processor.

[0061] The Information Service Provider (ISP) is used to process data fed back from camera modules 150 and 160. For example, when a camera module captures an image, light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimizations on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into camera modules 150 and 160.

[0062] Both camera modules 150 and 160 can be used to capture still images or videos. An object is projected onto a photosensitive element through the lens, generating an optical image. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into standard RGB, YUV, or other image formats.

[0063] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when a head-mounted display device is selecting a frequency, a DSP is used to perform Fourier transforms on the frequency energy.

[0064] Video codecs are used to compress or decompress digital video. Head-mounted display devices can support one or more video codecs. This allows the head-mounted display device to play or record video in various encoded formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0065] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can be used to enable intelligent cognitive applications in head-mounted displays, such as image recognition, facial recognition, speech recognition, and text understanding.

[0066] The external memory interface 502 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the head-mounted display device. The external memory card communicates with the processor 501 through the external memory interface 502 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0067] Internal memory 503 can be used to store computer executable program code, which includes instructions. Internal memory 503 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the head-mounted display device (such as audio data, phonebook, etc.). Furthermore, internal memory 503 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 501 executes various functional applications and data processing of the head-mounted display device by running instructions stored in internal memory 503 and / or instructions stored in memory disposed in processor 501.

[0068] The head-mounted display device can implement audio functions, such as music playback and recording, through an audio module 509, a speaker 510, a receiver 511, a microphone 512, a headphone jack 513, and an application processor.

[0069] The audio module 509 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 509 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 509 may be located in the processor 501, or some functional modules of the audio module 509 may be located in the processor 501.

[0070] The speaker 510, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Head-mounted display devices can listen to music or make hands-free calls through the speaker 510.

[0071] The receiver 511, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the head-mounted display device is used to answer phone calls or voice messages, the receiver 511 can be brought close to the ear to hear the voice.

[0072] Microphone 512, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When collecting voice information, the user can speak by bringing their mouth close to microphone 512, inputting the sound signal into microphone 512. A head-mounted display device can be equipped with at least one microphone 512. In some embodiments, the head-mounted display device can be equipped with two microphones 512, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, the head-mounted display device can also be equipped with three, four, or more microphones 512, enabling sound signal collection, noise reduction, sound source identification, and directional recording functions, etc.

[0073] The headphone jack 513 is used to connect wired headphones. The headphone jack 513 can be a USB interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0074] Buttons 514 include a power button, volume buttons, etc. Buttons 514 can be mechanical buttons or touch-sensitive buttons. The head-mounted display device can receive input from buttons 514, generating key signal inputs related to user settings and function control of the head-mounted display device.

[0075] Motor 515 can generate vibration alerts. Motor 515 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can be corresponding to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 515 can also correspond to different vibration feedback effects for touch operations performed on different areas of the left display 210 and right display 220. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0076] Indicator 516 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0077] Based on the head-mounted display device described above, the display method described in the embodiments of this application will be introduced below with reference to specific examples.

[0078] Figure 3 The diagram below illustrates a flowchart of the method shown in this embodiment. This method can be... Figure 1 and Figure 2The head-mounted display device shown can be used to execute this, specifically by... Figure 2 The processor in the head-mounted display device shown executes the commands. For example... Figure 3 As shown, the display method of this application embodiment includes:

[0079] S101. In response to the energy-saving command input by the user, obtain the first remaining power of the head-mounted display device at the current moment.

[0080] In this application embodiment, the methods by which the user inputs energy-saving commands to the head-mounted display device include, but are not limited to, the following:

[0081] In one embodiment of this application, the head-mounted display device includes two modes: a performance mode and an energy-saving mode. The performance mode prioritizes performance during operation. For example, in performance mode, the head-mounted display device uses the largest possible intermediate buffer and the highest possible sampling rate. While the display effect is good in performance mode, the corresponding power consumption is high. The energy-saving mode, on the other hand, increases battery life while maintaining display quality.

[0082] For example, a head-mounted display device displays as follows Figure 4 The interface shown allows users to... Figure 4 Click the "Energy Saving Mode" option on the interface shown to input energy saving commands to the head-mounted display device.

[0083] For example, the head-mounted display device can have physical buttons on its housing corresponding to performance mode and energy-saving mode. Users can input energy-saving commands to the head-mounted display device by pressing the physical button corresponding to the energy-saving mode.

[0084] For example, users can also input energy-saving commands to the head-mounted display device via voice or gestures, and this application embodiment does not limit this.

[0085] Method Two: The head-mounted display device monitors its remaining battery power in real time. When the remaining battery power is detected to be lower than a preset value, such as below 50%, the head-mounted display device controls the screen to display a prompt message. This prompt message is used to encourage the user to select to enter energy-saving mode. Figure 5 As shown, the prompt message could be "Enter power saving mode?", which the user can then select. Figure 5 Select the "Yes" option shown to complete the input of energy-saving commands to the head-mounted display device.

[0086] After receiving a power-saving command from the user, the head-mounted display device responds by entering power-saving mode and obtains the first remaining battery power of the head-mounted display device at the current moment.

[0087] S102. Based on the first remaining battery power, determine the first buffer and / or the first rendering sampling rate of the head-mounted display device.

[0088] The first buffer mentioned above is used to buffer the image to be rendered onto the display screen.

[0089] As mentioned above, head-mounted displays typically incorporate lenses between the user's eyes and the screen. Optical distortion and chromatic aberration caused by one or more lenses can impair the user's experience when operating the head-mounted display. To reduce distortion and chromatic aberration in head-mounted display imaging, the current approach is to not directly render the image onto the screen during rendering. Instead, an intermediate buffer is allocated, the content to be displayed is first rendered into the buffer, and then the buffer is texture-sampled at a certain rendering sampling rate before being drawn onto the screen.

[0090] In practical applications, it has been found that the larger the buffer, the more power the head-mounted display device consumes, and the higher the rendering sampling rate, the more power the head-mounted display device consumes.

[0091] Based on this, when the head-mounted display device enters the power-saving mode, in order to reduce optical distortion and color difference while increasing the battery life of the head-mounted display device, the embodiments of this application determine the first buffer and / or the first rendering sampling rate based on the first remaining power of the head-mounted display device at the current moment.

[0092] Specifically, the size of the buffer is positively correlated with the remaining battery power; that is, the larger the remaining battery power, the larger the corresponding buffer. Similarly, the rendering sampling rate is also positively correlated with the remaining battery power; that is, the larger the remaining battery power, the higher the corresponding rendering sampling rate. Thus, the head-mounted display device can determine its first buffer and / or first rendering sampling rate based on its current remaining battery power.

[0093] In this embodiment of the application, the methods for determining the first buffer include, but are not limited to, the following:

[0094] Method 1: The head-mounted display device pre-allocates multiple buffers of different sizes. When rendering an image, based on the current remaining battery power of the head-mounted display device, it selects a buffer from the pre-allocated buffers of different sizes that corresponds to the current remaining battery power as the first buffer. For example, the pre-allocated buffers of the head-mounted display device, from largest to smallest, are: Buffer 1, Buffer 2, and Buffer 3. In the previous image rendering, the head-mounted display device selected Buffer 1, and the remaining battery power of the head-mounted display device in the previous moment was 80%. The current remaining battery power is 60%. Since the current remaining battery power is less than the previous remaining battery power, in order to save energy, the head-mounted display device selects a buffer smaller than Buffer 1 from the above three buffers as the first buffer for the current moment, for example, selecting Buffer 2 as the first buffer.

[0095] Method Two: Instead of pre-allocating multiple different buffers, the head-mounted display device allocates a buffer corresponding to the current remaining battery level as the first buffer. During the next image rendering, if the buffer corresponding to the remaining battery level of the head-mounted display device changes, the previously allocated first buffer is destroyed, and a new buffer is allocated.

[0096] The size of the buffer in this embodiment is related to the resolution of the area visible to the human eye through the lens. For example, if a user can see 4096x4096 pixels on the display screen through the lens with one eye, the size of the first buffer is 4096x4096 (4K). If a user can see 2048x2048 pixels on the display screen through the lens with one eye, the size of the first buffer is 2048x2048 (2K), and so on.

[0097] In order to extend the battery life of the head-mounted display device, this embodiment of the application selects a smaller first buffer when the battery level of the head-mounted display device is detected to reduce the amount of data processed by the CPU and GPU, thereby reducing the power consumption of the head-mounted display device during image rendering, thus extending the battery life of the head-mounted display device and alleviating the phenomenon of display device lag.

[0098] For example, in this embodiment of the application, the rendering sampling rate of the head-mounted display device changes in a preset step size. For instance, the preset step size of the rendering sampling rate is c, where c is a positive number greater than 0. The remaining battery power of the head-mounted display device when rendering the previous image is a1, and the used rendering sampling rate is b1. The first remaining battery power of the head-mounted display device when rendering the current image is a2, where a2 is less than a1. Therefore, the first rendering sampling rate of the head-mounted display device at the current moment can be determined as b2 = b1 - n * c, where n is a positive number greater than 1. In this way, when the battery power of the head-mounted display device decreases, using a lower rendering sampling rate for image rendering can reduce the power consumption of the head-mounted display device, extend its battery life, and alleviate display device stuttering.

[0099] S103. Render the image based on the first buffer and / or the first rendering sampling rate.

[0100] According to the above steps, the head-mounted display device determines the first buffer and / or the first rendering sampling rate of the head-mounted display device at the current moment based on the first remaining power of the head-mounted display device at the current moment, and performs image rendering according to the first buffer and / or the first rendering sampling rate, so as to extend the battery life of the head-mounted display device and prevent the head-mounted display device from lag or unexpected shutdown.

[0101] In some embodiments, such as Figure 6 As shown, the image rendering based on the first buffer and / or the first rendering sampling rate in S103 above may include the following steps:

[0102] S1031. Render the image to be rendered into the first buffer.

[0103] In this embodiment of the application, in order to reduce optical distortion and color difference in imaging of the head-mounted display device, the image to be rendered is first rendered into the first buffer.

[0104] S1032. Based on the mapping relationship between display pixels and image points, the image points in the first buffer are texture sampled using the first rendering sampling rate.

[0105] The mapping relationship between display screen pixels and image points used in the embodiments of this application can be obtained through the Brown distortion model:

[0106] Xd(Yd)=Xu(Yu)*(K0+K1*r+K2*r 2 +K3*r 3 +K4*r 4 +K5*r 5 +K6*r 6 )

[0107] Where (Xu, Yu) are the pixel coordinates of the display screen, (Xd, Yd) are the corresponding image coordinates, and K1...K6 are the distortion coefficients, which are related to the optical characteristics of the lens.

[0108] Optionally, the mapping relationship between display screen pixels and image points used in the embodiments of this application can also be obtained through optical simulation experiments. The embodiments of this application do not limit the method for determining the mapping relationship between display screen pixels and image points.

[0109] In one possible implementation, the head-mounted display device performs texture sampling on all pixels in the first buffer using a first rendering sampling rate.

[0110] In another possible implementation, the head-mounted display device uses a preset maximum rendering sampling rate to perform texture sampling on the pixels corresponding to the first region in the first buffer, and uses the first rendering sampling rate to perform texture sampling on the pixels corresponding to the second region in the first buffer. Here, the first region is a preset area of ​​user interest on the display screen, and the second region is the area on the display screen excluding the first region. For example, the first region could be the area with a ±30° field of view on the display screen.

[0111] Optionally, the maximum preset rendering sampling rate can be 50, that is, sampling once every 50 pixels horizontally and vertically.

[0112] S1033. Interpolate and fill the pixels between key pixels to display the corrected image on the display screen.

[0113] For example, assuming the first rendering sampling rate is 50, in the previous sampling process, such as... Figure 7 As shown, based on the mapping relationship between display screen pixels and image points, the image point coordinates corresponding to the display screen pixels with coordinates (0,0), (0,50), (50,0), and (50,50) are (0,0), (10,50), (50,0), and (40,50), respectively. This allows the square area enclosed by (0,0), (10,50), (50,0), and (40,50) in the first buffer to be filled into the square area enclosed by (0,0), (0,50), (50,0), and (50,50) on the display screen, ultimately displaying the corrected image on the display screen.

[0114] In this embodiment, the image to be rendered is first rendered into a first buffer. Then, based on the mapping relationship between the display screen pixels and image points, the image points in the first buffer are texture sampled using a first rendering sampling rate. Finally, the pixels between key pixels are interpolated and filled, thereby correcting the displayed image, reducing optical distortion and color difference in head-mounted display devices, and improving the user display experience.

[0115] The display method provided in this application embodiment, in response to a user-inputted energy-saving command, acquires the first remaining power of the head-mounted display device at the current moment; based on the first remaining power, determines a first buffer and / or a first rendering sampling rate of the head-mounted display device, the first buffer being used to buffer the image to be rendered onto the display screen; and performs image rendering based on the first buffer and the first rendering sampling rate. That is, in this application embodiment, when the head-mounted display device, in response to a user-inputted energy-saving command, adjusts the size of the buffer and / or the rendering sampling rate of the head-mounted display device in real time according to the remaining power, and uses the adjusted buffer and / or rendering sampling rate for image rendering, it reduces optical distortion and color difference in the head-mounted display device's imaging, lowers the power consumption of the head-mounted display device, prevents the head-mounted display device from lag or unexpectedly shutting down, and thus improves the user experience.

[0116] Figure 8 The diagram illustrates a flowchart of the display method in this embodiment. Based on the above embodiments, this application describes an implementation of determining a first buffer and / or a first rendering sampling rate of a head-mounted display device based on a first remaining battery power. Figure 8 As shown, the method in this application embodiment includes:

[0117] S201. In response to the energy-saving command input by the user, obtain the first remaining power of the head-mounted display device at the current moment.

[0118] The implementation process of S201 is the same as that of S101. Referring to the specific description of S101, it will not be repeated here.

[0119] S202. Based on the correspondence between the remaining power of the head-mounted display device and the buffer, obtain the second buffer corresponding to the first remaining power.

[0120] In this embodiment of the application, as shown in Table 1, there is a corresponding relationship between the remaining power of the head-mounted display device and the buffer. It should be noted that Table 1 is an example of the correspondence between the remaining power and the buffer. The correspondence between the remaining power and the buffer involved in this application includes, but is not limited to, the one shown in Table 1. The specific correspondence can be determined according to actual needs.

[0121] Table 1

[0122] Remaining battery power Buffer 80％～100％ 8192x8192(8K) 20％～80％ 4096x4096 (4K) 10％～20％ 2048x2048 (2K) 0％～10％ 1024x1024(1K)

[0123] As shown in Table 1 above, there is a corresponding relationship between the remaining power of the head-mounted display device and the buffer of different sizes. Based on this relationship, the second buffer corresponding to the first remaining power of the head-mounted display device at the current moment can be obtained. For example, if the first remaining power of the head-mounted display device is 60%, based on the relationship described in Table 1, the second buffer of the head-mounted display device can be determined to be 4096x4096 (4K).

[0124] S203. The second buffer is designated as the first buffer.

[0125] In this embodiment of the application, the head-mounted display device obtains the second buffer corresponding to the first remaining power from the correspondence between the remaining power of the head-mounted display device and the buffer area, and uses the second buffer as the first buffer of the head-mounted display device.

[0126] It should be noted that the above steps S202 and S203 are related to the following steps S204 and S205 in an AND / OR relationship. When the above steps S202 and S203 are related to the following steps S204 and S205 in an AND relationship, the above steps S202 and S203 are not executed in any order with the following steps S204 and S205. That is, the above steps S202 and S203 can be executed before steps S204 and S205, or after steps S204 and S205, or simultaneously with steps S204 and S205.

[0127] S204. Based on the correspondence between the remaining power of the head-mounted display device and the rendering sampling rate, obtain the second rendering sampling rate corresponding to the first remaining power.

[0128] In this embodiment of the application, as shown in Table 2, there is also a corresponding relationship between the remaining battery power of the head-mounted display device and the rendering sampling rate. It should be noted that Table 2 is an example of the correspondence between the remaining battery power and the rendering sampling rate. The correspondence between the remaining battery power and the rendering sampling rate involved in this application includes, but is not limited to, the one shown in Table 2, and can be determined according to actual needs.

[0129] Table 2

[0130] Remaining battery power Rendering sampling rate 80％～100％ 50 20％～80％ 100 10％～20％ 150 0％～10％ 200

[0131] As shown in Table 2 above, there is a corresponding relationship between the remaining battery power of the head-mounted display device and different rendering sampling rates. Therefore, the head-mounted display device can obtain the second rendering sampling rate corresponding to its first remaining battery power at the current moment based on the relationship shown in Table 2. For example, if the head-mounted display device's first remaining battery power is 60%, based on the relationship described in Table 2, the second rendering sampling rate of the head-mounted display device can be determined to be 100, meaning that sampling is performed once every 100 pixels horizontally and vertically.

[0132] S205. The second rendering sampling rate is determined as the first rendering sampling rate.

[0133] In this embodiment of the application, the head-mounted display device obtains the second rendering sampling rate corresponding to the first remaining power from the correspondence between the remaining power of the head-mounted display device and the rendering sampling rate, and uses the second rendering sampling rate as the first rendering sampling rate of the head-mounted display device.

[0134] S206. Render the image based on the first buffer and / or the first rendering sampling rate.

[0135] The specific implementation process of S206 is the same as that of S103. Referring to the specific description of S103, it will not be repeated here.

[0136] The display method provided in this application embodiment involves a head-mounted display device obtaining a second buffer corresponding to a first remaining battery level based on the correspondence between the remaining battery power of the head-mounted display device and a buffer area, and determining the second buffer as the first buffer; and / or, the head-mounted display device obtaining a second rendering sampling rate corresponding to the first remaining battery level based on the correspondence between the remaining battery power of the head-mounted display device and a rendering sampling rate, and determining the second rendering sampling rate as the first rendering sampling rate. In this application embodiment, the process of determining the first buffer and / or the first rendering sampling rate is simple and easy to operate.

[0137] Figure 9 The diagram illustrates a flowchart of the display method in this embodiment. Based on the above embodiments, this application describes another implementation of determining the first buffer and / or first rendering sampling rate of a head-mounted display device based on a first remaining battery power. Figure 9 As shown, the method in this application embodiment includes:

[0138] S301. In response to the energy-saving command input by the user, obtain the first remaining power of the head-mounted display device at the current moment.

[0139] The specific implementation process of S301 is the same as that of S101. Referring to the specific description of S101, it will not be repeated here.

[0140] S302, In response to the energy-saving command input by the user, obtain the rendering frame rate of the head-mounted display device when rendering multiple frames of images.

[0141] Rendering frame rate refers to the number of frames rendered per second. The higher the rendering frame rate of a head-mounted display device, the more data the CPU processes, and the more power the device consumes.

[0142] Based on this, in this embodiment of the application, after the head-mounted display device receives the energy-saving command input by the user, the startup process not only monitors the remaining power of the head-mounted display device in real time, but also monitors the rendering frame rate of the head-mounted display device in real time.

[0143] S303. Based on the correspondence between the remaining power of the head-mounted display device and the buffer, obtain the second buffer corresponding to the first remaining power.

[0144] The specific implementation process of S303 is the same as that of S202. Referring to the specific description of S202, it will not be repeated here.

[0145] S304. Based on the correspondence between the remaining power of the head-mounted display device and the rendering sampling rate, obtain the second rendering sampling rate corresponding to the first remaining power.

[0146] The specific implementation process of S304 is the same as that of S204. Referring to the specific description of S204, it will not be repeated here.

[0147] S305. If the rendering frame rate corresponding to each frame in the multi-frame image is lower than the preset value, then the third buffer is obtained according to the second buffer and the third buffer is used as the first buffer, and / or, if the rendering frame rate corresponding to each frame in the multi-frame image is lower than the preset value, then the third rendering sampling rate is obtained according to the second rendering sampling rate and the third rendering sampling rate is used as the first rendering sampling rate.

[0148] The size of the third buffer is smaller than the size of the second buffer, and the third rendering sampling rate is smaller than the second rendering sampling rate.

[0149] The above preset values ​​can be set according to actual needs. Optionally, the above-mentioned corrupted value is, for example, 60 frames per second.

[0150] If, during the rendering of multiple frames of images, the head-mounted display (HMD) determines that the rendering frame rate for each frame is lower than a preset value, it indicates that the HMD may be experiencing stuttering due to low battery, causing the rendering frame rate to fall below the preset value. To resolve this stuttering issue, a third buffer smaller than the second buffer can be selected as the first buffer, and / or a third rendering sampling rate smaller than the second rendering sampling rate can be selected as the first rendering sampling rate.

[0151] In one possible implementation, as shown in Table 1, the buffer of the head-mounted display device includes multiple levels from large to small, and the third buffer is the next level of the second buffer. For example, if the second buffer obtained by S303 above is 2048x2048 (2K), then the third buffer is 1024x1024 (1K).

[0152] As shown in Table 2, the rendering sampling rate of the head-mounted display device includes multiple levels from large to small. The third rendering sampling rate is the next level of the second rendering sampling rate. For example, if the second rendering sampling rate is 150, then the third rendering sampling rate is 200.

[0153] According to the above steps, the head-mounted display device obtains a third buffer and uses this third buffer as the first buffer of the head-mounted display device, and / or obtains a third rendering sampling rate and uses this third rendering sampling rate as the first rendering sampling rate of the head-mounted display device. Then, the head-mounted display device performs image rendering based on the first buffer and the first rendering sampling rate.

[0154] S306. Render the image based on the first buffer and / or the first rendering sampling rate.

[0155] The specific implementation process of S306 is the same as that of S103. Referring to the specific description of S103, it will not be repeated here.

[0156] The display method provided in this application embodiment involves a head-mounted display device responding to a user-inputted energy-saving command. The device obtains a first remaining battery level and a rendering frame rate when rendering multiple frames of images. Then, based on the correspondence between the remaining battery level and a buffer, the device obtains a second buffer corresponding to the first remaining battery level. Based on the correspondence between the remaining battery level and a rendering sampling rate, the device obtains a second rendering sampling rate corresponding to the first remaining battery level. If the rendering frame rate for each frame in the multiple images is lower than a preset value, the device obtains a third buffer based on the second buffer and uses the third buffer as the first buffer. Alternatively, the device obtains a third rendering sampling rate based on the second rendering sampling rate and uses the third rendering sampling rate as the first rendering sampling rate. In other words, this application embodiment considers the rendering frame rate of the head-mounted display device when determining the first buffer and / or the first rendering sampling rate. This allows for further extension of the head-mounted display device's battery life when rendering images based on the first buffer and / or the first rendering sampling rate, preventing stuttering or unexpected shutdowns.

[0157] Figure 10 The diagram below illustrates the structure of the display device in this embodiment. The display device 700 exists in the form of a chip. The structure of the head-mounted display device includes a processor 701 and a memory 702. The memory 702 is used to couple with the processor 701. The memory 702 stores the necessary program instructions and data of the device. The processor 701 is used to execute the program instructions stored in the memory 702, so that the device performs the functions of the head-mounted display device in the above method embodiment.

[0158] The display device in this application embodiment can be used to execute the technical solutions of the head-mounted display devices in the above method embodiments. The implementation principle and technical effect are similar, and will not be repeated here.

[0159] Figure 11 The diagram below exemplifies the structure of the head-mounted display device in this embodiment. Figure 11 As shown, the head-mounted display device includes a display control unit 12, a left-eye display system 11a, and a right-eye display system 11b. The display control unit 12 may include a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement the display method provided in this application embodiment.

[0160] The computer equipment mentioned above can be a server, such as a cloud server, or an electronic device, such as a smartphone, smartwatch, or tablet computer. This embodiment does not limit the specific form of the computer equipment.

[0161] The display control unit 12 is presented in the form of a general-purpose computing device. The components of the display control unit 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0162] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0163] The display control unit 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the display control unit 12, including volatile and non-volatile media, removable and non-removable media.

[0164] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 41 and / or cache memory 42. Display control unit 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 44 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 11 Not shown; usually referred to as a "hard drive"). Although Figure 11 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media). In these cases, each drive can be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0165] A program / utility 40 having a set (at least one) of program modules 52 may be stored, for example, in memory 28. Such program modules 52 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 52 typically perform the functions and / or methods described in the embodiments of this application.

[0166] The display control unit 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the display control unit 12, and / or any device that enables the display control unit 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via the input / output (I / O) interface 22. Furthermore, the display control unit 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via the network adapter 20. Figure 11As shown, network adapter 20 communicates with other modules of display control unit 12 via bus 18. It should be understood that, although... Figure 11 As not shown, it can be used in conjunction with computer equipment with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0167] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the display method provided in the embodiments of this application.

[0168] This application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, can implement the display method provided in this application.

[0169] The aforementioned non-transitory computer-readable storage medium may be any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in connection with an instruction execution system, apparatus, or device.

[0170] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0171] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0172] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0173] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing device embodiments, and will not be repeated here. Furthermore, reference can be made between different device embodiments, and identical or corresponding content in different embodiments can be mutually referenced, without further elaboration.

Claims

1. A head-mounted display device, characterized in that, include: The display screen is configured to display an image rendered onto the display screen; The user input interface is configured to receive user input commands. A processor that communicates with both the display screen and the user input interface is configured to: In response to the energy-saving command input by the user, the first remaining power of the head-mounted display device at the current moment is obtained. Based on the first remaining power, the first buffer and the first rendering sampling rate of the head-mounted display device are determined, and image rendering is performed based on the first buffer and the first rendering sampling rate. The first buffer is used to buffer the image to be rendered onto the display screen, so that the image points in the first buffer are texture sampled according to the mapping relationship between image points and display screen pixels using the first rendering sampling rate.

2. The head-mounted display device according to claim 1, characterized in that, The remaining battery power of the head-mounted display device is correlated with the cache area and the rendering sampling rate, respectively, and the processor is further configured as follows: Based on the correspondence between the remaining power of the head-mounted display device and the buffer area, a second buffer corresponding to the first remaining power is obtained, and the first buffer is determined based on the second buffer. And / or, Based on the correspondence between the remaining power of the head-mounted display device and the rendering sampling rate, the second rendering sampling rate corresponding to the first remaining power is obtained, and the first rendering sampling rate is determined based on the second rendering sampling rate.

3. The head-mounted display device according to claim 2, characterized in that, The processor is also configured to: In response to the energy-saving command input by the user, the rendering frame rate of the head-mounted display device when rendering multiple frames of images is obtained; If the rendering frame rate corresponding to each frame in the multi-frame image is lower than a preset value, then a third buffer is obtained based on the second buffer, and the third buffer is used as the first buffer; and / or, if the rendering frame rate corresponding to each frame in the multi-frame image is lower than a preset value, then a third rendering sampling rate is obtained based on the second rendering sampling rate, and the third rendering sampling rate is used as the first rendering sampling rate. The size of the third buffer is smaller than the size of the second buffer, and the third rendering sampling rate is smaller than the second rendering sampling rate.

4. The head-mounted display device according to claim 3, characterized in that, The buffer of the head-mounted display device includes multiple levels from largest to smallest, and the third buffer is the next level buffer after the second buffer; and / or, The rendering sampling rate of the head-mounted display device includes multiple levels from large to small, and the third rendering sampling rate is the next level of the second rendering sampling rate.

5. The head-mounted display device according to any one of claims 2-4, characterized in that, The processor is further configured to: Using a preset maximum rendering sampling rate, texture sampling is performed on the image points in the first buffer corresponding to the first region, and the same applies to the image points in the first buffer corresponding to the second region. The first area is the preset area of ​​user focus on the display screen, and the second area is the area on the display screen other than the first area.

6. The head-mounted display device according to any one of claims 1-4, characterized in that, The processor is also configured to: When the battery level of the head-mounted display device is detected to be lower than a preset value, the display screen is controlled to show a prompt message, which prompts the user to select to enter the power-saving mode.

7. A display method, characterized in that, include: In response to the user's input energy-saving command, obtain the first remaining power of the head-mounted display device at the current moment; Based on the first remaining battery power, a first buffer and a first rendering sampling rate are determined for the head-mounted display device. The first buffer is used to buffer the image to be rendered onto the display screen. Image rendering is performed based on the first buffer and the first rendering sampling rate. The first buffer is used to buffer the image to be rendered onto the display screen, so as to perform texture sampling on the image points in the first buffer according to the mapping relationship between image points and display screen pixels using the first rendering sampling rate.

8. The method according to claim 7, characterized in that, The remaining battery power of the head-mounted display device is respectively related to the buffer and the rendering sampling rate. Determining the first buffer and / or the first rendering sampling rate of the head-mounted display device based on the first remaining battery power includes: Based on the correspondence between the remaining power of the head-mounted display device and the buffer area, a second buffer corresponding to the first remaining power is obtained, and the first buffer is determined based on the second buffer. And / or, Based on the correspondence between the remaining power of the head-mounted display device and the rendering sampling rate, the second rendering sampling rate corresponding to the first remaining power is obtained, and the first rendering sampling rate is determined based on the second rendering sampling rate.

9. The method according to claim 8, characterized in that, The method further includes: In response to the energy-saving command input by the user, the rendering frame rate of the head-mounted display device when rendering multiple frames of images is obtained; Determining the first buffer based on the second buffer, and / or determining the first rendering sampling rate based on the second rendering sampling rate, includes: If the rendering frame rate corresponding to each frame in the multi-frame image is lower than a preset value, then a third buffer is obtained based on the second buffer, and the third buffer is used as the first buffer; and / or, if the rendering frame rate corresponding to each frame in the multi-frame image is lower than a preset value, a third rendering sampling rate is obtained based on the second rendering sampling rate, and the third rendering sampling rate is used as the first rendering sampling rate. The size of the third buffer is smaller than the size of the second buffer, and the third rendering sampling rate is smaller than the second rendering sampling rate.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to implement the display method as described in any one of claims 7 to 9.

Images