Display method and electronic device

Abstract

A display method and an electronic device. The method comprises: displaying N frames of images by a display device; a first object on an i-th frame of the N frames of images has a first definition; the first object on a j-th frame of the N frames of images has a second definition; the first object on a k-th frame of the N frames of images has a third definition; wherein the first definition is greater than the second definition, the second definition is less than the third definition, i, j, k are all positive integers less than N, i < j < k; an average of the first definition and the second definition is a fourth definition, the fourth definition is higher than a first threshold; an average of the second definition and the third definition is a fifth definition, the fifth definition is higher than a second threshold. In this way, the definition of the virtual environment seen by the user through the display device is higher, which helps to improve the virtual reality experience.

Description

Technical Field

[0001] This application relates to the field of electronic technology, and more particularly to a display method and electronic device. Background Technology

[0002] Virtual Reality (VR) technology is a human-computer interaction method created using computer and sensor technologies. VR technology integrates various scientific technologies such as computer graphics, computer simulation, sensor technology, and display technology to create virtual environments. Users immerse themselves in these virtual environments by wearing VR wearable devices (such as VR glasses). The virtual environment is presented by continuously refreshing many three-dimensional images, which include objects at different depths of field, giving users a sense of three-dimensionality.

[0003] In VR technology, true immersion and a high-quality experience are related to image resolution; the higher the resolution, the better the experience. However, increased resolution means higher demands on performance and power consumption. Most devices cannot meet these high power consumption requirements, especially portable and compact standalone mobile devices (such as VR glasses), which are even more difficult to meet due to their heat dissipation characteristics and limited battery capacity. Summary of the Invention

[0004] The purpose of this application is to provide a display method and electronic device that can ensure a VR experience while reducing power consumption.

[0005] Firstly, a display method is provided, applicable to electronic devices, such as VR devices, AR devices, MR devices, etc. The method includes: displaying N frames of images via a display device; N being a positive integer; the sharpness of a first object in the i-th frame of the N frames being a first sharpness; the sharpness of the first object in the j-th frame of the N frames being a second sharpness; and the sharpness of the first object in the k-th frame of the N frames being a third sharpness; wherein the first sharpness is greater than the second sharpness, the second sharpness is less than the third sharpness, i, j, and k are all positive integers less than N, i < j < k; the average of the first sharpness and the second sharpness is a fourth sharpness, the fourth sharpness being higher than a first threshold; and the average of the second sharpness and the third sharpness is a fifth sharpness, the fifth sharpness being higher than the second threshold.

[0006] In this embodiment, the sharpness of the first object in different frames of the image stream displayed by the display device can be switched between high and low. For example, the first object has high sharpness in the i-th frame, low sharpness in the j-th frame, and high sharpness in the k-th frame. This avoids the need for the first object to maintain high sharpness in every frame, thus saving power. Furthermore, due to the persistence of vision, when the human eye views an image stream, the brain fuses the different images in the stream. That is, what the human eye perceives is the fused image of different frames, and the resolution of the fused image is the average sharpness of the fused images. In this embodiment, since the average sharpness of the first object in different frames is higher than a threshold, the sharpness of the first object seen by the human eye when fusing different frames is higher than the threshold. For example, if the average sharpness of the first object in the i-th and j-th frames is higher than a first threshold, the sharpness of the first object seen by the human eye when fusing the i-th and j-th frames is higher than the first threshold. For example, if the average sharpness of the first object in the j-th frame and the k-th frame is higher than the second threshold, then when the human eye merges the j-th frame and the k-th frame, the sharpness of the first object appears higher than the second threshold. Thus, the user sees a higher level of clarity in the virtual environment through the display device, which helps improve the visual experience.

[0007] In one possible design, the first threshold is a user-defined value, and / or the second threshold is a user-defined value. That is, the user can define a resolution threshold (i.e., the first threshold and the second threshold) for the first object in the virtual environment displayed by the electronic device. This ensures that the resolution of the first object in the virtual environment seen by the user through the electronic device is not lower than the threshold, thus improving the visual experience. Furthermore, the user can define the resolution threshold according to their own needs, further enhancing the user experience.

[0008] In one possible design, the first threshold is the average of the sixth and seventh sharpness values; wherein the sixth sharpness is the sharpness of the first object on the rendered i-th frame original image, and the i-th frame image is the processed image of the i-th frame original image; the sixth sharpness is less than the first sharpness; the seventh sharpness is the sharpness of the first object on the rendered j-th frame original image, and the j-th frame image is the processed image of the j-th frame original image; the seventh sharpness is less than or equal to the second sharpness. That is, after the electronic device renders N frames of original images, it can display these N frames, and the sharpness of the virtual environment seen by the user is sharpness 1. If, after the electronic device renders N frames of original images, it processes these N frames to improve their sharpness, and the electronic device displays the processed N frames, the sharpness of the virtual environment seen by the user is sharpness 2, which is greater than sharpness 1, thus helping to improve the user's virtual reality experience.

[0009] In one possible design, the second threshold is the average of the eighth and ninth sharpness levels; wherein the eighth sharpness is the sharpness of the first object on the rendered j-th frame original image, and the j-th frame image is the processed version of the j-th frame original image; the eighth sharpness is less than the second sharpness; the ninth sharpness is the sharpness of the first object on the rendered k-th frame original image, and the k-th frame image is the processed version of the k-th frame original image; the ninth sharpness is less than or equal to the third sharpness. That is, after the electronic device renders N frames of original images, it can display these N frames, and the sharpness of the virtual environment seen by the user is sharpness 1. If, after the electronic device renders N frames of original images, it processes these N frames to improve their sharpness, and the electronic device displays the processed N frames, the sharpness of the virtual environment seen by the user is sharpness 2, which is greater than sharpness 1, thus helping to improve the user's virtual reality experience.

[0010] In one possible design, the first object is located within a non-focal region, or it is located at a first depth of field greater than a preset depth of field. That is, the sharpness of objects within non-focal regions or distant objects (objects with a larger depth of field) in different frames of the image stream can be varied, thus saving rendering power. Furthermore, if the average sharpness of objects within non-focal regions or distant objects in different frames is greater than a threshold (a first threshold or a second threshold), then the human eye perceives objects within non-focal regions or distant objects in different frames as having a sharpness greater than the threshold, which helps improve the visual experience.

[0011] In one possible design, the sharpness of the second object remains constant across the N frames. The second object is located within the region of the foveation point, or it is located at a second depth of field, which is less than the preset depth of field. That is, when the sharpness of objects outside the foveation point changes between different frames in the image stream, the sharpness of objects within the foveation point can remain constant; or, when the sharpness of distant objects (objects with a larger depth of field) changes, the sharpness of nearby objects can remain constant. For example, the sharpness of objects outside the foveation point may change from high definition to low definition to high definition, while objects within the foveation point remain consistently high definition or ultra-high definition. Similarly, the sharpness of distant objects may change from high definition to low definition to high definition, while nearby objects remain consistently high definition or ultra-high definition. This reduces rendering power consumption.

[0012] In one possible design, the second object has a higher resolution than the first object. For example, when the resolution of objects in the non-focal region changes between different frames in an image stream, the resolution of objects in the focal region can remain constant; moreover, the resolution of objects in the focal region is higher than that of objects in the non-focal region. For instance, in N frames, objects in the focal region are always in ultra-high definition, while objects in the non-focal region exhibit a high-definition-low-high-definition variation. This ensures that the resolution of objects in the focal region or nearby objects is sufficiently clear, avoiding any impact on the user experience.

[0013] In one possible design, the sharpness of the second object on the i-th frame is tenth sharpness; the sharpness of the second object on the j-th frame is eleventh sharpness; the sharpness of the second object on the k-th frame is twelfth sharpness; the second object is located within the region of the fixation point, or the second object is located at a second depth of field, which is less than the preset depth of field; the average of the tenth and eleventh sharpnesses is the thirteenth sharpness, and the average of the eleventh and twelfth sharpnesses is the fourteenth sharpness; the thirteenth sharpness is higher than the fourth sharpness, and the fourteenth sharpness is higher than the fifth sharpness.

[0014] In other words, when the sharpness of objects within the non-focal region (i.e., the first object) changes between different frames in the image stream, the sharpness of objects within the focal region (i.e., the second object) also changes. Furthermore, the average sharpness of objects within the focal region across different frames is higher than the average sharpness of objects within the non-focal region. Therefore, in the virtual environment perceived by the user, objects within the focal region have higher sharpness than objects within the non-focal region. Similarly, when the sharpness of distant objects (i.e., the first object) changes between different frames in the image stream, the sharpness of nearby objects (i.e., the second object) also changes. Moreover, the average sharpness of nearby objects across different frames is higher than the average sharpness of distant objects. Therefore, in the virtual environment perceived by the user, nearby objects have higher sharpness than distant objects.

[0015] In one possible design, the first object is located within the region of the gaze point; or, the first object is located at a first depth of field, which is less than a preset depth of field. That is, the sharpness of objects within the gaze point region, or nearby objects (objects with a smaller depth of field), in different frames of the image stream can be varied, thus saving rendering power. Furthermore, if the average sharpness of objects within the gaze point region or nearby objects in different frames is greater than a threshold (a first threshold or a second threshold), then the human eye perceives objects within the gaze point region or nearby objects in different frames as having a sharpness greater than the threshold, which helps improve the visual experience.

[0016] In one possible design, the sharpness of the second object remains constant across the N frames. The second object is located within a non-focal region, or it is located at a second depth of field greater than a preset depth of field. That is, when the sharpness of objects within the focal region changes between frames in the image stream, the sharpness of objects within the non-focal region can remain constant; or, when the sharpness of nearby objects changes, the sharpness of distant objects can remain constant. For example, the sharpness of objects within the focal region may change from ultra-high definition to high definition to ultra-high definition, while objects within the non-focal region may remain consistently high definition or consistently low definition. Another example is that the sharpness of nearby objects may change from ultra-high definition to high definition to ultra-high definition, while distant objects may remain consistently high definition or consistently low definition. This reduces rendering power consumption.

[0017] In one possible design, the sharpness of the second object is lower than that of the first object. For example, when the sharpness of objects within the gaze area changes between different frames in an image stream, the sharpness of objects within the non-gaze area can remain constant; moreover, the sharpness of objects within the non-gaze area is lower than that of objects within the gaze area. For instance, in N frames, objects within the non-gaze area are consistently high-definition or low-definition, while objects within the gaze area exhibit a changing state from ultra-high definition to high definition to ultra-high definition. This ensures that the sharpness of objects in the gaze area or nearby objects is sufficiently clear, avoiding any impact on the user experience.

[0018] In one possible design, the sharpness of the second object on the i-th frame is tenth sharpness, the sharpness of the second object on the j-th frame is eleventh sharpness, and the sharpness of the second object on the k-th frame is twelfth sharpness; the second object is located in a region outside the focal point, or the second object is located in a second depth of field, which is greater than the preset depth of field; the average of the tenth and eleventh sharpnesses is the thirteenth sharpness, and the average of the eleventh and twelfth sharpnesses is the fourteenth sharpness; the thirteenth sharpness is lower than the fourth sharpness, and the fourteenth sharpness is lower than the fifth sharpness.

[0019] In other words, when the sharpness of objects within the gaze area (i.e., the first object) changes between different frames in the image stream, the sharpness of objects within the non-gaze area (i.e., the second object) also changes. Furthermore, the average sharpness of objects within the non-gaze area across different frames is lower than the average sharpness of objects within the gaze area. Therefore, in the virtual environment perceived by the user, the sharpness of objects within the non-gaze area is lower than that of objects within the gaze area. Similarly, when the sharpness of nearby objects (i.e., the first object) changes between different frames in the image stream, the sharpness of distant objects (i.e., the second object) also changes. Moreover, the average sharpness of distant objects across different frames is lower than the average sharpness of nearby objects. Therefore, in the virtual environment perceived by the user, the sharpness of distant objects is lower than that of nearby objects.

[0020] In one possible design, the user's gaze point remains unchanged during the display of the i-th frame, the j-th frame, and the k-th frame. It is understood that when the user's gaze point changes, the electronic device can redetermine the area where the user's gaze point is located and re-execute the technical solution provided in this application. For example, the redefined gaze point allows for switching between high and low sharpness of objects (i.e., the first object) within that area. Moreover, the average sharpness of the first object across different frames is higher than a threshold. In other words, when the user's gaze point changes, the sharpness of objects within the new gaze point area in the virtual environment perceived by the human eye is also higher than the threshold, resulting in a better user visual experience.

[0021] In one possible design, the time interval between the display time of the i-th frame and the display time of the j-th frame is less than or equal to the user's visual dwell time; and / or, the time interval between the display time of the j-th frame and the display time of the k-th frame is less than or equal to the user's visual dwell time. It should be understood that when the display time interval between two frames is small (e.g., less than the user's visual dwell time), due to human visual dwell efficiency, the brain will fuse the two frames, so the human eye sees the environment presented after the two frames are fused. Therefore, in this embodiment, when the electronic device displays an image stream, the display time interval between two frames is less than the user's visual dwell time, and the average sharpness of the first object in the two frames is higher than a threshold, so when the human eye fuses the two frames, the sharpness of the first object in the environment is higher.

[0022] In one possible design, before displaying N frames of images via a display device, the process further includes: detecting that the user triggers an operation to activate the sharpness optimization mode; the user's viewing time being greater than a first duration; and the number of times the user blinks or squints within a second duration being greater than a preset number.

[0023] For example, an electronic device has a normal mode and a sharpness optimization mode. In normal mode, after rendering an image stream, the device displays the image stream through a display device, resulting in a low-resolution virtual environment for the user. When the electronic device activates the sharpness optimization mode, it performs image processing on the rendered image stream, ensuring that the average sharpness of different frames in the image stream exceeds a threshold. This improves the sharpness of the virtual environment seen by the user, enhancing the user experience. There are several ways for the electronic device to activate the sharpness optimization mode. For example, it may activate when the user triggers an operation to activate the mode; or when the user's viewing time exceeds a first duration; or when the number of blinks or squints by the user exceeds a preset number within a second duration. In other words, the electronic device may not always be in sharpness optimization mode, but rather activate it when the user needs it or when a user need is detected (e.g., the user's viewing time exceeds the first duration, or the number of blinks or squints by the user exceeds a preset number within a second duration).

[0024] Secondly, a display method is also provided, comprising: when a first application is displayed through a first device, the display screen of the first device displays N frames of first images, where N is a positive integer, including an i-th frame of the first image and a j-th frame of the first image, wherein the clarity of a first object in the i-th frame of the first image is a first clarity; the clarity of the first object in the j-th frame of the first image is a second clarity; and the average of the first clarity and the second clarity is a third clarity; when the first application is displayed through a second device, the display screen of the second device displays N frames of second images, including an i-th frame of the second image and a j-th frame of the second image, wherein the clarity of the first object in the i-th frame of the second image is a fourth clarity; the clarity of the first object in the j-th frame of the second image is a fifth clarity; and the average of the fourth clarity and the fifth clarity is a sixth clarity; wherein the third clarity is higher than the sixth clarity.

[0025] In other words, there are two devices, such as two VR glasses, which will be referred to as VR Glasses A and VR Glasses B for easy distinction. VR Glasses A uses the technical solution provided in this application, while VR Glasses B does not. Both VR glasses can display images generated by the same application. This "same application" can mean the same application name, application version, etc. For example, both VR glasses may have the same VR driving application. When a user wears VR Glasses A, and VR Glasses A displays the application's image, the user sees a virtual environment with high clarity. When the user wears VR Glasses B, and VR Glasses B also displays the application's image, the user sees the same virtual environment as when wearing VR Glasses A, but the clarity of the virtual environment seen when wearing VR Glasses B is lower than that seen when wearing VR Glasses A.

[0026] Thirdly, an electronic device is also provided, comprising:

[0027] Processor, memory, and one or more programs;

[0028] The one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the processor, cause the electronic device to perform the method steps provided in the first or second aspect above.

[0029] Fourthly, a computer-readable storage medium is also provided for storing a computer program that, when run on a computer, causes the computer to perform the method described in the first or second aspect above.

[0030] Fifthly, a computer program product is also provided, comprising a computer program that, when run on a computer, causes the computer to perform the method described in the first or second aspect above.

[0031] For the beneficial effects described in aspects two through five above, please refer to the description of the beneficial effects in aspect one, which will not be repeated here. Attached Figure Description

[0032] Figure 1 A schematic diagram of a communication system provided in an embodiment of this application;

[0033] Figure 2 This is a schematic diagram of a VR wearable device provided in an embodiment of this application;

[0034] Figure 3 A schematic diagram of an image stream viewed by the human eye, provided as an embodiment of this application;

[0035] Figures 4A to 4C A schematic diagram of an image viewed by the human eye according to an embodiment of this application;

[0036] Figure 5 An illustration of the effect provided for one embodiment of this application;

[0037] Figures 6 to 9 A schematic diagram illustrating an application scenario provided in one embodiment of this application;

[0038] Figure 10 This is a schematic diagram of the structure of a VR wearable device provided in an embodiment of this application;

[0039] Figure 11 A schematic diagram of an image processing procedure provided in an embodiment of this application;

[0040] Figure 12 This is a schematic diagram of an image processing procedure provided in an embodiment of this application;

[0041] Figures 13A to 13B Another schematic diagram of an image processing procedure provided in an embodiment of this application;

[0042] Figures 14A to 14B This is yet another schematic diagram of an image processing procedure provided in an embodiment of this application;

[0043] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0044] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0045] (1) The at least one involved in the embodiments of this application includes one or more; wherein, multiple means two or more. Furthermore, it should be understood that in the description of this application, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order. For example, "first object" and "second object" do not represent the degree of importance of the two or their order, but are used to distinguish the objects.

[0046] In the embodiments of this application, "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0047] (2) Virtual Reality (VR) technology is a human-computer interaction method created using computer and sensor technologies. VR technology integrates various scientific technologies such as computer graphics, computer simulation, sensor technology, and display technology to create virtual environments. These virtual environments include computer-generated, real-time dynamic, three-dimensional, realistic images that provide visual perception to the user. In addition to the visual perception generated by computer graphics, there are also auditory, tactile, force, and motion perceptions, and even olfactory and gustatory perceptions, also known as multi-sensory perception. Furthermore, it can detect the user's head movements, eye movements, gestures, or other human actions. The computer processes the data corresponding to the user's actions and responds to them in real time, feeding back to the user's five senses, thus forming the virtual environment. For example, when wearing VR wearable devices, users can see a VR game interface and interact with it through gestures, controllers, etc., as if they were actually in the game.

[0048] (3) Augmented Reality (AR) technology refers to superimposing computer-generated virtual objects onto real-world scenes to enhance the real world. In other words, AR technology requires capturing real-world scenes and then adding a virtual environment to them. Therefore, the difference between VR and AR technologies is that VR creates a completely virtual environment where users see only virtual objects; while AR overlays virtual objects onto the real world, including both real and virtual objects. For example, a user wearing transparent glasses can see the surrounding real environment, and the glasses can also display virtual objects, allowing the user to see both real and virtual objects.

[0049] (4) Mixed Reality (MR) technology enhances the realism of the user experience by introducing real-world scene information (or real-world scene information) into a virtual environment, thus building an interactive feedback bridge between the virtual environment, the real world, and the user. Specifically, real-world objects are virtualized (for example, using a camera to scan real-world objects for 3D reconstruction to generate virtual objects), and the virtualized real objects are introduced into the virtual environment, so that users can see real objects in the virtual environment.

[0050] It should be noted that the technical solutions provided in this application can be applied to VR, AR, or MR scenarios. Of course, they can also be applied to other scenarios besides VR, AR, and MR. For example, naked-eye 3D scenarios (naked-eye 3D displays, naked-eye 3D projectors, etc.), cinemas (such as 3D movies), VR software in electronic devices, etc. In short, the technical solutions provided in this application can be applied to any scenario that requires displaying three-dimensional images.

[0051] For ease of understanding, the following text will mainly use VR scenarios as an example.

[0052] For example, please see Figure 1 This is a schematic diagram of a VR system according to an embodiment of this application. The VR system includes a VR wearable device 100 and an image generation device 200. The image generation device 200 includes a host (e.g., a VR host) or a server (e.g., a VR server). The VR wearable device 100 is connected to the VR host or VR server (wired or wireless connection). The VR host or VR server can be a device with significant computing power. For example, the VR host can be a mobile phone, tablet, laptop, etc., and the VR server can be a cloud server, etc. The VR host or VR server is responsible for generating images, etc., and then sending the images to the VR wearable device 100 for display. The user can see the images while wearing the VR wearable device 100. For example, the VR wearable device 100 can be a head-mounted display (HMD), such as glasses, a helmet, etc. Optionally, Figure 1 The VR system may also exclude the image generation device 200. For example, the VR wearable device 100 has its own image generation function, eliminating the need to obtain images from the image generation device 200 (VR host or VR server) for display. In short, the VR wearable device 100 can display three-dimensional images. Because different objects in the three-dimensional image have different depths of field (see below), a three-dimensional virtual environment can be displayed to the user through the three-dimensional image.

[0053] (5) Depth of Field (DOF): A 3D image includes objects at different depths. For example, when a VR wearable device displays a 3D image, the user sees a 3D scene (i.e., a virtual environment). Different objects in this 3D scene are at different distances from the user's eyes, creating a sense of depth. Therefore, image depth can be understood as the distance between objects in a 3D image and the user's eyes. The greater the image depth, the farther away an object appears to be from the user's eyes, making it seem like a distant scene; the smaller the image depth, the closer an object appears to be from the user's eyes, making it seem like a close-up scene. Image depth can also be called "depth of field".

[0054] In order to clearly explain the process by which VR wearable devices present virtual environments to users, the following is a brief introduction to the mechanism of human vision generation.

[0055] In real-world scenarios, when a user views an object, the human eye acquires light signals from that scene and processes these signals in the brain to achieve visual perception. These light signals can include reflected light from different objects and / or light signals directly emitted by a light source. Since the light signals in a real-world scene can carry information about various objects (such as size, position, and color), the brain can acquire this information—that is, obtain visual perception—by processing these signals. It should be understood that light signals are continuously generated in a real-world scene, and the human eye continuously acquires these signals at a certain frequency. These continuously acquired light signals, processed by the brain, form a dynamic visual perception.

[0056] The human eye consists of a left and a right eye, each using the aforementioned visual generation mechanisms to create dynamic visual perception. It's important to note that the left and right eyes have slightly different viewing angles when viewing the same object. Therefore, the images seen by the left and right eyes are actually different. For example, the left eye receives the light signal of a two-dimensional image (hereinafter referred to as the left-eye image) on the plane where the eye's focal point is perpendicular to its line of sight. Similarly, the right eye receives the light signal of a two-dimensional image (hereinafter referred to as the right-eye image) on the plane where the eye's focal point is perpendicular to its line of sight. The left-eye and right-eye images are slightly different. The brain can process the light signals from the left-eye and right-eye images to determine the depth of different objects in the actual scene, thus achieving stereoscopic vision. Stereoscopic vision is also known as binocular stereoscopic vision.

[0057] The above describes the mechanism of human vision. VR wearable devices utilize this mechanism to present a virtual environment to the user. For ease of understanding, we will use VR glasses as an example.

[0058] Please see Figure 2This is a schematic diagram of VR glasses. The VR glasses have two displays, for example, display 201 and display 202. Each display can show content to one of the user's eyes (e.g., left or right eye) through a corresponding eyepiece. For example, display 201 corresponds to eyepiece 203, and display 202 corresponds to eyepiece 204. Display 201 can display the left-eye image of the virtual environment. The light from this left-eye image can be focused onto the left eye through eyepiece 203, allowing the left eye to see the left-eye image. Similarly, display 202 can display the right-eye image of the virtual environment. The light from this right-eye image can be focused onto the right eye through eyepiece 204, allowing the right eye to see the right-eye image. Thus, the brain can fuse the left-eye and right-eye images, allowing the user to see objects in the virtual environment corresponding to the left-eye and right-eye images. It should be noted that display 201 and display 202 can be independent displays, or different display areas on the same display screen; this embodiment does not limit this.

[0059] To provide users with a dynamic visual experience, the VR glasses' display screens 201 and 202 can each display an image stream. The image stream consists of N frames, where N is an integer greater than or equal to 1.

[0060] The following example uses a display screen on VR glasses to illustrate the principle of how the screen displays an image stream.

[0061] Generally, image generation devices (e.g.) Figure 1 After the image generation device 200 generates N frames of images, it renders the N frames of images and displays the rendered N frames of images through a display screen of the VR glasses.

[0062] In some embodiments, each frame of the N-frame image is rendered using high resolution, and then the N-frame image rendered in high resolution is displayed on the VR glasses' screen. For example, see [link to example]. Figure 3 The VR glasses display a stream of images. This stream includes frames i, j, k, and so on. Each frame has a resolution of A (where A is a high resolution). For example, each frame can be an ultra-high-definition or high-definition image, resulting in high clarity for each frame.

[0063] To illustrate the technical solution of this application, we first introduce the Talbot-Platto law of the human eye. The Talbot-Platto law states that when a display screen shows an image stream, due to the persistence of vision, the brain fuses the different images in the stream. Simply put, what the human eye perceives is the fused image of different frames in the image stream, and the resolution of the fused image is the average of the sharpness of the fused images. Figure 3For example, when the display shows the i-th frame and the j-th frame, the human eye will fuse the i-th frame and the j-th frame. The resolution of the fused image is the average of the resolutions of the i-th and j-th frames. Since the resolutions of the i-th and j-th frames are both A, the resolution of the fused image is also A. Therefore, the resolution of the fused image perceived by the human eye is A. Similarly, when the display shows the j-th frame and the k-th frame, the human eye will fuse the j-th frame and the k-th frame. The resolution of the fused image is again the average of the resolutions of the j-th and k-th frames, which is A. Therefore, the resolution of the fused image perceived by the human eye is also A. In other words, when every frame in the image stream is ultra-high definition (or high definition), the fused image seen by the human eye is also ultra-high definition (or high definition), resulting in a better virtual reality experience.

[0064] However, it is understandable that maintaining high resolution (e.g., A resolution for each frame in an image stream) requires significant rendering power consumption, which most devices cannot meet.

[0065] To save power, one approach is to render different regions of the image using different resolutions. For example, the image can be divided into gaze regions and non-gaze regions. High-resolution rendering is used for gaze regions, while low-resolution rendering is used for non-gaze regions. Since user attention is low in non-gaze regions, the lower resolution has little impact on the visual experience, significantly saving power.

[0066] For example, such as Figure 4A The image includes a gaze region and a non-gaze region. The gaze region can be a circular area centered on the user's gaze point with a preset radius, or a square or rectangular area centered on the user's gaze point with a preset side length, etc. The non-gaze region is any area on the image other than the gaze region. Continuing... Figure 4A The resolution of the foveated region is A. The low resolution of the non-foveated region is B. A > B. In this way, it is not necessary to use high resolution for the entire image, reducing the power consumption of rendering.

[0067] It should be understood that Figure 4A Taking a single image as an example, as mentioned earlier, the VR glasses display a stream of images, i.e., N frames. In some embodiments, each frame in the image stream has a high resolution for the gaze area and a low resolution for the non-gaze area.

[0068] For example, such as Figure 4BAs shown, the image stream includes frames i, j, k, etc. In frame i, the resolution of the gaze area is A, and the resolution of the non-gaze area is B, where A > B. For example, A is ultra-high definition and B is high definition; or A is high definition and B is low definition. Similarly, in frame j, the resolution of the gaze area is A, and the resolution of the non-gaze area is B. In frame k, the resolution of the gaze area is A, and the resolution of the non-gaze area is B. That is, in each frame of the image stream, the gaze area resolution is high, and the non-gaze area resolution is low. In this case, when the VR glasses display frame i and frame j, the human eye will fuse them. The resolution of the fused image is the average of the resolutions of frame i and frame j. That is, the gaze area resolution of the fused image is still A, and the non-gaze area resolution is still B, because the non-gaze area resolution is (B+B) / 2. Similarly, when the VR glasses display the j-th frame and the k-th frame, the human eye will fuse the j-th frame and the k-th frame. The resolution of the fused image is the average of the resolutions of the j-th frame and the k-th frame. That is, the resolution of the gaze area in the fused image is still A, and the resolution of the non-gaze area is still B, because the resolution of the non-gaze area is (B+B) / 2. Therefore, Figure 4B In human visual perception, the sharpness of non-focused areas remains at B. Because B is low, the sharpness of non-focused areas remains low, which leads to the loss of details in non-focused areas and reduces the virtual reality experience.

[0069] It should be noted that, Figure 4B Taking the example of the fixation region having a constant resolution (e.g., A) across different frames in an image stream, the resolution of the fixation region can actually vary across different frames. For instance, the fixation region resolution might be A in the i-th frame, D in the j-th frame (where A is greater than D), and A again in the k-th frame. This means the resolution of the fixation region can switch between high and low across different frames. The principle of image fusion in the human eye is similar in this case. For example, the resolution of the fixation region in the fused image of the i-th and j-th frames is the average of the resolutions A and D of the fixation region in the i-th and j-th frames. Therefore, for the sake of brevity, the variations in the fixation region resolution across different frames will not be repeated.

[0070] It should be noted that, Figure 4B In the image stream, the resolution of the non-focused regions is the same in each frame, which is B. In other embodiments, the resolution of the non-focused regions in different frames of the image stream can be different; for example, the resolution of the non-focused regions in different frames of the image stream can switch between high and low.

[0071] For example, such as Figure 4C The image stream includes frames i, j, k, etc. In frame i, the resolution of the gaze region is A, and the resolution of the non-gaze region is B, where A > B. For example, A is ultra-high definition and B is high definition; or, A is high definition and B is low definition. In frame j, the resolution of the gaze region is A, and the resolution of the non-gaze region is C, where A > C and C > B. In frame k, the resolution of the gaze region is A, and the resolution of the non-gaze region is B. That is, compared with... Figure 4B Compared to the previous image, the resolution of the non-fixed region in the j-th frame has increased from B to C. Therefore, the resolution of the fixed region in the image obtained by fusing the i-th and j-th frames remains A, while the resolution of the non-fixed region is (B+C) / 2. Similarly, the resolution of the fixed region in the image obtained by fusing the j-th and k-th frames is A, while the resolution of the non-fixed region is (B+C) / 2. Since C>B, (B+C) / 2 is higher than B. Therefore, compared to the previous image, the resolution of the fixed region in the j-th frame is A, while the resolution of the non-fixed region is (B+C) / 2. Figure 4B compared to, Figure 4C In terms of human visual perception, the clarity of non-focused areas is improved.

[0072] Please combine Figure 4B and Figure 4C It's understood that by increasing the resolution of the non-focused regions in the j-th frame of the image stream (i.e., from B to C), the sharpness of these non-focused regions in the virtual environment as perceived by the human eye improves. Therefore, to further improve the sharpness of these non-focused regions in the virtual environment as perceived by the human eye, the resolution of the non-focused regions in certain frames of the image stream needs to be increased. Obviously, this would lead to increased rendering power consumption.

[0073] To improve the resolution of non-focused areas in the virtual environment as perceived by the human eye without increasing rendering power consumption, this application provides a method, such as... Figure 5 Image generation devices (e.g.) Figure 1 The image generation device 200 renders N frames of images. For ease of distinction, these rendered images are called the original images, including the i-th frame original image, the j-th frame original image, and the k-th frame original image, etc. The resolution of the gaze area in the i-th frame original image is A, and the resolution of the non-gaze area is B. The resolution of the gaze area in the j-th frame original image is A, and the resolution of the non-gaze area is C, where C is less than A and greater than B. The resolution of the non-gaze area in the k-th frame original image is B. Therefore, please compare... Figure 5 and Figure 4C It can be seen that the power consumption for image rendering has not increased.

[0074] See also Figure 5An image generation device can use image post-processing to process N original frames to obtain N new frames. For example, the resolution of the non-focused region in the i-th original frame can be increased from B to B' to obtain the i-th new frame, and / or the resolution of the non-focused region in the j-th original frame can be increased from C to C' to obtain the j-th new frame, and / or the resolution of the non-focused region in the k-th original frame can be increased from B to B' to obtain the k-th new frame. When the display device displays the N new frames, the resolution of the non-focused region in the image obtained by fusing the i-th and j-th new frames is (B'+C') / 2, and the resolution of the non-focused region in the image obtained by fusing the j-th and k-th new frames is (B'+C') / 2. Therefore, when a display device (e.g., VR glasses) displays a post-processed image stream, the clarity of the non-focused region is improved in the human visual perception, but the image rendering power consumption does not increase significantly. Figure 5 The image post-processing procedure described herein will be introduced later.

[0075] Therefore, in this embodiment of the application, the display device (e.g., VR glasses) can display... Figure 5 The image stream (including N new frames) contains frames i, j, and k. Assume the sharpness of the first object (e.g., an object located in a non-focused area) in frame i is a first sharpness, the sharpness of the first object in frame j is a second sharpness, and the sharpness of the first object in frame k is a third sharpness. The average of the first and second sharpnesses is the third sharpness, which is higher than the first threshold; for example, the third sharpness is (B'+C') / 2, and the first threshold is (B+C) / 2. Similarly, the average of the second and third sharpnesses is the fourth sharpness, which is higher than the second threshold; for example, the fourth sharpness is (B'+C') / 2, and the second threshold is (B+C) / 2. In this way, the sharpness of the first object in a non-focused area of ​​the virtual environment perceived by the user through a display device (e.g., VR glasses) is improved.

[0076] It should be noted that, Figure 5 In this example, the resolutions of non-focused and focused regions in the image stream are different. Taking the improvement of the resolution of non-focused regions as an example, for instance, the resolution of the non-focused region in the original image of frame i is improved to B', and the resolution of the non-focused region in the original image of frame j is improved to C'. In fact, the resolution of focused regions in an image can also be improved using the image post-processing method provided in this application. In other embodiments, the image is not divided into regions, i.e., the entire image has the same resolution (e.g., ...). Figure 3 In this case, the image post-processing method provided in this application can also be used to improve the resolution.

[0077] For ease of description, the following explanation uses increasing the resolution of a first object in the image as an example. The first object can be any object in the image. For example, the first object could be an object within a non-focused region of the image. Or, the first object could be an object within a focused region of the image. Or, the first object could be a distant object. Or, the first object could be a nearby object.

[0078] To facilitate understanding, an application scenario provided by an embodiment of this application is described below. This application scenario takes an object in a non-viewing region of an image as an example.

[0079] Figure 6 This is a schematic diagram illustrating an application scenario provided by an embodiment of this application. For example... Figure 6 For example, in this application scenario, a user wears VR glasses to drive in VR.

[0080] like Figure 6 As shown, VR glasses render an image stream, which includes N original frames. These include the i-th frame, j-th frame, and k-th frame, etc. The original images include objects such as a steering wheel, monitor, road, trees on the road, and vehicles ahead, providing a virtual driving experience for the user. Assuming the trees are located in a non-viewing area, the trees in the original images have low clarity; for example, the trees in the i-th frame are blurry, the trees in the j-th frame are clear, and the original image in the k-th frame is blurry. If the VR glasses display N original frames, the trees in the image obtained by fusing the i-th and j-th frames will have low clarity, and similarly, the trees in the image obtained by fusing the j-th and k-th frames will also have low clarity. The technical solution provided in this application embodiment can post-process the rendered N original frames to obtain N new images, including the i-th, j-th, and k-th frames, etc. Specifically, compared to the original image in frame i, the trees in the new image in frame i are clearer; compared to the original image in frame j, the trees in the new image in frame j have the same or better clarity; and compared to the original image in frame k, the trees in the new image in frame k have improved clarity. Therefore, when the VR glasses display N new images, the clarity of the trees in the image obtained by fusing the new images in frames i and j is improved compared to the image obtained by fusing the original images in frames i and j; similarly, the clarity of the trees in the image obtained by fusing the new images in frames j and k is improved compared to the image obtained by fusing the original images in frames j and j.

[0081] Optional, Figure 6Taking improving the sharpness of a first object (i.e., a tree) in a non-focused region of an image as an example, the sharpness of objects in the focused region (e.g., a vehicle in front) can remain unchanged, or the sharpness of objects in the focused region (e.g., a vehicle in front) can also be improved using the solution provided in this application embodiment. In some embodiments, the sharpness of objects in the focused region can be higher than the sharpness of objects in the non-focused region.

[0082] It should be noted that, Figure 6 Taking the example that the first object is an object within a non-focused region of the image, in other embodiments, the first object can also be an object within a focused region of the image, or a distant object in the image, or a nearby object in the image, etc., and the principle is the same. Figure 6 The principle is the same, so I will not repeat it.

[0083] The equipment related to this application is described below.

[0084] For example, please refer to Figure 7 Taking a VR wearable device (such as VR glasses) as an example, a structural schematic diagram of a VR wearable device provided in an embodiment of this application is shown. Figure 7 As shown, the VR wearable device 100 may include a processor 110, a memory 120, a sensor module 130 (which can be used to acquire the user's posture), a microphone 140, a button 150, an input / output interface 160, a communication module 170, a camera 180, a battery 190, an optical display module 1100, and an eye-tracking module 1200, etc.

[0085] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the VR wearable device 100. In other embodiments of this application, the VR wearable device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0086] Processor 110 is typically used to control the overall operation of VR wearable device 100 and may include one or more processing units. For example, processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a video processing unit (VPU) controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0087] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0088] In some embodiments of this application, the processor 110 can be used to control the optical power of the VR wearable device 100. For example, the processor 110 can be used to control the optical power of the optical display module 1100, thereby adjusting the optical power of the wearable device 100. For instance, the processor 110 can adjust the relative positions of the various optical components (such as lenses) in the optical display module 1100, thereby adjusting the optical power of the optical display module 1100. This, in turn, allows the position of the corresponding virtual image plane to be adjusted when the optical display module 1100 images onto the human eye, thus achieving the effect of controlling the optical power of the wearable device 100.

[0089] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, a serial peripheral interface (SPI) interface, etc.

[0090] In some embodiments, the processor 110 can blur objects at different depths of field to varying degrees so that the objects at different depths of field have different levels of clarity.

[0091] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses.

[0092] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the communication module 170. For example, the processor 110 communicates with the Bluetooth module in the communication module 170 via the UART interface to implement Bluetooth functionality.

[0093] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen and camera 180 in the optical display module 1100.

[0094] The GPIO interface is configurable via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 180, the display screen in the optical display module 1100, the communication module 170, the sensor module 130, the microphone 140, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc. Optionally, the camera 180 can capture images including real objects, and the processor 110 can fuse the images captured by the camera with virtual objects, displaying the fused image through the optical display module 1100. Optionally, the camera 180 can also capture images including human eyes. The processor 110 performs eye tracking using these images.

[0095] The USB interface conforms to the USB standard specification and can be a Mini USB interface, Micro USB interface, USB Type-C interface, etc. The USB interface can be used to connect a charger to charge the VR wearable device 100, and can also be used for data transfer between the VR wearable device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as mobile phones. The USB interface can be USB 3.0, used for compatibility with high-speed display port (DP) signal transmission, enabling the transmission of high-speed audio and video data.

[0096] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the wearable device 100. In other embodiments of this application, the wearable device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0097] Additionally, the VR wearable device 100 may include wireless communication functionality; for example, the VR wearable device 100 may receive images from other electronic devices (such as a VR host) for display. The communication module 170 may include a wireless communication module and a mobile communication module. The wireless communication functionality can be implemented using an antenna (not shown), a mobile communication module (not shown), a modem processor (not shown), and a baseband processor (not shown). The antenna is used to transmit and receive electromagnetic wave signals. The VR wearable device 100 may include multiple antennas, each of which can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in conjunction with a tuning switch.

[0098] The mobile communication module can provide solutions for wireless communication applications on VR wearable devices 100, including 2G, 3G, 4G, and 5G networks. The mobile communication module may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module can receive electromagnetic waves via an antenna, filter and amplify the received electromagnetic waves, and transmit them to a modem processor for demodulation. The mobile communication module can also amplify the signal modulated by the modem processor and radiate it as electromagnetic waves via the antenna. In some embodiments, at least some functional modules of the mobile communication module may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module and at least some modules of the processor 110 may be housed in the same device.

[0099] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to a speaker), or displays images or videos through a display screen in the optical display module 1100. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module or other functional modules.

[0100] The wireless communication module can provide solutions for wireless communication applications on the VR wearable device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module can be one or more devices integrating at least one communication processing module. The wireless communication module receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signals, and sends the processed signal to the processor 110. The wireless communication module can also receive signals to be transmitted from the processor 110, modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0101] In some embodiments, the antenna of the VR wearable device 100 is coupled to the mobile communication module, enabling the VR wearable device 100 to communicate with networks and other devices via wireless communication technologies. These wireless communication technologies may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies. GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0102] The VR wearable device 100 implements display functions through a GPU, an optical display module 1100, and an application processor. The GPU is a microprocessor for image processing, connecting the optical display module 1100 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0103] The memory 120 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the VR wearable device 100 by running the instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc. The data storage area may store data created during the use of the wearable device 100 (such as audio data, phone book, etc.). In addition, the memory 120 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0104] The VR wearable device 100 can implement audio functions through an audio module, speaker, microphone 140, headphone jack, and application processor. Examples include music playback and recording. The audio module is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module can also be used for encoding and decoding audio signals. In some embodiments, the audio module can be located in the processor 110, or some functional modules of the audio module can be located in the processor 110. The speaker, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. The wearable device 100 can listen to music or make hands-free calls through the speaker.

[0105] Microphone 140, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. VR wearable device 100 may have at least one microphone 140. In some embodiments, VR wearable device 100 may have two microphones 140, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, VR wearable device 100 may also have three, four, or more microphones 140, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions.

[0106] The headphone jack is used to connect wired headphones. The headphone jack can be a USB interface, or a 3.5 mm Open Mobile Terminal Platform (OMTP) standard interface, or a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0107] In some embodiments, the VR wearable device 100 may include one or more buttons 150 that can control the VR wearable device and provide users with access to functions on the VR wearable device 100. The buttons 150 may take the form of buttons, switches, dials, and touch or proximity sensing devices (such as touch sensors). Specifically, for example, a user can turn on the optical display module 1100 of the VR wearable device 100 by pressing a button. Buttons 150 include power buttons, volume buttons, etc. Buttons 150 may be mechanical buttons or touch buttons. The wearable device 100 can receive button input and generate key signal inputs related to user settings and function control of the wearable device 100.

[0108] In some embodiments, the VR wearable device 100 may include an input / output interface 160, which can connect other devices to the VR wearable device 100 via suitable components. Components may include, for example, audio / video jacks, data connectors, etc.

[0109] The optical display module 1100, under the control of the processor 110, presents images to the user. The optical display module 1100 can use one or more optical devices, such as mirrors, transmissive mirrors, or optical waveguides, to convert a real-pixel image display into a near-eye projection virtual image display, achieving a virtual interactive experience or a combined virtual and real interactive experience. For example, the optical display module 1100 receives image data information sent by the processor 110 and presents the corresponding image to the user.

[0110] In some embodiments, the VR wearable device 100 may further include an eye-tracking module 1200, which tracks the movement of the human eye to determine the gaze point. For example, the pupil position can be located and the pupil center coordinates can be obtained through image processing technology to calculate the gaze point.

[0111] For ease of understanding, the following is... Figure 7 The VR wearable device shown is a VR glasses example, which will be used to illustrate the display method of this application embodiment.

[0112] In this embodiment, the VR glasses display an image stream, which includes an i-th frame, a j-th frame, and a k-th frame. The sharpness of the first object in the i-th frame is a first sharpness, the sharpness of the first object in the j-th frame is a second sharpness, and the sharpness of the first object in the k-th frame is a third sharpness. The average of the first and second sharpnesses is the third sharpness, which is higher than a first threshold. The average of the second and third sharpnesses is the fourth sharpness, which is higher than the second threshold. This ensures that when the human eye views the image stream while wearing VR glasses, the sharpness of the first object is not perceived as too low. For example, the display time interval between the i-th and j-th frames is less than the user's visual dwell time. In some embodiments, j = i + 1, meaning the j-th frame is the next frame after the i-th frame. In other embodiments, j = i + n, where n > 1. n can be determined based on the user's visual dwell time and the image refresh rate. Assuming the user's visual dwell time is T and the image refresh rate is Z, then T*Z frames of images can be displayed within the time T, and n is less than or equal to T*Z. The visual dwell time T can be any value within the range of 0.1s to 3s, or it can be set by the user; this embodiment does not limit the value.

[0113] In some embodiments, the first threshold can be a user-defined value, and the second threshold can also be a user-defined value. Taking the first threshold as an example, the user can set a threshold such that when the VR glasses display the image stream, the sharpness of the first object is not lower than the threshold in terms of human visual perception. In this way, the user can customize the sharpness of objects in the virtual environment, resulting in a higher level of experience.

[0114] In other embodiments, the first threshold may be the average of the sharpness of the first object in the original image of frame i and the sharpness of the first object in the original image of frame j. Here, the original image of frame i is the original image corresponding to the image of frame i (or the new image of frame i), that is, the image of frame i is obtained by post-processing the original image of frame i; the original image of frame j corresponds to the image of frame j (or the new image of frame j), that is, the image of frame j is obtained by post-processing the original image of frame j. Figure 5 For example, assuming the first object is located within a non-focused region, the first threshold can be the average of the resolution of the non-focused region in the original image of frame i and the resolution of the non-focused region in the original image of frame j, i.e., (B+C) / 2. Similarly, the second threshold can be the average of the sharpness of the first object in the original image of frame j and the sharpness of the first object in the original image of frame k. Here, the original image of frame j corresponds to the image of frame j (or the new image of frame j), meaning that the image of frame j is obtained by post-processing the original image of frame j; the original image of frame k corresponds to the image of frame k (or the new image of frame k), meaning that the image of frame k is obtained by post-processing the original image of frame k. Figure 5For example, assuming the first object is located within a non-focused region, the second threshold can be the average of the resolution of the non-focused region in the original image of frame j and the resolution of the non-focused region in the original image of frame k, i.e., (B+C) / 2. In this way, the clarity of the first object in the virtual environment seen by the user is improved.

[0115] The following section describes the image post-processing procedure.

[0116] Please see Figure 8 The image generation device renders N original frames. These include the i-th original frame, the j-th original frame, the k-th original frame, and so on. Assume the resolution of the i-th original frame is P, the resolution of the j-th original frame is Q (where Q is less than P), and the resolution of the k-th original frame is P. Then, using the image post-processing method provided in this application, the resolution of the i-th original frame is increased to P', the resolution of the j-th original frame is increased to Q', and the resolution of the k-th original frame is increased to P'.

[0117] For example, taking the original image of frame i and the original image of frame j as examples, please refer to [link to relevant documentation]. Figure 9 The image post-processing workflow includes:

[0118] (1) Obtain the original image of the i-th frame, denoted as image H.

[0119] (2) Blur the image H to obtain the image L. The resolution of the image L is the same as that of the image H.

[0120] One possible implementation is that step (2) includes: downsampling image H, and then upsampling to obtain image L, so that the resolution of image L is the same as the resolution of image H. For ease of understanding, downsampling and upsampling are introduced first. Downsampling, also known as downsampling, refers to downsampling an image with a resolution of M*N by a factor of s, that is, obtaining an image with a resolution of (M / s)*(N / s). In other words, all pixels in the upper s*s window of the image with a resolution of M*N are turned into one pixel, and the value of this pixel is the average value of all pixels in the window. It can be seen that after downsampling, the details of the image are lost and the image resolution is reduced. The specific value of s is not limited in this application and can be the default setting or the user setting. Upsampling, also known as upsampling, refers to enlarging the image, for example, using an interpolation method, that is, inserting new elements between pixels based on the original image pixels using a suitable interpolation algorithm. There are many kinds of interpolation algorithms, such as edge-based image interpolation algorithms and region-based image interpolation algorithms, which are not limited in this application.

[0121] For example, please see Figure 10Image H contains multiple pixels (represented by solid black dots). Downsampling is performed on image H; for example, every 9 pixels in image H are processed into 1 pixel, meaning every 9 solid black dots in image H are processed into one hollow black dot. This reduces the resolution of image H. Then, continue to see... Figure 10 The downsampled image is then upsampled, meaning that interpolation is used to interpolate the pixels in the image to obtain image L, making image L have the same number of pixels as image H, i.e., the same resolution. It should be noted that image details are lost during downsampling. Although upsampling maintains the image resolution, the details in the image cannot be fully recovered. In other words, compared to image H, image L loses more detail.

[0122] (3) Based on image H and image L, image I is calculated, and image I satisfies I = 2H - L.

[0123] For example, please see Figure 10 Image H includes a first pixel, and image L includes a second pixel, which corresponds to the first pixel. This means the second pixel and the first pixel are in the same position, for example, both are pixels in the first row and first column. Image I includes a third pixel, which corresponds to both the first and second pixels. Furthermore, the pixel information of the third pixel is equal to the pixel information of the first pixel multiplied by 2 minus the pixel information of the second pixel. For example, the pixel information may include color information, such as RGB (R(red), G(green), B(blue)), and the pixel information can be color information from 0 to 255. It should be noted that the pixels in image L are obtained after blurring, and this information is generally low-frequency information. The pixels in image H are unblurred information, including high-frequency information. The pixel information in image H is twice that of the pixels in image H, which increases the high-frequency pixel information. Subtracting the pixel information from the pixels in image L removes low-frequency information, thus improving the contrast of image I.

[0124] It should be noted that, taking the image I satisfying I = 2H - L as an example above, in some embodiments, image I satisfies I = mH - L, where m can be a value greater than or equal to 1. The value of m can be a default setting or a user-defined value. When the user sets different values ​​for m, the R obtained in step (4) is different, and correspondingly, the image S displayed in step (5) is different. Moreover, it also affects the post-processing of the original image of frame j (denoted as image H') (because R is different). Therefore, when the user-defined value of m is different, the processing results of the original image of frame i and the original image of frame j are different, and the clarity of the image obtained by fusing the processed original image of frame i and the processed original image of frame j is different. In other words, the clarity of the image obtained by fusing the image of frame i and the image of frame j (i.e., the average clarity of the image of frame i and the image of frame j) varies with the value of m.

[0125] (4) Determine the overflow value R of the pixel information of the pixel points on image I.

[0126] In step (3), the pixel information of the third pixel in image I is twice the pixel information of the second pixel, and then the pixel information of the second pixel is subtracted. This results in an overflow value. For example, if the color information of the first pixel is 250 and the color information of the second pixel is 200, then the color information of the third pixel is 250*2-200=300. However, 300 is not within the range of 0-255, so the overflow value of the third pixel is determined to be 300-255=45. This is an example of determining the overflow value of the third pixel in image I. The calculation principle for the overflow values ​​of other pixels is the same.

[0127] (5) Display image S, where image S satisfies S = IR.

[0128] In other words, after subtracting the overflow value R from the pixel information of the pixels in image I, image S is obtained, and then image S is displayed. Step (5) can prevent over-sharpening of pixels (e.g., pixels with color information of 300), making the pixel information of the entire image smooth.

[0129] The above describes the post-processing procedure for the original image of frame i (i.e., image H). The following describes the post-processing procedure for the original image of frame j (denoted as image H'). See also... Figure 9 As shown, the post-processing procedure for image H' includes:

[0130] (1) Obtain the original image of the j-th frame, denoted as image H'. The resolution of image H' is lower than that of image H.

[0131] (2) Magnify the image H' to obtain the image L'. The resolution of the image L' is the same as that of the image L.

[0132] For example, image H' can be upsampled to increase its resolution, resulting in image L'. See [example example]. Figure 11 Image H' consists of multiple pixels (represented by hollow circles). Upsampling (i.e., pixel interpolation) is performed on image H' to increase the number of pixels and improve the resolution, resulting in image L'. The upsampling process has been described previously and will not be repeated here.

[0133] (3) Determine the offset coefficient or movement parameter W based on the image L' and the retained image L.

[0134] For example, W includes a movement parameter from the second pixel on image L to the fourth pixel on image L', where the fourth pixel corresponds to the second pixel, and the movement parameter satisfies the following formula:

[0135]

[0136] Where s is a user-defined parameter, and L and R are the values ​​stored during the processing of the i-th frame image.

[0137] (4) Based on W and the retained R, we obtain image I'. Image I' satisfies L'+WR.

[0138] For example, see [link to example]. Figure 11 Image I' includes a fifth pixel, and image L' includes a fourth pixel. The fifth pixel corresponds to the fourth pixel. The pixel information of the fifth pixel = the pixel information of the fourth pixel + the offset W of the fourth pixel relative to the second pixel multiplied by the overflow value R corresponding to the third pixel.

[0139] (5) Display image I'.

[0140] The above describes the post-processing of the original image (i.e., image H') in frame j.

[0141] It should be noted that when post-processing the original image of frame j, intermediate results from the post-processing of the original image of frame i are utilized, such as image L and overflow value R. In other words, the overflow portion R from the post-processing of the original image of frame i is added to the new image of frame j to enhance the details of the new image of frame j. Thus, when the human eye merges the new image of frame i and the new image of frame j, the clarity is improved.

[0142] exist Figures 8 to 11 Taking the post-processing of the entire image, including both the original image of frame i and the original image of frame j, as an example, this application can also perform post-processing only on the first object in the original image of frame i, or only on the first object in the original image of frame j, with the same implementation principle as for the entire image. For an example, please refer to [link to example]. Figure 12The first object is present in both the i-th and j-th original frames. Therefore, the post-processing procedure for the first object in the i-th original frame includes: determining the region containing the first object in the i-th original frame (e.g., the region enclosed by the edge contour of the first object), and then performing processing on that region. Figure 9 The processing flow in the first line increases the resolution of the first object to P'. The post-processing of the first object on the j-th frame original image includes: determining the region where the first object is located on the j-th frame original image (e.g., the region enclosed by the edge contour of the first object), and then performing processing on that region. Figure 9 The processing flow in the second line improves the resolution of the first object to Q'. The post-processing principle for the first object on the original image of the k-th frame is the same, and will not be repeated.

[0143] As mentioned earlier, the first object can be any object on the image. For ease of understanding, different cases will be introduced.

[0144] Case 1: The first object is an object outside the gaze area.

[0145] For example, please see Figure 13A The rendered image stream includes the original image of frame i, frame j, and frame k; each original image frame includes both a gaze region and a non-gaze region, and the gaze region in each original image frame is ultra-high definition. The non-gaze region in the original image of frame i is high definition, the non-gaze region in the original image of frame j is low definition, and the gaze region in the original image of frame k is high definition, meaning the non-gaze region in different original images switches between high definition and low definition. In this case, the technical solution provided by the embodiments of this application can... Figure 13A Sharpness enhancement is applied to objects in non-focused regions of the image stream.

[0146] For example, please see Figure 13B In the original image of the i-th frame, the gaze area is ultra-high definition, and the non-gaze area is high definition. The non-gaze area is processed using the post-processing method provided in this application (i.e.,...). Figure 9 The first line of the process (processing) is used to improve the resolution of the non-focused region. Then, the post-processed non-focused region is combined with the focused region to create a complete image, resulting in the new image of the i-th frame. See also... Figure 13B In the j-th frame of the original image, the gaze area is ultra-high resolution, while the non-gaze area is low resolution. The non-gaze area is processed using the post-processing method provided in this application (i.e., Figure 9 The processing flow in the second line of the image (referring to the non-focused region) is used to improve the resolution. Then, the processed non-focused region is combined with the focused region to create a complete image, resulting in the new image of frame j. This method improves the sharpness of objects within the non-focused regions in both the new image of frame i and the new image of frame j.

[0147] Case 2: The first object is an object within the gaze area.

[0148] For example, please see Figure 14A The system renders an image stream, which includes the original image of frame i, frame j, and frame k. Each original image frame includes a gaze region and a non-gaze region, and the non-gaze region in each original image frame is low-resolution. The gaze region in the original image of frame i is low-resolution, the gaze region in the original image of frame j is high-resolution, and the gaze region in the original image of frame k is low-resolution, meaning that the gaze region in different original images switches between high-resolution and low-resolution. In this case, the technical solution provided by the embodiments of this application can... Figure 14A Enhance the sharpness of objects within the gaze area in the image stream.

[0149] For example, please see Figure 14B The original image of frame i is rendered at a low resolution, meaning the gaze area and non-gaze areas in frame i have the same resolution, both being low-resolution. In frame j, the gaze area is high-resolution, while the non-gaze areas retain the low-resolution version of frame i. For frame i, post-processing can be performed on the gaze area (e.g.,...). Figure 9 The processing flow in the second line (see section 2) improves the resolution. For non-focused regions, post-processing is unnecessary. The post-processed focused and non-focused regions are then combined to obtain the new image of the i-th frame. See also... Figure 14B For the j-th frame of the original image, its gaze region can be post-processed (i.e., Figure 9 The processing flow in the first line (of the image) is used to improve the resolution. For non-focused regions, the low-resolution of the i-th frame can be reused. The post-processed focused and non-focused regions are then combined to obtain the new image of the j-th frame.

[0150] It should be noted that Case 1 and Case 2 can be used alone or in combination, and this application embodiment does not limit them.

[0151] In scenarios 3 and 4, it is necessary to determine which object is distant and which is near. In some embodiments, objects with a depth of field greater than a preset depth of field are distant objects, and objects with a depth of field less than a preset depth of field are near objects. The preset depth of field can be a specific depth value or a depth range; this embodiment does not limit this. The preset depth of field can be determined in various ways, including but not limited to at least one of the following methods.

[0152] Method 1: The preset depth of field can be determined based on the VR scene, and the preset depth of field varies depending on the VR scene. The VR scene includes, but is not limited to, at least one of VR games, VR movies, VR education, etc.

[0153] Taking VR games as an example, VR games include game characters, and the preset depth of field can be determined based on these characters. For instance, in a first-person game, the preset depth of field could be the depth of field of the user's corresponding game character within the game scene, or the depth of field of the user's corresponding game character's body parts, or the depth of field of the game equipment currently held by the user's corresponding game character. Assuming the game character is holding a gun, the depth of field of the arm or the gun can be determined as the preset depth of field.

[0154] Taking VR movie viewing as an example, VR movie viewing includes a display screen, and the depth of field where the display screen is located can be determined as a preset depth of field. Taking VR teaching as an example, VR teaching includes teaching equipment such as blackboards, display screens, and projectors, and the depth of field where the teaching equipment is located can be determined as a preset depth of field.

[0155] Method two: The preset depth of field can be user-defined. For example, a user can set the preset depth of field on VR glasses or an electronic device (such as a mobile phone) connected to the VR glasses. It should be understood that the electronic device includes various VR applications, and different VR applications can be set with different preset depths of field. Optionally, the user can set the preset depth of field for VR applications on the electronic device in batches, or they can set it individually for each VR application.

[0156] Method 3: The preset depth of field can also be the default depth of field. The default depth of field can be understood as the default setting of the VR glasses, or the default setting of the electronic device (such as a mobile phone) connected to the VR glasses, or the default setting of the VR application currently running on the electronic device (such as a mobile phone) connected to the VR glasses, etc. This application embodiment does not limit it.

[0157] Method four: The preset depth of field can also be based on the depth of field of the main object in the image currently displayed by the VR glasses. The main object can include the object occupying the largest area in the image, or the object located in the center of the image, or the virtual object in the image (such as the UI interface), etc.

[0158] Method 5: The preset depth of field is the depth of field where the user's gaze point is located. VR glasses may include an eye-tracking module, which can determine the user's gaze point, and the depth of field where the user's gaze point is located is the preset depth of field.

[0159] The above are several ways to determine the preset depth of field. Other methods are also feasible, and this application does not limit them.

[0160] In some embodiments, the electronic device uses the technical solution provided in this application when certain triggering conditions are met. For example, with Figure 8 For example, there are other images before the original image in the i-th frame ( Figure 8(Not shown in the image), these images may not undergo post-processing. Instead, the post-processing method provided in this application is used starting from the i-th frame. In other words, some images in the image stream do not use the post-processing method provided in this application. However, the post-processing method provided in this application is used when certain triggering conditions are detected.

[0161] For example, the triggering condition includes detecting that a user triggers an operation to activate the sharpness optimization mode. For instance, the electronic device has a normal mode and a sharpness optimization mode. In normal mode, after the electronic device renders an image stream, it displays the image stream through a display device, resulting in a low-resolution virtual environment for the user. When the electronic device activates the sharpness optimization mode, it performs image processing on the rendered image stream to improve its sharpness, thus enhancing the clarity of the virtual environment seen by the user and improving the user experience.

[0162] For example, the triggering condition also includes: detecting that the user's viewing time exceeds a first duration. That is, when the electronic device determines that the user's viewing time is relatively long, it automatically uses the technical solution provided in this application to improve the clarity of the virtual environment seen by the user.

[0163] For example, the triggering condition also includes: detecting that the number of times the user blinks or squints within the second time period is greater than a preset number. It is understood that if the object seen by the user is less clear, the user will unconsciously blink to try to see the object more clearly. Therefore, when the electronic device detects that the number of times the user blinks or squints within the second time period is greater than the preset number, it automatically uses the technical solution provided in this application to improve the clarity of the virtual environment seen by the user.

[0164] Although the description in this specification is presented in conjunction with some embodiments, this does not mean that the features of this application are limited to this embodiment. On the contrary, the purpose of describing the application in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of this specification. To provide a thorough understanding of this specification, many specific details will be included in the following description. This specification may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of this specification, some specific details will be omitted in the description. It should be noted that, unless otherwise specified, the embodiments and features described in this specification can be combined with each other.

[0165] Based on the same concept Figure 15 The image shows an electronic device 1500 provided in this application. This electronic device 1500 can be the VR glasses mentioned earlier. Figure 15As shown, the electronic device 1500 may include: one or more processors 1501; one or more memories 1502; a communication interface 1503; and one or more computer programs 1504. These devices can be connected via one or more communication buses 1505. The one or more computer programs 1504 are stored in the memory 1502 and configured to be executed by the one or more processors 1501. The one or more computer programs 1504 include instructions that can be used to perform the relevant steps of the VR glasses as described in the corresponding embodiments above. The communication interface 1503 is used to enable communication with other devices; for example, the communication interface may be a transceiver.

[0166] In the embodiments provided above, the methods provided by the present application are described from the perspective of an electronic device (e.g., VR glasses) as the executing entity. To implement the functions of the methods provided in the embodiments of the present application, the electronic device may include hardware structures and / or software modules, implementing the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is implemented in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0167] In the above embodiments, the terms "when..." or "after..." can be interpreted, depending on the context, as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, the phrases "when..." or "if (the stated condition or event) is detected" can be interpreted, depending on the context, as meaning "if...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)". Furthermore, in the above embodiments, relational terms such as "first" and "second" are used to distinguish one entity from another, without limiting any actual relationship or order between these entities.

[0168] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0169] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this embodiment are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)). Where there is no conflict, the solutions in the above embodiments can be combined.

[0170] It should be noted that a portion of this patent application contains copyrighted material. The copyright holder retains all rights except for making copies of the contents of patent documents or records from the patent office.

Claims

1. A display method, characterized in that, include: Display N frames of images using a display device; N is a positive integer; The clarity of the first object in the i-th frame of the N-frame images is the first clarity. The i-th frame image is obtained by processing the original i-th frame image based on image processing technology, and the clarity of the first object in the i-th frame image is higher than the clarity of the first object in the original i-th frame image. The clarity of the first object in the j-th frame of the N-frame images is the second clarity. The j-th frame image is obtained based on the original j-th frame image and intermediate results. The intermediate results are intermediate results generated during the process of processing the original i-th frame image to obtain the i-th frame image, so that the clarity of the first object in the j-th frame image is higher than the clarity of the first object in the original j-th frame image. The clarity of the first object in the k-th frame of the N-frame images is the third clarity. The k-th frame image is obtained by processing the original k-th frame image based on the image processing technology, and the clarity of the first object in the k-th frame image is higher than the clarity of the first object in the original k-th frame image. Wherein, the first resolution is greater than the second resolution, the second resolution is less than the third resolution, and i, j, and k are all positive integers less than N, i < j < k; The average of the first sharpness and the second sharpness is the fourth sharpness, which is higher than the average of the sixth sharpness and the seventh sharpness. The sixth sharpness is the sharpness of the first object on the i-th frame original image, and the seventh sharpness is the sharpness of the first object on the j-th frame original image. The average of the second and third sharpnesses is the fifth sharpness, which is higher than the average of the seventh and ninth sharpnesses. The ninth sharpness is the sharpness of the first object on the original image of the k-th frame.

2. The method according to claim 1, characterized in that, The first object is located within the region where the non-focal point is located, or, The first object is located at a first depth of field, which is greater than a preset depth of field.

3. The method according to claim 2, characterized in that, The sharpness of the second object remains unchanged in the N frames of images; The second object is located within the area of ​​the gaze point, or the second object is located at a second depth of field, which is less than the preset depth of field.

4. The method according to claim 3, characterized in that, The second object has a higher resolution than the first object.

5. The method according to claim 2, characterized in that, The sharpness of the second object on the i-th frame image is the tenth sharpness. The sharpness of the second object in the j-th frame image is the eleventh sharpness; The sharpness of the second object in the k-th frame image is the twelfth sharpness; The second object is located within the area of ​​the gaze point, or the second object is located at a second depth of field, which is less than the preset depth of field; The average of the tenth resolution and the eleventh resolution is the thirteenth resolution, and the average of the eleventh resolution and the twelfth resolution is the fourteenth resolution; The thirteenth level of resolution is higher than the fourth level of resolution, and the fourteenth level of resolution is higher than the fifth level of resolution.

6. The method according to claim 1, characterized in that, The first object is located within the area of ​​the gaze point; or, The first object is located at a first depth of field, which is less than a preset depth of field.

7. The method according to claim 6, characterized in that, The sharpness of the second object remains unchanged in the N frames of images; The second object is located within the area of ​​the non-focal point, or the second object is located at a second depth of field, which is greater than the preset depth of field.

8. The method according to claim 7, characterized in that, The clarity of the second object is lower than that of the first object.

9. The method according to claim 6, characterized in that, The resolution of the second object in the i-th frame image is the tenth resolution. The sharpness of the second object in the j-th frame image is the eleventh sharpness. The sharpness of the second object in the k-th frame image is the twelfth sharpness; The second object is located in the area where the non-focal point is located, or the second object is located in the second depth of field, which is greater than the preset depth of field; The average of the tenth resolution and the eleventh resolution is the thirteenth resolution, and the average of the eleventh resolution and the twelfth resolution is the fourteenth resolution; The thirteenth level of resolution is lower than the fourth level of resolution, and the fourteenth level of resolution is lower than the fifth level of resolution.

10. The method according to claim 2, characterized in that, During the display of the i-th frame, the j-th frame, and the k-th frame, the user's gaze point remains unchanged.

11. The method according to any one of claims 1-10, characterized in that, The time interval between the display time of the i-th frame and the display time of the j-th frame is less than or equal to the user's visual dwell time; and / or, The time interval between the display time of the j-th frame image and the display time of the k-th frame image is less than or equal to the user's visual dwell time.

12. The method according to any one of claims 1-10, characterized in that, Before displaying N frames of images via a display device, the following steps are also included: The system detects that the user has triggered an operation to activate the clarity optimization mode; the user's viewing time is longer than a first duration, or the number of times the user blinks or squints during a second duration is greater than a preset number.

13. An electronic device, characterized in that, include: Processor, memory, and one or more programs; The one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the processor, cause the electronic device to perform the steps of the method as described in any one of claims 1-12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 12.

15. A computer program product, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-12.

Images