Display method and system based on jitter super-resolution

Abstract

The invention provides a display method and system based on jitter super-resolution, and the method comprises the steps: obtaining a display mode of virtual reality equipment, and determining the weight of an initial image according to the display mode and a display content type; the display mode comprises a high-definition display mode, a low-power-consumption display mode and a self-adaptive display mode; performing image processing on the initial image according to the weight of the initial image to obtain a target image; updating a display frame rate and a jitter ratio according to the display mode; displaying the target image based on the display frame rate and the jitter multiplying power; therefore, the display mode can be adjusted according to application scene requirements, and the display frame rate and the jitter multiplying power can be updated in real time according to the display mode, so that the dynamic balance of the resolution ratio and the power consumption is realized, and the endurance time of equipment is prolonged while the display quality is ensured.

Description

Technical Field

[0001] This invention relates to the field of virtual reality display technology, and in particular to a display method and system based on dithering super-resolution. Background Technology

[0002] With the rapid development of Augmented Reality (AR), Mixed Reality (MR), and Extended Reality (XR) technologies, the demands on display systems are increasing, leading to the application of wobulation super-resolution technology in virtual reality devices. Wobulation super-resolution is a technique that improves display resolution by controlling minute displacements of the display. Its basic principle utilizes the persistence of vision in the human eye, displaying multiple slightly shifted images within a short period and using the minute displacement information between these images to reconstruct a high-resolution image.

[0003] However, in existing dithering super-resolution devices, there is a fixed relationship between resolution and power consumption. Generally speaking, the higher the resolution, the greater the power consumption. Therefore, it is impossible to achieve a dynamic balance between resolution and power consumption, and it is impossible to adapt to the dynamic needs of different display scenarios. Consequently, it is impossible to balance the display quality and battery life of the device. Summary of the Invention

[0004] To address the problems existing in the prior art, embodiments of the present invention provide a frame rate control method and system based on jitter super-resolution display, in order to solve or partially solve the technical problem in the prior art that it is impossible to achieve a dynamic balance between resolution and power consumption in virtual reality devices, thereby failing to extend the device's battery life while ensuring display quality.

[0005] A first aspect of the present invention provides a display method based on dithering super-resolution, the method comprising: The display mode of the virtual reality device is obtained, and the weight of the initial image is determined based on the display mode and the type of display content; the display mode includes: high-definition display mode, low-power display mode and adaptive display mode; The initial image is processed according to its weights to obtain the target image; Update the display frame rate and jitter rate according to the display mode; The target image is displayed based on the display frame rate and the jitter ratio.

[0006] In the above scheme, determining the weight of the initial image based on the display mode and the type of display content includes: When the displayed content type is text, determine the region location information of the initial image; The corresponding weight range is determined based on the regional location information; The target weights corresponding to each region in the initial image are determined based on the display mode and the weight range.

[0007] In the above scheme, determining the target weights corresponding to each region in the initial image based on the display mode and the weight range includes: When the region is the visual center region, the maximum weight of the weight range corresponding to the visual center region is determined as the target weight of the visual center region. When the region is an edge region, the maximum weight of the weight range corresponding to the edge region is determined as the target weight of the edge region. When the region is an intermediate transition region, the maximum weight of the weight range corresponding to the intermediate transition region is determined as the target weight of the intermediate transition region.

[0008] In the above scheme, updating the display frame rate and jitter ratio according to the display mode includes: If the display mode is the high-definition display mode, determine whether the battery level of the virtual reality device is greater than or equal to the first battery threshold. If so, adjust the display frame rate to 120Hz~240Hz and adjust the jitter factor of the super-resolution to the first factor.

[0009] In the above scheme, updating the display frame rate and jitter ratio according to the display mode includes: If the display mode is the low-power mode, the display frame rate is adjusted to 30Hz~60Hz, and the dithering ratio of the super-resolution is adjusted to a second multiple; the second multiple is less than the first multiple.

[0010] In the above scheme, updating the display frame rate and jitter ratio according to the display mode includes: If the display mode is the adaptive display mode, determine whether the battery level of the virtual reality device is greater than or equal to the first battery threshold. If so, determine the display content type corresponding to the eye gaze point. If the displayed content type is text, determine whether the ambient brightness is less than a preset brightness threshold. If so, update the display frame rate to 120Hz~240Hz and update the jitter factor of the super-resolution to the first factor. When the displayed content type is video and the screen motion speed is greater than a preset speed threshold, the display frame rate is updated to 30Hz~60Hz and the super-resolution jitter factor is updated to the second factor.

[0011] The method in the above scheme further includes: The power consumption of the virtual reality device is monitored according to a preset monitoring cycle. If the power consumption of the virtual reality device is less than or equal to a second power threshold, the display mode is switched to a low power mode.

[0012] In the above scheme, after the display mode is switched to low power mode, the method further includes: Multiple intermediate resolutions are generated using a smooth transition mechanism; these multiple high-definition resolutions are located between the original resolution and the resolution corresponding to the low-power display mode. The resolution is adjusted frame by frame until it reaches the resolution corresponding to the low-power display mode.

[0013] In the above scheme, displaying the target image based on the display frame rate and the jitter ratio includes: Extract the corresponding jitter ratio from the jitter ratio, and split each target image into a corresponding number of sub-frame images based on the jitter ratio; The subframe image is transmitted to the optical engine according to the display frame rate, and the optical engine is used to convert the subframe image into an optical signal; Under the action of the synchronization signal, the optical signal of each subframe image is offset according to the received control signal, the offset optical signal is transmitted to the optical waveguide for coupling, and the coupled optical signal is transmitted to the near-eye display screen for display.

[0014] A second aspect of the present invention provides a display system based on dithering super-resolution, the system comprising: The main control chip is used to acquire the display mode of the virtual reality device, and determine the weight of the initial image based on the display mode and the type of display content. The display modes include: high-definition display mode, low-power display mode, and adaptive display mode. The initial image is processed according to the weight of the initial image to obtain the target image. The display frame rate and jitter ratio are updated according to the display mode. Each target image is divided into a corresponding number of sub-frame images based on the jitter ratio, and each sub-frame image is transmitted to the optical engine according to the display frame rate. An optical engine is used to convert each of the subframe images into optical signals; The beam deflection element deflects the optical signal of each subframe image according to the received control signal, and transmits the deflected optical signal to the optical waveguide for coupling. Near-eye display screen, used to display coupled light signals.

[0015] This invention provides a display method and system based on jitter super-resolution. The method includes: acquiring the display mode of a virtual reality device; determining the weight of an initial image based on the display mode and the type of display content; the display mode includes: a high-definition display mode, a low-power display mode, and an adaptive display mode; performing image processing on the initial image according to the weight of the initial image to obtain a target image; updating the display frame rate and jitter ratio according to the display mode; and displaying the target image based on the display frame rate and the jitter ratio. Thus, the display mode can be adjusted according to the application scenario requirements, and the display frame rate and jitter ratio can be updated in real time according to the display mode, thereby achieving a dynamic balance between resolution and power consumption, ensuring display quality while extending device battery life. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A schematic diagram of a display system structure based on dithering super-resolution according to an embodiment of the present invention is shown; Figure 2 A schematic diagram of a beam deflection element according to an embodiment of the present invention is shown; Figure 3 A schematic diagram illustrating the segmentation of a target image based on jitter magnification according to an embodiment of the present invention is shown; Figure 4 A schematic flowchart of a dithering super-resolution display method according to an embodiment of the present invention is shown. Detailed Implementation

[0017] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0018] This invention provides a display system based on dithering super-resolution, such as... Figure 1 As shown, the system includes: a main control chip 1, an optical engine 2, a beam deflection element 3, a near-eye display screen 4, and an optical waveguide 5; The main control chip 1 can be an embedded system-on-a-chip (SOC) or other processors; no restrictions are placed here.

[0019] Optical engine 2 can be a miniature projector or a miniature display, specifically an organic light-emitting diode (OLED) display panel or a liquid crystal display (LCD). Both the resolution and frame rate of optical engine 2 are adjustable.

[0020] The beam deflection element 3 can be an adjustable wedge (TWedge) device based on a micro-electro-mechanical system (MEMS), or other optical devices capable of deflection; no limitation is made here.

[0021] refer to Figure 2 The adjustable wedge device includes a first transparent plate 211, a second transparent plate 212, and a compressible medium 213 disposed between the first transparent plate 211 and the second transparent plate 212. The driving device 22 includes at least one control electrode 221 disposed on the first transparent plate 211 and / or the second transparent plate 212. The control electrode 221 is used to drive the first transparent plate 211 and / or the second transparent plate 212 to deflect. Generally, the control electrode 221 is disposed on the outer surface of the first transparent plate 211 away from the second transparent plate 212 and / or on the outer surface of the second transparent plate 212 away from the first transparent plate 211. A driving circuit 222 is connected to the control electrode 221. The driving circuit 222 is disposed on the side of the first transparent plate 211 away from the second transparent plate 212.

[0022] When the adjustable wedge device is in the initial state, the first transparent plate 211 and the second transparent plate 212 are parallel to each other so that the adjustable wedge device is in a plate shape; when the adjustable wedge device is in the deflection state, the first transparent plate 211 and / or the second transparent plate 212 deflect and squeeze the compressible medium 213 so that the adjustable wedge device is in a wedge shape.

[0023] In an exemplary embodiment, the first transparent plate 211 and the second transparent plate 212 can be made of transparent glass or transparent plastic.

[0024] In an exemplary embodiment, the material of the compressible medium 213 can be a liquid or an elastically compressible solid. Specifically, the liquid can be liquid crystal, oil, water, or a mixture of the above substances, and the solid can be silicone resin, polymer gel, etc.

[0025] Therefore, TWedge devices possess the ability to electrically control various deformations such as bending, twisting, stretching, and compression, enabling precise and controllable deflection of the optical path in both the X and Y dimensions during transmission. Beam deflection element 3 has the following characteristics: supports I2C interface; response time less than 1ms; independent controllable deflection angles in both the X and Y dimensions; and deflection accuracy higher than 0.1 degrees.

[0026] The main control chip 1 is used to acquire the display mode of the virtual reality device, and determine the weight of the initial image according to the display mode and the type of display content. The display modes include: high-definition display mode, low-power display mode and adaptive display mode. The initial image is processed according to the weight of the initial image to obtain the target image. The display frame rate and jitter ratio are updated according to the display mode. Each target image is divided into a corresponding number of sub-frame images based on the jitter ratio, and each sub-frame image is transmitted to the optical engine according to the display frame rate. Optical engine 2 is used to convert each subframe image into an optical signal; The beam deflection element 3 deflects the optical signal of each subframe image according to the received control signal, and transmits the deflected optical signal to the optical waveguide 5 for coupling. Near-eye display screen 4 is used to display the coupled light signal.

[0027] Different types of display content require different frame rates and jitter levels. For example, text-based content generally requires sharp text edges, thus necessitating a higher frame rate and a larger jitter level. However, for video-based content, the human eye's sensitivity to dynamic image resolution decreases, so a higher frame rate and a larger jitter level are not necessary.

[0028] After the main control chip 1 obtains the display mode and jitter ratio, it can divide each target image into a corresponding number of sub-frame images based on the jitter ratio. The subframe image is transmitted to the optical engine 2 according to the display frame rate, and the optical engine 2 is used to convert the subframe image into an optical signal; Under the action of the synchronization signal, the beam deflection element 3 deflects the optical signal of each subframe image according to the received control signal, transmits the deflected optical signal to the optical waveguide 5 for coupling, and transmits the coupled optical signal to the near-eye display screen 4 for display.

[0029] Specifically, in order to further improve the display quality, the main control chip 1 needs to determine the weight of the initial image according to the display mode and the type of display content, and then process the initial image according to the weight of the initial image to obtain the target image, so as to obtain the target image that meets the needs of the actual application scenario.

[0030] The displayed content types generally include text, image, and video. For text, weights can be determined for the initial image based on a weight model of visual attention spatial location. For image, weights can be determined for the initial image based on a weight model of content detail features. For video, weights can be determined for the initial image based on motion stability. Content detail features can include frequency details (e.g., high-frequency, mid-frequency, and low-frequency features).

[0031] In one implementation, determining the weight of the initial image based on the display mode and the type of displayed content includes: When the displayed content type is text, determine the region location information of the initial image; Determine the corresponding weight range based on the regional location information; The target weights for each region in the initial image are determined based on the display mode and weight range.

[0032] In one implementation, determining the target weights corresponding to each region in the initial image based on the display mode and weight range includes: When the region is the visual center region, the maximum weight of the weight range corresponding to the visual center region is determined as the target weight of the visual center region. When the region is located in the edge region, the maximum weight of the weight range corresponding to the edge region is determined as the target weight of the edge region. When the region is an intermediate transition region, the maximum weight of the weight range corresponding to the intermediate transition region is determined as the target weight of the intermediate transition region.

[0033] Specifically, eye-tracking algorithms can be used to locate the gaze point of the human eye. The visual center region is defined as the visual field range of less than 3° around the gaze point, the intermediate transition region is defined as the visual field range of 3° to 8° around the gaze point, and the edge region is defined as the visual field range of more than 8° around the gaze point.

[0034] The process of locating the human eye's gaze point using an eye-tracking algorithm is as follows: Virtual reality devices are typically equipped with high-resolution near-infrared cameras and infrared light sources. The infrared light source emits near-infrared light to illuminate the eye. Since infrared light is invisible to the human eye, it does not cause interference and can provide stable illumination under various lighting conditions. When the infrared light hits the eye, it forms a bright reflective point on the cornea, known as the corneal reflector. Simultaneously, after entering the pupil, the infrared light undergoes diffuse reflection at the fundus, with some light re-exiting through the pupil, making the pupil appear as a darker area in the infrared image. The near-infrared camera continuously captures a video stream of the eye area, providing image data for subsequent processing.

[0035] The acquired eye images are processed to extract key features, such as the pupil center and corneal reflective points, using edge detection or template matching methods. For example, edge detection algorithms can identify the pupil boundary, thus determining the location of the pupil center; template matching methods can find the region in the image most similar to the corneal reflective point template, thereby determining the location of the corneal reflective point.

[0036] Based on extracted ocular features such as the pupil center and corneal reflection point, the user's current gaze point position is calculated using geometric models or machine learning algorithms. The geometric model can include a 3D eye model, a pupil-corneal reflection model, etc. The geometric model can calculate the gaze direction and thus determine the gaze point position using the known geometric positions of the camera and light source, as well as the vector relationship between the pupil center and the corneal reflection point.

[0037] Machine learning algorithms can learn the mapping relationship between eye features and fixation points by training on a large number of eye images and corresponding fixation point data. After obtaining the eye features, the eye features can be used as input parameters for the machine learning algorithm, and then the fixation point position can be output.

[0038] When users view text-based content (such as viewing drawings or reading text), they need clear, jagged edges. Therefore, the visual center area should have a higher weight, the intermediate transition area a medium weight, and the edge area a lower weight. For example, the weight range for the visual center area could be 0.5 to 0.7, the weight range for the intermediate transition area could be 0.2 to 0.4, and the weight range for the edge area could be 0.1 to 0.2.

[0039] Once the weight range is determined, the final weights can be determined based on the display mode. When the display mode is high definition, the weight corresponding to the visual center area can be 0.7, the weight corresponding to the intermediate transition area can be 0.4, and the weight corresponding to the edge area can be 0.2. After determining the weights for each area, it is necessary to normalize each weight so that the sum of the weights for each area is 1.

[0040] Similarly, when users are viewing image-based content (such as photos or website images), it's important to highlight high-frequency details. High-frequency features can be assigned higher weights, mid-frequency features can be assigned medium weights, and low-frequency features can be assigned lower weights. For example, the weight range for high-frequency features could be 0.5 to 0.7, for mid-frequency features 0.3 to 0.4, and for edge regions 0.1 to 0.2.

[0041] When users are watching video content, it's necessary to highlight stable regions. This can be achieved by calculating optical flow vectors for consecutive subframes in the video frame. Regions with optical flow vectors less than 0.5 pixels per frame are defined as stable regions, while regions with optical flow vectors greater than 0.5 pixels per frame are defined as unstable regions. For stable regions, the weighting range can be increased by 20% from the original weighting range; conversely, for unstable regions, the weighting range can be decreased by 20% from the original weighting range.

[0042] In addition, weights can be determined for target images of different display content types using deep neural networks, or based on user interaction data.

[0043] For example, some users prefer static images, while others prefer dynamic images. Therefore, user interaction data (such as zooming and clicking data) can be recorded and weighted accordingly. For instance, for designer users, the weight of edge areas could be increased by 20% on top of the existing weights, while for video enthusiasts, the weight of static areas could be decreased by 15% on top of the existing weights.

[0044] Once the weights of the initial image are determined, image processing can be performed on the initial image according to its weights to differentiate and enhance different regions of the image, thereby obtaining the target image.

[0045] For example, consider a nature image containing birds (the visual center region), tree branches (the intermediate transition region), and the sky (the edge region). The image is divided into three regions, and the normalized weights for each region are as follows: Visual center region (birds): weight W1=0.7; Intermediate transition region (tree branches): weight W2 = 0.2; Edge region (sky): weight W3=0.1.

[0046] This invention can sharpen and enhance images by adjusting pixel values, and the weight directly determines the sharpening intensity.

[0047] For all pixels in the three regions, the Sobel operator is used to calculate the edge gradient values, which are used to measure the strength of pixel edge features. For example, the gradient value of a pixel in the bird region is g11=40; the gradient value of a pixel in the tree branch region is g21=20; and the gradient value of a pixel in the sky region is g31=10.

[0048] Determine the sharpening increment based on weights. As shown in formula (1): (1) In formula (1), For the firsti The weight of each region For the first i The first region j Edge gradient values ​​of 1 pixel, This is the scaling factor (used to control the overall amplitude, typically set to 0.1).

[0049] Once the sharpening increment for each pixel is determined, the sum of the sharpening increment and the original brightness value can be used as the target brightness value for that pixel. This results in a new target image.

[0050] In the target image, pixels in high-weight regions (visual center) are adjusted significantly, with details being emphasized; pixels in low-weight regions (edges) are adjusted less, maintaining basic information. This mechanism ensures that image enhancement resources are concentrated on visually critical areas, improving the quality of core content while avoiding over-processing of irrelevant areas.

[0051] This allows for maximum differentiation enhancement of the initial image with limited computing resources, improving the image quality perceived by the user, especially the clarity and detail in the visual center area.

[0052] Furthermore, in order to dynamically balance resolution and power consumption, the main control chip 1 also needs to update the display frame rate of the optical engine 2 and the dithering ratio of the beam deflection element 3 according to the display mode. The dithering ratio is the dithering ratio of the super-resolution.

[0053] Dithering super-resolution technology uses optical deflection to achieve pixel displacement, allowing the same physical pixel to correspond to different virtual pixel positions at different times, thereby improving perceptual resolution through time integration. Therefore, the higher the dithering ratio, the higher the perceptual resolution.

[0054] Specifically, assuming the target image is a high-resolution 4K image (3840*2160 resolution), but because the physical resolution of the display panel of the optical engine 2 is low (1920*1080), this panel itself cannot display all 3840*2160 independent pixels in a static frame, as it only has 1920*1080 physical pixels. Therefore, this invention utilizes the persistence of vision characteristic of the human eye to distribute the pixels of a high-resolution target image across time (multiple frames) and space (pixel offset) for display, synthesizing a complete high-resolution image in the viewer's eye. First, the main control chip 1 needs to perform a splitting process on the initial image according to the jitter ratio, including: The number of subframes to be split is determined based on the jitter ratio, and the corresponding splitting strategy is determined based on the number of subframes; the jitter ratio and the number of subframes to be split are consistent. The splitting strategy extracts pixels from different locations in the target image to form multiple pixel subsets; each pixel subset corresponds to a subframe image.

[0055] The number of subframes can be determined based on the jitter factor. For example, if the jitter factor is 4, then there will be 4 subframes. Taking 4 subframes as an example, the target image can be split into 4 subframes.

[0056] like Figure 3 As shown, during the splitting process, pixels at the intersection of odd-numbered rows and odd-numbered columns in the target image are extracted as the first frame image. For example, pixels at coordinates (1,1), (1,3), (1,5) ... (3,1), (3,3) ... are extracted to form the first pixel subset, which is the first subframe image.

[0057] Pixels at the intersection of odd-numbered rows and even-numbered columns in the target image are extracted as the second frame image. For example, pixels at coordinates (1,2), (1,4), (1,6)...(3,2), (3,4)... are extracted to form the second pixel subset, which is the second subframe image.

[0058] Pixels at the intersection of even-numbered rows and odd-numbered columns in the target image are extracted as the third frame image. For example, pixels at coordinates (2,1), (2,3), (2,5)...(4,1), (4,3)... are extracted to form the third pixel subset, which is the third subframe image.

[0059] Pixels at the intersection of even-numbered rows and even-numbered columns in the target image are extracted as the fourth frame image. For example, pixels at coordinates (2,2), (2,4), (2,6) ... (4,2), (4,4) ... are extracted to form the fourth pixel subset, which is the fourth sub-frame image.

[0060] Each of these four subframe images has a resolution of 1920*1080 pixels (because 3840 / 2=1920, 2160 / 2=1080). Each subframe contains only 1 / 4 of the pixel information of the target image.

[0061] After obtaining multiple sub-frame images, the main control chip 1 needs to generate a corresponding control signal based on the target position of each sub-frame image and send the control signal to the beam deflection element 3. The control signal carries the sequence number of each sub-frame image, as well as the offset and offset direction of each sub-frame image. The control signal is used to drive the beam deflection element 3 to perform corresponding two-dimensional deflection (e.g., a small angular offset in the X / Y direction), thereby achieving precise displacement of the sub-frame image on the retina. After receiving the optical signal corresponding to the sub-frame image, the beam deflection element 3, under the action of the synchronization signal, offsets the optical signal of each sub-frame image according to the control signal, and transmits the offset optical signal to the optical waveguide 5. The optical waveguide 5 couples the signal into a high-resolution image and outputs it to the near-eye display screen 4 for display.

[0062] It is known that higher resolution requires more power, but some applications do not require high resolution. Therefore, this invention can update the display frame rate and jitter rate according to the display mode, including: If the display mode is the high-definition display mode, determine whether the battery level of the virtual reality device is greater than or equal to the first battery threshold. If so, adjust the display frame rate to 120Hz~240Hz and adjust the jitter factor of the super-resolution to the first factor.

[0063] Specifically, users can choose the display mode according to the application scenario, and the main control chip 1 can receive the corresponding display mode. For example, if the application scenario is viewing drawings, refining images, reading documents, and processing text, the user can choose the high-definition display mode, which is a high frame rate and high resolution display mode. After the main control chip 1 obtains the high-definition display mode, it will determine whether the battery level of the virtual reality device is greater than or equal to a first battery threshold (e.g., 70%). If so, it will adjust the display frame rate of the optical engine 2 to 120Hz~240Hz and adjust the super-resolution jitter multiplier to the first multiplier; the first multiplier can be 4 times.

[0064] The beam deflection element 3 generates subpixel displacement according to a 4x super-resolution. Subframe images are offset according to their respective subpixel displacements. Assuming each subframe image is moved by half a physical pixel, the offset physical positions are (0, 0), (0.5, 0), (0, 0.5), and (0.5, 0.5). When these four subframe images are displayed sequentially in this physical area at an extremely high speed (e.g., 240Hz), the human eye cannot distinguish the rapidly switching individual light spots. The retina will fuse the information of these four light spots—the upper left, upper right, lower left, and lower right corners—which appear at different times but are extremely close in position.

[0065] The brain ultimately perceives this fused information as the simultaneous existence of four independent pixels within this physical area, located at the top left, top right, bottom left, and bottom right positions, respectively. Thus, an area that could originally only display a single, "average" piece of information becomes clearly distinguishable to the human eye as containing the details of four independent pixels from the original 4K resolution, thereby achieving an improvement in resolution.

[0066] Furthermore, when the application scenario is video playback, the human eye's sensitivity to dynamic image resolution decreases; when the application scenario is game playback, social media interface, etc., the human eye's requirement for real-time performance is higher than resolution; and in outdoor strong light scenarios, brightness requirements take precedence over resolution. In these situations, to reduce power consumption, users can choose a low-power display mode, which is a low frame rate low-power display mode. After the main control chip 1 obtains the low-power display mode, it will adjust the display frame rate of the optical engine 2 to 30Hz~60Hz and adjust the super-resolution jitter factor to a second multiple. The second multiple is less than the first multiple.

[0067] Furthermore, if the device is in a strong light scene, after entering the low power mode, the device brightness needs to be adjusted to less than 50% of the total brightness to maintain the driving voltage of the beam deflection element 3 in a basic bias state.

[0068] In another implementation, updating the display frame rate and jitter ratio according to the display mode includes: If the display mode is adaptive display mode, determine whether the battery level of the virtual reality device is greater than or equal to the first battery threshold. If so, determine the type of display content corresponding to the eye gaze point. If the displayed content type is text, determine whether the ambient brightness is less than the preset brightness threshold. If so, update the display frame rate to 120Hz~240Hz and update the super-resolution jitter ratio to the first multiple. Specifically, in adaptive display mode, the main control chip 1 can adjust the display frame rate and jitter rate according to the type of displayed content, the user's eye movements, and the power consumption of the virtual reality device, finding the best balance between power consumption and resolution.

[0069] For example, eye-tracking algorithms can be used to determine the user's visual center area and assess the type of content displayed in that area. If the displayed content is text and the ambient brightness is lower than a preset brightness value, it indicates that the user requires a stronger visual experience. In this case, the display mode can be switched to high-definition display mode, which means updating the display frame rate to 120Hz~240Hz and updating the super-resolution jitter factor to the first multiple.

[0070] Conversely, if the user is watching a fast-moving video and their gaze is moving quickly, the jitter complexity and frame rate can be appropriately reduced to balance power consumption. So when the displayed content is a video and the speed of the screen movement is greater than the preset speed threshold, the display frame rate will be updated to 30Hz~60Hz and the jitter multiplier of the super-resolution will be updated to the second multiplier.

[0071] However, regardless of whether it is in high-definition display mode or adaptive display mode, the main control chip 1 is also used for: The power consumption of the virtual reality device is monitored according to a preset monitoring cycle. If the power consumption of the virtual reality device is less than or equal to the second power threshold, the display mode will be switched to low power mode.

[0072] As can be seen, during the display process, once the battery level of the virtual reality device is determined to be less than the second battery threshold (e.g., 30%), it automatically switches to low-power mode. This means updating the display frame rate to 30Hz~60Hz and updating the super-resolution jitter factor to the second multiple. The second multiple is less than the first multiple, and the second multiple can be 2 or 0. In other words, when the battery level of the virtual reality device is too low, the display frame rate and resolution will be reduced (by turning off the operation of the beam deflection element 3 or reducing the jitter factor), and the device will automatically switch to low-power mode to improve battery life.

[0073] Furthermore, to avoid screen flickering or stuttering during mode switching, which could affect image quality, the main control chip 1 needs to implement a smooth transition mechanism during mode switching (including switching between any modes). Taking the switch from high-definition mode to low-power mode as an example, in one implementation, after switching the display mode to low-power mode, the main control chip 1 is also used for: Multiple intermediate resolutions are generated using a smooth transition mechanism; multiple high-definition resolutions are located between the original resolution and the resolution corresponding to the low-power display mode. The resolution is adjusted frame by frame until it reaches the resolution corresponding to the low-power display mode.

[0074] In other words, during the mode switching process, the resolution does not switch directly from the resolution corresponding to the high-definition display mode to the resolution corresponding to the low-power display mode. Instead, a smooth transition mechanism is used to generate multiple intermediate resolutions between the high-definition display mode resolution and the low-power display mode resolution. Then, the resolution is adjusted frame by frame until the resolution corresponding to the low-power display mode is reached, thus completing the switching process.

[0075] Based on the same inventive concept as the foregoing embodiments, the present invention also provides a frame rate control method based on jitter-based super-resolution display, such as... Figure 4 The method includes the following steps: S410, acquire the display mode of the virtual reality device, and determine the weight of the initial image based on the display mode and the type of display content; the display mode includes: high-definition display mode, low-power display mode and adaptive display mode.

[0076] To further improve the display quality, the main control chip 1 needs to determine the weight of the initial image based on the display mode and the type of display content, and then process the initial image according to the weight of the initial image to obtain the target image, thereby processing the initial image to obtain the target image that meets the requirements of the actual application scenario.

[0077] The displayed content types generally include text, image, and video. For text, weights can be determined for the initial image based on a weight model of visual attention spatial location. For image, weights can be determined for the initial image based on a weight model of content detail features. For video, weights can be determined for the initial image based on motion stability. Content detail features can include frequency details (e.g., high-frequency, mid-frequency, and low-frequency features).

[0078] In one implementation, determining the weight of the initial image based on the display mode and the type of displayed content includes: When the displayed content type is text, determine the region location information of the initial image; Determine the corresponding weight range based on the regional location information; The target weights for each region in the initial image are determined based on the display mode and weight range.

[0079] In one implementation, determining the target weights corresponding to each region in the initial image based on the display mode and weight range includes: When the region is the visual center region, the maximum weight of the weight range corresponding to the visual center region is determined as the target weight of the visual center region. When the region is located in the edge region, the maximum weight of the weight range corresponding to the edge region is determined as the target weight of the edge region. When the region is an intermediate transition region, the maximum weight of the weight range corresponding to the intermediate transition region is determined as the target weight of the intermediate transition region.

[0080] Specifically, eye-tracking algorithms can be used to locate the gaze point of the human eye. The visual center region is defined as the visual field range of less than 3° around the gaze point, the intermediate transition region is defined as the visual field range of 3° to 8° around the gaze point, and the edge region is defined as the visual field range of more than 8° around the gaze point.

[0081] The process of locating the human eye's gaze point using an eye-tracking algorithm is as follows: Virtual reality devices are typically equipped with high-resolution near-infrared cameras and infrared light sources. The infrared light source emits near-infrared light to illuminate the eye. Since infrared light is invisible to the human eye, it does not cause interference and can provide stable illumination under various lighting conditions. When the infrared light hits the eye, it forms a bright reflective point on the cornea, known as the corneal reflector. Simultaneously, after entering the pupil, the infrared light undergoes diffuse reflection at the fundus, with some light re-exiting through the pupil, making the pupil appear as a darker area in the infrared image. The near-infrared camera continuously captures a video stream of the eye area, providing image data for subsequent processing.

[0082] The acquired eye images are processed to extract key features, such as the pupil center and corneal reflective points, using edge detection or template matching methods. For example, edge detection algorithms can identify the pupil boundary, thus determining the location of the pupil center; template matching methods can find the region in the image most similar to the corneal reflective point template, thereby determining the location of the corneal reflective point.

[0083] Based on extracted ocular features such as the pupil center and corneal reflection point, the user's current gaze point position is calculated using geometric models or machine learning algorithms. The geometric model can include a 3D eye model, a pupil-corneal reflection model, etc. The geometric model can calculate the gaze direction and thus determine the gaze point position using the known geometric positions of the camera and light source, as well as the vector relationship between the pupil center and the corneal reflection point.

[0084] Machine learning algorithms can learn the mapping relationship between eye features and fixation points by training on a large number of eye images and corresponding fixation point data. After obtaining the eye features, the eye features can be used as input parameters for the machine learning algorithm, and then the fixation point position can be output.

[0085] When users view text-based content (such as viewing drawings or reading text), they need clear, jagged edges. Therefore, the visual center area should have a higher weight, the intermediate transition area a medium weight, and the edge area a lower weight. For example, the weight range for the visual center area could be 0.5 to 0.7, the weight range for the intermediate transition area could be 0.2 to 0.4, and the weight range for the edge area could be 0.1 to 0.2.

[0086] Once the weight range is determined, the final weights can be determined based on the display mode. When the display mode is high definition, the weight corresponding to the visual center area can be 0.7, the weight corresponding to the intermediate transition area can be 0.4, and the weight corresponding to the edge area can be 0.2. After determining the weights for each area, it is necessary to normalize each weight so that the sum of the weights for each area is 1.

[0087] Similarly, when users are viewing image-based content (such as photos or website images), it's important to highlight high-frequency details. High-frequency features can be assigned higher weights, mid-frequency features can be assigned medium weights, and low-frequency features can be assigned lower weights. For example, the weight range for high-frequency features could be 0.5 to 0.7, for mid-frequency features 0.3 to 0.4, and for edge regions 0.1 to 0.2.

[0088] When users are watching video content, it's necessary to highlight stable regions. This can be achieved by calculating optical flow vectors for consecutive subframes in the video frame. Regions with optical flow vectors less than 0.5 pixels per frame are defined as stable regions, while regions with optical flow vectors greater than 0.5 pixels per frame are defined as unstable regions. For stable regions, the weighting range can be increased by 20% from the original weighting range; conversely, for unstable regions, the weighting range can be decreased by 20% from the original weighting range.

[0089] In addition, weights can be determined for target images of different display content types using deep neural networks, or based on user interaction data.

[0090] For example, some users prefer static images, while others prefer dynamic images. Therefore, user interaction data (such as zooming and clicking data) can be recorded and weighted accordingly. For instance, for designer users, the weight of edge areas could be increased by 20% on top of the existing weights, while for video enthusiasts, the weight of static areas could be decreased by 15% on top of the existing weights.

[0091] S411, perform image processing on the initial image according to the weight of the initial image to obtain the target image.

[0092] Once the weights of the initial image are determined, image processing can be performed on the initial image based on its weights to perform differential enhancement on the same region of the image, thereby obtaining the target image.

[0093] For example, consider a nature image containing birds (the visual center region), tree branches (the intermediate transition region), and the sky (the edge region). The image is divided into three regions, and the normalized weights for each region are as follows: Visual center region (birds): weight W1=0.7; Intermediate transition region (tree branches): weight W2 = 0.2; Edge region (sky): weight W3=0.1.

[0094] This invention can sharpen and enhance images by adjusting pixel values, and the weight directly determines the sharpening intensity.

[0095] For all pixels in the three regions, the Sobel operator is used to calculate the edge gradient values, which are used to measure the strength of pixel edge features. For example, the gradient value of a pixel in the bird region is g11=40; the gradient value of a pixel in the tree branch region is g21=20; and the gradient value of a pixel in the sky region is g31=10.

[0096] Determine the sharpening increment based on weights. As shown in formula (1): (1) In formula (1), For the first i The weight of each region For the first i The first region j Edge gradient values ​​of 1 pixel, This is the scaling factor (used to control the overall amplitude, typically set to 0.1).

[0097] Once the sharpening increment for each pixel is determined, the sum of the sharpening increment and the original brightness value can be used as the target brightness value for that pixel. This results in a new target image.

[0098] In the target image, pixels in high-weight regions (visual center) are adjusted significantly, with details being emphasized; pixels in low-weight regions (edges) are adjusted less, maintaining basic information. This mechanism ensures that image enhancement resources are concentrated on key visual areas, improving the quality of core content while avoiding over-processing of irrelevant areas. This maximizes the differential enhancement of the initial image with limited computational resources, improving the user's perceived image quality, especially the sharpness and detail in the visual center region.

[0099] S412, Update the display frame rate and jitter ratio according to the display mode.

[0100] To dynamically balance resolution and power consumption, the display frame rate of the optical engine and the dithering factor of the beam deflection elements also need to be updated according to the display mode. The dithering factor is the dithering factor of the super-resolution.

[0101] Dithering super-resolution technology uses optical deflection to achieve pixel displacement, allowing the same physical pixel to correspond to different virtual pixel positions at different times, thereby improving perceptual resolution through time integration. Therefore, the higher the dithering ratio, the higher the perceptual resolution.

[0102] Specifically, assuming the target image is a high-resolution 4K image (3840*2160 resolution), but the physical resolution of the optical engine's display panel is low (1920*1080), this panel itself cannot display all 3840*2160 independent pixels in a single static frame because it only has 1920*1080 physical pixels. Therefore, this invention utilizes the persistence of vision characteristic of the human eye to distribute the pixels of a high-resolution target image across time (multiple frames) and space (pixel offset) for display, synthesizing a complete high-resolution image in the viewer's eye. First, the initial image needs to be split according to the jitter ratio, including: The number of subframes to be split is determined based on the jitter ratio, and the corresponding splitting strategy is determined based on the number of subframes; the jitter ratio and the number of subframes to be split are consistent. The splitting strategy extracts pixels from different locations in the target image to form multiple pixel subsets; each pixel subset corresponds to a subframe image.

[0103] The number of subframes can be determined based on the jitter factor. For example, if the jitter factor is 4, then there will be 4 subframes. Taking 4 subframes as an example, the target image can be split into 4 subframes.

[0104] like Figure 3 As shown, during the splitting process, pixels at the intersection of odd-numbered rows and odd-numbered columns in the target image are extracted as the first frame image. For example, pixels at coordinates (1,1), (1,3), (1,5) ... (3,1), (3,3) ... are extracted to form the first pixel subset, which is the first subframe image.

[0105] Pixels at the intersection of odd-numbered rows and even-numbered columns in the target image are extracted as the second frame image. For example, pixels at coordinates (1,2), (1,4), (1,6)...(3,2), (3,4)... are extracted to form the second pixel subset, which is the second subframe image.

[0106] Pixels at the intersection of even-numbered rows and odd-numbered columns in the target image are extracted as the third frame image. For example, pixels at coordinates (2,1), (2,3), (2,5)...(4,1), (4,3)... are extracted to form the third pixel subset, which is the third subframe image.

[0107] Pixels at the intersection of even-numbered rows and even-numbered columns in the target image are extracted as the fourth frame image. For example, pixels at coordinates (2,2), (2,4), (2,6) ... (4,2), (4,4) ... are extracted to form the fourth pixel subset, which is the fourth sub-frame image.

[0108] Each of these four subframe images has a resolution of 1920*1080 pixels (because 3840 / 2=1920, 2160 / 2=1080). Each subframe contains only 1 / 4 of the pixel information of the target image.

[0109] After obtaining multiple sub-frame images, a corresponding control signal needs to be generated based on the target position of each sub-frame image. This control signal is then sent to the beam deflection element. The control signal carries the sequence number of each sub-frame image, as well as its offset and offset direction. The control signal drives the beam deflection element to perform corresponding two-dimensional deflection (e.g., a small angular offset in the X / Y direction), thereby achieving precise displacement of the sub-frame image on the retina. After receiving the optical signal corresponding to the sub-frame image, the beam deflection element, under the action of a synchronization signal, offsets the optical signal of each sub-frame image according to the control signal. The offset optical signal is then transmitted to the optical waveguide, coupled through the waveguide, and output as a high-resolution image to the near-eye display screen for display.

[0110] It is known that higher resolution requires more power consumption, but some applications do not require very high resolution. Therefore, this invention can update the display frame rate and jitter rate according to the display mode, including: If the display mode is the high-definition display mode, determine whether the battery level of the virtual reality device is greater than or equal to the first battery threshold. If so, adjust the display frame rate to 120Hz~240Hz and adjust the jitter factor of the super-resolution to the first factor.

[0111] Specifically, users can choose the display mode according to the application scenario, and the main control chip 1 can receive the corresponding display mode. For example, if the application scenario is viewing drawings, refining images, reading documents, and processing text, the user can choose the high-definition display mode, which is a high frame rate and high resolution display mode. After the main control chip 1 obtains the high-definition display mode, it will determine whether the battery level of the virtual reality device is greater than or equal to a first battery threshold (e.g., 70%). If so, it will adjust the display frame rate of the optical engine 2 to 120Hz~240Hz and adjust the super-resolution jitter multiplier to the first multiplier; the first multiplier can be 4 times.

[0112] The beam deflection element 3 generates subpixel displacement according to a 4x super-resolution. Subframe images are offset according to their respective subpixel displacements. Assuming each subframe image is moved by half a physical pixel, the offset physical positions are (0, 0), (0.5, 0), (0, 0.5), and (0.5, 0.5). When these four subframe images are displayed sequentially in this physical area at an extremely high speed (e.g., 240Hz), the human eye cannot distinguish the rapidly switching individual light spots. The retina will fuse the information of these four light spots—the upper left, upper right, lower left, and lower right corners—which appear at different times but are extremely close in position.

[0113] The brain ultimately perceives this fused information as the simultaneous existence of four independent pixels within this physical area, located at the top left, top right, bottom left, and bottom right positions, respectively. Thus, an area that could originally only display a single, "average" piece of information becomes clearly distinguishable to the human eye as containing the details of four independent pixels from the original 4K resolution, thereby achieving an improvement in resolution.

[0114] Furthermore, assuming the application scenario is video playback, the human eye's sensitivity to dynamic image resolution decreases; in scenarios such as game playback and social media interfaces, the human eye's requirements for real-time performance are higher than resolution; and in outdoor strong light scenarios, brightness requirements take precedence over resolution; in these cases, to reduce power consumption, the user can choose a low-power display mode, which is a low frame rate low-power display mode. After the main control chip 1 obtains the low-power display mode, it will adjust the display frame rate of the optical engine 2 to 30Hz~60Hz and adjust the super-resolution jitter multiplier to the second factor.

[0115] Furthermore, if the device is in a strong light scene, after entering the low power mode, the device brightness needs to be adjusted to less than 50% of the total brightness to maintain the driving voltage of the beam deflection element 3 in a basic bias state.

[0116] In another implementation, updating the display frame rate and jitter ratio according to the display mode includes: If the display mode is adaptive display mode, determine whether the battery level of the virtual reality device is greater than or equal to the first battery threshold. If so, determine the type of display content corresponding to the eye gaze point. If the displayed content type is text, determine whether the ambient brightness is less than the preset brightness threshold. If so, update the display frame rate to 120Hz~240Hz and update the super-resolution jitter ratio to the first multiple. Specifically, in adaptive display mode, the main control chip 1 can adjust the display frame rate and jitter rate according to the type of displayed content, the user's eye movements, and the power consumption of the virtual reality device, finding the best balance between power consumption and resolution.

[0117] For example, eye-tracking algorithms can be used to determine the user's visual center area and assess the type of content displayed in that area. If the displayed content is text and the ambient brightness is lower than a preset brightness value, it indicates that the user requires a stronger visual experience. In this case, the display mode can be switched to high-definition display mode, which means updating the display frame rate to 120Hz~240Hz and updating the super-resolution jitter factor to the first multiple.

[0118] Conversely, if the user is watching a fast-moving video and their gaze is moving quickly, the jitter complexity and frame rate can be appropriately reduced to balance power consumption. So when the displayed content is a video and the speed of the screen movement is greater than the preset speed threshold, the display frame rate will be updated to 30Hz~60Hz and the jitter multiplier of the super-resolution will be updated to the second multiplier.

[0119] However, regardless of whether it's in high-definition display mode or adaptive display mode, the method also includes: The power consumption of the virtual reality device is monitored according to a preset monitoring cycle. If the power consumption of the virtual reality device is less than or equal to the second power threshold, the display mode will be switched to low power mode.

[0120] As can be seen, during the display process, once the battery level of the virtual reality device is determined to be below the second battery threshold (e.g., 30%), it automatically switches to low-power mode. This means updating the display frame rate to 30Hz~60Hz and the super-resolution jitter factor to the second multiple. The second multiple is less than the first multiple, and the second multiple can be 2 or 0. In other words, when the battery level of the virtual reality device is too low, the display frame rate and resolution will be reduced (by turning off the beam deflection element or reducing the jitter factor), automatically switching to low-power mode to improve the device's battery life.

[0121] Furthermore, to avoid screen flickering or stuttering during mode switching, which could affect image quality, this invention requires a smooth transition mechanism for mode switching. Taking switching from high-definition mode to low-power mode as an example, in one embodiment, after switching the display mode to low-power mode, the method further includes: Multiple intermediate resolutions are generated using a smooth transition mechanism; multiple high-definition resolutions are located between the original resolution and the resolution corresponding to the low-power display mode. The resolution is adjusted frame by frame until it reaches the resolution corresponding to the low-power display mode.

[0122] In other words, during the mode switching process, the resolution does not switch directly from the resolution corresponding to the high-definition display mode to the resolution corresponding to the low-power display mode. Instead, a smooth transition mechanism is used to generate multiple intermediate resolutions between the high-definition display mode resolution and the low-power display mode resolution. Then, the resolution is adjusted frame by frame until the resolution corresponding to the low-power display mode is reached, thus completing the switching process.

[0123] S413, Display the target image based on the display frame rate and the jitter ratio.

[0124] Once the display frame rate and jitter ratio are determined, the target image can be displayed based on these parameters.

[0125] In one implementation, displaying the target image based on the display frame rate and jitter ratio includes: Extract the corresponding jitter ratio from the jitter ratio, and split each target image into a corresponding number of sub-frame images based on the jitter ratio; The subframe image is transmitted to the optical engine according to the display frame rate, and the optical engine converts the subframe image into an optical signal. Under the action of the synchronization signal, the optical signal of each subframe image is offset according to the received control signal, the offset optical signal is transmitted to the optical waveguide for coupling, and the coupled optical signal is transmitted to the near-eye display screen for display.

[0126] Through one or more embodiments of the present invention, the present invention has the following beneficial effects or advantages: This invention provides a display method and system based on jitter super-resolution. The method includes: acquiring the display mode of a virtual reality device; determining the weight of an initial image based on the display mode and the type of display content; the display mode includes: a high-definition display mode, a low-power display mode, and an adaptive display mode; performing image processing on the initial image according to the weight of the initial image to obtain a target image; updating the display frame rate and jitter ratio according to the display mode; and displaying the target image based on the display frame rate and the jitter ratio. Thus, the display mode can be adjusted according to the application scenario requirements, and the display frame rate and jitter ratio can be updated in real time according to the display mode. This allows for appropriate reduction of resolution in scenarios where high resolution requirements are not necessary, thereby achieving a dynamic balance between resolution and power consumption, ensuring display quality while extending device battery life.

[0127] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0128] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A display method based on dithering super-resolution, characterized in that, The method includes: The display mode of the virtual reality device is obtained, and the weight of the initial image is determined based on the display mode and the type of display content; the display mode includes: high-definition display mode, low-power display mode and adaptive display mode; The initial image is processed according to its weights to obtain the target image; Update the display frame rate and jitter rate according to the display mode; The target image is displayed based on the display frame rate and the jitter ratio.

2. The method as described in claim 1, characterized in that, Determining the weight of the initial image based on the display mode and the type of display content includes: When the displayed content type is text, determine the region location information of the initial image; The corresponding weight range is determined based on the regional location information; The target weights corresponding to each region in the initial image are determined based on the display mode and the weight range.

3. The method as described in claim 2, characterized in that, Determining the target weights corresponding to each region in the initial image based on the display mode and the weight range includes: When the region is the visual center region, the maximum weight of the weight range corresponding to the visual center region is determined as the target weight of the visual center region. When the region is an edge region, the maximum weight of the weight range corresponding to the edge region is determined as the target weight of the edge region. When the region is an intermediate transition region, the maximum weight of the weight range corresponding to the intermediate transition region is determined as the target weight of the intermediate transition region.

4. The method as described in claim 1, characterized in that, The step of updating the display frame rate and jitter ratio according to the display mode includes: If the display mode is the high-definition display mode, determine whether the battery level of the virtual reality device is greater than or equal to the first battery threshold. If so, adjust the display frame rate to 120Hz~240Hz and adjust the jitter factor of the super-resolution to the first factor.

5. The method as described in claim 1, characterized in that, The step of updating the display frame rate and jitter ratio according to the display mode includes: If the display mode is the low-power mode, the display frame rate is adjusted to 30Hz~60Hz, and the dithering ratio of the super-resolution is adjusted to a second multiple; the second multiple is less than the first multiple.

6. The method as described in claim 1, characterized in that, The step of updating the display frame rate and jitter ratio according to the display mode includes: If the display mode is the adaptive display mode, determine whether the battery level of the virtual reality device is greater than or equal to the first battery threshold. If so, determine the display content type corresponding to the eye gaze point. If the displayed content type is text, determine whether the ambient brightness is less than a preset brightness threshold. If so, update the display frame rate to 120Hz~240Hz and update the jitter factor of the super-resolution to the first factor. When the displayed content type is video and the screen motion speed is greater than a preset speed threshold, the display frame rate is updated to 30Hz~60Hz and the super-resolution jitter factor is updated to the second factor.

7. The method as described in any one of claims 4 and 6, characterized in that, The method further includes: The power consumption of the virtual reality device is monitored according to a preset monitoring cycle. If the power consumption of the virtual reality device is less than or equal to a second power threshold, the display mode is switched to a low power mode.

8. The method as described in claim 7, characterized in that, After switching the display mode to low-power mode, the method further includes: Multiple intermediate resolutions are generated using a smooth transition mechanism; these multiple high-definition resolutions are located between the original resolution and the resolution corresponding to the low-power display mode. The resolution is adjusted frame by frame until it reaches the resolution corresponding to the low-power display mode.

9. The method as described in claim 1, characterized in that, The process of displaying the target image based on the display frame rate and the jitter ratio includes: Extract the corresponding jitter ratio from the jitter ratio, and split each target image into a corresponding number of sub-frame images based on the jitter ratio; The subframe image is transmitted to the optical engine according to the display frame rate, and the optical engine is used to convert the subframe image into an optical signal; Under the action of the synchronization signal, the optical signal of each subframe image is offset according to the received control signal, the offset optical signal is transmitted to the optical waveguide for coupling, and the coupled optical signal is transmitted to the near-eye display screen for display.

10. A display system based on dithering super-resolution, characterized in that, The system includes: The main control chip is used to acquire the display mode of the virtual reality device, and determine the weight of the initial image based on the display mode and the type of display content. The display modes include: high-definition display mode, low-power display mode, and adaptive display mode. The initial image is processed according to the weight of the initial image to obtain the target image. The display frame rate and jitter ratio are updated according to the display mode. Each target image is divided into a corresponding number of sub-frame images based on the jitter ratio, and each sub-frame image is transmitted to the optical engine according to the display frame rate. An optical engine is used to convert each of the subframe images into optical signals; The beam deflection element deflects the optical signal of each subframe image according to the received control signal, and transmits the deflected optical signal to the optical waveguide for coupling. Near-eye display screen, used to display coupled light signals.

Images