Display method, display apparatus, display device, and nonvolatile computer-readable storage medium

Abstract

A display method, a display apparatus, a display device, and a nonvolatile computer-readable storage medium. The display method comprises: performing sampling and rendering on a current scene to generate a display image; and displaying the display image.

Description

Display method, display device, display apparatus, and non-volatile computer-readable storage medium

[0001] Priority information

[0002] This application claims priority and benefits of the patent application with patent application number "202311683598.4" filed with the State Intellectual Property Office of China on December 8, 2023, and priority and benefits of the patent application with patent application number "202311689382.9" filed with the State Intellectual Property Office of China on December 8, 2023, and the entire text of which is incorporated herein by reference. Technical Field

[0003] The present application relates to the field of virtual reality technology, and more specifically, to a display method, a display device, a display equipment, and a non-volatile computer-readable storage medium. Background Art

[0004] The resolution of the human retina is generally around 60 pixels per degree (PPD). When users use virtual reality (VR) devices, the closer the angular resolution of the VR device is to 60PPD, the closer the image clarity is to the resolution limit of the human eye, and the clearer the image seen by the human eye. The higher the image output frame rate of the VR device, the better the image display effect. Summary of the Invention

[0005] Embodiments of the present application provide a display method, a display apparatus, a display device, and a non-volatile computer-readable storage medium.

[0006] The display method of the embodiment of the present application includes sampling and rendering the current scene to generate a display image; and displaying the display image.

[0007] In some embodiments, the sampling rendering includes rasterization, and the sampling positions of the display images of consecutive frames are different during rasterization; and the displaying of the display images includes:

[0008] The display image is displayed, and the polarization angle of the display light corresponding to the display image is adjusted so that the polarization angle matches the sampling position corresponding to the current scene, and different polarization angles correspond to different display positions of the display image.

[0009] In some embodiments, the sampling rendering also includes pixel shading, and the sampling rendering of the current scene to generate a display image includes: determining the sampling position of the current scene according to the frame number corresponding to the current scene; rasterizing the current scene, and performing pixel shading on the current scene according to the sampling position of the current scene to generate the display image.

[0010] In some embodiments, determining the sampling position of the current scene based on the frame number corresponding to the current scene includes: when the frame number is an even number, determining the sampling position of the current scene as a first sampling position; when the frame number is an odd number, determining the sampling position of the current scene as a second sampling position, the first sampling position and the second sampling position are different.

[0011] In some embodiments, the continuous multiple frames of the display image include a first display image and a second display image, the first display image corresponds to the first sampling position, the second display image corresponds to the second sampling position, and in the diagonal direction of the pixel, there is a preset pixel displacement between the first sampling position and the second sampling position.

[0012] In some embodiments, the sampling rendering also includes vertex processing, and the sampling rendering of the current scene to generate a display image also includes: performing vertex processing on the current scene to determine a plurality of rendering areas to be sampled; rasterizing the current scene and performing pixel shading on the current scene according to the sampling position of the current scene to generate the display image, including: determining the rendering area to be sampled that matches each pixel of the current pixel array, the current pixel array being determined according to a preset sampling density; performing pixel shading on the pixels matching the rendering area to be sampled according to the sampling position of the current scene to generate the display image.

[0013] In some embodiments, the present invention further includes: sending a current frame signal to a sampling rendering module and a display control module respectively to synchronize the sampling rendering module and the display control module, the sampling rendering module is configured to generate the display image, and the display control module is configured to control the polarization modulator to adjust the polarization angle of the display light by sending a control signal to the polarization modulator; when the sampling rendering module and the display control module are synchronized, the polarization angle matches the sampling position corresponding to the current scene.

[0014] In some embodiments, the total display time of the multiple consecutive frames of the displayed image is less than a preset human eye integration time.

[0015] In some embodiments, the current scene includes multiple areas to be sampled and rendered, each of the areas to be sampled and rendered has M matching pixels to be rendered, the display image is generated by sampling and rendering according to the position of a target pixel to be rendered among the M pixels to be rendered, the positions of the target pixel to be rendered corresponding to M consecutive frames of the display image are different, and M is a positive integer; displaying the display image includes:

[0016] The target display pixel of the display screen is controlled to operate to display the display image, the display screen includes multiple pixel areas, the pixel area includes M display pixels, and the target display pixel is a display pixel among the M display pixels corresponding to the position of the target pixel to be rendered.

[0017] In some embodiments, each of the to-be-sampled rendering areas has M matching pixels to be rendered in a pixel array corresponding to an original sampling density, and the original sampling density corresponds to the resolution of the display screen; the sampling rendering includes rasterization and pixel shading, and the sampling rendering of the current scene to generate a display image includes: determining the target to-be-rendered pixel corresponding to the current scene according to a frame number of the current scene; and sequentially performing rasterization and pixel shading on the positions of the target to-be-rendered pixel corresponding to the current scene to generate the display image, and the rasterization sampling density is 1 / M of the original sampling density.

[0018] In some embodiments, determining the target pixel to be rendered corresponding to the current scene based on the frame number of the current scene includes: dividing the frame number of the current scene by M to determine a remainder; and determining the target pixel to be rendered corresponding to the current scene based on the remainder.

[0019] In some embodiments, the sampling rendering further includes vertex processing, and the sampling rendering of the current scene to generate a display image further includes: performing vertex processing on the current scene to determine a plurality of the areas to be sampled and rendered.

[0020] In some embodiments, the method further includes: sending a current frame signal to a sampling rendering module and a display control module respectively to synchronize the sampling rendering module and the display control module, wherein the sampling rendering module is configured to generate the display image, and the display control module is configured to control the operation of the target display pixels of the display screen by sending a control signal to the display screen; after the sampling rendering module and the display control module are synchronized, the target pixels to be rendered corresponding to the display image correspond to the target display pixels operating on the display screen.

[0021] In some embodiments, the total display time of the M consecutive frames of the displayed image is less than a preset human eye integration time.

[0022] In some embodiments, M is equal to 4, the target pixel to be rendered includes 1 pixel to be rendered, and the 4 pixels to be rendered are arranged in a matrix.

[0023] The display device of the embodiment of the present application includes a generation module and a display module. The generation module is used to perform sampling and rendering on a current scene to generate a display image, wherein the sampling and rendering includes rasterization, and the sampling positions of multiple consecutive frames of the display image are different during rasterization; the display module is used to display the display image and adjust the polarization angle of the display light corresponding to the display image so that the polarization angle matches the sampling position corresponding to the current scene.

[0024] The display device according to the embodiment of the present application includes:

[0025] a generation module that samples and renders a current scene to generate a display image, wherein the current scene includes a plurality of sampling and rendering areas, each of which has M matching pixels to be rendered. The display image is generated by sampling and rendering based on the position of a target pixel to be rendered among the M pixels to be rendered, and the positions of the target pixel to be rendered corresponding to M consecutive frames of the display image are different, where M is a positive integer;

[0026] A display module is configured to control a target display pixel of a display screen to operate so as to display the display image, wherein the display screen includes a plurality of pixel areas, each pixel area includes M display pixels, and the target display pixel is a display pixel among the M display pixels corresponding to a position of the target pixel to be rendered.

[0027] The display device of an embodiment of the present application includes a processor, a memory; and a computer program, wherein the computer program is stored in the memory and executed by the processor, and the computer program includes instructions for executing the display method described in any of the above embodiments.

[0028] In some embodiments, the display device also includes a display configured to display the display image; a polarization modulator configured to adjust the polarization angle of the display light emitted by the display; a sampling and rendering module configured to generate the display image; and a display control module configured to control the polarization modulator to adjust the polarization angle of the display light by sending a control signal to the polarization modulator.

[0029] In some embodiments, the display device further includes an optical module, which includes the polarization modulator, the birefringent crystal and the Fresnel lens, and the polarization modulator, the birefringent crystal and the Fresnel lens are arranged in sequence along the exit direction of the display light; or, the optical module includes the polarization modulator, the birefringent crystal and the folded optical path component, and the polarization modulator, the birefringent crystal and the folded optical path component are arranged in sequence along the exit direction of the display light.

[0030] In some embodiments, the display device of the present application further includes a sampling rendering module and a display control module, wherein the sampling rendering module is configured to generate the display image, and the display control module is configured to control the operation of the target display pixel of the display screen by sending a control signal to the display screen.

[0031] In some embodiments, the display device of the present application further includes an optical module, wherein the optical module includes a phase delay plate and a Fresnel lens, or the optical module includes a phase delay plate and a folded optical path component.

[0032] The non-volatile computer-readable storage medium of an embodiment of the present application includes a computer program. When the computer program is executed by a processor, the processor executes the display method described in any of the above embodiments.

[0033] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and / or additional aspects and advantages of the embodiments of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0035] FIG1 is a schematic diagram of an application scenario of a display method according to certain embodiments of the present application;

[0036] FIG2 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0037] FIG3 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0038] FIG4 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0039] FIG5 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0040] FIG6 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0041] FIG7 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0042] FIG8 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0043] FIG9 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0044] FIG10 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0045] FIG11 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0046] FIG12 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0047] FIG13 is a schematic diagram of a module of a display device according to certain embodiments of the present application;

[0048] FIG14 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0049] FIG15 is a flow chart of a display method according to certain embodiments of the present application;

[0050] FIG16 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0051] FIG17 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0052] FIG18 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0053] FIG19 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0054] FIG20 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0055] FIG21 is a schematic flow chart of a display method according to certain embodiments of the present application;

[0056] FIG22 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0057] FIG23 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0058] FIG24 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0059] FIG25 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0060] FIG26 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0061] FIG27 is a schematic structural diagram of a display device according to some embodiments of the present application;

[0062] FIG28 is a schematic diagram of a scene of a display method according to certain embodiments of the present application;

[0063] FIG29 is a schematic diagram illustrating the connection status of a non-volatile computer-readable storage medium and a processor in certain embodiments of the present application. DETAILED DESCRIPTION

[0064] The embodiments of the present invention will be further described below with reference to the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions.

[0065] In addition, the embodiments of the present invention described below in conjunction with the accompanying drawings are exemplary and are only used to explain the embodiments of the present invention, and should not be understood as limiting the present invention.

[0066] 1. Virtual reality technology (VR), also known as virtual reality or spiritual realm technology. Virtual reality technology encompasses computer, electronic information, and simulation technology. Its basic implementation method is based on computer technology, utilizing and integrating the latest developments in various high technologies such as three-dimensional graphics technology, multimedia technology, simulation technology, display technology, and servo technology. With the help of the graphics processing unit (GPU) in the VR device, the image in the current scene is processed to produce a realistic three-dimensional visual, tactile, olfactory and other sensory experiences in the virtual world, thereby giving people in the virtual world an immersive feeling. The technical solution provided in the embodiment of the present application is mainly used to enhance the resolution of VR devices without affecting the frame rate.

[0067] 2. Vertex processing: The vertices of 3D graphics have coordinates in three-dimensional space. Through linear algebra calculations, the coordinate data of each vertex in three-dimensional space is converted and drawn into the two-dimensional space of the display. At the same time, the color of the vertex is calculated for subsequent pixel color interpolation. This operation is called vertex processing.

[0068] 3. Rasterization: The primitive information generated during vertex processing is input into the rasterization stage. Primitives are assembled first, and then the fragments corresponding to the screen pixels covered by the primitives are determined through triangle traversal. After the rasterization stage, the primitives will be divided into basic units of pixel size, which are called fragments. Fragments are more like data representations of pixels. The final pixel is generated by the information in the fragments, and then the fragments are shaded by the fragment shader. When operating fragment by fragment, the fragments undergo a series of tests. The fragments that pass the test will be converted into pixels and finally presented in the frame buffer area.

[0069] 4. A Field Programmable Gate Array (FPGA) is a programmable logic device, which is a semiconductor chip consisting of a series of programmable logic gates. In the technical solution provided in the embodiments of this application, it is mainly used to send gate circuit signals to control the signal generator to change the state of the polarization modulator.

[0070] 5. Frame buffer: A video output device drives a video display device from a memory buffer containing complete frame data. The technical solution provided in the embodiment of the present application is mainly used to generate a display image after determining the required resolution size as the output frame through Framebuffer resampling.

[0071] 6. Fresnel lenses are thin lenses made of injection-molded polyolefin material. They have equidistant serrations on one side, which reflect or refract light within a specific spectral range. Fresnel lenses are used as the primary optical module in VR devices due to their thinness.

[0072] 7. Birefringent crystal. When a beam of light wave is projected onto the crystal interface, two refracted light beams are generally generated. This phenomenon is called birefringence. Due to the anisotropy of the crystal material, the dispersion angle of the two refracted light beams is related to the direction of the optical axis and the refractive index of the ordinary and extraordinary light of the crystal. The crystal that produces birefringence is called a birefringent crystal. The birefringent crystal mainly used in the technical solution provided in the embodiment of the present application is quartz crystal.

[0073] 8. Frame rate refers to the number of frames per second of a video, expressed as frames per second (fps). In VR technology, it's generally believed that a higher frame rate means a smoother image and a better user experience.

[0074] The human retinal resolution is approximately 60 pixels per degree (PPD). It's generally believed that the closer a VR device's angular resolution is to 60 PPD, the closer the image clarity is to the human eye's resolution limit, resulting in a clearer image. When a VR device's resolution is insufficient, the human eye directly sees the display's pixels, creating a screen door effect. Existing VR devices typically have an angular resolution of around 20 PPD, so their resolution is still significantly insufficient.

[0075] Currently, there is a method of increasing resolution by splicing multiple graphics cards and multiple displays to project different sampled images and superimpose them. However, the system structure of setting up multiple graphics cards and multiple displays is complex, and it takes up a large physical space, which makes the VR equipment too heavy. In addition, the hardware cost is high, the system adjustability is poor, and it is limited by the transmission signal bandwidth and processing performance. The displayed image resolution is still limited.

[0076] In order to solve the above technical problems, an embodiment of the present application provides a display method.

[0077] The application scenario of the technical solution of the present application is first introduced below. As shown in FIG1 , it is a schematic diagram of an application scenario of a display method provided in an embodiment of the present application, and the application scenario involves a display device 100 .

[0078] FIG1 exemplarily shows a display device 100 , and may actually include other numbers of display devices 100 , which is not limited in the embodiments of the present application.

[0079] In one embodiment, the display device 100 includes a body 10 , a display 20 , a processor 30 , and a memory 40 . The display 20 is disposed inside the body 10 and can display images.

[0080] In one embodiment, the processor 30 includes a main processor 31 and a graphics processing unit (GPU) 32 , and the GPU processor 32 samples and renders the current scene to generate a scene image.

[0081] In one embodiment, the display 20 is used to display the scene image generated after sampling and rendering by the GPU processor 30.

[0082] In one embodiment, the display device 100 also includes a display control module 60, and the GPU processor 30 includes a sampling rendering module 50. A frame signal is sent through the main processor 31 to synchronize the sampling rendering module 50 and the display control module 60 so that the polarization angle of the displayed image and the light modulated by the polarization modulator can correspond.

[0083] In one embodiment, the display device 100 includes an optical module, and the display light of the display image of the display 20 enters the human eye after passing through the optical module. The optical module includes a polarization modulator, a birefringent crystal and a Fresnel lens. The polarization modulator, the birefringent crystal and the Fresnel lens are arranged in sequence along the exit direction of the display light.

[0084] In one embodiment, the optical module also includes a polarization modulator, a birefringent crystal and a folded optical path component. The polarization modulator, the birefringent crystal and the folded optical path component are arranged in sequence along the exit direction of the display light. The folded optical path component can be a Pancake optical component, that is, an optical component including two 1 / 4 phase delay plates (Quarter-Wave Plate, QWP), a semi-transparent and semi-reflective mirror (Beam Splitter, BS), and a reflective polarizer (Reflective Polarizer, RP).

[0085] The display method of this application will be described in detail below:

[0086] Referring to FIG. 1 and FIG. 2 , an embodiment of the present application provides a display method, which includes:

[0087] Step 011: Sample rendering is performed on the current scene to generate a display image. Sampling rendering includes rasterization. The sampling positions of consecutive frames of display images are different during rasterization.

[0088] The current scene is a frame among multiple frames of scenes included in the scene data stream, and the scene data stream is generated by a virtual three-dimensional (3D) scene or a two-dimensional scene built by an application.

[0089] Specifically, taking the current scene as a 3D scene as an example, the GPU performs a view transformation on the current scene, converting the three-dimensional coordinates of the 3D scene graph into two-dimensional coordinates and transforming one or more vertices into completely different basic shapes (or primitives), thereby generating many more vertices than the original. Primitives are then assembled through rasterization, and then triangle traversal is performed to determine the fragments corresponding to the screen pixels covered by the primitives. Finally, the fragments are shaded and subjected to a series of tests based on different sampling positions to generate the display image of the current scene.

[0090] For example, please refer to Figure 3. The current scene is transformed in the GPU, the three-dimensional coordinates of the 3D scene graph are converted into two-dimensional coordinates, and multiple areas to be rendered in the current scene are determined. The pixels matching the area to be rendered (such as the triangular area in the figure) are determined to be 2*2 through rasterization. In the first frame, the area to be rendered is sampled and rendered to obtain the corresponding display image of the first frame. In the second frame, the diagonal position of the first frame is offset to the lower right corner by half a pixel for sampling and rendering to obtain the corresponding display image of the second frame. The Framebuffer is resampled to determine 2*2 pixels as the output frame to generate the display image of the current scene.

[0091] Optionally, the total display time of the continuous multiple frames of displayed images is less than a preset human eye integration time.

[0092] Specifically, it takes a certain integration time for an image to form on the retina. During this integration time, the inter-frame images can be superimposed on the retina. At this time, due to the persistence of vision of the human eye, the human eye obtains the image superimposed within the integration time. When the current scene is continuously sampled and rendered to generate multiple frames of display images, if the total display time of the continuous multi-frame display images is less than or equal to the preset human eye integration time, the resolution perceived by the human eye will exceed the original resolution of the screen, thereby achieving the effect of improving resolution.

[0093] For example, the human eye's daytime integration time is generally 50 milliseconds (ms). If the display's frame rate is 90 Hz, the interframe spacing is 1 second (s) / 90 Hz = 0.01111s = 11.11ms. When four frames are superimposed, the superposition time is 4 * 11.11ms = 44.44ms, which is less than the human eye's daytime integration time of 50ms. Interframe imaging can be superimposed on the retina during this integration time, bringing the perceived resolution of the human eye close to the screen's native resolution, thereby achieving a higher resolution.

[0094] Step 012: Display the display image and adjust the polarization angle of the display light corresponding to the display image so that the polarization angle matches the sampling position corresponding to the current scene. Different polarization angles correspond to different display positions of the display image.

[0095] Specifically, the generated display image is displayed by the display, and the polarization angle of the display light emitted corresponding to each frame of the display image displayed by the display is adjusted. The display light of different display images generated corresponding to different sampling positions is converted into different polarization states, and the polarization state corresponds to the sampling position corresponding to the current scene.

[0096] In this way, by acquiring the current scene from the scene data stream and performing sampling and rendering using the corresponding sampling positions, a display image corresponding to the current scene is generated. When rasterizing multiple consecutive frames of the current scene, different sampling positions are set for each frame, thereby obtaining multiple consecutive frames of display images corresponding to the multiple consecutive frames of the current scene.

[0097] Compared with the existing single-frame self-superposition method, the same scene is continuously sampled and rendered multiple times, and multiple display images of the same scene are superimposed to achieve resolution improvement, resulting in the continuous multi-frame display images containing fewer scenes, and the frame rate of the continuous multi-frame display images is lower than the frame rate of the scene data stream. The scenes corresponding to the various display images of this application are different, and the continuous multi-frame display images contain a larger number of scenes, which is basically the same as the frame rate generated by the scene data stream, thereby ensuring that the frame rate remains unchanged.

[0098] On the premise of ensuring that the frame rate remains unchanged, the display images of multiple consecutive frames corresponding to different sampling positions are displayed using the polarization angles corresponding to the sampling positions, so that the display positions of the multiple consecutive frames of display images are different. Based on the visual persistence characteristics of the human eye, the multiple consecutive frames of display images with different display positions are superimposed on the human eye to achieve resolution improvement.

[0099] Referring to FIG. 4 , in some embodiments, the display method further includes:

[0100] Step 013: Sending the current frame signal to the sampling and rendering module and the display control module respectively to synchronize the sampling and rendering module and the display control module. The sampling and rendering module is configured to generate a display image. The display control module is configured to send a control signal to the polarization modulator to control the polarization modulator to adjust the polarization angle of the display light.

[0101] Step 014: When the sampling rendering module and the display control module are synchronized, the polarization angle matches the sampling position corresponding to the current scene.

[0102] Among them, a polarization modulator is an optical modulator that changes the polarization state of light according to a certain rule.

[0103] Specifically, by sending the current frame signal to the sampling and rendering module and the display control module respectively, the sampling and rendering module and the display control module are synchronized so that the display light of the display image and the state of the polarization modulator can correspond. After receiving the current frame signal, the sampling and rendering module performs sampling and rendering on the current scene, such as through sampling and rendering processes such as vertex processing, rasterization, pixel shading and Framebuffer resampling to generate a display image. After receiving the current frame signal, the display control module sends a control signal to the polarization modulator to control the polarization modulator to adjust the polarization angle of the display light. For example, after receiving the current frame signal, the display control module sends a corresponding gate circuit signal to control the signal generator through the FPGA. The signal generator sends a control voltage according to the received signal to control the change of the state of the polarization modulator to achieve control of the polarization modulator to adjust the display light of the display image.

[0104] Referring to FIG. 5 , in some embodiments, sampling rendering further includes pixel shading. Step 011 : performing sampling rendering on the current scene to generate a display image includes:

[0105] Step 0111: Determine the sampling position of the current scene according to the frame number corresponding to the current scene;

[0106] Step 0112: Rasterize the current scene and perform pixel shading on the current scene according to the sampling position of the current scene to generate a display image.

[0107] Specifically, the frame number is used to identify the order of each frame. The sampling position of the current scene is determined according to the frame number corresponding to the current scene. Different sampling positions are set for the current scene through the frame numbers of multiple consecutive frames. The current scene is then rasterized, and pixel shading is performed according to different sampling positions to generate display images under different scenes.

[0108] Referring to FIG. 6 , in some embodiments, step 0111 : determining the sampling position of the current scene according to the frame number corresponding to the current scene, includes:

[0109] Step 01111: when the frame number is an even number, determine the sampling position of the current scene as the first sampling position;

[0110] Step 01112: When the frame number is an odd number, the sampling position of the current scene is determined to be the second sampling position, and the first sampling position and the second sampling position are different.

[0111] Specifically, the first sampling position and the second sampling position are different. The sampling position of the current scene is determined by whether the frame number corresponding to the current scene is odd or even. When the frame number is even, the sampling position of the current scene is determined to be the first sampling position; when the frame number is odd, the sampling position of the current scene is determined to be the second sampling position.

[0112] For example, referring to FIG7 , the current scene is distinguished into an even frame and an odd frame by the frame number corresponding to the current scene, so as to determine whether the sampling position of the current scene is the sampling position of the first display image or the sampling position of the second display image, and the even frames are rasterized and colored according to the sampling position; the odd frames are rasterized and colored according to the sampling position different from the even frames.

[0113] Optionally, the continuous multi-frame display image includes a first display image and a second display image, the first display image corresponds to a first sampling position, the second display image corresponds to a second sampling position, and in the diagonal direction of the pixel, there is a preset pixel displacement between the first sampling position and the second sampling position.

[0114] The preset pixel displacement may be 0.25 pixels, 0.3 pixels, 0.5 pixels, etc.

[0115] Specifically, the continuous multi-frame display image includes a first display image and a second display image, and the display image of each frame is the first display image or the second display image. The sampling positions of the first display image and the second display image are different, the first display image corresponds to the first sampling position, and the second display image corresponds to the second sampling position. There is a displacement of a preset number of pixels in the diagonal direction of the pixel between the sampling position corresponding to the first display image and the sampling position corresponding to the second display image. When the total display time of the first display image and the second display image of the continuous multi-frame is less than the preset human eye integration time, the resolution improvement effect can be achieved through the visual persistence characteristics of the human eye.

[0116] For example, please refer to Figure 8, taking the sampling position corresponding to the first display image as the original sampling position and the preset pixel as 0.5 pixels as an example for explanation, then the sampling position corresponding to the second display image is moved by a length D toward the lower right corner of the pixel diagonal relative to the sampling position corresponding to the first display image, where D is the diagonal length of half a pixel. When the first display image and the second display image are superimposed in the human eye after sampling and rendering, a superimposed effect as shown in the figure will be displayed.

[0117] That is to say, still taking the above example, when sampling and rendering the first display image and the second display image at different positions, assuming that counting starts from frame 0, the sampling position of frame 0 is set as the original sampling position, and the first display image is obtained after sampling and rendering the original sampling position, then the sampling position of the first frame will be determined to be the sampling position of the second display image at a position displaced by half a pixel from the original sampling position along the pixel diagonal direction relative to the original sampling position, and the second frame will be determined to be the sampling position corresponding to the first display image of the second frame, still at the first sampling position. Then, when the frame number is even, the sampling position of the current scene is determined to be the first sampling position; when the frame number is odd, the sampling position of the current scene is determined to be the second sampling position. After multiple consecutive frames, the first display image or the second display image with different sampling positions is obtained, and there is a half-pixel displacement in the diagonal direction of the pixels of the sampling positions of each consecutive first display image and second display image.

[0118] Continuing with Figure 7, using a preset pixel count of 0.5 pixels as an example, it can be seen that, assuming the count starts from the even frame, the sampling positions of consecutive odd and even frames are offset by 0.5 pixels in the diagonal direction. Due to the offset difference between consecutive odd and even frames, different rendering methods are required for odd and even frames. After rasterization and shading are performed on the even frames, the rasterization sampling positions are changed in the odd frames (shifted by 0.5 pixels diagonally). During the shading phase, shading is performed based on the changed sampling positions. Finally, framebuffer resampling is used to determine the corresponding resolution as the output frame. For example, assuming an LCD display has an n*n resolution, based on the acquired frame signals, rasterization and shading are performed on the even frames to obtain a first display image. Shading is performed on the odd frames at the changed rasterization sampling positions to obtain a second display image. Framebuffer resampling is then used to determine the output image with an n*n resolution to be displayed on the display. Compared with the current sampling rendering method, more details can be sampled.

[0119] Multiple consecutive frames are divided into even frames and odd frames to perform sampling and rendering of the first display image and the second display image. After the sampling and rendering is completed, when the generated display image is displayed on the display, the polarization angle of the display light corresponding to each frame of the display image displayed on the display is also adjusted based on the optical module. The optical module can include a polarization modulator, a birefringent crystal, and a Fresnel lens arranged in sequence. The birefringent crystal can be quartz crystal, calcite, etc. In the embodiment of the present application, the optical path principle is explained using a quartz crystal as the birefringent crystal.

[0120] The S-polarized light emitted by the LCD enters the polarization modulator, where it is modulated into either 0° or 90° linear polarization. 0° linear polarization propagates through the quartz crystal as o-ray, while 90° linear polarization propagates through the quartz crystal as e-ray. Therefore, the projected images of o-ray and e-ray are offset by half a pixel after passing through the quartz crystal. Based on the aforementioned optical path principle, even and odd frames can be distinguished based on the frame sequence number and transmitted to the FPGA. The FPGA controls the signal generator to issue the corresponding control voltage, changing the modulation state of the polarization modulator. In even frames, the polarization modulator modulates the light into 0° linear polarization, allowing it to exit along its original path. In odd frames, the polarization modulator modulates the light into 90° linear polarization, offsetting it by half a pixel. After passing through the quartz crystal and the Fresnel lens, the emitted light forms an image in the human eye.

[0121] Then, the thickness d of the quartz crystal should be designed as:

[0122] Where L is the width of a pixel on the display, and Φ is the dispersion angle of the e-light. The solution is:

[0123] Where θ is the angle between the incident light and the optical axis, n o is the refractive index of o light, n e is the refractive index of e-light.

[0124] For example, please refer to FIG9 . When two frames are superimposed, even frames and odd frames can be distinguished according to the frame number. Assume that counting starts from the 0th frame (even frame), the 0th frame (even frame) and the 1st frame (odd frame) are sampled and rendered, and two frames are superimposed and imaged in the human eye. The sampling position of the 0th frame is set to the original sampling position. After sampling and rendering the original sampling position, the first display image is obtained. The FPGA is controlled to send a corresponding gate circuit signal to control the signal generator. The signal generator sends a control voltage according to the received signal to control the change of the state of the polarization modulator, so that the first display image of the 0th frame enters the polarization modulator through the display light (S linear polarized light) emitted by the display and becomes 0° linear polarized light. After the 0° linear polarized light passes through the quartz crystal, it propagates from the original light emission path and passes through the Fresnel lens in the manner of o light to enter the human eye; determine the sampling position of the first display image along the 0th frame. The sampling position of the displacement position 0.5 pixels away from the diagonal direction is the sampling position of the first frame. After the sampling rendering is completed, the second display image is generated, and the FPGA is controlled to send a corresponding gate circuit signal to control the signal generator. The signal generator sends a control voltage according to the received signal to control the change of the state of the polarization modulator, so that the second display image of the first frame is generated by the display light (S linear polarized light) emitted by the display entering the polarization modulator and becoming 90° linear polarized light. After passing through the quartz crystal, the 90° linear polarized light is emitted from the position offset by half a pixel, propagates through the Fresnel lens in the form of e-light and enters the human eye.

[0125] When performing three-frame superposition, it is assumed that counting starts from frame 0 (even frame), that is, frame 0 (even frame), frame 1 (odd frame) and frame 2 (even frame) are sampled and rendered, and two consecutive frames are superimposed and imaged in the human eye. The adjustment of the display light of the display images of frame 0 and frame 1 after passing through the optical module is as described above, and no further details are given here. When sampling the second frame, the sampling position of the first display image along frame 0 is determined to be the sampling position of the second frame. After the sampling and rendering are completed, the first display image is generated. The display light of the first display image of the second frame will be adjusted to 0° linear polarized light by the polarization modulator. After passing through the quartz crystal, it will propagate from the original light emission path that is 0.5 pixels offset from the first frame, through the Fresnel lens and into the human eye in the manner of o light. Finally, the three frames of display images are superimposed in the human eye.

[0126] From the above, we can see that the o-light is emitted along the original path, and the e-light is emitted with a half-pixel offset relative to the o-light. The dispersion angle Φ of the e-light is:

[0127] Φ and the angle θ between the incident light and the optical axis, o The refractive index of light n o and the refractive index n of e-light e Related.

[0128] It is understandable that the rotation angle of the birefringent crystal around the optical axis can be adjusted by adding a mechanical device to offset the outgoing light in different directions, and combined with the corresponding sampling rendering position to achieve higher resolution enhancement.

[0129] It should be noted that quartz crystal has birefringence. When light passes through it, it undergoes refraction and changes in optical path difference, which in turn affect the light propagation path and imaging quality. This change in optical path difference can cause slight shifts in the light propagation path when raster sampling is performed at different locations, affecting the imaging at the sampling location after the pixel shift. Therefore, to achieve accurate imaging, the preset pixel displacement between the first and second sampling locations can be determined based on the thickness of the quartz crystal in actual applications.

[0130] In addition, the optical module can also be a polarization modulator, a birefringent crystal and a folded optical path component, wherein the folded optical path component can be a pancake component, and the polarization modulator, birefringent crystal and pancake component are arranged in sequence along the outgoing direction of the display light. The principles are similar and will not be elaborated here.

[0131] Please refer to Figure 10. Compared with the sampling rendering method described above, in which sampling and shading are not distinguished between odd and even frames, but sampling and rendering are performed twice at different positions on the same scene in one frame (i.e., a single-frame self-superposition method) to improve the resolution of the entire scene in each frame, the sampling rendering provided by the embodiment of the present application performs rasterization and shading on different scenes at different positions in odd and even frames (i.e., an inter-frame superposition method). At the same frame rate, the number of scenes displayed by this solution is greater than the number of scenes displayed by the single-frame self-superposition method, which can achieve sampling of more details and improve the resolution without reducing the frame rate.

[0132] This is because, in static scenes, the inter-frame superposition method of the present application has the same resolution improvement effect as the single-frame superposition method, and both methods will not lose frame rate; in dynamic scenes, when the objects in the current scene change, and the changes occur within the integration time of the human eye, the human eye cannot distinguish the actual objects displayed in each frame, and it is difficult to track the changes of the object in real time. The role of resolution is no longer reflected. At this time, the effect of improving the resolution of each frame by single-frame self-superposition cannot be reflected.

[0133] Since the human eye has an area of ​​focus, namely the area where the human eye tracks, when the human eye tracks a moving object in real time, the relative position between the human eye and the moving object does not change, that is, the human eye and the moving object are relatively still. Then the visual effect of the moving object tracked in real time in the human eye is very close to the superposition effect when the motion is compensated back to the previous frame position. Therefore, for the moving object in the area tracked by the human eye, there will be a resolution improvement effect similar to the single-frame self-superposition method; while in the area not tracked by the human eye, the resolution will be greater than or equal to the original resolution, which is in line with the usage requirements of the display of the VR device.

[0134] Therefore, it can be understood that when users use VR devices, the resolution requirements for the area tracked by the human eye are higher, and the resolution requirements for the area not tracked by the human eye are not high.

[0135] For example, please refer to Figure 11. When the human eye maintains the real-time tracking state in the figure and the GPU renders the moving scene and the static scene respectively, the human eye always looks at the shadow block in the scene. At this time, the shadow block in the moving scene is tracked in real time by the human eye, and the resolution is also enhanced. When calculating the resolution, since the relative position d1 of the human eye and the moving scene has not changed, that is, the human eye and the shadow object in the moving scene are relatively still, the motion compensation d can be automatically achieved by the human eye. Therefore, the resolution improvement effect can be achieved through the visual persistence of the human eye, and the resolution of the moving object tracked by the human eye in real time in the tracking area of ​​the human eye can be improved; while in the static scene, there is no motion compensation for the area tracked by the human eye, and the resolution can be improved.

[0136] In summary, the display method of the present application improves the resolution of tracked objects within the dynamic range of the human eye to a certain extent, maintains the resolution of the remaining areas, and keeps the frame rate stable. Compared with the current single-frame self-superposition method, when displaying multiple scenes continuously, the ghosting is reduced and the dynamic effect is improved.

[0137] Referring to FIG. 12 , in some embodiments, sampling rendering further includes vertex processing. In step 011 , sampling rendering is performed on the current scene to generate a display image, and further includes:

[0138] Step 0113: Perform vertex processing on the current scene to determine multiple rendering areas to be sampled;

[0139] Step 0112: rasterizing the current scene and performing pixel shading on the current scene according to the sampling position of the current scene to generate a display image, further comprising:

[0140] Step 01121: Determine the to-be-sampled rendering area that matches each pixel of the current pixel array, where the current pixel array is determined according to a preset sampling density;

[0141] Step 01122: Based on the sampling position of the current scene, perform pixel shading on the pixels that match the sampled rendering area to generate a display image.

[0142] Specifically, when converting a three-dimensional image to a two-dimensional screen, it is necessary to perform coordinate transformation operations on the vertices of the polygon. By using linear algebra calculations, the coordinate data of each vertex in the three-dimensional coordinates is converted and mapped to the two-dimensional space, thereby determining multiple sampling and rendering areas (primitives) to be sampled in the current scene. Then, the pixel array is determined by the preset sampling density (such as determining the preset sampling density according to the resolution of the display), and the fragments corresponding to the screen pixel points covered by the primitives are generated. Finally, through the different sampling positions of the current scene, the matching fragments in the sampling and rendering area are colored, and after a series of tests, the display image pixels are generated.

[0143] Referring to FIG. 13 , to facilitate better implementation of the display method of the embodiment of the present application, the embodiment of the present application further provides a display device 10, comprising a generation module 11 and a display module 11. The generation module 11 is configured to perform sampling and rendering on the current scene to generate a display image. The sampling and rendering includes rasterization, and the sampling positions of consecutive frames of display images are different during rasterization. The display module 11 is configured to display the display image and adjust the polarization angle of the display light corresponding to the display image so that the polarization angle matches the sampling position corresponding to the current scene.

[0144] In some embodiments, sampling rendering also includes pixel shading, and the generation module 11 is specifically used to determine the sampling position of the current scene according to the frame number corresponding to the current scene; rasterize the current scene, and pixel shade the current scene according to the sampling position of the current scene to generate a display image.

[0145] In some embodiments, the generation module 11 is further configured to determine the sampling position of the current scene as the first sampling position when the frame number is an even number; and to determine the sampling position of the current scene as the second sampling position when the frame number is an odd number, wherein the first sampling position and the second sampling position are different.

[0146] In some embodiments, sampling rendering also includes vertex processing, and the generation module 11 is specifically used to perform vertex processing on the current scene to determine multiple rendering areas to be sampled; determine the rendering areas to be sampled that match each pixel of the current pixel array, and the current pixel array is determined according to a preset sampling density; according to the sampling position of the current scene, pixel coloring is performed on the pixels matching the rendering areas to be sampled to generate a display image.

[0147] In some embodiments, the display device also includes a synchronization module 13, which is used to send the current frame signal to the sampling rendering module and the display control module respectively to synchronize the sampling rendering module and the display control module. The sampling rendering module is configured to generate a display image, and the display control module is configured to send a control signal to the polarization modulator to control the polarization modulator to adjust the polarization angle of the display light. When the sampling rendering module and the display control module are synchronized, the polarization angle matches the sampling position corresponding to the current scene.

[0148] The display device 10 is described above from the perspective of a functional module in conjunction with the accompanying drawings. The functional module can be implemented in the form of hardware, can be implemented by instructions in the form of software, or can be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly reflected as being executed by a hardware encoding processor, or can be executed by a combination of hardware and software modules in the encoding processor. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in combination with its hardware.

[0149] Please refer to Figure 1. The display device 100 of an embodiment of the present application includes a body 10, a processor 30, a memory 40 and a computer program, wherein the computer program is stored in the memory 40 and executed by the processor 30, and the computer program includes instructions for executing the display method of any of the above embodiments.

[0150] Optionally, the processor 30 includes a main processor 31 and a graphics processing unit (GPU) 32 , and the GPU processor 32 samples and renders the current scene to generate a scene image.

[0151] Optionally, the display device 100 also includes a display 20, a polarization modulator, a sampling and rendering module 50 and a display control module 60, the display 20 is configured to display a display image, the polarization modulator is configured to adjust the polarization angle of the display light emitted by the display, the sampling and rendering module 50 is configured to generate a display image, and the display control module 60 is configured to send a control signal to the polarization modulator to control the polarization modulator to adjust the polarization angle of the display light, and send a frame signal through the main processor 31 to synchronize the sampling and rendering module 50 and the display control module 60, so that the display image and the polarization angle of the light modulated by the polarization modulator can correspond.

[0152] Optionally, the display device 100 further includes an optical module, which includes a polarization modulator, a birefringent crystal and a Fresnel lens, and the polarization modulator, the birefringent crystal and the Fresnel lens are arranged in sequence along the exit direction of the display light; or, the optical module includes a polarization modulator, a birefringent crystal and a folded optical path component, and the polarization modulator, the birefringent crystal and the folded optical path component are arranged in sequence along the exit direction of the display light.

[0153] Please refer to Figures 1 and 14. When the current scene is sampled and rendered by the GPU image processor 32 to generate a scene image and the display 20 is controlled by the FPGA, the main processor synchronously sends a frame signal to the GPU image processor 32 and the FPGA. In the sampling and rendering module 50, the GPU image processor 32 performs vertex processing and rasterization according to the application scene, and performs pixel shading processing and Framebuffer resampling processing according to the frame signal to generate a display image. At the same time, after receiving the frame signal, the FPGA sends a gate circuit signal to control the signal generator, and the signal generator sends a gate circuit signal according to the received signal. The control voltage changes the state of the polarization modulator, and the GPU image processor 32 sends the generated display image to the display 20 for display. The display light emitted by the display 20 is modulated into 0° or 90° linear polarized light through the polarization modulator. The 0° or 90° linear polarized light passing through the polarization modulator is emitted as o light or e light through the birefringent crystal. The o light or e light emitted through the birefringent crystal is superimposed on the human eye after passing through the optical module. The GPU's pixel shading processing and Framebuffer resampling processing are continuously switched according to the frame signal, and the state of the polarization modulator is synchronously selected to improve the resolution.

[0154] At present, the frame rate of VR devices is mainly affected by the performance of the display and the speed of sampling and rendering the current scene. The performance of the display is mainly limited by the processing technology. Traditional displays use the interlaced display method to display odd or even rows of pixels each time, reducing the bandwidth of each transmission to increase the display frame rate. The current sampling rendering is generally rasterized according to the sampling density of 2n*2n, and after all the rasterized pixels are colored, they are directly output to the display for display. The existing pixel shifting and superposition method based on time division multiplexing used in VR devices achieves resolution improvement by continuously sampling and rendering the same scene multiple times, and superimposing multiple display images of the same scene through the visual persistence characteristics of the human eye.

[0155] Although the resolution increase method based on time division multiplexing can break through the resolution limit of the display, it will reduce the frame rate and the dynamic performance of the VR device when it is working, thereby affecting the user's viewing experience.

[0156] To solve the above technical problems, the present application provides a display method, which will be described in detail below:

[0157] Referring to FIG. 15 , an embodiment of the present application provides a display method, which includes:

[0158] Step 021: Sampling and rendering the current scene to generate a display image, wherein the current scene includes a plurality of sampling and rendering areas, each of which has M matching pixels to be rendered. The display image is generated by sampling and rendering based on the position of a target pixel to be rendered among the M pixels to be rendered. The positions of the target pixel to be rendered corresponding to M consecutive frames of the display image are different, where M is a positive integer.

[0159] The current scene is a frame among multiple frames of scenes included in the scene data stream, and the scene data stream is generated by a virtual three-dimensional (3D) scene or a two-dimensional scene built by an application.

[0160] Specifically, taking the current scene as a 3D scene as an example, the GPU of the VR device is used to perform view transformation on the current scene, converting the three-dimensional coordinates of the 3D scene graph into two-dimensional coordinates, determining multiple areas (primitives) to be sampled and rendered in the current scene, and there are M matching pixels to be rendered in the areas to be sampled and rendered. Then, the target pixel to be rendered among the M pixels to be rendered in the areas to be sampled and rendered (that is, the position of the target pixel to be rendered is used as the sampling position) is sampled and shaded, and a series of test sampling renderings are performed to finally generate a display image.

[0161] For example, vertex processing can be used to perform view transformation on the current scene in the scene data stream, determine multiple rendering areas to be sampled in the current scene, determine a preset sampling density based on the resolution of the display, determine a pixel array based on the preset sampling density, match the pixel array with the rendering area to be sampled through rasterization, and finally, shade the pixels in the rendering area to be sampled based on the information of the rendering area to be sampled (such as texture, color, etc.) and the preset sampling position to obtain corresponding pixel values, thereby generating a display image.

[0162] It is determined that the sampling and rendering of the current area to be sampled and rendered is completed through M consecutive frames. The position of the target pixel to be rendered corresponding to each display image of the consecutive M frames is different from each other. Therefore, after completing the continuous sampling of M frames, the M pixels to be rendered will all be sampled and rendered, thereby realizing the sampling and rendering of all pixels to be rendered in the current area to be sampled and rendered.

[0163] It is understood that M is a positive integer. When one pixel to be rendered is sampled in each frame, that is, the target pixel to be rendered is one pixel to be rendered, there are M target pixels to be rendered, and M sampling times are required to achieve sampling and rendering of the current sampling rendering area.

[0164] For example, referring to scene Figure 16, the current scene in the scene data stream is obtained. In the to-be-sampled rendering area of ​​the current scene, there is a 2*2 pixel array, which includes four 1*1 to-be-rendered pixels, that is, M=4. The to-be-rendered area needs to be sampled for four consecutive frames, and one 1*1 target to-be-rendered pixel is sampled and rendered in each frame. The four consecutive frames include a first frame, a second frame, a third frame, and a fourth frame. The target to-be-rendered pixel of the first frame is the to-be-rendered pixel in the upper left corner of the 2*2 matrix formed by the four to-be-rendered pixels; the target to-be-rendered pixel of the second frame is the to-be-rendered pixel in the upper right corner of the 2*2 matrix formed by the four to-be-rendered pixels; the target to-be-rendered pixel of the third frame is the to-be-rendered pixel in the lower left corner of the 2*2 matrix formed by the four to-be-rendered pixels; and the target to-be-rendered pixel of the fourth frame is the to-be-rendered pixel in the lower right corner of the 2*2 matrix formed by the four to-be-rendered pixels. The positions of the target pixels to be rendered in four consecutive frames of sampling and rendering are different. After completing the sampling and rendering of the first frame, the second frame, the third frame and the fourth frame, the four pixels to be rendered in the sampling and rendering area will all be sampled and rendered, and a display image will be generated.

[0165] Optionally, the total display time of the M consecutive frames of displayed images is less than a preset human eye integration time.

[0166] Specifically, it takes a certain integration time for an image to form on the retina. During this integration time, the inter-frame images are superimposed on the retina. Due to the human eye's persistence of vision, the human eye perceives the superimposed images within the integration time. When the total display duration of M consecutive frames is less than the preset human eye integration time, the perceived resolution exceeds the screen's native resolution, achieving a higher resolution.

[0167] For example, the human eye's daytime integration time is generally 50 milliseconds (ms). If the VR device's display has a frame rate of 90 Hz, the inter-frame spacing is 1 second (s) / 90 Hz = 0.01111s = 11.11ms. The time it takes to display four consecutive frames is 11.11ms*4 = 44.44ms, which is less than the human eye's daytime integration time of 50ms. Therefore, inter-frame imaging can be superimposed on the retina, making the resolution perceived by the human eye close to the screen's native resolution, thereby achieving the effect of improving resolution.

[0168] Step 022: Control the target display pixel of the display screen to operate to display the display image. The display screen includes multiple pixel areas, each pixel area includes M display pixels, and the target display pixel is a display pixel among the M display pixels corresponding to the position of the target pixel to be rendered.

[0169] Specifically, the display includes multiple pixel areas, each of which has M display pixels corresponding to the pixels to be rendered. By making a one-to-one correspondence between the position of the target display pixel of the display and the position of the target pixel to be rendered in the sampling and rendering area, after the target pixel to be rendered that has completed sampling and rendering generates a display image, the target display pixel on the display can be controlled to be displayed accordingly, so as to control other pixels that have not been sampled and rendered to not be displayed, so that the display image generated after sampling and rendering the target pixel to be rendered in each frame can correspond to the pixel controlled by the display each time.

[0170] In this way, during the sampling and rendering processing of M consecutive frames, the VR device generates a display image by sampling and rendering according to the position of the target pixel to be rendered among the M pixels to be rendered, and only samples and renders the target pixel to be rendered in the sampling rendering area, and does not process the non-target pixel to be rendered. This can reduce the sampling and rendering time of the sampling rendering area for each frame, thereby improving the sampling and rendering speed of the current scene, improving the efficiency of graphics processing, and achieving an increase in frame rate. Moreover, by making a one-to-one correspondence between the position of the target display pixel of the display screen and the position of the target pixel to be rendered in the sampling rendering area, the target display pixel of the control display screen each time works is the target pixel to be rendered corresponding to each frame, so that the display positions of the consecutive M frames of display images displayed on the display screen are different, and the resolution of the consecutive M frames of display images is superimposed in the human eye, which not only improves the frame rate but also ensures that the resolution is not lost, thereby improving the user experience when using the VR device.

[0171] Referring to FIG. 17 , in some embodiments, each area to be sampled and rendered has M matching pixels to be rendered in a pixel array corresponding to an original sampling density, where the original sampling density corresponds to the resolution of the display screen. Sampling and rendering include rasterization and pixel shading. Step 021: performing sampling and rendering on the current scene to generate a display image includes:

[0172] Step 0211: Determine the target pixel to be rendered corresponding to the current scene according to the frame number of the current scene;

[0173] Step 0212: Rasterize and pixel shade the positions of the target pixels to be rendered corresponding to the current scene in sequence to generate a display image. The rasterization sampling density is 1 / M of the original sampling density.

[0174] The rasterized sampling density is 1 / M of the original sampling density.

[0175] Specifically, sampling rendering includes rasterization and pixel shading. Each area to be sampled and rendered has M matching pixels to be rendered in the pixel array corresponding to the original sampling density, where the original sampling density corresponds to the resolution of the display screen. The target pixels to be rendered corresponding to the current scene are determined by the frame number of the current scene, such as by setting a mapping relationship between the parity of the current scene frame number and the target pixels to be rendered, or by using a mapping relationship between the remainder of dividing the current scene frame number by M and the target pixels to be rendered.

[0176] Since the original pixel information needs to be sampled, the original sampling density does not change. Each rendering area to be sampled has M matching pixels to be rendered in the pixel array corresponding to the original sampling density. The original sampling density corresponds to the resolution of the display. Therefore, when sampling and rendering the current rendering area to be sampled for M consecutive frames, the positions corresponding to the target pixels to be rendered in the M pixels to be rendered are rasterized and pixel shaded in sequence. At this time, the rasterized sampling density is 1 / M of the original sampling density.

[0177] For example, when the resolution of the LCD is 2n*2n, the current rendering area to be sampled includes 4 pixels to be rendered, and the original sampling density is 2n*2n pixels of the current rendering area to be sampled, corresponding to the resolution of the display. When 4 frames are continuously sampled and rendered, the rasterization of these 4 frames is only performed on the target pixels to be sampled and rendered, that is, only 1 n*n pixel is rasterized in each frame, so the rasterization sampling density is 1 / 4 of the original sampling density.

[0178] Referring to FIG. 18 , in some embodiments, step 0211: determining the target pixel to be rendered corresponding to the current scene according to the frame number of the current scene includes:

[0179] Step 02111: Divide the frame number of the current scene by M to determine the remainder; and

[0180] The target pixel to be rendered corresponding to the current scene is determined according to the remainder.

[0181] Specifically, the frame number of the current scene is divided by the number of pixels to be rendered to obtain a remainder, which is then mapped to the sampling position of the target pixel to be rendered in the current scene. The target pixel to be rendered in the current scene is then determined based on the remainder. For example, if there are M matching pixels to be rendered in the sampled rendering area of ​​the current scene, each pixel to be rendered is sequentially numbered from 0 to M. The remainder obtained by dividing the frame number by the number is the number of the target pixel to be rendered.

[0182] For example, referring to FIG19 , in the current scene's to-be-rendered area, there is a 2*2 pixel array including four 1*1 pixels to be rendered, i.e., M=4. The four pixels to be rendered are numbered sequentially, with pixel 0 being the top-left pixel to be rendered in the 2*2 pixel array, pixel 1 being the top-right pixel to be rendered in the 2*2 pixel array, pixel 2 being the bottom-left pixel to be rendered in the 2*2 pixel array, and pixel 3 being the bottom-right pixel to be rendered in the 2*2 pixel array. The to-be-rendered area is sampled for four consecutive frames, with each frame sampling and rendering a 1*1 target pixel to be rendered. Counting begins with frame 0, and the four consecutive frames include frame 0, frame 1, frame 2, and frame 3. That is, frame 0 has a frame number of 0, frame 1 has a frame number of 1, frame 2 has a frame number of 2, and frame 3 has a frame number of 3. The frame number 0 / 4 of the 0th frame has a remainder of 0, which determines that the target pixel to be rendered of the 0th frame sample is the pixel to be rendered in the upper left corner; the frame number 1 / 4 of the 1st frame has a remainder of 1, which determines that the target pixel to be rendered of the 1st frame sample is the pixel to be rendered in the upper right corner; the frame number 2 / 4 of the 2nd frame has a remainder of 2, which determines that the target pixel to be rendered of the 2nd frame sample is the pixel to be rendered in the lower left corner; the frame number 3 / 4 of the 3rd frame has a remainder of 3, which determines that the target pixel to be rendered of the 3rd frame sample is the pixel to be rendered in the lower right corner.

[0183] Referring to FIG. 20 , in some embodiments, sampling rendering further includes vertex processing, step 021 : performing sampling rendering on the current scene to generate a display image, further comprising:

[0184] Step 0213: Perform vertex processing on the current scene to determine multiple rendering areas to be sampled.

[0185] Specifically, when converting a three-dimensional image to a two-dimensional screen, it is necessary to perform coordinate transformation operations on the vertices of the polygon. By using linear algebra calculations, the coordinate data of each vertex in the three-dimensional coordinates is converted and mapped to the two-dimensional space, thereby determining multiple sampling and rendering areas (primitives) to be sampled in the current scene. Then, the pixel array is determined by the preset sampling density (such as determining the preset sampling density according to the resolution of the display), and the fragments corresponding to the screen pixel points covered by the primitives are generated. Finally, through the different sampling positions of the current scene, the matching fragments in the sampling and rendering area are colored, and after a series of tests, the display image pixels are generated.

[0186] Referring to FIG. 21 , in some embodiments, the display method further includes:

[0187] Step 023: Sending the current frame signal to the sampling and rendering module and the display control module respectively to synchronize the sampling and rendering module and the display control module. The sampling and rendering module is configured to generate a display image. The display control module is configured to control the operation of the target display pixels of the display screen by sending a control signal to the display screen.

[0188] Step 024: After the sampling rendering module and the display control module are synchronized, the target pixels to be rendered corresponding to the displayed image correspond to the target display pixel positions on the display screen.

[0189] Specifically, the sampling rendering module and the display control module are synchronized by sending the current frame signal to the sampling rendering module and the display control module respectively. After receiving the current frame signal, the sampling rendering module performs sampling rendering on the current scene, such as through sampling rendering processes such as vertex processing, rasterization, pixel shading and Framebuffer resampling to generate a display image. After receiving the current frame signal, the display control module sends a corresponding gate circuit signal through the FPGA to control the target display pixels in the display to perform display work. After synchronizing the sampling rendering module and the display control module, the target pixels to be rendered in the sampling rendering area of ​​the current scene in the sampling rendering module that have completed the sampling rendering process can be matched one-to-one with the positions of the target display pixels to be controlled in the pixel area of ​​the display in the display control module, thereby achieving consistency between the display position of the display image of the display and the sampling position during sampling rendering, so that the display position of the display image of the continuous M frames is constantly changing, thereby achieving resolution superposition of the display images of the continuous M frames, and ensuring the resolution.

[0190] For example, referring to scene diagrams 16 and 22, the sampling and rendering module and the display control module are synchronized. After receiving the current frame signal, the sampling and rendering module performs vertex processing on the current scene and determines multiple areas to be sampled and rendered. It determines that one of the areas to be sampled and rendered has four pixels to be rendered arranged in a 2*2 array, i.e., M=4. The sampling and rendering module sequentially samples and renders the target pixels to be rendered corresponding to the first frame, the second frame, the third frame, and the fourth frame. After receiving the current frame signal, the display control module sends corresponding gate circuit signals through the FPGA to sequentially control the display of the target display pixels corresponding to the first frame, the second frame, the third frame, and the fourth frame. When the sampling and rendering module performs sampling and rendering on the first frame, the target pixel to be rendered for the first frame is the pixel to be rendered in the upper left corner of the 2*2 matrix formed by the four pixels to be rendered. The target display pixel controlled by the display control module to operate on the display is the display pixel in the upper left corner of the 2*2 matrix corresponding to the area to be sampled and rendered on the display. The position of the pixel to be rendered in the upper left corner of the 2*2 matrix of the area to be rendered corresponds to the display pixel in the upper left corner of the 2*2 matrix of the display.

[0191] When the sampling and rendering module performs sampling and rendering on the second frame, the target to-be-rendered pixel of the second frame is the to-be-rendered pixel in the upper right corner of the 2*2 matrix formed by the four to-be-rendered pixels, and the display control module controls the target display pixel of the display to be the display pixel in the upper right corner of the 2*2 matrix corresponding to the to-be-sampled rendering area in the display, wherein the position of the to-be-rendered pixel in the upper right corner of the 2*2 matrix of the to-be-rendered area corresponds to the display pixel in the upper right corner of the 2*2 matrix of the display;

[0192] When the sampling and rendering module performs sampling and rendering on the third frame, the target to-be-rendered pixel of the third frame is the to-be-rendered pixel in the lower left corner of the 2*2 matrix formed by the four to-be-rendered pixels, and the display control module controls the display to operate as the target display pixel in the lower left corner of the 2*2 matrix corresponding to the to-be-sampled rendering area in the display, wherein the position of the to-be-rendered pixel in the lower left corner of the 2*2 matrix of the to-be-rendered area corresponds to the display pixel in the lower left corner of the 2*2 matrix of the display;

[0193] When the sampling and rendering module performs sampling and rendering on the fourth frame, the target to-be-rendered pixel of the fourth frame is the to-be-rendered pixel in the lower right corner of the 2*2 matrix formed by the four to-be-rendered pixels. The display control module controls the display to operate as the target display pixel in the lower right corner of the 2*2 matrix corresponding to the to-be-sampled rendering area in the display, wherein the position of the to-be-rendered pixel in the lower right corner of the 2*2 matrix of the to-be-rendered area corresponds to the display pixel in the lower right corner of the 2*2 matrix of the display.

[0194] In static scenes, the inter-frame superposition method can achieve the same resolution perception effect as the original image; however, in dynamic scenes, when the objects in the current scene change and the changes occur within the integration time of the human eye, the human eye cannot distinguish the actual objects displayed in each frame, and it is difficult to track the changes of the objects in real time. At this time, the role of resolution is no longer reflected.

[0195] Because the human eye has an area of ​​focus, namely the area that the eye tracks, when the human eye tracks a moving object in real time, the relative position between the human eye and the moving object does not change, that is, the human eye and the moving object are relatively stationary. At this time, the visual effect of the moving object tracked by the human eye in real time is very close to the effect of inter-frame superposition when the motion is compensated back to the previous frame position. Therefore, the moving object in the area tracked by the human eye will have a resolution enhancement effect close to that of the original image; while the resolution of the area not tracked by the human eye will be greater than or equal to the original resolution, which is in line with the usage requirements of the VR device display.

[0196] Therefore, it can be understood that when users use VR devices, the resolution requirements for the area tracked by the human eye are higher, and the resolution requirements for the area not tracked by the human eye are not high. For example, referring to FIG23 , when the human eye maintains real-time tracking, the GPU of the VR device samples and renders the static scene and the dynamic scene respectively based on the acquired current frame signal. At this time, in the dynamic scene, the human eye tracks the shadow block in the current scene. Since the shadow block is tracked in real time by the human eye, the human eye's persistence of vision can better maintain the resolution. Therefore, when calculating the resolution, since the relative position d between the human eye and the moving object does not change, that is, the human eye and the moving object are relatively stationary, motion compensation d1, d2, d3, and d4 can be automatically achieved by the human eye. Then, the visual effect of the moving object tracked in real time by the human eye is very close to the effect of inter-frame superposition when the motion is compensated back to the previous frame position. Therefore, the human eye's persistence of vision can achieve the effect of improving resolution, and the resolution of the moving object being tracked in real time by the human eye can be improved. In static scenes, there is no motion compensation d1, d2, d3, and d4 in the area tracked by the human eye, so the resolution can be improved.

[0197] Furthermore, as can be seen from the above, the display controls the target display pixels to display the display image. The display image displayed on the display will be displayed to the human eye through the optical system of the VR device. The optical system of the VR device can be a Pancake optical module, that is, an optical module consisting of two 1 / 4 phase delay plates (Quarter-Wave Plates, QWP), a semi-transparent and semi-reflective mirror (Beam Splitter, BS), and a reflective polarizer (Reflective Polarizer, RP), wherein the reflective polarizer (Reflective Polarizer, RP) is configured to reflect P-polarized light and transmit S-polarized light.

[0198] For example, please refer to Figure 24, which takes the display image based on a liquid crystal display (LCD) as an example for explanation. The S linearly polarized light emitted by the LCD will change the polarization form after passing through the QWP and become right-circularly polarized light (RCP). After the right-circularly polarized light RCP passes through the BS, the polarization form of the polarized light does not change and remains right-circularly polarized light RCP. The right-circularly polarized light RCP passes through the QWP and becomes P linear polarized light. The P linear polarized light reaches the RP and is emitted. The reflected light is P linear polarized light. The P linear polarized light passes through the QWP and changes back to right-circularly polarized light RCP. The right-circularly polarized light RCP is reflected by the BS and becomes left-circularly polarized light (LCP). The left-circularly polarized light LCP passes through the QWP and becomes S linear polarized light. The S linear polarized light reaches the RP and is transmitted to the human eye for display.

[0199] In addition, the optical system can also include an optical module consisting of a phase delay plate and a Fresnel lens. Based on the principle of light refraction, the curved surface of the Fresnel lens is used to fold and focus the light, thereby realizing the display image on the human eye.

[0200] In summary, when the total display time of M consecutive frames of display images is less than the preset human eye integration time, the target display pixels of the display are controlled to work to display the display image, and the generated display image is displayed on the human eye through the optical module of the VR device, and the display images of M consecutive frames are imaged on the human eye through inter-frame superposition imaging, that is, the target rendering pixels of the sampling rendering are switched, and the target display pixels of the sampling rendering are synchronously displayed on the display, and the inter-frame imaging is performed on the human eye through the optical system, thereby maintaining the resolution while improving the system frame rate.

[0201] Optionally, M is equal to 4, the target pixel to be rendered includes 1 pixel to be rendered, and the 4 pixels to be rendered are arranged in a matrix.

[0202] Specifically, in order to achieve the effect of enhancing the frame rate, the number of pixels to be rendered and the number of frames in the target area to be rendered in the current scene are determined to be 4, and the 4 pixels to be rendered are arranged in a matrix, such as a 4n*n matrix or a 2n*2n matrix. By performing sampling and rendering on the sampled rendering area of ​​the current scene for four consecutive frames, and determining that each frame samples and renders n*n target pixels to be rendered in the 2*2 target area to be rendered, it can be understood that, as shown in FIG. 15 , since, in the embodiment of the present application, during rendering, one frame only samples and renders n*n target pixels to be rendered, compared to the original method of sampling and rendering 2*2 target pixels to be rendered per frame, the sampling and rendering time is effectively shortened to 1 / 4 of the original time, thereby enhancing the frame rate.

[0203] It should be pointed out that when the frame rate = 180HZ and the spacing between frames is 1s / 180HZ, 1s / 180HZ*9=50ms. Please refer to Figure 26. The embodiment of the implementation method of the present application can realize 9-frame superposition. Let M=9, that is, it is determined that the number of pixels to be sampled and rendered in the area to be sampled and rendered is 9, and the number of frames is 9. Then, as shown in the scene graph, the n*n pixels in the area to be sampled and rendered can be sampled and rendered through 9 consecutive frames to generate a display image.

[0204] Please refer to FIG. 13 again. To facilitate better implementation of the display method of the embodiment of the present application, the embodiment of the present application further provides a display device 10. The display device 10 may include a generation module 11 and a display module 12. The generation module 11 is used to sample and render the current scene to generate a display image. The current scene includes multiple areas to be sampled and rendered. The areas to be sampled and rendered have M matching pixels to be rendered. The display image is generated by sampling and rendering according to the position of the target pixel to be rendered among the M pixels to be rendered. The positions of the target pixel to be rendered corresponding to the display images of M consecutive frames are different, and M is a positive integer. The display module 12 is used to control the target display pixel of the display screen to display the display image. The display screen includes multiple pixel areas. The pixel area includes M display pixels. The target display pixel is the display pixel among the M display pixels corresponding to the position of the target pixel to be rendered.

[0205] In some embodiments, each of the to-be-sampled rendering areas has M matching pixels to be rendered in the pixel array corresponding to the original sampling density, and the original sampling density corresponds to the resolution of the display screen; the sampling rendering includes rasterization and pixel shading, and the generation module 11 is further specifically used to determine the target pixels to be rendered corresponding to the current scene according to the frame number of the current scene; rasterization and pixel shading are performed on the positions of the target pixels to be rendered corresponding to the current scene in sequence to generate the display image, and the rasterization sampling density is 1 / M of the original sampling density.

[0206] In some embodiments, the generating module 11 is further configured to divide the frame number of the current scene by M to determine a remainder; and determine the target pixel to be rendered corresponding to the current scene according to the remainder.

[0207] In some embodiments, the generating module 11 is further configured to perform vertex processing on the current scene to determine a plurality of rendering areas to be sampled.

[0208] In some embodiments, the display device 10 also includes a synchronization module 13 for sending a current frame signal to the sampling rendering module and the display control module, respectively, to synchronize the sampling rendering module and the display control module. The sampling rendering module is configured to generate a display image, and the display control module is configured to control the operation of the target display pixels of the display screen by sending a control signal to the display screen, and after the sampling rendering module and the display control module are synchronized, the target display pixel to be rendered corresponding to the display image corresponds to the target display pixel position of the display screen.

[0209] The display device is described above from the perspective of a functional module in conjunction with the accompanying drawings. The functional module can be implemented in hardware, can be implemented by instructions in software, or can be implemented by a combination of hardware and software modules. Specifically, each step of the method implementation in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as being executed by a hardware encoding processor, or can be executed by a combination of hardware and software modules in the encoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method implementation in conjunction with its hardware.

[0210] Please refer to Figure 27. The display device 100 of an embodiment of the present application includes a display 20, a processor 30, a memory 40 and a computer program, wherein the computer program is stored in the memory 40 and executed by the processor 30, and the computer program includes instructions for executing the display method of any of the above embodiments.

[0211] Optionally, the processor 30 includes a main processor 31 and a graphics processing unit (GPU) 32 , and the GPU processor 32 samples and renders the current scene to generate a scene image.

[0212] Optionally, the display device of the embodiment of the present application also includes a sampling rendering module 50 and a display control module 60. The sampling rendering module 50 is configured to generate a display image, and the display control module 60 is configured to control the operation of the target display pixels of the display 20 by sending a control signal to the display screen. The frame signal is sent through the main processor 31 to synchronize the sampling rendering module 50 and the display control module 60 so that the polarization angle of the displayed image and the light modulated by the polarization modulator can correspond.

[0213] Optionally, the display device of the embodiment of the present application further includes an optical module, the optical module includes a phase delay plate and a Fresnel lens, or the optical module includes a phase delay plate and a folded optical path component.

[0214] Please refer to Figures 27 and 28. When the current scene is sampled and rendered by the GPU image processor 32 to generate a scene image and the display is controlled by the FPGA, the main processor 31 synchronously sends frame signals to the GPU and FPGA. In the sampling and rendering module 50, the GPU processor 32 performs vertex processing according to the application scenario, and generates a display image after rasterization, pixel shading and Framebuffer resampling processing according to the frame signal. At the same time, after receiving the frame signal, the FPGA sends a gate circuit signal to control the target display pixel in the display 20 to work. The GPU sends the generated display image to the display 20 with the selected target display pixel for display. The light of the scene image displayed by the display 20 is superimposed on the human eye through the optical module. The rasterization, pixel shading and Framebuffer resampling processing methods of the GPU processor 32 are continuously switched according to the frame signal, and the target display pixel of the display 20 is synchronously selected for display, and the image is superimposed on the human eye, so that the frame rate of the display image is superimposed to ensure the resolution.

[0215] Please refer to Figure 29. The embodiment of the present application also provides a computer-readable storage medium 600, on which a computer program 610 is stored. When the computer program 610 is executed by the processor 620, the steps of the display method of any of the above-mentioned embodiments are implemented. For the sake of brevity, they are not repeated here.

[0216] In the description of this specification, the reference terms "certain embodiments", "in an example", "exemplarily", etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are mutually inconsistent.

[0217] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0218] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A display method, characterized in that: include: Perform sampling and rendering on the current scene to generate a display image; The display image is displayed.

2. The display method according to claim 1, characterized in that: The sampling rendering includes rasterization, and the sampling positions of the display images of consecutive frames are different when rasterization is performed; and the displaying of the display images includes: The display image is displayed, and the polarization angle of the display light corresponding to the display image is adjusted so that the polarization angle matches the sampling position corresponding to the current scene, and different polarization angles correspond to different display positions of the display image.

3. The display method according to claim 1, characterized in that: The sampling rendering further includes pixel shading, and the sampling rendering of the current scene to generate a display image includes: Determining the sampling position of the current scene according to the frame number corresponding to the current scene; The current scene is rasterized, and the pixel shading is performed on the current scene according to the sampling position of the current scene to generate the display image.

4. The display method according to claim 3, characterized in that: The step of determining the sampling position of the current scene according to the frame sequence number corresponding to the current scene includes: When the frame sequence number is an even number, determining the sampling position of the current scene as the first sampling position; When the frame sequence number is an odd number, the sampling position of the current scene is determined to be a second sampling position, and the first sampling position is different from the second sampling position.

5. The display method according to claim 4, characterized in that: The continuous multiple frames of display images include a first display image and a second display image, the first display image corresponds to the first sampling position, the second display image corresponds to the second sampling position, and in the diagonal direction of the pixel, there is a preset pixel displacement between the first sampling position and the second sampling position.

6. The display method according to claim 4, characterized in that: The sampling rendering also includes vertex processing, and the sampling rendering of the current scene to generate a display image also includes: Performing vertex processing on the current scene to determine a plurality of rendering areas to be sampled; The rasterizing the current scene and performing pixel shading on the current scene according to the sampling position of the current scene to generate the display image includes: Determine the rendering area to be sampled that matches each pixel of the current pixel array, wherein the current pixel array is determined according to a preset sampling density; According to the sampling position of the current scene, the pixel shading is performed on the pixels matching the to-be-sampled rendering area to generate the display image.

7. The display method according to claim 1, characterized in that: Also includes: Sending a current frame signal to a sampling rendering module and a display control module respectively to synchronize the sampling rendering module and the display control module, wherein the sampling rendering module is configured to generate the display image, and the display control module is configured to control the polarization modulator to adjust the polarization angle of the display light by sending a control signal to the polarization modulator; When the sampling rendering module and the display control module are synchronized, the polarization angle matches the sampling position corresponding to the current scene.

8. The display method according to claim 1, characterized in that: The total display time of the continuous multiple frames of the displayed image is less than the preset human eye integration time.

9. The display method according to claim 1, characterized in that: The current scene includes a plurality of to-be-sampled rendering areas, wherein the to-be-sampled rendering areas have M matching to-be-rendered pixels, and the display image is generated by sampling and rendering according to the position of a target to-be-rendered pixel among the M to-be-rendered pixels, and the positions of the target to-be-rendered pixels corresponding to the display images of M consecutive frames are different from each other, and M is a positive integer; The displaying of the display image comprises: The target display pixel of the display screen is controlled to work to display the display image, the display screen includes a plurality of pixel areas, the pixel area includes M display pixels, and the target display pixel is a display pixel among the M display pixels corresponding to the position of the target pixel to be rendered.

10. The display method according to claim 1, characterized in that: Each of the to-be-sampled rendering areas has M matching to-be-rendered pixels in a pixel array corresponding to an original sampling density, and the original sampling density corresponds to a resolution of the display screen; The sampling rendering includes rasterization and pixel shading, and the sampling rendering of the current scene to generate a display image includes: Determining the target pixel to be rendered corresponding to the current scene according to the frame sequence number of the current scene; The positions of the target pixels to be rendered corresponding to the current scene are sequentially rasterized and pixel colored to generate the display image, and the sampling density of the rasterization is 1 / M of the original sampling density.

11. The display method according to claim 10, characterized in that: The step of determining the target pixel to be rendered corresponding to the current scene according to the frame sequence number of the current scene includes: Dividing the frame number of the current scene by M to determine a remainder; and The target pixel to be rendered corresponding to the current scene is determined according to the remainder.

12. The display method according to claim 9 or 10, characterized in that: The sampling rendering also includes vertex processing, and the sampling rendering of the current scene to generate a display image also includes: Vertex processing is performed on the current scene to determine a plurality of rendering areas to be sampled.

13. The display method according to claim 9, characterized in that: Also includes: Sending a current frame signal to a sampling rendering module and a display control module respectively to synchronize the sampling rendering module and the display control module, wherein the sampling rendering module is configured to generate the display image, and the display control module is configured to control the target display pixel of the display screen to work by sending a control signal to the display screen; After the sampling rendering module and the display control module are synchronized, the target pixels to be rendered corresponding to the display image correspond to the target display pixel positions on which the display screen operates.

14. The display method according to claim 9, characterized in that: The total display time of the M consecutive frames of the displayed image is less than the preset human eye integration time.

15. The display method according to claim 9, characterized in that: M is equal to 4, the target pixel to be rendered includes 1 pixel to be rendered, and the 4 pixels to be rendered are arranged in a matrix.

16. A display device, characterized in that: The display device comprises: A generation module, used for performing sampling rendering on the current scene to generate a display image, wherein the sampling rendering includes rasterization, and the sampling positions of consecutive frames of the display image are different when rasterization is performed; The display module is used to display the display image and adjust the polarization angle of the display light corresponding to the display image so that the polarization angle matches the sampling position corresponding to the current scene.

17. A display device, characterized in that: The display device comprises: A generation module performs sampling and rendering on a current scene to generate a display image, wherein the current scene includes a plurality of to-be-sampled rendering areas, wherein the to-be-sampled rendering areas have M matching to-be-rendered pixels, and the display image is generated by sampling and rendering according to the position of a target to-be-rendered pixel among the M to-be-rendered pixels, wherein the positions of the target to-be-rendered pixels corresponding to the display images of M consecutive frames are different from each other, and M is a positive integer; The display module is used to control the target display pixel of the display screen to work so as to display the display image. The display screen includes multiple pixel areas, and the pixel area includes M display pixels. The target display pixel is a display pixel among the M display pixels corresponding to the position of the target pixel to be rendered.

18. A display device, characterized in that: include: Processor, memory; and A computer program, wherein the computer program is stored in the memory and executed by the processor, and the computer program includes instructions for executing the display method according to any one of claims 1 to 15.

19. The display device according to claim 18, characterized in that Also includes: a display configured to display the display image; a polarization modulator configured to adjust the polarization angle of display light emitted by the display; A sampling and rendering module, configured to generate the display image; The display control module is configured to control the polarization modulator to adjust the polarization angle of the display light by sending a control signal to the polarization modulator.

20. The display device according to claim 18, characterized in that It also includes an optical module, which includes the polarization modulator, the birefringent crystal and the Fresnel lens, and the polarization modulator, the birefringent crystal and the Fresnel lens are arranged in sequence along the exit direction of the display light; or, the optical module includes the polarization modulator, the birefringent crystal and the folded optical path component, and the polarization modulator, the birefringent crystal and the folded optical path component are arranged in sequence along the exit direction of the display light.

21. A display device, characterized in that: include: Processor, memory, display; and A computer program, wherein the computer program is stored in the memory and executed by the processor, and the computer program includes instructions for executing the display method according to any one of claims 1 to 15.

22. The display device according to claim 21, characterized in that It also includes a sampling rendering module and a display control module, wherein the sampling rendering module is configured to generate the display image, and the display control module is configured to control the target display pixel of the display screen to operate by sending a control signal to the display screen.

23. The display device according to claim 21, characterized in that It also includes an optical module, which includes a phase delay plate and a Fresnel lens, or the optical module includes a phase delay plate and a folded optical path component.

24. A non-volatile computer-readable storage medium containing a computer program, wherein when the computer program is executed by a processor, the processor executes the display method according to any one of claims 1 to 15.

Images