Display method and device based on virtual resolution and electronic equipment

Abstract

The invention discloses a display method and device based on virtual resolution and electronic equipment. The method comprises the steps of obtaining graphics processor information of a hardware platform, determining a virtual resolution based on the graphics processor information, rendering a display image to a frame cache based on the virtual resolution in a first rendering stage, and performing enhanced rendering on the display image in the frame cache in a second rendering stage, and outputting the rendered display image to a display system. According to the method, the software rendering pressure is effectively reduced through the virtual resolution, the rendering fluency is improved, and the image quality of the display image with the reduced image quality in the software rendering stage is improved through enhanced rendering, so that the display image quality is improved while the rendering fluency is improved.

Description

Technical Field

[0001] The present invention relates to the field of display technology, and in particular to a display method and device based on virtual resolution, and electronic equipment. Background Art

[0002] Linux desktop systems usually use X11 (a bitmap display window system), Xserver (a server that communicates with hardware at the bottom level) or Wayland (a display service) as a graphics display manager. Xserver is responsible for interacting with the physical display engine. The display mechanism of Xserver requires that the frame buffer resolution be consistent with the output resolution. Today's multimedia embedded SOCs (system on chip), especially mid- and low-end chip platforms, usually have powerful video and display output capabilities, and can support resolutions up to 4K60 frames, while the performance of GPUs (graphics processors) is usually weak, and the rendering capabilities are generally within 1080P.

[0003] Therefore, in Linux desktop systems, it is impossible to ensure good display image quality while achieving high rendering fluency. Summary of the invention

[0004] The present invention provides a display method and device based on virtual resolution, and an electronic device, which can improve the quality of displayed images while improving rendering fluency.

[0005] In one aspect of the present invention, a display method based on virtual resolution is provided. The method includes: obtaining graphics processor information of a hardware platform, and determining a virtual resolution based on the graphics processor information; rendering a display image to a frame buffer based on the virtual resolution in a first rendering phase; performing enhanced rendering on the display image in the frame buffer in a second rendering phase to obtain a rendered display image; and outputting the rendered display image to a display system.

[0006] In another aspect of the present invention, a display device based on virtual resolution is provided. The device includes: a calculation module configured to obtain graphics processor information of a hardware platform and determine a virtual resolution based on the graphics processor information; a software rendering module configured to render a display image to a frame buffer based on the virtual resolution in a first rendering phase; a hardware rendering module configured to perform enhanced rendering on the display image in the frame buffer in a second rendering phase to obtain a rendered display image; and an output module configured to output the rendered display image to a display system.

[0007] In another aspect of the present invention, an electronic device is provided, comprising a memory configured to store an executable program; and a processor configured to execute the program to perform the above-mentioned display method based on virtual resolution.

[0008] In another aspect of the present invention, a computer-readable medium is provided, wherein a computer program is stored on the medium, and the computer program is executed by a processor to implement the above-mentioned display method based on virtual resolution.

[0009] According to the present invention, the graphics processor information of the hardware platform is obtained, and the virtual resolution is determined based on the graphics processor information. In the first rendering stage, the display image is rendered to the frame buffer based on the virtual resolution. In the second rendering stage, the display image in the frame buffer is enhanced and rendered. The rendered display image is output to the display system. The virtual resolution effectively reduces the software rendering pressure and improves the rendering smoothness. The image quality of the display image whose image quality is reduced in the software rendering stage is improved through enhanced rendering, thereby improving the display image quality while improving the rendering smoothness. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a flowchart of a display method based on virtual resolution according to an embodiment of the present invention;

[0011] Figure 2 is a control flow chart of a display method based on virtual resolution according to an embodiment of the present invention;

[0012] Figure 3 is a schematic structural diagram of a display device based on virtual resolution according to an embodiment of the present invention;

[0013] Figure 4 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0015] Today's multimedia embedded SOCs, especially mid- and low-end chip platforms, usually have powerful video and display output capabilities, with resolutions supporting up to 4K60 frames. However, GPU performance is usually weak, with rendering capabilities generally limited to 1080P. Therefore, in Linux desktop systems, it is impossible to ensure good display image quality while achieving high rendering fluency.

[0016] The rendering system disclosed in the present invention includes a rendering front end and a rendering back end. The rendering front end is responsible for processing the UI layout and drawing of the application program. The rendering back end is mainly related to hardware acceleration and screen synthesis display.

[0017] As the core service of the operating system, the rendering front end is mainly responsible for managing the hierarchical relationship, synthesis and display of UI components. The rendering front end outputs the synthesized layers to the final screen display content. SurfaceFlinger of the Android system and Xserver / Wayland of Linux are typical rendering front ends.

[0018] The rendering backend takes over all the rendered images of the rendering frontend, executes the layer synthesis strategy, and directly passes or merges the layers under the constraints of the number and characteristics of the hardware layers to maximize the use of the hardware images. In addition, the rendering backend associates the hardware layers with the display interface and executes the display output strategy. The chip generally has a built-in hardware-accelerated rendering backend.

[0019] Generally, the rendering front end calls the rendering back end, and the rendering back end provides rendering acceleration and display management capabilities to ensure the system's efficient image rendering efficiency and achieve the best display system.

[0020] According to the present disclosure, the rendering front end uses virtual resolution in the software rendering stage to reduce software rendering pressure, reduce image quality, improve rendering fluency, and achieve stable full-frame rendering. In addition, the rendering back end uses efficient rendering based on layered rendering and enhanced rendering based on image enhancement in the hardware rendering stage to improve display image quality.

[0021] Regarding efficient rendering based on layered rendering, the rendering synthesis system of the operating system is generally based on layered rendering. For example, there are three applications A, B, and C in the current system environment, and application A includes video playback. The rendering synthesis system of the operating system sees four software layers, namely, three image layers Sa, Sb, and Sc and a video layer SVa. The hardware rendering synthesis system of the electronic system is also layered, in which there are differences in the number of hardware layers and the resolution of the layers, as well as the correspondence between the hardware image and the display interface output. The rendering synthesis system of the operating system is generally bound to the hardware rendering synthesis system of the electronic device. The binding is not a simple mapping relationship, but includes complex image synthesis strategies to be compatible with and solve the hardware constraints of the number of images, image resolution, and image format. The technical solution disclosed in the present invention utilizes the hardware rendering synthesis system of the electronic system (chip). The hardware rendering synthesis system has the characteristics of layered rendering and hardware accelerated rendering.

[0022] The present disclosure provides a display method based on virtual resolution. According to the present disclosure, the graphics processor information of the hardware platform is obtained, and the virtual resolution is determined based on the graphics processor information. In the first rendering stage, the display image is rendered to the frame buffer based on the virtual resolution. In the second rendering stage, the display image in the frame buffer is enhanced and rendered, and the rendered display image is output to the display system. In this way, the software rendering pressure is effectively reduced and the rendering fluency is improved through virtual resolution. Then, the image quality of the display image whose image quality is reduced in the software rendering stage is improved through enhanced rendering, thereby improving the display image quality while improving the rendering fluency.

[0023] Hereinafter, the technical solution according to the present disclosure will be described with reference to specific embodiments and in conjunction with the accompanying drawings.

[0024] Figure 1 is a flow chart showing a display method 100 based on virtual resolution according to an embodiment of the present disclosure. Figure 1 The method 100 includes the following steps 102 to 108.

[0025] In step 102, graphics processor information of the hardware platform is obtained, and a virtual resolution is determined based on the graphics processor information.

[0026] In some embodiments, the computing power of the graphics processor is calculated according to the graphics processor information, a graphics processor load table including graphics processor loads at different resolutions is generated according to the computing power, and a resolution corresponding to a graphics processor load having a preset value is selected from the graphics processor load table, and the resolution is used as the virtual resolution. In this way, the resolution corresponding to the graphics processor load having a preset value, i.e., the optimal resolution, is selected from the graphics processor load table including graphics processor loads at different resolutions, so that the resolution can be reduced from the image source end, reducing the bandwidth pressure of the double-rate synchronous dynamic random access memory (DDR) for full-link image transmission and rendering.

[0027] In some embodiments, the graphics processor information includes the core frequency, the number of stream processors, and the number of arithmetic logic units of the graphics processor. The computing power of the image processor is calculated based on the core frequency, the number of stream processors, and the number of arithmetic logic units of the graphics processor of the hardware platform. In some embodiments, the computing power of the image processor = core frequency × number of stream processors × number of arithmetic logic units × 2. In this way, by calculating the computing power of the image processor, that is, the number of floating-point operations that can be performed per second, it can be used as data support for the load table, and the optimal virtual resolution can be determined according to the load table.

[0028] In some embodiments, the graphics processor information includes performance register information of the graphics processor. The real-time calculation amount and the maximum calculation amount are determined according to the performance register information of the graphics processor, and the real-time calculation amount and the maximum calculation amount are used as the computing power of the graphics processor. In this way, in addition to calculating the computing power of the image processor according to the formula, it can also be determined from the performance register information of the graphics processor, which is more flexible.

[0029] In step 104 , in a first rendering phase, the display image is rendered to a frame buffer based on the virtual resolution.

[0030] In some embodiments, it is determined whether the physical resolution is greater than the virtual resolution. If the physical resolution is greater than the virtual resolution, the virtual resolution is sent to the display service using the display driver, and in the software rendering stage, the display service calls the graphics processor to render the display image corresponding to the application to the frame buffer according to the virtual resolution. In this way, when the physical resolution is greater than the virtual resolution, it means that the resolution needs to be reduced, and the virtual resolution also maintains the best compatibility, and only the display driver needs to be adapted, which significantly improves the rendering pressure of the display service.

[0031] In some embodiments, during the software rendering stage, the display service determines the screen window size according to the virtual resolution, sends the screen window size to the application that displays the image, requests a frame buffer according to the virtual resolution through the display service, and calls the graphics processor through the display service to render and synthesize the display image corresponding to the application into the frame buffer. In this way, the display service renders according to the virtual resolution, reduces the rendering load of the graphics processor, and improves the rendering fluency of the system.

[0032] In step 106, enhanced rendering is performed on the display image in the frame buffer in a second rendering stage to obtain a rendered display image.

[0033] In some embodiments, during the hardware rendering stage, the frame buffer is configured to a physical display engine through a display service, the display image in the frame buffer is post-processed for image quality using the physical display engine to obtain a processed image, and the processed image is post-processed for video super-resolution using the physical display engine to obtain a rendered display image. In this way, enhanced rendering is achieved through image quality post-processing and video super-resolution post-processing, effectively improving image quality.

[0034] In step 108, the rendered display image is output to a display system.

[0035] In some embodiments, the rendered display image is magnified to an actual output resolution by using the physical display engine and then output to a display system, thereby ensuring the final display effect.

[0036] In some embodiments, the physical display engine is used to determine the actual output resolution corresponding to the virtual resolution according to a preset mapping relationship table, and the preset mapping relationship table includes virtual resolutions and actual output resolutions that correspond to each other; and the physical display engine is used to enlarge the rendered display image to the actual output resolution and then output it to the physical display interface corresponding to the display system. In this way, after determining the actual output resolution corresponding to the virtual resolution according to the preset mapping relationship table, the rendered display image is enlarged to the actual output resolution and output, which not only reduces the rendering pressure, but also can well ensure the image display effect.

[0037] Hereinafter, application scenarios of a display method and apparatus based on virtual resolution and an electronic device according to an embodiment of the present invention will be described by way of examples.

[0038] Figure 2 is a flow chart showing a display method based on virtual resolution according to an embodiment of the present invention. Figure 2 The method includes the following steps 201 to 210.

[0039] In step 201, the graphics processor information of the hardware platform is obtained, and the computing capability of the graphics processor is calculated according to the graphics processor (GPU) information. In some embodiments, the GPU is used as a rendering backend.

[0040] In some embodiments, the graphics processor information includes a core frequency, a number of stream processors, and a number of arithmetic logic units of the graphics processor.

[0041] In this embodiment, calculating the computing power of the graphics processor according to the graphics processor information includes: calculating the computing power of the graphics processor according to the core frequency, the number of stream processors and the number of arithmetic logic units (ALUs) of the graphics processor. In some embodiments, the calculation can be performed specifically by the following formula:

[0042] The computing power of the image processor = core frequency × number of stream processors × number of arithmetic logic units × 2.

[0043] In the formula, 2 means that the image processor can execute 2 instructions in a single clock cycle.

[0044] In some other embodiments, the graphics processor information includes performance register information of the graphics processor.

[0045] In this embodiment, calculating the computing capacity of the graphics processor according to the graphics processor information includes: determining the real-time computing amount and the maximum computing amount according to the performance register information of the graphics processor; and using the real-time computing amount and the maximum computing amount as the computing capacity of the graphics processor.

[0046] In step 202, a graphics processor load table including graphics processor loads at different resolutions is generated according to the computing capabilities.

[0047] Specifically, by executing the test program, the graphics processor is used to render images of different resolutions according to the computing power, and a graphics processor load table at different resolutions is obtained.

[0048] In some embodiments, the computing capacity of the graphics processor is recalculated at every predetermined period, and a graphics processor load table under different resolutions is regenerated.

[0049] In step 203, a resolution corresponding to a graphics processor load having a preset value is selected from the graphics processor load table, and the resolution is used as a virtual resolution.

[0050] In some embodiments, a resolution corresponding to a graphics processor load of 50% is selected from the graphics processor load table.

[0051] The graphics processor is the general rendering hardware in the system. Considering the system balance, the display system accounts for 50%, and the remaining rendering capacity is reserved for the user interface management (UI) system and the game system.

[0052] In step 204 , during the software rendering stage, it is determined whether the physical resolution is greater than the virtual resolution. If so, step 205 is executed.

[0053] In step 205, the virtual resolution is sent to the display service using the display driver.

[0054] In some embodiments, the display service is Xserver or Wayland.

[0055] In step 206 , the display service calls the graphics processor to render the display image corresponding to the application to the frame buffer according to the virtual resolution.

[0056] Specifically, the display service determines the screen window size according to the virtual resolution, and sends the screen window size to the application program that displays the image; the display service requests the frame buffer according to the virtual resolution; and the display service calls the graphics processor to render and synthesize the display image corresponding to the application program into the frame buffer. In some embodiments, the application program refers to an application program that complies with the Xclient protocol.

[0057] In step 207, the frame buffer is configured to the physical display engine through the display service in the hardware rendering stage.

[0058] In some embodiments, the physical display engine is a VOP, which serves as a display backend. The frame buffer is generally a physical display memory, which can be directly configured to the physical display engine VOP, and the physical display engine VOP outputs to a specified physical display output interface according to user configuration.

[0059] In step 208, the physical display engine is used to perform image quality post-processing on the display image in the frame buffer to obtain a processed image, so as to enhance the image quality of the image on the display medium.

[0060] In step 209, the processed image is subjected to video super-resolution post-processing using a physical display engine to obtain a rendered display image to enhance local texture details.

[0061] In step 210, the rendered display image is output to a display system.

[0062] Specifically, the rendered display image is enlarged to an actual output resolution using a physical display engine and then output to a display system.

[0063] In some embodiments, a physical display engine is used to determine an actual output resolution corresponding to a virtual resolution according to a preset mapping relationship table, where the preset mapping relationship table includes mutually corresponding virtual resolutions and actual output resolutions; the rendered display image is enlarged to the actual output resolution by the physical display engine, and then output to a physical display interface corresponding to the display system.

[0064] Assuming that the virtual resolution is 1920×1080@30FPS, the physical display engine VOP user configures the High Definition Multimedia Interface (HDMI) display resolution to 3840×2160@30FPS, that is, the virtual resolution 1080P is enabled, and the native Xserver framework and hardware display are consistent with 2160P, which reduces the rendering pressure of XClient and XServer. The physical display engine VOP finds the actual output resolution 2160P through the virtual resolution 1080P, and then renders the display image according to the actual output resolution 2160P.

[0065] X protocol is a set of protocols for Linux to manage display. Xserver and Wayland are widely used in Linux. X protocol includes Xclient protocol and Xserver protocol. Developers can develop UI window frameworks based on Xclient, such as QT / GTK. Developers can develop window synthesis managers based on Xserver protocol, such as Xserver and Wayland.

[0066] According to another aspect of the present invention, Figure 3 is a block diagram showing a display device 300 based on virtual resolution according to an embodiment of the present invention. Figure 3 The virtual resolution-based display device 300 includes a calculation module 302 , a software rendering module 304 , a hardware rendering module 306 and an output module 308 .

[0067] The calculation module 302 is configured to obtain graphics processor information of the hardware platform and determine the virtual resolution based on the graphics processor information.

[0068] The software rendering module 304 is configured to render the display image to a frame buffer based on the virtual resolution in a first rendering phase.

[0069] The hardware rendering module 306 is configured to perform enhanced rendering on the display image in the frame buffer in the second rendering stage to obtain a rendered display image.

[0070] The output module 308 is configured to output the rendered display image to a display system.

[0071] It should be understood that the computing module 302, the software rendering module 304, the hardware rendering module 306 and the output module 308 can be further configured to execute the corresponding steps or actions in the method described above, which will not be described in detail here.

[0072] According to yet another aspect of the present invention, Figure 4 4 is a schematic diagram showing a display device 400 based on virtual resolution according to an embodiment of the present invention. Figure 4 The electronic device 400 includes a memory 402 and a processor 404. The memory 402 is configured to store an executable program that can be run on the processor 404. The processor 404 is configured to execute the program to implement the various steps or actions of the display method based on virtual resolution as described above.

[0073] In summary, the display method and device based on virtual resolution, and the electronic device provided by the present invention obtain the graphics processor information of the hardware platform, and determine the virtual resolution based on the graphics processor information, render the display image to the frame buffer based on the virtual resolution in the first rendering stage, enhance the rendering of the display image in the frame buffer in the second rendering stage, and output the rendered display image to the display system, effectively reduce the software rendering pressure and improve the rendering fluency through the virtual resolution, and then enhance the rendering to improve the image quality of the display image whose image quality is reduced in the software rendering stage, thereby improving the rendering fluency and the display image quality. In addition, after determining the actual output resolution corresponding to the virtual resolution according to the preset mapping relationship table, the rendered display image is enlarged to the actual output resolution for output, which not only reduces the rendering pressure, but also can well ensure the image display effect.

[0074] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A display method based on virtual resolution, characterized in that: include: Acquire graphics processor information of the hardware platform, and determine the virtual resolution based on the graphics processor information; Rendering the display image to a frame buffer based on the virtual resolution in a first rendering stage; In a second rendering stage, enhanced rendering is performed on the display image in the frame buffer to obtain a rendered display image; as well as The rendered display image is output to a display system.

2. The method according to claim 1, characterized in that Determining the virtual resolution based on the graphics processor information includes: Calculating the computing capability of the graphics processor according to the graphics processor information; generating a graphics processor load table including graphics processor loads at different resolutions according to the computing capability; and A resolution corresponding to a graphics processor load having a preset value is selected from the graphics processor load table, and the resolution is used as the virtual resolution.

3. The method according to claim 2, characterized in that The graphics processor information includes the core frequency, the number of stream processors and the number of arithmetic logic units of the graphics processor, Calculating the computing power of the graphics processor according to the graphics processor information includes: calculating the computing power of the graphics processor according to the core frequency of the graphics processor of the hardware platform, the number of stream processors, and the number of arithmetic logic units.

4. The method according to claim 3, characterized in that Calculating the computing power of the image processor according to the core frequency of the graphics processor of the hardware platform, the number of stream processors, and the number of arithmetic logic units includes calculating by the following relationship: The computing power of the image processor = core frequency × number of stream processors × number of arithmetic logic units × 2.

5. The method according to claim 2, characterized in that: The graphics processor information includes performance register information of the graphics processor, Calculating the computing capacity of the graphics processor according to the graphics processor information includes: determining the real-time computing amount and the maximum computing amount according to the performance register information of the graphics processor, and using the real-time computing amount and the maximum computing amount as the computing capacity of the graphics processor.

6. The method according to claim 1, characterized in that Rendering the display image to the frame buffer based on the virtual resolution in the first rendering stage includes: Determining whether the physical resolution is greater than the virtual resolution; If the physical resolution is greater than the virtual resolution, sending the virtual resolution to a display service using a display driver; and In the software rendering stage, the display service calls a graphics processor to render a display image corresponding to the application to the frame buffer according to the virtual resolution.

7. The method according to claim 6, characterized in that In the software rendering stage, calling the graphics processor through the display service to render the display image corresponding to the application to the frame buffer according to the virtual resolution includes: In the software rendering stage, the display service determines the screen window size according to the virtual resolution, and sends the screen window size to the application program for displaying the image; requesting a frame buffer according to the virtual resolution by the display service; and The display service calls the graphics processor to render and synthesize the display image corresponding to the application into the frame buffer.

8. The method according to claim 1, characterized in that Performing enhanced rendering on the display image in the frame buffer in the second rendering stage to obtain a rendered display image includes: In the hardware rendering stage, the frame buffer is configured to the physical display engine through the display service; Performing image quality post-processing on the display image in the frame buffer using the physical display engine to obtain a processed image; and The processed image is subjected to video super-resolution post-processing by utilizing the physical display engine to obtain a rendered display image.

9. The method according to claim 8, characterized in that Outputting the rendered display image to a display system includes: The rendered display image is magnified to an actual output resolution using the physical display engine and then output to a display system.

10. The method according to claim 9, characterized in that The method of using the physical display engine to enlarge the rendered display image to an actual output resolution and then outputting the image to a display system comprises: Determining, by the physical display engine, an actual output resolution corresponding to the virtual resolution according to a preset mapping relationship table, wherein the preset mapping relationship table includes virtual resolutions and actual output resolutions corresponding to each other; and The rendered display image is magnified to the actual output resolution by using the physical display engine, and then output to a physical display interface corresponding to the display system.

11. A display device based on virtual resolution, characterized in that: include: A calculation module, configured to obtain graphics processor information of a hardware platform and determine a virtual resolution based on the graphics processor information; A software rendering module, configured to render a display image to a frame buffer based on the virtual resolution in a first rendering stage; A hardware rendering module is configured to perform enhanced rendering on the display image in the frame buffer in a second rendering stage to obtain a rendered display image; as well as The output module is configured to output the rendered display image to a display system.

12. An electronic device, characterized in that: include: a memory configured to store an executable program; as well as A processor configured to execute the program to perform the method according to any one of claims 1 to 10.

Images