Display picture rendering method and XR equipment
By adopting the dual-threaded rendering method in XR devices, sharing the pressure of the rendering process, solving the problem of poor rendering effect caused by insufficient resources of the core rendering process, and achieving a more efficient rendering effect.
Patent Information
- Application Number
- CN202410927425.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2025-03-14
AI Technical Summary
In the prior art, the core rendering process assumes too many functions, resulting in high rendering pressure and prone to insufficient resources, which leads to motion prediction failure and jitter problems. In severe cases, it will cause black screen, black edge tearing or even equipment restart.
Using the two-threaded rendering method, the first rendering process sends a start rendering instruction to the second rendering process, and the second rendering process performs rendering category determination and rendering method, obtains the rendered frame image and sends it to the first rendering process, and performs final motion prediction and anti-distortion processing.
It reduces the pressure of the first rendering thread, reduces the resource consumption of the first rendering process, avoids the motion prediction failure and jitter problems caused by insufficient resources, and improves the rendering effect.
Smart Images
Figure CN119621367A_ABST
Abstract
Description
Background Art
[0002] With the development and rise of games and other applications in mobile devices, XR has also made great progress in the gaming industry, producing high-quality game content. However, it is limited by the slow update and iteration of hardware level, especially the processor. It is difficult to produce higher quality content. Although higher-quality games can be streamed on the desktop, it is still the best solution to use all-in-one rendering due to the cumbersome operation and high latency.
[0003] All-in-one rendering uses the core rendering process to render the display screen, such as Figure 1 FIG. 1 is a schematic diagram of a core rendering process in the prior art. Figure 1 It can be seen that the rendering functions of the core rendering process include: VST (Virtual Studio Technology) rendering, application multi-layer synthesis, global function rendering, motion prediction and anti-distortion. Among them, VST rendering means that if the current display screen needs to be rendered by VST, it is necessary to collect color camera data for rendering. Application multi-layer synthesis means that if there are audio and video or other ordinary App compatible screens, it is necessary to obtain other application process rendering screens for synthesis. Global function rendering is to draw functions such as global handles. For example, system information such as Wi-Fi can be drawn on this layer, or rendered by other methods, and the screen can be synthesized on this layer. Motion prediction and anti-distortion are performed to obtain the rendered image, and finally motion prediction and anti-distortion processing are performed, and then it is displayed on the screen.
[0004] Therefore, the rendering mode in the prior art requires the core rendering process to perform multiple steps, and the core rendering process takes on more and more functions, resulting in greater rendering pressure on the core rendering process. However, the CPU, GPU and other computing power on the VR all-in-one machine are limited. When running high-load games or multiple applications at the same time, it is very easy to have insufficient resources for the core rendering process, resulting in motion prediction failure and jitter problems. In severe cases, it will cause black screens, black edge tearing, and even device restarts, resulting in poor rendering effects. Summary of the invention
[0005] The present application provides a display screen rendering method and an XR device for implementing dual-threaded rendering of display screens, reducing the pressure on the first rendering thread, reducing the resource consumption of the first rendering process, avoiding problems such as insufficient resources of the first rendering process and failure of motion prediction leading to jitter, and improving the rendering effect.
[0006] In a first aspect, an embodiment of the present application provides a method for rendering a display screen, which is applied to an XR device, wherein the XR device includes a first rendering process and a second rendering process, and the method includes:
[0007] When it is determined that the current display screen of the XR device needs to be updated, sending a start rendering instruction to the second rendering process through the first rendering process;
[0008] Determining, by the second rendering process based on the start rendering instruction, a rendering category corresponding to the start rendering instruction; and
[0009] The second rendering process updates and renders the current frame display image using the rendering method corresponding to the rendering category to obtain a rendered current frame display image, and sends the rendered current frame display image to the first rendering process; wherein the current frame display image is obtained based on the start rendering instruction;
[0010] A target operation is performed on the current frame display image through the first rendering process to obtain a target display picture, and the target display picture is displayed, wherein the target operation includes motion prediction and anti-distortion.
[0011] A second aspect of the present application provides an XR device, including a processor and a memory, wherein the processor and the memory are connected via a bus;
[0012] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:
[0013] When it is determined that the current display screen of the XR device needs to be updated, sending a start rendering instruction to the second rendering process through the first rendering process;
[0014] Determining, by the second rendering process based on the start rendering instruction, a rendering category corresponding to the start rendering instruction; and
[0015] The second rendering process updates and renders the current frame display image using the rendering method corresponding to the rendering category to obtain a rendered current frame display image, and sends the rendered current frame display image to the first rendering process; wherein the current frame display image is obtained based on the start rendering instruction;
[0016] A target operation is performed on the current frame display image through the first rendering process to obtain a target display picture, and the target display picture is displayed, wherein the target operation includes motion prediction and anti-distortion.
[0017] According to a third aspect provided by an embodiment of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a computer program, and the computer program is used to execute the method as described in the first aspect.
[0018] In the above-mentioned embodiment of the present application, after determining that the current display screen of the XR device needs to be updated, the first rendering process sends a start rendering instruction to the second rendering process; and the second rendering process determines the rendering category corresponding to the start rendering instruction based on the start rendering instruction; and the second rendering process updates and renders the current frame display image using the rendering method corresponding to the rendering category to obtain the rendered current frame display image, and sends the rendered current frame display image to the first rendering process; wherein the current frame display image is obtained based on the start rendering instruction; finally, the first rendering process performs motion prediction and anti-distortion on the current frame display image to obtain the target display screen, and displays the target display screen. Therefore, in the embodiment of the present application, dual rendering threads are used to render the display screen, which reduces the pressure on the first rendering thread, reduces the resource consumption of the first rendering process, avoids the problem of insufficient resources of the first rendering process causing motion prediction failure and thus causing jitter, and improves the rendering effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0020] Figure 1 A schematic diagram of a rendering method in the prior art provided by an embodiment of the present application is exemplarily shown;
[0021] Figure 2 One of the flowcharts of the method for rendering a display screen provided in an embodiment of the present application is exemplarily shown;
[0022] Figure 3 The schematic diagram of dual-thread rendering provided by the embodiment of the present application is exemplarily shown;
[0023] Figure 4 The following is a schematic diagram of a process for setting the current rendering frame rate provided in an embodiment of the present application;
[0024] Figure 5 A schematic diagram of the frame rate corresponding to the GPU occupancy rate provided in an embodiment of the present application is exemplarily shown;
[0025] Figure 6 The schematic diagram of determining the resolution of a rendered image provided by an embodiment of the present application is exemplarily shown;
[0026] Figure 7The following is a schematic diagram showing the process of function tailoring provided by an embodiment of the present application;
[0027] Figure 8 The second flowchart of the method for rendering a display screen provided in an embodiment of the present application is exemplarily shown;
[0028] Fig. 9 A schematic diagram of a rendering device for a display screen provided in an embodiment of the present application is exemplarily shown;
[0029] Fig.10 A schematic diagram of an XR device provided in an embodiment of the present application is exemplarily shown. DETAILED DESCRIPTION
[0030] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.
[0031] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.
[0032] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.
[0033] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a series of components is not necessarily limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.
[0034] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.
[0035] Before introducing the synchronous driving method of the VR virtual character in the embodiment of the present application, the terms in the embodiment of the present application are explained first:
[0036] Inverse Kinematics (IK): is an algorithm that can solve the joint angles of an object based on its end position and orientation, thereby achieving motion control. For example, it can determine the position of a child node first, and then inversely deduce the position of its n-level parent node on the node chain, thereby determining the method of the entire node chain.
[0037] The idea of the embodiments of the present application is summarized below.
[0038] The rendering mode in the existing technology requires the core rendering process to perform multiple steps. The core rendering process takes on more and more functions, resulting in greater rendering pressure on the core rendering process. However, the CPU, GPU and other computing power on the VR all-in-one machine are limited. When running high-load games or multiple applications at the same time, it is very easy to have insufficient resources for the core rendering process, resulting in motion prediction failure and jitter problems. In severe cases, it will cause black screens, black edge tearing, and even device restarts, resulting in poor rendering effects.
[0039] Based on the existing core rendering process in the prior art, the rendering pressure is relatively high, and it is very easy for the core rendering process to be insufficient in resources, resulting in motion prediction failure and then jitter. In severe cases, it will cause black screen, black edge tearing and even device restart, resulting in poor rendering effect. The present application proposes a rendering method for a display screen, which sends a start rendering instruction to the second rendering process through the first rendering process after determining that the current display screen of the XR device needs to be updated; and the second rendering process determines the rendering category corresponding to the start rendering instruction based on the start rendering instruction; and the second rendering process updates and renders the current frame display image using the rendering method corresponding to the rendering category to obtain the rendered current frame display image, and sends the rendered current frame display image to the first rendering process; wherein the current frame display image is obtained based on the start rendering instruction; finally, the first rendering process performs motion prediction and anti-distortion on the current frame display image to obtain the target display screen, and displays the target display screen. Therefore, in the embodiment of the present application, dual rendering threads are used to render the display screen, which reduces the pressure on the first rendering thread, reduces the resource consumption of the first rendering process, avoids problems such as insufficient resources of the first rendering process and failure of motion prediction, which in turn causes jitter, and improves the rendering effect.
[0040] The following is an introduction to the rendering method of the display screen in the embodiment of the present application in conjunction with the accompanying drawings. Figure 2 FIG. 1 is a flow chart of a method for rendering a display screen, which may specifically include the following steps:
[0041] Step 201: When it is determined that the current display screen of the XR device needs to be updated, a start rendering instruction is sent to the second rendering process through the first rendering process;
[0042] In the embodiment of the present application, before executing step 201, the first rendering process is initialized with the image rendering resolution and frame rate set by the current system. When the XR application is started, it first establishes a connection with the first rendering process and initializes based on the rendering requirements of the first rendering process. After the second rendering process is started, it establishes a connection with the first rendering process. The first rendering process sends rendering parameters to the second rendering process. The second rendering process sets its own image rendering resolution according to the initial image rendering resolution in the rendering parameters, and sets its own current frame rate according to the frame rate in the rendering parameters.
[0043] In the embodiment of the present application, after receiving the instruction sent by the system in the XR device, it is determined that the current display screen of the XR device needs to be updated.
[0044] Step 202: determining, by the second rendering process based on the start rendering instruction, a rendering category corresponding to the start rendering instruction;
[0045] The start rendering instruction in the embodiment of the present application includes a rendering category. The rendering category in the embodiment of the present application includes VST rendering, application multi-layer synthesis, and global function rendering described in the background technology.
[0046] Step 203: updating and rendering the current frame display image by the second rendering process using the rendering method corresponding to the rendering category to obtain a rendered current frame display image; wherein the current frame display image is obtained based on the start rendering instruction;
[0047] The rendering methods corresponding to the rendering categories in the embodiments of the present application are rendering methods in the prior art, and the embodiments of the present application do not limit the rendering methods.
[0048] In order to ensure the rendering effect, before executing step 203, in one embodiment, the rendering frame rate of the first rendering process is compared with the rendering frame rate of the second rendering process. If it is determined that the rendering frame rate of the first rendering of the vehicle is the same as the rendering frame rate of the second rendering process, the step of updating and rendering the current frame display image by the second rendering process using the rendering method corresponding to the rendering category to obtain the rendered current frame display image is executed; if the rendering frame rate of the first rendering process is less than the rendering frame rate of the second rendering process, it is determined based on the rendering frame rate of the second rendering process whether to execute the step of updating and rendering the current frame display image by the second rendering process using the rendering method corresponding to the rendering category to obtain the rendered current frame display image; if not, the rendered previous frame display image is determined as the rendered current frame display image.
[0049] For example, if the rendering frame rate of the first rendering process is 20 frames per second, and the rendering frame rate of the second rendering process is 10 frames per second, if the rendered image is 1-20 frames, the second rendering process will not update the rendering of the 2nd frame, the 4th frame, the 6th frame, the 8th frame, the 10th frame, the 12th frame, the 14th frame, the 16th frame, the 18th frame, and the 20th frame, and the rendered display image of the 1st frame will be determined as the rendered display image of the 2nd frame. The rendered display image of the 3rd frame will be determined as the rendered display image of the 4th frame. The rendered display image of the 5th frame will be determined as the rendered display image of the 6th frame... and so on.
[0050] Step 204: Sending the rendered current frame display image to the first rendering process;
[0051] Step 205: Perform a target operation on the current frame display image through the first rendering process to obtain a target display picture, and display the target display picture, wherein the target operation includes motion prediction and anti-distortion.
[0052] The motion prediction and anti-distortion in the embodiment of the present application are processing methods in the prior art, and will not be described in detail in the embodiment of the present application.
[0053] In the embodiment of the present application, if the application also renders a corresponding application image, the application image is synthesized with the current frame display image to obtain a synthesized image, and the synthesized image is determined as the current frame display image.
[0054] like Figure 3 As shown, it is a schematic diagram of dual-thread rendering. Figure 3As can be seen in the figure, the first rendering process is used to control the rendering process of the application and the second rendering process, and receives and processes the rendered images of both. The second rendering process is responsible for other rendering functions other than the first rendering process, including VST rendering, application multi-layer synthesis, and global function rendering.
[0055] In order to reduce the utilization of system resources, such as Figure 4 The figure is a schematic diagram of the process of setting the current rendering frame rate, which may specifically include the following steps:
[0056] Step 401: obtaining the current occupancy rate of system resources through the second rendering process at specified intervals, wherein the system resources include at least one of a CPU, a GPU, and a memory;
[0057] It should be noted that the specified duration in the embodiment of the present application can be set according to the specific actual situation, and the embodiment of the present application does not limit the specific value of the specified duration.
[0058] Step 402: Determine a target rendering frame rate of the second rendering process based on the current occupancy rate of the system resources by using the second rendering process;
[0059] In one embodiment, step 402 can be specifically implemented in the following three ways:
[0060] Method 1: If it is determined through the second rendering process that the current occupancy rate of the system resources meets the first specified condition, the target rendering frame rate is determined to be a first preset frame rate, wherein the first specified condition is used to indicate the condition when the system resources are sufficient.
[0061] Method 2: If it is determined through the second rendering process that the current occupancy rate of the system resources meets the second specified condition, the target rendering frame rate is determined to be a second preset frame rate, wherein the first preset frame rate is greater than the second preset frame rate, and the second specified condition is used to indicate the condition when the system resources are insufficient.
[0062] In the embodiment of the present application, the system resource is a GPU as an example for explanation, and the first specified condition in the embodiment of the present application may be that the current occupancy rate of the GPU is less than a first preset value. The second specified condition in the embodiment of the present application may be that the current occupancy rate of the GPU is greater than a second preset value, and the second preset value is greater than the first preset value.
[0063] It should be noted that the foregoing is only for illustration and does not limit the system resources, the first specified conditions and the second specified conditions in the embodiments of the present application. The system resources, the first specified conditions and the second specified conditions in the embodiments of the present application can be set according to specific actual conditions.
[0064] Method three: If it is determined through the second rendering process that the current occupancy rate of the system resources does not meet the first specified condition and does not meet the second specified condition, then the target rendering frame rate is determined to be a third preset frame rate, wherein the third preset frame rate is greater than the second preset frame rate and less than the first preset frame rate.
[0065] In the embodiment of the present application, the first preset frame rate is 90 Hz, the second preset frame rate is 45 Hz, and the third preset frame rate is 60 Hz. However, the specific values of the first preset frame rate, the second preset frame rate, and the third preset frame rate in the embodiment of the present application are not limited, and the specific values of the first preset frame rate, the second preset frame rate, and the third preset frame rate in the embodiment of the present application can be set according to actual conditions.
[0066] like Figure 5 As shown, Figure 5 This article takes GPU as an example to explain the system resources. Figure 5 It can be seen that when the GPU occupancy rate is less than A, the target rendering frame rate is 90hz. When the GPU occupancy rate is greater than B, the target rendering frame rate is 45hz. When the GPU occupancy rate is greater than A and less than B, the target rendering frame rate is 60hz.
[0067] Step 403: Setting the current rendering frame rate of the second rendering process as the target rendering frame rate through the second rendering process.
[0068] In order to ensure the quality of the rendered display picture without affecting the stability of the system, in one embodiment, after executing step 403, the resolution of the rendered image needs to be adjusted. Figure 6 The figure is a schematic diagram of a process for determining the resolution of a rendered image, which may specifically include the following steps:
[0069] Step 601: determining, through the second rendering process, whether the current rendering frame rate of the second rendering process is the second preset frame rate, if so, executing step 602, if not, executing step 604;
[0070] Step 602: Determine whether the current occupancy rate of the system resources meets the second specified condition, if yes, execute step 603, if no, execute step 604;
[0071] The second specified condition is used to indicate a condition when the system resources are insufficient.
[0072] Step 603: setting the rendering image resolution of the second rendering process to the first specified resolution;
[0073] In order to avoid insufficient system resources, such as Figure 7The figure is a schematic diagram of the process of function tailoring, which may specifically include the following steps:
[0074] Step 701: determining, through the second rendering process, whether the current occupancy rate of the system resources meets the second specified condition; if so, executing step 702; if not, ending;
[0075] Step 702: determining a first target rendering module based on the priorities of the rendering modules started in the second rendering process, and after shutting down the first target rendering module, returning to step 701;
[0076] In one embodiment, step 702 may be specifically implemented as: determining a rendering module with the lowest priority among the activated rendering modules as the first target rendering module.
[0077] Wherein, each rendering module is used to execute a module of each rendering mode. As mentioned above, the rendering modes in the embodiment of the present application include VST rendering mode, application multi-layer synthesis mode and global function rendering mode. Therefore, the rendering modules in the embodiment of the present application include VST rendering module, application multi-layer synthesis module and global function rendering module.
[0078] For example, if the priorities of the rendering modules are: VST rendering mode > application multi-layer synthesis mode > global function rendering mode, then the first target rendering module is the global function rendering mode.
[0079] Step 703: determining whether the system resources meet the first specified condition through the second rendering process, if yes, executing step 704, if no, ending;
[0080] Step 704: Determine a second target rendering module based on the priorities of the closed rendering modules through the second rendering process, start the second target rendering module, and then return to step 703.
[0081] In one embodiment, step 704 may be specifically implemented as: determining a rendering module with the highest priority among the closed rendering modules as the second target rendering module.
[0082] Step 604: Setting the rendering image resolution of the second rendering process to a second specified resolution.
[0083] The second specified resolution is greater than the first specified resolution.
[0084] In order to further understand the rendering method of the display screen in the embodiment of the present application, as shown in Figure 8 FIG. 1 is a flow chart of a method for rendering a display screen, which may specifically include the following steps:
[0085] Step 801: When it is determined that the current display screen of the XR device needs to be updated, a start rendering instruction is sent to the second rendering process through the first rendering process;
[0086] Step 802: determining, by the second rendering process based on the start rendering instruction, a rendering category corresponding to the start rendering instruction;
[0087] Step 803: Determine a target rendering frame rate of the second rendering process based on the current occupancy rate of the system resources by using the second rendering process;
[0088] Step 804: determining, through the second rendering process, whether the current rendering frame rate of the second rendering process is the second preset frame rate, if so, executing step 805, if not, executing step 807;
[0089] Step 805: determining, through the second rendering process, whether the current occupancy rate of the system resources meets the second specified condition; if so, executing step 806; if not, executing step 807;
[0090] Step 806: setting the rendering image resolution of the second rendering process to the first specified resolution;
[0091] Step 807: setting the rendering image resolution of the second rendering process to a second specified resolution;
[0092] Step 808: determining, through the second rendering process, whether the current occupancy rate of the system resources meets the second specified condition; if so, executing step 809; if not, ending;
[0093] Step 809: determining a first target rendering module based on the priorities of the rendering modules started in the second rendering process, and after shutting down the first target rendering module, returning to step 808;
[0094] Step 810: determining whether the system resources meet the first specified condition through the second rendering process, if yes, executing step 811, if no, ending;
[0095] Step 811: determining a second target rendering module based on the priorities of the closed rendering modules through the second rendering process, and starting the second target rendering module, and then returning to step 810;
[0096] Step 812: updating and rendering the current frame display image by the second rendering process using the rendering method corresponding to the rendering category to obtain a rendered current frame display image;
[0097] Step 813: Sending the rendered current frame display image to the first rendering process through the second rendering process; wherein the current frame display image is obtained based on the start rendering instruction;
[0098] Step 814: Perform a target operation on the current frame display image through the first rendering process to obtain a target display picture, and display the target display picture, wherein the target operation includes motion prediction and anti-distortion.
[0099] Based on the same inventive concept, the display screen rendering method disclosed above can also be implemented by a display screen rendering device. The effect of the display screen rendering device is similar to that of the above method, which will not be described in detail here.
[0100] Fig. 9 The figure is a schematic diagram of the structure of a rendering device for displaying a picture according to an embodiment of the present disclosure.
[0101] like Fig. 9 As shown, the rendering device 900 for displaying a screen of the present disclosure may include an instruction sending module 910 , a rendering category determining module 920 , a second rendering process rendering module 930 , and a first rendering process rendering module 940 .
[0102] An instruction sending module 910 is used to send a start rendering instruction to the second rendering process through the first rendering process when it is determined that the current display screen of the XR device needs to be updated;
[0103] A rendering category determination module 920, configured to determine, through the second rendering process and based on the start rendering instruction, a rendering category corresponding to the start rendering instruction; and
[0104] The second rendering process rendering module 930 is used to update and render the current frame display image by the second rendering process using the rendering method corresponding to the rendering category to obtain the rendered current frame display image, and send the rendered current frame display image to the first rendering process; wherein the current frame display image is obtained based on the start rendering instruction;
[0105] The first rendering process rendering module 940 is used to perform a target operation on the current frame display image through the first rendering process to obtain a target display picture and display the target display picture, wherein the target operation includes motion prediction and anti-distortion.
[0106] In one embodiment, the apparatus further comprises:
[0107] A rendering frame rate determination module 950, configured to obtain, through the second rendering process, a current occupancy rate of system resources at specified intervals, wherein the system resources include at least one of a CPU, a GPU, and a memory;
[0108] Determining a target rendering frame rate of the second rendering process based on a current occupancy rate of the system resources by using the second rendering process;
[0109] The current rendering frame rate of the second rendering process is set as the target rendering frame rate through the second rendering process.
[0110] In one embodiment, the rendering frame rate determination module 950 performs the step of determining the target rendering frame rate of the second rendering process based on the current occupancy rate of the system resources by using the second rendering process, specifically for:
[0111] If it is determined through the second rendering process that the current occupancy rate of the system resources meets a first specified condition, then determining the target rendering frame rate to be a first preset frame rate, wherein the first specified condition is used to indicate a condition when the system resources are sufficient; or,
[0112] If it is determined through the second rendering process that the current occupancy rate of the system resources meets the second specified condition, then the target rendering frame rate is determined to be a second preset frame rate, wherein the first preset frame rate is greater than the second preset frame rate, and the second specified condition is used to indicate a condition when the system resources are insufficient; or,
[0113] If it is determined through the second rendering process that the current occupancy rate of the system resources does not meet the first specified condition and does not meet the second specified condition, the target rendering frame rate is determined to be a third preset frame rate, wherein the third preset frame rate is greater than the second preset frame rate and less than the first preset frame rate.
[0114] In one embodiment, the apparatus further comprises:
[0115] A resolution setting module 960 is used for, after setting the current rendering frame rate of the second rendering process to the target rendering frame rate through the second rendering process, if it is determined through the second rendering process that the current rendering frame rate of the second rendering process is the second preset frame rate, and the current occupancy rate of the system resources meets the second specified condition, then setting the rendering image resolution of the second rendering process to the first specified resolution, wherein the second specified condition is used to indicate a condition when the system resources are insufficient; or,
[0116] If it is determined through the second rendering process that the current rendering frame rate of the second rendering process is the second preset frame rate, and the current occupancy rate of the system resources does not meet the second specified condition, then the rendering image resolution of the second rendering process is set to a second specified resolution; wherein the second specified resolution is greater than the first specified resolution; or,
[0117] If it is determined through the second rendering process that the current rendering frame rate of the second rendering process is not the second preset frame rate, the rendering image resolution of the second rendering process is set to the second specified resolution.
[0118] In one embodiment, the apparatus further comprises:
[0119] A closing module 970 is used for determining, through the second rendering process, whether the current occupancy rate of the system resources meets the second specified condition after the rendering image resolution of the second rendering process is set to the first specified resolution;
[0120] If so, based on the priority of each rendering module that has been started in the second rendering process, determine the first target rendering module, and after closing the first target rendering module, return to the step of determining whether the current occupancy rate of the system resources meets the second specified condition through the second rendering process until the current occupancy rate of the system resources does not meet the second specified condition, then end, wherein each rendering module is used to execute a module of each rendering mode respectively.
[0121] In one embodiment, the apparatus further comprises:
[0122] A starting module 980, configured to determine whether the system resources meet a first specified condition through the second rendering process after the target rendering module is closed;
[0123] If so, the second rendering process determines the second target rendering module based on the priorities of the closed rendering modules, and after starting the second target rendering module, returns to the step of determining whether the system resources meet the first specified condition through the second rendering process, until it is determined through the second rendering process that the system resources do not meet the first specified condition, then the process ends.
[0124] After introducing a method and apparatus for rendering a display screen according to an exemplary embodiment of the present invention, an XR device according to another exemplary embodiment of the present invention is introduced next.
[0125] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".
[0126] In some possible implementations, the XR device according to the present invention may include at least one processor and at least one computer storage medium. The computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps of the method for rendering a display screen according to various exemplary embodiments of the present invention described above in this specification. For example, the processor may execute the following steps: Figure 2 Steps 201-204 shown in .
[0127] Refer to the following Fig.10 hereinafter, an XR device 1000 according to this embodiment of the present invention is described. Fig.10 The XR device 1000 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0128] like Fig.10 As shown, the XR device 1000 is presented in the form of a general AR device. The components of the XR device 1000 may include, but are not limited to: the at least one processor 1001, the at least one computer storage medium 1002, and a bus 1003 connecting different system components (including the computer storage medium 1002 and the processor 1001).
[0129] Bus 1003 represents one or more of several types of bus structures, including a computer storage media bus or computer storage media controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.
[0130] The computer storage medium 1002 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM) 1021 and / or cache storage media 1022 , and may further include read-only computer storage media (ROM) 1023 .
[0131] The computer storage medium 1002 may also include a program / utility 1025 having a set (at least one) of program modules 1024, such program modules 1024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0132] The XR device 1000 may also communicate with one or more external devices 1004 (e.g., keyboards, pointing devices, etc.), one or more devices that enable a user to interact with the XR device 1000, and / or any device that enables the XR device 1000 to communicate with one or more other AR devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1005. In addition, the XR device 1000 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 1006. As shown, the network adapter 1006 communicates with other modules for the XR device 1000 via a bus 1003. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the XR device 1000, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0133] In some possible implementations, various aspects of a method for rendering a display screen provided by the present invention may also be implemented in the form of a program product, which includes a program code. When the program product runs on a computer device, the program code is used to enable the computer device to execute the steps of the method for rendering a display screen according to various exemplary embodiments of the present invention described above in this specification.
[0134] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for rendering a display screen, characterized in that: Applied in an XR device, the XR device includes a first rendering process and a second rendering process, and the method includes: When it is determined that the current display screen of the XR device needs to be updated, sending a start rendering instruction to the second rendering process through the first rendering process; Determining, by the second rendering process based on the start rendering instruction, a rendering category corresponding to the start rendering instruction; and The second rendering process updates and renders the current frame display image using the rendering method corresponding to the rendering category to obtain a rendered current frame display image, and sends the rendered current frame display image to the first rendering process; wherein the current frame display image is obtained based on the start rendering instruction; A target operation is performed on the current frame display image through the first rendering process to obtain a target display picture, and the target display picture is displayed, wherein the target operation includes motion prediction and anti-distortion.
2. The method according to claim 1, characterized in that The method further comprises: At specified intervals, obtaining, through the second rendering process, a current occupancy rate of system resources, wherein the system resources include at least one of a CPU, a GPU, and a memory; Determining a target rendering frame rate of the second rendering process based on a current occupancy rate of the system resources by using the second rendering process; The current rendering frame rate of the second rendering process is set as the target rendering frame rate through the second rendering process.
3. The method according to claim 2, characterized in that The determining, by using the second rendering process based on the current occupancy rate of the system resources, a target rendering frame rate of the second rendering process includes: If it is determined through the second rendering process that the current occupancy rate of the system resources meets a first specified condition, then determining the target rendering frame rate to be a first preset frame rate, wherein the first specified condition is used to indicate a condition when the system resources are sufficient; or, If it is determined through the second rendering process that the current occupancy rate of the system resources meets the second specified condition, then the target rendering frame rate is determined to be a second preset frame rate, wherein the first preset frame rate is greater than the second preset frame rate, and the second specified condition is used to indicate a condition when the system resources are insufficient; or, If it is determined through the second rendering process that the current occupancy rate of the system resources does not meet the first specified condition and does not meet the second specified condition, the target rendering frame rate is determined to be a third preset frame rate, wherein the third preset frame rate is greater than the second preset frame rate and less than the first preset frame rate.
4. The method according to claim 2, characterized in that: After setting the current rendering frame rate of the second rendering process to the target rendering frame rate through the second rendering process, the method further includes: If it is determined through the second rendering process that the current rendering frame rate of the second rendering process is the second preset frame rate, and the current occupancy rate of the system resources meets the second specified condition, then the rendering image resolution of the second rendering process is set to the first specified resolution, wherein the second specified condition is used to indicate a condition when the system resources are insufficient; or, If it is determined through the second rendering process that the current rendering frame rate of the second rendering process is the second preset frame rate, and the current occupancy rate of the system resources does not meet the second specified condition, then the rendering image resolution of the second rendering process is set to a second specified resolution; wherein the second specified resolution is greater than the first specified resolution; or, If it is determined through the second rendering process that the current rendering frame rate of the second rendering process is not the second preset frame rate, the rendering image resolution of the second rendering process is set to the second specified resolution.
5. The method according to claim 4, characterized in that After setting the rendering image resolution of the second rendering process to the first specified resolution, the method further includes: Determining, by the second rendering process, whether a current occupancy rate of the system resources meets the second specified condition; If so, based on the priority of each rendering module that has been started in the second rendering process, determine the first target rendering module, and after closing the first target rendering module, return to the step of determining whether the current occupancy rate of the system resources meets the second specified condition through the second rendering process until the current occupancy rate of the system resources does not meet the second specified condition, then end, wherein each rendering module is used to execute a module of each rendering mode respectively.
6. The method according to claim 5, characterized in that After closing the target rendering module, the method further includes: Determining, by the second rendering process, whether the system resources meet a first specified condition; If so, the second rendering process determines the second target rendering module based on the priorities of the closed rendering modules, and after starting the second target rendering module, returns to the step of determining whether the system resources meet the first specified condition through the second rendering process, until it is determined through the second rendering process that the system resources do not meet the first specified condition, then the process ends.
7. An XR device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: When it is determined that the current display screen of the XR device needs to be updated, sending a start rendering instruction to the second rendering process through the first rendering process; Determining, by the second rendering process based on the start rendering instruction, a rendering category corresponding to the start rendering instruction; as well as, The second rendering process updates and renders the current frame display image using the rendering method corresponding to the rendering category to obtain a rendered current frame display image, and sends the rendered current frame display image to the first rendering process; wherein the current frame display image is obtained based on the start rendering instruction; A target operation is performed on the current frame display image through the first rendering process to obtain a target display picture, and the target display picture is displayed, wherein the target operation includes motion prediction and anti-distortion.
8. The XR device according to claim 7, characterized in that The processor is further configured to: At specified intervals, obtaining, through the second rendering process, a current occupancy rate of system resources, wherein the system resources include at least one of a CPU, a GPU, and a memory; Determining a target rendering frame rate of the second rendering process based on a current occupancy rate of the system resources by using the second rendering process; The current rendering frame rate of the second rendering process is set as the target rendering frame rate through the second rendering process.
9. The XR device according to claim 8, characterized in that The processor performs the step of determining the target rendering frame rate of the second rendering process based on the current occupancy rate of the system resources by using the second rendering process, and is specifically configured as follows: If it is determined through the second rendering process that the current occupancy rate of the system resources meets a first specified condition, then determining the target rendering frame rate to be a first preset frame rate, wherein the first specified condition is used to indicate a condition when the system resources are sufficient; or, If it is determined through the second rendering process that the current occupancy rate of the system resources meets the second specified condition, then the target rendering frame rate is determined to be a second preset frame rate, wherein the first preset frame rate is greater than the second preset frame rate, and the second specified condition is used to indicate a condition when the system resources are insufficient; or, If it is determined through the second rendering process that the current occupancy rate of the system resources does not meet the first specified condition and does not meet the second specified condition, the target rendering frame rate is determined to be a third preset frame rate, wherein the third preset frame rate is greater than the second preset frame rate and less than the first preset frame rate.
10. The XR device according to claim 8, characterized in that The processor is further configured to: After setting the current rendering frame rate of the second rendering process to the target rendering frame rate through the second rendering process, if it is determined through the second rendering process that the current rendering frame rate of the second rendering process is the second preset frame rate, and the current occupancy rate of the system resources meets the second specified condition, then setting the rendering image resolution of the second rendering process to the first specified resolution, wherein the second specified condition is used to indicate a condition when the system resources are insufficient; or If it is determined through the second rendering process that the current rendering frame rate of the second rendering process is the second preset frame rate, and the current occupancy rate of the system resources does not meet the second specified condition, then the rendering image resolution of the second rendering process is set to a second specified resolution; wherein the second specified resolution is greater than the first specified resolution; or, If it is determined through the second rendering process that the current rendering frame rate of the second rendering process is not the second preset frame rate, the rendering image resolution of the second rendering process is set to the second specified resolution.