Rendering processing method and rendering device

Abstract

The application discloses a rendering processing method and a rendering device, relates to the technical field of image processing, and reduces terminal device heating and saves power consumption; the rendering processing method first intercepts first rendering instructions for rendering a to-be-rendered image of a target application sent by the target application; then acquires second rendering instructions for rendering a previous frame image of the to-be-rendered image; then determines field of view transformation information of the to-be-rendered image relative to the previous frame image according to the first rendering instructions and the second rendering instructions; adjusts original resolution parameters of at least one rendering channel in the first rendering instructions to lower preset resolution parameters in the case that the field of view transformation information meets a preset dynamic scene; and finally sends the adjusted first rendering instructions to render the to-be-rendered image.

Description

[0001] This application relates to the field of image processing technology, and in particular to rendering processing methods and rendering devices. Background Technology

[0002] As users demand increasingly higher visual effects from applications, the image resolution of applications is also increasing. The image resolution of an application depends on how the images are rendered. Specifically, the process of rendering a scene typically involves the central processing unit (CPU) of the terminal device sending rendering data to the graphics processing unit (GPU), which then renders the image based on that data. Since a dynamic scene generally consists of a large number of sequential frames, the GPU needs to render each frame of the dynamic scene, which is very time-consuming. Furthermore, the higher the image resolution of the application, the more detailed the rendering of each frame needs to be by the GPU, meaning the GPU needs to handle a huge amount of computation.

[0003] In summary, when rendering images for high-quality applications, the GPU load is high, causing the terminal device to heat up and consume power quickly. Summary of the Invention

[0004] This application provides a rendering processing method and rendering device that reduces heat generation in terminal devices and saves power consumption.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, a rendering processing method is provided. The method first intercepts a first rendering instruction sent by a target application for rendering an image to be rendered by the target application; then obtains a second rendering instruction for rendering the previous frame image of the image to be rendered; then determines the field of view transformation information of the image to be rendered relative to the previous frame image based on the first rendering instruction and the second rendering instruction; and when the field of view transformation information conforms to a preset dynamic scene, adjusts the original resolution parameter of at least one rendering channel in the first rendering instruction to a lower preset resolution parameter; and finally sends the adjusted first rendering instruction to render the image to be rendered.

[0007] In the first aspect, the first rendering instruction sent by the target application, indicating a higher resolution, is intercepted. When the field of view change information matches the preset dynamic scene, the original resolution parameters of some rendering channels in the first rendering instruction are lowered before being sent to the graphics processor for rendering. This reduces the amount of rendering computation while relatively ensuring the rendering quality, thereby improving the rendering efficiency to increase the frame rate of the target application and reducing the heat generation and power consumption of the terminal device.

[0008] In one possible implementation, the method further includes: generating a third rendering instruction when the view transformation information conforms to a preset static scene, the third rendering instruction including a first sub-instruction and a second sub-instruction, specifically, the first sub-instruction is used to indicate that the rendering result of at least one rendering channel of the previous frame image is used as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels in the first rendering instruction other than at least one rendering channel; and sending the third rendering instruction for rendering the image to be rendered.

[0009] In this implementation, when the field of view change information conforms to the preset static scene, a third rendering instruction is generated that indicates the reuse of the rendering result of a portion of the rendering channel of the previous frame image as the rendering result of the image to be rendered, thereby reducing the required amount of rendering computation.

[0010] In one possible implementation, adjusting the original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter includes: determining multiple rendering channels in the first rendering instruction, and specification parameters of the multiple rendering channels; the specification parameters include at least one of the following: rendering channel size, frame buffer attachment, and number of drawing call interfaces; adjusting the original resolution parameter of the target rendering channel among the multiple rendering channels to a preset resolution parameter, wherein the specification parameters of the target rendering channel meet the preset specification conditions.

[0011] In this implementation, only the original resolution parameters of the target rendering channel are adjusted to the preset resolution parameters. This reduces the amount of rendering computation while ensuring rendering quality, thereby improving rendering efficiency, increasing the frame rate of the target application, and reducing the heat generation and power consumption of the terminal device.

[0012] In one possible implementation, different sizes of field-of-view transformation information correspond to different preset dynamic scenes; the preset resolution parameters are different under different preset dynamic scenes, or the preset resolution parameters are the same under different preset dynamic scenes.

[0013] In this implementation, preset resolution parameters for different preset dynamic scenes are flexibly defined.

[0014] In one possible implementation, the view transformation information includes displacement transformation information and angle transformation information; the first rendering instruction includes a first view matrix and a first coordinate, and the second rendering instruction includes a second view matrix and a second coordinate; determining the view transformation information of the image to be rendered relative to the previous frame image based on the first rendering instruction and the second rendering instruction includes: calculating the difference between the first coordinate and the second coordinate, and outputting the displacement transformation information; calculating the first plane normal vector based on the first view matrix, and calculating the second plane normal vector based on the second view matrix; calculating the angle between the first plane normal vector and the second plane normal vector, and outputting the angle transformation information.

[0015] In this implementation, the field of view transformation information can be accurately calculated, and then the transformation scene of the camera can be determined as a preset dynamic scene or a preset static scene based on the field of view transformation information, so as to execute different rendering instructions and adjust strategies based on different scenes.

[0016] Secondly, a rendering processing method is provided, the method comprising: receiving a third rendering instruction; wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction being used to indicate that the rendering result of at least one rendering channel of the previous frame of the image to be rendered by the target application is used as a first rendering result of the image to be rendered, the second sub-instruction including the original resolution parameters of the remaining rendering channels in the first rendering instruction other than at least one rendering channel, the first rendering instruction being an instruction sent by the target application for rendering the image to be rendered; rendering the image to be rendered according to the third rendering instruction to obtain a full rendering result of the image to be rendered, wherein the full rendering result includes the first rendering result and a second rendering result, the second rendering result being obtained by rendering the image to be rendered based on the original resolution parameters of the remaining rendering channels.

[0017] In the second aspect, the image to be rendered is rendered based on the third rendering instruction. Since the rendering channel corresponding to the reused rendering result does not need to be rendered again, the amount of rendering computation is reduced, the rendering efficiency is increased, the frame rate of the target application is improved, and the heat generation and power consumption of the terminal device are reduced.

[0018] Thirdly, a rendering processing apparatus is provided, comprising: an interception module for intercepting a first rendering instruction sent by a target application, wherein the first rendering instruction is used to render an image to be rendered by the target application; an acquisition module for acquiring a second rendering instruction, wherein the second rendering instruction is used to render the previous frame image of the image to be rendered; a determination module for determining view transformation information of the image to be rendered relative to the previous frame image based on the first rendering instruction and the second rendering instruction; an adjustment module for adjusting the original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter, wherein the preset resolution parameter is lower than the original resolution parameter, provided that the view transformation information conforms to a preset dynamic scene; and a sending module for sending the adjusted first rendering instruction; the adjusted first rendering instruction is used to render the image to be rendered.

[0019] In one possible implementation, the apparatus further includes: a generation module, configured to generate a third rendering instruction when the view transformation information conforms to a preset static scene, wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction being configured to indicate that the rendering result of a portion of the rendering channels of the previous frame image is used as a portion of the rendering result of the image to be rendered, and the second sub-instruction including the original resolution parameters of the remaining rendering channels in the first rendering instruction excluding the portion of the rendering channels; and a sending module, further configured to send the third rendering instruction, the third rendering instruction being used to render the image to be rendered.

[0020] In one possible implementation, the adjustment module is specifically used to: determine multiple rendering channels in the first rendering instruction, and the specification parameters of the multiple rendering channels; adjust the original resolution parameters of the target rendering channel among the multiple rendering channels to preset resolution parameters; wherein, the specification parameters include at least one of the following: rendering channel size, frame buffer attachment, and number of drawing call interfaces, and the specification parameters of the target rendering channel meet the preset specification conditions.

[0021] In one possible implementation, different sizes of field-of-view transformation information correspond to different preset dynamic scenes; the preset resolution parameters are different under different preset dynamic scenes, or the preset resolution parameters are the same under different preset dynamic scenes.

[0022] In one possible implementation, the view transformation information includes displacement transformation information and angle transformation information; the first rendering instruction includes a first view matrix and a first coordinate, and the second rendering instruction includes a second view matrix and a second coordinate; the determining module is specifically used to: calculate the difference between the first coordinate and the second coordinate, and output displacement transformation information; calculate the first plane normal vector based on the first view matrix, and calculate the second plane normal vector based on the second view matrix; calculate the angle between the first plane normal vector and the second plane normal vector, and output angle transformation information.

[0023] Fourthly, a rendering processing apparatus is provided, comprising: a receiving module for receiving a third rendering instruction; wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction indicating that the rendering result of at least one rendering channel of the previous frame of the image to be rendered by the target application is used as a first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels in the first rendering instruction other than at least one rendering channel, the first rendering instruction being an instruction sent by the target application for rendering the image to be rendered; and a rendering module for rendering the image to be rendered according to the third rendering instruction to obtain a full rendering result of the image to be rendered, wherein the full rendering result includes a first rendering result and a second rendering result, the second rendering result being obtained by rendering the image to be rendered based on the original resolution parameters of the remaining rendering channels.

[0024] Fifthly, this application provides a rendering apparatus, which includes a processor and a transceiver. The processor and transceiver are used to support the rendering apparatus in executing the methods of the first aspect or the second aspect. Furthermore, the rendering apparatus may also include a memory storing computer instructions, which the processor can execute to perform the methods of the first aspect or the second aspect.

[0025] Sixthly, this application provides a computer-readable storage medium that stores computer instructions, wherein when the computer instructions are executed, the method of the first or second aspect is performed.

[0026] In a seventh aspect, this application provides a computer program product containing instructions that, when run on a computer, enables the computer to perform the methods described in the first or second aspect.

[0027] Eighthly, this application provides a chip including a processor and a transceiver, the processor and transceiver being used to support a rendering processing apparatus in performing the methods of the first aspect or the second aspect.

[0028] The beneficial effects described in aspects three through six of this application can be referred to the beneficial effect analysis of aspect one or aspect two, and will not be repeated here. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the structure of a rendering device provided in an embodiment of this application;

[0030] Figure 2 This application provides a schematic diagram of the architecture of a rendering system.

[0031] Figure 3 A flowchart illustrating a rendering process provided in an embodiment of this application;

[0032] Figure 4 A schematic flowchart of another rendering processing method provided in an embodiment of this application;

[0033] Figure 5 A schematic flowchart of another rendering processing method provided in an embodiment of this application;

[0034] Figure 6 A schematic diagram of a rendering channel provided in an embodiment of this application;

[0035] Figure 7 A schematic flowchart of another rendering processing method provided in an embodiment of this application;

[0036] Figure 8 This is a schematic diagram of the structure of a rendering processing apparatus provided in an embodiment of this application;

[0037] Figure 9 This is a schematic diagram of another rendering processing apparatus provided in an embodiment of this application. Detailed Implementation

[0038] The network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0039] It should be noted that the terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0040] It should be understood that in the embodiments of this application, "at least one (item)" refers to one or more, "more than one" refers to two or more, "at least two (items)" refers to two or three or more, and "and / or" is used to describe the association relationship of related objects, indicating that there can be three relationships. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple. It should be understood that in the embodiments of this application, "B corresponding to A" means that B is associated with A. For example, B can be determined based on A. It should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0041] Before introducing the embodiments of this application, some terms involved in the embodiments of this application will be explained.

[0042] GPU: Also known as a graphics processing unit, visual processor, or display chip, it is a microprocessor specifically designed for performing image and graphics-related calculations on personal computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones).

[0043] The Model-View-Projection Matrix (MVP Matrix) is a combination of three matrices: the Model matrix, the View matrix, and the Projection matrix. The Model matrix transforms the object from the local coordinate system to the world coordinate system, the View matrix transforms the world coordinate system to the camera coordinate system, and the Projection matrix transforms the camera coordinate system to the clipping coordinate system.

[0044] Draw Call Interface: A draw call refers to the process by which the CPU sends a drawing instruction to the GPU. In order to draw an object, the CPU needs to send Draw Calls to the GPU. The more Draw Calls there are, the greater the load on the CPU and GPU, thus affecting the performance of the terminal device.

[0045] A rendering pass (or simply pass) is a crucial component of the rendering pipeline. Every detail seen in the final rendered frame (also known as the rendered result) is calculated through a series of passes. A pass generates the textures, buffers, and render targets needed for an image. Each image generated by a pass contains scene-specific information, such as color, normals, and depth. This image information is combined to produce more complex effects, such as shadows, lighting, blur, glow, and other post-processing effects. A pass is a logical grouping of rendering tasks with defined input and output slots. It receives input data through the input slot, performs calculations on it, and then produces output data, which is passed to other passes through the output slot.

[0046] In one embodiment, a pass can be a render pass, and the applied rendering instructions include a set of associated render passes.

[0047] Camera: This can be called a virtual camera, and it is very similar to a real-world video camera. The Camera component is responsible for capturing game footage and projecting it onto the screen. The footage captured by the Camera determines the view of the Game panel. When creating a scene, A camera is created by default, so the game screen only appears when the user clicks the Game panel. In game applications, the camera is generally movable; for example, in a first-person game, the camera will continuously move following the user's actions. Specifically, the camera's coordinates can correspond to a point in the world coordinate system. The visible range determined by the camera determines the game's screen area. In various embodiments of this application, the camera is also referred to as the field of view.

[0048] To meet users' ever-increasing demands for higher image quality in applications, games, movies, design, and other applications are increasingly moving towards higher resolutions and higher frame rates. Higher image resolution requires a greater amount of computation from the GPU, leading to increased heat generation and faster power consumption in the device.

[0049] To address the aforementioned technical problems, this application provides a rendering processing method. The method provided in this application embodiment is described below with reference to the accompanying drawings.

[0050] The communication method provided in this application can be applied to rendering devices. For example, the rendering device can be terminal equipment, which can be user equipment (UE), mobile station (MS), or mobile terminal (MT), etc. Specifically, the terminal device can be a mobile phone, tablet computer, or computer with wireless transceiver capabilities, and can also be a virtual reality (VR) terminal, augmented reality (AR) terminal, etc. In this application embodiment, the device used to implement the functions of the terminal device can be a terminal or a device capable of supporting the terminal in implementing the functions, such as a chip system. This device can be installed in the terminal device or used in conjunction with the terminal device.

[0051] For example, this application provides a rendering device 100, the structure of which is as follows: Figure 1 As shown, the rendering device 100 includes: a processor 101, a graphics processor 102, and a memory 103.

[0052] The processor 101 can be a CPU, or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof.

[0053] The processor 101 is used to generate a first rendering instruction according to the rendering request of the application, adjust the first rendering instruction using the rendering processing method of this application embodiment, and then convert the adjusted rendering instruction into hardware code and send it to the graphics processor 102 for execution.

[0054] The graphics processor 102 is used to render the application's image by executing the hardware code corresponding to the rendering instructions.

[0055] The memory 103 may be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, used to store rendering instructions, rendering resources, shader parameters, rendering states, applications, configuration files, data information, or other content that can implement the methods of this application.

[0056] This application embodiment also provides a rendering system that can be deployed in Figure 1 On the rendering device shown. (As shown) Figure 2 As shown, Figure 2 The illustrated rendering system includes an application layer, a software framework layer, a driver layer, and a hardware layer.

[0057] Specifically, the application layer, software framework layer, and driver layer can run on the processor 101 of the rendering device 100, while the hardware layer can run on the graphics processor 102 of the rendering device 100.

[0058] The application layer primarily carries the graphics rendering program, which includes games, graphics applications (APP), user interfaces (UI), cameras, virtual and augmented reality (VR / AR) technologies, etc. The graphics rendering program in the application layer can run on the CPU, receive rendering requests from users, and, based on the rendering requests, call the target application to generate the first rendering instruction, and then send the generated first rendering instruction to the software framework layer.

[0059] The software framework layer primarily hosts the rendering engine, which includes the game engine. Unreal Engine, graphics display system HWUI), game development engine (Cocos), etc., the software framework layer can run on the CPU, receive the first rendering instruction from the application layer, and execute the rendering processing method of the embodiment of this application based on the first rendering instruction to obtain the adjusted first rendering instruction, and improve the adjusted first rendering instruction based on the rendering engine.

[0060] The driver layer mainly carries the graphics driver program. The driver layer can run on the CPU and translate the adjusted first rendering instructions into hardware code and send it to the hardware layer.

[0061] The hardware layer primarily hosts the graphics hardware; for example, the hardware could be a GPU. The hardware layer can run on the GPU, executing the hardware code corresponding to the adjusted first rendering instructions to render the application's image.

[0062] For example, when a user runs a game application on a mobile phone, a scene is rendered based on a certain operation of the user. The game application, which runs on the application layer of the CPU, responds to the user's operation, generates a rendering request and initially generates the first rendering instruction. Then, the game engine, which runs on the software framework layer of the CPU, adjusts the rendering instruction. The driver program, which runs on the driver layer of the CPU, converts the adjusted rendering instruction into hardware code and sends it to the hardware layer. The hardware layer can run the hardware code on the GPU to render the application's image.

[0063] Specifically, the process by which the software framework layer of the rendering system executes the rendering processing method of this application embodiment will be described in detail in the following method description, and will not be repeated here.

[0064] It should be noted that the above Figure 1 Rendering device shown Figure 2 The rendering system shown is only an example to illustrate the application scenario of the solution in this application, and is not intended to limit the application scenario of the solution in this application.

[0065] Based on the rendering device described in the embodiments of this application, the embodiments of this application provide a rendering processing method. Figure 3 A flowchart illustrating the rendering processing method provided in an embodiment of this application is shown. Figure 3 As shown, the method may include the following steps:

[0066] S310, the processor intercepts the first rendering command sent by the target application.

[0067] When processing rendering commands sent by various applications, the processor can determine whether the command belongs to the target application based on application information. If the rendering command is identified as belonging to the target application, it will be intercepted. Target applications are those with high resolution requirements, such as games, VR applications, and AR applications. Due to these high resolution requirements, if the image to be rendered is based on the first rendering command generated by the target application, the resulting rendering result will also have a high resolution, and the rendering process will place a high load on the graphics processor.

[0068] Referring to the description of the application layer in the rendering system provided in this application embodiment, the application layer receives a rendering request from the user and, based on the request, calls the target application to generate a first rendering instruction, which is then sent to the software framework layer. The software framework layer typically refines the first rendering instruction directly. However, in this application embodiment, the first rendering instruction is intercepted, adjusted, and then refined by the rendering engine of the software framework layer. In other words, after intercepting the first rendering instruction, the terminal device cannot perform rendering based on it.

[0069] Installed on terminal devices Taking the operating system as an example, one can utilize The reserved interfaces of the operating system OpenGL and Vulkan intercept the first rendering instruction. Before the software framework layer calls the driver layer, the first rendering instruction is intercepted and cached into an instruction stream.

[0070] S320, the processor receives the second rendering instruction.

[0071] The second rendering instruction is used to render the previous frame of the image to be rendered. In the target application's rendering scenario, images are rendered frame by frame. When rendering the image to be rendered, the previous frame has already been rendered, and the rendering of the previous frame is also triggered by the target application generating a rendering instruction (i.e., the second rendering instruction). The second rendering instruction is generally stored as historical data in a cache, and can be retrieved from the cache.

[0072] S330, the processor determines the field-view transformation information of the image to be rendered relative to the previous frame image based on the first rendering instruction and the second rendering instruction.

[0073] The first rendering instruction and the second rendering instruction record the field of view information of the image to be rendered and the previous frame image, respectively. By comparing these two sets of field of view information, the field of view transformation information of the image to be rendered and the previous frame image can be determined.

[0074] S340, when the field of view change information matches the preset dynamic scene, the processor adjusts the original resolution parameter of at least one rendering channel in the first rendering instruction to the preset resolution parameter.

[0075] Different sizes of view transformation information correspond to different preset dynamic scenes; the preset resolution parameters under different preset dynamic scenes can be the same or different, or they can be the same. A preset dynamic scene can be a type of scene where the view transformation information represents a significant degree of view transformation between the image to be rendered and the previous frame. For example, if the target application is a game application, the preset dynamic scene could be a scene where a character is walking, running, or the user's view is rotating. In this case, the original resolution parameter of at least one rendering channel in the first rendering instruction is lowered (i.e., adjusted to the preset resolution parameter) to reduce the amount of rendering computation.

[0076] S350: The processor sends the adjusted first rendering instruction to the graphics processor, and the graphics processor receives the adjusted first rendering instruction accordingly.

[0077] In this process, the processor sends a modified first rendering instruction to the graphics processor, which then renders the image to be rendered based on this instruction.

[0078] In this embodiment, the first rendering instruction sent by the target application indicating a higher resolution is intercepted. When the field of view change information conforms to the preset dynamic scene, the original resolution parameters of some rendering channels in the first rendering instruction are lowered before being sent to the graphics processor for rendering. This reduces the amount of rendering computation while relatively ensuring the rendering quality, thereby improving the rendering efficiency to increase the frame rate of the target application and reducing the heat generation and power consumption of the terminal device.

[0079] Furthermore, it requires no code adaptation for applications and application engines, is independent of any specific hardware platform, and leverages the contextual transmission characteristic of rendering instructions as data streams to determine the scene changes between consecutive frames. It then adjusts the resolution of the rendering channel accordingly, reducing the load on the target application and increasing its frame rate. This allows the target application to reduce terminal device power consumption and delay the time when the system limits the target application's frequency and frame rate, thereby extending the target application's runtime and improving the user experience, all without the user noticing.

[0080] In one embodiment, such as Figure 4 As shown, the method may further include:

[0081] S410: When the field of view change information matches the preset static scene, the processor generates a third rendering instruction.

[0082] The preset static scene can be a type of scene where the field of view change information represents a small degree of change in the field of view of the image to be rendered relative to the previous frame. For example, if the target application is a game application, the preset static scene can be a scene where the camera is stationary in the game. In this case, reusing the rendering result of some rendering channels of the previous frame image as the rendering result of the image to be rendered can still ensure the quality of the target application image. At this time, a third rendering instruction is generated, which includes a first sub-instruction and a second sub-instruction. The first sub-instruction is used to indicate that the rendering result of at least one rendering channel of the previous frame image is used as the first rendering result of the image to be rendered. The second sub-instruction includes the original resolution parameters of the remaining rendering channels in the first rendering instruction, excluding at least one rendering channel.

[0083] In another example, if the user does not interact with the game character or the game UI, the rendering results of all rendering channels of the previous frame can be reused. In this case, the first sub-instruction indicates that the rendering results of all rendering channels of the previous frame should be used as the first rendering result of the image to be rendered. Therefore, the second sub-instruction does not exist, meaning the third rendering instruction includes the first sub-instruction. In this case, the third rendering instruction simply indicates that the rendering result should be reused, without requiring further rendering.

[0084] In S420, the processor sends a third rendering instruction to the graphics processor. Correspondingly, the graphics processor receives the third rendering instruction.

[0085] S430, the graphics processor renders the image to be rendered according to the third rendering instruction, and obtains the full rendering result of the image to be rendered.

[0086] The graphics processor renders the image to be rendered according to the third rendering instruction in two processes. The first process uses the rendering result of at least one rendering channel of the previous frame of the image to be rendered as the first rendering result of the image to be rendered. The second process uses the remaining rendering channels to render the image to be rendered based on the original resolution parameters to obtain the second rendering result, and finally obtains the full rendering result.

[0087] In this embodiment of the application, when the field of view change information conforms to the preset static scene, a third rendering instruction is generated that indicates the reuse of the rendering result of the rendering channel of the previous frame image as the rendering result of the image to be rendered, and the image to be rendered is rendered based on the third rendering instruction. Since the rendering channel corresponding to the reused rendering result does not need to be rendered again, the rendering computation is reduced, the rendering efficiency is increased, the frame rate of the target application is improved, and the heat generation and power consumption of the terminal device are reduced.

[0088] In one possible implementation, such as Figure 5 As shown, S340 (adjusting the original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter) may include:

[0089] S510, determine the multiple rendering channels in the first rendering instruction, as well as the specification parameters of the multiple rendering channels.

[0090] Referring to the description of the rendering channels in the embodiments of this application, every detail seen in the final rendered frame is calculated through a series of passes, for example, such as Figure 6 As shown, rendering channels can include: main scene rendering channel, special effects rendering channel, UI rendering channel, motion blur rendering channel, bloom rendering channel, etc. The first rendering instruction includes specification parameters for multiple rendering channels. Different rendering channels are responsible for rendering different parts of the image; for example, rendering channel A is responsible for rendering the main scene, rendering channel B is responsible for rendering special effects, and rendering channel C is responsible for rendering the UI, etc. Specification parameters include at least one of the following: rendering channel size, frame buffering attachment, and the number of draw calls. Based on the specification parameters of the rendering channels, it is possible to distinguish which part of the image each rendering channel is responsible for rendering.

[0091] S520 adjusts the original resolution parameters of the target rendering channel in multiple rendering channels to the preset resolution parameters.

[0092] The target rendering channel's specifications must conform to preset conditions. These preset conditions can be flexibly set, taking into account both image quality and improved terminal performance. If too many rendering channels have their resolution parameters reduced, while improving terminal performance, it will result in poorer image quality, which users may perceive, thus degrading the user experience. Reducing the rendering channels for the main scene strikes a balance between image quality and improved terminal performance. The target rendering channels determined by the preset specifications should achieve this balance.

[0093] For example, the target rendering channel determined by the preset specifications can be the rendering channel for the main scene. In this case, the preset specifications require the target rendering channel's Size to be greater than a certain threshold, the framebuffer attachments to include three attachments: Color, Depth, and Stencil, and the number of Draw Calls to be the highest among multiple rendering channels. Adjusting only the original resolution parameters of the rendering channel for the main scene to the preset resolution parameters can significantly reduce the rendering computation while slightly reducing rendering quality.

[0094] In another example, the target rendering channel determined by the preset specifications can be the rendering channel of the main scene and the associated rendering channel (the associated rendering channel shares the depth texture with the rendering channel of the main scene). Adjusting only the original resolution parameters of the rendering channel of the main scene and the associated rendering channel to the preset resolution parameters can significantly reduce the amount of rendering computation while reducing the rendering quality by a small amount.

[0095] In one embodiment, the rendering channel specifications can also be applied to the generation of a third rendering instruction in S410. This third rendering instruction includes a first sub-instruction and a second sub-instruction. The first sub-instruction indicates that the rendering result of at least one rendering channel from the previous frame image is used as the first rendering result of the image to be rendered. The second sub-instruction includes the original resolution parameters of the remaining rendering channels in the first rendering instruction, excluding the at least one rendering channel. Specifically, the rendering channel specifications can be used to delineate which rendering channels the first and second sub-instructions respectively indicate. For example, the rendering channels for the main scene and the remaining rendering channels besides the main scene rendering channels can be determined by the rendering channel specifications. Then, the first sub-instruction can indicate that the rendering result of the main scene rendering channel from the previous frame image is used as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels besides the main scene rendering channel in the first rendering instruction. Rendering based on this defined third rendering instruction eliminates the need to render the main scene, greatly reducing the amount of rendering computation.

[0096] In this embodiment, only the original resolution parameters of the target rendering channel are adjusted to the preset resolution parameters. In this way, when rendering based on the adjusted first rendering instruction, the amount of rendering computation can be reduced while ensuring the rendering quality, thereby improving the rendering efficiency to increase the frame rate of the target application and reducing the heat generation and power consumption of the terminal device.

[0097] In one possible implementation, the view transformation information includes displacement transformation information and angle transformation information, such as Figure 7 As shown, S330 (determining the view transformation information of the image to be rendered relative to the previous frame image according to the first rendering instruction and the second rendering instruction) may include:

[0098] S710 calculates the difference between the first coordinate and the second coordinate, and outputs the displacement transformation information.

[0099] The first rendering instruction includes the first coordinates of the Camera, and the second rendering instruction includes the second coordinates of the Camera. By calculating the difference between the two, the displacement transformation information of the Camera can be obtained.

[0100] S720, calculates the normal vector of the first plane based on the first field of view matrix, and calculates the normal vector of the second plane based on the second field of view matrix.

[0101] The first rendering instruction includes a first view matrix, and the second rendering instruction includes a second view matrix. The plane normal vector of the view matrix can be calculated based on it. For example, the first view matrix is:

[0102]

[0103] The calculated normal vector of the first plane is (a,b,c). (a,b,c) is calculated using the following function: ax + by + cz + d = 0. In this function:

[0104] a = m 41 +m 31

[0105] b = m 42 +m 32

[0106] c = m 43 +m 33

[0107] d = m 44 +m 34

[0108] S730 calculates the angle between the normal vector of the first plane and the normal vector of the second plane, and outputs the angle transformation information.

[0109] Once the normal vectors of the first and second planes are determined, the angle transformation information can be calculated. For example, the normal vector of the first plane is... The normal vector of the second plane is The angle transformation information θ can be calculated using the following formula:

[0110]

[0111] Where θ∈[0°, 180°].

[0112] Once the field of view change information is determined, the field of view change scene can be identified based on this information. For example, if the target application is a game application, preset dynamic scenes include a character walking, running, or the user's view rotating; preset dynamic scenes also include static scenes in the game (characters without significant movements, such as a character meditating, standing, or fishing). In this case, the specific scene can be determined based on the field of view change information. For example, the following scene can be defined based on the field of view change information:

[0113] For static scenes: displacement transformation information is less than threshold 1; angle transformation information is less than threshold 3.

[0114] Character walking scene: Displacement transformation information is greater than threshold 1 and less than threshold 2; angle transformation information is less than threshold 3.

[0115] In a running scenario, the displacement transformation information is greater than threshold 2; the angle transformation information is less than threshold 3.

[0116] User perspective rotation scenario: Angle change information is greater than the threshold 3.

[0117] In this embodiment, the field of view transformation information can be accurately calculated, and then the transformation scene of the Camera can be determined as a preset dynamic scene or a preset static scene based on the field of view transformation information, so as to execute different rendering instructions and adjust strategies based on different scenes.

[0118] In one embodiment, experiments were conducted on various game apps based on the rendering processing method provided in the embodiments of this application, as shown in Table 1. It can be seen that the performance of different game apps is improved by 10%-20% or the power consumption is reduced by 10%-20%.

[0119] Table 1

[0120]

[0121] In this embodiment, no code adaptation is required for the application or application engine, and it does not rely on any specific hardware platform. Based on the contextual transmission characteristic of rendering instructions in the form of data streams, it determines the scene change between the two frames and adjusts the resolution of the rendering channel accordingly to reduce the load on the target application and increase its frame rate. This allows the target application to reduce the power consumption of the terminal device without the user noticing, and delays the time when the target application is subject to system frequency and frame throttling, thereby extending the runtime of the target application and improving the user experience.

[0122] The foregoing mainly describes the solution provided by the embodiments of this application from the perspective of the execution logic of each step. It is understood that each node, such as a processor, includes corresponding hardware structures and / or software modules for executing each function in order to achieve the above-mentioned functions. Those skilled in the art should readily recognize that, in conjunction with the algorithm steps of the examples described in the embodiments disclosed herein, the method of the embodiments of this application can be implemented in hardware, software, or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software-driven hardware manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0123] This application embodiment can divide the processor or graphics processor into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0124] Figure 8 A structural diagram of a rendering processing device 800 is shown. Figure 8 Each module in the device shown has the ability to implement Figures 3-7 The corresponding steps in the module implement their functions and achieve their respective technical effects. The beneficial effects of each module's execution steps can be found in [reference]. Figures 3-7 The corresponding steps will not be elaborated further. The functions described can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The rendering processing device can be a terminal device or a chip or system-on-a-chip within the terminal device. For example, the rendering processing device includes:

[0125] The interception module 810 is used to intercept the first rendering command sent by the target application.

[0126] The first rendering instruction is used to render the image to be rendered for the target application.

[0127] The acquisition module 820 is used to acquire the second rendering instruction, wherein the second rendering instruction is used to render the previous frame of the image to be rendered.

[0128] The determination module 830 is used to determine the field of view transformation information of the image to be rendered relative to the previous frame image based on the first rendering instruction and the second rendering instruction.

[0129] The adjustment module 840 is used to adjust the original resolution parameter of at least one rendering channel in the first rendering instruction to the preset resolution parameter when the field of view change information conforms to the preset dynamic scene.

[0130] The preset resolution parameter is lower than the original resolution parameter.

[0131] The sending module 850 is used to send the adjusted first rendering command.

[0132] The adjusted first rendering instruction is used to render the image to be rendered.

[0133] In one embodiment, the apparatus further includes: a generation module 860, configured to generate a third rendering instruction when the field-view transformation information conforms to a preset static scene, wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction being configured to indicate that the rendering result of a portion of the rendering channels of the previous frame image is used as a portion of the rendering result of the image to be rendered, and the second sub-instruction including the original resolution parameters of the remaining rendering channels in the first rendering instruction excluding the portion of the rendering channels. A sending module 850 is further configured to send the third rendering instruction, which is used to render the image to be rendered.

[0134] In one embodiment, the adjustment module 840 is specifically used to: determine multiple rendering channels in the first rendering instruction, and the specification parameters of the multiple rendering channels. The original resolution parameters of the target rendering channel among the multiple rendering channels are adjusted to preset resolution parameters. The specification parameters include at least one of the following: rendering channel size, frame buffer size, and number of drawing call interfaces, and the specification parameters of the target rendering channel meet preset specification conditions.

[0135] In one embodiment, different sizes of field-of-view transformation information correspond to different preset dynamic scenes. The preset resolution parameters are different under different preset dynamic scenes, or the preset resolution parameters are the same under different preset dynamic scenes.

[0136] In one embodiment, the view transformation information includes displacement transformation information and angle transformation information. The first rendering instruction includes a first view matrix and first coordinates, and the second rendering instruction includes a second view matrix and second coordinates. The determining module 830 is specifically configured to: calculate the difference between the first coordinates and the second coordinates, and output displacement transformation information; calculate a first plane normal vector based on the first view matrix, and calculate a second plane normal vector based on the second view matrix; calculate the angle between the first plane normal vector and the second plane normal vector, and output angle transformation information.

[0137] Figure 9 A structural diagram of a rendering processing device 900 is shown. Figure 9 Each module in the device shown has the ability to implement Figures 3-7The corresponding steps in the module implement their functions and achieve their respective technical effects. The beneficial effects of each module's execution steps can be found in [reference]. Figures 3-7 The corresponding steps will not be elaborated further. The functions described can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The rendering processing device can be a terminal device or a chip or system-on-a-chip within the terminal device. For example, the rendering processing device includes:

[0138] The receiving module 910 is used to receive the third rendering instruction.

[0139] The third rendering instruction includes a first sub-instruction and a second sub-instruction. The first sub-instruction is used to indicate that the rendering result of at least one rendering channel of the previous frame of the image to be rendered by the target application is used as the first rendering result of the image to be rendered. The second sub-instruction includes the original resolution parameters of the remaining rendering channels in the first rendering instruction, excluding at least one rendering channel. The first rendering instruction is an instruction sent by the target application for rendering the image to be rendered.

[0140] The rendering module 920 is used to render the image to be rendered according to the third rendering instruction, and obtain the full rendering result of the image to be rendered.

[0141] The full rendering result includes the first rendering result and the second rendering result. The second rendering result is obtained by rendering the image to be rendered based on the original resolution parameters of the other rendering channels.

[0142] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. This program can be stored in the computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be a terminal device of any of the foregoing embodiments, such as an internal storage unit including a data sending end and / or a data receiving end, such as a hard disk or memory of the terminal device. The computer-readable storage medium can also be an external storage device of the terminal device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device. Further, the computer-readable storage medium can include both the internal storage unit and the external storage device of the terminal device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0143] This application also provides computer instructions. All or part of the processes in the above method embodiments can be executed by computer instructions to instruct related hardware (such as computers, processors, network devices, and terminals). The program can be stored in the aforementioned computer-readable storage medium.

[0144] This application also provides a chip system. The chip system may be composed of chips or may include chips and other discrete devices, without limitation. The chip system includes a processor and a transceiver. All or part of the processes in the above method embodiments can be completed by this chip system. For example, the chip system can be used to implement the functions performed by the processor in the above method embodiments, or to implement the functions performed by the graphics processor in the above method embodiments.

[0145] In one possible design, the chip system further includes a memory for storing program instructions and / or data. When the chip system is running, the processor executes the program instructions stored in the memory to cause the chip system to perform the functions performed by the processor in the above method embodiments or the functions performed by the graphics processor in the above method embodiments.

[0146] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0147] In the embodiments of this application, the memory can be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as random-access memory (RAM). Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in the embodiments of this application can also be a circuit or any other device capable of implementing storage functions, used to store instructions and / or data.

[0148] Unless otherwise specified, the term "transmission" in the embodiments of this application refers to bidirectional transmission, encompassing the actions of sending and / or receiving. Specifically, "transmission" in the embodiments of this application includes sending data, receiving data, or both sending and receiving data. In other words, data transmission here includes uplink and / or downlink data transmission. Data may include channels and / or signals; uplink data transmission refers to uplink channel and / or uplink signal transmission, and downlink data transmission refers to downlink channel and / or downlink signal transmission. The terms "network" and "system" in the embodiments of this application refer to the same concept; a communication system is a communication network.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0150] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device, such as a microcontroller, chip, or processor, to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A rendering processing method, characterized in that, include: The first rendering instruction sent by the target application is intercepted, wherein the first rendering instruction is used to render the image to be rendered by the target application, and the first rendering instruction includes a first view matrix and a first coordinate. Obtain a second rendering instruction, wherein the second rendering instruction is used to render the previous frame image of the image to be rendered, and the second rendering instruction includes a second view matrix and a second coordinate. The view transformation information of the image to be rendered relative to the previous frame image is determined according to the first rendering instruction and the second rendering instruction. The view transformation information includes displacement transformation information and angle transformation information. The displacement transformation information is determined based on the first coordinate and the second coordinate, and the angle transformation information is determined based on the first view matrix and the second view matrix. When the field of view transformation information conforms to a preset dynamic scene, the original resolution parameter of at least one rendering channel in the first rendering instruction is adjusted to a preset resolution parameter, wherein the preset resolution parameter is lower than the original resolution parameter; Send the adjusted first rendering instruction; the adjusted first rendering instruction is used to render the image to be rendered.

2. The rendering processing method according to claim 1, characterized in that, The method further includes: When the field of view transformation information conforms to a preset static scene, a third rendering instruction is generated. The third rendering instruction includes a first sub-instruction and a second sub-instruction. The first sub-instruction is used to indicate that the rendering result of at least one rendering channel of the previous frame image is used as the first rendering result of the image to be rendered. The second sub-instruction includes the original resolution parameters of the remaining rendering channels in the first rendering instruction other than the at least one rendering channel. Send the third rendering instruction, which is used to render the image to be rendered.

3. The rendering processing method according to claim 1, characterized in that, Adjusting the original resolution parameter of at least one rendering channel in the first rendering instruction to a preset resolution parameter includes: Determine multiple rendering channels in the first rendering instruction, as well as the specification parameters of the multiple rendering channels; the specification parameters include at least one of the following: rendering channel size, frame buffer attachment, and number of drawing call interfaces; The original resolution parameters of the target rendering channel among the multiple rendering channels are adjusted to preset resolution parameters, wherein the specification parameters of the target rendering channel meet the preset specification conditions.

4. The rendering processing method according to claim 1 or 3, characterized in that, Different sizes of field-of-view transformation information correspond to different preset dynamic scenes; the preset resolution parameters are different under different preset dynamic scenes, or the preset resolution parameters are the same under different preset dynamic scenes.

5. The rendering processing method according to any one of claims 1-3, characterized in that, Determining the view transformation information of the image to be rendered relative to the previous frame image based on the first rendering instruction and the second rendering instruction includes: Calculate the difference between the first coordinate and the second coordinate, and output the displacement transformation information; Calculate the first plane normal vector based on the first field of view matrix, and calculate the second plane normal vector based on the second field of view matrix; Calculate the angle between the normal vector of the first plane and the normal vector of the second plane, and output the angle transformation information.

6. A rendering processing method, characterized in that, include: Receive a third rendering instruction; wherein the third rendering instruction includes a first sub-instruction and a second sub-instruction, the first sub-instruction is used to indicate that the rendering result of at least one rendering channel of the previous frame image of the image to be rendered by the target application is used as the first rendering result of the image to be rendered, and the second sub-instruction includes the original resolution parameters of the remaining rendering channels in the first rendering instruction other than the at least one rendering channel, and the first rendering instruction is an instruction sent by the target application for rendering the image to be rendered; The image to be rendered is rendered according to the third rendering instruction to obtain the full rendering result of the image to be rendered. The full rendering result includes the first rendering result and the second rendering result. The second rendering result is obtained by the remaining rendering channels rendering the image to be rendered based on the original resolution parameters.

7. A rendering device, characterized in that, The rendering device includes a processor and a transceiver, the processor and the transceiver being configured to support the rendering device in performing the method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed, perform the method as described in any one of claims 1-6.

Images