Rendering processing method and electronic device

Abstract

The embodiments of the present application relate to the technical field of terminals. Provided are a rendering processing method and an electronic device. The method comprises: determining a plurality of rendering instructions corresponding to first graphic data to be rendered of a first application program, wherein the plurality of rendering instructions include a first rendering instruction; and then reading, from a graphics processing unit (GPU) memory, a first rendering resource corresponding to the first rendering instruction, and executing the plurality of rendering instructions on the basis of rendering resources respectively corresponding to the plurality of rendering instructions, so as to render the first graphic data. In this way, a rendering resource is read from a GPU memory, so as to reduce data interaction between a GPU and a main memory, thereby reducing the bandwidth consumption of a terminal device.

Description

Rendering processing method and electronic device TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, and in particular to a rendering processing method and an electronic device. BACKGROUND

[0002] With the continuous development of rendering scenes, the corresponding rendering pipeline is also becoming more and more complex. The rendering pipeline is an abstract concept of the process and steps for generating images in computer graphics. It can be understood that the rendering pipeline includes multiple stages of graphics processing and calculation.

[0003] In the rendering pipeline, the rendering resources usually need to be processed accordingly to complete the corresponding rendering task. In the related art, the execution of the rendering task is usually performed by a GPU, and the rendering resources are usually stored in the main memory. This results in that the GPU needs to obtain the rendering resources from the main memory, and after the GPU generates the intermediate rendering resources, the rendering resources also need to be written into the main memory.

[0004] In this way, the GPU needs to frequently interact with the main memory during the rendering process, thereby causing high bandwidth consumption of the terminal device.

[0005] SUMMARY

[0006] Embodiments of the present application provide a rendering processing method and an electronic device, which are applied to the technical field of terminals to reduce the data interaction between the GPU and the main memory, thereby reducing the bandwidth consumption of the terminal device.

[0007] In a first aspect, an embodiment of the present application provides a rendering processing method. The method comprises:

[0008] determining a plurality of rendering instructions corresponding to first graphics data to be rendered by a first application program, the plurality of rendering instructions including a first rendering instruction. The first application program can be any type of application program as long as the first application program has a demand for graphics data rendering.

[0009] Then, a first rendering resource corresponding to the first rendering instruction is read from a GPU memory, and based on the rendering resources corresponding to the plurality of rendering instructions respectively, the plurality of rendering instructions are executed to perform rendering processing on the first graphics data.

[0010] The execution of the plurality of rendering instructions can be understood as a processing process of rendering processing on the first graphics data. After the execution of the plurality of rendering instructions is completed, a rendering result, i.e., the first graphics data after rendering, can be obtained.

[0011] It can be understood that, compared with the implementation manner of reading data from the main memory, the implementation manner of reading data from the GPU memory is equivalent to reading data locally on the GPU, and therefore the bandwidth consumption of reading data from the GPU memory is relatively low, and the reading speed is relatively fast. Therefore, based on the implementation manner introduced above, reading the rendering resource corresponding to the part of the rendering instruction from the GPU memory can effectively reduce the data interaction between the GPU and the main memory, so as to reduce the bandwidth consumption of the terminal device.

[0012] In a possible implementation manner, the plurality of rendering instructions further includes a second rendering instruction; and the method further includes: reading a first rendering resource corresponding to the second rendering instruction from the GPU memory.

[0013] It can be understood that the first rendering instruction and the second rendering instruction both correspond to the first rendering resource, and therefore the first rendering resource can be stored in the GPU memory, and then the first rendering resource is read from the GPU memory to execute the first rendering instruction, and the first rendering resource is also read from the GPU memory to execute the second rendering instruction. Therefore, in this implementation manner, the rendering resource required by the plurality of rendering instructions can be written into the GPU memory, and then this part of the rendering resource is read from the GPU memory, so that the bandwidth consumption caused by repeatedly reading the same rendering resource from the main memory can be effectively saved.

[0014] In a possible implementation manner, the second rendering instruction is adjacent to the first rendering instruction in the instruction sequence of the plurality of rendering instructions. Based on this implementation manner, in the case that the adjacent two rendering instructions need to read the same rendering resource, this part of the same rendering resource can be written into the GPU memory for subsequent reading, so that writing this part of the rendering resource into the GPU memory can effectively save the bandwidth consumption.

[0015] Alternatively, the number of rendering instructions between the second rendering instruction and the first rendering instruction is less than or equal to a first threshold. Based on this implementation manner, in the case that the several rendering instructions close in the instruction sequence need to read the same rendering resource, this part of the same rendering resource can be written into the GPU memory for subsequent reading, so that writing this part of the rendering resource into the GPU memory can effectively save the bandwidth consumption.

[0016] Alternatively, the number of rendering instructions corresponding to the first rendering resource in the plurality of rendering instructions is greater than or equal to a second threshold, and the plurality of rendering instructions includes the second rendering instruction. Based on this implementation manner, in the case that a large number of rendering instructions need to read the same rendering resource, this part of the same rendering resource can be written into the GPU memory for subsequent reading, so that writing this part of the rendering resource into the GPU memory can effectively save the bandwidth consumption.

[0017] In a possible implementation, the plurality of rendering instructions further comprises a third rendering instruction; and the executing the plurality of rendering instructions comprises:

[0018] The first rendering instruction is executed based on the first rendering resource to obtain a second rendering resource, the second rendering resource being an intermediate rendering resource generated in a process of executing a rendering process;

[0019] The third rendering instruction is executed based on the second rendering resource.

[0020] In a possible implementation, after the second rendering resource is obtained, the method further comprises:

[0021] The second rendering resource is written into the GPU memory.

[0022] Before the third rendering instruction is executed based on the second rendering resource, the method further comprises:

[0023] The second rendering resource corresponding to the third rendering instruction is read from the GPU memory.

[0024] It can be understood that the second rendering resource is obtained after the first rendering instruction is executed, and then the second rendering resource is written into the GPU memory, and the second rendering resource is read from the GPU memory to execute the third rendering instruction subsequently. Based on this implementation, the rendering resource generated by the rendering instruction in front of the instruction sequence and the rendering resource required to be read by the rendering instruction in back of the execution sequence can be written into the GPU memory, and then the rendering resource is read from the GPU memory, so that the bandwidth consumption caused by writing the rendering resource into the main memory and then reading the rendering resource from the main memory can be effectively saved.

[0025] In a possible implementation, the third rendering instruction is adjacent to the first rendering instruction in the instruction sequence of the plurality of rendering instructions. Based on this implementation, in the case that the rendering resource generated by the previous rendering instruction is required to be read by the subsequent rendering instruction immediately after the previous rendering instruction, the generated rendering resource can be written into the GPU memory for subsequent reading, so that the bandwidth consumption caused by writing the rendering resource into the GPU memory can be effectively saved.

[0026] Alternatively, the number of rendering instructions between the third rendering instruction and the first rendering instruction is less than or equal to a third threshold. Based on this implementation, in the case that the rendering resource generated by the previous rendering instruction is required to be read by the subsequent rendering instruction immediately after the previous rendering instruction, the generated rendering resource can be written into the GPU memory for subsequent reading, so that the bandwidth consumption caused by writing the rendering resource into the GPU memory can be effectively saved.

[0027] In a possible implementation, the method further includes:

[0028] The indication field corresponding to each of the plurality of rendering resources is obtained, and each of the plurality of rendering resources corresponds to a respective indication field; for example, the indication field can be a field included in metadata of the rendering resource, or the indication field can be a field associated with the rendering resource, and the embodiment does not limit the specific field position and implementation of the indication field.

[0029] Subsequently, the plurality of rendering resources can be written based on the indication field corresponding to each of the plurality of rendering resources. When the indication field is of a first value, the write operation is used to write the rendering resource corresponding to the indication field to the GPU memory; when the indication field is of a second value, the write operation is used to write the rendering resource corresponding to the indication field to the main memory. For example, the first value can be 1, and the second value can be 0. Alternatively, the first value and the second value can also be other values, and the application does not limit this.

[0030] In this implementation, by determining the respective indication field corresponding to each of the plurality of rendering resources, the storage location of each rendering resource can be quickly and effectively indicated, so that subsequent determination can be made on whether the rendering resource is to be written to the GPU memory or the main memory according to the indication field.

[0031] In a possible implementation, the plurality of rendering resources includes a first rendering resource and a second rendering resource.

[0032] The indication field corresponding to each of the plurality of rendering resources is obtained, and each of the plurality of rendering resources corresponds to a respective indication field; for example, the indication field can be a field included in metadata of the rendering resource, or the indication field can be a field associated with the rendering resource, and the embodiment does not limit the specific field position and implementation of the indication field.

[0033] In a case where the usage type of the first rendering resource by the first rendering instruction is a read type, if the usage type of the first rendering resource by the second rendering instruction is a read type, it is determined that the value of the indication field corresponding to the first rendering resource is a first value. If the usage type of the first rendering resource by the second rendering instruction is not a read type, it is determined that the value of the indication field corresponding to the first rendering resource is a second value.

[0034] In this implementation, when a certain rendering instruction needs to read a certain rendering resource, if it is determined that another rendering instruction also needs to read the rendering resource, it can be determined that the indication field of the rendering resource indicates that the storage location of the rendering resource is the GPU memory, so as to effectively save the bandwidth consumption in the subsequent data interaction process and improve the necessity of writing the rendering resource to the GPU memory. The instruction relationship between the certain rendering instruction and the another rendering instruction can be understood with reference to the first rendering instruction and the second rendering instruction described above.

[0035] Or, in the case that the use type of the first rendering instruction on the second rendering resource is a write type, and the use type of the third rendering instruction on the second rendering resource is a read type, it is determined that the value of the indication field corresponding to the second rendering resource is the first value.

[0036] In this implementation, when a certain rendering instruction generates a certain rendering resource, if it is determined that there is another rendering instruction that needs to read the rendering resource generated by the rendering instruction, it can be determined that the indication field of the rendering resource indicates that the storage location of the rendering resource is the GPU memory, so as to effectively save the bandwidth consumption in the subsequent data interaction process and improve the necessity of writing the rendering resource into the GPU memory. The instruction relationship between the certain rendering instruction and the another rendering instruction introduced herein can be understood with reference to the first rendering instruction and the third rendering instruction introduced above.

[0037] In a possible implementation, before determining that the indication field corresponding to the first rendering resource is the first value, the method further includes:

[0038] determining whether the data amount of the first rendering resource is less than or equal to the remaining storage space of the GPU memory;

[0039] determining that the indication field corresponding to the first rendering resource is the first value includes:

[0040] determining that the indication field corresponding to the first rendering resource is the first value when the data amount of the first rendering resource is less than or equal to the remaining storage space of the GPU memory.

[0041] In addition, before determining that the indication field corresponding to the second rendering resource is the first value, the method further includes:

[0042] determining whether the data amount of the second rendering resource is less than or equal to the remaining storage space of the GPU memory;

[0043] determining that the indication field corresponding to the second rendering resource is the first value includes:

[0044] determining that the indication field corresponding to the second rendering resource is the first value when the data amount of the second rendering resource is less than or equal to the remaining storage space of the GPU memory.

[0045] In this implementation, in the case that it is determined that the rendering resource meets the conditions introduced above, the data amount of the rendering resource can be further determined to determine whether the rendering resource can be stored in the GPU memory. If it is determined that the rendering resource can be stored, it is further determined that the indication field of the rendering resource is the first value, so as to improve the success rate of storing the rendering resource in the GPU memory.

[0046] In a possible implementation, the plurality of rendering instructions are executed, including:

[0047] In a case where the rendering mode corresponding to the first graphics data is the first rendering mode, the plurality of rendering instructions are executed in sequence.

[0048] In a case where the rendering mode corresponding to the first graphics data is the second rendering mode, the plurality of tiles obtained by the division are rendered in sequence, and in the process of rendering the plurality of tiles, the rendering instruction associated with each of the plurality of tiles is executed.

[0049] In a case where the region corresponding to the first graphics data covers the full screen range of the terminal device, it can be determined that the rendering mode corresponding to the first graphics data is the first rendering mode. In a case where the region corresponding to the first graphics data covers the partial screen range of the terminal device, it can be determined that the rendering mode corresponding to the first graphics data is the second rendering mode.

[0050] Exemplarily, the first rendering mode can be direct rendering, and the second rendering mode can be tile-based rendering. Therefore, in the technical solution of the present application, different rendering modes can be adopted for different types of graphics data.

[0051] In a possible implementation, rendering the plurality of tiles obtained by the division comprises:

[0052] Based on the geometry data associated with each of the plurality of tiles and the rendering resource corresponding to the rendering instruction associated with each of the plurality of tiles, the plurality of tiles are rendered.

[0053] The method further comprises:

[0054] The geometry data associated with the plurality of tiles is read from the GPU memory.

[0055] In this implementation, by writing the geometry data associated with the tile obtained by the division of the first graphics data into the GPU memory, the bandwidth consumption caused by reading the geometry data from the GPU memory can be further saved.

[0056] In a possible implementation, the method further comprises:

[0057] In a case where the total data amount of the geometry data contained in the first graphics data is less than or equal to the remaining storage space of the GPU memory, the geometry data contained in the first graphics data is written into the GPU memory.

[0058] In this implementation, in a case where the geometry data can be stored in the GPU memory in full amount, the geometry data contained in the first graphics data is written into the GPU memory, thereby effectively saving the bandwidth consumption of reading the geometry data from the main memory.

[0059] In a possible implementation, the method further comprises:

[0060] In a case that the total data amount of the geometry data contained in the first graphics data is greater than the remaining storage space of the GPU memory, the plurality of tiles are rendered in the first order one by one;

[0061] For the rendered tile, if the geometry data associated with the rendered tile is not stored in the GPU memory, the geometry data associated with the rendered tile is written into the GPU memory;

[0062] The plurality of tiles include a first tile and a second tile, the geometry data associated with the first tile is the same as the geometry data associated with the second tile, and the first tile and the second tile are adjacent in the first order.

[0063] In this implementation, in a case that the geometry data cannot be stored in the GPU memory in full amount, for each rendered tile, it is determined whether the geometry data associated with the tile is stored in the GPU memory, if the geometry data associated with the tile is already stored in the GPU memory, the geometry data associated with the tile can be directly read from the GPU memory. When the geometry data associated with the tile is not stored in the GPU memory, the geometry data associated with the tile is written into the GPU memory, so that part of the geometry data can be written into the GPU memory according to the geometry data actually associated with the rendered tile. At the same time, the tiles with the same associated geometry data are adjacent in the first order, and the probability that the geometry data associated with the tile is already stored in the GPU memory can also be improved, so as to avoid the bandwidth consumption caused by frequently writing the same geometry data into the GPU memory.

[0064] In addition, for each rendered tile, it can also be determined whether there is remaining geometry data other than the geometry data associated with the tile in the geometry data stored in the GPU memory, if there is, the remaining geometry data can be deleted from the GPU memory, so that the storage space of the GPU memory can be effectively cleaned up, and the success rate of storing the geometry data associated with the tile into the GPU memory can be improved.

[0065] In a possible implementation, the method further includes:

[0066] The tile rendering results of the plurality of tiles are written into the GPU memory. In this implementation, the GPU memory can also serve as a storage for the tile rendering results.

[0067] In a possible implementation, for any target rendering resource with the first value of the indication field, after the reading of the target rendering resource is completed, the method further includes:

[0068] If there is no read instruction for the target rendering resource, the target rendering resource is deleted from the GPU memory, and the read instruction is sent by a central processing unit (CPU);

[0069] After the reading of the target rendering resource is completed, if there is a read instruction for the target rendering resource, the target rendering resource is sent to the CPU, and the target rendering resource is deleted from the GPU memory.

[0070] In this implementation manner, before the stored rendering resource is deleted from the GPU memory, it can be first determined whether the CPU needs to read the rendering resource, and if the CPU needs to read, the deletion is performed after the rendering resource is sent to the CPU, so that the logical correctness of the data interaction can be effectively guaranteed.

[0071] In a second aspect, an embodiment of the present application provides a rendering processing apparatus. The rendering processing apparatus can be an electronic device, or a chip or chip system in the electronic device.

[0072] The rendering processing apparatus can include a display unit and a processing unit.

[0073] When the rendering processing apparatus is an electronic device, the display unit can be a display screen. The display unit is configured to perform the displaying, so that the electronic device implements a rendering processing method described in the first aspect or any possible implementation manner of the first aspect.

[0074] When the rendering processing apparatus is an electronic device, the processing unit can be a processor. The rendering processing apparatus can further include a storage unit, which can be a memory. The storage unit is configured to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the electronic device implements a rendering processing method described in the first aspect or any possible implementation manner of the first aspect.

[0075] When the rendering processing apparatus is a chip or chip system in an electronic device, the processing unit can be a processor. The processing unit executes instructions stored in a storage unit, so that the electronic device implements a rendering processing method described in the first aspect or any possible implementation manner of the first aspect. The storage unit can be a storage unit (for example, a register, a cache, etc.) in the chip, or a storage unit (for example, a read-only memory, a random access memory, etc.) in the electronic device and located outside the chip.

[0076] In a third aspect, an embodiment of the present application provides an electronic device including a processor and a memory. The memory is configured to store code instructions, and the processor is configured to execute the code instructions to perform a method described in the first aspect or any possible implementation manner of the first aspect.

[0077] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0078] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program, and when the computer program is run on a computer, the computer is caused to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0079] In a sixth aspect, the present application provides a chip or chip system, which includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is configured to run a computer program or instructions to execute the method described in the first aspect or any possible implementation manner of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0080] In a possible implementation, the chip or chip system described in the present application further includes at least one memory, and the at least one memory stores instructions. The memory can be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0081] It should be understood that the second aspect to the sixth aspect of the present application correspond to the technical solution of the first aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding possible implementation manner are similar, which will not be repeated. BRIEF DESCRIPTION OF DRAWINGS

[0082] FIG. 1 is an implementation schematic diagram of a rendering architecture provided by an embodiment of the present application;

[0083] FIG. 2 is a schematic diagram of a rendering pipeline of direct rendering;

[0084] FIG. 3 is a schematic diagram of a rendering pipeline based on tile rendering;

[0085] FIG. 4 is a hardware structure schematic diagram of a terminal device provided by an embodiment of the present application;

[0086] FIG. 5 is a software structure schematic diagram of a terminal device provided by an embodiment of the present application;

[0087] FIG. 6 is a processing flow schematic diagram of a rendering instruction provided by an embodiment of the present application;

[0088] FIG. 7 is a logical judgment schematic diagram of saving a rendering resource provided by an embodiment of the present application;

[0089] FIG. 8 is a schematic diagram of usage of rendering resources in an instruction buffer according to an embodiment of the present application;

[0090] FIG. 9 is a schematic diagram of usage of rendering resources in an instruction buffer according to an embodiment of the present application;

[0091] FIG. 10 is a schematic diagram of implementation of reading and writing of rendering resources according to an embodiment of the present application;

[0092] FIG. 11 is a schematic diagram of implementation of reading and writing of rendering resources according to an embodiment of the present application;

[0093] FIG. 12 is a schematic diagram of implementation of reading and writing of rendering resources according to an embodiment of the present application;

[0094] FIG. 13a is a schematic diagram of implementation of reading and writing of rendering resources according to an embodiment of the present application;

[0095] FIG. 13b is a schematic diagram of implementation of reading and writing of rendering resources according to an embodiment of the present application;

[0096] FIG. 14a is a schematic diagram of implementation of reading and writing of rendering resources according to an embodiment of the present application;

[0097] FIG. 14b is a schematic diagram of implementation of reading and writing of rendering resources according to an embodiment of the present application;

[0098] FIG. 15 is a schematic block diagram of a rendering processing apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0099] For the purpose of clearly describing the technical solutions of the embodiments of the present application, the following briefly introduces some terms and technologies involved in the embodiments of the present application:

[0100] 1. Rendering pipeline

[0101] A rendering pipeline is an abstract concept in computer graphics that describes the process and steps for generating images. It describes the entire process from 3D scene data to the final visualized image on the screen, including various stages of graphics processing and computation.

[0102] 2. Rendering resources

[0103] Rendering resources can include, for example, textures, materials, attributes, and vertices, etc.

[0104] Among them, texture is an image used to represent the details and appearance of an object's surface. In the rendering process, texture is usually mapped onto the surface of geometry to increase visual details and realism. Texture can include color information, lighting information, surface roughness, etc. Common texture types include color texture, normal texture, height texture, ambient occlusion texture, etc. Texture can be used for various effects, such as coloring objects, simulating surface details, implementing parallax mapping, etc.

[0105] Material describes how an object reflects light and represents its appearance characteristics during rendering. Material usually includes properties such as surface color, glossiness, reflectivity, transparency, etc. In rendering, material is combined with texture to determine the final visual effect of the object. Different types of material can simulate different surfaces, such as metal, wood, glass, etc.

[0106] Attributes refer to data that describes the characteristics of vertices or fragments. Vertex attributes usually include position coordinates, color, normal direction, texture coordinates, etc. These attributes can be processed and modified in the rendering pipeline to determine the final appearance of each vertex or fragment. For example, the color attribute of a vertex can determine the basic color of the vertex, and the texture coordinate attribute can be used to determine the mapping position of the texture.

[0107] Vertices are the basic points that make up a three-dimensional model. Each vertex usually contains position coordinates and other attributes such as color, normal, texture coordinates, etc. In the rendering pipeline, vertices undergo a series of processes, including transformation, lighting calculation, texture mapping, etc., and are finally converted into pixels on the screen to form the final image. The attributes of the vertex determine the appearance and behavior of the object's surface.

[0108] In the rendering process, the above-mentioned rendering resources and intermediate results obtained by processing the rendering resources, such as UBO (Uniform Buffer Object, Uniform Buffer Object) or SSBO (Shader Storage Buffer Object, Shader Storage Buffer Object), can be carried by the two mechanisms used to transfer data between GPU and CPU, but they have some differences in use and characteristics.

[0109] Among them, UBO is a buffer object used to transfer constant data to the shader. It is mainly used to transfer data that does not change during rendering, such as projection matrix, view matrix, lighting parameters, etc.

[0110] SSBO is a buffer object used to read and write data in the shader. Compared with UBO, it is more flexible and can be used to store and process data that may change during rendering, such as intermediate calculation results in compute shaders, transformation feedback, etc.

[0111] 3、RenderPass

[0112] RenderPass (RenderPass) is a concept used to describe a series of rendering operations and resource usage during the graphics processing process.

[0113] For example, RenderPass can define a series of rendering operations that are executed in a specified order, ensuring correct resource usage and synchronization during rendering. In addition, RenderPass can also define various resources used during rendering, such as color buffers, depth buffers, additional input / output buffers, etc. RenderPass can define multiple color attachments, supporting simultaneous rendering to multiple targets.

[0114] The components of RenderPass may include, for example, Attachment (Attachment), Subpass (Subpass), and Dependency (Dependency).

[0115] Attachment is a resource defined in RenderPass, which can be a color buffer, a depth buffer, or other types of image resources. Each Subpass can use one or more attachments.

[0116] Subpass is a stage in RenderPass, containing a set of rendering operations. Subpass can specify which attachments to use as input / output, and can specify dependencies.

[0117] Dependency defines the dependency and execution order between Subpasses. It ensures correct synchronization and management of resource usage during rendering.

[0118] 4、Other terms

[0119] In embodiments of the present application, "first", "second", etc. are used to distinguish the same or similar items with basically the same function and effect. For example, the first chip and the second chip are only used to distinguish different chips, and do not limit the order. Those skilled in the art can understand that "first", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. also do not necessarily mean different.

[0120] It should be noted that the terms "exemplary" and "for example" are used herein to mean "serving as an example, instance, or illustration," and not "preferred" or "advantageous over other examples." The usage of these terms in this application is not intended to convey any preference or advantage for the embodiments or examples described with such terms.

[0121] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, and c can be single or multiple.

[0122] 5. Electronic device

[0123] The electronic device in the embodiments of the present applicationapplicationinclude a handheld device with rendering function, a vehicle-mounted device, etc. For example, some electronic devices are: a mobile phone, a tablet computer, a palm computer, a notebook computer, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc. The embodiments of the present application are not limited thereto.

[0124] By way of example and not limitation, in the embodiments of the present application, the electronic deviceapplicationalso be a wearable device. In addition, in the embodiments of the present application, the electronic deviceapplicationalso be a terminal device in an internet of things (IoT) system. IoT is an important component of future information technology development, and its main technical feature is to connect objects to a network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-object interconnection.

[0125] The electronic device in the embodiments of the present applicationapplicationalso be referred to as: a terminal device, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), an access terminal, a subscriber unit, a subscriber station, a mobile station, a mobile terminal, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent, or a user apparatus, etc.

[0126] In the embodiments of the present application, the electronic device or each network device comprises a hardware layer, an operating system layer running above the hardware layer, and an application layer running on the operating system layer. The hardware layer comprises hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also referred to as main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system. The application layer comprises applications such as a browser, an address book, word processing software, and instant messaging software.

[0127] In order to better understand the technical solutions of the present application, the related technologies involved in the present application are further described in detail below.

[0128] With the continuous launch of high-quality and high-definition mobile games, the rendering scenes of mobile games are becoming more and more rich, and the rendering effects are becoming more and more gorgeous, and accordingly the rendering pipeline is becoming more and more complex, and the requirement for rendering resources is becoming higher and higher.

[0129] In the rendering pipeline, a large amount of intermediate resources and rendering information will be generated, wherein the intermediate resources may, for example, include the textures, materials, attributes, and vertices described above, and the rendering information can be understood as information related to rendering operations. These intermediate resources and rendering information are often used only within the rendering pipeline, and outside the rendering pipeline, these resources are usually not needed.

[0130] The current rendering architecture will be briefly introduced below in combination with FIG. 1, which is an implementation schematic diagram of the rendering architecture provided in the embodiments of the present application.

[0131] As shown in FIG. 1, the current rendering architecture comprises a GPU (Graphics Processing Unit) and a main memory, wherein the main memory may, for example, be a DDR (Double Data Rate).

[0132] In addition, the GPU comprises a compute wave (also referred to as a compute beam) and a vertex wave / fragment wave (also referred to as a vertex beam / fragment beam). The compute wave is used to obtain instructions of a compute task, and the vertex wave / fragment wave is used to obtain instructions of a graphics task. In an implementation manner, the instructions of the compute task and the instructions of the graphics task can be sent to the GPU by the CPU.

[0133] Referring to FIG. 1, a job scheduler is also included in the GPU, which is responsible for distributing the received instructions to each shader core. A plurality of shader cores are included in the GPU, which can process the instructions of related tasks in parallel to complete the specific requirements of the instructions.

[0134] Here, the shader core is briefly introduced again. The shader core is a key component responsible for executing graphics rendering and general computing tasks. The shader core is a processor core in the GPU, which is specifically used to execute various shader programs, which can include vertex shaders, fragment shaders, geometry shaders, compute shaders, etc. Each shader core includes a plurality of processing units for parallel execution of shader instructions to achieve efficient graphics rendering and computing functions.

[0135] In addition, a vertex cache is also included in the GPU. After the shader core processes the vertex-related data, it can first store the related data in the vertex cache. This implementation can avoid repeated calculations for the same vertex to improve the efficiency of graphics processing.

[0136] In addition, a GPU memory (GMEM) is also included in the GPU. Because the GMEM is located in the GPU, the GPU can read and write data in the GMEM at a faster access speed.

[0137] In addition, an L2 cache is also included in the GPU. The L2 cache is a temporary memory located between the processor and the memory (i.e., the main memory), which has a smaller capacity but faster exchange speed than the memory.

[0138] On the basis of the architecture introduced in FIG. 1, in the related art, when performing a graphics processing task, the graphics unit can send the obtained instructions to the job scheduler, and then the job scheduler distributes the instructions to each shader core.

[0139] In addition, the shader core usually needs to interact with rendering resources when executing instructions. The rendering resources in the embodiment can include, for example, the above-mentioned textures, materials, attributes, vertices, etc., and can also include UBO or SSBO used in the computing process, etc.

[0140] The data interaction performed by the shader core can include, for example, reading of rendering resources and writing of rendering resources.

[0141] In a possible implementation, the shader core can write the rendering resource into the main memory when performing the writing of the rendering resource. In addition to being written by the shader core, the rendering resource stored in the main memory can also be written by other software units generating the rendering resource. In the prior art, the rendering resource is stored in the main memory.

[0142] When the shader core reads the rendering resource, the shader core reads the rendering resource from the main memory. The implementation of the shader core reading data can be understood with reference to FIG. 1. As shown in FIG. 1, when the GPU needs to use some rendering resource, the GPU reads data from the main memory, and then stores the read data in the L2 cache. Then, the shader core can read the rendering resource required by the shader core from the L2 cache through the data path 1 in FIG. 1, to complete the related graphics processing instruction.

[0143] As shown in FIG. 1, there is also a data path 2 between the shader core and the GMEM. Therefore, the shader core can also store some data in the GMEM through the data path 2, and read some data from the GMEM through the data path 2. The implementation of the GMEM storing data will be further described in the following embodiments.

[0144] Based on the rendering architecture described above, two existing rendering technologies will be introduced below. The two rendering technologies are direct rendering and tile based rendering.

[0145] The direct rendering is a traditional graphics rendering method, also known as immediate mode rendering (IMR). In an implementation, when the area corresponding to the to-be-rendered graphics data covers the full screen range of the terminal device, the direct rendering can be used to render the graphics data.

[0146] Direct rendering refers to directly submitting rendering for each primitive. A primitive is a basic geometric primitive or basic graphics object in graphics, and is used to construct a complex graphics scene. Common graphics primitives include a point, a line segment, a triangle, and the like. The following embodiments take the triangle as an example.

[0147] It can be understood that the rendering pipeline in the direct rendering technology does not interrupt, and the rendering speed is fast. The rendering pipeline of the direct rendering will be understood in combination with FIG. 2. FIG. 2 is a schematic diagram of the rendering pipeline of the direct rendering.

[0148] As shown in Figure 2, the rendering pipeline for direct rendering is:

[0149] The GPU reads geometric data from main memory (or system memory), which can be understood as primitives as described above.

[0150] The GPU then performs vertex processing based on the geometric data, transforming multiple vertices in the geometric data so that the calculated vertices conform to the viewing direction of the virtual camera. Therefore, after the vertex processing stage is completed, multiple vertices and their individual positions are obtained.

[0151] Next, the GPU performs culling on the calculated vertices. This is because, depending on the viewport, some objects in the scene may be occluded, or some rendered objects may be outside the viewport. Therefore, the rendering pipeline can perform culling on some invisible vertices (clip.project.cull). The purpose of culling is to remove invisible vertices to avoid unnecessary rendering operations.

[0152] Then, rasterization can be performed on the remaining vertices after culling. Rasterization involves finding the pixels covered by each triangle.

[0153] Furthermore, because triangles may overlap, when some pixels of one triangle are covered by other triangles, these covered pixels do not need to participate in subsequent rendering processes, as they would be invisible even if rendered. This effectively avoids wasting resources. Therefore, it is necessary to perform early depth testing (early Z) on the pixels of each triangle separately.

[0154] As shown in Figure 2, the GPU can read the depth information of pixels from the depth buffer in main memory, then compare the depth information of the pixels to determine which pixels are visible, and then perform subsequent processing on the visible pixels. Additionally, at this stage, the GPU can also write the depth information of pixels into the depth buffer in main memory.

[0155] The GPU can then read texture data from main memory to execute the corresponding rendering algorithm, thereby coloring the corresponding triangles. The rendering algorithm can include the texture algorithm and shading algorithm shown in Figure 2.

[0156] Further, because some rendering algorithms can change the depth information of pixels, after the rendering algorithm is executed, a late Z can also be performed, for example, the implementation of the late Z is similar to the implementation of the early Z described above, which will not be described here. It can be understood that whether the late Z is executed or not is optional, if the rendering algorithm changes the depth information of the pixel, the late Z can be enabled. Otherwise, the late Z can be cancelled.

[0157] Finally, the result of the coloring is subjected to alpha blend processing to blend the rendered result with the original rendering result on the screen to obtain a final rendering result, and then the final rendering result can be stored in the frame buffer of the main memory.

[0158] The rendering pipeline described above is a direct rendering pipeline, which directly executes the complete rendering process described above for each primitive. However, there is a problem, if there is a large graph (such as a triangle) to be rendered, the frame buffer will be large, such as the color rendering or depth rendering of the entire screen will consume a lot of storage resources, but there is not enough resources on the GPU. Therefore, in the direct rendering mode, the rendering related data is usually stored in the main memory, based on the above, it can be determined that this rendering method needs to frequently read the main memory, thereby increasing the bandwidth.

[0159] In order to alleviate the problem of frequent reading of the main memory, a tile-based rendering method is currently proposed. In one implementation, when the area corresponding to the to-be-rendered graphics data covers part of the screen range of the terminal device, the graphics data can be rendered by using the direct rendering method.

[0160] Among them, the tile-based rendering is also called tile-based rendering or small block-based rendering, which is a process of subdividing an image by a regular grid in optical space and rendering each grid or tile after subdivision. The core idea is to divide the frame buffer into multiple small tiles, and then render each tile block by block.

[0161] The rendering pipeline based on tile-based rendering can be introduced below in combination with FIG. 3, which is a schematic diagram of the rendering pipeline based on tile-based rendering.

[0162] As shown in FIG. 3, in the rendering pipeline, the GPU reads the geometry data from the main memory, calculates the vertices, and performs culling, the implementation of the three parts is similar to that described above in FIG. 2, which will not be described here.

[0163] The difference between the above direct rendering and the current rendering is that after the execution of the culling operation, the current rendering needs to perform binning processing, wherein the binning processing is used to divide the rendering object to obtain a plurality of tiles.

[0164] Respective tile data can be generated for each of the tiles, wherein the tile data can include which vertices are included in the current tile, how the vertices are combined, and what color the triangle included in the tile is to be drawn, and the like. Any data related to the divided tile can be the tile data in the present application, and the specific tile data is not limited in the embodiment.

[0165] The tile data of each tile can then be stored in the main memory, for example, in FIG. 3, the tile data can be stored in the intermediate store of the main memory.

[0166] Then, rendering is performed for each tile in turn. Referring to FIG. 3, the GPU can first read the tile data of the first tile from the intermediate store of the main memory, and then perform rasterization, early depth test, rendering algorithm, late depth test, and alpha blending operations for the first tile in turn. The implementation manners of these operations are similar to those introduced in the above embodiment, and will not be described herein again.

[0167] As shown in FIG. 3, in the current implementation manner, because the data amount of one tile is relatively small, the depth information required for performing the depth test can be stored in the on-chip buffer of the GPU on-chip memory. In an implementation manner, the on-chip buffer in the embodiment is equivalent to the above-mentioned GMEM.

[0168] After the alpha blending is performed on one tile, the rendering result of the tile can be first stored in the on-chip buffer, and when it is determined that the rendering of all pixels on the tile is completed, the rendering result of the tile is written back to the frame buffer of the main memory.

[0169] In this way, the rendering processing of the first tile is completed, and then the tile data of the second tile is read from the main memory, and the rendering process introduced above is performed on the second tile to obtain the rendering result of the second tile. By analogy, the rendering of each tile is completed.

[0170] Based on the above-mentioned rendering pipeline, it can be understood that the implementation manner of the tile-based rendering can perform rendering in a tile-by-tile manner, and the rendering result of the tile can be stored in the GMEM, so that the access to the main memory can be reduced to a certain extent, thereby reducing the bandwidth consumption.

[0171] The above introduces two rendering technologies currently existing, in the actual implementation process, which rendering technology can be determined according to the drawing target. In an implementation mode, for example, when multiple objects need to be drawn in a single RenderPass (rendering canvas), the geometry is complex, the above-mentioned tile-based rendering can be used. And when a small number of triangles need to be drawn in a single RenderPass, the geometry is relatively simple, the above-mentioned direct rendering can be used. The embodiment does not limit the implementation of which rendering technology is used in the specific rendering process, which can be selected according to actual needs.

[0172] The rendering pipeline of the above two rendering technologies can be understood that the rendering method of direct rendering completely interacts with the main memory when processing rendering resources. And although the rendering method based on tiles makes certain improvements, it only temporarily stores the rendering results of a single tile in the GMEM, and the remaining rendering resources still need to interact with the main memory. Therefore, both methods inevitably interact frequently with the main memory, resulting in increased bandwidth and performance degradation of the device, and further causing device heating.

[0173] It can be understood that because the storage space of the GMEM is relatively small compared to the main memory, the rendering resources cannot be directly stored in the GMEM, which leads to the current related technology mainly relying on the main memory for rendering resource storage, and the problem of frequent interaction between the GPU and the main memory during the rendering process is inevitable.

[0174] Based on this, the present application proposes a new technical solution, which can specifically analyze the use of rendering resources in the rendering process. The use may, for example, indicate whether the GPU will continuously use the rendering resources. Then, according to the use of the rendering resources, some of the rendering resources are stored in the GMEM under the premise that the GMEM can store, which can greatly reduce the access of the GPU to the main memory, and thus effectively reduce the bandwidth consumption.

[0175] The technical solution provided by the present application can be applied to a terminal device. First, the terminal device is briefly introduced.

[0176] For example, FIG. 4 is a hardware structure schematic diagram of a terminal device provided by an embodiment of the present application.

[0177] The terminal device can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0178] The processor 110 can include one or more processing units. For example, in this embodiment, the processor 110 can include a CPU and a GPU. Different processing units can be independent devices or integrated into one or more processors.

[0179] The processor 110 can also be provided with a memory for storing instructions and data. In an implementation manner, the above-mentioned main memory can be a memory in the CPU, or the main memory can also be the internal memory 121 shown in FIG. 4, and the specific settings of the main memory in this embodiment are not limited.

[0180] In this embodiment, the terminal device can realize the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connected with the display screen 194 and the application processor. The GPU in this embodiment is used to perform mathematical and geometric calculations, so as to realize the processing of graphic rendering.

[0181] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the terminal device can include 1 or N display screens 194, and N is a positive integer greater than 1.

[0182] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative, and does not constitute a structural limitation on the terminal device. In some other embodiments of the present application, the terminal device can also use different interface connection modes or combinations of multiple interface connection modes in the above embodiments.

[0183] The software system of the terminal device can use a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present application take a system with a layered architecture as an example to exemplarily illustrate the software structure of the terminal device.

[0184] Fig. 5 is a schematic diagram of a software structure of a terminal device according to an embodiment of the present application.

[0185] As shown in Fig. 5, the layered architecture divides the software into several layers, each of which has a clear role and division of labor. The layers communicate with each other through interfaces. In some embodiments, the system can include an application layer, an application framework layer, a system library, a hardware abstraction layer (HAL), a driver layer, and a hardware layer.

[0186] It should be noted that the embodiments of the present application are exemplified by the Android system, and in other operating systems (such as the Harmony system, the IOS system, etc.), as long as the functions of the various functional modules are similar to the embodiments of the present application, the scheme of the present application can also be implemented.

[0187] The application layer can include a series of application packages. As shown in Fig. 5, the application packages can include camera, gallery, game, and other application programs.

[0188] It can be understood that there are usually a large number of rendering requirements in game applications. When the game application has rendering requirements, for example, rendering resources and rendering instructions can be submitted to the lower layer to enable the lower layer to perform corresponding rendering processing, and after the rendering is completed, the rendering result can be displayed in the game application. In actual implementation, any application can have rendering requirements, so the embodiments of the present application do not limit the application that submits rendering resources and rendering instructions, which depends on the actual needs of the application. The following takes the game application as an example for introduction, and the implementation of the remaining applications is similar.

[0189] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the application programs of the application layer. For example, as shown in Fig. 5, the application program framework layer includes some pre-defined functions. For example, it can include an activity manager, a window manager, a content provider, a view system, a resource manager, a notification manager, and a camera server unit, etc., which are not limited by the embodiments of the present application.

[0190] In addition, the system library can include a plurality of functional modules. For example: a surface manager, media libraries, a three-dimensional graphics processing library (such as: OpenGL ES), a 2D graphics engine (such as: SGL), etc. The Android runtime module can also be included in the system library.

[0191] Further, the hardware abstraction layer is used to abstract hardware, encapsulate Linux kernel drivers, and provide an interface upward, shielding the implementation details of the low-level hardware. In the example of FIG. 5, the HAL layer includes an audio HAL and a code software library. In actual implementation, the specific units included in the HAL layer can be extended according to actual needs, for example, an image HAL and a video HAL can also be included.

[0192] Further, the driver layer is a layer between hardware and software. The kernel layer at least includes display drivers, camera drivers, audio drivers, sensor drivers, charging drivers, and the like. In this embodiment, a command buffer is also included in FIG. 5. The command buffer is used to store rendering instructions submitted by a game application.

[0193] Further, the hardware layer can include the CPU and GPU described in this embodiment. The CPU and GPU are used to execute rendering instructions stored in the command buffer, thereby completing corresponding rendering tasks. It can be understood that, generally, rendering instructions are executed by the GPU, and the CPU can be responsible for performing some subsequent processing after rendering.

[0194] Meanwhile, in the process of executing rendering instructions by the GPU, the rendering instructions can involve some rendering resources, for example, the GPU needs to read rendering resources as inputs of the rendering instructions, thereby performing corresponding rendering processing. Or the rendering instructions can generate some rendering resources in the process of performing rendering processing. Therefore, the rendering resources corresponding to the rendering instructions can include the rendering resources submitted by the game application described above, and can also include rendering resources generated in the process of executing the rendering instructions.

[0195] It can be understood that the rendering resources corresponding to the rendering instructions need to be saved in a storage space, so as to facilitate subsequent reading or writing of the corresponding rendering resources.

[0196] When the rendering resources are saved in the main memory, it means that the GPU needs to read the rendering resources from the main memory or write the rendering resources into the main memory in the process of executing the rendering resources. Therefore, the problem of high bandwidth caused by frequent access of the GPU to the main memory is introduced.

[0197] When the rendering resources are saved in the GMEM, it means that the GPU can quickly and conveniently read the rendering resources from the GMEM or write the rendering resources into the GMEM in the process of executing the rendering resources, thereby avoiding frequent access of the GPU to the main memory.

[0198] However, the storage space of the GMEM is limited, and thus all rendering resources cannot be stored in the GMEM. In this application, how to determine to store the rendering resources in which storage space is introduced in detail with respect to the rendering resources.

[0199] On the basis of the above introduction, the rendering resource processing method provided in this application is introduced in detail as follows. In an implementation manner, the rendering resource processing method provided in this application includes the following three parts:

[0200] 1. Obtain the usage of the rendering resources.

[0201] 2. According to the usage of the rendering resources, the GPU saves the part of the rendering resources with acceleration demand in the GMEM.

[0202] 3. In the rendering process, the GPU interacts with the GMEM and the main memory to execute the rendering instructions, so as to obtain the rendering result.

[0203] The specific implementation of the above three parts is introduced as follows:

[0204] First, the implementation of obtaining the usage of the rendering resources is described in combination with FIG. 6. FIG. 6 is a processing flow diagram of a rendering instruction provided in an embodiment of this application.

[0205] The software architecture layers shown in FIG. 6 have been described in the above embodiment, and thus will not be described herein. Referring to FIG. 6, when the first application (or also referred to as the first application program) has a rendering demand, for example, the first application has first graphics data to be rendered, the first application can submit rendering resources and rendering instructions to the lower layer.

[0206] Taking the first application as a game application as an example, in the process of game rendering, the game engine can submit rendering resources and rendering instructions through a graphics API (such as OpenGL / GLES / Vulkan / Metal / WebGL / DirectX 3D, etc.). In the actual implementation process, the first application can be any application with a rendering demand, and the embodiment does not limit this.

[0207] The rendering resource can at least include data, textures, parameters and the like required for rendering. The rendering resource usually needs to be saved in a certain form. For example, the data required for rendering can be saved in vertex attributes, UBOs or SSBOs, the textures can be saved in texture objects provided by a graphics API (Application Programming Interface), and the parameters can be stored in descriptor sets / Uniforms. Further, the content that the rendering resource can specifically include is described in detail in the concept introduction in the beginning of the embodiment, and thus the content introduced above can also be referred to for understanding the rendering resource.

[0208] Currently, modern graphics APIs all adopt an asynchronous command execution mechanism, that is, after the first application generates rendering instructions, the rendering instructions are first transmitted to the instruction buffer of the driver layer. The first application can directly transmit the rendering instructions to the instruction buffer of the driver layer, or the first application can transmit the rendering instructions to the instruction buffer of the driver layer after processing and forwarding by at least one intermediate layer in the application framework layer, the system library and the HAL layer, which is not limited in the embodiment.

[0209] Referring to FIG. 6, after receiving the rendering instructions, the driver layer temporarily stores the rendering instructions in the instruction buffer, such as instructions 1, 2, 3 and 4 stored in the instruction buffer shown in FIG. 6, which are the rendering instructions currently introduced.

[0210] After the driver layer receives some special instructions or when the number of rendering instructions stored in the instruction buffer exceeds a preset number, the driver layer triggers the instructions stored in the instruction buffer to be issued to the hardware layer, so that the GPU in the hardware layer executes specific rendering instructions to complete the corresponding rendering task.

[0211] It can be determined from the above introduction that all rendering instructions included in a rendering task will pass through the instruction buffer, and each rendering instruction itself will indicate which rendering resources it needs to obtain and output which rendering resources. Therefore, the rendering instructions can be analyzed based on the instruction buffer in the present application to obtain the resource usage information corresponding to the rendering instructions.

[0212] The resource usage information can include rendering resources associated with the rendering instructions. The rendering resources associated with the rendering instructions can include the rendering resources obtained by the rendering instructions and the rendering resources output by the rendering instructions, which can be understood as having an association relationship with the rendering instructions. Alternatively, the rendering resources associated with the rendering instructions can also be described as rendering resources corresponding to the rendering instructions, and the meanings of the two descriptions are actually the same.

[0213] In addition, the resource usage information can further include a usage type of the associated rendering resource by the rendering instruction, where the usage type can be a read type or a write type. When the rendering instruction acquires the rendering resource, the usage type of the rendering instruction for the rendering resource is the read type, and when the rendering instruction generates the rendering resource, the usage type of the rendering instruction for the rendering resource is the write type.

[0214] It should be further noted that although the number of instructions that can be stored in the instruction buffer is limited, after the driving layer delivers the rendering instructions stored in the instruction buffer to the hardware layer, new rendering instructions included in a rendering task continue to be written into the instruction buffer. Therefore, each rendering instruction included in a rendering task is stored in the instruction buffer, and the overall analysis of the plurality of rendering instructions included in the rendering task can be comprehensively and effectively implemented based on the instruction buffer, so as to obtain the resource usage information corresponding to each rendering instruction of the rendering task.

[0215] The implementation of obtaining the resource usage information of the rendering instruction will be introduced below in combination with specific examples.

[0216] For example, it is assumed that the rendering instruction 1 indicates to acquire the rendering resource a, and the rendering resource b is obtained after the rendering resource a is processed. Therefore, the rendering resources involved (or associated) by the rendering instruction 1 include the rendering resource a and the rendering resource b, the usage type of the rendering instruction 1 for the rendering resource a is the read type, and the usage type of the rendering instruction 1 for the rendering resource b is the write type.

[0217] For another example, it is assumed that the rendering instruction 2 indicates to acquire the rendering resource b, and the rendering resource b is adjusted, but a new rendering resource can not be generated. Therefore, the rendering resources involved (or associated) by the rendering instruction 2 include only the rendering resource b, and the usage type of the rendering instruction 2 for the rendering resource b is the read type.

[0218] Based on the above-described analysis process, the plurality of rendering instructions included in a rendering task are analyzed, and at least one rendering resource associated with each rendering instruction and the usage type of each rendering instruction for the corresponding rendering resource can be obtained.

[0219] In the embodiment, the analysis of the rendering instruction based on the instruction buffer in the driving layer can comprehensively and effectively implement the acquisition of the usage of the rendering resource.

[0220] Secondly, the implementation of saving part of the rendering resources in the GMEM according to the usage of the rendering resources will be introduced in combination with FIG. 7 to FIG. 9. FIG. 7 is a logical judgment diagram for saving the rendering resources provided by the embodiment of the present application, FIG. 8 is a usage diagram of the rendering resources in the instruction buffer provided by the embodiment of the present application, and FIG. 9 is another usage diagram of the rendering resources in the instruction buffer provided by the embodiment of the present application.

[0221] As shown in FIG. 7, the method comprises:

[0222] S701, judging whether the usage type of the first rendering instruction for the first rendering resource is a write type, if yes, executing S701, and if no, executing S706.

[0223] Based on the analysis process introduced in the above embodiment, it can be determined that the determination process introduced in the embodiment can be executed for each rendering instruction corresponding to at least one rendering resource, so as to write the rendering resource into the corresponding storage position.

[0224] In the embodiment, the first rendering resource corresponding to the first rendering instruction is taken as an object for introduction, wherein the first rendering instruction can be any rendering instruction, and the first rendering resource can be any rendering resource. It can be understood that the determination manners of any rendering resource corresponding to each rendering instruction are similar.

[0225] The usage type of the rendering instruction for the rendering resource can be divided into a read type and a write type, and the rendering resources of the read type and the write type can have different processing manners when the rendering resource is written. Therefore, it can be firstly judged whether the usage type of the first rendering instruction for the first rendering resource is a write type.

[0226] Alternatively, it can also be judged whether the usage type of the first rendering instruction for the first rendering resource is a read type, and the implementation is equivalent, and the logic of the subsequent steps can be replaced accordingly.

[0227] Exemplarily, when the usage type of the first rendering instruction for a certain rendering resource is a write type, the rendering resource can also be called a second rendering resource. It should be understood that whether it is the first rendering resource or the second rendering resource, it is only a designation of a name, and does not particularly refer to a certain rendering resource, and the subsequent processing manners for each rendering resource are similar.

[0228] S702, judging whether there is a third rendering instruction for reading the first rendering resource, if yes, executing S703, and if no, executing S705.

[0229] In an implementation, if the usage type of the first rendering resource by the first rendering instruction is a write type, it can be determined that the first rendering instruction needs to generate the first rendering resource.

[0230] In the field of rendering, when a rendering instruction needs to generate a certain rendering resource, the first rendering resource is usually an attachment of a framebuffer. Therefore, the judgment performed in S701 can also be whether the first rendering resource is an attachment of a framebuffer. If the first rendering resource is an attachment of a framebuffer, it means that the usage type of the first rendering resource by the first rendering instruction is a write type.

[0231] Regardless of the implementation, in the case where the first rendering instruction indicates to generate the first rendering resource, it can be further determined whether there is a third rendering instruction that needs to read the first rendering resource, that is, whether there is a rendering instruction that is associated with the first rendering resource and has a read type of usage of the first rendering resource. The instruction order of the third rendering instruction is after the first rendering instruction.

[0232] In an implementation, the third rendering instruction is adjacent to the first rendering instruction in the instruction order of the plurality of rendering instructions, that is, the third rendering instruction can be the next instruction of the first rendering instruction. In this implementation, because the third rendering instruction is a certain instruction, the judgment logic of the current step can also be described as: judging whether the third rendering instruction reads the first rendering resource.

[0233] For example, it can be understood with reference to FIG. 8. It is assumed that the first rendering instruction is rendering instruction 1, and it is assumed that rendering instruction 1 needs to obtain rendering resource a and generate rendering resource b after processing rendering resource a. It can be determined that the usage type of rendering resource b by rendering instruction 1 is a write type.

[0234] In addition, the next instruction of rendering instruction 1 is rendering instruction 2, and rendering instruction 2 is the third rendering instruction. With reference to FIG. 8, rendering instruction 2 needs to obtain rendering resource b and generate rendering resource c after rendering processing of rendering resource b. Therefore, it can be determined that the usage type of rendering resource b by rendering instruction 2 is a read type. Then the judgment result corresponding to the current step is: it is determined that there is a third rendering instruction that reads the first rendering resource, or it is described that the third rendering instruction reads the first rendering resource.

[0235] In another implementation, the number of rendering instructions between the third rendering instruction and the first rendering instruction is less than or equal to a third threshold n3, where n3 is an integer greater than or equal to 1. That is, the current step needs to determine whether there is a third rendering instruction that reads the first rendering resource in the n3 rendering instructions sequentially located after the first rendering instruction.

[0236] For example, referring to FIG. 9, assume that the first rendering instruction is rendering instruction 1, which is similar to the rendering instruction 1 introduced in FIG. 8, and it can be determined that the use type of rendering instruction 1 for rendering resource b is the write type.

[0237] Meanwhile, assume that n3 = 3, then the n rendering instructions sequentially located after rendering instruction 1 and adjacent to rendering instruction 1 include rendering instruction 2, rendering instruction 3 and rendering instruction 4 shown in FIG. 9, and it can be determined whether there is a rendering instruction that reads rendering resource a in rendering instruction 2, rendering instruction 3 and rendering instruction 4. Referring to FIG. 9, it can be determined that rendering instruction 3 reads rendering resource a, and thus there can be a third rendering instruction that needs to read rendering resource a, i.e., rendering instruction 3.

[0238] It can be understood that in the implementation introduced in the current embodiment, there can be multiple third rendering instructions, for example, in the case that rendering instruction 4 also needs to obtain rendering resource a, rendering instruction 4 is actually the third rendering instruction introduced above. In the current example, the determination result corresponding to the current step is that it is determined that there is a third rendering instruction that reads the first rendering resource.

[0239] No matter which of the above cases, the current step is essentially to determine whether there is a third rendering instruction that reads the first rendering resource sequentially adjacent to the first rendering instruction after the first rendering instruction generates the first rendering resource, and the specific interval relationship between the third rendering instruction and the first rendering instruction in the instruction sequence can be selected and set according to actual needs.

[0240] It can be understood that if the third rendering instruction that is sequentially adjacent to the first rendering instruction reads the first rendering resource after the first rendering instruction generates the first rendering resource. Then storing the first rendering resource in the GMEM can effectively save the bandwidth consumed by writing the first rendering resource to the main memory and then reading it from the main memory. In this case, it can also be understood that there is an acceleration demand for the first rendering resource.

[0241] Therefore, in the current step, if it is determined that the third rendering instruction reads the first rendering resource, it can be determined that the first rendering resource is a rendering resource with an acceleration demand, and in an implementation, for example, the first rendering resource can be directly stored in the GMEM, thereby saving bandwidth consumption.

[0242] Further, because the storage space of the GMEM is limited, before directly storing the first rendering resource in the GMEM, step S703 can be performed first to determine whether the first rendering resource can be written into the GMEM.

[0243] Alternatively, if there is no third rendering instruction that needs to read the first rendering resource after the first rendering instruction generates the first rendering resource, storing the first rendering resource in the GMEM only occupies storage space and does not help reduce bandwidth. Therefore, it can be determined that there is no acceleration demand for the first rendering resource, so in this embodiment, the first rendering resource can be directly written into the main memory.

[0244] S703, determine whether the resource amount of the first rendering resource is less than or equal to the remaining storage space of the GMEM, if yes, perform S704, if no, perform S705.

[0245] S704, write the first rendering resource into the GMEM after the first rendering instruction generates the first rendering resource.

[0246] S705, write the first rendering resource into the main memory after the first rendering instruction generates the first rendering resource.

[0247] The steps of S703-S705 are introduced as follows.

[0248] Because the storage space of the GMEM is limited, before finally determining the write position of the first rendering resource, it can be further determined whether the resource amount of the first rendering resource is less than or equal to the remaining storage space of the GMEM, and then determine whether the first rendering resource can be written into the GMEM.

[0249] In the case that the resource amount of the first rendering resource is less than or equal to the remaining storage space of the GMEM, the first rendering resource can be written into the GMEM.

[0250] In the case that the resource amount of the first rendering resource is greater than the remaining storage space of the GMEM, the first rendering resource cannot be stored in the GMEM, so the first rendering resource can be written into the GMEM.

[0251] And it can be understood that the current S704 and S705 are execution steps under the branch of "the use type of the first rendering instruction for the first rendering resource is a write type", so it can be determined that the first rendering resource is actually generated by the first rendering instruction, and therefore, whether the first rendering resource is written into the GMEM or the main memory, the action of writing the first rendering resource cannot be completed before the first rendering instruction is executed.

[0252] Therefore, after the first rendering resource is generated by executing the first rendering instruction, the first rendering resource is written into the GMEM or the main memory. In an implementation manner, an indication field corresponding to each rendering resource can be set to mark the writing position of each rendering resource, so that when the rendering resource is applied subsequently, the indication field can be used to quickly and efficiently determine the storage position to which the rendering resource should be written, which can be the GMEM or the main memory.

[0253] That is, the indication field of the rendering resource is used to indicate the writing position of the rendering resource. In an implementation manner, when the indication field is a first value, the indication field is used to indicate that the writing position of the rendering resource is the GMEM. When the indication field is a second value, the indication field is used to indicate that the writing position of the rendering resource is the main memory. The first value can be 1, and the second value can be 0. Alternatively, the first value and the second value can also be other values as long as the corresponding indication can be realized, which is not limited in the embodiment.

[0254] Then, corresponding to the above-mentioned steps, in an implementation manner, after the judgment step of S703 or S702 is executed, if the judgment result is yes, it can be determined that the indication field corresponding to the first rendering resource is the first value. Then, after the first rendering resource is generated by the first rendering instruction, the first rendering resource is written into the GMEM according to the indication field corresponding to the first rendering resource.

[0255] Alternatively, after the judgment step of S703 or S702 is executed, if the judgment result is no, it can be determined that the indication field corresponding to the first rendering resource is the first value. Then, after the first rendering resource is generated by the first rendering instruction, the first rendering resource is written into the main memory according to the indication field corresponding to the first rendering resource.

[0256] And the above-mentioned "remaining storage space of the GMEM" is further introduced, which has two implementation manners.

[0257] The first implementation manner is that the remaining storage space of the GMEM is the size of the remaining storage space of the GMEM at the moment when the judgment step of S703 is executed.

[0258] The second implementation manner is that the remaining storage space of the GMEM is the size of the remaining storage space of the GMEM after a rendering instruction preceding the first rendering instruction is executed.

[0259] It can be determined according to the implementation of obtaining the resource usage information that the GPU can analyze the resource usage information of each rendering instruction included in the rendering task, and then the GPU can determine which rendering instructions each rendering resource needs to read and which rendering instructions will be generated. According to the description of the current embodiment, the GPU can also determine whether each rendering resource is to be written into the GMEM or the main memory. For the rendering resource written into the GMEM, the GPU can also determine that the rendering resource is used up after the execution of the rendering instruction, and the GPU can remove the rendering resource from the GMEM to avoid unnecessary occupation of the storage space of the GMEM. That is, the remaining storage space of the GMEM actually dynamically changes with the execution of the rendering instruction.

[0260] According to the above information, before the execution of the rendering instruction, the GPU can actually know which rendering resources are to be written into the GMEM and which rendering resources are to be removed from the GMEM after the execution of each rendering instruction, and thus the GPU can determine the real-time remaining storage space of the GMEM in advance.

[0261] In this implementation, the GPU determines whether the first rendering resource can be written into the GMEM according to the size of the real-time remaining storage space of the GMEM after the execution of the previous rendering instruction of the first rendering instruction, so that the dynamic scheduling of the storage space of the GMEM can be implemented, and the utilization rate of the storage space of the GMEM can be effectively improved.

[0262] In this embodiment, for the rendering resource of the write type, after the first rendering instruction generates the first rendering resource, if it is determined that there is a third rendering instruction that needs to read the first rendering resource, and the first rendering resource can be stored in the GMEM, the first rendering resource is written into the GMEM. In this way, the first rendering resource can be avoided to be written into the main memory and then read from the main memory in a short time, which causes additional bandwidth consumption. At the same time, when the above condition is not met, the first rendering resource is stored in the main memory, which can also avoid storing the rendering resource without acceleration demand in the GMEM, so as to reduce the bandwidth consumption of accessing the main memory as much as possible.

[0263] It should be understood that the step of S703 is optional. If the step of S703 does not exist, after the execution of S702, the yes branch can correspond to the execution of S704, and the no branch can execute S705, and the implementation is similar to the above description, which will not be described here.

[0264] S706, determining whether there is a second rendering instruction for reading the first rendering resource, if yes, performing S707, if no, performing S705.

[0265] Corresponding to another branch of the S701 determination result, if the use type of the first rendering instruction for the first rendering resource is the read type, it can be determined that the first rendering instruction needs to read the first rendering resource.

[0266] As can be understood from the above description of S702, the judgment performed by S701 can also be whether the first rendering resource is an attachment of the framebuffer. If the first rendering resource is not an attachment of the framebuffer, it means that the use type of the first rendering instruction for the first rendering resource is the read type.

[0267] In the case where the first rendering instruction needs to read the first rendering resource, the embodiment can further determine whether there is a second rendering instruction for reading the first rendering resource. The implementation is similar to the implementation of S702, but there are some differences in understanding. The following describes the understanding of the steps under the current branch.

[0268] In one implementation, the second rendering instruction is adjacent to the first rendering instruction in the instruction order of the plurality of rendering instructions, that is, the second rendering instruction can be the next instruction of the first rendering instruction, and the implementation is similar to the above embodiment.

[0269] For example, referring to FIG. 9, it is assumed that the first rendering instruction is rendering instruction 1, and it is assumed that rendering instruction 1 needs to obtain rendering resource a and generate rendering resource b after processing rendering resource a. It can be determined that the use type of rendering instruction 1 for rendering resource a is the read type.

[0270] In addition, the next instruction of rendering instruction 1 is rendering instruction 2, and rendering instruction 2 is the second rendering instruction. Referring to FIG. 9, rendering instruction 2 needs to obtain rendering resource a. Then the determination result corresponding to the current step is: it is determined that there is a second rendering instruction for reading the first rendering resource, or it is described that the second rendering instruction reads the first rendering resource.

[0271] In another implementation, the number of rendering instructions between the second rendering instruction and the first rendering instruction is less than or equal to a first threshold n1, where n1 is an integer greater than or equal to 1, and the implementation is similar to the above embodiment.

[0272] Similarly, referring to FIG. 9, it is assumed that the first rendering instruction is rendering instruction 1, and it can be determined that the use type of rendering instruction 1 for rendering resource a is the read type.

[0273] Meanwhile, assuming that n1=3, then after the rendering instruction 1, and sequentially adjacent to the rendering instruction 1, the n rendering instructions include the rendering instruction 2, the rendering instruction 3 and the rendering instruction 4 shown in Fig. 9, thus it can be determined whether there is a rendering instruction to read the rendering resource a in the rendering instruction 2, the rendering instruction 3 and the rendering instruction 4. It can be determined with reference to Fig. 9 that the rendering instruction 3 and the rendering instruction 4 read the rendering resource a, thus there can be the second rendering instruction to read the rendering resource a, i.e. the rendering instruction 3 and the rendering instruction 4.

[0274] No matter which case is introduced above, then the current step is essentially to determine whether there is a second rendering instruction to also read the first rendering resource sequentially adjacent to the first rendering instruction to read the first rendering resource.

[0275] It can be understood that if the second rendering instruction to read the first rendering resource is sequentially adjacent to the first rendering instruction to read the first rendering resource, then storing the first rendering resource in the GMEM can effectively save the bandwidth consumed by repeatedly reading the first rendering resource from the main memory by the second rendering resource. In this case, it can also be understood that the first rendering resource exists the acceleration demand.

[0276] Therefore, in the current step, if it is determined that the second rendering instruction reads the first rendering resource, it can be determined that the first rendering resource is the rendering resource with the acceleration demand, and in an implementation manner, the first rendering resource can be directly stored in the GMEM, thereby saving the bandwidth consumption.

[0277] Further, because the storage space of the GMEM is limited, before the first rendering resource is directly stored in the GMEM, step S707 can be first performed to determine whether the first rendering resource can be written into the GMEM.

[0278] Or, if there is no second rendering instruction to read the first rendering resource in the case that the first rendering instruction reads the first rendering resource, then storing the first rendering resource in the GMEM only occupies the storage space, and does not help to reduce the bandwidth. Therefore, it can be determined that the first rendering resource does not exist the acceleration demand, so the first rendering resource can be directly written into the main memory in the embodiment.

[0279] It can be understood that the implementation manner is from the continuity of reading the first rendering resource, and considers whether the first rendering resource exists the acceleration demand, and in the implementation manner, the first rendering resource to be continuously read by the GPU can be stored in the GMEM, thereby avoiding the bandwidth consumption caused by the continuous access of the GPU to the main memory to obtain the same rendering resource.

[0280] In another implementation, if the use type of the first rendering resource in the first rendering instruction is a read type, it can be further determined whether the number of rendering instructions reading the first rendering resource is greater than a preset threshold. That is, the number of rendering instructions corresponding to the first rendering resource in the plurality of rendering instructions is greater than or equal to a second threshold n2, wherein the plurality of rendering instructions includes the second rendering instruction.

[0281] It can be understood that if the number of rendering instructions reading the first rendering resource is greater than the second threshold, it can be determined that the GPU needs to access the first rendering resource multiple times in the current rendering process. Then storing the first rendering resource in the GMEM can effectively save the bandwidth consumed by the GPU reading the first rendering resource from the main memory multiple times. In this case, it can also be understood that the first rendering resource has acceleration demand.

[0282] Therefore, if it is determined that the number of rendering instructions reading the first rendering resource is greater than the second threshold, it can be determined that the first rendering resource has acceleration demand, and the first rendering resource can be considered to be stored in the GMEM to save bandwidth consumption. Then step S703 is further executed to determine the storage location of the first rendering resource.

[0283] Alternatively, if it is determined that the number of rendering instructions reading the first rendering resource is not greater than the second threshold, it means that the GPU does not need to read the first rendering resource multiple times, and storing the first rendering resource in the GMEM only occupies storage space and does not help to reduce bandwidth. Therefore, it can be determined that the first rendering resource does not have acceleration demand, and therefore the storage location of the first rendering resource in this embodiment is determined to be the main memory.

[0284] It can be understood that the above implementation is from the perspective of the number of times of reading the first rendering resource to consider whether the first rendering resource has acceleration demand. In this implementation, the first rendering resource that needs to be read multiple times by the GPU can be stored in the GMEM, thereby avoiding the bandwidth consumption caused by the GPU accessing the main memory multiple times to obtain the same rendering resource.

[0285] Alternatively, the two implementation modes introduced above can also be combined. If the use type of the first rendering resource in the first rendering instruction is a read type, it can be further determined whether the number of rendering instructions continuously reading the first rendering resource is greater than a preset threshold. The subsequent implementation is similar to the above introduction. If yes, S708 is executed, and if no, S709 is executed.

[0286] S707, determining whether the resource amount of the first rendering resource is less than or equal to the remaining storage space of the GMEM, if yes, S708 is executed, and if no, S708 is executed.

[0287] S708, write the first rendering resource into the GMEM before the first rendering instruction reads the first rendering resource.

[0288] S709, write the first rendering resource into the main memory before the first rendering instruction reads the first rendering resource.

[0289] Wherein, S707-S709 are similar to the implementation manners of S703-S705 described above, and only the differences will be introduced below.

[0290] It can be understood that the current S708 and S709 belong to the execution steps under the branch of "the use type of the first rendering instruction for the first rendering resource is the reading type", and it can be determined that the first rendering resource is the rendering resource that the first rendering instruction needs to read. If the first rendering resource is written into the GMEM too early, the first rendering resource actually only occupies the storage space of the GMEM before the first rendering instruction is executed, which does not help to reduce the bandwidth.

[0291] Therefore, in the embodiment, the first rendering resource can be written into the GMEM or the main memory before the first rendering instruction reads the first rendering resource (which can also be understood as the moment when the first rendering instruction is executed or the moment when the first rendering instruction is executed) after the first rendering instruction is executed, so as to improve the utilization rate of the storage space of the GMEM.

[0292] Then in an implementation manner, a corresponding indication field can be set for each rendering resource, and the implementation manner of the indication field is similar to that described in the above embodiment, which will not be described here.

[0293] Then corresponding to the steps described above, in an implementation manner, after the judgment step of S707 or S706 is executed, if the judgment result is yes, it can be determined that the indication field corresponding to the first rendering resource is the first value. Then, before the first rendering instruction reads the first rendering resource, the first rendering resource is written into the GMEM according to the indication field corresponding to the first rendering resource.

[0294] Or, after the judgment step of S707 or S706 is executed, if the judgment result is no, it can be determined that the indication field corresponding to the first rendering resource is the first value. Then, before the first rendering instruction reads the first rendering resource, the first rendering resource is written into the main memory according to the indication field corresponding to the first rendering resource.

[0295] In the embodiment, for the rendering resource of the read type, if it is determined that there are continuous multiple rendering instructions that need to read the first rendering resource (or if it is determined that the number of rendering instructions that need to read the first rendering resource is large), and the first rendering resource can be stored in the GMEM, it is determined that the storage location of the first rendering resource is the GMEM. In this way, the first rendering resource that needs to be continuously read (or read multiple times) can be avoided from being written into the main memory, and the subsequent additional bandwidth consumption caused by resource reading from the main memory can be avoided. At the same time, when the above condition is not met, the first rendering resource is stored in the main memory, and the rendering resource that does not have acceleration demand can also be avoided from being stored in the GMEM, so as to cause unnecessary occupation of the storage space of the GMEM, thereby reducing the bandwidth consumption of accessing the main memory as much as possible.

[0296] It needs to be understood that the step of S707 is optional. If the step of S707 does not exist, after the execution of S706, the yes branch can correspond to the execution of S708, and the no branch can execute S708, and the implementation manner is similar to the above introduction, which will not be described herein.

[0297] In the above embodiment, the rendering resource that needs to be accelerated is stored in the GMEM, so that the GPU can read the required rendering resource from the GMEM to realize the rendering data. Further, in order to ensure the effective utilization of the storage space in the GMEM, when the rendering resource stored in the GMEM is used up, the part of the rendering resource can be deleted from the GMEM.

[0298] Based on the above introduction, it can be determined that when the use type of the first rendering instruction for the first rendering resource is the write type, if there is a third rendering instruction that reads the first rendering resource, it can be determined that the first rendering resource has acceleration demand, and then when the third rendering instruction reads the first rendering resource, it can be determined that the first rendering resource is used up, and therefore the part of the rendering resource can be deleted from the GMEM, so as to save the occupation of the storage space of the GMEM.

[0299] Or, when the use type of the first rendering instruction for the first rendering resource is the read type, if there is a second rendering instruction that also reads the first rendering resource, it can be determined that the first rendering resource is used up after the first rendering instruction and the second rendering instruction read the first rendering resource, and then the part of the rendering resource is deleted from the GMEM, so as to save the occupation of the storage space of the GMEM.

[0300] Further, because the CPU can also read the rendering resource from the GMEM in some cases, before deleting the first rendering resource from the GMEM, it can be first determined whether a read instruction for reading the first rendering resource is received by the CPU. If the read instruction is received, the first rendering resource can be first written back to the main memory, and then the first rendering resource can be deleted. If the read instruction is not received, the first rendering resource can be directly deleted, so as to ensure the correctness of the read logic between the CPU and the GPU.

[0301] In the embodiments introduced in FIGS. 7-9, it can be determined whether the rendering resource is stored in the GMEM according to the usage of the rendering resource. This is because the storage space of the GMEM is limited, and thus it is necessary to consider whether the rendering resource has an acceleration demand, and then the rendering resource having the acceleration demand is stored in the GMEM.

[0302] However, it can be understood that compared with the way of storing the rendering resource in the main memory, the rendering resource stored in the GMEM can certainly save bandwidth. Then for the rendering resource whose data amount is very small, that is, the rendering resource which does not occupy much storage space, it can also not be necessary to determine whether the rendering resource has an acceleration demand, that is, it is not necessary to perform the steps S701, S702 and S706, and the steps S703-S705 are directly performed, so as to determine the storage position of the rendering resource.

[0303] For example, still taking the first rendering resource as an example, when the data amount of the first rendering resource is less than or equal to the preset size, the steps S703-S705 can be directly performed, so as to determine the storage position of the first rendering resource. When the data amount of the first rendering resource is greater than the preset size, the complete judgment process of the embodiment of FIG. 7 is still adopted to determine the storage position of the first rendering resource.

[0304] Alternatively, the data amount of different types of rendering resources is different. For example, the data amount of the rendering resource of the texture type is usually large, and the data amount of the rendering resource of the attribute type is usually small. Therefore, the GPU can also determine the specific way for determining the storage position of the rendering resource according to the resource type of the rendering resource. The rendering resource of the attribute type can also be understood as the attribute parameter of the material, which can include smoothness, roughness, reflection coefficient, glossiness and scattering coefficient, etc.

[0305] For example, still taking the first rendering resource as an example, when the resource type of the first rendering resource is the attribute type, the steps S703-S705 can be directly executed to determine the storage location of the first rendering resource. When the resource type of the first rendering resource is the texture type, the complete judgment process of the embodiment of FIG. 7 is still used to determine the storage location of the first rendering resource.

[0306] In actual implementation, which types of rendering resource have small data volume and only need to execute the steps S703-S705, and which types of rendering resource have large data volume and need to execute the complete judgment process of the embodiment of FIG. 7 can be set according to actual requirements.

[0307] Based on this implementation mode, rendering resources that are not continuously used can also be quickly read and written, and the storage space of the GMEM is not excessively occupied, thereby reducing the bandwidth of the GPU accessing the main memory.

[0308] The current part introduces the implementation mode of how to determine the storage location of the rendering resource. It can be understood that for each rendering resource involved in each rendering instruction included in a rendering process, the above-mentioned process is executed to determine the storage location corresponding to each rendering resource.

[0309] In addition, because the storage space of the GMEM is limited, it is assumed that the storage locations of x rendering resources are finally determined to be the GMEM, where x is an integer greater than or equal to 1. If the total amount of resources of the x rendering resources is less than or equal to the remaining storage space of the GMEM, the x rendering resources can be stored in the GMEM, thereby saving the bandwidth consumption to the greatest extent. The remaining storage space introduced in each embodiment of the present application is similar to the concept of the remaining storage space introduced in the above-mentioned embodiments, and will not be described in detail.

[0310] However, if the total amount of resources of the x rendering resources is greater than the remaining storage space of the GMEM, the x rendering resources cannot be stored in the GMEM, and certain screening is needed.

[0311] In an implementation mode, for example, the x rendering resources can be combined in any manner to determine y rendering resources whose total amount of resources is the largest and does not exceed the remaining storage space of the GMEM, and then the y rendering resources are stored in the GMEM, so that the rendering resources with larger total amount of resources can be stored in the GMEM as much as possible under the premise that the GMEM can store, thereby saving the bandwidth consumption of the GMEM accessing the main memory as much as possible.

[0312] After introducing the implementation of determining the storage location of the GMEM, the last part, i.e., the implementation of the interaction of the GPU with the GMEM and the main memory to render resources to execute rendering instructions in the rendering process, will be described in combination with FIGS. 10-14. FIG. 10 is an implementation diagram for reading and writing rendering resources according to an embodiment of the present application, FIG. 11 is an implementation diagram for reading and writing rendering resources according to an embodiment of the present application, FIG. 12 is an implementation diagram for reading and writing rendering resources according to an embodiment of the present application, FIG. 13a is an implementation diagram for reading and writing rendering resources according to an embodiment of the present application, FIG. 13b is an implementation diagram for reading and writing rendering resources according to an embodiment of the present application, FIG. 14a is an implementation diagram for reading and writing rendering resources according to an embodiment of the present application, and FIG. 14b is an implementation diagram for reading and writing rendering resources according to an embodiment of the present application.

[0313] As can be determined from the above introduction, there are two rendering mechanisms at present, i.e., direct rendering and rendering based on tiles, and the implementation of reading rendering resources to execute rendering instructions in the two rendering mechanisms will be introduced below.

[0314] First, the implementation of direct rendering will be described in combination with FIGS. 10 and 11. In order to better understand the beneficial effects of the present application, FIG. 10 introduces an implementation manner of storing rendering resources in the main memory, and FIG. 11 introduces an implementation manner of storing part of the rendering resources in the GMEM.

[0315] In the implementation manner of direct rendering, the manner adopted is that the GPU executes the rendering instruction of each submitted graphics object (or referred to as a primitive, and the following introduction takes a triangle as an example) by the first application, thereby completing the drawing of the triangle (i.e., the graphics object).

[0316] It can be understood that one rendering instruction is used to indicate the drawing of one triangle. Meanwhile, a rendering pass needs to be constructed in the rendering process, and in the mechanism of direct rendering, one rendering pass is used to draw one triangle. One rendering pass can be understood as a canvas, and thus the current implementation can also be understood as drawing one triangle on one canvas.

[0317] Then in the example of FIG. 10, it is assumed that the rendering instruction 1 is used to indicate the drawing of a certain triangle, and then the GPU executes the rendering instruction 1 based on the rendering pass 1 to draw the triangle. Meanwhile, it is assumed that the above example is continued, and it is assumed that the rendering instruction 1 needs to perform corresponding processing on the rendering resource a, thereby generating the rendering resource b.

[0318] Because in the prior art, the rendering resources are stored in the main memory, as shown in FIG. 10, the GPU needs to read the rendering resource a from the main memory, and then execute the rendering instruction 1, and then the rendering instruction 1 generates the rendering resource b. The GPU writes the rendering resource b into the main memory. Then, the rendering instruction 2 needs to process the rendering resource b, and then the GPU needs to read the rendering resource b from the main memory.

[0319] The technical solution of the present application can be understood with reference to FIG. 11. It is assumed that the rendering instruction 1 needs to process the rendering resource a, and thus generates the rendering resource b. It is also assumed that, through the processing of the above embodiment, it is determined that the storage location of the rendering resource a is the main memory, and the storage location of the rendering resource b is the GMEM.

[0320] Then, as shown in FIG. 11, the GPU can read the rendering resource a from the main memory, and then execute the rendering instruction 1 to generate the rendering resource b. Then, the GPU writes the rendering resource b into the GMEM. Then, the rendering instruction 2 needs to process the rendering resource b, and the GPU can directly read the rendering resource b from the GMEM.

[0321] Then, by comparing FIG. 10 and FIG. 11, it can be understood that, compared with the implementation manner of directly storing the rendering resource in the main memory, the technical solution of the present application can save the bandwidth consumption of writing the rendering resource b into the main memory and reading the rendering resource b from the main memory. When the technical solution of the present application covers more rendering resources for acceleration processing, the bandwidth consumption of the GPU accessing the main memory can be effectively reduced.

[0322] Secondly, the implementation based on tile rendering will be described in combination with FIG. 12 to FIG. 14. In order to better understand the beneficial effects of the present application, FIG. 12 introduces an implementation manner of storing the rendering resource in the main memory, and FIG. 13 and FIG. 14 introduce an implementation manner of storing part of the rendering resource in the GMEM.

[0323] In the implementation manner based on tile rendering, the first application can submit rendering instructions of multiple triangles, and the multiple triangles jointly constitute a graphical object to be rendered. It can be understood that one rendering instruction is used to indicate the drawing of one triangle. In the rendering process, a rendering pass also needs to be constructed. In the mechanism based on tile rendering, one rendering pass is used to draw multiple triangles. One rendering pass can be understood as a canvas, and thus the current implementation can also be understood as drawing multiple triangles on one canvas.

[0324] In the embodiment, the GPU divides the canvas into multiple tiles, and then draws each tile. When drawing each tile, the GPU executes the drawing instructions of the triangles covered by the tile, so as to draw the triangles covered by the tile.

[0325] For example, in the example of FIG. 12, it is assumed that the rendering instruction 1 indicates drawing of triangle A, the rendering instruction 2 indicates drawing of triangle B, the rendering instruction 3 indicates drawing of triangle C, the rendering instruction 4 indicates drawing of triangle D, and the rendering instruction 5 indicates drawing of triangle E. It is assumed that the GPU executes the rendering instruction 1 to the rendering instruction 5 based on the rendering pass 1, so as to draw the triangles A to E.

[0326] Meanwhile, it is assumed that the GPU divides the canvas into 7*3 tiles shown in FIG. 12, wherein the tile in the first row and the first column covers the triangles A and B, the tile in the second row and the first column covers the triangles A, B and C, and the triangle coverage of the remaining tiles can be determined with reference to FIG. 12, which will not be described herein.

[0327] Taking the tile in the first row and the first column as an example, when drawing the tile, the GPU executes the rendering instruction 1 to draw the part of the triangle A covered by the tile, and executes the rendering instruction 2 to draw the part of the triangle B covered by the tile. In the process of drawing the tile, the GPU needs the following three aspects of data:

[0328] Firstly, the GPU needs the rendering data involved in the rendering instruction 1 and the rendering data involved in the rendering instruction 2, so as to execute the rendering instructions.

[0329] Secondly, the GPU also needs to determine the tile data corresponding to the tile, so as to determine which triangles are covered by the tile and which parts of the triangles are covered by the tile. As described above, the tile data is stored in the intermediate store of the main memory, so the GPU can directly access the intermediate store to obtain the tile data of the tile.

[0330] Thirdly, the GPU needs the geometry data of the triangle A and the geometry data of the triangle B, so as to draw the triangles A and B. The geometry data may include, but is not limited to, vertex, normal and tangent information of the triangle. The GPU can draw the tile based on the geometry data. The geometry data of the triangle A and the geometry data of the triangle B can be understood as the geometry data associated with the tile in the first row and the first column.

[0331] The acceleration of the rendering data is similar to the above-mentioned embodiments, and thus will not be described in detail. The tile data corresponding to the tile can be directly obtained from the main memory, and thus will not be described in detail. However, because the triangles associated with different tiles can be partially the same, the geometry data usually also needs to be accelerated, and the acceleration of the geometry data will be further described in the embodiment.

[0332] In the related art, referring to FIG. 12, when drawing the tile in the first row and the first column (corresponding to AB), the GPU can obtain the geometry data of triangle A and the geometry data of triangle B from the main memory, and then execute the rendering instruction 1 and the rendering instruction 2, so as to realize the drawing of the tile. Then, when drawing the tile in the first row and the second column (corresponding to AB), the GPU also needs to obtain the geometry data of triangle A and the geometry data of triangle B from the main memory again, and then execute the rendering instruction 1 and the rendering instruction 2, so as to realize the drawing of the tile.

[0333] The remaining tiles in FIG. 12 are similar to the above-mentioned embodiments, and thus will not be described in detail. Referring to FIG. 12, it can be understood that the GPU needs to repeatedly read the geometry data from the main memory, so as to cause large bandwidth consumption.

[0334] Therefore, the geometry data is considered to be accelerated in the present application, and the following will be understood in combination with FIG. 13 and FIG. 14:

[0335] In an implementation mode, the geometry data of each triangle included in the first graphics data rendered by the current drawing task can be stored in the GMEM. Referring to FIG. 13a, the geometry data of triangle A, the geometry data of triangle B, the geometry data of triangle C, the geometry data of triangle D, and the geometry data of triangle E can be stored in the GMEM. Then, no matter which tile is drawn, the GPU directly reads the required geometry data from the GMEM, so as to realize the rendering of the corresponding tile, and thus the bandwidth consumption caused by the frequent access of the GPU to the main memory can be effectively reduced.

[0336] In the actual implementation process, the geometry data can be accelerated. Alternatively, the geometry data and the rendering data can be simultaneously accelerated, and referring to FIG. 13b, the geometry data and the rendering resource can be simultaneously stored in the GMEM. The implementation mode of the geometry data and the rendering resource stored in the GMEM is similar to the above-mentioned embodiments, and thus will not be described in detail.

[0337] When the storage space of the GMEM is large, the technical solution introduced in FIG. 13a can be used, but if the storage space of the GMEM is small and cannot guarantee to accommodate the geometric data of all the triangles contained in the first graphic data, another implementation manner shown in FIG. 14 can also be used.

[0338] In this implementation manner, the geometric data contained in the first graphic data does not need to be all stored in the GMEM, but only the geometric data associated with the rendered tiles need to be stored in the GMEM. However, it can be understood that the geometric data associated with some tiles is the same, for example, in the example of FIG. 14a, the geometric data associated with the tile in the first row and the first column and the tile in the first row and the second column is the geometric data A and the geometric data B.

[0339] Then, instead of sequentially rendering according to the arrangement order of the tiles, the rendering order of the tiles can be reordered, and the rendering order of the tiles with the same associated geometric data can be arranged together.

[0340] That is, in this embodiment, the rendering order of the tiles can be sorted according to the geometric data associated with each tile. The sorting of the tiles with the same associated geometric data is adjacent. In this way, it can be realized that part of the geometric data is stored in the GMEM, and then the tiles associated with this part of the geometric data are continuously rendered, and then this part of the geometric data is removed.

[0341] For example, a rendering order of the tiles is shown in FIG. 14a. In order to distinguish which one of the 21 tiles shown in FIG. 14a each tile in the rendering order is, the row and column corresponding to the tile are marked in the brackets beside each tile. The number before the comma represents the row of the tile, and the number after the comma represents the column of the tile.

[0342] It can be determined from FIG. 14a that the rendering order of each tile is in turn the tile (A) in the first row and the third column→the tile (A) in the first row and the fourth column→the tile (AB) in the first row and the first column→the tile (AB) in the first row and the second column→the tile (AB) in the first row and the fifth column→the tile (ABC) in the first row and the second column→…, etc. The rendering order in FIG. 14a does not show all the tiles, but it can be determined that the sorting of the tiles with the same associated geometric data in the rendering order is adjacent.

[0343] Afterwards, each tile is rendered in turn according to the reordered rendering order. As can be determined with reference to Figure 14a, it is possible that the geometry data associated with adjacent tiles is identical. In this case, the geometry data associated with the previous tile can be loaded into the GMEM before rendering the next tile, so that the geometry data can be read directly from the GMEM, thereby saving the bandwidth consumption of repeated reading of geometry data from memory.

[0344] Alternatively, the geometry data associated with adjacent tiles can not be identical, but there can still be some overlap, so that some of the geometry data already stored in the GMEM does not need to be read repeatedly.

[0345] Therefore, when rendering each tile in turn according to the reordered rendering order, it can be determined first whether the geometry data associated with the tile is stored in the GMEM. The following describes the first tile as an example, which can be any one of the plurality of tiles. The following processing is performed for any geometry data associated with the first tile:

[0346] If the geometry data is stored in the GMEM, the geometry data can be obtained directly from the GMEM, thereby saving the bandwidth consumption of repeated reading of data from memory.

[0347] Alternatively, if the geometry data is not stored in the GMEM, the geometry data needs to be obtained from the main memory and stored in the GMEM, so that the GMEM stores all the geometry data associated with the rendering of the first tile. Afterwards, when rendering the first tile, the geometry data is read directly from the GMEM for rendering.

[0348] In addition, in order to clean up the storage space in the GMEM to store the geometry data associated with the current tile, the geometry data stored in the GMEM but not required (or can also be understood as not associated) by the current tile can be removed, so as to improve the possibility that the geometry data associated with the current tile can be stored in the GMEM.

[0349] The following describes a specific example with reference to Figure 14a, assuming that each tile is rendered in turn according to the rendering order shown in Figure 14a:

[0350] The first tile is associated with geometry data A, and in the initial case, the GMEM does not store any geometry data, so it can be determined that the geometry data A is not stored in the GMEM, and therefore the geometry data A can be read from the main memory and stored in the GMEM. With reference to the case in 1401 in Figure 14a, it can be determined that the geometry data A can be read from the GMEM to perform rendering of the first tile.

[0351] After that, the 2nd patch is to be rendered according to the rendering order, and the 2nd patch is also associated with geometry data A, since geometry data A has been stored in the GMEM, geometry data A can be directly read from the GMEM to perform rendering of the 2nd patch.

[0352] After that, the 3rd patch is to be rendered according to the rendering order, and the 3rd patch is associated with geometry data A and geometry data B, wherein geometry data A has been stored in the GMEM, so there is no need to repeatedly read geometry data A from the main memory to effectively save bandwidth consumption. However, geometry data B is not stored in the GMEM, so geometry data B can be read from the main memory and stored in the GMEM. Referring to the case in 1402 in FIG. 14a, it can be determined that geometry data A and geometry data B can be read from the GMEM to perform rendering of the 2nd patch.

[0353] The rendering of the 4th, 5th, and 6th patches is similar, and will not be described here. The rendering of the 7th patch will be described below.

[0354] After the rendering of the 6th patch is completed, geometry data A, geometry data B, and geometry data C can be stored in the GMEM. Then, the 7th patch is to be rendered according to the rendering order, and the 7th patch is associated with geometry data A and geometry data C, wherein geometry data A and geometry data C have been stored in the GMEM, so there is no need to repeatedly read these two geometry data from the main memory to effectively save bandwidth consumption.

[0355] However, geometry data B stored in the GMEM is not needed for the 7th patch that is currently being rendered, so geometry data B can be removed from the GMEM to clean up a part of the storage space in the GMEM to store the remaining data needed.

[0356] Based on the above description, it can be determined that if the geometry data associated with the latter patch overlaps with the geometry data associated with the former patch, part of the geometry data associated with the latter patch can be stored in the GMEM, so that a part of the bandwidth consumed by data reading can be effectively saved. Therefore, when reordering the rendering order of the patches, on the basis of ensuring that the patches associated with the same geometry data are adjacent in the rendering order, the geometry data associated with the patches can be regarded as a set, and the sets of different geometry data are set to be adjacent in the rendering order.

[0357] That is, in one implementation of determining the tile rendering order, the tiles associated with the first geometry data set can be sequentially and adjacently ordered first, and then in the geometry data sets associated with the remaining tiles, it is determined whether there is a second geometry data set that overlaps with the first geometry data set. If there is, then the tiles associated with the second geometry data set are sequentially and adjacently ordered next. If not, the tiles associated with any one geometry data set can be sequentially and adjacently ordered. Thus, the bandwidth consumed by reading data from the main memory can be further saved.

[0358] In actual implementation, only the geometry data can be accelerated. Alternatively, the geometry data and the rendering data can be accelerated simultaneously. Referring to FIG. 14b, the geometry data and the rendering resources can be stored in the GMEM simultaneously, for example. The implementation of the geometry data and the rendering resources stored in the GMEM is similar to that described in the above embodiments, which will not be described here.

[0359] Further, for the case of simultaneously accelerating the geometry data and the rendering data, it can be understood that the multiple rendering resources that need to be accelerated in the above embodiments and the geometry data that need to be stored in the GMEM described herein are actually this part of data, and both parts of data have acceleration needs. Therefore, storing any data in the GMEM can effectively save bandwidth consumption. If both parts of data can be stored in the GMEM, it is the best result. However, when the storage space of the GMEM is limited and both parts of data cannot be stored, the implementation described in the above embodiments can still be used:

[0360] According to the multiple rendering resources that need to be accelerated and the geometry data that need to be stored in the GMEM, any combination is made to determine multiple rendering resources and geometry data after combination whose total sum is maximum and does not exceed the remaining storage space of the GMEM. Then, the multiple rendering resources and the geometry data are stored in the GMEM. Thus, under the premise that the GMEM can store, the rendering resources with the larger total amount of resources are stored in the GMEM as much as possible, thereby saving the bandwidth consumption of the GMEM accessing the main memory as much as possible.

[0361] The above embodiments introduce that the GPU can store the rendering resources that need to be accelerated in the GMEM according to the use of the rendering resources, so as to save the bandwidth consumption of the GPU accessing the main memory. At the same time, the delay caused by the GPU accessing the main memory to obtain data can also be saved, thereby improving the performance.

[0362] In another implementation, the developer can explicitly indicate in the program design that some rendering resources need to be accelerated, and therefore, the API hint interface can be provided to the developer to send an acceleration instruction to the GPU in the process of program design, where the acceleration instruction indicates that a certain rendering resource needs to be accelerated. Then, the GPU can determine that a certain rendering resource needs to be accelerated according to the acceleration instruction, and then directly store the rendering resource in the GMEM.

[0363] In an implementation, the API hint interface can be implemented in the following forms, for example:

[0364] glGMEMHint(Resource Type, Resource Id, Flag);

[0365] VKGMEMHint(Resource Type, Resource handle, Flag);

[0366] wherein, Resource Type is the resource type of the rendering resource, Resource Id is the resource identifier of the rendering resource, and Flag is used to indicate whether the rendering resource needs to be accelerated, for example, Flag is 1 to indicate that the rendering resource needs to be accelerated, and Flag is 0 to indicate that the rendering resource does not need to be accelerated.

[0367] Based on this implementation, the GPU can quickly and efficiently determine that some rendering resources need to be accelerated, thereby saving the related judgment operation, improving the operation efficiency of the resource acceleration processing, and effectively saving the system resources of the GPU.

[0368] The rendering processing method of the embodiments of the present application has been described above, and the device provided by the embodiments of the present application for executing the above method is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the related device provided by the embodiments of the present application can execute the steps in the method of the above list sorting.

[0369] FIG. 15 is a schematic block diagram of a rendering processing apparatus 1500 according to an embodiment of the present application. The apparatus 1500 includes a processor 1501, a communication interface 1502, and a memory 1503. The processor 1501, the communication interface 1502, and the memory 1503 communicate with each other through an internal connection path. The memory 1503 is configured to store instructions, and the processor 1501 is configured to execute the instructions stored in the memory 1503. The communication interface 1502 can be configured to send signals to other apparatuses (for example, the processor 1501 or a touch screen of an electronic device) and receive signals from other apparatuses (for example, the memory 1503). For example, the communication interface 1502 reads the instructions stored in the memory 1503 and sends the instructions to the processor 1501.

[0370] It should be understood that the apparatus 1500 can be specifically an electronic device in the above-described embodiments, and can be configured to perform each step and / or process corresponding to the electronic device in the above-described method embodiments. Alternatively, the memory 1503 can include a read-only memory and a random access memory, and provide instructions and data for the processor. A part of the memory can also include a non-volatile random access memory. For example, the memory can also store device type information. The processor 1501 can be configured to execute the instructions stored in the memory, and when the processor 1501 executes the instructions stored in the memory, the processor 1501 is configured to perform each step and / or process of the above-described method embodiments.

[0371] It should be understood that in the embodiments of the present application, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0372] In the implementation process, each step of the above-described method can be completed by the integrated logic circuit of hardware in the processor or the instructions in the form of software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as the execution of hardware processor, or executed by the combination of hardware and software modules in the processor. The software modules can be located in the random access memory, the flash memory, the read-only memory, the programmable read-only memory, the electrically erasable programmable memory, the register, or other mature storage media in the field. The storage medium is located in the memory, and the processor executes the instructions in the memory to combine the hardware to complete the steps of the above-described method. To avoid repetition, they will not be described in detail here.

[0373] The rendering processing method provided in the embodiments of the present application can be applied to an electronic device with an image processing function. The electronic device includes a terminal device, and the specific device form of the terminal device can refer to the related description above, which will not be repeated here. The embodiments of the present application provide a terminal device, which includes: a processor and a memory; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory, so that the terminal device executes the above method.

[0374] The embodiments of the present application provide a chip. The chip includes a processor, and the processor is configured to invoke a computer program in a memory to execute the technical solutions in the above embodiments. The implementation principle and technical effects are similar to those of the above related embodiments, which will not be repeated here.

[0375] The embodiments of the present application also provide a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by the processor to implement the above method. The method described in the above embodiments can be implemented by software, hardware, firmware or any combination thereof, in whole or in part. If implemented in software, the functions can be stored as one or more instructions or codes on a computer readable medium or transmitted on a computer readable medium. The computer readable medium can include computer storage medium and communication medium, and can also include any medium that can carry computer programs from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0376] In a possible implementation, the computer readable medium can include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that is targeted to carry the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately called a computer readable medium. For example, if software is transmitted from a website, server or other remote source using a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology, the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave is included in the definition of medium. As used herein, magnetic disks and optical disks include compact disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, in which magnetic disks usually reproduce data magnetically, and optical disks reproduce data optically with laser. The above combinations should also be included in the scope of computer readable medium.

[0377] The embodiment of the present application provides a computer program product, the computer program product comprises a computer program, when the computer program is executed, the computer executes the above method.

[0378] The embodiment of the present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate a device for realizing the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0379] The above detailed description is further detailed for the purpose, technical scheme and beneficial effects of the present application, and it should be understood that the above is only the specific embodiment of the present application, and is not used to limit the protection scope of the present application, and any modification, equivalent replacement, improvement, etc. made on the basis of the technical scheme of the present application should be included in the protection scope of the present application.

Claims

1. A rendering processing method, characterized in that, include: Determine multiple rendering instructions corresponding to the first graphics data to be rendered in the first application, wherein the multiple rendering instructions include the first rendering instruction; Read the first rendering resource corresponding to the first rendering instruction from the graphics processor GPU memory; Based on the rendering resources corresponding to each of the plurality of rendering instructions, the plurality of rendering instructions are executed to render the first graphic data.

2. The method according to claim 1, characterized in that, The plurality of rendering instructions further includes a second rendering instruction; the method further includes: Read the first rendering resource corresponding to the second rendering instruction from the GPU memory.

3. The method according to claim 2, characterized in that, The second rendering instruction is adjacent to the first rendering instruction in the instruction sequence of the plurality of rendering instructions; or, The number of rendering instructions between the second rendering instruction and the first rendering instruction is less than or equal to the first threshold. or, The number of rendering instructions corresponding to the first rendering resource among the plurality of rendering instructions is greater than or equal to a second threshold, and the plurality of rendering instructions includes the second rendering instruction.

4. The method according to any one of claims 1-3, characterized in that, The plurality of rendering instructions further includes a third rendering instruction; executing the plurality of rendering instructions includes: The first rendering instruction is executed based on the first rendering resource to obtain the second rendering resource; The third rendering instruction is executed based on the second rendering resource.

5. The method according to claim 4, characterized in that, After obtaining the second rendering resource, the method further includes: Write the second rendering resource into the GPU memory; Before executing the third rendering instruction based on the second rendering resource, the method further includes: Read the second rendering resource corresponding to the third rendering instruction from the GPU memory.

6. The method according to claim 5, characterized in that, The third rendering instruction is adjacent to the first rendering instruction in the instruction sequence of the plurality of rendering instructions; or, The number of rendering instructions between the third rendering instruction and the first rendering instruction is less than or equal to the third threshold.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Retrieve indicator fields corresponding to multiple rendering resources; Based on the indicator fields corresponding to the multiple rendering resources, write operations are performed on the multiple rendering resources; Wherein, when the indicator field is a first value, the write operation is used to write the rendering resource corresponding to the indicator field into the GPU memory; when the indicator field is a second value, the write operation is used to write the rendering resource corresponding to the indicator field into the main memory.

8. The method according to claim 7, characterized in that, The plurality of rendering resources includes the first rendering resource and the second rendering resource; The step of obtaining the indicator fields corresponding to multiple rendering resources includes: When the first rendering instruction uses the first rendering resource in a read-type manner, there exists... If the second rendering instruction uses the first rendering resource as a read type, then the value of the indicator field corresponding to the first rendering resource is determined to be the first value. If the first rendering instruction uses the second rendering resource in a write manner, and the third rendering instruction uses the second rendering resource in a read manner, then the value of the indicator field corresponding to the second rendering resource is determined to be the first value.

9. The method according to claim 7, characterized in that, Before determining that the indicator field corresponding to the first rendering resource is the first value, the method further includes: Determine whether the data volume of the first rendering resource is less than or equal to the remaining storage space of the GPU memory; The step of determining that the indicator field corresponding to the first rendering resource is the first value includes: When the amount of data in the first rendering resource is less than or equal to the remaining storage space of the GPU memory, the indicator field corresponding to the first rendering resource is determined to be the first value.

10. The method according to any one of claims 1-9, characterized in that, The execution of the plurality of rendering instructions includes: When the rendering mode corresponding to the first graphic data is the first rendering mode, the plurality of rendering instructions are executed sequentially; When the rendering mode corresponding to the first graphic data is the second rendering mode, the multiple tiles obtained by division are rendered sequentially. During the rendering of the multiple tiles, the rendering instructions associated with each tile in the multiple tiles are executed.

11. The method according to claim 10, characterized in that, The multiple tiles obtained from the rendering division include: Render the multiple tiles based on their associated geometric data and the rendering resources corresponding to the rendering instructions associated with each of the multiple tiles. The method further includes: Read the geometric data associated with the multiple tiles from the GPU memory.

12. The method according to claim 11, characterized in that, The method further includes: If the total amount of geometric data contained in the first graphics data is less than or equal to the remaining storage space of the GPU memory, the geometric data contained in the first graphics data is written into the GPU memory.

13. The method according to claim 11, characterized in that, The method further includes: If the total amount of geometric data contained in the first graphics data is greater than the remaining storage space of the GPU memory, the plurality of tiles are rendered sequentially in a first order. For a rendered tile, if the geometric data associated with the rendered tile is not stored in the GPU memory, then the geometric data associated with the rendered tile is written into the GPU memory. The plurality of blocks include a first block and a second block, wherein the geometric data associated with the first block and the geometric data associated with the second block are the same, and the first block and the second block are adjacent in the first order.

14. The method according to claim 10, characterized in that, When the area corresponding to the first graphic data covers the entire screen area of ​​the terminal device, the rendering method corresponding to the first graphic data is the first rendering method. When the area corresponding to the first graphic data covers part of the screen area of ​​the terminal device, the first... The rendering method corresponding to a graphic data is the second rendering method.

15. The method according to claim 13 or 14, characterized in that, The method further includes: The tile rendering results of the multiple tiles are written to the GPU memory.

16. The method according to any one of claims 7-15, characterized in that, For any target rendering resource whose indicator field value is the first value, after the target rendering resource has been read, the method further includes: If no read instruction is available for the target rendering resource, the target rendering resource is deleted from the GPU memory. The read instruction is sent by the central processing unit (CPU). If a read instruction for the target rendering resource exists, the target rendering resource is sent to the CPU, and the target rendering resource is deleted from the GPU memory.

17. An electronic device, characterized in that, The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 16.

18. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 16.

20. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 16.

Images