Method and apparatus for graphics rendering

Abstract

Embodiments of the present disclosure provide a method and apparatus for graphics rendering. In the method, a script file defining a combined rendering path is loaded in a graphics rendering SDK package. The combined rendering path includes a plurality of atomic rendering paths, at least one texture, and texture relationship information between the plurality of atomic rendering paths. Each atomic rendering path in the plurality of atomic rendering paths represents a shader. The texture relationship information indicates a relationship of an output texture of each atomic rendering path with an input texture of another atomic rendering path. Then, script code in the script file is converted into object code. Next, the object code is run to perform a graphics rendering operation.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of computer technology, and more specifically, to methods and apparatus for graphics rendering. Background Technology

[0002] Currently, graphics rendering operations are performed by the GPU (commonly known as a graphics card). GPU encoding and decoding speeds are faster than those of the Central Processing Unit (CPU). Using the GPU for encoding and decoding is called hardware encoding and decoding. Graphics APIs used for hardware encoding and decoding (e.g., OpenGL, Metal, Vulkan) require compilation and linking. The logic for performing graphics rendering is embedded in the executable file or dynamic link library of the graphics processing application. Following the principle of high cohesion and low coupling in program code, the code related to graphics rendering operations is centralized in a graphics rendering SDK package. The graphics processing application depends on and links to this graphics rendering SDK package to implement graphics rendering functionality. When post-processing effects (which are part of graphics rendering) need to be updated or added, the source code of the graphics rendering SDK package needs to be updated and a new SDK version released. The graphics processing application also needs to release a new version to integrate the new graphics rendering SDK package to have the new post-processing effects. Summary of the Invention

[0003] The embodiments described herein provide a method, apparatus, and computer-readable storage medium storing a computer program for graphics rendering.

[0004] According to a first aspect of this disclosure, a method for graphics rendering is provided. In this method, a script file defining a combined rendering path is loaded into a graphics rendering SDK package. The combined rendering path includes multiple atomic rendering paths, at least one texture, and texture relationship information between the multiple atomic rendering paths. Each of the multiple atomic rendering paths represents a shader. The texture relationship information indicates the relationship between the output texture of each atomic rendering path and the input textures of the other atomic rendering paths. Then, the script code in the script file is converted into object code. Next, the object code is run to perform graphics rendering operations.

[0005] In some embodiments of this disclosure, the graphics rendering SDK package includes a binding file. The binding file defines a mapping relationship between a plurality of first objects defined in object code and a plurality of second objects defined in script code. Converting script code in the script file into object code includes mapping each second object in the script file to a corresponding first object according to the mapping relationship.

[0006] In some embodiments of this disclosure, at least one script file loading path is defined in the graphics rendering SDK package. The script file loading path defines the address where the script file is stored. Loading a script file in the graphics rendering SDK package includes loading the script file into the graphics rendering SDK package via one of the at least one script file loading paths.

[0007] In some embodiments of this disclosure, the combined rendering paths are represented in the form of a directed acyclic graph (DAG). The DAG includes multiple nodes and edges connecting the nodes. Each node in the DAG corresponds to a multiple atomic rendering path. The edge between each pair of connected nodes in the DAG carries texture relationship information between the corresponding two atomic rendering paths.

[0008] In some embodiments of this disclosure, each of a plurality of nodes in a directed acyclic graph is connected to at least one other node among a plurality of nodes.

[0009] In some embodiments of this disclosure, the number of nodes with an in-degree of 0 in a directed acyclic graph is 1. The number of nodes with an out-degree of 0 in a directed acyclic graph is 1. In-degree represents the number of edges pointing to that node. Out-degree represents the number of edges pointing from that node to other nodes.

[0010] In some embodiments of this disclosure, the atomic rendering path is established through the following steps: constructing a shader program, wherein the shader program includes: a vertex shader program and a fragment shader program; setting graphics pipeline related parameters of the shader program; setting vertex data of the shader program; setting parameters within the shader program; associating each input sampler of the shader with a first blank texture, wherein the first blank texture is used to store the input image data to be processed; and associating each output of the shader with a second blank texture, wherein the second blank texture is used to store the rendered output image data.

[0011] In some embodiments of this disclosure, the target code file is a GPU-executable code file.

[0012] According to a second aspect of this disclosure, an apparatus for graphics rendering is provided. The apparatus includes at least one processor and at least one memory storing a computer program. When the computer program is executed by the at least one processor, the apparatus causes the following actions: loading a script file in a graphics rendering SDK package, the script file defining a combined rendering path, the combined rendering path including a plurality of atomic rendering paths, at least one texture, and texture relationship information between the plurality of atomic rendering paths, each of the plurality of atomic rendering paths representing a shader, the texture relationship information indicating the relationship between the output texture of each atomic rendering path and the input textures of other atomic rendering paths; converting script code in the script file into object code; and running the object code to perform graphics rendering operations.

[0013] In some embodiments of this disclosure, the graphics rendering SDK package includes a binding file. The binding file defines a mapping relationship between a plurality of first objects defined in object code and a plurality of second objects defined in script code. When executed by at least one processor, a computer program causes a device to convert script code in a script file into object code by mapping each second object in the script file to a corresponding first object according to the mapping relationship.

[0014] In some embodiments of this disclosure, at least one script file loading path is defined in the graphics rendering SDK package. The script file loading path defines the address where the script file is stored. When executed by at least one processor, a computer program causes a device to load a script file in the graphics rendering SDK package by loading the script file into the graphics rendering SDK package via one of the at least one script file loading paths.

[0015] In some embodiments of this disclosure, when executed by at least one processor, the computer program causes the apparatus to establish an atomic rendering path by: constructing a shader program, wherein the shader program includes: a vertex shader program and a fragment shader program; setting graphics pipeline-related parameters of the shader program; setting vertex data of the shader program; setting parameters within the shader program; associating each input sampler of the shader with a first blank texture, wherein the first blank texture is used to store input image data to be processed; and associating each output of the shader with a second blank texture, wherein the second blank texture is used to store rendered output image data.

[0016] According to a third aspect of this disclosure, a computer-readable storage medium is provided storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described according to a first aspect of this disclosure. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. It should be understood that the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure, wherein:

[0018] Figure 1 This is an exemplary flowchart of a method for graphics rendering according to embodiments of the present disclosure;

[0019] Figure 2 This is an exemplary schematic diagram of a combined rendering path according to an embodiment of the present disclosure;

[0020] Figure 3 This is an exemplary flowchart illustrating the steps of establishing an atomic rendering path according to embodiments of the present disclosure; and

[0021] Figure 4 This is an exemplary block diagram of an apparatus for graphics rendering according to embodiments of the present disclosure.

[0022] The elements in the attached diagram are schematic and not drawn to scale. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are also within the scope of protection of this disclosure.

[0024] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having meanings consistent with their meanings in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. Terms such as “first” and “second” are used only to distinguish one component (or part of a component) from another component (or another part of a component).

[0025] As mentioned above, developing post-processing effects using the existing SDK release process incurs significant time costs. Assuming both the graphics rendering SDK development team and the graphics processing application development team release a new version every two weeks, a completed post-processing effect would take approximately two weeks to a month to be deployed online. This update speed clearly cannot meet user needs.

[0026] Embodiments of this disclosure provide a method for graphics rendering. Figure 1 An exemplary flowchart of a method for graphics rendering according to embodiments of the present disclosure is shown. The following is in conjunction with... Figure 1 To describe method 100 for graphics rendering.

[0027] exist Figure 1 At box S102, a script file is loaded from the graphics rendering SDK package. This script file defines a combined rendering path. The combined rendering path includes multiple atomic rendering paths, at least one texture, and texture relationship information between these atomic rendering paths. Each atomic rendering path represents a shader. The texture relationship information indicates the relationship between the output texture of each atomic rendering path and the input textures of other atomic rendering paths. The term "atomic" in an atomic rendering path indicates that it is the simplest, indivisible rendering path. The concept of atomic rendering paths exists because many complex post-processing effects cannot be achieved with a single rendering path and require combining multiple rendering paths to achieve the desired effect. These combined rendering paths are what we call a combined rendering path (e.g., a filter chain).

[0028] In some embodiments of this disclosure, at least one script file loading path is defined in the graphics rendering SDK package. The script file loading path defines the address where the script files are stored. The address where the script files are stored can be a web address (URL) or a local address. In one example, each script file can provide a filter chain. Assume that two script file loading paths are defined in the graphics rendering SDK package: a first script file loading path and a second script file loading path. During the execution of the graphics rendering SDK package, a first script file can be loaded into the graphics rendering SDK package via the first script file loading path to implement a first filter chain. A second script file can be loaded into the graphics rendering SDK package via the second script file loading path to implement a second filter chain.

[0029] Figure 2 An exemplary schematic diagram of a combined rendering path according to an embodiment of the present disclosure is shown. Figure 2In the example, the combined rendering path 200 includes: a first atomic rendering path 204, a second atomic rendering path 208, a third atomic rendering path 212, texture a 202, texture b 206, texture c 210, texture d 214, and texture relationship information between these multiple atomic rendering paths (the specific content of the texture relationship information is not shown). The combined rendering path 200 is represented in the form of a directed acyclic graph (DAG). A directed acyclic graph is a data structure. A directed acyclic graph includes multiple nodes and edges connecting the multiple nodes. The multiple nodes in the directed acyclic graph correspond to multiple atomic rendering paths. The edge between each pair of connected nodes in the directed acyclic graph carries the texture relationship information between the corresponding two atomic rendering paths.

[0030] exist Figure 2 In the example, texture a 202 is the input texture of the sampler of the first atomic rendering path 204, and texture b 206 is the output texture of the first atomic rendering path 204. The output texture of the first atomic rendering path 204 (texture b 206) is the input texture of the sampler of the second atomic rendering path 208 (i.e., the texture relationship information between the first atomic rendering path 204 and the second atomic rendering path 208). The output texture of the first atomic rendering path 204 (texture b 206) is the input texture of the sampler input2 of the third atomic rendering path 212 (i.e., the texture relationship information between the first atomic rendering path 204 and the third atomic rendering path 212). The output texture of the second atomic rendering path 208 (texture c 210) is the input texture of the sampler input1 of the third atomic rendering path 212 (i.e., the texture relationship information between the second atomic rendering path 208 and the third atomic rendering path 212). The output texture of the third atomic rendering path 212 (texture d 214) is the output texture of the combined rendering path 200.

[0031] In some embodiments of this disclosure, each node in a directed acyclic graph representing a composite rendering path is connected to at least one other node. In other words, there cannot be isolated nodes when the number of nodes in the directed acyclic graph is greater than one. An isolated node is one that is not connected to any other node. Isolated nodes are meaningless in the composite rendering path and do not participate in the rendering process. Figure 2 In the example, any one of the first atomic rendering path 204, the second atomic rendering path 208, and the third atomic rendering path 212 is connected to at least one other atomic rendering path.

[0032] Furthermore, in some embodiments of this disclosure, the number of nodes with an in-degree of 0 among the multiple nodes in the directed acyclic graph is 1. In-degree represents the number of edges pointing to that node. The number of nodes with an out-degree of 0 among the multiple nodes in the directed acyclic graph is 1. Out-degree represents the number of edges pointing to other nodes from that node. Figure 2 In the example, the node with an in-degree of 0 is the first atomic rendering path 204. The node with an out-degree of 0 is the third atomic rendering path 212. It should be noted that edges in a directed acyclic graph exist only between two nodes. Figure 2 In the example, although texture a 202 points to the first atomic rendering path 204 via an arrow, this arrow does not originate from a node in the directed acyclic graph (DAG), therefore, the arrow cannot be considered an edge. Thus, the in-degree of the first atomic rendering path 204 is 0. Similarly, although the third atomic rendering path 212 points to texture d 214 via an arrow, this arrow does not point to a node in the DAG, therefore, the arrow cannot be considered an edge. Thus, the out-degree of the third atomic rendering path 212 is 0.

[0033] Figure 2 The example only illustrates one possible structure of a combined rendering path. Those skilled in the art will understand that combined rendering paths can also have other structures (e.g., having the same as...). Figure 2 The different numbers of nodes and their connections are not detailed here.

[0034] Reference Figure 2 After introducing the process of establishing a combined rendering path, the following is a reference. Figure 3 This section will introduce the steps for establishing an atomic rendering path.

[0035] exist Figure 3 At box S302, the shader program is constructed. The graphics algorithm is implemented in the shader. A shader program can include multiple shader programs, such as a vertex shader program and a fragment shader program. Graphics algorithm developers need to write the source code for the vertex and fragment shaders, and then use a graphics API (e.g., OpenGL) to compile and link the shader source code into a shader program. Generally, one shader program internally implements one graphics algorithm.

[0036] At box S304, set the graphics pipeline parameters of the shader program, such as viewport size, blending parameters, primitive type, and drawing type.

[0037] At box S306, set the vertex data for the shader program. For 2D post-processing effects, the vertex data here is fixed as the vertex data and vertex layout of a rectangle composed of two triangles.

[0038] In box S308, set the parameters within the shader procedure. Shader procedure parameters can be set during the creation of the atomic render pass (RenderPass) stage, or after creation and before rendering.

[0039] At box S310, each input sampler of the shader is associated with a first blank texture. This first blank texture is used to store the input image data to be processed. Figure 2 In the example, the second atomic rendering path 208 has one input sampler. Texture b 206 can be bound to this input sampler. Before rendering, texture b 206 is a blank texture. Texture b 206 is only filled with texture data after the first atomic rendering path 204 outputs rendered image data to texture b 206. Similarly, the third atomic rendering path 212 has two input samplers, input1 and input2. Texture c 210 can be bound to input sampler input1. Texture b 206 can be bound to input sampler input2. Before rendering, texture c 210 is a blank texture. Texture c 210 is only filled with texture data after the second atomic rendering path 208 outputs rendered image data to texture c 210.

[0040] At box S312, each output of the shader is associated with a second blank texture, which is used to store the rendered output image data. In normal mode, the shader has one output. When depth testing is enabled, the shader has two outputs. Figure 2 In the example, the first atomic rendering path 204 has an output terminal. Texture b206 can be bound to this output terminal. Before rendering, texture b206 is a blank texture. Texture b206 is only filled with texture data after the first atomic rendering path 204 outputs the rendered image data to texture b206.

[0041] It should be noted that the operations performed in blocks S302 to S312 can be performed in parallel, or in accordance with... Figure 3 The different orders shown will be executed.

[0042] Back Figure 1 At box S104, the script code in the script file is converted into target code. In some embodiments of this disclosure, the script code in the script file may be written, for example, in the Lua scripting language. The target code is GPU-executable code, such as compiled OpenGL (or other graphics API) code. In some embodiments of this disclosure, the script code in the script file may be converted into target code during the execution of the script file.

[0043] In some embodiments of this disclosure, the graphics rendering SDK package may include a binding file. The binding file defines a mapping relationship between a plurality of first objects defined in object code and a plurality of second objects defined in script code. In other words, in the binding file, the plurality of first objects defined in object code are bound to the plurality of second objects defined in script code. Each second object in the script file can be mapped to a corresponding first object according to the mapping relationship.

[0044] At box S106, target code is run to perform graphics rendering operations. In some embodiments of this disclosure, during the execution of the script file, second objects are called according to the logical order (combined rendering path) defined in the script file. When each second object is called, the first object corresponding to that second object may be called. The process of calling the first object is equivalent to the process of running the target code. The process of running the target code is equivalent to the process of performing graphics rendering operations according to the combined rendering path defined in the script file.

[0045] Since the graphics rendering SDK package only defines the script file loading path and does not include the code for composite rendering paths, it is not necessary to redistribute the graphics rendering SDK package when updating composite rendering paths. Using the method described above 100, the code describing the composite rendering path structure can be extracted from the graphics rendering SDK package and placed into a script file. The graphics rendering SDK package only needs to include code that can translate the description object of the composite rendering path into graphics API instructions to support various composite rendering paths.

[0046] Figure 4 A schematic block diagram of an apparatus 400 for graphics rendering according to an embodiment of the present disclosure is shown. Figure 4 As shown, the device 400 may include a processor 410 and a memory 420 storing a computer program. When the computer program is executed by the processor 410, the device 400 is made capable of performing actions such as... Figure 1 The steps of method 100 are shown. In one example, device 400 may be a computer device or a cloud computing node. Device 400 may load a script file in a graphics rendering SDK package, which defines a combined rendering path. The combined rendering path includes multiple atomic rendering paths, at least one texture, and texture relationship information between the multiple atomic rendering paths. Each of the multiple atomic rendering paths represents a shader. The texture relationship information indicates the relationship between the output texture of each atomic rendering path and the input textures of other atomic rendering paths. Device 400 may convert the script code in the script file into object code. Device 400 may run the object code to perform graphics rendering operations.

[0047] In some embodiments of this disclosure, the graphics rendering SDK package includes a binding file. The binding file defines a mapping relationship between a plurality of first objects defined in object code and a plurality of second objects defined in script code. The apparatus 400 can map each second object in the script file to a corresponding first object based on the mapping relationship.

[0048] In some embodiments of this disclosure, at least one script file loading path is defined in the graphics rendering SDK package. The script file loading path defines the address where the script file is stored. The apparatus 400 can load a script file into the graphics rendering SDK package via one of these at least one script file loading paths.

[0049] In some embodiments of this disclosure, the apparatus 400 can construct a shader program. The shader program includes a vertex shader program and a fragment shader program. The apparatus 400 can set graphics pipeline-related parameters of the shader program. The apparatus 400 can set vertex data of the shader program. The apparatus 400 can set parameters within the shader program. The apparatus 400 can associate each input sampler of the shader with a first blank texture. The first blank texture is used to store input image data to be processed. The apparatus 400 can associate each output of the shader with a second blank texture. The second blank texture is used to store rendered output image data.

[0050] In embodiments of this disclosure, processor 410 may be, for example, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a processor based on a multi-core processor architecture, etc. Memory 420 may be any type of memory implemented using data storage technologies, including but not limited to random access memory, read-only memory, semiconductor-based memory, flash memory, disk storage, etc.

[0051] Furthermore, in embodiments of this disclosure, device 400 may also include an input device 430, such as a keyboard or mouse, for inputting script files. Additionally, device 400 may also include an output device 440, such as a display, for outputting rendered graphics.

[0052] In other embodiments of this disclosure, a computer-readable storage medium storing a computer program is also provided, wherein the computer program, when executed by a processor, is capable of performing the following functions: Figures 1 to 3 The steps of the method shown.

[0053] The method for graphics rendering according to embodiments of this disclosure can extract the code describing the structure of a combined rendering path from the graphics rendering SDK package and place it in a script file. When it is necessary to update or replace the combined rendering path, only the script file needs to be updated, not the graphics rendering SDK package. The graphics rendering SDK package only needs to include code that can translate the description object of the combined rendering path into graphics API instructions to support a scalable number of different rendering paths.

[0054] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses and methods according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0055] Unless otherwise expressly indicated by the context, the singular form of words used herein and in the appended claims includes the plural form, and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Similarly, the terms “comprising” and “including” shall be interpreted as including rather than exclusively. Likewise, the terms “including” and “or” shall be interpreted as including unless such interpretation is expressly prohibited herein. Where the term “example” is used herein, particularly when it follows a set of terms, the “example” is merely exemplary and illustrative and should not be considered exclusive or extensive.

[0056] Further aspects and scope of adaptation become apparent from the description provided herein. It should be understood that various aspects of this application may be implemented individually or in combination with one or more other aspects. It should also be understood that the descriptions and specific embodiments herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0057] Several embodiments of this disclosure have been described in detail above. However, it is obvious that those skilled in the art can make various modifications and variations to the embodiments of this disclosure without departing from the spirit and scope of this disclosure. The scope of protection of this disclosure is defined by the appended claims.

Claims

1. A method for graphics rendering, comprising: loading a script file in a graphics rendering SDK package, the script file defining a combined rendering path, the combined rendering path comprising a plurality of atomic rendering paths, at least one texture, and texture relationship information between the plurality of atomic rendering paths, each of the plurality of atomic rendering paths representing a shader, the texture relationship information indicating a relationship of an output texture of each atomic rendering path with an input texture of another atomic rendering path; converting script code in the script file into target code; and running the target code to perform a graphics rendering operation; wherein at least one script file loading path is defined in the graphics rendering SDK package, the script file loading path defining an address where the script file is stored, the address where the script file is stored being a web address URL or a local address, loading the script file in the graphics rendering SDK package comprising loading the script file in the graphics rendering SDK package via one of the at least one script file loading path; wherein the combined rendering path is represented in a form of a directed acyclic graph, the directed acyclic graph comprising a plurality of nodes and edges connecting the plurality of nodes, the plurality of nodes in the directed acyclic graph respectively corresponding to the plurality of atomic rendering paths, each edge between two connected nodes in the directed acyclic graph carrying texture relationship information between the corresponding two atomic rendering paths.

2. The method of claim 1, wherein, the graphics rendering SDK package comprises a binding file defining a mapping relationship between a plurality of first objects defined in the target code and a plurality of second objects defined in the script code, converting the script code in the script file into the target code comprises mapping each second object in the script file to a first object corresponding to the second object according to the mapping relationship.

3. The method of claim 1, wherein, each node in the plurality of nodes in the directed acyclic graph is connected to at least one other node in the plurality of nodes.

4. The method of claim 1 or 3, wherein, a number of nodes in the plurality of nodes in the directed acyclic graph having an in-degree of 0 is 1, a number of nodes in the plurality of nodes in the directed acyclic graph having an out-degree of 0 is 1, the in-degree representing a number of edges pointing to the node, the out-degree representing a number of edges pointing from the node to other nodes.

5. The method of claim 1, wherein, the atomic rendering path is established by: constructing a shader program, wherein the shader program comprises a vertex shader program and a fragment shader program; setting a graphics pipeline related parameter of the shader program; setting vertex data of the shader program; setting a parameter within the shader program; associating each input sampler of the shader to a first blank texture, wherein the first blank texture is used to save input picture data to be processed; and associating each output end of the shader to a second blank texture, wherein the second blank texture is used to save rendered output picture data.

6. The method of claim 1, wherein, the target code file is a GPU executable code file.

7. An apparatus for graphics rendering, comprising: at least one processor; and at least one memory having a computer program stored therein; wherein the computer program, when executed by the at least one processor, causes the apparatus to perform the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, wherein, The computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images