Rendering processing method and apparatus, electronic device, storage medium, and program product

Abstract

The application provides a rendering processing method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: in the process of calling the interface of an open graphics library, at least one interface call is intercepted, so that the at least one interface call is respectively converted into at least one near-zero driver overhead data, the driver overhead of the near-zero driver overhead data is small, the calling overhead of the open graphics library is reduced; and then the at least one near-zero driver overhead data is processed in parallel into corresponding rendering tasks and sent to a graphics processor, so that the rendering tasks are integrated, the waste of rendering time and rendering resources caused by the multiple rendering of the image processor is avoided, the graphics processor performs fast rendering processing and display on the rendering tasks, the original code program of the open graphics library is not changed, the operation is convenient and fast, the low overhead of the near-zero driver overhead data is utilized, the parallel processing advantage is combined, and the rendering efficiency is effectively improved.

Description

Technical Field

[0001] This application relates to the field of graphics rendering technology, and in particular to a rendering processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] In existing technologies, rendering is typically accomplished by using the Open Graphics Library (OpenGL) to interface with the graphics processor.

[0003] However, this method of graphics rendering through repeated API calls is cumbersome and requires frequent resource switching, resulting in high resource consumption and low efficiency. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a rendering processing method, apparatus, electronic device, storage medium and program product to solve or partially solve the above-mentioned technical problems.

[0005] To achieve the above objectives, this application provides a rendering processing method, comprising:

[0006] During the process of making interface calls to the Open Graphics Library, intercept at least one interface call of the Open Graphics Library;

[0007] Each of the above-mentioned interface calls is converted into at least one near-zero driver overhead data; wherein, the interface call and the near-zero driver overhead data correspond one-to-one.

[0008] At least one of the near-zero driving overhead data is processed in parallel to obtain a rendering task, and the rendering task is sent to the graphics processor for the graphics processor to render and display the rendering task.

[0009] Based on the same concept, this application also provides a rendering processing method, including:

[0010] Receive rendering tasks from the central processing unit, wherein the rendering tasks include at least one near-zero drive overhead data;

[0011] Rendering results are obtained by rendering based on at least one of the near-zero driving overhead data in the rendering task, and the rendering results are displayed on the screen.

[0012] Based on the same concept, this application also provides a rendering processing apparatus, including:

[0013] The interception module is configured to intercept at least one interface call of the open graphics library during the interface call process; wherein, the interface call corresponds one-to-one with the near-zero driver overhead data;

[0014] The conversion module is configured to convert at least one of the interface calls into at least one near-zero driver overhead data, respectively;

[0015] A parallel processing module is configured to process at least one of the near-zero driving overhead data in parallel to obtain a rendering task, and send the rendering task to a graphics processor for the graphics processor to render and display the rendering task.

[0016] Based on the same concept, this application also provides a rendering processing apparatus, including:

[0017] The rendering task receiving module is configured to receive a rendering task fed back by the processor, which is transformed from at least one near-zero drive overhead data, wherein the near-zero drive overhead data is data obtained by the processor from at least one interface call to intercept open graphics libraries.

[0018] The rendering module is configured to render the rendering task to obtain the rendering result and display the rendering result on the screen.

[0019] Based on the same concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described above.

[0020] Based on the same concept, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to perform the methods described above.

[0021] Based on the same concept, this application also provides a computer program product, including computer program instructions that, when the computer program instructions are run on a computer, cause the computer to perform the method described above.

[0022] As can be seen from the above, the rendering processing method, apparatus, electronic device, storage medium, and program product provided in this application intercept at least one interface call during the interface call process of the open graphics library. This allows each of the at least one interface call to be converted into at least one near-zero driving overhead data. The driving overhead of this near-zero driving overhead data is small, thus reducing the call overhead of the open graphics library. Then, this at least one near-zero driving overhead data is processed in parallel into corresponding rendering tasks and sent to the graphics processor. This integration into a single rendering task avoids the graphics processor wasting rendering time and resources through multiple renderings, allowing the graphics processor to quickly render and display the task. This solution does not change the original code of the open graphics library, is convenient and fast to operate, and effectively improves rendering efficiency by utilizing the low overhead of the near-zero driving overhead data combined with the advantages of parallel processing. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A schematic diagram illustrating the rendering process in related technologies;

[0025] Figure 2 This is a schematic diagram illustrating an application scenario of an embodiment of this application;

[0026] Figure 3 This is a flowchart of a rendering processing method according to an embodiment of this application;

[0027] Figure 4 This is a flowchart of a rendering processing method according to another embodiment of this application;

[0028] Figure 5 This is a schematic diagram illustrating the rendering process performed by the central processing unit (CPU) and graphics processing unit (GPU) in an embodiment of this application.

[0029] Figure 6 This is a schematic diagram illustrating the conversion performed in a thread after interception, as described in an embodiment of this application.

[0030] Figure 7 This is a schematic diagram illustrating parallel rendering for a scene that does not support CommandList, according to an embodiment of this application.

[0031] Figure 8 This is a schematic diagram illustrating parallel rendering for a scene that supports CommandList, according to an embodiment of this application.

[0032] Figure 9 This is a structural block diagram of a rendering processing apparatus according to an embodiment of this application;

[0033] Figure 10 This is a structural block diagram of a rendering processing apparatus according to another embodiment of this application;

[0034] Figure 11 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0035] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0036] The principles and spirit of this application will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this application, and are not intended to limit the scope of this application in any way. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0037] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0038] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0039] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0040] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0041] In this document, it should be understood that any quantity of elements in the accompanying drawings is for illustration rather than limitation, and any nomenclature is only used for distinction without any limiting meaning.

[0042] Term Glossary:

[0043] AZDO: Approaching Zero Driver Overhead, that is, the driver overhead of the CPU is close to zero.

[0044] buffer: buffer, also known as buffer register, including input buffer and output buffer.

[0045] OpenGL: Open Graphics Library.

[0046] CPU: Central Processing Unit.

[0047] GPU: Graphics Processing Unit.

[0048] VBO: Vertex Buffer Objects.

[0049] UBO: Uniform Buffer Object.

[0050] Texture: texture.

[0051] API: Application Programming Interface.

[0052] ANGLE: Almost Native Graphics Layer Engine.

[0053] OpenGLES: OpenGL for Embedded Systems, which is a subset of the OpenGL three-dimensional graphics API.

[0054] Vulkan: a cross-platform 2D and 3D drawing application programming interface.

[0055] Based on the above description of the background art, the following situations still exist in the related art:

[0056] OpenGL is a widely used cross-platform API for 3D rendering. By providing diverse graphics processing capabilities, it enables developers to efficiently utilize GPUs to complete complex graphics rendering tasks. However, with the rapid improvement of hardware performance and the increasing demands of applications, the traditional binding and state management methods of OpenGL have gradually revealed their limitations. For example, in parallelized scenarios with frequent binding, performance cannot be optimally utilized. Therefore, improving the efficiency of the OpenGL interface in high-performance, multi-threaded scenarios has become an important research direction in the industry.

[0057] like Figure 1 As shown, this generally corresponds to multiple interface calls in the traditional OpenGL architecture of the CPU (such as...). Figure 1 The interface calls shown (0 to 3) will use a single thread (e.g., Figure 1 The thread 0 in the middle processes the data serially, which requires frequent slot binding and unbinding operations for each interface call, making it inconvenient to use.

[0058] OpenGL employs a state machine model, influencing the rendering process by setting global states and executing API operations. In this model, GPU resources (such as VBOs, UBOs, and Textures) typically need to be bound to an OpenGL context to be used in the rendering pipeline. This binding pattern requires that each time a specific resource is manipulated, it must first be bound to the appropriate context before the operation is performed, resulting in numerous binding and unbinding operations. This pattern easily becomes a performance bottleneck during multiple draw calls, especially when the program frequently switches resources, significantly increasing API overhead.

[0059] The core goal of Google's ANGLE project is to translate OpenGL ES calls into Vulkan API calls, overcoming the performance limitations of traditional OpenGL interfaces. Leveraging Vulkan's underlying features, ANGLE can utilize GPU resources more efficiently, achieving lower CPU load and higher rendering performance. The specific implementation steps of ANGLE are as follows:

[0060] 1. OpenGLES API Capture: ANGLE intercepts and captures OpenGLES API calls at the application layer, acting as an intermediary layer.

[0061] 2. API Translation: After capturing OpenGLES API calls, ANGLE translates these calls into equivalent Vulkan API calls. Since Vulkan is a lower-level graphics API with more efficient resource control and command management capabilities, translation reduces the overhead of binding and state switching.

[0062] 3. Low-level rendering execution: Communicates with the GPU via the Vulkan API to achieve high-performance drawing calls. In this way, ANGLE can bypass the OpenGLES state machine model and use Vulkan's CommandBuffer and Pipeline for batch rendering calls, achieving higher parallelism and lower CPU-GPU synchronization overhead.

[0063] While ANGLE theoretically solves the performance bottleneck problem of traditional OpenGL ES, it also faces high adaptation costs in practical applications. The main reasons include:

[0064] (1) Compatibility challenges: OpenGLES and Vulkan have significant differences in resource management and state models, making it difficult to directly correspond and convert some features, especially in advanced features and customized hardware functions.

[0065] (2) Development and debugging workload: Translating OpenGLES into Vulkan API requires ensuring consistency and deep debugging on different hardware platforms to ensure consistent rendering effects and performance optimization.

[0066] (3) Adaptation complexity: ANGLE has a high adaptation cost, especially when it needs to be deeply optimized for specific hardware or applications. It requires a lot of resources to solve compatibility issues and ensure consistent rendering quality and performance on different devices.

[0067] Based on the above description, the principles and spirit of this application will be explained in detail below with reference to several representative embodiments.

[0068] refer to Figure 2 This diagram illustrates an application scenario of the rendering processing method provided in this application. The application scenario includes a central processing unit 101 (CPU) and a graphics processing unit 102 (GPU), which are connected via wired or wireless means. The CPU and GPU can be located in the same terminal device or in different terminal devices. Terminal devices include, but are not limited to, desktop computers, mobile phones, mobile computers, tablet computers, media players, smart wearable devices, personal digital assistants (PDAs), in-vehicle terminals, or other electronic devices capable of performing the above functions.

[0069] During the process of making interface calls using the Open Graphics Library, the Central Processing Unit 101 intercepts at least one interface call from the Open Graphics Library, and converts each of the at least one interface call into at least one near-zero driving overhead data. Thus, each interface call corresponds to one near-zero driving overhead data. Then, the at least one near-zero driving overhead data is processed in parallel into a rendering task, and the rendering task is sent to the Graphics Processing Unit 102. Upon receiving the rendering task, the Graphics Processing Unit 102 performs rendering processing on the task and finally displays the rendering result. This process does not change the original code of the Open Graphics Library, is convenient and fast, and effectively improves rendering efficiency by utilizing the low overhead of the near-zero driving overhead data and the advantages of parallel processing.

[0070] The following is combined Figure 2 The above application scenarios are used to describe the rendering processing method according to exemplary embodiments of this application. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect. On the contrary, the embodiments of this application can be applied to any applicable scenario.

[0071] This application provides a rendering processing method applied to a central processing unit (e.g., CPU).

[0072] like Figure 3 As shown, the rendering processing methods include:

[0073] Step 201: During the process of making interface calls in the Open Graphics Library, intercept at least one interface call of the Open Graphics Library.

[0074] In practice, when the central processing unit (e.g., CPU) controls the execution of graphics rendering, it uses the application layer to hook (e.g., use a hook) the interface calls of open graphics libraries (e.g., OpenGL) (e.g., glBindBuffer, glDrawArrays, etc.). This achieves the goal of intercepting the interface calls of open graphics libraries.

[0075] In this way, from the perspective of the application layer above the CPU, the original interface calls of the open graphics library can be maintained without changing the corresponding program code. By intercepting at least one interface call of the open graphics library through hooking (e.g., hooking), the conversion and optimization of the interface call can be controlled more freely, while providing an operation entry point for subsequent conversion of near-zero driver overhead data (e.g., AZDO).

[0076] Step 202: Convert at least one of the interface calls into at least one near-zero driver overhead data; wherein, the interface call and the near-zero driver overhead data correspond one-to-one.

[0077] In practice, to improve the conversion rate, the CPU will convert at least one interface call into at least one near-zero driver overhead data (e.g., AZDO). Since near-zero driver overhead data is data with driver overhead close to zero, it has the advantage of low overhead. Therefore, converting interface calls into near-zero driver overhead data (e.g., AZDO) can significantly reduce the CPU's driver overhead, thereby improving the efficiency of resource access.

[0078] Step 203: Perform parallel processing on at least one of the near-zero driving overhead data to obtain a rendering task, and send the rendering task to the graphics processor for the graphics processor to render and display the rendering task.

[0079] In practice, because near-zero drive overhead data (e.g., AZDO) has the characteristic of parallel processing, at least one near-zero drive overhead data (e.g., AZDO) is processed in parallel to form a rendering task that the graphics processing unit (e.g., GPU) can receive. The GPU then directly retrieves the corresponding rendering resources based on the near-zero drive overhead data (e.g., AZDO) in the rendering task and performs rendering. After rendering, the content is displayed on the screen for the user to view. This process does not alter the original code within the graphics processing unit (e.g., GPU), making it simple and convenient.

[0080] The above scheme intercepts at least one interface call during the open graphics library's interface calls. This allows each interface call to be converted into at least one near-zero driver overhead data. The low driver overhead of this near-zero driver overhead data reduces the call overhead of the open graphics library. Then, this near-zero driver overhead data is processed in parallel into corresponding rendering tasks and sent to the graphics processor. This integration into a single rendering task avoids the graphics processor wasting rendering time and resources through multiple rendering processes, allowing the graphics processor to quickly render and display the task. This scheme does not alter the original code of the open graphics library, is convenient and fast, and effectively improves rendering efficiency by leveraging the low overhead of near-zero driver overhead data combined with the advantages of parallel processing.

[0081] In some embodiments, step 202 includes:

[0082] Step 2021: Assign at least one of the interface calls to a thread.

[0083] Step 2022: For any of the interface calls, the thread is used to convert the interface call into the corresponding near-zero driver overhead data.

[0084] In practice, each time an open graphics library interface call is made, it is intercepted, and a thread is used to convert the interface call into near-zero driver overhead data corresponding to the interface call (e.g., AZDO).

[0085] The above approach fully utilizes the thread resources in the central processing unit, enabling threads to quickly convert interface calls and obtain data with near-zero driver overhead.

[0086] In some embodiments, step 2022 includes:

[0087] The process of using each thread to convert various interface calls is as follows:

[0088] Step 20221: For any of the aforementioned interface calls, use a thread to determine the cached data for that interface call.

[0089] In practice, each interface call will use a thread to generate corresponding cached data (e.g., glBufferData).

[0090] Step 20222: Retrieve the memory address corresponding to the cached data from the graphics processor (e.g., glGetBufferParameteri64v).

[0091] Step 20223: Associate and match the cached data with the corresponding memory address to obtain the near-zero driver overhead data.

[0092] In practice, cached data is typically placed in corresponding slots and retrieved from those slots when needed. This requires continuous retrieval from slots, which is inconvenient. Therefore, to accurately implement subsequent rendering calls by the image processor, the corresponding memory address is retrieved based on the cached data. This cached data is then associated and matched with its corresponding memory address to form near-zero driver overhead data (e.g., AZDO).

[0093] With the above approach, based on the near-zero driver overhead data (e.g., AZDO), rendering data can be called directly according to the memory address. This eliminates the need to put cached data into the corresponding slot or to retrieve it from the slot, thus saving driver overhead.

[0094] In some embodiments, step 20221 includes:

[0095] Step 202211: Generate a cache corresponding to the interface call.

[0096] In practice, the buffer generation process is executed in parallel for each interface call, so that each interface call gets a corresponding buffer (e.g., glGenBuffers).

[0097] Step 202212: Bind the cache to the context data corresponding to the Open Graphics Library to obtain the bound cache.

[0098] In practice, to enable the buffer to perform the corresponding functions, it is bound to the context data of the open graphics library (e.g., OpenGL). This binding process is also performed in parallel, so that each buffer is bound to its corresponding context data, resulting in bound buffers (e.g., glBindBuffers).

[0099] Among them, the context data corresponding to open graphics libraries (such as OpenGL) is the various graphics rendering functions.

[0100] Step 202213: Add data content to the bound cache to obtain a cache with data content, and use the cache with data content as the cached data.

[0101] In practice, after obtaining the binding buffer, there is no data in the binding buffer yet. Therefore, it is necessary to add the corresponding data content to be rendered to the binding buffer to obtain a buffer with data content, which is then used as cached data (e.g., glBuffersData).

[0102] The above method can quickly and accurately obtain the corresponding cached data, which facilitates further processing of the cached data in subsequent steps 20222 and 20223.

[0103] In some embodiments, step 203 includes:

[0104] Step 2031: Determine whether the Open Graphics Library supports command list processing functionality.

[0105] In practice, some open graphics libraries (e.g., OpenGL) support command list processing, while others do not. Therefore, it is necessary to first determine whether the open graphics library supports command list processing to determine which step to proceed to.

[0106] Step 2032: In response to determining that the command list processing function is not supported, at least one near-zero overhead data is processed in parallel to obtain at least one rendering subtask. The at least one rendering subtask is combined as a rendering task and sent to the graphics processor, so that the graphics processor can perform parallel rendering of the at least one rendering subtask and then superimpose the rendering results to obtain the rendering results and display them.

[0107] In practice, if the open graphics library (e.g., OpenGL) does not support command list processing, it cannot directly convert at least one near-zero overhead data (e.g., AZDO) into a rendering task command list. Instead, each near-zero overhead data (e.g., AZDO) needs to be converted in parallel to obtain one or more rendering subtasks. These rendering subtasks are then packaged and combined into a rendering task and sent to the graphics processor (e.g., GPU) for parallel rendering. Since each rendering subtask only renders a part of the corresponding image, the rendered parts of each rendering subtask need to be superimposed to obtain the final rendering result, which is then displayed on the screen.

[0108] Alternatively, in step 2033, in response to determining that the command list processing function is supported, at least one near-zero overhead data obtained is converted into a rendering task command list, and the rendering task command list is sent as a rendering task to the graphics processor so that the graphics processor can perform unified rendering based on the rendering task command list to obtain and display the rendering result.

[0109] In practice, if the open graphics library (e.g., OpenGL) supports command list processing, it can directly convert this near-zero overhead data (e.g., AZDO) into a series of rendering commands recorded in the rendering task command list (e.g., CommandList). The rendering task command list (e.g., CommandList) is then sent to the graphics processor (e.g., GPU). The graphics processor (e.g., GPU) can then perform parallel rendering processing according to the rendering commands in the rendering task command list (e.g., CommandList) to obtain the final rendering result, which is then displayed on the screen.

[0110] With the above approach, open graphics libraries (e.g., OpenGL) can process the obtained near-zero overhead data in parallel into corresponding rendering tasks, regardless of whether they support command list functionality. These tasks are then sent to a graphics processor (e.g., GPU) for unified rendering, and the resulting rendering results are displayed. This method can accelerate the rendering process of the graphics processor.

[0111] In some embodiments, step 2032 includes:

[0112] Step 20321: In response to determining that the command list processing function is not supported, at least one of the near-zero overhead data is transformed in parallel using at least one thread to obtain a rendering subtask corresponding to each of the threads.

[0113] Step 20322: Integrate at least one of the rendering subtasks to form the rendering task, and send the rendering task to the graphics processor.

[0114] In practice, each thread can perform parallel transformation processing on near-zero-overhead data, converting it into a corresponding rendering subtask. This results in a number of rendering subtasks equal to the number of threads processing in parallel. These rendering subtasks are then integrated into a single rendering task using shared resources (such as shared VBOs or FBOs).

[0115] The above solution provides a way to process at least one near-zero overhead data in parallel when the open image library does not support command list processing. It can utilize the resources of at least one thread, reduce the waste of thread resources, and improve processing speed and efficiency, effectively avoiding the bottleneck of single-threaded processing.

[0116] In some embodiments, step 2033 includes:

[0117] Step 20331: In response to determining that the command list processing function is supported, at least one rendering task command is generated for each of the at least one near-zero overhead data, and at least one of the rendering task commands is recorded in the rendering task command list.

[0118] Step 20332: Send the rendering task command list as a rendering task to the graphics processor so that the graphics processor can perform unified rendering based on the rendering task command list to obtain and display the rendering result.

[0119] In practice, if an open graphics library (e.g., OpenGL) supports command list processing, the subsequent process is relatively simple. These near-zero overhead data are directly converted into corresponding rendering task commands and recorded in the rendering task command list. The list is then sent to the graphics processor, which can then perform unified rendering in the corresponding window based on the various rendering task commands in the rendering task command list, thereby obtaining the rendering result of the window and displaying the rendering result on the screen.

[0120] The above approach, which uses a unified rendering method based on a list of rendering commands, is more convenient and faster, and can further improve the rendering speed.

[0121] Based on the same concept, the rendering processing method proposed in the embodiments of this application is applied to a graphics processor (e.g., a GPU).

[0122] like Figure 4 As shown, the rendering process includes:

[0123] Step 301: Receive a rendering task fed back by the central processing unit, wherein the rendering task includes at least one near-zero drive overhead data.

[0124] In specific implementation, the graphics processor receives the rendering task obtained from steps 201 to 203 of the rendering processing method executed by the central processing unit. This process is the same as the process in steps 201 to 203 above, and will not be described again.

[0125] Step 302: Render the data based on at least one of the near-zero driving overhead data in the rendering task to obtain a rendering result, and display the rendering result on the screen.

[0126] In practice, the graphics processor directly retrieves the corresponding rendering data from the memory address corresponding to the near-zero drive overhead data in the rendering task, performs rendering processing based on the rendering data, and obtains the final rendering result to be displayed on the screen.

[0127] Using the above approach, the graphics processor can achieve fast rendering and display based on the near-zero driving overhead data in the rendering task. This approach does not change the original code of the open graphics library, is convenient and quick to operate, and can effectively improve rendering efficiency by taking advantage of the low overhead of near-zero driving overhead data.

[0128] In some embodiments, step 302 includes:

[0129] Step 3021: Determine that the rendering task includes at least one rendering subtask. For each rendering subtask, determine the near-zero driving overhead data in the rendering subtask, and perform off-screen rendering according to the windowless open graphics library context corresponding to the near-zero driving overhead data to obtain the off-screen rendering result corresponding to the rendering subtask.

[0130] Step 3022: Overlay and combine at least one of the off-screen rendering results to obtain a rendering result, and display the rendering result on the rendering window of the screen.

[0131] In practice, the rendering task sent by the central processing unit (CPU) contains at least one rendering subtask, proving that the rendering task was obtained through step 2032 above. Then, the near-zero driver overhead data in each rendering subtask is used to determine its corresponding windowless Open Graphics Library (OPL) context. This windowless OPL context runs in memory and is not displayed on the screen. Off-screen rendering is performed based on the windowless OPL context to obtain the corresponding off-screen rendering result. The graphics processor then combines all the off-screen rendering results corresponding to the rendering task to obtain the corresponding rendering result, which is displayed on the rendering window on the screen for the user to view.

[0132] With the above approach, for a rendering task that includes at least one rendering subtask, the graphics processor performs off-screen rendering on these subtasks and then combines them together. This achieves the goal of parallel rendering without changing the original hardware structure of the graphics processor.

[0133] In some embodiments, step 302 includes:

[0134] Step 3021': Determine that the received rendering task includes a rendering task command list, and retrieve at least one rendering task command from the rendering task command list.

[0135] Step 3022': Construct a rendering window, retrieve rendering resources according to the near-zero driving overhead data corresponding to each rendering task command, perform rendering in the rendering window, and display the rendering window on the screen after rendering is completed.

[0136] In practice, if the rendering task received by the graphics processor is a list of rendering task commands, the rendering process is relatively simpler. The memory addresses of the near-zero overhead data of the rendering task commands in the list can be directly extracted, and the corresponding rendering data can be retrieved. Rendering is then performed directly in the rendering window based on this data. After confirming that the rendering window has finished rendering, the rendered window is displayed directly on the screen.

[0137] The above approach, which uses a unified rendering command list, is more convenient and faster, reduces the overhead of calling each interface individually, and further improves rendering efficiency.

[0138] The following is a detailed description of the rendering process executed by the central processing unit (CPU) and the graphics processing unit (GPU) as an example:

[0139] 1. Hook OpenGL standard interface calls (i.e., interface calls)

[0140] First, such as Figure 5 As shown, the application layer of the central processing unit (CPU) hooks the OpenGL standard interface (i.e., intercepts at least one interface call from the open graphics library). This means that when a program calls standard OpenGL APIs (such as glBindBuffer, glDrawArrays, etc.), these interface calls are intercepted and subsequently converted into calls with AZDO features (specifically executed in step 2). The significance of this process is:

[0141] (1) Transparency: For the upper application layer, the standard OpenGL API call format is maintained, and the application code does not need to be changed.

[0142] (2) Compatibility: By intercepting standard API calls, it ensures compatibility with OpenGL applications and OpenGL context, especially in cases where direct use of AZDO optimization technology is not supported, thus leveraging the performance advantages of AZDO.

[0143] By using hooks, we can have more freedom to control the transformation and optimization of API interface calls, while providing an entry point for the subsequent AZDO optimization (i.e., the transformation process executed in step 2).

[0144] 2. Convert OpenGL standard API (i.e., interface calls) into AZDO Bindless Graphics operations.

[0145] like Figure 6 As shown, after hooking to the OpenGL standard API (i.e., interface calls), the OpenGL standard API (such as binding buffers, textures, etc.) is converted into BindlessGraphics calls. First, a buffer corresponding to the interface call (e.g., glGenBuffers) is generated. Then, the buffer is bound to the context data corresponding to the OpenGL library to obtain a bound buffer (e.g., glBindBuffers). Then, data content is added to the bound buffer to obtain cached data with data content (e.g., glBuffersData). Then, the memory address corresponding to the cached data is retrieved from the graphics processor (e.g., glGetBufferParameteri64V). Then, the cached data is associated and matched with the corresponding memory address (e.g., glMakeBufferHandleResidentARB) to obtain the near-zero driving overhead data (e.g., glBufferAddressRangeNV).

[0146] The main objectives are as follows:

[0147] (1) Bindless VBO, UBO, Texture: Once a resource is created, it is bound to a persistent memory address on the GPU to achieve "unbound" access. The specific process is as follows:

[0148] (11) Convert to bindlesshandle: When the application first creates a VBO, UBO or Texture, a bindlesshandle (i.e., near-zero overhead data) is generated, so that these resources can be accessed directly through the GPU's memory address, instead of having to build a cache, bind the cache, add data content, and bind it to the corresponding slot before each rendering call.

[0149] (12) Resource persistence: By using the converted near-zero overhead data (e.g., AZDO interfaces such as glMakeBufferResidentNV and glMakeTextureHandleResidentARB), the access to these near-zero overhead data resources is persisted throughout the rendering process, avoiding frequent operations of binding to and unbinding from the corresponding slots.

[0150] (2) Performance improvement: By converting to bindlesshandle (i.e., near-zero overhead data), the CPU no longer needs to frequently bind and manage resource states every time it is updated or drawn. This conversion greatly reduces the CPU driver overhead of OpenGL, making resource access more efficient.

[0151] In this mode, standard OpenGL APIs (i.e., interface calls) are converted into unbound, near-zero overhead data that directly accesses GPU addresses, reducing the CPU-GPU synchronization frequency and thus improving performance.

[0152] 3. In scenarios that do not support CommandList (i.e., command list processing functionality), the parallel rendering process is performed as follows:

[0153] like Figure 7 As shown, the near-zero overhead data obtained by each thread forms a rendering subtask. In this way, the GPU's various rendering subtasks construct multiple windowless OpenGL contexts in parallel for offscreen rendering. These contexts can run in different threads, with each context responsible for a different rendering subtask (such as...). Figure 7 Threads 0 to 2 correspond to rendering subtasks 0 to 2. Then the GPU completes all rendering subtasks to obtain their respective off-screen rendering results (e.g., ...). Figure 7 After rendering the off-screen result (0 to 2), the completion instruction is sent to the CPU. The CPU generates compositing instructions, which are then sent to the GPU. The GPU creates an OpenGL context with a window (i.e., a rendering window) to display the final rendering result (e.g., ...). Figure 7 (As shown).

[0154] By sharing resources between OpenGL contexts (e.g., VBOs, textures, framebuffer objects, FBOs), rendering results can be shared between multiple contexts. Shared resources can be textures, buffers, etc., which are used to render content in the off-screen rendering context and are ultimately passed to the context with a window for display.

[0155] Use synchronization mechanisms (such as glFenceSync and glWaitSync) to ensure that rendering tasks across multiple contexts do not conflict with each other. Synchronization ensures that rendering operations in each context are executed in the correct order, avoiding data races or the misuse of incomplete rendering operations.

[0156] 4. For scenarios that support CommandList (i.e., command list processing functionality), execute the parallel rendering process:

[0157] like Figure 8 As shown, CommandList technology allows for the pre-recording of drawing and state setting operations. The CPU converts standard API calls (i.e., interface calls) into near-zero overhead data (AZDO), which is then distributed to various threads (such as...). Figure 8 After the transformation of threads 0 to 2, all near-zero overhead data (AZDO) obtained are converted into a CommandList (i.e., a list of rendering tasks) to form rendering tasks. This allows the CPU to submit rendering tasks to the GPU in batches, enabling the GPU to perform unified rendering and obtain the rendering results, thus improving rendering efficiency. The specific implementation idea is as follows:

[0158] (1) Record the rendering task command list (CommandList): Before the GPU performs graphics rendering, the CPU uses OpenGL at the application layer to convert the near-zero overhead data (AZDO) after standard API calls (i.e. interface calls), converts the near-zero overhead data (AZDO) into a set of rendering task commands, and records them in the rendering task command list (CommandList).

[0159] (2) Batch submission: When drawing is required, the entire list of rendering task commands is submitted to the GPU for execution at once, thus avoiding the situation of submitting each command one by one as in traditional API calls.

[0160] Using a command list for rendering tasks for unified batch rendering reduces the frequency of API calls and allows the GPU to process instructions more efficiently in batches, thereby achieving higher rendering performance.

[0161] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0162] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0163] Based on the same concept, and the above Figure 3 Corresponding to the rendering processing method of the corresponding embodiment, this application also provides a rendering processing apparatus, which is disposed in a central processing unit (e.g., CPU).

[0164] refer to Figure 9 The device includes:

[0165] Interception module 401 is configured to intercept at least one interface call of the open graphics library during the interface call process of the open graphics library;

[0166] The conversion module 402 is configured to convert at least one of the interface calls into at least one near-zero driver overhead data respectively; wherein, the interface call and the near-zero driver overhead data correspond one-to-one.

[0167] The parallel processing module 403 is configured to process at least one of the near-zero driving overhead data in parallel to obtain a rendering task, and send the rendering task to the graphics processor for the graphics processor to render and display the rendering task.

[0168] In some embodiments, the conversion module 402 includes:

[0169] A thread allocation unit is configured to allocate at least one of the interface calls to a thread;

[0170] The conversion unit is configured to, for any of the aforementioned interface calls, use the thread to convert the interface call into the corresponding near-zero driver overhead data.

[0171] In some embodiments, the conversion unit is specifically configured as follows:

[0172] For any of the aforementioned interface calls, a thread is used to determine the cached data for that interface call;

[0173] The memory address corresponding to the cached data is retrieved from the graphics processor using a thread;

[0174] The cached data is associated and matched with the memory address using a thread to obtain the near-zero driver overhead data corresponding to the interface call.

[0175] In some embodiments, the conversion unit is specifically configured as follows:

[0176] Generate a cache corresponding to the API call;

[0177] The buffer is bound to the context data corresponding to the open graphics library to obtain the bound buffer;

[0178] Add data content to the bound cache to obtain a cache with data content, and use the cache with data content as the cached data.

[0179] In some embodiments, the parallel processing module 403 includes:

[0180] The judgment unit is configured to determine whether the open image library supports command list processing functionality;

[0181] A rendering task parallel conversion unit is configured to, in response to determining that command list processing functionality is not supported, process at least one near-zero overhead data in parallel to obtain at least one rendering subtask, combine the at least one rendering subtask as a rendering task, and send it to the graphics processor, so that the graphics processor can perform parallel rendering of the at least one rendering subtask and then superimpose the results to obtain a rendering result for display; or...

[0182] The list transformation unit is configured to, in response to determining that it supports the command list processing function, process at least one near-zero overhead data in parallel into a rendering task command list, and send the rendering task command list as a rendering task to the graphics processor, so that the graphics processor can perform unified rendering based on the rendering task command list to obtain and display the rendering result.

[0183] In some embodiments, the rendering task parallel transformation unit is specifically configured as follows:

[0184] In response to determining that the command list processing function is not supported, at least one of the near-zero overhead data is transformed in parallel using at least one thread to obtain a rendering subtask corresponding to each of the threads;

[0185] At least one of the rendering subtasks is integrated to form the rendering task, and the rendering task is sent to the graphics processor.

[0186] In some embodiments, the list transformation unit is specifically configured as follows:

[0187] In response to determining that the command list processing function is supported, at least one rendering task command is generated from at least one of the near-zero overhead data, and at least one of the rendering task commands is recorded in the rendering task command list.

[0188] The rendering task command list is sent as a rendering task to the graphics processor, so that the graphics processor can perform unified rendering based on the rendering task command list to obtain and display the rendering result.

[0189] Based on the same concept, and the above Figure 4 Corresponding to the rendering processing method of the corresponding embodiment, this application also provides a rendering processing apparatus disposed in a graphics processor (e.g., GPU).

[0190] like Figure 10 As shown, the device includes:

[0191] The rendering task receiving module 501 is configured to receive rendering tasks fed back by the central processing unit, wherein the rendering task includes at least one near-zero driving overhead data.

[0192] The rendering module 502 is configured to render a rendering result based on at least one of the near-zero driving overhead data in the rendering task, and to display the rendering result on the screen.

[0193] In some embodiments, the rendering module 502 includes a first rendering unit, configured to:

[0194] The rendering task is determined to include at least one rendering subtask. For each rendering subtask, the near-zero driving overhead data in the rendering subtask is determined. Off-screen rendering is performed according to the windowless open graphics library context corresponding to the near-zero driving overhead data to obtain the off-screen rendering result corresponding to the rendering subtask.

[0195] At least one of the off-screen rendering results is superimposed and combined to obtain a rendering result, which is then displayed on the rendering window of the screen.

[0196] In some embodiments, the rendering module 502 includes a second rendering unit, configured to:

[0197] The received rendering task is determined to include a list of rendering task commands, and at least one rendering task command from the list of rendering task commands is retrieved.

[0198] A rendering window is constructed, rendering resources are retrieved according to the near-zero driving overhead data corresponding to each rendering task command, rendering is performed in the rendering window, and the rendering window after rendering is completed is displayed on the screen.

[0199] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0200] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0201] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in any of the above embodiments.

[0202] Figure 11 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0203] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0204] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0205] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0206] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0207] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0208] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0209] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0210] Based on the same concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.

[0211] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0212] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0213] Based on the same concept, corresponding to any of the above embodiments, this application also provides a computer program product, including computer program instructions, which, when run on a computer, cause the computer to perform the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0214] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0215] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0216] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0217] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A rendering processing method, characterized in that, include: During the process of making interface calls to the open graphics library, at least one interface call of the open graphics library is intercepted; Each of the above-mentioned interface calls is converted into at least one near-zero driver overhead data; wherein, the interface call and the near-zero driver overhead data correspond one-to-one. At least one of the near-zero driving overhead data is processed in parallel to obtain a rendering task, and the rendering task is sent to the graphics processor for the graphics processor to render and display the rendering task.

2. The method according to claim 1, characterized in that, Converting at least one of the aforementioned interface calls into at least one near-zero driver overhead data, including: Assign at least one of the aforementioned interface calls to a thread; For any of the aforementioned interface calls, the thread is used to convert the interface call into the corresponding near-zero driver overhead data.

3. The method according to claim 2, characterized in that, For any of the aforementioned interface calls, the thread converts the interface call into the corresponding near-zero driver overhead data, including: For any of the aforementioned interface calls, a thread is used to determine the cached data for that interface call; The memory address corresponding to the cached data is retrieved from the graphics processor using a thread; The cached data is associated and matched with the memory address using a thread to obtain the near-zero driver overhead data corresponding to the interface call.

4. The method according to claim 3, characterized in that, The determination of the cached data for the interface call includes: Generate a cache corresponding to the API call; The buffer is bound to the context data corresponding to the open graphics library to obtain the bound buffer; Add data content to the bound cache to obtain a cache with data content, and use the cache with data content as the cached data.

5. The method according to claim 1, characterized in that, Parallel processing of at least one of the near-zero driving overhead data to obtain a rendering task, and sending the rendering task to a graphics processor for rendering and display, including: Determine whether the open image library supports command list processing functionality; In response to the determination that command list processing functionality is not supported, at least one near-zero overhead data is processed in parallel to obtain at least one rendering subtask. The at least one rendering subtask is then combined into a rendering task and sent to the graphics processor, which performs parallel rendering of the at least one rendering subtask before overlaying the results for display; or... In response to determining that the command list processing function is supported, at least one near-zero overhead data obtained is processed in parallel to form a rendering task command list. The rendering task command list is then sent as a rendering task to the graphics processor, so that the graphics processor can perform unified rendering based on the rendering task command list to obtain and display the rendering result.

6. The method according to claim 5, characterized in that, In response to determining that command list processing functionality is not supported, the process of parallel processing at least one near-zero overhead data to obtain at least one rendering subtask, and combining the at least one rendering subtask as a rendering task and sending it to the graphics processor, includes: In response to determining that the command list processing function is not supported, at least one of the near-zero overhead data is transformed in parallel using at least one thread to obtain a rendering subtask corresponding to each of the threads; At least one of the rendering subtasks is integrated to form the rendering task, and the rendering task is sent to the graphics processor.

7. The method according to claim 5, characterized in that, In response to determining that the command list processing function is supported, the step of converting at least one near-zero overhead data into a rendering task command list and sending the rendering task command list as a rendering task to the graphics processor includes: In response to determining that the command list processing function is supported, at least one rendering task command is generated from at least one of the near-zero overhead data, and at least one of the rendering task commands is recorded in the rendering task command list. The list of rendering task commands is sent as rendering tasks to the graphics processor, so that the graphics processor can perform unified rendering based on the list of rendering task commands to obtain and display the rendering results.

8. A rendering processing method, characterized in that, include: Receive rendering tasks from the central processing unit, wherein the rendering tasks include at least one near-zero drive overhead data; Rendering results are obtained by rendering based on at least one of the near-zero driving overhead data in the rendering task, and the rendering results are displayed on the screen.

9. The method according to claim 8, characterized in that, The process of rendering based on at least one of the near-zero driving overhead data in the rendering task to obtain a rendering result, and displaying the rendering result on the screen, includes: The rendering task is determined to include at least one rendering subtask. For each rendering subtask, the near-zero driving overhead data in the rendering subtask is determined. Off-screen rendering is performed according to the windowless open graphics library context corresponding to the near-zero driving overhead data to obtain the off-screen rendering result corresponding to the rendering subtask. At least one of the off-screen rendering results is superimposed and combined to obtain a rendering result, which is then displayed on the rendering window of the screen.

10. The method according to claim 8, characterized in that, The process of rendering based on at least one of the near-zero driving overhead data in the rendering task to obtain a rendering result, and displaying the rendering result on the screen, includes: The received rendering task is determined to include a list of rendering task commands, and at least one rendering task command from the list of rendering task commands is retrieved. A rendering window is constructed, rendering resources are retrieved according to the near-zero driving overhead data corresponding to each rendering task command, rendering is performed in the rendering window, and the rendering window after rendering is completed is displayed on the screen.

11. A rendering processing apparatus, characterized in that, include: The interception module is configured to intercept at least one interface call of the open graphics library during the interface call process; wherein, the interface call corresponds one-to-one with the near-zero driver overhead data; The conversion module is configured to convert at least one of the interface calls into at least one near-zero driver overhead data, respectively; A parallel processing module is configured to process at least one of the near-zero driving overhead data in parallel to obtain a rendering task, and send the rendering task to a graphics processor for the graphics processor to render and display the rendering task.

12. A rendering processing apparatus, characterized in that, include: The rendering task receiving module is configured to receive rendering tasks fed back by the central processing unit, wherein the rendering task includes at least one near-zero driving overhead data. The rendering module is configured to render a result based on at least one of the near-zero driving overhead data in the rendering task, and to display the rendering result on the screen.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 10.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method as described in any one of claims 1 to 10.

15. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed on a computer, the computer causes the computer to perform the method as described in any one of claims 1 to 10.

Images