A data processing method, apparatus and electronic device

Abstract

Embodiments of the present application disclose a data processing method, apparatus, and electronic device, which relate to the field of image processing, enabling the electronic device to have the rendering ability for a variety of different rendering instructions while only maintaining the Vulkan driver, while improving the instruction conversion efficiency. The specific solution is as follows: The electronic device configures at least two resource groups according to the first graphics API, and then the electronic device can use the at least two resource groups to concurrently convert a plurality of rendering instructions corresponding to the first graphics API.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular, to a data processing method, apparatus, and electronic device. Background Art

[0002] When an application runs on an electronic device, it needs to call various application programming interfaces (APIs) to implement its functions. For example, when an image needs to be displayed, the application needs to call the graphics API to render the data of each frame image according to the rendering instructions issued by the application. Thus, the rendered image data is obtained and displayed on the display screen.

[0003] In different operating systems (such as Android, or Windows, etc.), the graphics APIs may be different. For example, in the Android system, the Open Graphics Library for Embedded Systems (OpenGL ES) is most widely used in embedded systems. Another example is that in the Windows system, OpenGL corresponding to OpenGL ES has also been widely applied. Taking the Android operating system as an example, as the requirements for graphics processing on electronic devices are getting higher and higher, Vulkan can provide developers with more control and a more transparent programming design, thus greatly reducing the "API overhead" of the processor when providing important features, performance, and image quality. At the same time, it can greatly improve the performance of 3D applications by giving developers the ability to access the underlying hardware. Therefore, Vulkan may gradually replace OpenGL ES. During the process of replacing the two APIs, Vulkan and OpenGL ES will coexist for a long time. Similar to the Android system, since Vulkan can support the Windows system at the same time and has the above advantages, it will also gradually replace OpenGL. In the Windows system, the image rendering requirements for Vulkan and OpenGL will also coexist for a long time. The following will be described taking the Android system as an example.

[0004] The operation of Vulkan and OpenGL ES requires the support of their respective underlying drivers. Therefore, in order to support both Vulkan and OpenGL ES at the same time, the electronic device needs to maintain the underlying drivers corresponding to Vulkan and OpenGL ES respectively, and the code volume exceeds 700,000 lines, which will bring a huge burden to the electronic device.

[0005] In order to reduce the burden on an electronic device for maintaining a large number of underlying drivers, currently, the underlying driver corresponding to Vulkan with a relatively small amount of code (about 150,000 lines) can be maintained in the electronic device. When OpenGL ES image rendering is required, the instructions of OpenGL ES are converted into the instructions of Vulkan so that they can be processed in the underlying driver corresponding to Vulkan. In this way, while the electronic device does not need to maintain the underlying driver corresponding to OpenGL ES, it can also achieve OpenGL ES image rendering. However, the instruction conversion process in the current solution is inefficient, resulting in low efficiency of OpenGL ES image rendering. Summary of the Invention

[0006] Embodiments of the present application provide a data processing method, apparatus, and electronic device. By parallelly converting rendering instructions, it is possible to simultaneously convert multiple OpenGL instructions or OpenGL ES instructions into Vulkan instructions, thereby improving the instruction conversion efficiency and enabling the electronic device to have the rendering ability for multiple different rendering instructions while only maintaining the Vulkan driver. Since the conversion of multiple rendering instructions can be executed in parallel, the rendering efficiency of rendering instructions including OpenGL ES instructions can be improved.

[0007] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a data processing method is provided, which is applied to an electronic device. The electronic device maintains a first driver, and the first driver corresponds to a first graphics application programming interface (API). The method includes: The electronic device configures at least two resource groups according to the first graphics API, and the resource groups are used to convert rendering instructions. The electronic device converts a first rendering instruction through a first resource group to obtain a second rendering instruction. The first rendering instruction is a rendering instruction corresponding to a second graphics API, and the second rendering instruction is a rendering instruction corresponding to the first graphics API. The first resource group is included in the at least two configured resource groups. The electronic device executes the second rendering instruction through the first driver.

[0009] Based on this solution, a specific example of the data processing method described in this application is provided. In this example, the electronic device can perform parallel conversion on one or more rendering instructions through multiple configured resource groups, thereby improving the efficiency of rendering instruction conversion. This enables the electronic device to have the ability to process rendering instructions corresponding to multiple different graphics APIs while only maintaining a driver corresponding to one graphics API. It can be understood that since the conversion of a subsequent rendering instruction does not need to wait for the completion of the previous rendering instruction, but multiple rendering instructions can be converted in parallel and processed at a reasonable time, the processing efficiency of rendering instructions can be significantly improved, especially for rendering instructions that require instruction conversion, and the improvement effect is particularly obvious.

[0010] In a possible design, the method further includes: the electronic device converts a third rendering instruction through a second resource group to obtain a fourth rendering instruction. The third rendering instruction is a rendering instruction corresponding to a second graphics API, and the fourth rendering instruction is a rendering instruction corresponding to a first graphics API. The second resource group is included in at least two configured resource groups. The electronic device executes the fourth rendering instruction through a first driver. Based on this solution, another example in this solution is provided. In this example, the multiple resource groups configured by the electronic device can also be used to perform instruction conversion on other rendering instructions except the first rendering instruction, such as the third rendering instruction.

[0011] In a possible design, the conversion of the first rendering instruction and the conversion of the third rendering instruction are executed in parallel. Based on this solution, it is clear that in this solution, when there are multiple instructions that need to be converted, such as the conversion of the first rendering instruction and the third rendering instruction, the electronic device can use multiple different resource groups to separately convert different instructions, thereby avoiding the problem of low efficiency introduced by serial execution.

[0012] In a possible design, the electronic device configures at least two resource groups according to the first graphics API, including: the electronic device configures N resource groups according to the maximum number of frame buffers M supported by the first graphics API, where N is an integer greater than 1 and less than or equal to M. Based on this solution, a solution for configuring resource groups is proposed. It can be understood that different graphics APIs can support different maximum numbers of frame buffers. Therefore, in this example, the electronic device can configure the corresponding number of resource groups according to the maximum number of frame buffers M supported by the first graphics API (for example, M can be 3). So that the N rendering instructions obtained after parallel instruction conversion through N (for example, N can be set to 3 equal to M) resource groups can be cached in the cache corresponding to the first graphics API.

[0013] In a possible design, the data to be rendered indicated by the first rendering instruction is the data of the P-th frame image, and the data to be rendered indicated by the third rendering instruction is the data of the (P + 1)-th frame image, where P is an integer greater than or equal to 1. Based on this solution, an example of the relative relationship between the first rendering instruction and the third rendering instruction is shown. For example, the first rendering instruction can be one of a plurality of consecutive rendering instructions for different frame images issued by an application in an electronic device, and the third rendering instruction can be another one of the consecutive rendering instructions for different frame images. These two rendering instructions can be adjacent. Of course, these two rendering instructions can also be non-adjacent. For different rendering instructions, the conversion of the instructions can be performed in parallel according to the method described in this application.

[0014] In a possible design, a rendering execution module is provided in the electronic device, and the rendering execution module is used to perform a rendering operation according to a rendering instruction. The electronic device executes a fourth rendering instruction through a first driver, including: when the number of instructions of the second rendering instruction is less than a first threshold, the electronic device submits the fourth rendering instruction to the rendering execution module so that the rendering execution module executes the fourth rendering instruction. Based on this solution, a solution for performing a rendering operation according to the converted rendering instruction is provided. It can be understood that when submitting the rendering instruction corresponding to the current frame image, the rendering execution module may have a large pressure due to the excessive number of instructions of the rendering instruction corresponding to the previous frame image. To avoid the situation where submitting the rendering instruction corresponding to the current frame image causes too much processing pressure on the rendering execution module, in this example, before submitting the rendering instruction of the current frame image, it can be determined whether the number of instructions of the rendering instruction of the previous frame image is less than the first threshold. If so, it is considered that the pressure on the rendering execution module is not large. At this time, the current rendering instruction can be directly submitted for quick execution. Correspondingly, when the number of instructions of the rendering instruction of the previous frame image is greater than the first threshold, it is considered that the current processing pressure of the rendering execution module is large. At this time, the submission of the rendering instruction corresponding to the current frame image can be paused and then submitted when the processing pressure of the rendering execution module is small, thereby achieving the effect of flexibly adjusting the submission timing of the rendering instruction of the current frame image according to the processing pressure of the rendering execution module.

[0015] It should be understood that the rendering execution module in this application can be understood as a graphics processing unit (GPU) from a hardware perspective and as a computer program that the GPU needs to read and implement from a software perspective.

[0016] In a possible design, a rendering execution module is provided in the electronic device, and the rendering execution module is used to perform a rendering operation according to a rendering instruction. The electronic device executes a fourth rendering instruction through a first driver, including: when the number of instructions of the second rendering instruction is greater than a first threshold and the number of instructions of the fourth rendering instruction is less than a second threshold, the electronic device submits the fourth rendering instruction to the rendering execution module so that the rendering execution module can execute the fourth rendering instruction. Based on this solution, the electronic device can determine the submission timing of the rendering instruction of the current frame image by referring to the number of rendering instructions of the current frame image on the basis of the number of rendering instructions of the previous frame image. For example, taking both the first threshold and the second threshold as 100 as an example. If the number of rendering instructions of the previous frame image is less than 100, then the rendering instruction of the current frame image can be directly submitted. If the number of rendering instructions of the previous frame image is more than 100, then when the number of rendering instructions of the current frame image is less than 100, it can be considered that the submission of the current frame image will not bring too much execution pressure to the rendering execution module, so the rendering instruction of the current frame image can be directly submitted. If the number of instructions of the previous frame image is more than 100 and the number of instructions of the current frame image is also more than 100, then the submission of the rendering instruction of the current frame image may cause excessive pressure on the rendering execution module. Therefore, in this case, the electronic device can wait for a period of time (such as a preset duration, which can be pre-set or determined according to the number of instructions of the current frame image and / or the number of instructions of the previous frame image). After this time, the electronic device then submits the rendering instruction of the current frame image, thereby preventing the rendering execution module from being overloaded due to the submission of the current frame image. It should be noted that the above first threshold and second threshold may also be different, and the configuration of the threshold can be pre-set or determined by the electronic device itself.

[0017] In a possible design, a rendering execution module is provided in the electronic device, and the rendering execution module is used to perform a rendering operation according to a rendering instruction. The method further includes: the electronic device converts a fifth rendering instruction through a third resource group to obtain a sixth rendering instruction, where the fifth rendering instruction is a rendering instruction corresponding to a second graphics API, the sixth rendering instruction is a rendering instruction corresponding to a first graphics API, and the third resource group is included in at least two configured resource groups. The data to be rendered indicated by the fifth rendering instruction is the data of the (P + 2)-th frame image. The electronic device executes a fourth rendering instruction through a first driver, including: when the number of instructions of the second rendering instruction is greater than a first threshold, the number of instructions of the fourth rendering instruction is greater than a second threshold, and the number of instructions of the sixth rendering instruction is greater than a third threshold, the electronic device submits the fourth rendering instruction to the rendering execution module so that the rendering execution module executes the fourth rendering instruction. Based on this solution, the electronic device can also comprehensively determine the submission timing of the rendering instruction of the current frame image in combination with the number of rendering instructions of the previous frame image, the number of rendering instructions of the current frame image, and the number of rendering instructions of the subsequent frame image. For example, still taking the first threshold, the second threshold, and the third threshold all being 100 as an example. If the number of rendering instructions of the previous frame image is less than 100, the rendering instruction of the current frame image can be directly submitted. If the number of rendering instructions of the previous frame image is greater than 100 and the number of rendering instructions of the current frame image is less than 100, the rendering instruction of the current frame image can be directly submitted. If the number of instructions of both the current frame image and the previous frame image is greater than 100, then the electronic device can determine whether to directly submit the rendering instruction of the current frame image according to the number of instructions of the subsequent frame image. It can be understood that if the number of instructions of the subsequent frame image is large, such as greater than 100, then the delayed submission of the current frame image may cause the rendering process of the next frame image to be untimely, thereby affecting subsequent displays. Therefore, in this example, if the number of instructions of both the current frame image and the previous frame image is greater than 100, then if the number of instructions of the subsequent frame image is also greater than 100, the rendering instruction of the current frame image is directly submitted. If the number of instructions of both the current frame image and the previous frame image is greater than 100, then if the number of instructions of the subsequent frame image is less than 100, the rendering instruction of the current frame image can be submitted after waiting until the processing pressure of the rendering execution module is smaller. It should be noted that the above first threshold, second threshold, and third threshold may also be different or partially different, and the configuration of the threshold can be set in advance or determined by the electronic device itself.

[0018] In a possible design, the first graphics API is Vulkan. The second graphics API is the Open Graphics Library (OpenGL), or the second graphics API is the Embedded Open Graphics Library (OpenGL ES). Based on this solution, an implementation scenario of the current solution is clarified. For example, when the electronic device maintains the driver of Vulkan and needs to process the rendering instructions of OpenGL or OpenGL ES, the efficient conversion of the rendering instructions and subsequent execution can be achieved through the above solution.

[0019] In a second aspect, a data processing device is provided, which is applied to an electronic device that maintains a first driver corresponding to a first graphics application programming interface (API). The device includes: a configuration unit configured to configure at least two resource groups according to the first graphics API, where the resource groups are used to convert rendering instructions; a conversion unit configured to convert a first rendering instruction through a first resource group to obtain a second rendering instruction, where the first rendering instruction is a rendering instruction corresponding to the second graphics API, and the second rendering instruction is a rendering instruction corresponding to the first graphics API, and the first resource group is included in the at least two configured resource groups; and an execution unit configured to execute the second rendering instruction through the first driver.

[0020] In a possible design, the conversion unit is further configured to convert a third rendering instruction through a second resource group to obtain a fourth rendering instruction, where the third rendering instruction is a rendering instruction corresponding to the second graphics API, and the fourth rendering instruction is a rendering instruction corresponding to the first graphics API, and the second resource group is included in the at least two configured resource groups. The execution unit is further configured to execute the fourth rendering instruction through the first driver.

[0021] In a possible design, the conversion unit converts the first rendering instruction and the third rendering instruction in parallel.

[0022] In a possible design, the configuration unit is configured to configure N resource groups according to the maximum number of frame buffers M supported by the first graphics API, where N is an integer greater than 1 and less than or equal to M.

[0023] In a possible design, the data indicated by the first rendering instruction is the data of the Pth frame image, and the data indicated by the third rendering instruction is the data of the (P + 1)th frame image, where P is an integer greater than or equal to 1.

[0024] In a possible design, the device further includes: a submission unit configured to submit the fourth rendering instruction to the execution unit when the number of instructions of the second rendering instruction is less than a first threshold, so that the execution unit can execute the fourth rendering instruction.

[0025] In a possible design, the device further includes: a submission unit, configured to submit the fourth rendering instruction to an execution unit when the number of instructions of the second rendering instruction is greater than a first threshold and the number of instructions of the fourth rendering instruction is less than a second threshold, so that the execution unit executes the fourth rendering instruction.

[0026] In a possible design, a conversion unit is further configured to convert a fifth rendering instruction according to a third resource group to obtain a sixth rendering instruction, where the fifth rendering instruction is a rendering instruction corresponding to a second graphics API, the sixth rendering instruction is a rendering instruction corresponding to a first graphics API, and the third resource group is included in at least two configured resource groups. The data indicated by the fifth rendering instruction is the data of the (P + 2)-th frame image. The device further includes: a submission unit, configured to submit the fourth rendering instruction to the execution unit when the number of instructions of the second rendering instruction is greater than the first threshold, the number of instructions of the fourth rendering instruction is greater than the second threshold, and the number of instructions of the sixth rendering instruction is greater than a third threshold, so that the execution unit executes the fourth rendering instruction.

[0027] In a possible design, the first graphics API is Vulkan. The second graphics API is the Open Graphics Library (OpenGL), or the second graphics API is the embedded Open Graphics Library for Embedded Systems (OpenGL ES).

[0028] In a third aspect, an electronic device is provided. The electronic device maintains a driver for a first graphics API. The electronic device includes one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions; when the one or more processors execute the computer instructions, the electronic device executes the data processing method as described in any one of the first aspect and various possible designs above.

[0029] In a fourth aspect, a chip system is provided. The chip system includes an interface circuit and a processor; the interface circuit and the processor are interconnected through a line; the interface circuit is configured to receive a signal from a memory and send the signal to the processor, and the signal includes computer instructions stored in the memory; when the processor executes the computer instructions, the chip system executes the data processing method as described in any one of the first aspect and various possible designs above.

[0030] In a fifth aspect, a computer-readable storage medium is provided. The computer-readable storage medium includes computer instructions, and when the computer instructions run, the data processing method as described in any one of the first aspect and various possible designs above is executed.

[0031] Sixthly, a computer program product is provided. The computer program product includes instructions. When the computer program product runs on a computer, the computer can execute the data processing method described in any one of the first aspect and various possible designs as described above according to the instructions.

[0032] It should be understood that for the technical solutions provided in the second aspect, the third aspect, the fourth aspect, the fifth aspect, and the sixth aspect above, their technical features can all correspond to the data processing method provided in the first aspect and its possible designs. Therefore, the beneficial effects that can be achieved are similar and will not be elaborated here. Description of the Drawings

[0033] Figure 1 It is a schematic flowchart of an instruction conversion method;

[0034] Figure 2 It is a schematic diagram of an instruction conversion mechanism;

[0035] Figure 3 It is a schematic diagram of the composition of an electronic device provided by an embodiment of the present application;

[0036] Figure 4 It is a schematic diagram of the composition of another electronic device provided by an embodiment of the present application;

[0037] Figure 5 It is a schematic diagram of the composition of a GL2VK system provided by an embodiment of the present application;

[0038] Figure 6 It is a schematic diagram of the composition of a GL module provided by an embodiment of the present application;

[0039] Figure 7 It is a schematic diagram of the composition of a conversion layer provided by an embodiment of the present application;

[0040] Figure 8 It is a schematic flowchart of a data processing method provided by an embodiment of the present application;

[0041] Figure 9 It is a schematic flowchart of another data processing method provided by an embodiment of the present application;

[0042] Figure 10 It is a schematic flowchart of another data processing method provided by an embodiment of the present application;

[0043] Figure 11 It is a schematic flowchart of another data processing method provided by an embodiment of the present application;

[0044] Figure 12 It is a schematic diagram of the composition of a data processing device provided by an embodiment of the present application;

[0045] Figure 13 Schematic diagram of another electronic device provided by an embodiment of this application;

[0046] Figure 14 Schematic diagram of a chip system provided by an embodiment of this application. Detailed implementation manners

[0047] In order to enable an electronic device to support Vulkan and OpenGL (or OpenGL ES) simultaneously, the electronic device can maintain the underlying drivers corresponding to Vulkan and OpenGL (or OpenGL ES) at the same time, which will lead to an excessive maintenance burden on the electronic device.

[0048] Currently, there is a solution that enables the electronic device to convert the instructions corresponding to OpenGL or OpenGL ES into instructions corresponding to Vulkan when rendering processing for OpenGL or OpenGL ES is required, and execute the instructions by running the underlying driver corresponding to Vulkan, thereby implementing the rendering processing for OpenGL or OpenGL ES. In this way, only the underlying driver corresponding to Vulkan needs to be maintained in the electronic device. Combining the foregoing description, since the number of codes of the underlying driver corresponding to Vulkan is much smaller than that of the underlying driver corresponding to OpenGL (or OpenGL ES), this solution can significantly reduce the problem of excessive maintenance burden of the electronic device caused by maintaining a large number of codes. In this application, the component for implementing the above solution can also be called an Angle component.

[0049] Exemplarily, in combination with Figure 1 , a flowchart of this solution is shown. Among them, taking the operating system of the electronic device as the Android system and the need to render OpenGL ES as an example. An application can send the rendering instructions of OpenGL ES to the Angle component. This component can convert the rendering instructions to obtain the rendering instructions corresponding to Vulkan. The rendering instructions of Vulkan can be transmitted to a Graphics Processing Unit (GPU) through a Vulkan driver module to perform corresponding rendering processing. As Figure 1As shown in the figure, the Angle component may include multiple modules to implement the above-mentioned instruction conversion function. Among them, the instruction interception module (EGL Entry) can be used to receive OpenGL ES rendering instructions issued by an application. The GL authentication module (GL Validation) can authenticate the OpenGL ES rendering instruction in combination with the GL context of the OpenGL ES rendering instruction to determine the feasibility and legality of the instruction. After the authentication is completed, the OpenGL ES rendering instruction can be stored in the front-end storage module (Front-End). In this front-end storage module, the rendering instructions corresponding to OpenGL ES can be converted to obtain the corresponding Vulkan rendering instructions. Then, the Vulkan rendering instruction can be stored in the Vulkan instruction storage module (Back-End) and transmitted to the GPU via the Vulkan driver module (Vulkan Driver) to perform the corresponding rendering process.

[0050] In the above example, the rendering instruction to be processed is taken as OpenGL ES for illustration. When the operating system of the electronic device is Windows, the Angle component can also convert the input OpenGL instruction into the corresponding Vulkan rendering instruction in the front-end storage module and store it in the Vulkan instruction storage module, so as to be transmitted to the GPU via the Vulkan driver module to perform the corresponding rendering process. In addition, in the Windows system, Direct3D (abbreviated as D3D) is also a frequently used graphics API. When the underlying driver corresponding to D3D is maintained in the electronic device, combined with Figure 1 , the Angle component can also be used to convert OpenGL rendering instructions into D3D rendering instructions to implement the corresponding rendering process. The specific process is similar to the foregoing process and will not be elaborated here.

[0051] It should be noted that in the rendering process based on the Angle component, the conversion of instructions and the submission to the GPU can be carried out in units of frame images. For example, referring to Figure 2 , when rendering a rendering instruction stream corresponding to a frame image, the Angle component can convert multiple draw calls corresponding to the rendering instruction stream respectively to convert multiple draw calls corresponding to OpenGL ES (such as Figure 2 the draw instructions 1 - draw instruction N shown) into multiple draw calls corresponding to Vulkan respectively. Then, the GPU can perform rendering according to the draw instructions 1 - draw instruction N corresponding to Vulkan submitted by the Angle component and display each frame image in the foreground.

[0052] It should be understood that after the electronic device renders each frame image, it needs to display each frame image in a certain order. Therefore, after the Angle component completes the conversion of the instructions, the obtained corresponding Vulkan drawing instructions 1 - N can be stored in the swap buffer respectively. A Vulkan wait queue (vkQueueWaitidle) can be set in the SwapBuffer, and these Vulkan drawing instructions corresponding to different frame images can be sequentially acquired by the Vulkan driver in the order of the frame images in this Vulkan wait queue. For example, when the Vulkan driver finishes displaying one frame image and needs to display the next frame image, it can acquire the drawing instructions corresponding to the corresponding frame image from the Vulkan wait queue and transmit these drawing instructions to the GPU for rendering, so as to obtain the corresponding frame image for display in the foreground.

[0053] It can be seen that in the current instruction conversion scheme based on the Angle component, the electronic device can achieve compatibility processing for OpenGL ES (Open GL) while only maintaining the underlying driver corresponding to Vulkan. However, due to the need to support multi-platform rendering, the Angle component does not optimize the Vulkan framework. As a result, during the execution of this scheme, when the application issues rendering instructions for multiple frame images, the Angle component will only convert the instructions corresponding to the next frame after the instructions corresponding to the first frame (such as the drawing instructions corresponding to the first frame image) are converted. That is to say, the instruction conversion of the subsequent frame image needs to wait until the instruction conversion of the previous frame image is completed. This makes the execution efficiency of the instruction conversion process of the Angle component low. Through comparative experiments, it can be verified that currently, the performance of this scheme is only about half of that of native OpenGL ES.

[0054] In addition, in the current scheme provided by the Angle component, the GPU can only acquire the drawing instructions corresponding to the frame image when the next frame image needs to be displayed, which results in insufficient utilization of the GPU. In addition, since the GPU may also need to perform other processing tasks, when the drawing instructions corresponding to the next frame image are transmitted to the GPU, the GPU may be in a high-load state. If the GPU is forced to execute the drawing instructions to draw this frame image at this time, it will cause the problem of excessive workload of the GPU.

[0055] In response to this, an embodiment of the present application provides a data processing method, which enables an electronic device to utilize the cache mechanism in the Vulkan framework to implement multi-frame concurrent instruction conversion, thereby improving the conversion efficiency of instructions. Additionally, in some implementation manners, the data processing method provided by the embodiment of the present application can also flexibly adjust the submission timing of the current frame image according to the number of drawing instructions included in other frame images adjacent to the current frame image that needs to be submitted to the GPU for processing (such as the previous frame image and / or the next frame image), thereby achieving the effect of adaptively adjusting the submission of drawing instructions according to the GPU load condition, so that the GPU will not be overloaded due to performing rendering work.

[0056] The following details the solution provided by the embodiment of the present application in conjunction with the accompanying drawings.

[0057] Please refer to Figure 3 , which is a schematic diagram of the composition of an electronic device 300 provided by an embodiment of the present application. As Figure 3 shown, the electronic device 300 may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a sensor module 380, a key 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc. Among them, the sensor module 380 may include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc. In some embodiments, the electronic device 300 may further include components such as a speaker, a receiver, a microphone, and a headphone interface to implement the audio-related functions of the electronic device.

[0058] It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 300. In other embodiments, the electronic device 300 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0059] The processor 310 may include one or more processing units. For example, the processor 310 may include a Central Processing Unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0060] The controller may be the nerve center and command center of the electronic device 300. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0061] A memory may also be provided in the processor 310 for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. This memory may save the instructions or data that the processor 310 has just used or recycled. If the processor 310 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0062] In some embodiments, the processor 310 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0063] As an example, take the case where the processor 310 includes a CPU and a GPU. The CPU can be used to perform one or more of the following operations: receiving rendering instructions issued by an application, intercepting the rendering instructions corresponding to OpenGL ES in the rendering instructions, converting the rendering instructions into rendering instructions corresponding to Vulkan, and submitting the rendering instructions corresponding to Vulkan to the GPU for rendering operations according to preset rules through the Vulkan driver.

[0064] The electronic device 300 can implement the shooting function through the ISP, the camera 393, the video codec, the GPU, the display screen 394, the application processor 310, etc.

[0065] The ISP is used to process the data fed back by the camera 393. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the photosensitive element of the camera 393. The optical signal is converted into an electrical signal, and the photosensitive element of the camera 393 transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP may be provided in the camera 393.

[0066] The camera 393 is used to capture still images or videos. An object generates an optical image through a lens and projects it onto a photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the electronic device 300 may include one or N cameras 393, where N is a positive integer greater than 1.

[0067] The digital signal processor 310 is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 300 selects a frequency point, the digital signal processor 310 is used to perform a Fourier transform on the frequency point energy, etc.

[0068] The video codec is used to compress or decompress digital videos. The electronic device 300 can support one or more video codecs. In this way, the electronic device 300 can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0069] The NPU is a neural-network (NN) computing processor 310. By drawing on the structure of a biological neural network, such as the transmission pattern between human brain neurons, it can quickly process input information and can also continuously learn on its own. Through the NPU, applications such as intelligent cognition of the electronic device 300 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0070] The charging management module 340 is used to receive charging input from a charger. Herein, the charger can be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 340 can receive the charging input of the wired charger through the USB interface 330. In some embodiments of wireless charging, the charging management module 340 can receive the wireless charging input through the wireless charging coil of the electronic device 300. While charging the battery 342, the charging management module 340 can also supply power to the electronic device 300 through the power management module 341. The power management module 341 is used to connect the battery 342, the charging management module 340, and the processor 310. The power management module 341 receives the inputs from the battery 342 and / or the charging management module 340 and supplies power to the processor 310, the internal memory 321, the external memory, the display screen 394, the camera 393, the wireless communication module 360, etc. The power management module 341 can also be used to monitor parameters such as the capacity of the battery 342, the number of battery 342 charge cycles, and the health status (leakage, impedance) of the battery 342. In some other embodiments, the power management module 341 can also be disposed in the processor 310. In some other embodiments, the power management module 341 and the charging management module 340 can also be disposed in the same device.

[0071] The wireless communication function of the electronic device 300 can be implemented by the antenna 1, the antenna 2, the mobile communication module 350, the wireless communication module 360, the modulation and demodulation processor, and the baseband processor, etc.

[0072] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 300 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0073] The mobile communication module 350 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device 300. The mobile communication module 350 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves by the antenna 1, filter and amplify the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 350 can be disposed in the processor 310. In some embodiments, at least some functional modules of the mobile communication module 350 and at least some modules of the processor 310 can be disposed in the same device.

[0074] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to speakers, receivers, etc.), or displays an image or video through the display screen 394. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 310 and be provided in the same device as the mobile communication module 350 or other functional modules.

[0075] The wireless communication module 360 may provide solutions for wireless communications applied to the electronic device 300, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 360 may be one or more devices integrating at least one communication processing module. The wireless communication module 360 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to the processor 310. The wireless communication module 360 may also receive the signal to be transmitted from the processor 310, perform frequency modulation and amplification on it, and convert it into electromagnetic waves through the antenna 2 and radiate them out.

[0076] In some embodiments, antenna 1 of electronic device 300 is coupled to mobile communication module 350, and antenna 2 is coupled to wireless communication module 360, enabling electronic device 300 to communicate with a network and other devices via wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0077] Electronic device 300 implements a display function via a GPU, display screen 394, and an application processor, etc. The GPU is a microprocessor for image processing, connected to display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 310 may include one or more GPUs, which execute program instructions to generate or change display information.

[0078] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 300 may include one or N display screens 394, where N is a positive integer greater than 1.

[0079] The external memory interface 320 can be used to connect to an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 300. The external memory card communicates with the processor 310 through the external memory interface 320 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0080] The internal memory 321 can be used to store computer-executable program code, and the executable program code includes instructions. The processor 310 executes various functional applications and data processing of the electronic device 300 by running the instructions stored in the internal memory 321. The internal memory 321 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device 300 (such as audio data, a phone book, etc.). In addition, the internal memory 321 can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0081] In the embodiments of the present application, the internal memory 321 can be used to store cached network videos. Of course, in some embodiments, the network video can also be stored in an external storage medium connected through the external memory interface 320.

[0082] The electronic device 300 can implement audio functions through the audio module 370, the speaker, the receiver, the microphone, the headphone interface, and the application processor 310, etc. Such as music playback, recording, etc.

[0083] The audio module 370 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 370 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 370 can be disposed in the processor 310, or some functional modules of the audio module 370 can be disposed in the processor 310. A speaker, also known as a "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device 300 can listen to music or a hands-free call through the speaker. A receiver, also known as a "handset", is used to convert an audio electrical signal into a sound signal. When the electronic device 300 answers a call or a voice message, the voice can be listened to by holding the receiver close to the human ear. A microphone, also known as a "microphone" or "transmitter", is used to convert a sound signal into an electrical signal. When making a call, sending a voice message, or when it is necessary to trigger the electronic device 300 to perform certain functions through a voice assistant, the user can speak by bringing the mouth close to the microphone to input the sound signal into the microphone. The electronic device 300 can be provided with at least one microphone. In some other embodiments, the electronic device 300 can be provided with two microphones, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device 300 can also be provided with three, four or more microphones, which can collect sound signals, reduce noise, identify the sound source, and implement functions such as directional recording. The headphone jack is used to connect a wired headphone. The headphone jack can be a USB interface 330, or a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0084] The touch sensor, also known as a "touch panel". The touch sensor can be disposed on the display screen 394, and together with the display screen 394, it forms a touch screen, also known as a "touch screen". The touch sensor is used to detect a touch operation acting thereon or nearby. The touch sensor can transmit the detected touch operation to the application processor 310 to determine the type of touch event. In some embodiments, a visual output related to the touch operation can be provided through the display screen 394. In some other embodiments, the touch sensor can also be disposed on the surface of the electronic device 300, at a different position from the display screen 394.

[0085] A pressure sensor is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, the pressure sensor may be disposed on the display screen 394. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor may include at least two parallel plates having conductive materials. When a force acts on the pressure sensor, the capacitance between the electrodes changes. The electronic device 300 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 394, the electronic device 300 detects the intensity of the touch operation according to the pressure sensor. The electronic device 300 can also calculate the position of the touch based on the detection signal of the pressure sensor. In some embodiments, touch operations with the same touch position but different touch operation intensities may correspond to different operation instructions. For example: When a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, the instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, the instruction to create a new short message is executed.

[0086] The gyroscope sensor can be used to determine the motion posture of the electronic device 300. In some embodiments, the angular velocity of the electronic device 300 around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor. The gyroscope sensor can be used for anti-shake during shooting. Exemplarily, when the shutter is pressed, the gyroscope sensor detects the angle of jitter of the electronic device 300, calculates the distance that the lens module needs to compensate based on the angle, and enables the lens to offset the jitter of the electronic device 300 through reverse movement to achieve anti-shake. The gyroscope sensor can also be used for navigation and somatosensory game scenarios.

[0087] The barometric pressure sensor is used to measure barometric pressure. In some embodiments, the electronic device 300 calculates the altitude based on the barometric pressure value measured by the barometric pressure sensor to assist in positioning and navigation.

[0088] The magnetic sensor includes a Hall sensor. The electronic device 300 can use the magnetic sensor to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 300 is a flip phone, the electronic device 300 can detect the opening and closing of the flip according to the magnetic sensor. Furthermore, according to the detected opening and closing state of the leather case or the flip, features such as automatic flip unlocking are set.

[0089] The acceleration sensor can detect the magnitude of the acceleration of the electronic device 300 in various directions (generally three axes). When the electronic device 300 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device 300 and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0090] A distance sensor for measuring distance. The electronic device 300 can measure distance through infrared or laser. In some embodiments, when shooting a scene, the electronic device 300 can use the distance sensor to measure distance for quick focusing.

[0091] The proximity light sensor may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode may be an infrared light-emitting diode. The electronic device 300 emits infrared light outward through the light-emitting diode. The electronic device 300 uses the photodiode to detect the infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 300. When insufficient reflected light is detected, the electronic device 300 can determine that there is no object near the electronic device 300. The electronic device 300 can use the proximity light sensor to detect when the user holds the electronic device 300 close to the ear for a call, so as to automatically turn off the screen to achieve power-saving purposes. The proximity light sensor can also be used in the holster mode and the pocket mode for automatic unlocking and locking of the screen.

[0092] The ambient light sensor is used to sense the ambient light brightness. The electronic device 300 can adaptively adjust the brightness of the display screen 394 according to the sensed ambient light brightness. The ambient light sensor can also be used to automatically adjust the white balance during photography. The ambient light sensor can also cooperate with the proximity light sensor to detect whether the electronic device 300 is in the pocket to prevent accidental touch.

[0093] The fingerprint sensor is used to collect fingerprints. The electronic device 300 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access to the application lock, fingerprint photography, fingerprint answering of incoming calls, etc.

[0094] The temperature sensor is used to detect temperature. In some embodiments, the electronic device 300 uses the temperature detected by the temperature sensor to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor exceeds the threshold, the electronic device 300 reduces the performance of the processor 310 located near the temperature sensor to reduce power consumption and implement thermal protection. In some other embodiments, when the temperature is lower than another threshold, the electronic device 300 heats the battery 342 to avoid abnormal shutdown of the electronic device 300 caused by low temperature. In some other embodiments, when the temperature is lower than yet another threshold, the electronic device 300 boosts the output voltage of the battery 342 to avoid abnormal shutdown caused by low temperature.

[0095] The bone conduction sensor can acquire vibration signals. In some embodiments, the bone conduction sensor can acquire the vibration signals of the vibrating bone mass of the human vocal part. The bone conduction sensor can also contact the human pulse to receive blood pressure pulsation signals. In some embodiments, the bone conduction sensor can also be disposed in the earphone to form a bone conduction earphone. The audio module 370 can parse out voice signals based on the vibration signals of the vibrating bone mass acquired by the bone conduction sensor to implement the voice function. The application processor 310 can parse out heart rate information based on the blood pressure pulsation signals acquired by the bone conduction sensor to implement the heart rate detection function.

[0096] The keys 390 include a power-on key, volume keys, etc. The keys 390 can be mechanical keys 390 or touch keys 390. The electronic device 300 can receive the input of the keys 390 to generate key signal inputs related to the user settings and function controls of the electronic device 300.

[0097] The motor 391 can generate vibration prompts. The motor 391 can be used for incoming call vibration prompts and also for touch vibration feedback. For example, touch operations on different applications (such as taking pictures, audio playing, etc.) can correspond to different vibration feedback effects. For touch operations on different areas of the display screen 394, the motor 391 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminder, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0098] The indicator 392 can be an indicator light, which can be used to indicate the charging state, power change, and can also be used to indicate messages, missed calls, notifications, etc.

[0099] The SIM card interface 395 is used to connect the SIM card. The SIM card can be in contact with and separated from the electronic device 300 by inserting into or pulling out from the SIM card interface 395. The electronic device 300 can support 1 or N SIM card interfaces 395, where N is a positive integer greater than 1. The SIM card interface 395 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 395 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 395 can also be compatible with different types of SIM cards. The SIM card interface 395 can also be compatible with external memory cards. The electronic device 300 interacts with the network through the SIM card to implement functions such as calls and data communication. In some embodiments, the electronic device 300 uses an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 300 and cannot be separated from the electronic device 300.

[0100] It should be understood that the above Figure 3shows a hardware structure composition in an electronic device. In this application, the electronic device 300 can also be divided from another perspective. For example, referring to Figure 4 , shows another logical composition of the electronic device 300.

[0101] In this example, the electronic device 300 may include an application layer 401, a framework layer 402, a driver layer 403, and a physical layer 404.

[0102] Among them, one or more application programs can be installed and run in the application layer 401, such as Application 1, Application 2, etc. In some examples, the application programs installed in the application layer 401 may include applications that issue rendering instructions during operation. For example, game applications (such as Peacekeeper Elite etc.).

[0103] The framework layer 402 may include a system framework that runs basic program code to support a basic system (such as operating systems like Android and / or Windows). For the sake of illustration, hereinafter, it is taken as an example that the electronic device runs an Android system, and correspondingly, the rendering instructions issued by the application are OpenGL ES - corresponding rendering instructions (hereinafter simply referred to as OpenGL ES instructions).

[0104] In the embodiments of this application, the system framework can also be used to intercept rendering instructions corresponding to OpenGL or OpenGL ES. As Figure 4 shown, a GL2VK system can also be set in the framework layer 402. This system can be used to convert rendering instructions corresponding to OpenGL or OpenGL ES into rendering instructions corresponding to Vulkan.

[0105] As an example, in combination with Figure 5 , the GL2VK system may include an instruction interception module 501, a GL module 502, a shader conversion module 503, a conversion layer 504, a Vulkan backend 505 and other modules.

[0106] Among them, the instruction interception module 501 can serve as the receiving port of GL2VK, and is used to receive the OpenGL ES - corresponding rendering instructions intercepted by the system framework. For example, the instruction interception module 501 can be used to intercept the OpenGL ES instructions issued by the application and convert them into window system instructions that can be recognized and processed by the GL2VK system. It can be understood that in some other implementation manners, this instruction interception module 501 can also be used to intercept OpenGL instructions or EGL instructions issued by the application and perform the above - mentioned similar operations on the intercepted instructions. The specific implementation is not elaborated here.

[0107] The GL module 502 can be used to process the instructions obtained from the instruction interception module 501. In some implementations, in combination with Figure 6 , the GL module 502 may include a resource management 601, a state management 602, an operation management 603, etc. Among them, the resource management 601 can be used to maintain operations such as creation, deletion, and update of all resources. The state management 602 can be used to record all state information, and subsequent rendering instructions will use this state information. The operation management 603 can be used to manage relevant operation information in the rendering instructions, such as rendering instructions, clear operations, blit operations, etc.

[0108] The shader conversion module 503 can be used to convert the language corresponding to the OpenGL ES instructions into the language corresponding to the Vulkan driver, so as to facilitate the subsequent instruction conversion processing. It can be understood that the language formats and other rules followed by the rendering instructions based on different frameworks are different. In this example, since the OpenGL ES instructions need to be converted into Vulkan instructions, therefore, before the instruction conversion, the OpenGL ES language can be converted into the Vulkan language (such as SPIR-V binary data) through the shader conversion module 503, so as to facilitate subsequent processing.

[0109] The conversion layer 504 can be used to convert OpenGL ES instructions into Vulkan instructions. Exemplarily, in combination with Figure 7 , the conversion layer 504 may include an instruction cache module 701, which is used to save the instruction information and the current state information into multiple pre-configured lock-free queues. Among them, the lock-free queue can be a first in first out (FIFO) lock-free queue. In some implementations, one frame image can correspond to one lock-free queue, and the number of lock-free queues can be determined according to the maximum number of frame buffers supported by the current Vulkan framework. For example, taking the maximum number of frame buffers supported by the current Vulkan framework as 3 frames as an example. The instruction cache module 701 can be used to save the instruction information and the current state information corresponding to the three frame images into their respective corresponding three lock-free queues respectively.

[0110] As Figure 7 shown, the conversion layer 504 may further include an instruction conversion module 702, which is used to complete the conversion of specific OpenGL or OpenGL ES graphics APIs to Vulkan graphics APIs. An instruction submission module 703, which is used to submit Vulkan instructions to the hardware GPU for drawing.

[0111] In addition, in some implementation manners, a synchronization module 704 may also be included in the conversion layer 504. It can be understood that since there is a strict order for the display of frame images, in order to ensure the correct display of frame images, it is necessary to make the rendering of each frame image also correspond to this order. That is, when submitting the rendering instructions for each frame image, it is necessary to correspond to the above order. In this example, the instruction conversion for each frame image is performed in its respective lock-free queue. Since they are relatively independent of each other, there may be a deviation in the output time of the instruction conversion. Therefore, in this example, the conversion layer 504 can adjust the output order of the rendering instructions after the instruction conversion in each lock-free queue through the synchronization module 704, thereby achieving the effect of corresponding to the display order of the frame images.

[0112] After the conversion layer 504 completes the instruction conversion, these Vulkan instructions can be transmitted to the Vulkan backend 505 to packetize these instructions, and then transmit these instructions to the Vulkan driver. Subsequently, the Vulkan driver can separately transmit the Vulkan instructions corresponding to each frame image after the conversion to the GPU for rendering processing.

[0113] This enables the electronic device to only maintain the underlying driver of Vulkan to complete the processing of rendering instructions under other frameworks, including those of OpenGLES. It can be seen that in combination with the above description, since the instructions corresponding to each frame image can perform the instruction conversion in parallel in different lock-free queues, the conversion efficiency of the instructions can be effectively improved, and thus the overall performance can be enhanced.

[0114] In combination with the above description, the electronic device can store the OpenGL ES instructions corresponding to multiple frame images into their respective lock-free queues, so that the conversion layer in the GL2VK system can independently convert the instructions in each lock-free queue. In the embodiments of the present application, the electronic device can pre-configure the resources corresponding to each lock-free queue according to a preset rule before using the lock-free queue.

[0115] Exemplarily, taking the running application as a game application as an example. Refer to Figure 8 , which shows a schematic diagram of a method for configuring resources provided by the embodiments of the present application. As Figure 8 shown, the method may include:

[0116] S801. The game starts.

[0117] In this example, the electronic device can be triggered by the start of a game application and execute the following S802 - S803. For example, the start of a game application can refer to the electronic device receiving an operation of touch or click input on the icon of the game application, and in response to this operation, calling the API of the game application to start running. The electronic device can know that the game starts by monitoring when the corresponding API is called and execute the following steps.

[0118] S802. Obtain the maximum number of frame buffers supported by the Vulkan framework.

[0119] When the game starts, the electronic device can obtain the maximum number of frame buffers supported by the Vulkan framework corresponding to the game application. It can be understood that the Vulkan framework may support different maximum numbers of frame buffers in different application scenarios. Therefore, in this application, the electronic device can obtain the maximum number of frame buffers that the Vulkan framework corresponding to the current game application can support. For example, the maximum number of true frame buffers supported by the Vulkan framework can be M, where M is an integer greater than or equal to 1. In some specific implementation scenarios, M can be 3.

[0120] S803. Create resources corresponding to the lock - free queue.

[0121] After the electronic device determines the maximum number of frame buffers supported by the Vulkan framework (such as M), it can create corresponding resources (such as N). Where N is a positive integer less than or equal to M. As an example, taking M = 3 as an example. The electronic device can create resources corresponding to 3 lock - free queues. The resources corresponding to the lock - free queue can include the thread corresponding to the lock - free queue, the lock - free queue, and the command buffer. For example, the electronic device can create 3 groups of resources, which are: The first group of resources includes thread 1, lock - free queue 1, and command buffer 1. The second group of resources includes thread 2, lock - free queue 2, and command buffer 2. The third group of resources includes thread 3, lock - free queue 3, and command buffer 3. In the embodiments of this application, the resources corresponding to a lock - free queue can also be referred to as a resource group.

[0122] It should be noted that in the above example, N = M is taken as an example for illustration. It can be understood that, with resources permitted, configuring the resource combination of the maximum number of frame buffers supported by the Vulkan framework can maximize the efficiency of parallel instruction conversion.

[0123] In some other implementations, the electronic device can also flexibly adjust the number of N according to the amount of resources currently allowed to be used. For example, when there are more threads currently in the queue, the electronic device can set N to an integer less than M, thereby reducing the pressure on CPU parallel processing. Another example is that when the cache space available for storing instructions is in short supply, the electronic device can also set N to an integer less than M, thereby reducing the demand for instruction buffering in the configured resources, and further achieving the effect of reducing the pressure on cache space usage.

[0124] For ease of explanation, hereinafter, taking N = M = 3 as an example, in combination with Figures 3 to 7 the composition description of any one of the electronic devices, the data processing method provided in the embodiments of the present application will be described in detail. Please refer to Figure 9 , when the electronic device is running a game application, the application program at the application layer can issue OpenGL ES rendering instructions. In combination with the method description shown in the above Figure 8 , the electronic device can also load the GL2VK system when the application program is started, and configure the corresponding number of resources according to the maximum number of frame buffers supported by the current Vulkan framework. For example, configure 3 groups of resources including lock-free queue 1, lock-free queue 2, and lock-free queue 3. The instruction interception module in the GL2VK system can intercept OpenGL ES instructions and write the intercepted instructions into the 3 groups of resources. In this way, 3 groups of resources storing OpenGL ES instructions can be obtained in the GL2VK system. In some implementations of the present application, each group of resources can correspond to multiple drawing commands for a frame image. For example, lock-free queue 1 can store the drawing commands for the first frame image. Lock-free queue 2 can store the drawing commands for the second frame image. Lock-free queue 3 can store the drawing commands for the third frame image. Among them, taking the extraction of data from a data array through glDrawElements to render primitive primitives as an example, the electronic device can write the wrapped instructions into lock-free queue 1 after processing the data state (pipeline information, status information, etc.) required for the rendering instructions of the first frame image. Similarly, the electronic device can also perform the above operations on the second frame image and the third frame image to achieve the writing of instructions.

[0125] Next, the conversion layer in the GL2VK system can perform instruction conversion. Exemplarily, such as Figure 9As shown, steps such as instruction reading, instruction conversion, and instruction recording can be performed separately for each group of resources, thereby obtaining three groups of resources storing Vulkan instructions. For example, in the conversion layer of the GL2VK system, the draw instructions corresponding to the first frame image stored in the lock-free queue 1 can be read, glDrawElements can be converted to the corresponding vkCmdDrawIndexed in Vulkan, and then instruction recording can be performed through vkBeginCommandBuffer and vkEndCommandBuffer, and the draw instructions under the Vulkan framework obtained by recording are stored in the lock-free queue 1. Similarly, the conversion layer can also convert the instructions in the other two lock-free queues, thereby obtaining three lock-free queues of resources storing Vulkan instructions. It should be noted that since each lock-free queue can correspond to an independent thread, therefore, during the instruction conversion process of the GL2VK system, the CPU can call the corresponding threads respectively to complete the instruction conversion, that is to say, the CPU can use the multi-threaded concurrent processing mechanism to achieve parallel processing of the instruction conversions corresponding to multiple different frame images. Thus, compared with the serial mechanism of processing one by one in the prior art, the instruction conversion efficiency can be significantly improved.

[0126] After the instruction conversion is completed, the corresponding instructions can be transmitted to the Vulkan backend for packetization, so that the packetized instructions can be submitted to the GPU through the Vulkan driver to perform the corresponding rendering process. It can be understood that the instructions in each lock-free queue can include the rendering instructions for one frame image, that is to say, after packetization by the Vulkan backend, the number of rendering instruction packets corresponding to the frame image can be obtained. In some implementation manners, the rendering instruction packet can be directly transmitted to the GPU after the packetization is completed, so that the GPU can render the frame image as soon as possible. In other implementation manners, the transmission of the rendering instructions to the GPU can also adopt other methods, so as to be able to flexibly adjust the submission timing and / or quantity of the rendering instructions in combination with the workload brought by the current rendering process of the GPU, so as to achieve the purpose of reducing the GPU workload and improving the system working stability.

[0127] Exemplarily, the GL2VK system can determine the submission timing of the rendering instructions for the current frame image in combination with the quantity of the rendering instructions corresponding to the previous frame image.

[0128] As an example, please refer to Figure 10 , a method for submitting rendering instructions provided in an embodiment of the present application. Among them, taking the rendering instructions currently to be submitted to the GPU for rendering corresponding to the frame image being the Nth frame image as an example. As Figure 10 shown, the method may include:

[0129] S1001. Whether the number of rendering instructions for the (N - 1)-th frame image is less than the first threshold.

[0130] It can be understood that since the rendering instructions for the (N - 1)-th frame image are also transmitted to the GPU after being processed by the GL2VK system, the GL2VK system can know the quantity of the rendering instructions for the (N - 1)-th frame image, that is, the frame image corresponding to the rendering instructions currently being rendered by the GPU. Before submitting the rendering instructions for the N-th frame image, the GL2VK system can determine the size relationship between the rendering instructions for the (N - 1)-th frame image and a preset first threshold (such as 100), and accordingly determine whether to immediately submit the rendering instructions for the N-th frame image to the GPU for processing. For example, when the number of rendering instructions for the (N - 1)-th frame image is less than 100, then execute the following S1002. When the number of rendering instructions for the (N - 1)-th frame image is greater than 100, then execute the following S1003.

[0131] S1002. Submit the rendering instructions for the N-th frame image.

[0132] It should be understood that if the rendering instructions for the (N - 1)-th frame image are less than the first threshold, it can be considered that the current rendering pressure on the GPU is not large. Therefore, the rendering instructions for the N-th frame image can be immediately submitted to the GPU for rendering processing.

[0133] S1003. Whether the number of rendering instructions for the N-th frame image is less than the second threshold.

[0134] If the number of rendering instructions for the (N - 1)-th frame image is greater than the first threshold, then it can be considered that the current rendering pressure on the GPU for processing the (N - 1)-th frame image is large. Therefore, it is impossible to process a large amount of newly submitted rendering instructions in a short time. In this example, when the number of rendering instructions for the (N - 1)-th frame image is large, the GL2VK system can determine the size relationship between the number of rendering instructions for the current frame image (such as the N-th frame image) and the second threshold (such as 100), determine whether submitting the current frame image will cause too much burden on the GPU's rendering processing, and then determine whether to submit the rendering instructions corresponding to the current frame image. For example, when the number of rendering instructions for the N-th frame image is greater than 100, it is considered that submitting the rendering instructions for the N-th frame image immediately will bring a large burden to the GPU. Therefore, the following S1004 can be executed. Correspondingly, when the number of rendering instructions for the N-th frame image is less than 100, it is considered that submitting the rendering instructions for the N-th frame image immediately will not bring a large burden to the GPU. Therefore, S1002 can be executed.

[0135] S1004. The submission of the rendering instructions for the N-th frame image enters the waiting state.

[0136] It can be understood that when it is determined that the submission of the current frame image will impose a relatively large burden on the GPU, the submission of the rendering instruction for the current frame (such as the Nth frame image) can be temporarily postponed to avoid imposing an excessive processing burden on the GPU. In some implementation manners, when the processing burden is relatively light, the GPU can actively report its current status to the CPU to end the waiting state of the Nth frame image and instruct the CPU to submit the rendering instruction for the Nth frame image. Alternatively, the CPU can trigger a preset timer when entering the waiting state, and when the timer expires, execute the submission of the rendering instruction for the Nth frame image. Alternatively, the CPU can also determine to submit the rendering instruction for the Nth frame image when the GPU processing pressure is relatively small based on other information. Thus, the effect of flexibly adjusting the submission of the rendering instruction based on the magnitude of the GPU workload is achieved.

[0137] In this way, for the submission of the rendering instruction for the current Nth frame image, the submission timing can be flexibly adjusted in combination with the burden imposed on the GPU by the rendering instruction of the previous frame image, so as to avoid the situation where the submission of the rendering instruction for the current frame image brings an excessive burden to the GPU. It should be noted that in some other implementation manners of this application, when executing S1001, if the number of rendering instructions of the (N - 1)th frame image is greater than the first threshold, the electronic device can also directly execute S1004 to enter the waiting state, and until the conditions in any of the above examples are met, execute S1002 to submit the rendering instruction for the Nth frame image.

[0138] In the above examples, it is described by taking the determination of the submission timing of the rendering instruction for the current frame image by combining the number of rendering instructions of the previous frame image and the number of rendering instructions of the current frame image as an example. In some other examples of this application, the GL2VK system can also refer to the number of rendering instructions of the subsequent frame (such as the (N + 1)th frame) image to determine the submission timing of the rendering instruction for the current frame image (such as the Nth frame). Exemplarily, combined with Figure 11 . Compared with Figure 10The method shown is different in that when executing S1003, if the number of rendering instructions for the Nth frame image is less than the second threshold, then S1005 can be executed, that is, to determine whether the number of rendering instructions for the (N + 1)th frame image is greater than the third threshold (set to 100 for example). If the number of rendering instructions for the (N + 1)th frame image is greater than 100, it is considered that in the queue of frame images to be rendered, the rendering workload of subsequent frame images is still relatively large. Therefore, it is necessary to render the Nth frame image as soon as possible. Then S1002 can be executed. On the contrary, if the number of rendering instructions for the (N + 1)th frame image is less than 100, then S1004 is executed. In this way, in order to ensure that the (N + 1)th frame image can be rendered and processed in a timely manner, the GL2VK system can immediately submit the rendering execution of the Nth frame image to the GPU, that is, execute S1002. Thus, although it may cause a slightly higher processing pressure on the current GPU, it can ensure that the (N + 1)th frame image can be processed in a timely manner. Therefore, it can ensure the smoothness of the rendering process of all frame images, and further ensure that there will be no situation of frame freezing during the display of each frame image in the foreground due to waiting for rendering and processing. Thus, the stability of the system can be improved from a global perspective.

[0139] Based on the above description, those skilled in the art should be able to accurately and clearly understand the specific implementation process of the solution provided by the embodiments of the present application. At the same time, during the implementation of this solution, compared with the prior art, the game effects that can be brought are very significant. Exemplarily, take the game application as Peacekeeper Elite as an example. The frame rate comparison in the CPU heavy load scenario is shown in Table 1.

[0140] Table 1

[0141] Solution Resolution FPS OpenGL ES Native 1080*2068 41.4 Angle Component 1080*2068 25.2 This Invention Patent 1080*2068 45.8

[0142] Due to the solution based on the Angle component only performing instruction conversion in a single thread and not fully utilizing the multi-threaded characteristics of Vulkan, the frame rate is only 25.2 frames, far inferior to the frame rate of native OpenGL ES. And because the present invention is based on a multi-frame concurrent and two-way inter-frame prediction mechanism, it can fully utilize the multi-core characteristics of the CPU and at the same time improve the GPU utilization rate, and the frame rate exceeds that of native OpenGL ES.

[0143] The above mainly introduces the solution provided by the embodiments of the present application from the perspective of an electronic device. To implement the above functions, it includes the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0144] The embodiments of the present application can divide the devices involved according to the above method examples into functional modules. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0145] Please refer to Figure 12 , a data processing device 1200 provided by the embodiments of the present application, can be applied to an electronic device to implement the method provided in the above embodiments. Exemplarily, the electronic device maintains a first driver, and the first driver corresponds to a first graphics application programming interface (API).

[0146] The data processing device 1200 may include: a configuration unit 1201, configured to configure at least two resource groups according to the first graphics API, and the resource groups are used to convert rendering instructions. A conversion unit 1202, configured to convert a first rendering instruction through the first resource group to obtain a second rendering instruction, where the first rendering instruction is a rendering instruction corresponding to a second graphics API, the second rendering instruction is a rendering instruction corresponding to the first graphics API, and the first resource group is included in the at least two configured resource groups. An execution unit 1203, configured to execute the second rendering instruction through the first driver. It should be noted that, in this example, the execution unit 1203 may correspond to the rendering execution module described above. For example, the execution unit 1203 can implement its corresponding function through a GPU.

[0147] In a possible design, the conversion unit 1202 is further configured to convert a third rendering instruction through a second resource group to obtain a fourth rendering instruction, where the third rendering instruction is a rendering instruction corresponding to a second graphics API, the fourth rendering instruction is a rendering instruction corresponding to a first graphics API, and the second resource group is included in at least two configured resource groups. The execution unit 1203 is further configured to execute the fourth rendering instruction through a first driver.

[0148] In a possible design, the conversion unit 1202 performs the conversion of the first rendering instruction and the conversion of the third rendering instruction in parallel.

[0149] In a possible design, the configuration unit 1201 is configured to configure N resource groups according to the maximum number of frame buffers M supported by the first graphics API, where N is an integer greater than 1 and less than or equal to M.

[0150] In a possible design, the data indicated to be rendered by the first rendering instruction is the data of the Pth frame image, and the data indicated to be rendered by the third rendering instruction is the data of the (P + 1)th frame image, where P is an integer greater than or equal to 1.

[0151] In a possible design, the device further includes: a submission unit 1204, configured to submit the fourth rendering instruction to the execution unit 1203 when the number of instructions of the second rendering instruction is less than a first threshold, so that the execution unit 1203 executes the fourth rendering instruction.

[0152] In a possible design, the device further includes: a submission unit 1204, configured to submit the fourth rendering instruction to the execution unit 1203 when the number of instructions of the second rendering instruction is greater than the first threshold and the number of instructions of the fourth rendering instruction is less than a second threshold, so that the execution unit 1203 executes the fourth rendering instruction.

[0153] In a possible design, the conversion unit 1202 is further configured to convert a fifth rendering instruction through a third resource group to obtain a sixth rendering instruction, where the fifth rendering instruction is a rendering instruction corresponding to a second graphics API, the sixth rendering instruction is a rendering instruction corresponding to a first graphics API, and the third resource group is included in at least two configured resource groups. The data indicated to be rendered by the fifth rendering instruction is the data of the (P + 2)th frame image. The device further includes: a submission unit 1204, configured to submit the fourth rendering instruction to the execution unit 1203 when the number of instructions of the second rendering instruction is greater than the first threshold, the number of instructions of the fourth rendering instruction is greater than the second threshold, and the number of instructions of the sixth rendering instruction is greater than a third threshold, so that the execution unit 1203 executes the fourth rendering instruction.

[0154] In a possible design, the first graphics API is Vulkan. The second graphics API is the Open Graphics Library (OpenGL), or the second graphics API is the Embedded Open Graphics Library (OpenGL ES).

[0155] It should be noted that all relevant content of each step involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, and will not be elaborated here.

[0156] Figure 13 Shows a schematic diagram of the composition of an electronic device 1300. As Figure 13 shown, the electronic device 1300 may include: a processor 1301 and a memory 1302. The memory 1302 is used to store computer-executable instructions. Exemplarily, in some embodiments, when the processor 1301 executes the instructions stored in the memory 1302, the electronic device 1300 may be caused to execute any of the data processing methods shown in the above embodiments.

[0157] It should be noted that all relevant content of each step involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, and will not be elaborated here.

[0158] Figure 14 Shows a schematic diagram of the composition of a chip system 1400. The chip system 1400 may include: a processor 1401 and a communication interface 1402, which are used to support related devices to implement the functions involved in the above embodiments. In a possible design, the chip system further includes a memory for storing necessary program instructions and data of the terminal. The chip system may be composed of chips or may include chips and other discrete devices. It should be noted that in some implementation manners of the present application, the communication interface 1402 may also be referred to as an interface circuit.

[0159] It should be noted that all relevant content of each step involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, and will not be elaborated here.

[0160] The functions, actions, operations, steps, etc. in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more integrated media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0161] Although the present application has been described in connection with specific features and their embodiments, it is obvious that various modifications and combinations can be made without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A data processing method, characterized in that, it is applied to an electronic device, and the electronic device maintains a first driver, and the first driver corresponds to a first graphics application programming interface (API); The method includes: The electronic device configures at least two resource groups according to the first graphics API, and the at least two resource groups are used for parallel conversion of one or more rendering instructions; The electronic device converts a first rendering instruction through a first resource group to obtain a second rendering instruction, where the first rendering instruction is a rendering instruction corresponding to a second graphics API, the second rendering instruction is a rendering instruction corresponding to the first graphics API, and the first resource group is included in the at least two configured resource groups; The electronic device executes the second rendering instruction through the first driver.

2. The method according to claim 1, characterized in that, the method further includes: The electronic device converts a third rendering instruction through a second resource group to obtain a fourth rendering instruction, where the third rendering instruction is a rendering instruction corresponding to a second graphics API, the fourth rendering instruction is a rendering instruction corresponding to the first graphics API, and the second resource group is included in the at least two configured resource groups; The electronic device executes the fourth rendering instruction through the first driver.

3. The method according to claim 1 or 2, characterized in that, the conversion of the first rendering instruction and the conversion of the third rendering instruction are executed in parallel.

4. The method according to claim 1 or 2, characterized in that, the electronic device configures at least two resource groups according to the first graphics API, including: The electronic device configures N resource groups according to the maximum number of frame buffers M supported by the first graphics API, where N is an integer greater than 1 and less than or equal to M.

5. The method according to claim 1 or 2, characterized in that, the data indicated to be rendered by the first rendering instruction is the data of the Pth frame image, and the data indicated to be rendered by the third rendering instruction is the data of the (P + 1)th frame image, where P is an integer greater than or equal to 1.

6. The method according to claim 2, characterized in that, a rendering execution module is provided in the electronic device, and the rendering execution module is used to perform a rendering operation according to a rendering instruction; The electronic device executes the fourth rendering instruction through the first driver, including: When the number of instructions of the second rendering instruction is less than a first threshold, the electronic device submits the fourth rendering instruction to the rendering execution module so that the rendering execution module executes the fourth rendering instruction.

7. The method according to claim 2, characterized in that, a rendering execution module is provided in the electronic device, and the rendering execution module is used to perform a rendering operation according to a rendering instruction; The electronic device executes the fourth rendering instruction through the first driver, including: When the number of instructions of the second rendering instruction is greater than a first threshold and the number of instructions of the fourth rendering instruction is less than a second threshold, the electronic device submits the fourth rendering instruction to the rendering execution module so that the rendering execution module executes the fourth rendering instruction.

8. The method according to claim 2, wherein, a rendering execution module is provided in the electronic device, and the rendering execution module is configured to perform a rendering operation according to a rendering instruction; the method further includes: the electronic device converts a fifth rendering instruction through a third resource group to obtain a sixth rendering instruction, where the fifth rendering instruction is a rendering instruction corresponding to a second graphics API, the sixth rendering instruction is a rendering instruction corresponding to the first graphics API, and the third resource group is included in at least two configured resource groups; the data indicated to be rendered by the fifth rendering instruction is the data of the (P + 2)-th frame image; the electronic device executes the fourth rendering instruction through the first driver, including: when the number of instructions of the second rendering instruction is greater than a first threshold, the number of instructions of the fourth rendering instruction is greater than a second threshold, and the number of instructions of the sixth rendering instruction is greater than a third threshold, the electronic device submits the fourth rendering instruction to the rendering execution module so that the rendering execution module executes the fourth rendering instruction.

9. The method according to any one of claims 6-8, wherein, the rendering execution module is a graphics processing unit GPU.

10. The method according to any one of claims 1-2, 6-8, wherein, the first graphics API is Vulkan; the second graphics API is the Open Graphics Library OpenGL, or the second graphics API is the embedded Open Graphics Library OpenGL ES.

11. An electronic device, wherein, the electronic device maintains a first driver; the electronic device includes one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions; when the one or more processors execute the computer instructions, the electronic device is caused to execute the data processing method according to any one of claims 1-10.

12. A computer program product, wherein, the computer program product includes computer instructions, and when the computer instructions run, the data processing method according to any one of claims 1-10 is executed.

Images