Image rendering method and electronic equipment

Abstract

The embodiment of the invention is applicable to the field of image processing, and provides an image rendering method and electronic equipment, the electronic equipment executes at least two image rendering instruction streams, the at least two image rendering instruction streams comprise a first instruction stream and a second instruction stream, and the first instruction stream and the second instruction stream both comprise a first instruction. Comprising the following steps: before executing a first instruction, storing an initial parameter corresponding to the first instruction, when executing the first instruction, determining a first parameter corresponding to the first instruction from the initial parameter and parameters carried in the first instruction, executing the first instruction based on the first parameter, and performing image rendering processing on a to-be-rendered image to obtain a rendered image, when the electronic device executes two or more image rendering tasks including the same first instruction, the first parameter in the instruction stream executed later is the parameter corresponding to the first instruction, and the effect of image rendering obtained based on execution of the first instruction in the instruction stream executed later is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more specifically, to an image rendering method and an electronic device. Background Art

[0002] With the development of graphics processing unit (GPU) in terminal devices, GPU plays an important role in image rendering of terminal devices. For example, rendering instruction reorganization optimization and image rendering technologies such as super frame and super resolution are inseparable from the use of GPU.

[0003] At present, electronic devices usually call the GPU by executing a standard image rendering instruction stream to implement the image rendering function. It should be understood that different image rendering functions are implemented through different image rendering instruction streams. For example, the image rendering instruction stream corresponding to the super-resolution function is different from the image rendering instruction stream corresponding to the super-frame function. However, it is understandable that the image rendering instruction streams corresponding to different functions will use some of the same instructions, such as the first instruction. In this case, if two or more image rendering tasks are performed at the same time, due to the change in the parameters of the first instruction in one instruction stream, when executing another instruction stream, the changed parameters are called, making the parameters called by the first instruction of another instruction stream wrong, which ultimately leads to poor image rendering effect.

[0004] Based on this, how to improve the effect of image rendering has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides an image rendering method, which can improve the image rendering effect.

[0006] In a first aspect, an image rendering method is provided. The method is applied in an electronic device. The electronic device executes at least two image rendering instruction streams. The at least two image rendering instruction streams include a first instruction stream and a second instruction stream. The first instruction stream includes a first instruction, and the second instruction stream includes the first instruction. The method includes:

[0007] Before executing the first instruction, storing initial parameters corresponding to the first instruction;

[0008] When executing the first instruction, determining a first parameter corresponding to the first instruction from the initial parameter and the parameter carried by the first instruction;

[0009] The first instruction is executed based on the first parameter to perform image rendering processing on the image to be rendered to obtain a rendered image.

[0010] An image rendering method provided by an embodiment of the present application is applied to an electronic device, which executes at least two image rendering instruction streams, wherein the at least two image rendering instruction streams include a first instruction stream and a second instruction stream, wherein the first instruction stream includes the first instruction, and the second instruction stream includes the first instruction. The method includes: before executing the first instruction, storing an initial parameter corresponding to the first instruction, and when executing the first instruction, determining a first parameter corresponding to the first instruction from the initial parameter and the parameter carried in the first instruction, executing the first instruction based on the first parameter, performing image rendering processing on an image to be rendered, and obtaining a rendered image, which is equivalent to when the electronic device executes two or more image rendering tasks including the same first instruction, if the first parameter in the first executed instruction stream changes after being executed, the first parameter in the later executed instruction stream is a corresponding parameter selected from the initial parameter and the parameter carried in the first instruction, so that the parameter called by the first instruction in the later executed instruction stream is a correct parameter, thereby preventing resource conflicts between the two image rendering tasks, ensuring that the image rendering task results are correct, and thus improving the image rendering effect obtained based on the first instruction.

[0011] In combination with the first aspect, in certain embodiments of the first aspect, the method also includes: determining whether the first instruction stream and the second instruction stream are instruction streams corresponding to a preset image rendering task; if the first instruction stream and the second instruction stream are instruction streams corresponding to the preset image rendering task, before executing the first instruction, storing initial parameters corresponding to the first instruction.

[0012] The image rendering method provided in the embodiment of the present application first determines whether the first instruction stream and the second instruction stream are whitelist instruction streams before executing the first instruction. If the first instruction stream and the second instruction stream are whitelist instruction streams, the initial parameters corresponding to the first instruction are stored. This can reduce the number of initial parameters stored in the instruction stream management module, thereby avoiding the problem of too many initial parameters stored in the instruction stream management module occupying system memory.

[0013] In combination with the first aspect, in some embodiments of the first aspect, the electronic device includes an instruction flow management module, and the instruction flow management module stores initial parameters.

[0014] In combination with the first aspect, in some embodiments of the first aspect, the instruction stream management module is further used to determine whether the first instruction stream and the second instruction stream are instruction streams corresponding to a preset image rendering task.

[0015] In combination with the first aspect, in some embodiments of the first aspect, the instruction flow management module further includes a first submodule, and the first submodule is used to store initial parameters before executing the first instruction.

[0016] Among them, the first submodule may refer to the Local Layer module.

[0017] In combination with the first aspect, in some embodiments of the first aspect, the instruction flow management module also includes a second sub-module, and the second sub-module is used to store a second parameter, where the second parameter is a parameter sent to the GPU corresponding to the first instruction.

[0018] Among them, the second submodule may refer to the Base Layer module.

[0019] In combination with the first aspect, in certain embodiments of the first aspect, the instruction flow management module also includes a third sub-module, and the third sub-module is used to provide an external library interface to the GPU. The method also includes: the GPU calls the parameters in the external library based on the external library interface provided by the third sub-module to execute the second instruction.

[0020] Among them, the third submodule may refer to the Callback Layer module.

[0021] In combination with the first aspect, in certain embodiments of the first aspect, the instruction flow management module also includes a fourth sub-module, and the fourth sub-module is used to provide a GPU extension driver interface. The method also includes: driving the GPU extension driver through the GPU extension driver interface to drive the GPU to execute a third instruction.

[0022] Among them, the fourth submodule may refer to an External Layer module.

[0023] In a second aspect, an image rendering device is provided, comprising a unit for executing any one of the methods in the first aspect. The device may be a server, a terminal device, or a chip in a terminal device. The device may include an acquisition unit and a processing unit.

[0024] When the apparatus is a terminal device, the processing unit may be a processor, and the acquisition unit may be a communication interface; the terminal device may further include a memory for storing computer program code, and when the processor executes the computer program code stored in the memory, the terminal device executes any one of the methods in the first aspect.

[0025] When the device is a chip in a terminal device, the processing unit may be a processing unit inside the chip, and the acquisition unit may be an output interface, a pin or a circuit, etc.; the chip may also include a memory, which may be a memory inside the chip (for example, a register, a cache, etc.) or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the methods in the first aspect.

[0026] In one possible implementation, a memory is used to store computer program code; a processor executes the computer program code stored in the memory, and when the computer program code stored in the memory is executed, the processor is used to perform: before executing a first instruction, storing initial parameters corresponding to the first instruction; when executing the first instruction, determining a first parameter corresponding to the first instruction from the initial parameters and the parameters carried by the first instruction; executing the first instruction based on the first parameter, performing image rendering processing on the image to be rendered, and obtaining a rendered image.

[0027] According to a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an image rendering device, the image rendering device executes any one of the image rendering methods according to the first aspect.

[0028] In a fourth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an image rendering device, the image rendering device executes any one of the device methods in the first aspect.

[0029] An image rendering method and an electronic device provided by an embodiment of the present application, the electronic device executes at least two image rendering instruction streams, the at least two image rendering instruction streams include a first instruction stream and a second instruction stream, the first instruction stream includes the first instruction, and the second instruction stream includes the first instruction. The method includes: before executing the first instruction, storing an initial parameter corresponding to the first instruction, when executing the first instruction, determining a first parameter corresponding to the first instruction from the initial parameter and the parameter carried in the first instruction, executing the first instruction based on the first parameter, performing image rendering processing on the image to be rendered, and obtaining a rendered image, which is equivalent to when the electronic device executes two or more image rendering tasks including the same first instruction, if the first parameter in the first executed instruction stream changes after being executed, the first parameter in the later executed instruction stream is a corresponding parameter selected from the initial parameter and the parameter carried in the first instruction, so that the parameter called by the first instruction in the later executed instruction stream is the correct parameter, thereby preventing resource conflicts between the two image rendering tasks, ensuring that the image rendering task results are correct, and thus improving the image rendering effect obtained based on the first instruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a schematic diagram of a hardware system of an electronic device applicable to the present application;

[0031] Figure 2 is a software structure block diagram of an electronic device applicable to the present application;

[0032] Figure 3is a schematic diagram of an application scenario of the image rendering method provided in an embodiment of the present application;

[0033] Figure 4 It is a flowchart of an image rendering method provided in an embodiment of the present application;

[0034] Figure 5 It is a software structure block diagram applicable to an image rendering method provided in an embodiment of the present application;

[0035] Figure 6 is a flowchart of another image rendering method provided by an embodiment of the present application;

[0036] Figure 7 It is a software structure block diagram applicable to another image rendering method provided in an embodiment of the present application;

[0037] Figure 8 It is a software structure block diagram applicable to another image rendering method provided in an embodiment of the present application;

[0038] Fig. 9 is a flowchart of another image rendering method provided by an embodiment of the present application;

[0039] Fig.10 is a schematic diagram of an image rendering device provided by the present application;

[0040] Fig.11 It is a schematic diagram of an electronic device for image rendering provided by the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0042] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0043] In order to facilitate the understanding of the embodiments of the present application, the relevant concepts involved in the embodiments of the present application are first briefly described.

[0044] 1. Image Rendering

[0045] Image rendering may include the process of generating an image based on a three-dimensional model, wherein the three-dimensional model may be a three-dimensional graphic or virtual scene positioned using a programming language or data result, and typically includes rendering information such as geometric shape, viewpoint, image texture, lighting and shadow.

[0046] 2. Super resolution

[0047] Super-resolution refers to adjusting the resolution of an image from low resolution to high resolution. Electronic devices can find the jagged edges of objects in the image and eliminate these jagged edges to improve the resolution of the image.

[0048] 3. Superframe

[0049] Superframe uses the interpolation function of the image processing chip to interpolate the low frame rate game screen into a higher frame rate screen. It should be understood that the higher the frame rate, the better the screen display effect. Superresolution and superframe are usually concurrent, that is, superframe and superresolution are turned on at the same time.

[0050] 4. Instruction Flow

[0051] The instruction stream usually includes multiple drawing instructions. The GPU obtains display data by running the drawing instructions in the instruction stream in sequence, so that the display interface displays images according to the display data. In order to improve the efficiency of image rendering, electronic devices usually pre-store some instruction streams for common image rendering tasks, such as the instruction stream corresponding to the super-resolution function and the instruction stream corresponding to the super-frame function. The GPU runs the instruction stream corresponding to the super-resolution function to achieve image super-resolution.

[0052] 5. Drawing instructions

[0053] Drawing instructions usually refer to instructions corresponding to drawing targets in image frames. Drawing instructions are run by the GPU, and the corresponding parameters are called to implement the drawing function corresponding to the drawing execution.

[0054] It should be understood that during the image rendering process, the GPU executes the drawing instruction and calls the corresponding parameters from the database to perform image rendering. After the drawing instruction is executed, the called parameters usually change, and the updated parameters are stored in the database. In this case, if the electronic device executes the drawing instruction again, the called parameters are usually the updated parameters.

[0055] At present, with the development of GPU in terminal devices, GPU plays an important role in the image rendering of terminal devices. For example, image rendering technologies such as rendering instruction reorganization optimization, super frame, super resolution, etc. are inseparable from the use of GPU.

[0056] Electronic devices usually call the GPU by executing a standard image rendering instruction stream to implement different image rendering functions. It should be understood that different image rendering functions are implemented through different image rendering instruction streams. For example, the image rendering instruction stream corresponding to the super-resolution function is different from the image rendering instruction stream corresponding to the super-frame function. However, it is understandable that there may be some identical instructions in the image rendering instruction streams corresponding to different functions. In this case, if two or more image rendering tasks are performed at the same time, the parameters change after the execution of the previous instruction stream in the same instruction, resulting in the calling of the changed parameters when the instructions in the subsequent instruction stream are executed, making the parameters called by the instructions in the subsequent instruction stream wrong, which ultimately leads to a poor image rendering effect.

[0057] In view of this, an embodiment of the present application provides an image rendering method and an electronic device, which executes at least two image rendering instruction streams, the at least two image rendering instruction streams including a first instruction stream and a second instruction stream, the first instruction stream including the first instruction, and the second instruction stream including the first instruction, the method including: before executing the first instruction, storing an initial parameter corresponding to the first instruction, when executing the first instruction, determining a first parameter corresponding to the first instruction from the initial parameter and the parameter carried in the first instruction, executing the first instruction based on the first parameter, performing image rendering processing on the image to be rendered, and obtaining a rendered image, which is equivalent to when the electronic device executes two or more image rendering tasks including the same first instruction, if the first parameter in the first executed instruction stream changes after being executed, the first parameter in the later executed instruction stream is a corresponding parameter selected from the initial parameter and the parameter carried in the first instruction, so that the parameter called by the first instruction in the later executed instruction stream is the correct parameter, thereby preventing resource conflicts between the two image rendering tasks, ensuring that the image rendering task results are correct, and thus improving the image rendering effect obtained based on the first instruction.

[0058] The image rendering method provided in the embodiment of the present application can be applied to electronic devices. Optionally, the electronic device includes a terminal device, which can also be called a terminal, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The terminal device can be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, etc. The embodiment of the present application does not limit the specific technology and specific device form adopted by the terminal device.

[0059] For example, Figure 1 1 shows a schematic diagram of the structure of the electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0060] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0061] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0062] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0063] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0064] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0065] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0066] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0067] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0068] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0069] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0070] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0071] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0072] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0073] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0074] It should be noted that any electronic device mentioned in the embodiments of the present application may include more or fewer modules in the electronic device 100.

[0075] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0076] Figure 2 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0077] The layered architecture of the electronic device 100 divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0078] The application layer can include a series of application packages.

[0079] like Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, game and other applications.

[0080] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0081] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, an instruction flow management module, and the like.

[0082] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0083] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0084] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0085] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0086] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0087] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0088] The instruction stream management module is used to intercept the instruction stream sent by the game application to the GPU and store the address information of the initial parameters carried in the instruction stream so that the GPU can call the initial parameters when executing the drawing instructions in the instruction stream.

[0089] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0090] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0091] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0092] In one possible case, when the electronic device executes two or more image rendering functions simultaneously, the two or more image rendering tasks, that is, the instruction streams corresponding to the two or more image rendering functions, can be run simultaneously through a virtual machine.

[0093] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc. When the GPU executes drawing instructions, it usually needs to call parameters in OpenGL ES.

[0094] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0095] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0096] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0097] A 2D graphics engine is a drawing engine for 2D drawings.

[0098] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, sensor driver, Wi-Fi driver, GPU driver, etc.

[0099] It should be noted that the electronic device mentioned in the embodiments of the present application may include more or fewer modules in the above electronic device. For example, the electronic device may also include a memory, a timer, etc.

[0100] The following is an example of the workflow of the software and hardware of the electronic device 100 in conjunction with a game application scenario.

[0101] When the game application starts running, the electronic device needs to generate multiple frames of images through image rendering, and display each frame of image in sequence, so as to display the game screen. In the process of generating multiple frames of images, the game application in the application layer sends an instruction stream to the GPU so that the GPU executes the instructions in the instruction stream to perform image rendering and generate images. Among them, the instruction stream passes through the application layer, the application framework layer, and the system library in sequence to the kernel layer, and the GPU is called by the GPU driver in the kernel layer so that the GPU can execute the drawing instructions in the instruction stream to complete the image rendering and generate images. Among them, when the GPU executes the drawing instructions in the instruction stream, it needs to read the parameter address carried in the drawing instructions and call the parameters stored in the parameter address to execute the drawing instructions. At this time, the image rendering task is run in the Android runtime in the system library. In some possible cases, rendering a frame of image requires multiple image rendering tasks to complete, that is, multiple instruction streams are executed simultaneously to complete the image rendering of the frame of image. At this time, multiple image rendering tasks will be run simultaneously in the Android runtime.

[0102] The application scenarios provided by the embodiments of the present application are described below with reference to the accompanying drawings.

[0103] Figure 3 Schematic diagram of an application scenario of the image rendering method provided in the embodiment of the present application. Figure 3 As shown in (a) of FIG. 1 , the user may click on the control 101 of the game application. In response to the user's click operation, the electronic device displays a game screen 200, such as Figure 3 As shown in (b) in FIG. 2 , a picture quality enhancement control 201 and a frame rate enhancement control 202 are displayed on the game screen 200. The user clicks the picture quality enhancement control 201 and the frame rate enhancement control 202, as shown in FIG. Figure 3 As shown in (c) of FIG. 1 , in response to the user clicking the image quality enhancement control 201 and the frame rate enhancement control 202, the game screen 300 is displayed, such as Figure 3 As shown in (d) in the figure. Among them, the game screen 300 is obtained by the electronic device rendering the game screen 200 in response to the operation of clicking the image quality enhancement control 201, so that the image quality of the game screen 300 is better than the image quality of the game screen 200. It can be understood that the image rendering of the game screen 200 may include one or more of image super-frame and image super-resolution, and the embodiment of the present application does not limit this. If the image rendering of the game screen 200 includes both image super-frame and image super-resolution, that is, the user clicks the image quality enhancement control 201 and the frame rate enhancement control 202, the image super-frame task and the image super-resolution task are run simultaneously during Android.

[0104] It should be understood that when the user clicks on the control 101 of the game application, the electronic device also needs to generate the game screen through image rendering. In the process of generating the game screen, the Android in the electronic device can also run the image super-frame task and the image super-resolution task at the same time. The embodiment of the present application does not limit this.

[0105] It should be understood that the above is an example of an application scenario and does not limit the application scenario of the present application.

[0106] Exemplarily, the image rendering method provided in the embodiments of the present application can also be applied to scenarios such as home design, image modeling, enhanced display applications, and virtual display applications, and the embodiments of the present application are not limited to this.

[0107] Combine the following Figure 4 The image rendering method provided in the embodiment of the present application is described in detail.

[0108] Figure 4 A flowchart of an image rendering method provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the method is applied in an electronic device, the electronic device executes at least two image rendering instruction streams, the at least two image rendering instruction streams include a first instruction stream and a second instruction stream, the first instruction stream includes a first instruction, and the second instruction stream includes the first instruction, the method includes:

[0109] S101. Before executing a first instruction, store initial parameters corresponding to the first instruction.

[0110] It should be understood that when executing the first instruction in the first instruction stream and executing the first instruction in the second instruction stream, if the parameters of the call are different, different results will be produced. For example, the first instruction is to call an image texture to fill the image outline. However, the image texture called by executing the first instruction in the first instruction stream and the image texture called by executing the first instruction in the second instruction stream can be different image textures, so that the rendered image obtained based on executing the first instruction in the first instruction stream is different from the rendered image obtained by executing the first instruction in the second instruction stream. This is equivalent to calling different parameters when executing the same instruction, so that the rendered images obtained after executing the same instruction are different.

[0111] When an electronic device starts to render an image, for example, starts to render an image of a game screen, the game application at the application layer sends an instruction stream to the GPU, wherein the instructions in the instruction stream sent by the game application carry parameter addresses corresponding to each instruction specified by the game application.

[0112] It can be understood that the first instruction stream and the second instruction stream can be OpenGLES instruction streams or Vulkan instruction streams, and the embodiment of the present application does not limit this.

[0113] Take the first instruction in the first instruction stream sent by the game application as an example for explanation. Figure 5 As shown, when the game application sends the first instruction stream to the GPU, when the first instruction in the first instruction stream passes through the instruction stream management module of the application framework layer along with the first instruction stream, the instruction stream management module intercepts the first instruction, obtains the parameter address of the initial parameter carried in the first instruction, reads the initial parameter in the parameter address, and stores the initial parameter in the instruction stream management module. Then the instruction stream management module sends the first instruction stream to the GPU driver of the kernel layer, and then sends the first instruction stream to the GPU in the hardware layer through the GPU driver. After receiving the first instruction stream, the GPU executes the first instruction in the first instruction stream.

[0114] In one possible case, after the GPU executes the first instruction, the initial parameters in the parameter address are usually modified to obtain new parameters. At this time, the parameters carried in the first instruction are the new parameters, that is, the initial parameters are different from the parameters carried in the first instruction.

[0115] In one possible case, after the GPU executes the first instruction, the initial parameters in the parameter address are not modified. In this case, the parameters carried in the first instruction are the initial parameters, that is, the initial parameters are the same as the parameters carried in the first instruction.

[0116] S102: When executing the first instruction, determine a first parameter corresponding to the first instruction from the initial parameters and the parameters carried in the first instruction.

[0117] It should be understood that when an application renders an image, it usually sends multiple instruction streams simultaneously. For example, a game application sends a first instruction stream and a second instruction stream simultaneously, and both the first instruction stream and the second instruction stream include the first instruction.

[0118] As described in S101, in one possible case, after the GPU executes the first instruction in the first instruction stream, the initial parameter in the parameter address is usually modified to obtain a new parameter. At this time, the parameter carried in the first instruction is the new parameter, that is, the initial parameter is different from the parameter carried in the first instruction.

[0119] At this time, if the GPU directly calls the parameters carried in the first instruction when executing the first instruction in the second instruction stream, a parameter call error may occur, resulting in a final image rendering error. Therefore, the image rendering method provided in the embodiment of the present application needs to first determine the first parameter corresponding to the first instruction from the initial parameters and the second parameters when the later executed instruction (the first instruction in the second instruction stream) is executed, and then call the first parameter to execute the first instruction.

[0120] For example, when a game application sends a second instruction stream to the GPU, it first calls the first instruction to the rendering task. The rendering task decides whether the parameters required to execute the first instruction in the second instruction stream are the initial parameters or the parameters carried in the first instruction.

[0121] S103 . Execute the first instruction based on the first parameter to perform image rendering processing on the image to be rendered to obtain a rendered image.

[0122] The image rendering method provided by the embodiment of the present application is applied to an electronic device, which executes at least two image rendering instruction streams, the at least two image rendering instruction streams include a first instruction stream and a second instruction stream, the first instruction stream includes the first instruction, and the second instruction stream includes the first instruction. The method includes: before executing the first instruction, storing an initial parameter corresponding to the first instruction, when executing the first instruction, determining a first parameter corresponding to the first instruction from the initial parameter and the parameter carried in the first instruction, executing the first instruction based on the first parameter, performing image rendering processing on the image to be rendered, and obtaining a rendered image, which is equivalent to when the electronic device executes two or more image rendering tasks including the same first instruction, if the first parameter in the first executed instruction stream changes after being executed, the first parameter in the later executed instruction stream is a corresponding parameter selected from the initial parameter and the parameter carried in the first instruction, so that the parameter called by the first instruction in the later executed instruction stream is the correct parameter, thereby preventing resource conflicts between the two image rendering tasks, ensuring that the image rendering task results are correct, and thus improving the image rendering effect obtained based on the first instruction.

[0123] In one possible case, the image rendering task needs to first check whether the first instruction stream and the second instruction stream are instruction streams corresponding to the image rendering function in the whitelist, and then intercept the first instruction in the first instruction stream. Figure 6 The illustrated embodiment is described in detail.

[0124] Figure 6 A flowchart of an image rendering method provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the method is applied in an electronic device, the electronic device executes at least two image rendering instruction streams, the at least two image rendering instruction streams include a first instruction stream and a second instruction stream, the first instruction stream includes a first instruction, and the second instruction stream includes the first instruction, the method includes:

[0125] S201. A game application sends a first instruction stream and a second instruction stream to an instruction stream management module.

[0126] The game application starts rendering the image when displaying the game screen, and sends the first instruction stream and the second instruction stream. It can be understood that the first instruction stream and the second instruction stream can be OpenGLES instruction streams or Vulkan instruction streams, and the embodiment of the present application does not limit this.

[0127] For example, Figure 7 As shown, the game application can send the first instruction stream and the second instruction stream to the instruction stream management module through the EGL / GL Wrapper or Vulkan Layer in the application framework layer.

[0128] For example, if the first instruction stream is an OpenGLES instruction stream, the game application can send the first instruction stream to the instruction stream management module through the EGL / GL Wrapper.

[0129] For example, if the second instruction stream is a Vulkan instruction stream, the game application can send the second instruction stream to the instruction stream management module through the Vulkan Layer.

[0130] S202: The instruction stream management module determines whether the first instruction stream and the second instruction stream are whitelist instruction streams. If the first instruction stream and the second instruction stream are whitelist instruction streams, S203 is executed.

[0131] The instruction stream management module can determine whether the first instruction stream and the second instruction stream are instructions in the whitelist when the image rendering task is initialized. If the first instruction stream and the second instruction stream are instruction streams in the whitelist, the first instruction is intercepted, that is, S203 is executed.

[0132] The image rendering method provided in the embodiment of the present application first determines whether the first instruction stream and the second instruction stream are whitelist instruction streams before executing the first instruction. If the first instruction stream and the second instruction stream are whitelist instruction streams, the initial parameters corresponding to the first instruction are stored. This can reduce the number of initial parameters stored in the instruction stream management module, thereby avoiding the problem of too many initial parameters stored in the instruction stream management module occupying system memory.

[0133] S203. Before executing the first instruction, the instruction flow management module stores initial parameters corresponding to the first instruction.

[0134] S204: The instruction flow management module calls the first instruction to the GPU.

[0135] S205. When executing the first instruction, the GPU determines a first parameter corresponding to the first instruction from the initial parameters and the parameters carried in the first instruction.

[0136] For example, when executing the first instruction in the first instruction stream, the image rendering task corresponding to the first instruction stream determines whether the GPU needs to call the initial parameters or the parameters carried in the first instruction. If the image rendering task corresponding to the first instruction stream determines that the GPU needs to call the initial parameters, the initial parameters are called; if the image rendering task corresponding to the first instruction stream determines that the GPU needs to call the parameters carried in the first instruction, the parameters carried in the first instruction are called.

[0137] For example, when executing the first instruction in the second instruction stream, the image rendering task corresponding to the second instruction stream determines whether the GPU needs to call the initial parameters or the parameters carried in the first instruction. If the image rendering task corresponding to the second instruction stream determines that the GPU needs to call the initial parameters, the initial parameters are called; if the image rendering task corresponding to the second instruction stream determines that the GPU needs to call the parameters carried in the first instruction, the parameters carried in the first instruction are called.

[0138] It should be understood that if the first instruction in the first instruction stream is the instruction executed first, and the first parameter of the image rendering task decision corresponding to the second instruction stream is the parameter carried in the first instruction, then the second instruction stream is usually an instruction stream related to the first instruction stream, that is, the first instruction in the second instruction stream is usually an instruction for further performing image rendering based on the image obtained based on the first instruction in the first instruction stream after the first instruction in the first instruction stream is executed.

[0139] It should be understood that if the first instruction in the first instruction stream is the instruction executed first, and the first parameter of the image rendering task corresponding to the second instruction stream is determined as an initial parameter, then the second instruction stream is usually an instruction stream independent of the first instruction stream, that is, the image rendering task corresponding to the second instruction stream is unrelated to the image rendering task corresponding to the first instruction stream.

[0140] S206: The GPU executes the first instruction based on the first parameter, performs image rendering processing on the image to be rendered, and obtains a rendered image.

[0141] In one possible scenario, Figure 8As shown, the instruction flow management module also includes Local Layer (equivalent to the first submodule), Base Layer (equivalent to the second submodule), Callback Layer (equivalent to the third submodule) and External Layer (equivalent to the fourth submodule). Among them, Local Layer is used to intercept the instruction stream sent by the application program of the application layer and store the initial parameters; Base Layer is connected to the GPU driver and is used to intercept the instructions sent by the image rendering task in the Framework layer to the GPU driver and store the parameters sent to the GPU driver; External Layer is used to map the GPU extension interface provided by the GPU manufacturer so that the image rendering task can easily access the GPU manufacturer's extension interface; CallbackLayer is used to manage the registration of callback functions and can call the interface of the native layer, including the interface of non-standard OpenGL, so that the CPU can call the parameters in the external library through the external library interface.

[0142] In addition, the image rendering task can map a management module at the application framework layer to manage the instructions for implementing the image rendering task. Figure 8 Rendering task management module 1 and rendering task management module 2 are shown in FIG.

[0143] It should be understood that one image rendering task corresponds to one management module at the application framework layer. That is, rendering task management module 1 corresponds to one image rendering task, and rendering task management module 2 corresponds to another image rendering task. Exemplarily, rendering task management module 1 corresponds to image rendering task 1, and rendering task management module 2 corresponds to image rendering task 2.

[0144] For example, a game application usually displays each frame of the game screen, including the drawing and post-processing of hundreds of objects, so it usually needs to use multiple application programming interface (API) instructions. For example, when a game application draws one of the objects, it needs to execute the following API instructions in sequence:

[0145] glBindFramebuffer(target, fbID); / / Specify the frame buffer target for the binding operation;

[0146] glViewport(x,y,width,height); / / Specify the coordinate position and size of the drawing window of the current frame buffer;

[0147] glScissor(x,y,width,height); / / Specify the coordinate position and size of the clipping window of the current frame buffer;

[0148] glActiveTexture(texNumber); / / Specify the enabled texture unit;

[0149] glBindTexture(target,texID); / / Specify the bound texture corresponding to the enabled texture unit;

[0150] glUseProgram(progID); / / Specify the program to be used;

[0151] glUniform4fv(location,count,value); / / Specify the parameter value of the uniform variable of the current program;

[0152] glBindBuffer(target,buffer1); / / Specify the bound cache object buffer1;

[0153] glBufferSubData(buffer,data); / / Specify the data submitted to the buffer object;

[0154] glBindBuffer(target,buffer2); / / Specify the bound cache object buffer2;

[0155] glVertexAttribPointer(index,size,type,...); / / Specify the array of general vertex attribute data;

[0156] glBindBuffer(target,buffer3); / / Specify the bound cache object buffer3;

[0157] glDrawElements(mode,count,type,indices); / / Draw the image according to the specified vertex attributes;

[0158] glBindFramebuffer(target,fbID); / / Specify the frame buffer target for the binding operation;

[0159] glBlitFramebuffer(srcX0,srcY0,srcX1,srcY1,...); / / Instruct the GPU to copy the specified frame buffer to another specified frame buffer;

[0160] eglSwapBuffers(eglDisplay, surface); / / Instruct the GPU to pass the drawn image to the display module.

[0161] Among them, the Local Layer in the instruction flow management module can intercept the glBindFramebuffer, glBindTexture, glUseProgram, glUniform4fv, glDrawElements, glBlitFramebuffer and eglSwapBuffers instructions, obtain the addresses of these instructions, and then modify these addresses to new addresses (that is, the addresses managed by LocalLayer), and return the new addresses to the game application. When the GPU executes the above instructions, it can call the instructions in the new address and execute the instructions.

[0162] It should be understood that different instructions are executed in different image rendering tasks. Therefore, when executing different image rendering tasks, the instructions intercepted by the Local Layer may be part or all of the above instructions.

[0163] Exemplarily, the instructions intercepted by the Local Layer in image rendering task 1 include: glBindFramebuffer, glBindTexture, glUseProgram, glUniform4fv, glBlitFramebuffer, and eglSwapBuffers.

[0164] Exemplarily, the instructions intercepted by the Local Layer in the image rendering task 2 include: glBindFramebuffer, glBindTexture, glUseProgram, and glDrawElements.

[0165] The Base Layer in the instruction flow management module can intercept the glBindFramebuffer instruction, obtain the addresses carried in these instructions, read the parameters in these addresses, store the parameters in these addresses in new addresses, and return the new addresses to the game application. When the GPU executes the above instructions, it can call the parameters in the new address to execute the instructions.

[0166] Combine the following Fig. 9 Describe how the Local Layer, Base Layer, External Layer and Callback Layer in the embodiments of the present application intercept instructions.

[0167] like Fig. 9 As shown, the method includes:

[0168] S301: The game application calls a first instruction to the Local Layer.

[0169] It should be understood that the display of each frame of the game screen by the game application usually includes the drawing and post-processing of hundreds of objects, so multiple API instructions are usually required, and these instructions are usually executed in the GPU. The game application calling the first instruction to the Local Layer may refer to the game application transmitting a message to execute the first instruction to the GPU through the Local Layer, and usually this message carries the initial parameters of the game application for the first instruction.

[0170] S302. Local Layer stores the initial parameters carried in the first instruction.

[0171] S303, Local Layer calls the first instruction to the rendering task management module 1.

[0172] Among them, the local layer calling the first instruction to the rendering task management module 1 may refer to the local layer transmitting a message to execute the first instruction to the GPU through the rendering task management module 1.

[0173] The rendering task management module 1 may refer to a management module corresponding to the image rendering task 1 , that is, the rendering task management module 1 manages instructions required to be executed to implement the functions of the image rendering task 1 .

[0174] The rendering task management module 2 may refer to a management module corresponding to the image rendering task 2 , that is, the rendering task management module 2 manages the instructions required to be executed to implement the functions of the image rendering task 2 .

[0175] If the first instruction currently required to be executed is an instruction required to be executed to implement the function of image rendering task 2, then S304 is executed.

[0176] S304: Rendering task management module 1 calls a first instruction to rendering task management module 2.

[0177] The rendering task management module 2 may refer to a management module corresponding to the image rendering task 2 , that is, the rendering task management module 2 manages the instructions required to be executed to implement the functions of the image rendering task 2 .

[0178] In one possible case, the first instruction is an instruction that needs to be executed to implement the function of the image rendering task 2, and the rendering task management module 2 calls the first instruction to the GPU.

[0179] In this case, the electronic device can determine the parameters used when executing the first instruction through the rendering task management module 2. For example, if the rendering task management module 2 decides to use the initial parameters sent by the game application to execute the first instruction, the rendering task management module 2 calls the first instruction to the GPU, and the GPU executes the first instruction based on the initial parameters. Or if the rendering task management module 2 decides to use the first parameters to execute the first instruction, the rendering task management module 2 calls the first instruction to the GPU, and the GPU executes the first instruction based on the first parameters.

[0180] S305 , the rendering task management module 2 decides to replace the initial parameter in the first instruction with the first parameter to obtain an updated first instruction.

[0181] S306: The rendering task management module 2 calls the first instruction to the Base Layer.

[0182] S307. The Base Layer stores the first parameter based on the first instruction.

[0183] S308. The Base Layer calls the first instruction to the GPU.

[0184] S309: The GPU executes the first instruction based on the first parameter.

[0185] It should be understood that if the first instruction is called when the image rendering task 2 is executed again, the GPU obtains the first parameter passed to the GPU from the Base Layer, and then executes the first instruction based on the first parameter.

[0186] S301 to S309 mainly describe that the Base Layer is used to store parameters and send the first parameter to the GPU when the GPU needs to call the parameter.

[0187] The following describes through S310 to S313 the specific process of how to obtain initial parameters from the Local Layer when executing the image rendering task 1.

[0188] S310 , the game application calls a first instruction to the rendering task management module 2 .

[0189] S311, the rendering task management module 2 decides to obtain initial parameters from the Local Layer.

[0190] S312: The rendering task management module 2 calls a first instruction to the GPU.

[0191] It should be understood that the rendering task management module 1 needs to call the first instruction to the GPU through the GPU driver.

[0192] S313: The GPU executes the first instruction based on the initial parameters.

[0193] The following describes through S314 to S317 the specific process of how to obtain the first parameter from the Base Layer when executing the image rendering task 1.

[0194] S314: The game application calls a first instruction to the rendering task management module 1.

[0195] S315, the rendering task management module 1 decides to obtain the first parameter from the Base Layer.

[0196] S316. The rendering task management module 1 calls a first instruction to the GPU.

[0197] It should be understood that the rendering task management module 1 calling the first instruction to the GPU may mean that the GPU reads the first parameter in the Base Layer and executes the first instruction based on the first parameter.

[0198] S317: The GPU executes the first instruction based on the first parameter.

[0199] In one possible case, the instruction executed by the GPU (for example, the second instruction) is not a standard instruction, that is, the second instruction is an instruction in an external database, and the rendering task management module 1 cannot use the standard instruction interface to directly call the second instruction. Therefore, an external database interface can be provided to the GPU through the Callback Layer, so that the GPU can call the second instruction in the external database through the external database interface provided by the Callback Layer, and then execute the second instruction.

[0200] S318: The game application calls a second instruction to the rendering task management module 1.

[0201] The parameter used when executing the second instruction is to obtain the second parameter from the external library.

[0202] It should be understood that the game application can also call the second instruction to the rendering task management module 2, and the embodiment of the present application does not limit this. The following is an example of the game application calling the second instruction to the rendering task management module 1.

[0203] S319, the rendering task management module 1 calls the external interface based on the second instruction Callback Layer.

[0204] S320. Callback Layer obtains external interface information.

[0205] S321. Callback Layer sends external interface information to rendering task management module 1.

[0206] S322. The rendering task management module 1 calls a second instruction to the GPU based on the external interface information.

[0207] S323: The GPU obtains a second parameter from the external library based on the external interface information, and executes a second instruction based on the second parameter.

[0208] In one possible case, calling the third instruction to the GPU requires calling the third instruction to the GPU through a GPU extension driver, and the GPU extension driver cannot be directly called by the rendering task management module. The rendering task management module 1 can obtain the interface of the GPU extension driver through the External Layer, and when calling the third instruction to the GPU, call the third instruction to the GPU through the GPU extension driver.

[0209] S324: The game application calls a third instruction to the rendering task management module 1.

[0210] When executing the third instruction, it is necessary to call the GPU to execute the third instruction through the GPU extension driver.

[0211] The following description will be made by taking an example where a game application calls the third instruction to the rendering task management module 1 .

[0212] S325. The rendering task management module 1 calls the GPU extension driver to the External Layer based on the third instruction.

[0213] S326. The External Layer calls the third instruction to the GPU through the GPU extension driver interface.

[0214] S327: The GPU executes the third instruction.

[0215] It should be understood that, although each step in the flow chart in the above-described embodiment is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0216] It is understandable that in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0217] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. It should be noted that the names of the modules in the embodiment of the present application are schematic and are not limited to the names of the modules in actual implementation.

[0218] Fig.10 A schematic diagram of the structure of an image rendering device provided in an embodiment of the present application.

[0219] It should be understood that the image rendering device 600 may execute Figures 4 to 9 The image rendering method shown; the image rendering device 600 includes: an acquisition unit 610 and a processing unit 620.

[0220] The acquisition unit 610 is used to store initial parameters corresponding to the first instruction before executing the first instruction;

[0221] The processing unit 620 is used to determine a first parameter corresponding to the first instruction from the initial parameter and the parameter carried by the first instruction when executing the first instruction;

[0222] The processing unit 620 is used to execute the first instruction based on the first parameter, perform image rendering processing on the image to be rendered, and obtain a rendered image.

[0223] In one embodiment, the processing unit 620 is further configured to determine whether the first instruction stream and the second instruction stream are instruction streams corresponding to a preset image rendering task;

[0224] The processing unit 620 is specifically configured to store initial parameters corresponding to the first instruction before executing the first instruction if the first instruction stream and the second instruction stream are instruction streams corresponding to the preset image rendering task.

[0225] In one embodiment, the electronic device includes an instruction flow management module, and the instruction flow management module stores initial parameters.

[0226] In one embodiment, the instruction stream management module is further used to determine whether the first instruction stream and the second instruction stream are instruction streams corresponding to a preset image rendering task.

[0227] In one embodiment, the instruction flow management module further includes a first submodule, and the first submodule is used to store initial parameters before executing the first instruction.

[0228] In one embodiment, the instruction flow management module further includes a second submodule, and the second submodule is used to store a second parameter, where the second parameter is a parameter corresponding to the first instruction and sent to the GPU.

[0229] In one embodiment, the instruction flow management module further includes a third submodule, which is used to provide an external library interface to the GPU, so that the GPU calls parameters in the external library based on the external library interface provided by the third submodule to execute the second instruction.

[0230] In one embodiment, the instruction flow management module further includes a fourth submodule, which is used to provide a GPU extension driver interface, drive the GPU extension driver through the GPU extension driver interface, and drive the GPU to execute the third instruction.

[0231] The image rendering device provided in this embodiment is used to execute the image rendering method of the above embodiment. The technical principle and technical effect are similar and will not be described in detail here.

[0232] It should be noted that the above-mentioned image rendering device 600 is embodied in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0233] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0234] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0235] Fig.11A structural schematic diagram of an electronic device provided by the present application is shown. Fig.11 The dotted line in indicates that the unit or the module is optional. The electronic device 700 can be used to implement the image rendering method described in the above method embodiment.

[0236] The electronic device 700 includes one or more processors 701, which can support the electronic device 700 to implement the image rendering method in the method embodiment. The processor 701 can be a general-purpose processor or a special-purpose processor. For example, the processor 701 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, such as discrete gates, transistor logic devices or discrete hardware components.

[0237] The processor 701 may be used to control the electronic device 700, execute software programs, and process data of the software programs. The electronic device 700 may also include a communication unit 705 to implement input (reception) and output (transmission) of signals.

[0238] For example, the electronic device 700 may be a chip, the communication unit 705 may be an input and / or output circuit of the chip, or the communication unit 705 may be a communication interface of the chip, and the chip may be a component of a terminal device or other electronic devices.

[0239] For another example, the electronic device 700 may be a terminal device, and the communication unit 705 may be a transceiver of the terminal device, or the communication unit 705 may be a transceiver circuit of the terminal device.

[0240] The electronic device 700 may include one or more memories 702 on which a program 704 is stored. The program 704 can be executed by the processor 701 to generate instructions 703, so that the processor 701 executes the impedance matching method described in the above method embodiment according to the instructions 703.

[0241] Optionally, data may be stored in the memory 702. Optionally, the processor 701 may read data stored in the memory 702. The data may be stored at the same storage address as the program 704, or may be stored at a different storage address than the program 704.

[0242] The processor 701 and the memory 702 may be provided separately or integrated together; for example, integrated on a system on chip (SOC) of a terminal device.

[0243] Exemplarily, the memory 702 can be used to store the related program 704 of the image rendering method provided in the embodiment of the present application, and the processor 701 can be used to call the related program 704 of the image rendering method stored in the memory 702 when performing image rendering, and execute the image rendering method of the embodiment of the present application; including: before executing the first instruction, storing the initial parameters corresponding to the first instruction; when executing the first instruction, determining the first parameter corresponding to the first instruction from the initial parameters and the parameters carried by the first instruction; executing the first instruction based on the first parameter, performing image rendering processing on the image to be rendered, and obtaining a rendered image.

[0244] The present application also provides a computer program product, which, when executed by the processor 701, implements the image rendering method described in any method embodiment of the present application.

[0245] The computer program product may be stored in the memory 702 , for example, a program 704 , which is finally converted into an executable target file that can be executed by the processor 701 after preprocessing, compiling, assembling, and linking.

[0246] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the image rendering method described in any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0247] The computer-readable storage medium is, for example, a memory 702. The memory 702 may be a volatile memory or a nonvolatile memory, or the memory 702 may include both a volatile memory and a nonvolatile memory. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0248] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0249] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0250] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0251] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0252] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic; for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0253] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0254] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0255] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image rendering method, characterized in that: The method is applied in an electronic device, the electronic device executes at least two image rendering instruction streams, the at least two image rendering instruction streams include a first instruction stream and a second instruction stream, the first instruction stream includes a first instruction, and the second instruction stream includes the first instruction, the method includes: Before executing the first instruction, storing initial parameters corresponding to the first instruction; When executing the first instruction, determining a first parameter corresponding to the first instruction from the initial parameter and the parameter carried by the first instruction; The first instruction is executed based on the first parameter to perform image rendering processing on the image to be rendered to obtain a rendered image.

2. The method according to claim 1, characterized in that The method further comprises: Determining whether the first instruction stream and the second instruction stream are instruction streams corresponding to a preset image rendering task; The storing of initial parameters corresponding to the first instruction before executing the first instruction includes: If the first instruction stream and the second instruction stream are instruction streams corresponding to the preset image rendering task, before executing the first instruction, initial parameters corresponding to the first instruction are stored.

3. The method according to claim 1 or 2, characterized in that: The electronic device comprises an instruction flow management module, and the instruction flow management module stores the initial parameters.

4. The method according to claim 3, characterized in that The instruction stream management module is further used to determine whether the first instruction stream and the second instruction stream are instruction streams corresponding to a preset image rendering task.

5. The method according to claim 3 or 4, characterized in that: The instruction flow management module also includes a first submodule, and the first submodule is used to store the initial parameters before executing the first instruction.

6. The method according to claim 5, characterized in that The instruction flow management module also includes a second submodule, and the second submodule is used to store a second parameter, where the second parameter is a parameter sent to the GPU corresponding to the first instruction.

7. The method according to claim 5, characterized in that The instruction flow management module further includes a third submodule, and the third submodule is used to provide an external library interface to the GPU. The method further includes: The GPU calls the parameters in the external library based on the external library interface provided by the third submodule to execute the second instruction.

8. The method according to claim 5, characterized in that The instruction flow management module further includes a fourth submodule, and the fourth submodule is used to provide a GPU extension driver interface. The method further includes: The GPU extension driver is driven through the GPU extension driver interface to drive the GPU to execute a third instruction.

9. An electronic device, characterized in that: The electronic device comprises means for performing the method as claimed in any one of claims 1 to 8.

10. An electronic device, characterized in that: include: one or more processors; Memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the method as claimed in any one of claims 1 to 8.

11. A chip system, characterized in that: The chip system includes a processor for calling and running a computer program from a memory, so that an electronic device equipped with the chip system executes the method as described in any one of claims 1 to 8.

12. A computer-readable storage medium comprising a computer program, characterized in that: When the computer program is executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 8.

Images