Image rendering method and device, terminal equipment, storage medium and computer program product

Abstract

The invention relates to an image rendering method and device, terminal equipment, a storage medium and a computer program product. The image rendering method comprises the steps of generating a query request in response to an image rendering instruction for a to-be-rendered image frame; the query request carries identification information associated with a to-be-rendered image frame; based on the identification information, determining a target rendering pipeline for the to-be-rendered image frame; and performing image rendering on the to-be-rendered image frame based on the target rendering pipeline. According to the embodiment of the invention, not only can the flexibility of image rendering of the to-be-rendered image frame be improved, but also the rendering efficiency of the to-be-rendered image frame can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of communications, and in particular to an image rendering method and apparatus, a terminal device, a storage medium, and a computer program product. Background Art

[0002] Currently, existing terminal devices such as Android systems of mobile phones use the Open Graphics Library for Embedded Systems (OpenGL ES) rendering pipeline by default to render images for mobile phone applications.

[0003] However, if the rendering pipeline of the graphics application programming interface (Vulkan) is used, all applications on the mobile phone need to switch to the Vulkan rendering pipeline. This switching method will affect the display of all applications in the mobile phone, and it is impossible to dynamically control the rendering pipeline of a certain application in the mobile phone to switch, resulting in poor flexibility. Summary of the Invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an image rendering method and apparatus, a terminal device, a storage medium and a computer program product, which can not only improve the flexibility of image rendering for image frames to be rendered, but also improve the rendering efficiency for image frames to be rendered.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image rendering method, comprising at least:

[0006] In response to an image rendering instruction for an image frame to be rendered, generating a query request; the query request carries identification information associated with the image frame to be rendered;

[0007] Determining a target rendering pipeline for the image frame to be rendered based on the identification information;

[0008] Based on the target rendering pipeline, image rendering is performed on the image frame to be rendered.

[0009] In some embodiments, determining a target rendering pipeline for the image frame to be rendered based on the identification information includes:

[0010] Based on the identification information, query the rendering pipeline corresponding to the image frame to be rendered to obtain a query result;

[0011] If the query result satisfies a preset condition, switching the current rendering pipeline of the image frame to be rendered;

[0012] The switched rendering pipeline is used as the target rendering pipeline.

[0013] In some embodiments, when the query result satisfies a preset condition, switching the current rendering pipeline of the image frame to be rendered includes:

[0014] When the query result indicates that the rendering pipeline corresponding to the to-be-rendered image frame is different from the current rendering pipeline of the target application, the current rendering pipeline is switched to a rendering pipeline different from the current rendering pipeline.

[0015] In some embodiments, the current rendering pipeline includes a first preset rendering pipeline, and the rendering pipeline different from the current rendering pipeline includes a second preset rendering pipeline; and switching the current rendering pipeline to a rendering pipeline different from the current rendering pipeline includes:

[0016] The pipeline information corresponding to the first preset rendering pipeline in the pipeline configuration object of the target application associated with the image frame to be rendered is replaced with the pipeline information corresponding to the second preset rendering pipeline.

[0017] In some embodiments, querying the rendering pipeline corresponding to the image frame to be rendered based on the identification information to obtain the query result includes:

[0018] Determining, based on the identification information, a first keyword of an application package name of the target application program associated with the image frame to be rendered;

[0019] Based on a preset keyword list, query the first keyword to obtain the query result;

[0020] The keyword list includes at least one second keyword.

[0021] In some embodiments, querying the first keyword based on a preset keyword list to obtain the query result includes:

[0022] In a case where the keyword list contains the second keyword that is identical to the first keyword, it is determined that the rendering pipeline corresponding to the to-be-rendered image frame is different from the current rendering pipeline of the target application.

[0023] In some embodiments, the method further comprises:

[0024] In response to a screening instruction of a terminal device, determining a candidate application from designated applications installed on the terminal device;

[0025] The keyword list is formed based on keywords of the application package names of the candidate application programs.

[0026] In some embodiments, querying the first keyword based on a preset keyword list to obtain the query result includes:

[0027] In response to an update instruction for the keyword list, updating the keyword list;

[0028] Based on the updated keyword list, the first keyword is queried to obtain the query result.

[0029] According to a second aspect of an embodiment of the present disclosure, there is provided an image rendering apparatus, comprising at least:

[0030] A generating module configured to generate a query request in response to an image rendering instruction for an image frame to be rendered; the query request carries identification information associated with the image frame to be rendered;

[0031] a determination module configured to determine a target rendering pipeline for the image frame to be rendered based on the identification information;

[0032] The image rendering module is configured to perform image rendering on the image frame to be rendered based on the target rendering pipeline.

[0033] According to a third aspect of an embodiment of the present disclosure, a terminal device is provided, comprising at least:

[0034] processor;

[0035] memory for storing computer programs or instructions;

[0036] The processor executes the computer program or instructions to implement the steps of the image rendering method described in the first aspect.

[0037] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided, which stores a computer program or instructions. When the computer program or instructions in the storage medium are executed by a processor, the steps of the image rendering method described in the first aspect are implemented.

[0038] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implements the steps of the image rendering method described in the first aspect.

[0039] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0040] An image rendering method provided by an embodiment of the present disclosure generates a query request in response to an image rendering instruction for an image frame to be rendered; the query request carries identification information associated with the image frame to be rendered; based on the identification information, a target rendering pipeline for the image frame to be rendered is determined; and based on the target rendering pipeline, image rendering is performed on the image frame to be rendered.

[0041] That is to say, unlike the related art that cannot switch the rendering pipeline for a certain image frame of the application, the embodiment of the present disclosure can determine the target rendering pipeline for the image frame to be rendered through the identification information associated with the image frame to be rendered, so that the target rendering pipeline that better matches the image frame to be rendered can be used to render the image of the image frame to be rendered, thereby not only improving the flexibility of image rendering for the image frame to be rendered, but also improving the rendering efficiency of the image frame to be rendered; and, by generating a query request for each image frame to be rendered, the corresponding target rendering pipeline can be used in real time when rendering the next image frame.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0044] Figure 1 This is a flow diagram of an image rendering method according to an exemplary embodiment. Figure 1 .

[0045] Figure 2 is a schematic diagram showing a traditional image rendering method according to an exemplary embodiment.

[0046] Figure 3 This is a flow diagram of an image rendering method according to an exemplary embodiment. Figure 2 .

[0047] Figure 4 This is a flow diagram of an image rendering method according to an exemplary embodiment. Figure 3 .

[0048] Figure 5 The figure is a schematic diagram showing an image rendering method according to an exemplary embodiment.

[0049] Figure 6 The figure is a structural block diagram of an image rendering device according to an exemplary embodiment.

[0050] Figure 7The figure is a schematic structural diagram of a terminal device according to an exemplary embodiment. DETAILED DESCRIPTION

[0051] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0052] The technical solutions provided by various embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0053] Figure 1 This is a flow diagram of an image rendering method according to an exemplary embodiment. Figure 1 ,like Figure 1 As shown, the image rendering method provided by the embodiment of the present disclosure includes at least the following steps:

[0054] Step 110: generating a query request in response to an image rendering instruction for an image frame to be rendered; the query request carries identification information associated with the image frame to be rendered;

[0055] Step 120: Determine a target rendering pipeline for the image frame to be rendered based on the identification information;

[0056] Step 130: Perform image rendering on the image frame to be rendered based on the target rendering pipeline.

[0057] In the embodiments of the present disclosure, the above-mentioned image rendering method can be applied to terminal devices, which may include: mobile phones, tablet computers, smart watches, digital cameras, head-mounted displays (HMDs), etc., and the embodiments of the present disclosure do not impose any restrictions.

[0058] In step 110 , the image frame to be rendered may be an image frame that needs to be displayed in the terminal device; for example, the image frame to be rendered may be an image frame that is to be displayed by the terminal device at the next moment.

[0059] The image rendering instruction may be an instruction for a target application associated with the image frame to be rendered to request the image frame to be rendered to be rendered, wherein the target application may be any application currently running in the terminal device.

[0060] It is understandable that after the target application is started, the terminal device needs to display the screen corresponding to the target application. At this time, the application process corresponding to the target application can generate the image rendering instruction.

[0061] In the disclosed embodiment, the query request may be a query request sent by an application process corresponding to a target application to a central processing unit (CPU) of a terminal device. The query request may carry identification information of the target application associated with the image frame to be rendered, thereby querying the identification information of the target application.

[0062] Here, the identification information associated with the image frame to be rendered may include, but is not limited to: the application package name of the target application, keywords of the application package name of the target application, or the application type of the target application.

[0063] In step 120 , the target rendering pipeline may be a rendering pipeline used by the target application to render the image frame to be rendered; for example, the target rendering pipeline may include but is not limited to: an OpenGL ES rendering pipeline or a Vulkan rendering pipeline.

[0064] The Open Graphics Library (OpenGL) is a cross-platform graphics application programming interface (API) that specifies a standard software interface for 3D graphics processing hardware. OpenGL ES is a form of the OpenGL specification suitable for embedded devices. Android devices can use OpenGL ES to render 2D and 3D graphics.

[0065] Vulkan is a cross-platform API for rendering 2D and 3D graphics. It's been called the "next version of OpenGL," and aims to provide lower CPU overhead and greater control over the Graphics Processing Unit (GPU) than OpenGL. Vulkan offers improved performance compared to OpenGL, and using the Vulkan rendering pipeline offers several advantages, such as reducing the time required to render each frame and improving smoothness.

[0066] Here, the rendering pipeline, also known as the rendering pipeline, pixel pipeline, or pixel pipeline, is a series of independent, parallel processing units within the display chip that process image signals. The rendering pipeline converts a 3D scene model into screen pixel space for output. The image rendering process within the rendering pipeline can be abstracted into three stages: the application stage, which outputs rendering primitives; the geometry stage, which outputs vertex information in screen space; and the rasterization stage.

[0067] It is understandable that the terminal device can perform a query based on the identification information of the target application to determine the target rendering pipeline for the image frame to be rendered.

[0068] It should be noted that the target rendering pipeline can be the same as or different from the rendering pipeline used by the target application at the previous moment, and this is not limited in the embodiments of the present disclosure. For example, when the rendering pipeline used by the target application at the previous moment is the OpenGL ES rendering pipeline, the target rendering pipeline can be the OpenGL ES rendering pipeline or the Vulkan rendering pipeline; or, when the rendering pipeline used by the target application at the previous moment is the Vulkan rendering pipeline, the target rendering pipeline can also be the OpenGL ES rendering pipeline or the Vulkan rendering pipeline, etc.

[0069] In step 130, after determining the target rendering pipeline for the image frame to be rendered, the terminal device can use the target rendering pipeline to render the image frame to be rendered, so as to display the image frame rendered by the target application in the terminal device.

[0070] For example, Figure 2 As shown, in the traditional image rendering method, the Android system of terminal devices such as mobile phones uses the OpenGL ES rendering pipeline for image rendering by default. Specifically, the system can decide to create the use type SkiaOpenGLPipeline or SkiaVulkanPipeline according to the return value of the Properties::peekRenderPipelineType function, which corresponds to the OpenGL ES or Vulkan rendering pipeline respectively. In this function, the system property "debug.hwui.renderer" will be read. The value of this property setting may include: null value or skiagl, which corresponds to the use of OpenGL ES rendering pipeline; or skiavk, which corresponds to the use of Vulkan rendering pipeline. The return value of this function defaults to RenderPipelineType::SkiaGL, and the display process of all applications will call this function, that is, the Android system uses the OpenGL rendering pipeline by default to render images for applications.

[0071] In the related art, if an existing mobile phone uses the Vulkan rendering pipeline for image rendering, it is necessary to modify the property value of debug.hwui.renderer to skiavk by issuing a command (such as adb shell setprop). Then, the application process corresponding to each application can read the property value, so that the return value of the Properties::SkiaOpenGLPipeline function is the RenderPipelineType::SkiaVulkan type, that is, it is modified to use the Vulkan rendering pipeline to render images for the application. However, this method of switching the rendering pipeline requires restarting the application after executing the command in order to take effect on the application. All applications belonging to the mobile phone are switched to the Vulkan rendering pipeline. This switching method will affect the display of all applications in the mobile phone, and cannot dynamically control the rendering pipeline of an application in the mobile phone to switch, resulting in poor flexibility.

[0072] Based on this, an image rendering method provided by an embodiment of the present disclosure generates a query request in response to an image rendering instruction for an image frame to be rendered; the query request carries identification information of a target application associated with the image frame to be rendered; based on the identification information, a target rendering pipeline for the image frame to be rendered is determined; and based on the target rendering pipeline, image rendering is performed on the image frame to be rendered. That is, unlike the related art in which a rendering pipeline cannot be switched for a certain image frame of an application, the embodiment of the present disclosure can determine the target rendering pipeline for the image frame to be rendered by the identification information associated with the image frame to be rendered, so that the image rendering of the image frame to be rendered can be performed using a target rendering pipeline that better matches the image frame to be rendered, thereby not only improving the flexibility of image rendering for the image frame to be rendered, but also improving the rendering efficiency for the image frame to be rendered; and, by generating a query request for each image frame to be rendered, the corresponding target rendering pipeline can be used in real time when rendering the next image frame.

[0073] In one embodiment of the present disclosure, step 120 may be implemented in various ways.

[0074] A specific implementation example is given below. It should be noted that the following is only an example and is not intended to be limiting.

[0075] Figure 3 This is a flow diagram of an image rendering method according to an exemplary embodiment. Figure 2 See also Figure 3 , step 120 in the image rendering method provided by the embodiment of the present disclosure may include the following steps:

[0076] Step 121: Based on the identification information, query the rendering pipeline corresponding to the image frame to be rendered to obtain a query result;

[0077] Step 122: If the query result satisfies a preset condition, the current rendering pipeline for the image frame to be rendered is switched;

[0078] Step 123: Use the switched rendering pipeline as the target rendering pipeline.

[0079] In step 121 , the query result may be used to indicate that the rendering pipeline corresponding to the image frame to be rendered needs to be switched, or may be used to indicate that the rendering pipeline corresponding to the image frame to be rendered does not need to be switched.

[0080] In an embodiment of the present disclosure, the terminal device may query the rendering pipeline corresponding to the image frame to be rendered according to the identification information of the target application associated with the image frame to be rendered in a pre-stored identification list to obtain a query result.

[0081] It should be noted that the query on the rendering pipeline corresponding to the image frame to be rendered based on the identification information may include but is not limited to: when the identification information is the application package name of the target application, the rendering pipeline corresponding to the image frame to be rendered can be queried according to the preset application package name list; or, when the identification information is the keyword of the application package name of the target application, the rendering pipeline corresponding to the image frame to be rendered can be queried according to the preset keyword list; or, when the identification information is the application type of the target application, the rendering pipeline corresponding to the image frame to be rendered can be queried according to the preset application type list.

[0082] In step 122, the query result meeting the preset condition can be understood as indicating that the rendering pipeline corresponding to the image frame to be rendered is different from the rendering pipeline used by the target application at the current moment, that is, the rendering pipeline corresponding to the image frame to be rendered needs to be switched.

[0083] Here, if the query result does not meet the preset conditions, the terminal device can directly use the rendering pipeline currently used by the target application as the target rendering pipeline. In other words, the query result can indicate that the rendering pipeline corresponding to the image frame to be rendered is the same as the rendering pipeline currently used by the target application, that is, the rendering pipeline corresponding to the image frame to be rendered does not need to be switched.

[0084] For example, when the rendering pipeline used by the target application at the current moment is the OpenGL ES rendering pipeline, if the rendering pipeline corresponding to the image frame to be rendered queried based on the identification information is the Vulkan rendering pipeline, then the query result can meet the preset conditions; if the rendering pipeline corresponding to the image frame to be rendered queried based on the identification information is the OpenGL ES rendering pipeline, then the query result does not meet the preset conditions. Similarly, when the rendering pipeline used by the target application at the current moment is the Vulkan rendering pipeline, if the rendering pipeline corresponding to the image frame to be rendered queried based on the identification information is the OpenGLES rendering pipeline, then the query result can meet the preset conditions; if the rendering pipeline corresponding to the image frame to be rendered queried based on the identification information is the Vulkan rendering pipeline, then the query result does not meet the preset conditions.

[0085] In step 123, the terminal device can perform image rendering on the image frame to be rendered according to the switched rendering pipeline, ie, the target rendering pipeline, so that the target application can use the target rendering pipeline that better matches the image frame to be rendered for image rendering.

[0086] In an embodiment of the present disclosure, the terminal device can query the rendering pipeline corresponding to the image frame to be rendered through the identification information of the target application, obtain the query result, and when the query result meets the preset conditions, switch the current rendering pipeline of the target application, so that the target application can use the switched rendering pipeline to render the image frame to be rendered, thereby ensuring the smoothness of the target application while improving the rendering efficiency of the image frame to be rendered in the target application.

[0087] In some embodiments, step 122 may include:

[0088] When the query result indicates that the rendering pipeline corresponding to the image frame to be rendered is different from the current rendering pipeline, the current rendering pipeline is switched to a rendering pipeline different from the current rendering pipeline.

[0089] In an embodiment of the present disclosure, the query result indicates that the rendering pipeline corresponding to the image frame to be rendered is different from the current rendering pipeline of the target application, indicating that pipeline switching needs to be performed according to the current rendering pipeline of the target application, that is, the current rendering pipeline of the target application needs to be switched to a rendering pipeline different from the current rendering pipeline of the target application.

[0090] Here, the current rendering pipeline of the target application is the rendering pipeline currently used by the target application.

[0091] It should be noted that switching the current rendering pipeline to a rendering pipeline different from the current rendering pipeline may include: if the current rendering pipeline of the target application is the second preset rendering pipeline, switching the current rendering pipeline of the target application from the second preset rendering pipeline to the first preset rendering pipeline; if the current rendering pipeline of the target application is the first preset rendering pipeline, switching the current rendering pipeline of the target application from the first preset rendering pipeline to the second preset rendering pipeline.

[0092] In some embodiments, the image rendering method further includes:

[0093] If the query result indicates that the rendering pipeline corresponding to the image frame to be rendered is the same as the current rendering pipeline, the current rendering pipeline is used as the target rendering pipeline. In other words, the embodiment of the present disclosure can determine whether the current rendering pipeline of the target application needs to be switched based on different query results.

[0094] In an embodiment of the present disclosure, the terminal device can switch the current rendering pipeline of the target application to a rendering pipeline different from the current rendering pipeline based on the query results, so that the target application can use the switched rendering pipeline to render the image frames to be rendered, thereby ensuring the smoothness of the target application while improving the rendering efficiency of the image frames to be rendered in the target application.

[0095] In one embodiment of the present disclosure, the current rendering pipeline includes a first preset rendering pipeline, and the rendering pipeline different from the current rendering pipeline includes a second preset rendering pipeline; and switching the current rendering pipeline to the rendering pipeline different from the current rendering pipeline includes:

[0096] The pipeline information corresponding to the first preset rendering pipeline in the pipeline configuration object of the target application associated with the image frame to be rendered is replaced with the pipeline information corresponding to the second preset rendering pipeline.

[0097] In the embodiment of the present disclosure, the first preset rendering pipeline and the second preset rendering pipeline may be two different rendering pipelines after the target application is switched.

[0098] For example, if the first preset rendering pipeline is the Vulkan rendering pipeline, the second preset rendering pipeline is the OpenGL ES rendering pipeline; that is, if the current rendering pipeline is the Vulkan rendering pipeline, the rendering pipeline different from the current rendering pipeline can be the OpenGL ES rendering pipeline. If the first preset rendering pipeline is the OpenGL ES rendering pipeline, the second preset rendering pipeline is the Vulkan rendering pipeline; that is, if the current rendering pipeline is the OpenGL ES rendering pipeline, the rendering pipeline different from the current rendering pipeline can be the Vulkan rendering pipeline.

[0099] In the embodiment of the present disclosure, the pipeline configuration object of the target application may be an object storing the running properties and configuration information of the current rendering pipeline of the target application; for example, the pipeline configuration object may be a rendering pipeline context.

[0100] The pipeline information corresponding to the first preset rendering pipeline may include the operating properties and configuration information corresponding to the first preset rendering pipeline, etc.; the pipeline information corresponding to the second preset rendering pipeline may include the operating properties and configuration information corresponding to the second preset rendering pipeline, etc.

[0101] It can be understood that each application in the terminal device can create two pipeline configuration objects when it is started for the first time, namely a first pipeline configuration object and a second pipeline configuration object; the first pipeline configuration object may include the pipeline information corresponding to the image rendering of the application, and the second pipeline configuration object may include pipeline information corresponding to different rendering pipelines in the first pipeline configuration object.

[0102] Exemplarily, when the rendering pipeline used by the target application at the current moment is the OpenGL ES rendering pipeline, the first pipeline configuration object may include pipeline information corresponding to the OpenGL ES rendering pipeline, and the second pipeline configuration object may include pipeline information corresponding to the Vulkan rendering pipeline; at this time, when the query result indicates that the rendering pipeline corresponding to the image frame to be rendered is the Vulkan rendering pipeline, the terminal device can exchange the pipeline information in the first pipeline configuration object of the target application with the pipeline information in the second pipeline configuration object to complete the switching of the current rendering pipeline of the target application.

[0103] Similarly, when the rendering pipeline used by the target application at the current moment is the Vulkan rendering pipeline, the first pipeline configuration object may include pipeline information corresponding to the Vulkan rendering pipeline, and the second pipeline configuration object may include pipeline information corresponding to the OpenGL ES rendering pipeline; at this time, when the query result indicates that the rendering pipeline corresponding to the image frame to be rendered is the OpenGL ES rendering pipeline, the terminal device may also exchange the pipeline information in the first pipeline configuration object of the target application with the pipeline information in the second pipeline configuration object to complete the switching of the current rendering pipeline of the target application.

[0104] In an embodiment of the present disclosure, the terminal device can replace the pipeline information in the pipeline configuration object of the target application to achieve the switching of the current rendering pipeline of the target application, thereby making it easier to complete the switching of the rendering pipeline of the target application and improving the flexibility of the target application in image rendering.

[0105] In some embodiments, step 121 may include:

[0106] Determining, based on the identification information, a first keyword of an application package name of a target application program associated with the image frame to be rendered;

[0107] Based on a preset keyword list, query the first keyword to obtain a query result;

[0108] The keyword list includes at least one second keyword.

[0109] In the embodiment of the present disclosure, the application package name of the target application may be a character string used to identify the target application; that is, one application package name represents one application, and the application package names of different applications are different.

[0110] The first keyword may be a keyword in the application package name of the target application, that is, a keyword in a character string that identifies the target application. Similarly, the first keywords of different applications are also different.

[0111] Here, the keyword list may be a list of second keywords pre-stored in the terminal device and containing at least one application package name corresponding to an application program that can be switched to the first preset rendering pipeline.

[0112] It is understandable that the terminal device can query whether there is a second keyword that is the same as the first keyword in the preset keyword list to obtain a query result that can indicate the rendering pipeline corresponding to the image frame to be rendered, and then determine whether the current rendering pipeline of the target application needs to be switched.

[0113] In an embodiment of the present disclosure, the terminal device can query the first keyword of the application package name of the target application through a preset keyword list to obtain a query result that can indicate the rendering pipeline corresponding to the image frame to be rendered, so as to better determine whether the current rendering pipeline of the target application needs to be switched, thereby improving the flexibility of the target application in image rendering.

[0114] In some embodiments, in the above image rendering method, querying the first keyword based on a preset keyword list to obtain a query result includes:

[0115] When a second keyword identical to the first keyword is found in the keyword list, it is determined that the rendering pipeline corresponding to the to-be-rendered image frame is different from the current rendering pipeline of the target application.

[0116] In some embodiments, in the above image rendering method, querying the first keyword based on a preset keyword list to obtain a query result includes:

[0117] When it is found that the keyword list does not contain a second keyword identical to the first keyword, it is determined that the rendering pipeline corresponding to the to-be-rendered image frame is identical to the current rendering pipeline of the target application.

[0118] In the disclosed embodiment, since the keyword list pre-stores at least one second keyword corresponding to the application package name of an application that can be switched to the first preset rendering pipeline, when a second keyword identical to the first keyword is found in the keyword list, it can be determined that the first preset rendering pipeline can be used to render the image frame to be rendered in the target application. Similarly, when a second keyword identical to the first keyword is found in the keyword list, it can be determined that the second preset rendering pipeline can be used to render the image frame to be rendered in the target application.

[0119] It should be noted that the specific number of the second keywords in the above keyword list can be set according to actual application conditions, and is not limited in the embodiment of the present disclosure.

[0120] In an embodiment of the present disclosure, the terminal device can determine the query result that can indicate the rendering pipeline corresponding to the image frame to be rendered by querying whether there is a second keyword that is the same as the first keyword in the keyword list, thereby making it easier to determine whether the current rendering pipeline of the target application needs to be switched in the subsequent process.

[0121] In some embodiments, the image rendering method further includes:

[0122] In response to a screening instruction of the terminal device, determining a candidate application from designated applications installed in the terminal device;

[0123] A keyword list is formed based on keywords of the application package names of the candidate applications.

[0124] In the embodiment of the present disclosure, the screening instruction may be an instruction issued by the terminal device for screening candidate applications that can form a keyword list.

[0125] The designated application may be an application that has been installed in the terminal device; the candidate application may be an application in the designated application that can be switched to a rendering pipeline different from the current rendering pipeline.

[0126] It is understandable that the terminal device can determine the candidate application based on the refresh rate of the display screen corresponding to the specified application and the preset refresh rate threshold; or, it can also determine the candidate application based on the frame rate of the display screen corresponding to the specified application and the preset frame rate threshold, etc.

[0127] Here, the terminal device may select a designated application whose refresh rate is greater than or equal to a preset refresh rate threshold as a candidate application; or, may select a designated application whose frame rate is greater than or equal to a preset frame rate threshold as a candidate application.

[0128] It should be noted that the preset refresh rate threshold or the preset frame rate threshold can be set according to actual application conditions, and the embodiments of the present disclosure do not limit this.

[0129] In the embodiment of the present disclosure, the terminal device can determine specified applications with high refresh rate or frame rate requirements as candidate applications through filtering instructions, and form a keyword list based on the keywords of the application package names of the candidate applications, so as to better prepare for the subsequent target application to query the rendering pipeline corresponding to the image frame to be rendered, so as to determine whether the current rendering pipeline of the target application needs to be switched.

[0130] In some embodiments, in the above image rendering method, querying the first keyword based on a preset keyword list to obtain a query result includes:

[0131] In response to an update instruction for the keyword list, updating the keyword list;

[0132] Based on the updated keyword list, a query is performed on the first keyword to obtain a query result.

[0133] Here, the update instruction may be an instruction issued by the terminal device for updating the keyword list.

[0134] In the embodiment of the present disclosure, when the current rendering pipeline of the target application is switched, the terminal device can update the keyword list. When the current rendering pipeline of the target application is not switched, the terminal device can keep the keyword list unchanged.

[0135] For example, when the current rendering pipeline of the target application is switched from the OpenGL ES rendering pipeline to the Vulkan rendering pipeline, the terminal device may delete the second keyword in the keyword list that is identical to the first keyword of the application package name of the target application; or, when the current rendering pipeline of the target application is switched from the Vulkan rendering pipeline to the OpenGL ES rendering pipeline, the terminal device may add the second keyword that is identical to the first keyword of the application package name of the target application to the keyword list, etc.

[0136] In the embodiment of the present disclosure, the terminal device can update the keyword list through an update instruction so that the query results obtained when the target application queries the rendering pipeline corresponding to the next image frame are more accurate, thereby improving the rendering efficiency of the image frame to be rendered in the target application.

[0137] Figure 4 A schematic diagram of the process of an image rendering method provided in an embodiment of the present disclosure Figure 3 .like Figure 4 As shown, the image rendering method provided in the embodiment of the present disclosure is only an example and not a limitation, and is intended to help those skilled in the art better understand the technical solution of the present disclosure. Figure 4 The image rendering method provided by the embodiment of the present disclosure can be applied to a terminal device and may specifically include the following steps:

[0138] Step 401: Generate a query request in response to an image rendering instruction for an image frame to be rendered; the query request carries identification information associated with the image frame to be rendered.

[0139] Step 402: Determine a first keyword of an application package name of a target application program associated with the image frame to be rendered based on the identification information.

[0140] Step 403 , query whether there is a second keyword that is the same as the first keyword in the keyword list; if so, execute step 404 ; if not, execute step 410 .

[0141] Step 404 , determine whether the current rendering pipeline of the target application is the second preset rendering pipeline; if so, execute step 405 ; if not, execute step 407 .

[0142] Step 405: Switch the current rendering pipeline of the target application from the second preset rendering pipeline to the first preset rendering pipeline.

[0143] For example, the first preset rendering pipeline may be a Vulkan rendering pipeline, and the second preset rendering pipeline may be an OpenGL ES rendering pipeline.

[0144] Step 406: Perform image rendering on the image frame to be rendered based on the first preset rendering pipeline.

[0145] Step 407 , determine whether the current rendering pipeline of the target application is the first preset rendering pipeline; if so, execute step 408 ; if not, end.

[0146] Step 408: Switch the current rendering pipeline of the target application from the first preset rendering pipeline to the second preset rendering pipeline.

[0147] Step 409: Perform image rendering on the image frame to be rendered based on the second preset rendering pipeline.

[0148] Step 410: Based on the current rendering pipeline of the target application, perform image rendering on the image frame to be rendered.

[0149] For example, in one embodiment of the present disclosure, when a target application of a terminal device such as a mobile phone is started for the first time, Figure 5 The Properties values ​​shown create two pipeline configuration objects simultaneously, for example, a first render pipeline context (e.g., the mContext member) and a second render pipeline context (e.g., the mSecondaryContext member). The first render pipeline context can correspond to the OpenGL ES render pipeline, while the second render pipeline context can correspond to the Vulkan render pipeline. To switch the target application's current render pipeline, call the switchContext interface, swapping the mContext and mSecondaryContext members of the two render pipeline contexts. This means assigning the original mSecondaryContext to the mContext member, and the original mContext to the mSecondaryContext member.

[0150] In the embodiment of the present disclosure, Figure 5 As shown, the mobile phone can trigger the corresponding setSwitchFlag method in the Window Manager Service (WMS) by issuing an adb shell command. This means that the adb shell command can be used to dynamically switch the current rendering pipeline of the target application. For example, the adb shell command can be adb shellwm. In other words, the mobile phone can use the adb shell command to trigger the application process of the target application to generate image rendering instructions for the image frame to be rendered, thereby dynamically switching the current rendering pipeline of the target application in subsequent processes.

[0151] In the disclosed embodiment, a mobile phone can add a keyword list to the WMS service to store keywords for application package names. A checkSwitch interface can be defined in the WMS service. Upon receiving a query request from a target application's application process, the WMS service can check whether the keywords for the application package name corresponding to the identification information carried in the query request are in the keyword list. The query result, such as a Boolean value (True / False), can be returned to indicate whether the target application's current rendering pipeline needs to be switched.

[0152] In an embodiment of the present disclosure, when the target application draws the next image frame, such as the image frame to be rendered, the application process of the target application can call the checkSwitch interface of the WMS service to query whether the current rendering pipeline of the target application needs to be switched. If the query result returned is true (True), the switchContext interface can be called to exchange the members of the first rendering pipeline context and the second rendering pipeline context to complete the switching of the current rendering pipeline of the target application; if the query result returned is false (False), there is no need to switch the current rendering pipeline of the target application. In addition, each time the switching of the current rendering pipeline of the target application is completed, the keyword list needs to be updated so that the query result obtained when the target application queries the rendering pipeline corresponding to the next image frame is more accurate.

[0153] An image rendering method provided by an embodiment of the present disclosure can determine the target rendering pipeline for the image frame to be rendered through the identification information of the target application, so that the target application can use the target rendering pipeline that better matches the image frame to be rendered for image rendering, thereby not only improving the flexibility of the target application in image rendering, but also improving the rendering efficiency of the image frame to be rendered in the target application while ensuring the smoothness of the target application; and, by generating a query request for each image frame to be rendered by the target application, the target application can use the corresponding target rendering pipeline when rendering the next image frame without restarting.

[0154] Figure 6 This is a structural diagram of an image rendering device provided by an embodiment of the present disclosure. Figure 6 The image rendering apparatus 600 provided in an embodiment of the present disclosure is applied to a terminal device and may include: a generating module 610 , a determining module 620 and an image rendering module 630 .

[0155] The generating module 610 is configured to generate a query request in response to an image rendering instruction for an image frame to be rendered; the query request carries identification information associated with the image frame to be rendered;

[0156] A determination module 620 is configured to determine a target rendering pipeline for the image frame to be rendered based on the identification information;

[0157] The image rendering module 630 is configured to perform image rendering on the image frame to be rendered based on the target rendering pipeline.

[0158] The image rendering device provided by the embodiment of the present disclosure generates a query request in response to an image rendering instruction for an image frame to be rendered; the query request carries identification information associated with the image frame to be rendered; based on the identification information, a target rendering pipeline for the image frame to be rendered is determined; and based on the target rendering pipeline, image rendering is performed on the image frame to be rendered. That is, unlike the related art in which a rendering pipeline cannot be switched for a certain image frame of an application, the embodiment of the present disclosure can determine the target rendering pipeline for the image frame to be rendered by the identification information associated with the image frame to be rendered, so that the image rendering of the image frame to be rendered can be performed using a target rendering pipeline that better matches the image frame to be rendered, thereby not only improving the flexibility of image rendering for the image frame to be rendered, but also improving the rendering efficiency for the image frame to be rendered; and, by generating a query request for each image frame to be rendered, the corresponding target rendering pipeline can be used in real time when rendering the next image frame.

[0159] for Figure 6 In the technical solution shown, in one possible implementation, the determination module 620 can be specifically configured as follows: based on the identification information, query the rendering pipeline corresponding to the image frame to be rendered to obtain the query result; when the query result meets the preset conditions, switch the current rendering pipeline of the image frame to be rendered; and use the switched rendering pipeline as the target rendering pipeline.

[0160] for Figure 6 In the technical solution shown, in one possible implementation, the determination module 620 can also be configured to: when the query result indicates that the rendering pipeline corresponding to the image frame to be rendered is different from the current rendering pipeline, switch the current rendering pipeline to a rendering pipeline different from the current rendering pipeline.

[0161] for Figure 6 In the technical solution shown, in one possible implementation, the current rendering pipeline includes a first preset rendering pipeline, and the rendering pipeline different from the current rendering pipeline includes a second preset rendering pipeline; the determination module 620 can also be configured to: replace the pipeline information corresponding to the first preset rendering pipeline in the pipeline configuration object of the target application associated with the image frame to be rendered with the pipeline information corresponding to the second preset rendering pipeline.

[0162] for Figure 6 In the technical solution shown, in one possible implementation, the determination module 620 can also be configured to: determine a first keyword of the application package name of the target application associated with the image frame to be rendered based on the identification information; query the first keyword based on a preset keyword list to obtain a query result; wherein the keyword list includes at least one second keyword.

[0163] for Figure 6In the technical solution shown, in one possible implementation, the determination module 620 can also be configured to: when a second keyword that is the same as the first keyword is found in the keyword list, determine that the rendering pipeline corresponding to the image frame to be rendered is different from the current rendering pipeline of the target application.

[0164] for Figure 6 In a possible implementation of the technical solution shown, the determination module 620 can also be configured to: determine candidate applications from specified applications installed on the terminal device in response to a screening instruction of the terminal device; and form a keyword list based on keywords of the application package names of the candidate applications.

[0165] for Figure 6 In a possible implementation of the technical solution shown, the determination module 620 may also be configured to: update the keyword list in response to an update instruction for the keyword list; and query the first keyword based on the updated keyword list to obtain a query result.

[0166] It should be noted that the image rendering device provided by the embodiment of the present disclosure corresponds to the image rendering method mentioned above. For related content, please refer to the description of the image rendering method above, and no further details will be given here.

[0167] Figure 7 7 is a schematic diagram illustrating the structure of a terminal device according to an exemplary embodiment. For example, the terminal device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0168] Reference Figure 7 The terminal device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0169] The processing component 702 generally controls the overall operation of the terminal device 700, such as operations associated with at least one of display, phone calls, data communications, camera operation, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-described method. In addition, the processing component 702 may include one or more modules to facilitate interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate interaction between the multimedia component 708 and the processing component 702.

[0170] The memory 704 is configured to store various types of data to support operations on the terminal device 700. Examples of such data include at least one of the following: instructions for any application or method operating on the terminal device 700, contact data, phone book data, messages, pictures, and videos. The memory 704 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0171] The power supply component 706 provides power to various components of the terminal device 700. The power supply component 706 may include at least one of the following: a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 700.

[0172] The multimedia component 708 includes a screen that provides an output interface between the terminal device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the terminal device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0173] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), which is configured to receive external audio signals when the terminal device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0174] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as a keyboard, click wheel, and buttons. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.

[0175] The sensor assembly 714 includes one or more sensors for providing various aspects of the terminal device 700's status assessment. For example, the sensor assembly 714 can detect the open / closed state of the terminal device 700, the relative positioning of components, such as the display and keypad of the terminal device 700. The sensor assembly 714 can also detect changes in the position of the terminal device 700 or a component thereof, the presence or absence of user contact with the terminal device 700, the orientation or acceleration / deceleration of the terminal device 700, and changes in the temperature of the terminal device 700. The sensor assembly 714 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 can also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 can also include, but is not limited to, at least one of the following: an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, and a temperature sensor.

[0176] The communication component 716 is configured to facilitate wired or wireless communication between the terminal device 700 and other devices. The terminal device 700 can access a wireless network based on a communication standard, such as Wi-Fi, 4G, 5G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0177] In an exemplary embodiment, the terminal device 700 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0178] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including executable instructions or a computer program, which can be executed by a processor of the charger to perform the charging method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0179] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a terminal device, enables the terminal device to perform any of the image rendering methods described in the above embodiments. For example, the image rendering method includes: generating a query request in response to an image rendering instruction for an image frame to be rendered; the query request carries identification information of a target application associated with the image frame to be rendered; determining a target rendering pipeline for the image frame to be rendered based on the identification information; and performing image rendering on the image frame to be rendered based on the target rendering pipeline.

[0180] The present disclosure provides a computer program product comprising a computer program or executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or executable instructions from the computer-readable storage medium and executes the computer program or executable instructions, causing the computer device to perform any of the image rendering methods described in the above embodiments.

[0181] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

[0182] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image rendering method, characterized in that: include: generating a query request in response to an image rendering instruction for an image frame to be rendered; The query request carries identification information associated with the image frame to be rendered; Determining a target rendering pipeline for the image frame to be rendered based on the identification information; Based on the target rendering pipeline, image rendering is performed on the image frame to be rendered.

2. The method according to claim 1, characterized in that The determining, based on the identification information, a target rendering pipeline for the image frame to be rendered includes: Based on the identification information, query the rendering pipeline corresponding to the image frame to be rendered to obtain a query result; If the query result satisfies a preset condition, switching the current rendering pipeline of the image frame to be rendered; The switched rendering pipeline is used as the target rendering pipeline.

3. The method according to claim 2, characterized in that When the query result satisfies a preset condition, switching the current rendering pipeline of the image frame to be rendered includes: When the query result indicates that the rendering pipeline corresponding to the to-be-rendered image frame is different from the current rendering pipeline, the current rendering pipeline is switched to a rendering pipeline different from the current rendering pipeline.

4. The method according to claim 3, characterized in that The current rendering pipeline includes a first preset rendering pipeline, and the rendering pipeline different from the current rendering pipeline includes a second preset rendering pipeline; The switching the current rendering pipeline to a rendering pipeline different from the current rendering pipeline includes: The pipeline information corresponding to the first preset rendering pipeline in the pipeline configuration object of the target application associated with the image frame to be rendered is replaced with the pipeline information corresponding to the second preset rendering pipeline.

5. The method according to claim 2, characterized in that The querying of the rendering pipeline corresponding to the image frame to be rendered based on the identification information to obtain a query result includes: Determining, based on the identification information, a first keyword of an application package name of a target application program associated with the image frame to be rendered; Based on a preset keyword list, query the first keyword to obtain the query result; The keyword list includes at least one second keyword.

6. The method according to claim 5, characterized in that The querying of the first keyword based on the preset keyword list to obtain the query result includes: In a case where the keyword list contains the second keyword that is identical to the first keyword, it is determined that the rendering pipeline corresponding to the to-be-rendered image frame is different from the current rendering pipeline of the target application.

7. The method according to claim 5, characterized in that The method further comprises: In response to a screening instruction of a terminal device, determining a candidate application from designated applications installed on the terminal device; The keyword list is formed based on keywords of the application package names of the candidate application programs.

8. The method according to claim 5, characterized in that The querying of the first keyword based on the preset keyword list to obtain the query result includes: In response to an update instruction for the keyword list, updating the keyword list; Based on the updated keyword list, the first keyword is queried to obtain the query result.

9. An image rendering device, characterized in that: include: a generating module configured to generate a query request in response to an image rendering instruction for an image frame to be rendered; The query request carries identification information associated with the image frame to be rendered; a determination module configured to determine a target rendering pipeline for the image frame to be rendered based on the identification information; The image rendering module is configured to perform image rendering on the image frame to be rendered based on the target rendering pipeline.

10. A terminal device, characterized in that: include: processor; memory for storing computer programs or instructions; The processor executes the computer program or instructions to implement the steps of the image rendering method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium storing a computer program or instruction, characterized in that: When the computer program or instructions in the storage medium are executed by a processor, the steps of the image rendering method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the image rendering method according to any one of claims 1 to 8 are implemented.

Images