Application program control method and device and computer readable storage medium

Abstract

The embodiment of the invention provides an application program control method, electronic equipment and a computer readable storage medium, and the method comprises the steps: recording the writing times of a Vulkan module for writing data into a command cache region of the Vulkan module through a counter within the rendering time of the current frame data of a target application program, obtaining the total writing frequency of the Vulkan module into the command cache region in the rendering time of the current frame data; determining the current rendering pressure of the target application program according to the total writing times; and controlling at least one of performance resources and display resolution of the target application program according to the current rendering pressure. In this way, the performance of the target application program developed by using Vulkan can be improved.

Description

Technical Field

[0001] This document relates to the field of application interface rendering technology, and in particular to a control method, device and computer-readable storage medium for an application. Background Art

[0002] With the advancement of hardware technology and the increasing complexity of application interfaces (such as game applications), the traditional graphics application programming interface (Application Programming Interface, API) can no longer meet the needs of developers. Therefore, a more low-level, more efficient and more scalable graphics API is needed to support the display requirements of application interfaces. Against this background, Vulkan came into being.

[0003] The Vulkan graphics API has been widely adopted by game developers because of its asynchronous command submission model. This asynchronous command submission model allows game applications to submit multiple commands to the GPU for processing at the same time without waiting for the previous commands to be processed; but this also makes game applications developed based on this API consume more GPU computing power, and the GPU itself is the largest source of heat in the hardware composition of electronic devices. Therefore, when applications developed based on the Vulkan graphics API run on electronic devices, they will have poor performance problems such as poor frame rate and excessive heat.

[0004] At present, related technologies release performance resources by collecting the temperature of the mobile phone, which has low energy efficiency and unreasonable performance release; or, by identifying application threads and setting priorities for different threads to improve some frame rates, this requires high computing power from the CPU itself and the performance improvement is not obvious. In addition, these solutions are not applicable to applications developed using the Vulkan graphics API.

[0005] It can be seen that there is currently a lack of effective performance improvement solutions for applications developed using the Vulkan graphics API. Summary of the invention

[0006] The embodiments of this specification provide an application control method, device, and computer-readable storage medium, which are used to solve the problem that there is currently a lack of effective performance improvement solutions for applications developed using the Vulkan graphics API.

[0007] To solve the above technical problems, the embodiments of this specification are implemented as follows:

[0008] In a first aspect, a method for controlling an application is provided, the method comprising:

[0009] During the rendering time of the current frame data of the target application, the number of times the Vulkan module writes data to the command buffer of the Vulkan module is recorded by a counter, so as to obtain the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data;

[0010] Determine the current rendering pressure of the target application according to the total number of writes;

[0011] At least one of the performance resources and the display resolution of the target application is controlled according to the current rendering pressure.

[0012] In a second aspect, a device for controlling an application is provided, the device comprising:

[0013] a counting module, configured to record, by a counter, the number of times a Vulkan module writes data to a command buffer of the Vulkan module during the rendering time of the current frame data of the target application, and obtain a total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data;

[0014] A load determination module, configured to determine a current rendering pressure of the target application according to the total number of writes;

[0015] A performance control module is used to control at least one of the performance resources and display resolution of the target application according to the current rendering pressure.

[0016] According to a third aspect, an electronic device is provided, including:

[0017] processor;

[0018] a memory for storing instructions executable by the processor;

[0019] The processor is configured to execute the instructions to implement the method as described in the first aspect.

[0020] According to a fourth aspect, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method described in the first aspect.

[0021] In the embodiments of the present specification, since the number of times the Vulkan module writes data to the command buffer of the Vulkan module can be recorded by a counter during the rendering time of the current frame data of the target application, the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data can be obtained, and the current actual rendering pressure (equivalent to the actual load) of the target application can be accurately calculated based on the total number of writes, and then at least one of the performance resources and display resolution of the target application can be dynamically controlled based on the current actual rendering pressure of the target application, thereby improving the performance of the target application developed using Vulkan. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 It is a flowchart of a control method of an application provided by an embodiment of this specification.

[0024] Figure 2 It is a schematic diagram of an implementation architecture of a control method for an application provided by an embodiment of this specification.

[0025] Figure 3 It is a detailed flowchart of a control method of an application provided by an embodiment of this specification.

[0026] Figure 4 It is a structural diagram of a control device of an application provided by an embodiment of this specification.

[0027] Figure 5 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present specification. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0029] The terms "first", "second", etc. in this specification and claims are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of this specification can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in this specification and claims means at least one of the connected objects, and the character " / " generally means that the objects associated with each other are in an "or" relationship.

[0030] In the past, graphics APIs were mainly based on fixed function pipelines, which had certain limitations on the flexibility and performance optimization of graphics rendering. With the advancement of hardware technology and the increasing complexity of graphics application requirements, traditional graphics APIs can no longer meet the needs of developers. Therefore, a more low-level, more efficient, and more scalable graphics API is needed to support the needs of modern graphics applications. Against this background, Vulkan came into being. Vulkan is based on the shortcomings of traditional graphics APIs and the recognition of the needs of modern graphics applications, as well as the need for a more low-level, more efficient, and more scalable graphics API. Vulkan has become one of the important tools for modern graphics application development. Therefore, more and more manufacturers are gradually developing graphics applications based on the Vulkan graphics API. However, mainstream manufacturers currently either release performance resources by collecting the temperature of electronic devices, which has low energy efficiency and unreasonable performance release; or, by identifying application threads and setting priorities for different threads to improve some frame rates, which requires high computing power for the CPU itself and does not significantly improve performance. Moreover, these solutions are not applicable to applications developed using the Vulkan graphics API.

[0031] In order to solve the problem of lack of effective performance improvement solutions for applications developed using Vulkan, a graphics API, this specification proposes a control method, device and computer-readable storage medium for an application. The method can be executed by an electronic device or software installed in an electronic device. The electronic device includes but is not limited to any of: smart phones, personal computers (PCs), laptops, tablets, e-readers, network TVs, wearable devices and other smart devices.

[0032] This specification provides a method for controlling an application, which aims to collect the actual load of the target application at all times, and assign performance resources to the target application and / or adjust the display resolution according to different actual loads. Because, taking the target application as a game application as an example, it is impossible for any scene and operation to have a constant load on the electronic device throughout the game. By identifying the actual load and then releasing performance resources in real time according to the actual load, it is possible to ensure that the performance is continuously optimized while improving the energy efficiency ratio. In addition, according to the actual load identified, the display resolution of each screen is dynamically adjusted to enhance the image quality of the main object in the screen, and maintain or reduce the image quality of the non-main object (edge), which can save some energy consumption while greatly improving the viewing experience and smoothness of the entire game.

[0033] A method for controlling an application program provided by an embodiment of this specification is described in detail below with reference to the accompanying drawings.

[0034] like Figure 1 As shown, an embodiment of this specification provides a method for controlling an application program, which may include:

[0035] Step 101, during the rendering time of the current frame data of the target application, the number of times the Vulkan module writes data to the command buffer of the Vulkan module is recorded by a counter, so as to obtain the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data.

[0036] The target application may be any graphics application. Optionally, the target application may be a graphics application with relatively high display performance requirements, such as game applications, drawing applications, and other applications with high graphics rendering frequency and complex rendering.

[0037] The current frame data of the target application refers to the frame data that the target application currently needs to synthesize and display. Usually, the second interface (such as notifyVKCmdCountFlagNeeded) of the display synthesis system (generally referring to SurfaceFlinger) continuously reads each frame data of the target application, then synthesizes the Framebuffer and saves it to the frame buffer, and finally, the screen of the electronic device reads the Framebuffer from the frame buffer and displays it.

[0038] Optionally, before step 101, the first interface in the display composition system may be used to identify whether the graphics API used by the target application is Vulkan, if so, step 101 is performed, otherwise, the process ends. The first interface may be a newly added interface.

[0039] Specifically, the Vulkan module can call the method SurfaceFlingerBlastBufferQueue in the display synthesis system - SurfaceFlinger, and add an interface (first interface) in this method to monitor the integrity and cycle of the current frame data in real time, and before rendering the current frame data, determine whether the drawing API used by the target application is Vulkan. Among them, the main function of SurfaceFlingerBlastBufferQueue is to process the graphics buffer queue, and its specific function is to add the graphics buffer to the queue, and take it out of the queue and display it on the screen when needed. Generally speaking, when an application or system needs to display a new graphical interface, it will add the corresponding graphics buffer area to the BufferQueue queue, and take it out and display it on the screen when needed.

[0040] The first interface may include but is not limited to notifyVKCmdCountFlagIfNeeded added in the SurfaceFlingerBlastBufferQueue method of SurfaceFlinger.

[0041] The counter is pre-inserted into the Vulkan module. Optionally, before the Vulkan module starts rendering the current frame data, the counter is cleared to ensure that the current actual load can be correctly collected.

[0042] The counter (vk_messager) inserted into the Vulkan module is used to record the number of times a series of commands for performing rendering and computing tasks are submitted in parallel in a multi-threaded environment to the vulkan buffer, and through algorithm filtering and refreshing, each frame of data is accurately and real-time counted. That is, the counter inserted into the Vulkan module is used to record the number of times the Vulkan module writes data to the command buffer of the Vulkan module (vulkan count). The number of writes can be represented by an integer.

[0043] The counter of this graphics API inserted into the Vulkan module implements counting through statistical queries (occlusion queries). The main function logic of the code is used to query the number of times a specific event occurs and count it.

[0044] It can be understood that the total number of writes corresponding to the current frame data can reflect the rendering pressure of the current frame data or the current actual load of the target application. The core of the control method of an application provided in an embodiment of this specification is to use the number of writes to the command buffer of the Vulkan module - the actual number of submissions (vulkancount) to approximate the real-time rendering pressure of the target application. If this value is recorded with a counter, the different rendering pressures at different times during the operation of the target application can be identified. The larger the value at a certain moment, the greater the rendering pressure of the target application at this point in time.

[0045] Since the rendering pressure corresponding to each frame of data is different, the number of threads that perform rendering and computing tasks in the Vulkan graphics API is different, and the sum of the number of times it writes to its own buffer (total number of writes) is also different. Taking the target application as a game application as an example, for the flying scene in the game, the rendering pressure when the flying object just takes off may be relatively low. Under this rendering pressure, there are N1 computing tasks. The Vulkan module needs to write N1 times to its own command buffer (Command Buffer). Correspondingly, the counter counts N1 times; then, as the scene changes, the number of thread tasks in the synthesis process will also change. For this change value, the counter continuously adds 1 to the integer data with a base of N1 to achieve the increase of this count value until the rendering of this frame is completed (dequenbuff writes to quenbuff). For example, assuming the frame rate is 60 frames, it takes 16ms.

[0046] Optionally, after step 101 and before the following step 102, the method may further include: recording the total number of write times in the command buffer area.

[0047] Furthermore, after step 101 and before the following step 102, the method may further include: encrypting the total number of writes recorded in the command buffer by the encryption code in the Vulkan module to ensure the security of the total number of writes. Specifically, the total number of writes may be encrypted by the code (verifySocket) pre-implanted in the Vulkan module. Optionally, the display synthesis system (rendering layer) may query the total number of writes by specifying a switch (vkCmdCount).

[0048] Step 102: Determine the current rendering pressure of the target application according to the total number of write times.

[0049] As an example, step 102 may include: using the total number of write times as the current rendering pressure value of the target application.

[0050] As another example, the total number of writes may be converted into a numerical value and used as the current rendering pressure value of the target application. The numerical conversion may include but is not limited to proportional conversion (such as conversion into a numerical value between 0 and 1), level conversion (such as conversion into a matching level), etc.

[0051] It should be noted that, no matter whether the total number of writes itself or the converted value of the total number of writes is used to represent the current rendering pressure value of the target application, the current rendering pressure value of the target application is always positively correlated with the number of writes.

[0052] Step 103: Control at least one of the performance resources and the display resolution of the target application according to the current rendering pressure.

[0053] It should be noted that, in addition to performance resources and display resolution, other performance indicators of the target application may also be controlled according to the current rendering pressure, and this specification does not impose any restrictions on this.

[0054] As an example, step 103 may include but is not limited to at least one of the following two methods:

[0055] 1) determining a performance resource usage strategy of the target application according to the current rendering pressure value, and controlling the target application to run according to the performance resource usage strategy (or, releasing performance resources to the target application according to the performance resource usage strategy); wherein the performance resources include but are not limited to at least one of a CPU and a GPU.

[0056] 2) Determine a display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data according to the current rendering pressure value, and control the target application to display the screen corresponding to the current frame data according to the display resolution adjustment strategy.

[0057] The above two methods are explained one by one by examples below.

[0058] For the first method, determining the performance resource usage strategy of the target application according to the current rendering pressure value may include: first reporting the current rendering pressure value to the performance central control through the Vulkan module, so that the performance central control determines the performance resource usage strategy of the target application according to the current rendering pressure value. The performance central control refers to a performance control module in an electronic device.

[0059] Specifically, the Vulkan module can package the current rendering pressure value into a what type and then report it to the performance central control through a socket, so that the performance central control determines the performance resource usage strategy of the target application according to the current rendering pressure value.

[0060] Optionally, the Vulkan module may report the current rendering pressure value to the performance central control according to a reporting numerical interval, wherein the report is made when the current rendering pressure value reaches the upper limit of the reporting numerical interval. That is to say, after the current rendering pressure value is obtained, the current rendering pressure value is interval matched, and a suitable reporting interval is set for reporting. For example, 1 to 5 may be used as a limited interval. If the current rendering pressure value falls within this interval, it is reported only when the current rendering pressure value is equal to 5. The next time the current rendering pressure value falls within this interval but is not equal to 5, it will not be reported. This can reduce the number of reports, reduce system overhead, and save energy consumption.

[0061] Furthermore, determining the performance resource usage strategy of the target application according to the current rendering pressure value may include:

[0062] When the current rendering pressure value is greater than or equal to the second threshold, determining the performance resource usage strategy of the target application to be a first resource usage strategy, wherein the first resource usage strategy includes: controlling at least one of the CPU and the GPU to be at a first frequency point;

[0063] When the current rendering pressure value is less than or equal to the first threshold, determining the performance resource usage policy of the target application to be a second resource usage policy, wherein the second resource usage policy includes: controlling at least one of the CPU and the GPU to be at a second frequency point lower than the first frequency point;

[0064] The first threshold is less than or equal to the second threshold.

[0065] The significance of the above two resource usage strategies is that: when the current rendering pressure value is large, more CPU or GPU resources are released to the target application, such as making the CPU or GPU at the highest frequency point; and when the current rendering pressure value is small, fewer CPU or GPU resources are released to the target application, such as making the CPU or GPU at two-thirds of the highest frequency point, so as to ensure continuous optimal performance while improving energy efficiency.

[0066] Of course, in addition to the first resource usage strategy and the second resource usage strategy mentioned above, those skilled in the art can also expand other resource usage strategies based on the above technical concept, and are not limited to these two.

[0067] For the second method, determining the display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data based on the current rendering pressure value may include: obtaining the current rendering pressure value through the extended function of the Vulkan module, and determining the display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data based on the current rendering pressure value.

[0068] Furthermore, the determining, according to the current rendering pressure value, a display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data may include:

[0069] When the current rendering pressure value is less than or equal to the first threshold, determining that the display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data is a first display resolution adjustment strategy, wherein the first display resolution adjustment strategy includes: controlling the display resolution of the area where the main object is located in the screen corresponding to the current frame data to increase, and / or controlling the display resolution of the surrounding area of ​​the main object to remain unchanged;

[0070] When the current rendering pressure value is greater than or equal to the second threshold, determining that the display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data is a second display resolution adjustment strategy, wherein the second display resolution adjustment strategy includes: controlling the display resolution of the surrounding area of ​​the main object in the screen corresponding to the current frame data to be reduced, and controlling the display resolution of the area where the main object is located to remain unchanged;

[0071] The first threshold is less than or equal to the second threshold.

[0072] The main object refers to the main display object in the display screen, such as the central character in the game screen.

[0073] The significance of the above two display resolution adjustment strategies is that: when the current rendering pressure value is large (under high rendering pressure), the display resolution of the central character remains unchanged, and the display resolution of the edge scene is appropriately reduced; and when the current rendering pressure value is small (under low rendering pressure), the display resolution of the central character is appropriately increased, and the display resolution of the edge scene remains unchanged, so as to save some energy consumption while greatly improving the viewing experience and smoothness of the entire game.

[0074] In a specific implementation, the display synthesis system can pass the current rendering pressure value to the vulkan core API through an interface for vulkan expansion, so as to read the data in the buffer queue, construct rendering instructions for the rendering channel, capture the resolution under different display screens, and then match the current rendering pressure value through the ResolutionDynamic function to dynamically change the display resolution of different areas in the display screen corresponding to the current frame data.

[0075] Of course, in addition to the first display resolution adjustment strategy and the second display resolution adjustment strategy, those skilled in the art can also expand other display resolution adjustment strategies based on the above technical concept, and are not limited to these two.

[0076] An embodiment of the present specification provides a method for controlling an application. During the rendering time of the current frame data of the target application, a counter is used to record the number of times a Vulkan module writes data to a command buffer of the Vulkan module, thereby obtaining the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data. The current actual rendering pressure (equivalent to the actual load) of the target application can be accurately calculated based on the total number of writes, and then at least one of the performance resources and display resolution of the target application can be dynamically controlled based on the current actual rendering pressure of the target application. This can improve the performance of the target application developed using Vulkan.

[0077] The following uses SurfaceFlinger as an example to introduce a possible implementation architecture of a control method for an application provided in an embodiment of this specification.

[0078] like Figure 2 As shown, a possible implementation architecture of a control method for an application provided in an embodiment of the present specification may include: SurfaceFlingerBlastBufferQueue 21, a counter 22 inserted into a Vulkan module, an extended function 23 of the Vulkan module, a performance central control 24, a performance adjustment module 25, and a ResolutionDynamic function 26.

[0079] SurfaceFlingerBlastBufferQueue 21 is a method in SurfaceFlinger. Its main function is to process the graphics buffer queue, add the graphics buffer to the queue, and take it out of the queue and display it on the screen when needed. Specifically, when an application or system needs to display a new graphics interface, it adds the corresponding graphics buffer to the BufferQueue queue, and takes it out and displays it on the screen when needed. Adding an interface (notifyVKCmdCountFlagIfNeeded) in SurfaceFlingerBlastBufferQueue 21 can identify the graphics API and rendering process used by the target application.

[0080] The counter 22 inserted into the Vulkan module is used to record the number of times a series of commands for executing rendering and computing tasks are submitted in parallel in a multi-threaded environment to the Vulkan cache area, and through algorithm filtering and refreshing, each frame of data is counted accurately and in real time.

[0081] The extended function 23 of the Vulkan module is used to extend the functions of the Vulkan core API. The principle of Vulkan extension is to implement the extended functions by adding additional commands, functions and data structures to the Vulkan core API. The extended functions are mainly used to add new graphics function methods: ResolutionDynamic function 26.

[0082] The performance central control 24 is used to receive the vulkan count (the total number of write times) uploaded by framewors, and send different performance resource usage strategies to the performance adjustment module 25 according to the current rendering pressure value.

[0083] The performance adjustment module 25 is used to release performance resources to the target application according to the issued performance resource usage policy.

[0084] The ResolutionDynamic function 26 is used to receive the vulkan count (the total number of write times) to dynamically improve the display resolution of the display image to improve the user experience while optimizing the performance.

[0085] exist Figure 2 Based on the implementation architecture shown, Figure 3 As shown, a control method of an application provided in an embodiment of this specification may specifically include:

[0086] Step 300, start.

[0087] Step 301: Add a first interface in the display synthesis system.

[0088] Specifically, by adding a first interface (such as notifyVKCmdCountFlagIfNeeded) to SurfaceFlingerBlastBufferQueue of SurfaceFlinger (a display synthesis system), the graphics API and rendering process used by the target application can be identified.

[0089] Among them, SurfaceFlingerBlastBufferQueue is a method in SurfaceFlinger, which is mainly used to process the graphics buffer queue, add the graphics buffer to the queue, and take it out of the queue and display it on the screen when needed. Specifically, when an application or system needs to display a new graphics interface, it will add the corresponding graphics buffer to the BufferQueue queue, and take it out and display it on the screen when needed.

[0090] Step 302 , identifying through the first interface whether the graphics API used by the target application is Vulkan; if yes, executing step 303 , otherwise, executing step 309 .

[0091] Step 303: read the frame data of the target application through the second interface in the display composition system.

[0092] Step 304, during the rendering time of the current frame data of the target application, the number of times the Vulkan module writes data to the command buffer of the Vulkan module is recorded by a counter, so as to obtain the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data.

[0093] Step 305: Report the total number of write times to the performance control center.

[0094] That is, the total number of write times is reported as the current rendering pressure value.

[0095] Optionally, the total number of write times is matched with a reporting value interval, and reporting is performed when the total number of write times reaches an upper limit of the reporting value interval.

[0096] Specifically, the total number of write times is packaged into a what data format, and then reported to the frameworks in the electronic device via a socket. The frameworks parse the data and transmit it to the performance central control in sections. The performance central control gives different levels of performance resources according to different numerical ranges.

[0097] Step 306: The performance central control determines a performance resource usage strategy for the target application based on the total number of writes.

[0098] Step 307: The extended function of the Vulkan module passes the total number of write times to the core of the Vulkan module.

[0099] That is, the data is obtained through the extended function of Vulkan and passed to the core of the Vulkan module.

[0100] Step 308: The core of the Vulkan module determines and adjusts the display resolution adjustment strategy of the target application according to the total number of write times.

[0101] Specifically, the display resolution of different areas of the display screen is dynamically adjusted through the ResolutionDynamic function. The main implementation principle of the ResolutionDynamic function is to match the rendering detail data with the size of the collected vulkan count value, to readjust the rendering detail data, focus on the main object screen, and dynamically enhance or reduce the rendering details of the main screen or the edge screen.

[0102] Step 309, end.

[0103] After applying a control method of an application provided in an embodiment of this specification to a game application, firstly, the game interface freezes less and the frame rate experience is improved; secondly, the display resolution of the game screen is dynamically refreshed. When the rendering pressure is high, the display resolution of the center screen is enhanced, and the display resolution of the edge screen is reduced more. When the rendering pressure is low, the display resolution of the center screen is highly enhanced, and the display resolution of the edge screen is reduced less. In addition, by adjusting the performance resource release strategy (resource use strategy), the frame rate can be kept full when the user is using a high-load game, and the heat generation of the device is significantly reduced, and the energy efficiency ratio is greatly improved; by dynamically adjusting the display resolution, the performance can be improved while enhancing the user's gaming visual experience.

[0104] The above is an introduction to a control method of an application provided in this specification. Corresponding to the above method, this specification also provides a control device of an application, which is briefly described below.

[0105] Figure 4 is a schematic diagram of the structure of a control device 400 of an application program according to an embodiment of this specification. Figure 4 In a software implementation, the control device 400 of the application may include: a counting module 401, a load determination module 402 and a performance control module 403.

[0106] The counting module 401 is used to record the number of times the Vulkan module writes data to the command buffer of the Vulkan module through a counter during the rendering time of the current frame data of the target application, and obtain the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data.

[0107] The target application may be any graphics application. Optionally, the target application may be a graphics application with relatively high display performance requirements, such as game applications, drawing applications, and other applications with high graphics rendering frequency and complex rendering.

[0108] The current frame data of the target application refers to the frame data that the target application currently needs to synthesize and display. Usually, the second interface (such as notifyVKCmdCountFlagNeeded) of the display synthesis system (generally referring to SurfaceFlinger) continuously reads each frame data of the target application, then synthesizes the Framebuffer and saves it to the frame buffer, and finally, the screen of the electronic device reads the Framebuffer from the frame buffer and displays it.

[0109] Optionally, the device may further include: a judgment module, used to identify whether the drawing API used by the target application is Vulkan through the first interface in the display synthesis system before triggering the counting module 401, and if so, trigger the counting module 401, otherwise, do not trigger. The first interface may be a newly added interface.

[0110] Specifically, the Vulkan module can call the method SurfaceFlingerBlastBufferQueue in the display synthesis system - SurfaceFlinger, and add an interface (first interface) in this method to monitor the integrity and cycle of the current frame data in real time, and before rendering the current frame data, determine whether the drawing API used by the target application is Vulkan. Among them, the main function of SurfaceFlingerBlastBufferQueue is to process the graphics buffer queue, and its specific function is to add the graphics buffer to the queue, and take it out of the queue and display it on the screen when needed. Generally speaking, when an application or system needs to display a new graphical interface, it will add the corresponding graphics buffer area to the BufferQueue queue, and take it out and display it on the screen when needed.

[0111] The first interface may include but is not limited to notifyVKCmdCountFlagIfNeeded added in the SurfaceFlingerBlastBufferQueue method of SurfaceFlinger.

[0112] The counter is pre-inserted into the Vulkan module. Optionally, the device may further include: a clearing module, which is used to clear the counter before triggering the counting module 401 to ensure that the current actual load can be correctly collected.

[0113] Optionally, the device may further include: a writing module, configured to record the total number of write times in the command buffer area.

[0114] Furthermore, the device may also include: an encryption module, which is used to encrypt the total number of writes recorded in the command buffer through the encryption code in the Vulkan module to ensure the security of the total number of writes. Specifically, the total number of writes can be encrypted by the code (verifySocket) pre-implanted in the Vulkan module. Optionally, the display synthesis system (rendering layer) can query the total number of writes through a specified switch (vkCmdCount).

[0115] The load determination module 402 is used to determine the current rendering pressure of the target application according to the total number of write times.

[0116] As an example, the load determination module 402 may use the total number of write times as the current rendering pressure value of the target application.

[0117] As another example, the load determination module 402 may also convert the total number of write times into a numerical value as the current rendering pressure value of the target application. The numerical conversion may include but is not limited to proportional conversion (such as conversion into a numerical value between 0 and 1), level conversion (such as conversion into a matching level), etc.

[0118] It should be noted that, no matter whether the total number of writes itself or the converted value of the total number of writes is used to represent the current rendering pressure value of the target application, the current rendering pressure value of the target application is always positively correlated with the number of writes.

[0119] The performance control module 403 is used to control at least one of the performance resources and the display resolution of the target application according to the current rendering pressure.

[0120] It should be noted that, in addition to performance resources and display resolution, other performance indicators of the target application may also be controlled according to the current rendering pressure, and this specification does not impose any restrictions on this.

[0121] As an example, the performance control module 403 may be used to:

[0122] 1) determining a performance resource usage policy of the target application according to the current rendering pressure value, and controlling the target application to run according to the performance resource usage policy (or, releasing performance resources to the target application according to the performance resource usage policy); wherein the performance resources include but are not limited to at least one of a CPU and a GPU;

[0123] and / or

[0124] 2) Determine a display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data according to the current rendering pressure value, and control the target application to display the screen corresponding to the current frame data according to the display resolution adjustment strategy.

[0125] The above two methods are explained one by one by examples below.

[0126] For the first method, the performance control module 403 can be used to: first report the current rendering pressure value to the performance central control through the Vulkan module, so that the performance central control determines the performance resource usage strategy of the target application according to the current rendering pressure value. The performance central control refers to the performance control module in the electronic device.

[0127] Specifically, the Vulkan module can package the current rendering pressure value into a what type and then report it to the performance central control through a socket, so that the performance central control determines the performance resource usage strategy of the target application according to the current rendering pressure value.

[0128] Optionally, the Vulkan module may report the current rendering pressure value to the performance central control according to a reporting numerical interval, wherein the report is made when the current rendering pressure value reaches the upper limit of the reporting numerical interval. That is to say, after the current rendering pressure value is obtained, the current rendering pressure value is interval matched, and a suitable reporting interval is set for reporting. For example, 1 to 5 may be used as a limited interval. If the current rendering pressure value falls within this interval, it is reported only when the current rendering pressure value is equal to 5. The next time the current rendering pressure value falls within this interval but is not equal to 5, it will not be reported. This can reduce the number of reports, reduce system overhead, and save energy consumption.

[0129] Furthermore, the performance control module 403 may be used to:

[0130] When the current rendering pressure value is greater than or equal to the second threshold, determining that the performance resource usage policy of the target application is a first resource usage policy, wherein the first resource usage policy includes: controlling at least one of the CPU and the GPU to be at a first frequency point;

[0131] When the current rendering pressure value is less than or equal to the first threshold, determining the performance resource usage policy of the target application to be a second resource usage policy, wherein the second resource usage policy includes: controlling at least one of the CPU and the GPU to be at a second frequency point lower than the first frequency point;

[0132] The first threshold is less than or equal to the second threshold.

[0133] The significance of the above two resource usage strategies is that: when the current rendering pressure value is large, more CPU or GPU resources are released to the target application, such as making the CPU or GPU at the highest frequency point; and when the current rendering pressure value is small, fewer CPU or GPU resources are released to the target application, such as making the CPU or GPU at two-thirds of the highest frequency point, so as to ensure continuous optimal performance while improving energy efficiency.

[0134] Of course, in addition to the first resource usage strategy and the second resource usage strategy mentioned above, those skilled in the art can also expand other resource usage strategies based on the above technical concept, and are not limited to these two.

[0135] For the second method, the performance control module 403 can be used to: obtain the current rendering pressure value through the extended function of the Vulkan module, and determine the display resolution adjustment strategy of different areas in the display screen corresponding to the current frame data according to the current rendering pressure value.

[0136] Furthermore, the performance control module 403 may be used to:

[0137] When the current rendering pressure value is less than or equal to the first threshold, determining that the display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data is a first display resolution adjustment strategy, wherein the first display resolution adjustment strategy includes: controlling the display resolution of the area where the main object is located in the screen corresponding to the current frame data to increase, and / or controlling the display resolution of the surrounding area of ​​the main object to remain unchanged;

[0138] When the current rendering pressure value is greater than or equal to the second threshold, determining that the display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data is a second display resolution adjustment strategy, wherein the second display resolution adjustment strategy includes: controlling the display resolution of the surrounding area of ​​the main object in the screen corresponding to the current frame data to be reduced, and controlling the display resolution of the area where the main object is located to remain unchanged;

[0139] The first threshold is less than or equal to the second threshold.

[0140] The main object refers to the main display object in the display screen, such as the central character in the game screen.

[0141] The significance of the above two display resolution adjustment strategies is that: when the current rendering pressure value is large (under high rendering pressure), the display resolution of the central character remains unchanged, and the display resolution of the edge scene is appropriately reduced; and when the current rendering pressure value is small (under low rendering pressure), the display resolution of the central character is appropriately increased, and the display resolution of the edge scene remains unchanged, so as to save some energy consumption while greatly improving the viewing experience and smoothness of the entire game.

[0142] In a specific implementation, the display synthesis system can pass the current rendering pressure value to the vulkan core API through an interface for vulkan expansion, so as to read the data in the buffer queue, construct rendering instructions for the rendering channel, capture the resolution under different display screens, and then match the current rendering pressure value through the ResolutionDynamic function to dynamically change the display resolution of different areas in the display screen corresponding to the current frame data.

[0143] Of course, in addition to the first display resolution adjustment strategy and the second display resolution adjustment strategy, those skilled in the art can also expand other display resolution adjustment strategies based on the above technical concept, and are not limited to these two.

[0144] An application control device provided in an embodiment of the present specification can record the number of times a Vulkan module writes data to a command buffer of the Vulkan module through a counter during the rendering time of the current frame data of the target application, thereby obtaining the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data, thereby accurately calculating the current actual rendering pressure (equivalent to the actual load) of the target application based on the total number of writes, and then dynamically controlling at least one of the performance resources and display resolution of the target application based on the current actual rendering pressure of the target application, thereby improving the performance of the target application developed using Vulkan.

[0145] The control device 400 of the application program provided in the embodiment of this specification can also execute Figure 1 method, and implement Figure 1 The functions of the embodiments shown in the figure are the same and the same technical effects are achieved, so the embodiments of this specification will not be described in detail here.

[0146] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0147] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this specification. Figure 5 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0148] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0149] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0150] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a control device for the application program at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0151] During the rendering time of the current frame data of the target application, the number of times the Vulkan module writes data to the command buffer of the Vulkan module is recorded by a counter, so as to obtain the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data;

[0152] Determine the current rendering pressure of the target application according to the total number of writes;

[0153] At least one of the performance resources and the display resolution of the target application is controlled according to the current rendering pressure.

[0154] The above is as in this manual Figure 5 The method executed by the control device of the application disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in software form. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of this specification can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of this specification can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0155] The electronic device may also perform Figure 1 The method and the control device of the application are implemented in Figure 1 The functions in the illustrated embodiments will not be described in detail in this specification.

[0156] Of course, in addition to software implementation, the electronic device of this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0157] The embodiment of the present specification also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of target application programs, enable the portable electronic device to execute Figure 1 The method of the embodiment shown is specifically used to perform the following operations:

[0158] During the rendering time of the current frame data of the target application, the number of times the Vulkan module writes data to the command buffer of the Vulkan module is recorded by a counter, so as to obtain the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data;

[0159] Determine the current rendering pressure of the target application according to the total number of writes;

[0160] At least one of the performance resources and the display resolution of the target application is controlled according to the current rendering pressure.

[0161] In short, the above description is only a preferred embodiment of this specification and is not intended to limit the protection scope of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the protection scope of this specification.

[0162] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0163] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0164] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0165] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A method for controlling an application, comprising: During the rendering time of the current frame data of the target application, the number of times the Vulkan module writes data to the command buffer of the Vulkan module is recorded by a counter, so as to obtain the total number of times the Vulkan module writes data to the command buffer during the rendering time of the current frame data; Determine the current rendering pressure of the target application according to the total number of writes; At least one of the performance resources and the display resolution of the target application is controlled according to the current rendering pressure.

2. The method according to claim 1, before determining the current rendering pressure of the target application according to the total number of writes, the method further comprises: The total number of write times is recorded in the command buffer area.

3. The method according to claim 2, further comprising: The total number of write times recorded in the command buffer is encrypted by the encryption code in the Vulkan module.

4. The method according to claim 1, wherein: The counter is pre-inserted into the Vulkan module.

5. The method according to any one of claims 1 to 4, wherein: The determining the current rendering pressure of the target application according to the total number of write times includes: The total number of write times is used as the current rendering pressure value of the target application.

6. The method according to claim 5, wherein: The controlling at least one of the performance resources and the display resolution of the target application according to the current rendering pressure includes: Determine a performance resource usage policy of the target application according to the current rendering pressure value, and control the target application to run according to the performance resource usage policy; and / or, A display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data is determined according to the current rendering pressure value, and the target application is controlled to display the screen corresponding to the current frame data according to the display resolution adjustment strategy.

7. The method according to claim 6, wherein: The determining the performance resource usage strategy of the target application according to the current rendering pressure value includes: The current rendering pressure value is reported to the performance central control through the Vulkan module, so that the performance central control determines the performance resource usage strategy of the target application according to the current rendering pressure value.

8. The method according to claim 7, wherein: The reporting the current rendering pressure value to the performance central control through the Vulkan module includes: The current rendering pressure value is reported to the performance central control through the Vulkan module according to the reporting value interval, wherein the reporting is performed when the current rendering pressure value reaches the upper limit of the reporting value interval.

9. The method according to any one of claims 6 to 8, wherein: The determining the performance resource usage strategy of the target application according to the current rendering pressure value includes: When the current rendering pressure value is greater than or equal to the second threshold, determining the performance resource usage strategy of the target application to be a first resource usage strategy, wherein the first resource usage strategy includes: controlling at least one of the CPU and the GPU to be at a first frequency point; When the current rendering pressure value is less than or equal to the first threshold, determining the performance resource usage policy of the target application to be a second resource usage policy, wherein the second resource usage policy includes: controlling at least one of the CPU and the GPU to be at a second frequency point lower than the first frequency point; The first threshold is less than or equal to the second threshold.

10. The method according to claim 6, wherein: The step of determining, according to the current rendering pressure value, a display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data includes: The current rendering pressure value is obtained through the extended function of the Vulkan module, and the display resolution adjustment strategy of different areas in the display screen corresponding to the current frame data is determined according to the current rendering pressure value.

11. The method according to claim 6 or 10, wherein: The step of determining, according to the current rendering pressure value, a display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data includes: When the current rendering pressure value is less than or equal to the first threshold, determining that the display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data is a first display resolution adjustment strategy, wherein the first display resolution adjustment strategy includes: controlling the display resolution of the area where the main object is located in the screen corresponding to the current frame data to increase, and / or controlling the display resolution of the surrounding area of ​​the main object to remain unchanged; When the current rendering pressure value is greater than or equal to the second threshold, determining that the display resolution adjustment strategy for different areas in the display screen corresponding to the current frame data is a second display resolution adjustment strategy, wherein the second display resolution adjustment strategy includes: controlling the display resolution of the surrounding area of ​​the main object in the screen corresponding to the current frame data to be reduced, and controlling the display resolution of the area where the main object is located to remain unchanged; The first threshold is less than or equal to the second threshold.

12. The method according to claim 1, before recording the number of times the Vulkan module writes data to the command buffer of the Vulkan module by a counter, the method further comprises: Identify, through a first interface in a display composition system, whether a drawing API used by the target application is Vulkan; If yes, then execute the step of recording the number of times the Vulkan module writes data to the command buffer of the Vulkan module through a counter.

13. The method according to claim 1, further comprising: Before the Vulkan module starts rendering the current frame data, the counter is cleared.

14. An electronic device comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 13. 15 . A computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to claim 1 .

Images