Task scheduling method, electronic equipment, storage medium and chip

Abstract

The invention provides a task scheduling method, electronic equipment, a storage medium and a chip, and relates to the technical field of terminals. The method comprises the following steps: acquiring a plurality of GPU (Graphic Processing Unit) tasks, and sorting the tasks according to the priority of GPU task types from high to low; allocating resources for the plurality of GPU tasks in a plurality of ring buffers, so that the resource proportion of the GPU tasks which are ranked in the front is higher when the GPU tasks which are ranked in the front are closer to the head positions of the plurality of ring buffers; and sequentially executing the plurality of GPU tasks. By utilizing the method, the lagging of the current display picture of the electronic equipment can be reduced in a GPU heavy load scene, the use experience of a user is improved, and the scheme does not need to improve the GPU frequency, so that the power consumption of the GPU cannot be increased.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a task scheduling method, electronic equipment, storage medium and chip. Background Art

[0002] With the development of technology, the refresh rate of display screens of electronic devices such as mobile phones and tablets is increasing. The refresh rate is the number of frames of images drawn on the screen of an electronic device per second. The refresh rate of electronic devices has gradually evolved from 30 frames and 60 frames to 90 frames and 120 frames.

[0003] As the refresh rate increases, the load on the graphics processing unit (GPU) of electronic devices increases day by day. When there are many GPU tasks, the GPU will queue the GPU tasks and process them in sequence. In the scenario where the GPU is heavily loaded, when queuing for GPU tasks, the GPU tasks are queued for execution in the order of first creator first executed. At this time, the GPU tasks created first will block the GPU tasks created later because of their longer execution time. When the GPU tasks created later are tasks related to the current display screen of the electronic device, the current display screen of the electronic device may be stuck or have frame drops, affecting the user experience. Summary of the invention

[0004] In order to solve the above problems, the present application provides a task scheduling method, an electronic device, a storage medium and a chip, which can reduce the lag of the current display screen of the electronic device in the scenario of GPU overload, thereby improving the user experience.

[0005] In a first aspect, the present application provides a task scheduling method, which includes: obtaining multiple graphics processor GPU tasks, and sorting them in descending order of priority of GPU task type; allocating resources to the multiple GPU tasks in multiple ring buffers so that GPU tasks with higher sorting have a higher resource share when they are closer to the head position of the multiple ring buffers; and executing the multiple GPU tasks in sequence.

[0006] With this solution, the resources of the ring buffer ringbuffer are no longer directly allocated to GPU tasks in the order in which the GPU tasks are generated. Instead, the GPU tasks are first sorted according to the priority of the GPU task type corresponding to the GPU tasks, and the GPU tasks are sequentially preempted for the resources of the ring buffer according to the sorted order. At this time, the higher the priority of the GPU task, the higher the resource share of the head position of the ring buffer, so that the GPU task with high priority can be executed earlier and completed earlier, while the GPU task with low priority will be executed later. At this time, even if the execution time of the GPU task with low priority is long, it will not block the GPU task with high priority. The tasks related to the display screen of the electronic device and the foreground interface are GPU synthesis tasks with high priority, which can be executed first. Therefore, this solution can reduce the jamming of the current display screen of the electronic device in the scenario of GPU overload, and improve the user experience. In addition, the solution adjusts the execution order of GPU tasks by resource preemption without increasing the GPU frequency, so it will not increase the power consumption of the GPU, and has high practicality.

[0007] In a possible implementation, the method of obtaining multiple graphics processor GPU tasks and sorting them in descending order of priority of GPU task type specifically includes: obtaining each GPU task instruction generated by each process, each GPU task instruction corresponding to a GPU task; determining the GPU task corresponding to each GPU task instruction, and determining the GPU task type corresponding to each GPU task; and sorting each GPU task in descending order of priority of GPU task type.

[0008] The resource preemption of GPU tasks occurs in the kernel layer of the system, specifically in the kernel graphics system layer (KGSL). The present application method adds a to-do task scheduling module in KGSL, in which GPU tasks are reordered according to the priority of GPU task types.

[0009] In one possible implementation, when there are at least two GPU tasks corresponding to the same GPU task type priority, the GPU tasks are sorted in descending order according to the priority of the GPU task type, specifically including: obtaining the task generation time corresponding to the at least two GPU tasks; and sorting the at least two GPU tasks in descending order according to the task generation time.

[0010] In a possible implementation, each of the GPU tasks includes at least the following two GPU tasks: a first GPU task and a second GPU task, wherein the first GPU task is ranked higher than the second GPU task, and the allocating resources to the multiple GPU tasks in multiple ring buffers specifically includes:

[0011] Make the number of ring buffers occupied by the first GPU task at the first position greater than or equal to the number of ring buffers occupied by the second GPU task at the first position;

[0012] Furthermore, the position farthest from the head of the ring buffer occupied by the first GPU task is a second position, the position farthest from the head of the ring buffer occupied by the second GPU task is a third position, and the second position is closer to the head of the ring buffer than the third position.

[0013] At this time, the first GPU task with a higher priority can be executed earlier and completed earlier.

[0014] In a possible implementation, making the number of ring buffers occupied by the first GPU task at the first position greater than or equal to the number of ring buffers occupied by the second GPU task at the first position specifically includes:

[0015] The first GPU task is made to occupy the entire ring buffer at the first position.

[0016] At this time, the first GPU task with a higher priority occupies the most resources and can be executed first, ensuring that the first GPU task is completed quickly.

[0017] In a possible implementation, each of the GPU tasks further includes a third GPU task and a fourth GPU task, the third GPU task is ranked higher than the fourth GPU task, and the fourth GPU task is the GPU task with the lowest ranking among the GPU tasks, and the allocating resources to the multiple GPU tasks in multiple ring buffers specifically includes:

[0018] When the remaining ring buffer at the fourth position is occupied by the fourth GPU task, such that the fourth GPU task has just completed occupying all required ring buffers, setting the remaining ring buffer at the fourth position to a dormant state;

[0019] The fourth GPU task is enabled to occupy a fifth position of the ring buffer, wherein the fourth position is closer to a head of the ring buffer than the fifth position.

[0020] By setting the remaining ring buffers at the fourth position to a sleep state, power consumption can be reduced.

[0021] In one possible implementation, the GPU task type is any one of the following: a GPU synthesis task, a GPU rendering task, or a GPU algorithm task; the priority order of each GPU task type is: the priority of the GPU synthesis task is higher than the priority of the GPU rendering task, and the priority of the GPU rendering task is higher than the priority of the GPU algorithm task.

[0022] In a possible implementation, the GPU algorithm task is generated by the following application process:

[0023] The camera capture process, the navigation process of the map application, or the image recognition process of the image recognition software.

[0024] The solution of the present application can be used to alleviate the problem of foreground display screen freeze caused by GPU algorithm tasks generated by the camera shooting process, GPU algorithm tasks generated by the navigation process of the map application, and GPU rendering tasks generated by video editing applications, and has high usability.

[0025] In a second aspect, the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, the processor being used to call and execute the computer program to implement the task scheduling method provided by the first aspect above and any one of the implementation methods of the first aspect.

[0026] In a third aspect, the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by an electronic device, the task scheduling method provided by any of the above implementations is implemented.

[0027] In a fourth aspect, the present application also provides a chip, which includes a processor, and the processor is used to call a computer program in a memory to execute the task scheduling method provided by any of the above implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the principle provided for this application Figure 1 ;

[0029] Figure 2 Scenario illustration provided for this application Figure 1 ;

[0030] Figure 3 Scenario illustration provided for this application Figure 2 ;

[0031] Figure 4 A schematic diagram of an electronic device architecture provided in an embodiment of the present application;

[0032] Figure 5 Timing diagram drawn for the GPU provided by this application;

[0033] Figure 6 A flowchart of a task scheduling method provided in an embodiment of the present application;

[0034] Figure 7 Schematic diagram of the principle provided by the embodiment of this application Figure 1 ;

[0035] Figure 8 Schematic diagram of the principle provided by the embodiment of this application Figure 2 ;

[0036] Fig. 9 The execution log of the GPU task after adopting this solution provided in the embodiment of the present application;

[0037] Fig.10 A flowchart of another task scheduling method provided in an embodiment of the present application;

[0038] Fig.11 Schematic diagram of the principle provided by the embodiment of this application Figure 3 ;

[0039] Fig.12 Schematic diagram of the principle provided by the embodiment of this application Figure 4 ;

[0040] Fig.13 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to enable persons skilled in the art to more clearly understand the solution of the present application, the application scenario of the technical solution of the present application is first described below.

[0042] In current technology, when there are many GPU tasks, the GPU will queue up the GPU tasks and process them in sequence. In the scenario where the GPU is heavily loaded, when queuing up GPU tasks, the GPU tasks are queued up in the order of first created first executed.

[0043] See also Figure 1 , which is a schematic diagram of the principle provided by this application Figure 1 .

[0044] The central processing unit (CPU) of the electronic device sends a drawing instruction to the GPU, and the GPU processes the task corresponding to the drawing instruction.

[0045] At this time, a freeze occurs in the second frame. The cause of the freeze is GPU task 2, which is related to the foreground display screen. However, since GPU task 1 is executed first and the processing time of GPU task 1 is long, GPU task 2 is blocked and is not completed within one frame, causing the frame to freeze.

[0046] The following is an example using specific scenarios.

[0047] See also Figure 2 and Figure 3 .in, Figure 2 Scenario illustration provided for this application Figure 1 ; Figure 3 Scenario diagram provided for the embodiment of the present application Figure 3 .

[0048] Figure 2 2-(a) is a schematic diagram of a preview interface for taking photos of an electronic device. When a user clicks the photo button 21, the photo interface briefly flashes and generates a photo as shown in 2-(b), and then returns to the preview interface shown in 2-(c).

[0049] After the user takes multiple photos in a row using the camera, the user can Figure 3 The animation effect shown is as follows: Figure 3 - Swipe up on the camera interface shown in (a) to return to the desktop, and slide on the desktop to switch desktop pages, for example, switch from the desktop home page to Figure 3 -(b) The second page shown in Figure 3 - On the second page shown in (b), click to launch the Mail app and open Figure 3 -(c) shows the mailbox interface.

[0050] At this time, the animation effects of application startup or desktop sliding may be stuck and dropped. It is understandable that the above animation effect triggering scenarios are only for distance explanation. In actual applications, the animation effect triggering scenarios on the desktop include but are not limited to status bar pull-down, status bar pull-up, side sliding, application shifting, desktop editing mode opening animation, desktop folder opening, desktop folder closing, folder creation and merging, folder release, desktop application position swap, click effect, long press effect, application installation prompt corresponding animation or entering one-handed operation mode and other scenes that require GPU rendering processing, which will not be repeated here.

[0051] By analyzing the GPU tasks when the animation effects are triggered after the camera continuous shooting scene, it is found that the camera algorithm occupies the GPU resources first, resulting in the GPU being unable to complete the GPU tasks corresponding to the current display screen within the duration of one frame, which in turn causes the jamming. Taking the screen refresh rate of an electronic device as 60 frames per second as an example, the GPU rendering task for each frame needs to be completed within 16.7 milliseconds. However, combined with the camera continuous shooting log, it can be found that because the camera algorithm task occupies the GPU resources first, the GPU task for executing the animation effect needs to wait until the camera algorithm task is completed before it can be executed, and the waiting time can be as long as tens of milliseconds, which is more than the rendering time of one frame of 16.7 milliseconds, which in turn causes the animation effect of the foreground interface to drop frames and jam.

[0052] The above description only takes the camera algorithm occupying GPU resources as an example. In actual applications, other types of applications may also occupy GPU resources, which may cause the foreground screen to freeze and drop frames. For example, the navigation algorithm of the map application occupies GPU resources, causing the foreground screen to freeze and drop frames; for example, after the user adds special effects or edits the video through a video editing application or plug-in, during the video rendering process, when other windows or interfaces are called out, the foreground screen may freeze because the GPU performs rendering tasks.

[0053] To sum up, the current GPU resource allocation method will allocate limited GPU resources to the first created GPU task. However, the first created GPU task may block the later created GPU task due to its long execution time. When the later created GPU task is a task related to the current display screen of the electronic device, it may cause the current display screen of the electronic device to be stuck or have frame drops, affecting the user experience.

[0054] In order to solve the above technical problems, the embodiment of the present application provides a task scheduling method, an electronic device, a storage medium and a chip. The method first sorts the tasks of each graphics processor GPU in order from high to low according to the priority of the corresponding GPU task type, and then makes each GPU task successively seize the resources of each ring buffer in the sorted order, so that the closer to the head position of each ring buffer, the higher the resource share of the GPU task that is ranked higher. Among them, the GPU task that is closer to the head position in each ring buffer is executed first. Using this solution, the jamming of the current display screen of the electronic device can be reduced in the scenario of GPU overload, which improves the user experience. The solution of the present application can be used to alleviate the problem of jamming of the foreground display screen caused by the GPU algorithm tasks generated by the camera shooting process, the GPU algorithm tasks generated by the navigation process of the map application, and the GPU rendering tasks generated by the video editing application, and has high usability. And the solution does not need to increase the GPU frequency, so it will not cause the power consumption of the GPU to increase.

[0055] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0056] The electronic device in the embodiment of the present application may be a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted terminal device, or other electronic device. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device. The following description takes the electronic device as an example of a mobile phone.

[0057] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the drawing process of the GPU of the electronic device is first described below.

[0058] See also Figure 4 , which is a schematic diagram of the electronic device architecture provided in an embodiment of the present application.

[0059] The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device.

[0060] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into the application layer, the application framework layer (Framework), the system runtime layer, the hardware abstraction layer (HAL) and the kernel layer (Linux Kernel) from top to bottom. For the convenience of explanation, Figure 4 Some hardware layer devices are also shown.

[0061] The application layer can include a series of application packages. Both built-in system applications and non-system-level applications belong to the application layer, which is responsible for direct interaction with users. For example, it can include instant messaging apps, cameras, browsers, galleries, and launchers.

[0062] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions. This layer is generally written in Java code and can be called a Java Framework.

[0063] The application framework layer may include but is not limited to the following services and components.

[0064] Window Manager Service (WMS): Mainly responsible for managing the window view on the Android device, and controlling the application interface and correct display and input. WMS is responsible for managing the position, size and layout of application windows, starting, adding and deleting windows, taking care of multitasking and switching between applications, thus ensuring a stable, smooth and consistent user interface. The window manager can also obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0065] Activity Manager Service (AMS) is a key component in the Android system. It is responsible for managing the application life cycle, task stack, and interaction between applications, as well as starting, switching, and scheduling the four major components in the system and managing and scheduling application processes.

[0066] View system: includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. The display interface can be composed of one or more views. For example, the display interface of a text notification icon can include a view for displaying text and a view for displaying images. View is an abstraction of controls at the interface layer. It is the basis of all controls in Android and can sense events such as pressing and sliding.

[0067] The system runtime layer mainly includes the Android runtime library and program library (Native C / C++ Libraries).

[0068] Native C / C++ Libraries may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0069] The Android runtime library includes: the core library and the Android Runtime (ART). ART is responsible for the scheduling and management of the Android system and is a running environment on the Android operating system. In the ART environment, when the application is installed for the first time, the bytecode will be pre-compiled into machine code, making it a truly native application.

[0070] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0071] HWUI (Hardware Accelerated Drawing UI) is a module added by Android for 2D hardware drawing. Before HWUI, Android mainly used skia for software drawing. Later, due to problems such as drawing performance, Android now almost all uses hwui hardware accelerated drawing. HWUI mainly uses OpenGL for GPU hardware drawing to improve the drawing performance of the entire system.

[0072] libgui is a GUI library used to create graphical user interfaces.

[0073] Libui is the native framework of the Android graphics library, responsible for providing the framework of the graphical interface (Surface).

[0074] The role of SurfaceFlinger is to accept graphics display data from multiple sources, synthesize them, and then send them to the display device.

[0075] The Hardware Abstraction Layer (HAL) is a routine package of the software layer. It is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware and simulate the details of a specific system platform so that the program can directly access the hardware resources.

[0076] HAL Interface Definition Language (HIDL) is an interface definition language that specifies the interface between a HAL and its users. HIDL allows the specification of types and method calls (which are aggregated into interfaces and packages).

[0077] Composer is a hardware compositor that accepts the composite layer parameters sent from SurfaceFlinger for synthesis.

[0078] The allocator class is defined in the header file memory to help separate memory allocation and object construction.

[0079] Mapper is a mapping mechanism from logical devices to physical devices. Under this mechanism, users can easily manage storage resources according to their needs.

[0080] The GPU Device Development Kit (DDK) includes device drivers that enable access to GPU hardware.

[0081] The Linux Kernel is the layer between hardware and software.

[0082] The display subsystem frame buffer (DSS FrameBuffer) is used to implement LCD display driving and output DP interface data.

[0083] Figure 4 The dashed box on the right side of the figure shows the GPU drawing process. Figure 5 , which is a timing diagram drawn by the GPU provided in this application.

[0084] When performing the GPU drawing process, the main thread of the upper-level application such as the Gallery or Launcher application creates a view and calls the OpenGL interface of the HWUI through the RenderThread thread of the application.

[0085] Among them, the RenderThread thread is a new system-controlled processing thread that can keep the animation smooth when the UI thread is blocked.

[0086] Then call the GPU DDK interface, and then call the GPU Driver through the GPU DDK to finally translate the instructions into various register operations that the GPU hardware can recognize, driving the GPU hardware to complete the task.

[0087] An embodiment of the present application provides a method for determining the execution order of GPU tasks. Through the GPU resource preemption strategy, different GPU tasks preempt different resources. GPU resource preemption occurs at the kernel layer, which is explained below in conjunction with the accompanying drawings.

[0088] See also Figure 6 , which is a flowchart of a method for determining the execution order of GPU tasks provided in an embodiment of the present application.

[0089] The method comprises the following steps:

[0090] S11: Acquire multiple graphics processor GPU tasks, and sort them in descending order according to the priority of the GPU task type.

[0091] S12: Allocate resources to the multiple GPU tasks in the multiple ring buffers, so that the GPU tasks that are ranked higher have a higher resource share when they are closer to the head positions of the multiple ring buffers.

[0092] The GPU tasks that are closer to the head position in each ring buffer are executed first.

[0093] S13: Execute multiple GPU tasks in sequence.

[0094] By preempting resources, the GPU tasks that are closer to the head of each ring buffer and ranked higher have a higher resource share. The principle of this solution is described in detail below. In the following description, the execution order of the four GPU tasks is determined. In actual applications, the number of GPU tasks that exist simultaneously can be more or less, and this embodiment of the application does not specifically limit this.

[0095] See also Figure 7 , which is a schematic diagram of the principle provided by the embodiment of the present application Figure 1 .

[0096] In the current solution without ring buffer preemption, the GPU tasks issued by different processes such as proc1, proc2, proc3 and proc4 generate corresponding GPU task instructions after passing through the Graphics System Layer (GSL). The GPU task instruction corresponding to proc1 is cmd1pro12, the GPU task instruction corresponding to proc2 is cmd2pro8, the GPU task instruction corresponding to proc3 is cmd3pro4, and the GPU task instruction corresponding to proc4 is cmd4pro1.

[0097] Among them, cmd1pro12 represents a GPU task with a priority of 12, cmd2pro8 represents a GPU task with a priority of 8, cmd3pro14 represents a GPU task with a priority of 4, and cmd4pro1 represents a GPU task with a priority of 1. For the sake of convenience, in the embodiments of the present application, when the priority is n, the smaller n is, the higher the priority is considered to be, and it does not constitute a limitation on the technical solution of the present application.

[0098] These GPU task instructions occupy the resources of the ring buffer in the kernel graphics system layer (KGSL) in the order in which the process issues the GPU tasks. At this time, the GPU task corresponding to cmd1pro12 generated first occupies the head position of S1, followed by the GPU task corresponding to cmd2pro8 occupying the position of S2, the GPU task corresponding to cmd3pro4 occupying the position of S3, and the GPU task corresponding to cmd4pro1 occupying the position of S4 closest to the tail.

[0099] In the ring buffer, since the GPU tasks closer to the head are executed first, the GPU tasks corresponding to cmd1pro12 are always executed first, and the GPU tasks corresponding to cmd4pro1 are always executed later. This will cause cmd4pro1 to be affected if the GPU tasks such as cmd1pro12, cmd2pro8 or cmd3pro4 have long execution times. If the GPU tasks corresponding to cmd4pro1 are GPU tasks related to the foreground display screen, that is, GPU synthesis tasks with higher priority, the foreground will be stuck.

[0100] See also Figure 8 , which is a schematic diagram of the principle provided by the embodiment of the present application Figure 2 .

[0101] The technical solution of the present application adds a preemption function for ring buffer resources. A dispatcher pending list module 80 is added to KGSL. In this module, GPU tasks of different priorities are reordered, with high-priority GPU tasks being placed in the front and low-priority GPU tasks being placed at the end.

[0102] After sorting, the task queue is put into the ring buffer. At this time, the execution order of GPU tasks is no longer first come first served, but the important tasks with high priority are executed first. At this time, the GPU task corresponding to cmd1pro12 is task4, and task4 has the lowest priority; the GPU task corresponding to cmd2pro8 is task3, and task3 has a higher priority than task4; the GPU task corresponding to cmd3pro4 is task2, and task2 has a higher priority than task3; the GPU task corresponding to cmd4pro1 is task1, and task1 has the highest priority and is executed first.

[0103] Through the above technical solution, if the GPU task corresponding to cmd4pro1 is the GPU task related to the foreground display screen, even if cmd4pro1 is generated the latest, but because the GPU task task1 corresponding to cmd4pro1 has the highest priority, task1 can also be executed first, thereby avoiding foreground lag.

[0104] The technical effect of the present application scheme is explained below in combination with actual test results.

[0105] See also Fig. 9 , this figure is the execution log of the GPU task after adopting this solution provided in an embodiment of the present application.

[0106] The GPU task with a low priority of ts=265 is submitted first. This task is the GPU task corresponding to the camera algorithm. ts=265 is the number of the task. The priority of ts=265 is ctx_prio=12.

[0107] During this period, three new GPU tasks with higher priority appear, ts=21858720, ts=21864960 and ts=21871200. The priority of these three GPU tasks is ctx_prio=8, which is higher than ctx_prio=12. Therefore, ts=21858720, ts=21864960 and ts=21871200 are executed in sequence before ts=265. These three GPU tasks are GPU rendering tasks corresponding to video rendering. After ts=21858720, ts=21864960 and ts=21871200 are all executed, ts=265 is completed last.

[0108] The system user interface (system UI) of an electronic device is one of the core applications of the system, responsible for feedback on the system and application status and maintaining a large amount of interaction with the user. With system UI as the base class, subclasses of system UI implement different system interfaces. The main view structure of system UI includes the negative one screen, lock screen wallpaper (wallpaper), status bar (Status Bar), lock screen interface (keyguard) and lock screen password interface (Bouncer). The status bar (Status Bar) is used to display system status information. When the electronic device applies the ring buffer resource preemption solution, the test shows that the GPU rendering task issued by system UI through RenderThead has a shorter runtime and can be completed within 16.7ms of one frame, so it will not cause visual lag.

[0109] In summary, by utilizing the solution provided by the embodiment of the present application, the resources of the ring buffer are no longer directly allocated to GPU tasks in the order in which the GPU tasks are generated. Instead, the GPU tasks are first sorted according to the priority of the GPU task type corresponding to the GPU tasks, and each GPU task sequentially seizes the resources of the ring buffer according to the sorted order. As a result, GPU tasks with high priority can be executed earlier, while GPU tasks with lower priority will be executed later. At this time, even if the execution time of the GPU tasks with lower priority is longer, they will not block the GPU tasks with higher priority. The tasks of the electronic device related to the display screen of the foreground interface are GPU synthesis tasks with higher priority, which can be executed preferentially. Therefore, this solution can reduce the jamming of the current display screen of the electronic device in the scenario of GPU overload, thereby improving the user experience.

[0110] The above description only takes the resource occupying one ring buffer as an example. In actual applications, GPU tasks can often occupy multiple ring buffers. In this case, the preemption weight can be determined according to the sorting result. The following is an explanation in combination with the specific implementation method.

[0111] See also Fig.10 and Fig.11 .in, Fig.10 A flowchart of another method for determining the execution order of GPU tasks provided in an embodiment of the present application; Fig.11 Schematic diagram of the principle provided by the embodiment of this application Figure 3 .

[0112] The method comprises the following steps.

[0113] S21: Each process generates multiple GPU tasks.

[0114] In each process running on the electronic device, multiple GPU tasks may be generated. The GPU task types in the embodiment of the present application may include but are not limited to GPU synthesis tasks, GPU rendering tasks or GPU algorithm tasks.

[0115] In the process of android layer buffer synthesis and display, HWC (hwcomposer) is the HAL layer module used for synthesis and display. When the processing power of HWC is insufficient, GPU synthesis is usually required. SurfaceFlinger can set the drawing instructions to the GPU through the RenderEngine thread, that is, send the GPU synthesis task to the GPU. In heavy load scenarios, GPU synthesis tasks may include but are not limited to application startup and exit scenarios, real-time window blur scenarios, multi-layer display scenarios, and other scenes that can be displayed on the foreground interface, that is, scenes that users can directly perceive visually. Therefore, the GPU synthesis task has the highest priority, and it is necessary to ensure that the GPU synthesis task is completed first.

[0116] GPU rendering tasks are mainly generated when electronic devices perform image and video rendering processing. For example, when a user runs a video editing application on an electronic device and renders the edited video, the process of the application will generate corresponding GPU rendering tasks.

[0117] GPU algorithm tasks are generated when the application of an electronic device uses GPU resources to implement algorithm processing. For example, the navigation process of the map application of an electronic device will use GPU resources to implement the navigation algorithm, the camera shooting process of the camera application of an electronic device will use GPU resources to implement the camera algorithm, and the image recognition process of the image recognition software of an electronic device will use GPU resources to implement the image recognition algorithm.

[0118] At the same time, the processes of various applications on an electronic device can generate multiple GPU tasks.

[0119] by Fig.11 In the example, processes proc1, proc2, proc3, and proc4 generate four GPU tasks in sequence.

[0120] S22: Graphics System Layer generates GPU task instructions corresponding to each of the GPU tasks.

[0121] After the Graphics System Layer obtains the GPU tasks issued by each process, it generates GPU task instructions corresponding to each GPU task.

[0122] Each GPU task instruction corresponds to a GPU task.

[0123] like Fig.11 As shown, the GPU task instruction corresponding to proc1 is cmd1pro12, which is a GPU algorithm task with a priority of 12; the GPU task instruction corresponding to proc2 is cmd2pro8, which is a GPU rendering task with a priority of 8; the GPU task instruction corresponding to proc3 is cmd3pro4, which is a GPU algorithm task with a priority of 4; the GPU task instruction corresponding to proc4 is cmd4pro1, which is a GPU synthesis task with a priority of 1.

[0124] S23: Graphics System Layer sends each GPU task instruction to the DispatcherPending List of KGSL.

[0125] S24: The Dispatcher Pending List determines the GPU task corresponding to each of the GPU task instructions, and determines the GPU task type corresponding to each of the GPU tasks.

[0126] S25: Dispatcher Pending List sorts the GPU tasks in descending order according to the priority of the GPU task type.

[0127] When sorting by task type, the priority order of GPU task types is: the priority of GPU synthesis tasks is higher than that of GPU rendering tasks, and the priority of GPU rendering tasks is higher than that of GPU algorithm tasks. For example, at this time, the priority of GPU synthesis tasks can be represented by pro1, the priority of GPU rendering tasks can be represented by pro8, and the priority of GPU algorithm tasks can be represented by pro12. At this time, when sorting, since the priorities of GPU tasks of the same type are the same, the GPU tasks generated first can be arranged in front and then executed first, and the GPU tasks generated later can be arranged in the back and then executed later.

[0128] In another possible implementation, the GPU tasks can be sorted according to the priority of the services they correspond to. When the services implemented by the processes that trigger the GPU tasks are different, the services corresponding to the GPU tasks are different and their priorities are also different. The priority of the GPU tasks triggered by each process is predetermined, and a priority identifier is pre-assigned to the priority of the GPU tasks triggered by each process as a basis for determining the priority, for example Fig.11 The pro1, pro4, pro8, and pro12 in the table are all priority identifiers. At this time, they are not directly classified according to the type of GPU tasks, but are targeted at specific businesses. Therefore, GPU tasks of the same type can have different priorities due to different specific businesses. For example, proc12 is a GPU algorithm task with a priority of 12, and proc4 is a GPU algorithm task with a priority of 4, but the two algorithm tasks correspond to different businesses.

[0129] The solution of the present application adds a Dispatcher Pending List module in KGSL. In this module, GPU tasks of different priorities are reordered, with high-priority GPU tasks being placed at the front and low-priority GPU tasks being placed at the back.

[0130] The difference in sorting methods is only reflected in the order of GPU tasks in the Dispatcher Pending List. The sorting method does not affect the resource preemption strategy in the ring buffers.

[0131] It is understandable that for both sorting methods, i.e. sorting by priority of GPU task type or sorting by the business corresponding to GPU tasks, the same priority may occur. When there are two or more GPU tasks with the same priority, Dispatcher Pending List obtains the task generation time corresponding to each GPU task with the same priority, and sorts the two or more GPU tasks with the same priority in the order of task generation time from earliest to latest.

[0132] See also Fig.12 The principle diagram shown Figure 4 .

[0133] exist Fig.12 In the scheme without ring buffer preemption shown in the figure, GPU tasks occupy each ring buffer in the order of first come first served and wait for execution. Specifically, the GPU tasks issued by proc1, proc2, proc3 and proc4 are generated by the Graphics System Layer in turn as cmd1pro12 (GPU algorithm task with priority 12), cmd2pro8 (GPU algorithm task with priority 8), cmd3pro14 (GPU rendering task with priority 4) and cmd4pro1 (GPU synthesis task with priority 1). These tasks are executed in the ring buffers of KGSL according to the logic of first come first served and occupying all resources. For the three ring buffers A, B and C, as long as the GPU task is generated first, it will occupy the resources near the head first. As long as the GPU task is generated later, it can only passively occupy the resources near the tail. At this time, if the GPU task generated by proc1 takes a long time to execute, it will affect the GPU synthesis task generated by the key process proc4, resulting in jams.

[0134] Continue to see Fig.11 In the present application scheme, the Dispatcher Pending List module reorders GPU tasks of different priorities. At this time, cmd4pro1 (the synthesis task of the GPU with a priority of 1) has the highest priority and is sorted as task1. cmd3pro4 (the synthesis task of the GPU with a priority of 4) has the second priority and is sorted as task2. cmd2pro8 (the synthesis task of the GPU with a priority of 8) has the third priority and is sorted as task3. cmd1pro12 (the synthesis task of the GPU with a priority of 12) has the fourth priority and is sorted as task4.

[0135] S26: Allocate resources to the GPU tasks so that GPU tasks that are ranked higher have a higher resource share when they are closer to the head positions of the multiple ring buffers.

[0136] When preempting the resources of the ring buffers, the high-priority task4 is put in front and takes priority in preempting the three ring buffers at position S1.

[0137] The second priority task 2 preempts two ring buffers at position S2 and one ring buffer at position S3.

[0138] Task 3 of the third priority occupies a ring buffer at position S2, position S3, and position S4 respectively.

[0139] After being preempted, the lowest priority task 4 (e.g., corresponding to the camera algorithm) only occupies the positions of S3, S4, and S5, each occupying one ring buffer.

[0140] The following describes in detail how to implement resource preemption.

[0141] For the first GPU task and the second GPU task, the ranking of the first GPU task is higher than the ranking of the second GPU task.

[0142] The number of ring buffers occupied by the first GPU task at the first position is made greater than or equal to the number of ring buffers occupied by the second GPU task at the first position.

[0143] Furthermore, the position farthest from the head of the ring buffer occupied by the first GPU task is the second position, the position farthest from the head of the ring buffer occupied by the second GPU task is the third position, and the second position is closer to the head of the ring buffer than the third position.

[0144] For example: when the first GPU task is task1, and the second GPU task is task2, the first position is S1, and the number of ring buffers occupied by task1 in the first position S1 is 3, and the number of ring buffers occupied by task2 in the first position S1 is 0. And the position farthest from the head of the ring buffer occupied by task1 is the second position, which is also S1, and the position farthest from the head of the ring buffer occupied by task2 is the third position, which is S3, and S1 is closer to the head of the ring buffer than S3. And at this time, the first GPU task task1 occupies all the ring buffers in the first position, so that the first GPU task task1 with the highest priority can be started and completed as soon as possible.

[0145] For another example: when the first GPU task is task2, and the second GPU task is task3, the first position is S2, and the number of ring buffers occupied by task2 in the first position S2 is 2, and the number of ring buffers occupied by task3 in the first position S2 is 1. And the position farthest from the head of the ring buffer occupied by task2 is the second position, the second position is S3, and the position farthest from the head of the ring buffer occupied by task3 is the third position, the third position is S4, and S3 is closer to the head of the ring buffer than S4.

[0146] For another example: when the first GPU task is task3 and the second GPU task is task4, the first position is S2, at which time the number of ring buffers occupied by task3 in the first position S2 is 1, and the number of ring buffers occupied by task4 in the first position S2 is 0. And the position farthest from the head of the ring buffer occupied by task3 is the second position, the second position is S4, and the position farthest from the head of the ring buffer occupied by task4 is the third position, the third position is S5, and S4 is closer to the head of the ring buffer than S5.

[0147] In addition, for the third GPU task and the fourth GPU task included in each GPU task, when the ranking of the third GPU task is higher than the ranking of the fourth GPU task, when the remaining ring buffer at the fourth position is occupied by the fourth GPU task, just when the fourth GPU task has completed occupying all the required ring buffers, the remaining ring buffer at the fourth position is set to a dormant state, and the fourth GPU task occupies the ring buffer at the fifth position, and the fourth position is closer to the head of the ring buffer than the fifth position.

[0148] For example, when the third GPU task is task3 and the fourth GPU task is task4, the fourth position is S4 and the fifth position is S5. After task3 completes the occupation of all required ring buffers, the S4 position of ring buffer C is the remaining ring buffer at the fourth position. When the S4 position of ring buffer C is occupied by the fourth GPU task, the fourth GPU task just completes the occupation of all required ring buffers. At this time, the S4 position of ring buffer C will not be occupied by the fourth GPU task, but is set to a dormant state. At this time, the fourth GPU task task4 occupies the ring buffer at the fifth position S5, that is, occupies S5 of ring buffer A, so that the fourth GPU task completes the occupation of all required ring buffers. By setting the S4 position of ring buffer C to a dormant state, power consumption can be saved.

[0149] In summary, using the technical solution provided by the embodiment of the present application, the GPU tasks are first sorted according to the priority of the GPU task type corresponding to the GPU task, and then each GPU task is sequentially preempted from the ringbuffer resources according to the sorted order. As a result, GPU tasks with high priority can be executed earlier and completed earlier, while GPU tasks with lower priority will be executed later. At this time, even if the execution time of the GPU tasks with lower priority is longer, it will not block the GPU tasks with higher priority. The tasks of the electronic device related to the display screen of the foreground interface are GPU synthesis tasks with higher priority, which can be executed preferentially. Therefore, the solution can reduce the jamming of the current display screen of the electronic device in the scenario of GPU overload, thereby improving the user experience. In addition, the solution adjusts the execution order of GPU tasks by resource preemption without increasing the GPU frequency, so it will not cause the power consumption of the GPU to increase, and has high practicality.

[0150] Based on the task scheduling method provided in the above embodiment, the embodiment of the present application further provides an electronic device, which is described in detail below with reference to the accompanying drawings.

[0151] See also Fig.13 , which is a schematic diagram of an electronic device provided in an embodiment of the present application.

[0152] The electronic device shown in the figure may be a mobile phone, a laptop computer, a wearable electronic device (such as a smart watch), a tablet computer, an augmented reality (AR) device, a virtual reality (VR) device, and an in-vehicle device, etc.

[0153] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0154] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0155] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0156] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0157] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, at least one application required for a function, etc. The data storage area may store data created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0158] In an embodiment of the present application, when the computer executable program code stored in the internal memory 121 is called and executed by the processor, the method for determining the execution order of the GPU tasks described in the above description is implemented. By implementing this method, the jamming of the current display screen of the electronic device can be reduced in the scenario of GPU overload, thereby improving the user experience. In addition, the solution adjusts the execution order of GPU tasks by resource preemption without increasing the GPU frequency, so it does not cause an increase in the power consumption of the GPU, and is highly practical.

[0159] Specifically, when the user opens a video editing application for video rendering and returns to the desktop to trigger the animation effect, the user does not feel any freeze in the foreground screen; or when the user opens a map navigation APP for navigation and positioning and returns to the desktop to trigger the animation effect, the user does not feel any freeze in the foreground screen; or when the user takes a photo or shoots a video through the camera application and returns to the desktop to trigger the animation effect, the user does not feel any freeze in the foreground screen. This is because after adopting the technical solution of the present application, the GPU task corresponding to the foreground animation effect has a high priority and can therefore be executed first, so that the GPU task corresponding to the foreground animation effect can be processed within one frame, and the user cannot visually feel any freeze or frame drop.

[0160] In addition, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by an electronic device, the task scheduling method provided in the above description is implemented.

[0161] In addition, an embodiment of the present application further provides a chip, which includes a processor, and the processor is used to call a computer program in a memory to execute the task scheduling method provided by any of the above implementations.

[0162] It should be understood that in this application, "at least one (item)" means one or more, and "more" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist, where A and B can be singular or plural.

[0163] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A task scheduling method, characterized in that: Applied to electronic equipment, the method includes: Get multiple GPU tasks and sort them in descending order of priority based on GPU task type. Allocating resources to the plurality of GPU tasks in a plurality of ring buffers so that GPU tasks that are ranked higher have a higher resource share when they are closer to the head of the plurality of ring buffers; The multiple GPU tasks are executed sequentially.

2. The method according to claim 1, characterized in that The method of obtaining multiple graphics processing unit (GPU) tasks and sorting them in descending order of priority of GPU task types includes: Obtaining each GPU task instruction generated by each process, each GPU task instruction corresponding to a GPU task; Determine the GPU task corresponding to each of the GPU task instructions, and determine the GPU task type corresponding to each of the GPU tasks; The GPU tasks are sorted in descending order of priority of the GPU task type.

3. The method according to claim 2, characterized in that When there are at least two GPU tasks corresponding to the same GPU task type priority, sorting the GPU tasks in descending order of priority of the GPU task type specifically includes: Obtaining task generation times corresponding to the at least two GPU tasks; The at least two GPU tasks are sorted in descending order of task generation time.

4. The method according to claim 3, characterized in that Each of the GPU tasks includes at least the following two GPU tasks: a first GPU task and a second GPU task, wherein the first GPU task is ranked higher than the second GPU task, and allocating resources to the multiple GPU tasks in multiple ring buffers specifically includes: Make the number of ring buffers occupied by the first GPU task at the first position greater than or equal to the number of ring buffers occupied by the second GPU task at the first position; Furthermore, the position farthest from the head of the ring buffer occupied by the first GPU task is a second position, the position farthest from the head of the ring buffer occupied by the second GPU task is a third position, and the second position is closer to the head of the ring buffer than the third position.

5. The method according to claim 4, characterized in that The step of making the number of ring buffers occupied by the first GPU task at the first position greater than or equal to the number of ring buffers occupied by the second GPU task at the first position specifically includes: The first GPU task is enabled to occupy the entire ring buffer at the first position.

6. The method according to claim 4 or 5, characterized in that Each of the GPU tasks further includes a third GPU task and a fourth GPU task, the third GPU task is ranked higher than the fourth GPU task, and the fourth GPU task is the GPU task with the lowest ranking among the GPU tasks, and allocating resources to the multiple GPU tasks in multiple ring buffers specifically includes: When the remaining ring buffer at the fourth position is occupied by the fourth GPU task, such that the fourth GPU task has just completed occupying all required ring buffers, setting the remaining ring buffer at the fourth position to a dormant state; The fourth GPU task is caused to occupy a fifth position of the ring buffer, where the fourth position is closer to the head of the ring buffer than the fifth position.

7. The method according to claim 1, wherein The GPU task type is any one of the following: GPU synthesis task, GPU rendering task or GPU algorithm task; The priority order of each GPU task type is: The priority of the GPU synthesis task is higher than the priority of the GPU rendering task, and the priority of the GPU rendering task is higher than the priority of the GPU algorithm task.

8. The method according to claim 7, characterized in that The GPU algorithm task is generated by the following application process: The camera capture process, the navigation process of a map application, or the image recognition process of an image recognition software.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor is used to call and execute the computer program to implement the task scheduling method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by an electronic device, the task scheduling method according to any one of claims 1 to 8 is implemented.

11. A chip, characterized in that: The chip includes a processor, and the processor is used to call a computer program in a memory to execute the task scheduling method according to any one of claims 1 to 8.

Images