Resource scheduling method, electronic device and computer readable storage medium

Abstract

The application relates to the software technical field and provides a resource scheduling method, an electronic device and a computer readable storage medium. The method is applied to an electronic device, and a target application program runs on the electronic device. The method comprises the following steps: acquiring a frame identifier of a current frame, wherein the current frame is a frame image of the target application program; determining whether the current frame is a real frame or a virtual frame according to the frame identifier; if the current frame is the real frame, performing resource scheduling on the current frame by using a target scheduling module, and the target scheduling module is used for performing resource scheduling on the frame image of the target application program. The above method can optimize resource configuration and save power consumption.

Description

Technical Field

[0001] This application relates to the field of software technology, specifically to a resource scheduling method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the development of terminal devices and software technology, people have increasingly higher requirements for display effects and power consumption in high-performance scenarios.

[0003] Taking gaming as an example, when users operate game applications on their terminal devices, frame interpolation can be used to ensure the smoothness of the video playback in the game application.

[0004] However, traditional frame interpolation methods have unreasonable resource allocation and relatively high power consumption. Summary of the Invention

[0005] This application provides a resource scheduling method, apparatus, chip, electronic device, computer-readable storage medium, and computer program product that can optimize resource allocation and save power consumption.

[0006] Firstly, a resource scheduling method is provided, applied to an electronic device running a target application. The method includes: obtaining the frame identifier of the current frame, where the current frame is a frame image of the target application; determining whether the current frame is a real frame or a virtual frame based on the frame identifier; if the current frame is a real frame, using a target scheduling module to perform resource scheduling on the current frame, whereby the target scheduling module is used to perform resource scheduling on the frame images of the target application.

[0007] The target application can be a video playback application or a game application, or other high-performance applications. In high-performance scenarios, frame interpolation or frame supplementation is often required. Frame interpolation and frame supplementation are collectively referred to as virtual frames. The key frames used to synthesize virtual frames are called real frames. The target scheduling module can be an AI scheduling module.

[0008] Electronic devices can identify real frames through frame identifiers and perform resource scheduling for real frames via an AI scheduling module, avoiding resource scheduling for virtual frames. This prevents virtual frames from consuming dedicated resources, allowing for more efficient resource scheduling. Furthermore, the AI ​​scheduling module and image processing enhancement module use binder communication during resource scheduling. By focusing resource scheduling only on real frames, the frequency of binder communication is reduced, thereby lowering power consumption.

[0009] In some possible implementations, the method further includes: if the current frame is a virtual frame, using the system scheduling module to perform resource scheduling on the current frame.

[0010] The electronic device identifies the current frame as a virtual frame based on the frame identifier. Since virtual frames are relatively lightweight, other system scheduling strategies (system scheduling module) can be used for resource scheduling, such as EAS scheduling.

[0011] Based on this, virtual frames are scheduled for resources through the system scheduling module. This stepping ensures that the overhead of virtual frames does not occupy the dedicated resources of real frames, making resource scheduling more reasonable.

[0012] In some possible implementations, obtaining the frame identifier of the current frame includes: generating an enable command when an enable condition is detected; in response to the enable command, creating an independent thread in the process of the target application, the independent thread being used to synthesize the frame image of the target application; and obtaining the frame identifier of the current frame based on the independent thread.

[0013] The electronic device can monitor the operating environment and generate an activation command when the environment meets the activation conditions. Based on the activation command, the electronic device creates a new independent thread within the target application's process. This independent thread is used to generate frame images for the target application.

[0014] After establishing an independent thread, the electronic device can obtain the identifier of that independent thread. The electronic device can also identify the frame identifier of the current frame based on the independent thread.

[0015] In some possible implementations, the activation conditions include: obtaining an activation command to enable the frame interpolation function, and / or detecting a scene that needs to be interpolated.

[0016] In some possible implementations, the scenarios for which frame interpolation is to be performed include: a game application playing a game video in the foreground, or a video playback application playing a video in the foreground.

[0017] Optionally, the user can operate the management application corresponding to the target application to enable or disable the frame interpolation function of the target application. When the electronic device detects that the user has enabled the frame interpolation function through the management application, it can determine that it is currently in the enabled state, thus meeting the enabling conditions.

[0018] Optionally, the electronic device can also automatically monitor the current scene. If it detects that it is in a scene where frames need to be filled, such as a game application running in the foreground or a video playback application running in the foreground and playing a video, it can determine that it is now in the on state and meets the on conditions.

[0019] Among some possible implementations, the method also includes: obtaining the timestamp of the current frame based on a separate thread.

[0020] After establishing an independent thread, the electronic device can also identify the timestamp of the current frame based on the independent thread.

[0021] In some possible implementations, the method further includes: in response to an enable command, obtaining the package name and target frame rate of the target application; and using a target scheduling module to perform resource scheduling on the current frame, including: using the target scheduling module to add the identifier of an independent thread to a preset group to perform resource scheduling on the current frame, wherein the resource configuration priority of processes in the preset group is higher than that of processes outside the preset group.

[0022] The electronic device also responds to the activation command by determining the target frame rate of the target application. Different applications have their own corresponding playback frame rates. It can be understood that an application will have a smooth playback effect when played at its corresponding playback frame rate. Taking game A as an example, game A has a smooth playback effect at a frame rate of 120Hz. When the graphics processing enhancement module detects that the frame interpolation function is enabled, and game A is currently running at a frame rate of 60Hz, the graphics processing enhancement module can set the target frame rate of game A to 120Hz, allowing game A's video to play at a frame rate of 120Hz, ensuring the smoothness of the game screen.

[0023] Optionally, the graphics processing enhancement module can, after receiving the package name of the game application, query a preset whitelist configuration file to obtain the playback frame rate corresponding to the game application's package name, and set that playback frame rate as the target frame rate. It can be understood that the preset whitelist configuration file may include the package names of multiple applications, each application's package name corresponding to its own playback frame rate.

[0024] The preset groups can be either VIP groups or game groups. Optionally, threads in the preset groups have higher resource allocation priority than threads outside the preset groups. The target scheduling module adds the identifier of the independent thread to the preset group, which ensures that the independent thread has a high resource allocation priority, thereby guaranteeing the resource supply for the independent thread and enabling timely resource scheduling.

[0025] In some possible implementations, the electronic device may include: a frame stabilization engine, an image processing enhancement service, an image processing enhancement core, an application interest module, a graphical user interface building module, an image compositing module, a toolkit module, a graphical interface, and a target scheduling module.

[0026] Obtaining the frame identifier of the current frame includes: when the frame stabilization engine detects the activation condition, it sends an activation command to the image processing enhancement service; in response to the activation command, the image processing enhancement service sends a creation command to the image processing enhancement core; in response to the creation command, the image processing enhancement core creates an independent thread; the image processing enhancement core obtains the frame identifier and the timestamp of the current frame based on the independent thread.

[0027] Determining whether the current frame is a real frame or a virtual frame based on the frame identifier includes: the image processing enhancement core sending the frame identifier and the timestamp of the current frame to the graphical user interface construction module; the graphical user interface construction module sending the frame identifier and the timestamp of the current frame to the image compositing module; the application interest module identifying the identifier of the independent thread and sending the identifier of the independent thread to the image compositing module; and the image compositing module determining whether the current frame is a real frame or a virtual frame based on the identifier of the independent thread and the frame identifier.

[0028] If the current frame is a real frame, the target scheduling module performs resource scheduling on the current frame, including: if the current frame is a real frame, the image synthesis module sends the timestamp of the current frame to the toolkit module; the toolkit module sends the timestamp of the current frame to the target scheduling module; in response to the start command, the image processing enhancement service also determines the target frame rate based on the package name of the target application; the image processing enhancement service sends the target frame rate and the package name of the target application to the target scheduling module through a graphical interface; the target scheduling module performs resource scheduling on the current frame based on the timestamp of the current frame and the target frame rate.

[0029] In a second aspect, a resource scheduling apparatus is provided, comprising a unit composed of software and / or hardware, the unit being used to execute any one of the methods described in the first aspect.

[0030] Thirdly, embodiments of this application provide a chip including a processor; the processor is used to read and execute a computer program stored in a memory to perform any one of the methods described in the first aspect.

[0031] Optionally, the chip further includes a memory, which is connected to the processor via a circuit or wire.

[0032] Alternatively, the chip may further include a communication interface.

[0033] Fourthly, an electronic device is provided, comprising: a processor, a memory, and an interface; the processor, memory, and interface cooperate with each other to enable the electronic device to perform any one of the methods described in the first aspect.

[0034] Fifthly, an electronic device is provided, which includes any one of the chips described in the third aspect.

[0035] In a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, the processor performs any one of the methods described in the first aspect.

[0036] In a seventh aspect, a computer program product is provided, the computer program product comprising: computer program code, which, when executed on an electronic device, causes the electronic device to perform any one of the methods described in the first aspect. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the structure of a terminal device 100 provided in an embodiment of this application;

[0038] Figure 2 This is a software structure block diagram of the terminal device 100 provided in the embodiments of this application;

[0039] Figure 3 This is a flowchart illustrating an example of a frame interpolation method provided in an embodiment of this application;

[0040] Figure 4 This is an example software architecture diagram provided in an embodiment of this application;

[0041] Figure 5 This is an example of a resource scheduling interaction diagram provided in an embodiment of this application;

[0042] Figure 6 This is a flowchart illustrating an example of a resource scheduling method provided in an embodiment of this application;

[0043] Figure 7 This is a schematic diagram of the structure of a resource scheduling device provided in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0045] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0046] The resource scheduling method provided in this application can be applied to terminal devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of terminal device.

[0047] For example, Figure 1 This is a schematic diagram of the structure of a terminal device 100 provided in an embodiment of this application. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0048] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0049] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0050] The controller can serve as the central nervous system and command center of the terminal device 100. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0051] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0052] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0053] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0054] Terminal device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0055] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a minimized display, a microLED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0056] Terminal device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0057] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP performs Fourier transforms on the frequency energy.

[0058] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. Thus, terminal device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0059] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0060] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0061] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of terminal device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of terminal device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0062] The software system of terminal device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of terminal device 100.

[0063] Figure 2 This is a software structure block diagram of the terminal device 100 according to an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0064] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0065] The Application Framework (FWK) layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes a set of predefined functions.

[0066] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system (SurfaceView), phone manager, resource manager, notification manager, etc.

[0067] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0068] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0069] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0070] The phone manager is used to provide communication functions for terminal device 100. For example, it manages call status (including connection, hang-up, etc.).

[0071] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0072] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0073] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0074] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0075] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0076] The system library (Native) can include multiple functional modules. For example: surface manager, surfaceflinger (SF) module, media libraries, 3D graphics processing library (e.g., OpenGL ES), 2D graphics engine (e.g., SGL), etc.

[0077] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0078] The image compositing module is responsible for compositing and displaying graphics buffers from different applications.

[0079] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0080] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0081] A 2D graphics engine is a graphics engine for 2D drawing.

[0082] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0083] For ease of understanding, the following embodiments of this application will be described using the following methods: Figure 1 and Figure 2 Taking the terminal device with the structure shown as an example, and in conjunction with the accompanying drawings and application scenarios, the resource scheduling method provided in this application embodiment will be specifically described.

[0084] To clearly describe the technical solution of this application, the terms involved in this application will be explained first:

[0085] Frame interpolation: Frame interpolation generates realistic intermediate frames between two keyframes by predicting the motion trajectory of an object. Frame interpolation can improve the smoothness and detail of the image. However, frame interpolation is computationally complex.

[0086] Frame interpolation: Frame interpolation generates new frames by calculating pixel changes between adjacent keyframes. Frame interpolation can improve the smoothness of the video, but its effect is not significant when the original video has a high frame rate. Compared to frame interpolation, frame interpolation has lower computational complexity and is relatively simpler to implement.

[0087] Frame interpolation and frame supplementation can be collectively referred to as virtual frames, while the keyframes used to generate interpolation or frame supplementation can be called real frames.

[0088] In some scenarios, management software, such as game assistant applications, can be installed on the terminal device to manage frame interpolation. Users can use the game assistant to enable frame interpolation for game applications, thus achieving frame interpolation in high-performance scenarios like games, making the video playback smoother. The terminal device can also automatically identify scenarios requiring frame interpolation, such as when the game video or the video application's playback interface is displayed in the foreground, and automatically perform frame interpolation to improve video smoothness.

[0089] Taking a gaming scenario as an example, in addition to the game application, a game assistant is also installed on the terminal device. The specific process of frame interpolation for game videos on the terminal device can be found in [link to documentation]. Figure 3 As shown, it includes:

[0090] S301, the game assistant responds to user operations and sends the frame interpolation function to the graphics processing enhancement module when it is enabled.

[0091] Users can open the game assistant and manually enable or disable the frame interpolation function by clicking the toggle switch in the game assistant's interface. Optionally, users can also operate the game assistant via voice or other means to enable or disable the frame interpolation function. Optionally, the frame interpolation function can also be called a frame rate enhancement function.

[0092] When a user enables the frame interpolation function through the game assistant, the game assistant can respond to the user's operation by sending the enabled status of the frame interpolation function (called the enabled status) to the graphics processing enhancement module, such as GTX (GPU TurboX).

[0093] Optionally, the game assistant can also send the package name of the currently running game application to the graphics processing enhancement module.

[0094] S302. When the graphics processing enhancement module detects that the frame interpolation function is enabled, it sets the frame rate of the game application to the target frame rate.

[0095] The graphics processing enhancement module manages the frame rate of applications. Different applications have their own corresponding playback frame rates. It's understood that an application will have a smooth playback effect when playing at its designated frame rate. Taking game A as an example, game A has a smooth playback effect at a frame rate of 120Hz. When the graphics processing enhancement module detects that frame interpolation is enabled, and game A is currently running at a frame rate of 60Hz, the graphics processing enhancement module can set the target frame rate of game A to 120Hz, allowing game A's video to play at a 120Hz frame rate, ensuring smooth gameplay.

[0096] Optionally, the graphics processing enhancement module can, after receiving the package name of the game application, query a preset whitelist configuration file to obtain the playback frame rate corresponding to the game application's package name, and set that playback frame rate as the target frame rate. It can be understood that the preset whitelist configuration file may include the package names of multiple applications, each application's package name corresponding to its own playback frame rate.

[0097] S303, the graphics processing enhancement module sends the frame interpolation function's on / off status and the target frame rate to the graphics interface.

[0098] The graphics processing enhancement module can send the frame interpolation function's on / off status and target frame rate to the graphics interface, which then forwards them to the lower layer; details will not be elaborated here. The graphics interface serves as a converter between the upper and lower layers; for example, it could be an accelerated graphics port (AGP).

[0099] S304, the image compositing module sends the timestamp of each frame to the graphics interface.

[0100] The image compositing module (SurfaceFlinger, SF) is used to compose each frame of the image, including virtual frames and real frames. It can also obtain the timestamp of each frame of the synthesized image.

[0101] Specifically, under the control of the graphics processing enhancement module, the image compositing module can synthesize each frame of image according to the target frame rate and obtain the timestamp of each synthesized frame. After obtaining the timestamp of each frame, the image compositing module can send the timestamp to the graphics interface.

[0102] S305, the graphics processing enhancement module sends the frame interpolation function's enabled status to the AI ​​scheduling module.

[0103] The AI ​​scheduling module is used for resource scheduling of currently running applications. For example, in a gaming scenario, the AI ​​scheduling module is specifically designed for resource scheduling of game applications, such as CPU allocation and memory scheduling.

[0104] Optionally, the graphics processing enhancement module sends the frame interpolation function's enabled status to the AI ​​scheduling module, which can be done by passing the value of the skiphint flag to the AI ​​scheduling module as 1.

[0105] S306, the graphical interface forwards the target frame rate to the AI ​​scheduling module.

[0106] Specifically, the graphics interface forwards the target frame rate received from the graphics processing enhancement module to the AI ​​scheduling module.

[0107] S307: When the frame interpolation function is enabled, the AI ​​scheduling module schedules resources according to the number of frames that need to be added.

[0108] Specifically, the AI ​​scheduling module performs resource scheduling for the currently running game application to ensure the application's resource supply. When frame interpolation is enabled, if the game application's actual frame rate is 60Hz and the target frame rate is 120Hz, the AI ​​scheduling module will allocate resources for the additional 60 frames that need to be interpolated.

[0109] Specifically, the AI ​​scheduling module can randomly select any 60 frames from the 120 frames at 120Hz and perform resource scheduling on these 60 frames. These 60 frames can include both real frames and virtual frames. In other words, the AI ​​scheduling module can schedule resources according to half of the resources required for the target frame rate. The resources required for other frames can be supplied by other scheduling modules in the system.

[0110] Figure 3 In the method shown in the embodiment, the AI ​​scheduling module does not distinguish between real frames and virtual frames when performing resource scheduling. Since the AI ​​scheduling module schedules resources for both real and virtual frames, scheduling GPU resources for virtual frames as well leads to resource waste. Furthermore, the AI ​​scheduling module and the image processing enhancement module use binder communication during resource scheduling. This means that binder communication is also used for resource scheduling for virtual frames, which increases the frequency of binder communication, potentially doubling it, resulting in high power consumption.

[0111] Based on this, this application provides a resource scheduling method that can filter out real frames by identifying frame identifiers and perform resource scheduling on real frames through an AI scheduling module, thereby avoiding resource scheduling on virtual frames. Virtual frames will not occupy dedicated resources, making resource scheduling more efficient. Furthermore, the AI ​​scheduling module's focus on real frames reduces the frequency of binder communication, thus lowering power consumption.

[0112] Figure 4An example software architecture diagram provided in an embodiment of this application is shown. Figure 2 Based on the above, the software architecture can be further referenced. Figure 4 , Figure 4 Only content relevant to the embodiments of this application is shown; the remaining unshown parts do not necessarily mean they do not exist. Figure 4 The following explanation will continue using the scenario of running a game application as an example.

[0113] like Figure 4 As shown, the application package in the application layer (APP layer) can also include applications such as game applications and game assistants.

[0114] The system library also includes: frame stabilization engine (GUE), image processing enhancement service (GTX service), image processing enhancement core (GTX core), graphical user interface building module (libGUI), native window system (libnativewindow), libEGL, application interest module (Gamesaware), image compositing module (surfaceflinger, SF), toolkit module (Aware GameSdk), and graphics interface (AGP).

[0115] The frame stabilization engine can detect whether the frame interpolation function or the frame supplementation function is enabled, and then initiate frame interpolation or frame supplementation based on the detected enabled status of the frame interpolation or frame supplementation function.

[0116] LibEGL provides a platform that can identify all thread identifiers.

[0117] The image processing enhancement core is an application-side module, while the image processing enhancement service is a system-side module. Together, they enable image enhancement processing.

[0118] The image processing enhancement service can determine the target frame rate and send the target frame rate, target refresh rate, and game application package name when starting interpolation or frame interpolation.

[0119] The image processing enhancement core can create a new independent thread when starting interpolation or frame interpolation, and obtain frame tags and timestamps.

[0120] The graphical user interface building module, supported by the extended functionality (Entend) of the local window system, sends the frame tags and timestamps transmitted by the image processing enhancement core to the image compositing module.

[0121] The application's interest module can obtain the identifier of an independent thread based on libEGL.

[0122] The image synthesis module is used to filter out real frames based on the identifier of the independent thread and the frame label, and send the timestamp of the real frame to the toolkit module.

[0123] The toolkit module is used to send the timestamps of real frames to the AI ​​scheduling module.

[0124] The graphical interface serves as the interface between layers, enabling the image processing enhancement service to send the target frame rate and the package name of the target application to the AI ​​scheduling module.

[0125] The graphics interface and frame stabilization engine reside in the same process. The game application, view system, libEGL, image processing enhancement core, application interest modules, graphical user interface building modules, and native window system also reside in the same process. Communication between the image processing enhancement service and the image processing enhancement core is cross-process.

[0126] The HAL layer is located between the kernel layer and the system libraries. The HAL layer includes the hardware synthesizer (hwcomposer, HWC) and the AI ​​scheduling module.

[0127] HWC is used to adjust the screen refresh rate;

[0128] The AI ​​scheduling module is used to schedule dedicated resources for high-performance applications, such as games, including GPU resources.

[0129] based on Figure 4 The software architecture and structure shown Figure 5 The interactive diagrams shown illustrate the embodiments of this application in detail. For example... Figure 5 As shown, the resource scheduling methods include:

[0130] S501, the frame stabilization engine is confirmed to be in the enabled state of synthesizing virtual frames.

[0131] Optionally, the application layer can install a management application (i.e., management software) to manage the frame interpolation function. Different applications can manage the frame interpolation function through different management applications.

[0132] For example, when a user enables frame interpolation, the management application can send a status indicating that frame interpolation is enabled to the frame stabilization engine to perform frame interpolation on the currently running application. After receiving the status indicating that frame interpolation is enabled, the frame stabilization engine can determine that virtual frames need to be synthesized and thus enter the enabled state.

[0133] Taking game applications as an example, the game assistant, acting as a management application for game applications, can enable or disable the frame interpolation function for a game application by modifying the value of the flag bit corresponding to the game application's package name. For instance, the game assistant can change the value of the flag bit corresponding to the game application's package name to 1 to enable the frame interpolation function, or change the value of the flag bit corresponding to the game application's package name to 0 to disable the frame interpolation function. Then, the terminal device can obtain the on / off status of the game application's frame interpolation function by reading the flag bit of the frame interpolation function switch.

[0134] When a user enables the frame interpolation function through the game assistant, the game assistant modifies the value of the flag corresponding to the game application's package name to 1 and sends it to the frame stabilization engine.

[0135] Optionally, the management application can also send the package names of currently running applications that require frame rate management, such as the package name of a currently running game application, to the graphics processing enhancement service.

[0136] Optionally, the frame stabilization engine can also obtain the current working scene of the terminal device. If it is in a scene that requires frame interpolation, it can enter the state of simulating virtual frames. For example, when the frame stabilization engine detects that a game application is running in the foreground, or a video playback application is running in the foreground and playing video, it can determine that it is currently in the active state.

[0137] By configuring a frame stabilization engine, it can identify not only frame interpolation scenarios but also frame supplementation scenarios. In other words, the frame stabilization engine is compatible with both frame interpolation and frame supplementation scenarios, achieving compatibility between frame interpolation and frame supplementation functions, thus broadening its application scenarios and making it more flexible.

[0138] S502, the frame stabilization engine responds to being enabled by sending an enable command to the image processing enhancement service.

[0139] When the frame stabilization engine determines that it is currently enabled, it can send a compositing command to the image processing enhancement service. This compositing command is used to instruct the start of compositing virtual frames, that is, to enable the frame interpolation / frame supplementation scheme.

[0140] For example, when the frame stabilization engine receives a notification from the management application to enable frame interpolation, or when it determines that it is currently in a scene that needs to be interpolated, it can send an enable command to the image processing enhancement service to instruct it to enable frame interpolation.

[0141] S503, In response to the start command, the image processing enhancement service sends a creation command to the image processing enhancement core.

[0142] When the image processing enhancement service receives an enable command, it sends a create command to the image processing enhancement core in response. This create command instructs the creation of an independent thread to synthesize frame images, which runs within the process of the currently running application.

[0143] The GTX core is an application-side module. When the application requiring frame interpolation or frame supplementation is a game, the GTX core runs within the game application's process, while the independent thread is a thread within that game process.

[0144] 504. The image processing enhancement kernel responds to the creation command and creates a new independent thread.

[0145] The image processing enhancement core responds to a creation command by creating a new, independent thread within the currently running application's process. This independent thread is used to composite virtual and real frames when enabled.

[0146] If the currently running application is a game, this independent thread can be a separate thread within the game application's process, and it can belong exclusively to the currently running application. Taking a game application as an example, this independent thread can copy game resources for compositing virtual and real frames.

[0147] The libEGL platform can send game commands from the game process to the image processing enhancement core on an independent thread. The image processing enhancement core then controls the frame interpolation or frame replacement operations instructed by the game commands. libEGL can also recognize the identifiers of all threads, including independent threads.

[0148] The image processing enhancement kernel can also obtain the identifier (thread ID, TID) of the independent thread with the support of the libEGL platform.

[0149] In addition to obtaining the identifier of an independent thread, the image processing enhancement core can also obtain an identifier for each frame, indicating whether it is a virtual frame or a real frame; this is called a frame identifier. The image processing enhancement core can then pass the frame identifier and the timestamp of each frame (q-buffer timestamp) to the lower layer.

[0150] By creating a separate thread to support frame output, the resource supply for the independent thread can be guaranteed without preempting other resources, thus ensuring the display effect.

[0151] The S505 image processing enhancement core, based on a new independent thread, marks the frame identifiers of real and / or virtual frames and obtains timestamps.

[0152] S506, the image processing enhancement core, sends the frame identifier and timestamp to the graphical user interface building module.

[0153] The image processing enhancement core, based on a new independent thread, marks the frame identifiers of real and / or virtual frames and also obtains the timestamp of each frame. Then, the image processing enhancement core sends the frame identifier and timestamp of the current frame (real or virtual frame) to the graphical user interface building module (libGUI).

[0154] S507, the graphical user interface construction module sends the frame identifier and timestamp of each frame to the image compositing module.

[0155] Optionally, the graphical user interface (GUI) building module and the image compositing module use binder communication. The GUI building module can send the frame identifier and timestamp of each frame (whether it is a real frame or a virtual frame) to the image compositing module via binder communication.

[0156] S508, the application interest module obtains the identifier of the independent thread.

[0157] With the support of libEGL, the application interest module can identify the identifier of this independent thread. If the currently running application is a game application, the application interest module can be gamesaware.

[0158] S509, The application interest module sends the identifier of the independent thread to the image synthesis module.

[0159] The application's interest module sends the identifiers of the identified independent threads to the image synthesis module (surfaceflinger).

[0160] S510 The image synthesis module filters out independent threads based on the identifier of the independent thread, and then filters out the timestamp of the real frame synthesized by the independent thread according to the frame identifier and timestamp.

[0161] The image compositing module can parse the currently received frame information, including the identifier of the independent thread, frame identifier, and timestamp. Based on the identifier of the independent thread, the image compositing module selects the independent thread, and further selects the real frames synthesized by the independent thread based on the frame identifier, thereby obtaining the timestamp of the real frame.

[0162] S510, the image synthesis module will send the timestamp of the real frame to the toolkit module of the currently running application.

[0163] If the currently running application is a game application, the image compositing module will send the timestamps of the real frames to the game application's toolkit module (AwareGameSdk).

[0164] Optionally, the image synthesis module can also send process information (such as pendingnum) to the toolkit module. The pendingnum information indicates whether blocking is likely to occur. If the pendingnum information indicates a large backlog of buffers, blocking may occur, and the AI ​​scheduling module can use this information to perform appropriate resource scheduling to prevent blocking.

[0165] S511, the toolkit module sends the timestamp of the real frame to the AI ​​scheduling module.

[0166] The toolkit module sends the timestamps of the received real frames to the AI ​​scheduling module. Optionally, the toolkit module can also send process information to the AI ​​scheduling module.

[0167] S512, the graphics processing enhancement service responds to the enable command and sets the target frame rate.

[0168] The graphics processing enhancement service responds to the enable command by setting the target frame rate.

[0169] The Graphics Processing Enhancement Service manages the frame rate of applications. Different applications have their own corresponding playback frame rates. It's understood that an application will have good image quality when played at its designated frame rate. For example, Game A has good image quality at a frame rate of 120Hz. When the Graphics Processing Enhancement Service detects that frame interpolation is enabled, and Game A is currently running at a frame rate of 60Hz, the service can set the target frame rate for Game A to 120Hz, ensuring smooth gameplay.

[0170] Optionally, the graphics processing enhancement service can also, after receiving the package name of the game application, query a preset whitelist configuration file to obtain the playback frame rate corresponding to the game application's package name, and use that playback frame rate as the target frame rate. It can be understood that the preset whitelist configuration file may include the package names of multiple applications, and the playback frame rates corresponding to each of these application package names.

[0171] Optionally, if the game application is already running at a high enough frame rate, such as being running at its corresponding playback frame rate, the graphics enhancement service can further reduce the current playback frame rate before interpolating frames to the target frame rate for playback. For example, if the game application is currently running at 120Hz, and its corresponding playback frame rate is also 120Hz, the graphics enhancement service can first reduce the game application's frame rate to 60Hz, and then interpolate frames to the target frame rate of 120Hz for playback. It's important to note that the resources and power consumption of generating virtual frames (such as interpolated or supplemented frames) on the terminal device are lower than those of generating the game application's actual frames. Therefore, reducing the frame rate of the game application's actual frames reduces rendering overhead and power consumption, while interpolating or supplementing frames to a higher frame rate ensures smooth video playback. Thus, this method can reduce resource and power consumption while ensuring smooth game streaming.

[0172] S513, the graphics processing enhancement service sends the target frame rate, target refresh rate, and application package name to the graphics interface.

[0173] Specifically, the target refresh rate is the screen refresh rate corresponding to the currently running application's frame rate output. Generally speaking, if the screen refresh rate is lower than the output frame rate, a higher frame rate will not improve image quality. Therefore, the target refresh rate needs to be adjusted according to the target frame rate to adapt to it. Optionally, the target refresh rate can be twice the target frame rate.

[0174] S514, the graphics interface sends the identifier of the independent thread and the target frame rate to the AI ​​scheduling module.

[0175] The graphics interface forwards the received independent thread identifier and target frame rate to the AI ​​scheduling module.

[0176] Optionally, the graphics interface can also communicate with the HWC to update the screen refresh rate according to the target refresh rate.

[0177] The S515 AI scheduling module adds the identifier of the independent thread to the VIP group.

[0178] Threads in the VIP group have higher resource allocation priority than threads outside the VIP group. The AI ​​scheduling module adds the identifier of independent threads to the VIP group, ensuring that independent threads have high resource allocation priority, thereby guaranteeing resource supply for independent threads and enabling timely resource scheduling. Optionally, this application embodiment does not limit the group with high resource allocation priority; it can also be the game group.

[0179] S516, the AI ​​scheduling module performs resource scheduling on real frames based on the timestamp of the real frames and the target frame rate.

[0180] The AI ​​scheduling module can obtain the timestamps of two consecutive real frames to determine the frame interval. If the frame interval is greater than the scheduled frame rate corresponding to the target frame rate, a frequency increase operation can be performed to increase the frame rate; if the frame interval is less than the scheduled frame rate corresponding to the target frame rate, a frequency decrease operation can be performed to decrease the frame rate, so that the frame rate is stabilized at the target frame rate for frame output.

[0181] For example, if the frame interval obtained by the AI ​​scheduling module is greater than 16.6 milliseconds, it indicates that the frame rate is lower than 60Hz. For the required target frequency of 60Hz, a frequency increase operation can be performed. Conversely, if the obtained frame interval is less than 16.6 milliseconds, it indicates that the corresponding frame rate is higher than 60Hz. For the required target frequency of 60Hz, a frequency increase operation can be performed.

[0182] Specifically, the AI ​​scheduling module schedules resources for real frames based on their timestamps and target frame rates, without scheduling virtual frames. This means that for virtual frames, GPU resource allocation can be reduced while ensuring resource availability for real frames, resulting in more rational resource allocation. Since virtual frames have lower computational load and are less computationally demanding, other system scheduling strategies, such as energy-aware scheduling (EAS), can be used.

[0183] When performing resource scheduling, the AI ​​scheduling module communicates with the image processing enhancement module using binder. The AI ​​scheduling module only performs resource scheduling for real frames, not for virtual frames, which also reduces the frequency of binder communication and thus reduces power consumption.

[0184] It should be noted that the step numbers above do not restrict the order in which the steps are executed. The order of the steps can be freely changed as long as the implementation is guaranteed. For example, the order of S512 and S503 can be changed without restriction.

[0185] Figure 6 This is a flowchart illustrating an example of a resource scheduling method provided in an embodiment of this application. The method is applied to an electronic device running a target application, and includes:

[0186] S601. Obtain the frame identifier of the current frame, which is a frame image of the target application.

[0187] S602. Determine whether the current frame is a real frame or a virtual frame based on the frame identifier.

[0188] S603. If the current frame is a real frame, the target scheduling module is used to perform resource scheduling on the current frame. The target scheduling module is used to perform resource scheduling on the frame image of the target application.

[0189] The target application can be a video playback application or a game application, or other high-performance applications. In high-performance scenarios, frame interpolation or frame supplementation is often required. Frame interpolation and frame supplementation are collectively referred to as virtual frames. The key frames used to synthesize virtual frames are called real frames. The target scheduling module can be an AI scheduling module.

[0190] Electronic devices can identify real frames through frame identifiers and perform resource scheduling for real frames via an AI scheduling module, avoiding resource scheduling for virtual frames. This prevents virtual frames from consuming dedicated resources, allowing for more efficient resource scheduling. Furthermore, the AI ​​scheduling module and image processing enhancement module use binder communication during resource scheduling. By focusing resource scheduling only on real frames, the frequency of binder communication is reduced, thereby lowering power consumption.

[0191] In some embodiments, the method further includes: if the current frame is a virtual frame, using a system scheduling module to perform resource scheduling on the current frame.

[0192] The electronic device identifies the current frame as a virtual frame based on the frame identifier. Since virtual frames are relatively lightweight, other system scheduling strategies (system scheduling module) can be used for resource scheduling, such as EAS scheduling.

[0193] Based on this, virtual frames are scheduled for resources through the system scheduling module. This stepping ensures that the overhead of virtual frames does not occupy the dedicated resources of real frames, making resource scheduling more reasonable.

[0194] In some embodiments, obtaining the frame identifier of the current frame includes: generating an enable command when an enable condition is detected; in response to the enable command, creating an independent thread in the process of the target application, the independent thread being used to synthesize the frame image of the target application; and obtaining the frame identifier of the current frame based on the independent thread.

[0195] The electronic device can monitor the operating environment and generate an activation command when the environment meets the activation conditions. Based on the activation command, the electronic device creates a new independent thread within the target application's process. This independent thread is used to generate frame images for the target application.

[0196] After establishing an independent thread, the electronic device can obtain the identifier of that independent thread. The electronic device can also identify the frame identifier of the current frame based on the independent thread.

[0197] In some embodiments, the activation conditions include: obtaining an activation command to enable the frame interpolation function, and / or detecting a scene to be interpolated.

[0198] In some embodiments, the scenarios to be frame-filled include: a game application playing a game video in the foreground, or a video playback application playing a video in the foreground.

[0199] Optionally, the user can operate the management application corresponding to the target application to enable or disable the frame interpolation function of the target application. When the electronic device detects that the user has enabled the frame interpolation function through the management application, it can determine that it is currently in the enabled state, thus meeting the enabling conditions.

[0200] Optionally, the electronic device can also automatically monitor the current scene. If it detects that it is in a scene where frames need to be filled, such as a game application running in the foreground or a video playback application running in the foreground and playing a video, it can determine that it is now in the on state and meets the on conditions.

[0201] In some embodiments, the method further includes: obtaining the timestamp of the current frame based on a separate thread.

[0202] After establishing an independent thread, the electronic device can also identify the timestamp of the current frame based on the independent thread.

[0203] In some embodiments, the method further includes: in response to an enable command, obtaining the package name of the target application and the target frame rate of the target application; and using a target scheduling module to perform resource scheduling on the current frame, including: using the target scheduling module to add the identifier of an independent thread to a preset group to perform resource scheduling on the current frame, wherein the resource configuration priority of processes in the preset group is higher than that of processes outside the preset group.

[0204] The electronic device also responds to the activation command by determining the target frame rate of the target application. Different applications have their own corresponding playback frame rates. It can be understood that an application will have a smooth playback effect when played at its corresponding playback frame rate. Taking game A as an example, game A has a smooth playback effect at a frame rate of 120Hz. When the graphics processing enhancement module detects that the frame interpolation function is enabled, and game A is currently running at a frame rate of 60Hz, the graphics processing enhancement module can set the target frame rate of game A to 120Hz, allowing game A's video to play at a frame rate of 120Hz, ensuring the smoothness of the game screen.

[0205] Optionally, the graphics processing enhancement module can, after receiving the package name of the game application, query a preset whitelist configuration file to obtain the playback frame rate corresponding to the game application's package name, and set that playback frame rate as the target frame rate. It can be understood that the preset whitelist configuration file may include the package names of multiple applications, each application's package name corresponding to its own playback frame rate.

[0206] The preset groups can be either VIP groups or game groups. Optionally, threads in the preset groups have higher resource allocation priority than threads outside the preset groups. The target scheduling module adds the identifier of the independent thread to the preset group, which ensures that the independent thread has a high resource allocation priority, thereby guaranteeing the resource supply for the independent thread and enabling timely resource scheduling.

[0207] In some embodiments, the electronic device is equipped with: a frame stabilization engine, an image processing enhancement service, an image processing enhancement core, an application interest module, a graphical user interface construction module, an image compositing module, a toolkit module, a graphical interface, and a target scheduling module. Obtaining the frame identifier of the current frame includes: when the frame stabilization engine detects an activation condition, it sends an activation command to the image processing enhancement service; in response to the activation command, the image processing enhancement service sends a creation command to the image processing enhancement core; in response to the creation command, the image processing enhancement core creates an independent thread; the image processing enhancement core, based on the independent thread, obtains the frame identifier and the timestamp of the current frame; determining whether the current frame is a real frame or a virtual frame based on the frame identifier includes: the image processing enhancement core sending the frame identifier and the timestamp of the current frame to the graphical user interface construction module; the graphical user interface construction module sending the frame identifier and the timestamp of the current frame to the graphical user interface construction module; and the graphical user interface construction module sending the frame identifier and the timestamp of the current frame to the graphical user interface construction module. The timestamp of the current frame is sent to the image compositing module; the application interest module identifies the identifier of the independent thread and sends the identifier of the independent thread to the image compositing module; the image compositing module determines whether the current frame is a real frame or a virtual frame based on the identifier of the independent thread and the frame identifier; if the current frame is a real frame, the target scheduling module performs resource scheduling on the current frame, including: if the current frame is a real frame, the image compositing module sends the timestamp of the current frame to the toolkit module; the toolkit module sends the timestamp of the current frame to the target scheduling module; in response to the start command, the image processing enhancement service also determines the target frame rate based on the package name of the target application; the image processing enhancement service sends the target frame rate and the package name of the target application to the target scheduling module through the graphical interface; the target scheduling module performs resource scheduling on the current frame based on the timestamp of the current frame and the target frame rate.

[0208] The implementation principle and technical effects of this embodiment can be found in [reference]. Figure 5 The relevant descriptions of the embodiments will not be repeated here.

[0209] The foregoing has detailed examples of the methods provided in this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0210] This application can divide the resource scheduling device into functional modules based on the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0211] Figure 7 A schematic diagram of a resource scheduling device provided in this application is shown. The device 700 is applied to an electronic device on which a target application is running. The device 700 includes:

[0212] The acquisition module 701 is used to acquire the frame identifier of the current frame, which is a frame image of the target application.

[0213] The determination module 702 is used to determine whether the current frame is a real frame or a virtual frame based on the frame identifier;

[0214] The scheduling module 703 is used to perform resource scheduling on the current frame when the current frame is a real frame, using the target scheduling module. The target scheduling module is used to perform resource scheduling on the frame image of the target application.

[0215] In some embodiments, the scheduling module is further configured to use the system scheduling module to perform resource scheduling on the current frame when the current frame is a virtual frame.

[0216] In some embodiments, the acquisition module 701 is specifically used to generate an opening instruction when an opening condition is detected; in response to the opening instruction, an independent thread is created in the process of the target application, the independent thread is used to synthesize the frame image of the target application; and based on the independent thread, the frame identifier of the current frame is obtained.

[0217] In some embodiments, the activation conditions include: obtaining an activation command to enable the frame interpolation function, and / or detecting a scene to be interpolated.

[0218] In some embodiments, the scenarios to be frame-filled include: a game application playing a game video in the foreground, or a video playback application playing a video in the foreground.

[0219] In some embodiments, the acquisition module 701 is further configured to acquire the timestamp of the current frame based on an independent thread.

[0220] In some embodiments, the apparatus 700 further includes a frame rate determination module, configured to obtain the package name of the target application and the target frame rate of the target application in response to an enable command; and a scheduling module 703, specifically configured to use a target scheduling module to add the identifier of an independent thread to a preset group to perform resource scheduling on the current frame, wherein the resource configuration priority of processes in the preset group is higher than that of processes outside the preset group.

[0221] In some embodiments, the electronic device may be equipped with: a frame stabilization engine, an image processing enhancement service, an image processing enhancement core, an application interest module, a graphical user interface building module, an image compositing module, a toolkit module, a graphical interface, and a target scheduling module.

[0222] The acquisition module 701 is specifically used for: controlling the frame stabilization engine to send an activation command to the image processing enhancement service when it detects the activation condition; in response to the activation command, the image processing enhancement service sends a creation command to the image processing enhancement core; in response to the creation command, the image processing enhancement core creates an independent thread; and the image processing enhancement core obtains the frame identifier and the timestamp of the current frame based on the independent thread.

[0223] The determination module 702 is specifically used for: controlling the image processing enhancement core to send the frame identifier and the timestamp of the current frame to the graphical user interface construction module; the graphical user interface construction module sending the frame identifier and the timestamp of the current frame to the image compositing module; the application interest module identifying the identifier of the independent thread and sending the identifier of the independent thread to the image compositing module; and the image compositing module determining whether the current frame is a real frame or a virtual frame based on the identifier of the independent thread and the frame identifier.

[0224] The scheduling module 703 is specifically used to control the image synthesis module to send the timestamp of the current frame to the toolkit module when the current frame is a real frame; the toolkit module sends the timestamp of the current frame to the target scheduling module; in response to the start command, the image processing enhancement service also determines the target frame rate based on the package name of the target application; the image processing enhancement service sends the target frame rate and the package name of the target application to the target scheduling module through a graphical interface; the target scheduling module performs resource scheduling on the current frame based on the timestamp of the current frame and the target frame rate.

[0225] The specific manner in which the device 700 executes the resource scheduling method and the beneficial effects thereof can be found in the relevant descriptions in the method embodiments, and will not be repeated here.

[0226] This application also provides an electronic device, including the processor described above. The electronic device provided in this embodiment may be... Figure 1 The terminal device 100 shown is used to execute the resource scheduling method described above. When using integrated units, the terminal device may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the actions of the terminal device; for example, it can support the terminal device in executing the steps performed by the display unit, detection unit, and processing unit. The storage module can be used to support the terminal device in executing stored program code and data. The communication module can be used to support communication between the terminal device and other devices.

[0227] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a digital signal processor (DSP), and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other terminal devices.

[0228] In one embodiment, when the processing module is a processor and the storage module is a memory, the terminal device involved in this embodiment can be a device having... Figure 1 The device with the structure shown.

[0229] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the resource scheduling method described in any of the above embodiments.

[0230] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the resource scheduling method described in the above embodiments.

[0231] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0232] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units. The replaced units may or may not be physically separate. The component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0233] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0234] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0235] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A resource scheduling method, characterized in that, Applied to an electronic device on which a target application is running, the method includes: Obtain the frame identifier of the current frame, where the current frame is a frame image of the target application; The current frame is determined to be either a real frame or a virtual frame based on the frame identifier; If the current frame is a real frame, the target scheduling module performs resource scheduling on the current frame. The target scheduling module is used to perform resource scheduling on the frame images of the target application.

2. The method according to claim 1, characterized in that, The method further includes: If the current frame is a virtual frame, the system scheduling module performs resource scheduling on the current frame.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining the frame identifier of the current frame includes: When the activation condition is detected, an activation command is generated; In response to the start command, an independent thread is created in the process of the target application, the independent thread being used to synthesize frame images of the target application; Based on the independent thread, the frame identifier of the current frame is obtained.

4. The method according to claim 3, characterized in that, The activation conditions include: obtaining an activation command to enable the frame interpolation function, and / or detecting a scene that needs to be interpolated.

5. The method according to claim 4, characterized in that, The scenes to be supplemented include: A game app plays game videos in the foreground, or a video playback app plays videos in the foreground.

6. The method according to claim 4 or 5, characterized in that, The method further includes: Based on the independent thread, obtain the timestamp of the current frame.

7. The method according to any one of claims 3 to 6, characterized in that, The method further includes: In response to the start command, obtain the package name of the target application and the target frame rate of the target application; The step of using the target scheduling module to perform resource scheduling on the current frame includes: The target scheduling module is used to add the identifier of the independent thread to a preset group to perform resource scheduling on the current frame. The resource configuration priority of processes in the preset group is higher than that of processes outside the preset group.

8. The method according to claim 1, characterized in that, The electronic device is equipped with: a frame stabilization engine, an image processing enhancement service, an image processing enhancement core, an application interest module, a graphical user interface construction module, an image compositing module, a toolkit module, a graphical interface, and a target scheduling module. The step of obtaining the frame identifier of the current frame includes: When the frame stabilization engine detects the activation condition, it sends an activation command to the image processing enhancement service. In response to the start command, the image processing enhancement service sends a creation command to the image processing enhancement core; In response to the creation instruction, the image processing enhancement core creates an independent thread; The image processing enhancement core obtains the frame identifier and the timestamp of the current frame based on the independent thread; The step of determining whether the current frame is a real frame or a virtual frame based on the frame identifier includes: The image processing enhancement core sends the frame identifier and the timestamp of the current frame to the graphical user interface construction module; The graphical user interface construction module sends the frame identifier and the timestamp of the current frame to the image synthesis module; The application interest module identifies the identifier of the independent thread and sends the identifier of the independent thread to the image synthesis module; The image synthesis module determines whether the current frame is a real frame or a virtual frame based on the identifier of the independent thread and the frame identifier. If the current frame is a real frame, the target scheduling module performs resource scheduling on the current frame, including: If the current frame is a real frame, the image synthesis module sends the timestamp of the current frame to the toolkit module; The toolkit module sends the timestamp of the current frame to the target scheduling module; In response to the activation command, the image processing enhancement service further determines the target frame rate based on the package name of the target application; The image processing enhancement service sends the target frame rate and the package name of the target application to the target scheduling module through a graphical interface; The target scheduling module performs resource scheduling on the current frame based on the timestamp of the current frame and the target frame rate.

9. An electronic device, characterized in that, include: Processor, memory, and interface; The processor, the memory, and the interface cooperate with each other to enable the electronic device to perform the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 8.

Images