Model scheduling method and electronic device
By rationally scheduling models and determining the execution order based on type and state, the problem of resource contention in model execution in electronic devices is solved, thereby improving system efficiency and user experience.
Patent Information
- Application Number
- PCT/CN2026/093331
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-08
- Filing Date
- 2026-04-27
- Publication Date
- 2026-11-12
AI Technical Summary
Running multiple models in electronic devices can easily lead to resource contention, resulting in decreased system efficiency and lag, thus affecting the user experience.
By determining the execution order, priority, and parallel execution strategies based on model type and running status, models can be rationally scheduled to avoid resource contention.
It improves model running efficiency, avoids system lag, and enhances user experience.
Smart Images

Figure CN2026093331_12112026_PF_FP_ABST
Abstract
Description
A model scheduling method and electronic device
[0001] Cross-references to related applications
[0002] This application claims priority to Chinese Patent Application No. 202510592172.0, filed on May 8, 2025, entitled "A Model Scheduling Method and Electronic Device", the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to the field of electronic equipment technology, and in particular to a model scheduling method and an electronic device. Background Technology
[0004] The rapid development of large models has spurred numerous applications such as intelligent agents and content generation. To meet demands for data privacy and cost control, current electronic devices, especially mobile devices, are rapidly integrating diverse models, such as language models, code models, and multimodal models. Some models, especially large models, consume significant system resources such as computing power and memory, while edge devices have limited computing and memory resources. Running these models can pose risks to the operating system. For example, when applications on an electronic device call models, resource contention may occur between different models, leading to decreased model efficiency, excessive consumption of electronic device resources, and even system lag or crashes, resulting in a poor user experience. Summary of the Invention
[0005] This application provides a model scheduling method and electronic device to achieve efficient and reasonable model scheduling, and to avoid or reduce problems such as system lag and freezing during model operation.
[0006] In a first aspect, embodiments of this application provide a model scheduling method applied to an electronic device. The method includes: determining the running order of a first model and a second model based on a first reference item and / or a second reference item; wherein the first reference item includes the type of the first model and the type of the second model, and the second reference item includes the running state of the first model and the running state of the second model; and running the first model and the second model based on the running order.
[0007] In this method, electronic devices can determine the running order of different models based on their type and / or running status, and then run them sequentially according to that order. This efficiently and rationally schedules model inference and improves the efficiency of multi-model operation. Furthermore, by running different models in an orderly manner, this method also prevents resource contention between models, thus avoiding or reducing system lag and crashes during model operation, resulting in a superior user experience.
[0008] In one possible design, any model type conforms to at least one of the following: a system model or a non-system model; a large model or a small model; wherein the large model is a model with a size greater than or equal to a set size, and the small model is a model with a size less than the set size; or, the large model is a model with an inference latency greater than or equal to a set latency, and the small model is a model with an inference latency less than the set latency. Inference latency, also known as runtime latency, refers to the total time required for the model to generate a complete response from the start of inference during the execution of an inference task. Optionally, the inference latency of any model is the average inference latency, maximum inference latency, or minimum inference latency of any model executing at least one inference task within a historical time period.
[0009] Based on the above method, electronic devices can flexibly and efficiently determine the running order of different models based on information such as model size and whether the model is a system model, which is highly practical.
[0010] In one possible design, the system model has a higher priority than the non-system models. Determining the execution order of different models using this method helps to prioritize the operational efficiency of the system model.
[0011] In one possible design, the smaller model has a higher priority than the larger model.
[0012] In this method, the smaller model is typically the model for which the user expects an immediate response. Larger models run slower and have less stringent real-time requirements, allowing for interruptions from the smaller model. Therefore, prioritizing the smaller model over the larger model helps ensure that the model the user expects runs first, thereby improving the user experience.
[0013] In one possible design, the operating state of any model is either the foreground state or any other state besides the foreground state.
[0014] In this method, the running state of the model can reflect the user's desired model running order to some extent. For example, the model in the foreground state may be the model the user currently prefers to run. Therefore, determining the running order of different models by referring to their running states helps to make the model running order closer to the user's expectations, thereby improving the user experience.
[0015] In one possible design, the model of the foreground state has a higher priority than the models of the other states.
[0016] In this method, the inference tasks processed by the model in the foreground state are the inference tasks that users are relatively more concerned about. Therefore, by setting the priority of the model in the foreground state higher than the priority of the models in other states, it helps to ensure that the inference tasks that users are relatively more concerned about are executed first, thereby improving the user experience.
[0017] In one possible design, the foreground state includes a foreground focus state or a foreground non-focus state.
[0018] This method allows for the determination of priorities for different models by referencing finer-grained information, which helps to achieve finer-grained model scheduling.
[0019] In one possible design, the model of the foreground focused state has a higher priority than the model of the foreground non-focused state.
[0020] In this method, the inference tasks handled by the model in the foreground focus state are generally the inference tasks that users are most concerned about. Therefore, by setting the model in the foreground focus state to have the highest priority, it helps to ensure that the inference tasks that users are most concerned about are executed first, thereby improving the user experience.
[0021] In one possible design, when both the first model and the second model belong to the small model, the running order is the same as the priority order of the first model and the second model; or, when both the first model and the second model belong to the large model, and the running states of the first model and the second model are different, the running order is parallel execution; or, when both the first model and the second model belong to the large model, and the running states of the first model and the second model are the same, the running order is determined according to a set rule; or, when the first model belongs to both the large model and the non-system model, and the second model belongs to both the large model and the system model... When modeling, the running order is parallel execution; or, when the first model belongs to the small model, the second model belongs to the large model, and the running state of the first model is the other state and the running state of the second model is the foreground state, the running order is parallel execution; or, when the first model belongs to the small model, the second model belongs to the large model, and the running state of the first model is not the other state and the running state of the second model is the foreground state, the running order is the same as the priority order of the first model and the second model; wherein, the priority of the first model and the second model is determined based on the first reference item and / or the second reference item.
[0022] Based on this method, electronic devices can determine a more suitable model running order in different scenarios, thereby improving model scheduling efficiency and user experience in different scenarios, which is highly practical.
[0023] In one possible design, determining the running order of the first model and the second model based on the first reference item and / or the second reference item includes: determining the priority of the first model and the second model based on the first reference item and / or the second reference item; and determining the running order based on the priority of the first model and the second model.
[0024] In this method, the priority of model running status can reflect business priority, and the priority of model type can reflect the priority of running resources. By combining business priority and computing resource priority, electronic devices can more reasonably determine the priority of different models, and then determine the running order of different models more reasonably based on the priority of different models.
[0025] In one possible design, when the priorities of the first model and the second model are different, the running order is the same as the order of their priorities; or, when the priorities of the first model and the second model are different, the running order is parallel running; or, when the priorities of the first model and the second model are the same, the running order is the order determined according to a set rule.
[0026] Based on this method, electronic devices can determine a more suitable model running order in different priority scenarios, thereby improving model scheduling efficiency and user experience in different scenarios, which is highly practical.
[0027] In one possible design, running the first model and the second model based on the running order includes: when the running order is parallel running and the priority of the first model is higher than the priority of the second model, running the first model at a first running speed and running the second model at a second running speed; wherein the first running speed is higher than the second running speed. Optionally, the first running speed is the speed obtained by reducing the maximum running speed of the first model by a set percentage. Optionally, the second running speed is less than or equal to the set speed.
[0028] Based on this method, in multi-model concurrent scenarios, electronic devices can ensure the high-speed operation of high-priority models, thereby guaranteeing the corresponding high-priority service experience, while simultaneously preventing low-priority models from starving, thus ensuring that low-priority services are not starved. This method enables efficient and reasonable concurrent scheduling of multiple models and prevents resource contention between different models, thereby avoiding system lag and crashes.
[0029] In one possible design, the method further includes: when a set control condition is detected during the operation of the first model and / or the second model, performing control processing; wherein the control processing includes at least one of the following: reducing the running speed of the first model and / or the second model; pausing the operation of the first model and / or the second model; stopping the operation of the first model and / or the second model; and not running a new model.
[0030] Based on this method, electronic devices can control the operation of models in combination with the actual scenario during the operation of one or more models. This can effectively avoid excessive consumption of the system capabilities and resources of electronic devices during model operation, thereby avoiding problems such as system lag, freezing, overheating, and crashes.
[0031] In one possible design, the control conditions include at least one of the following: the electronic device is in a set high-load state; the temperature of the electronic device is higher than a set temperature; the power consumption of the electronic device is higher than a set power consumption; the battery level of the electronic device is lower than a set battery level. Optionally, the electronic device being in a set high-load state specifically includes at least one of the following: the load of the processor used to run the model in the electronic device is higher than or equal to a set load; the bandwidth of the processor used to run the model in the electronic device is higher than or equal to a set bandwidth; the free memory capacity of the electronic device is less than or equal to a set capacity.
[0032] Based on this method, electronic devices can control the operation of one or more models by combining overall status such as load, temperature, power consumption, and battery level. This can effectively avoid excessive consumption of the electronic device's system capabilities and resources during model operation, thereby ensuring the overall performance of the electronic device.
[0033] In one possible design, the first model and / or the second model are in other running states; or, the first model and / or the second model are in the foreground state and the first model and / or the second model belong to the large model; or, the first model and / or the second model belong to the large model.
[0034] In the above methods, the real-time requirements for other states of the model or the large model are relatively low. Therefore, by combining the overall system state to control the operation of such models, we can ensure the overall system performance while minimizing the impact on the user experience.
[0035] In one possible design, the method further includes: if the control condition includes the electronic device being in the high-load state, during the execution of the control process, outputting a prompt message indicating insufficient operating resources for the first model and / or the second model. This method can promptly remind the user of insufficient system resources, thus improving the user experience.
[0036] In one possible design, pausing the operation of the first model and / or the second model includes: pausing the operation of the first model and / or the second model when the control conditions are still met after reducing the operating speed of the first model and / or the second model.
[0037] In this method, electronic devices can first reduce the speed of the model operation when the overall system condition is poor. If the overall system condition remains poor after reducing the speed of the model operation, meaning that reducing the speed of the model operation is insufficient to alleviate the overall system pressure, the model operation can be paused to alleviate the overall system pressure more effectively. Through this method, electronic devices can achieve tiered and refined model control, avoiding over-control and achieving reasonable and effective control efficiency.
[0038] In one possible design, stopping the operation of the first model and / or the second model includes: stopping the operation of the first model and / or the second model when the control conditions are still met after pausing the operation of the first model and / or the second model.
[0039] In this method, if the overall condition of the electronic device remains poor after pausing the model, meaning that pausing the model is insufficient to alleviate the overall pressure, then the model can be stopped to further alleviate the overall pressure. This method allows for tiered and refined model control of electronic devices, avoiding over-control and achieving reasonable and effective control efficiency.
[0040] In one possible design, the method further includes: when the first model is in the foreground focus state and cannot run at its maximum speed, outputting a prompt message indicating insufficient running resources for the first model. This method can promptly remind the user of insufficient system resources, thus improving the user experience.
[0041] In one possible design, the method further includes: when the running state of at least one of the first model and the second model changes, redetermining the running order of the first model and the second model based on the first reference item and / or the third reference item; wherein the third reference item includes the state of the first model and the state of the second model after the running state of the at least one model changes; and running the first model and the second model according to the redetermined running order of the first model and the second model.
[0042] Based on this method, electronic devices can adjust the model scheduling strategy according to specific scenarios to achieve dynamic model scheduling, which is highly flexible and practical.
[0043] In a second aspect, this application provides an electronic device, the electronic device including a memory and one or more processors; wherein the memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by the one or more processors, the electronic device causes the electronic device to perform the method described in the first aspect or any possible design of the first aspect.
[0044] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when run on an electronic device, causes the electronic device to perform the method described in the first aspect or any possible design of the first aspect.
[0045] Fourthly, this application provides a computer program product comprising a computer program or instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect or any possible design of the first aspect.
[0046] Fifthly, this application provides a chip system including a processor and a memory, wherein the memory stores instructions; when the instructions are executed by the processor, they implement the methods described in the first aspect or any possible design of the first aspect. The chip system may be composed of chips or may include chips and other discrete devices.
[0047] The beneficial effects of the second to fifth aspects mentioned above can be found in the beneficial effects of the first aspect mentioned above, and will not be repeated here. Attached Figure Description
[0048] Figure 1 is a schematic diagram of the hardware architecture of an electronic device provided in an embodiment of this application;
[0049] Figure 2 is a schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;
[0050] Figure 3 is a schematic diagram of an application window provided in an embodiment of this application;
[0051] Figure 4 is a schematic diagram of a prompt message provided in an embodiment of this application;
[0052] Figure 5 is a schematic diagram of the architecture of a model scheduling system provided in an embodiment of this application;
[0053] Figure 6 is a schematic diagram of a model scheduling method provided in an embodiment of this application;
[0054] Figure 7 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.
[0056] In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.
[0057] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.
[0058] To efficiently and rationally schedule models in electronic devices, this application provides a model scheduling method and an electronic device. In this method, the electronic device can schedule models based on information such as the foreground / background state of the model's corresponding window, whether the model's corresponding window is focused, the model's inference latency or size, and whether the model belongs to the system model. This helps improve model running efficiency and user experience, and enables concurrent model scheduling. The electronic device can also dynamically manage the model's operation process based on the overall system status (e.g., model speed limiting) to prevent excessive consumption of electronic device resources by the model, thereby avoiding problems such as system freezes, overheating, and crashes.
[0059] The technical solutions provided in this application can be executed by any operable electronic device. In some embodiments of this application, the electronic device can be a portable device, such as a mobile phone, tablet computer, wearable device (e.g., smart glasses), in-vehicle terminal device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), smart home device (e.g., smart TV), smart robot, workshop equipment, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, or wireless terminal in smart home, flying equipment (e.g., smart robot, drone, airplane), etc. Among them, wearable devices are portable devices that users can wear directly on their bodies or integrate into their clothing or accessories.
[0060] In some embodiments of this application, the electronic device may also be a portable terminal device that includes other functions. Exemplary embodiments of the portable terminal device include, but are not limited to, carrying... Alternatively, it could be a portable terminal device with another operating system. The aforementioned portable terminal device could also be other portable terminal devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of this application, the aforementioned electronic device may not be a portable terminal device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).
[0061] Referring to Figure 1 below, the structure of a possible electronic device to which the method provided in the embodiments of this application is applicable will be described.
[0062] As shown in Figure 1, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a SIM card interface 195, etc.
[0063] The sensor module 180 may include a gyroscope sensor, an accelerometer, a proximity sensor, a fingerprint sensor, a touch sensor, a temperature sensor, a pressure sensor, a distance sensor, a magnetic sensor, an ambient light sensor, a barometric pressure sensor, a bone conduction sensor, etc.
[0064] It is understood that the electronic device 100 shown in Figure 1 is merely an example and does not constitute a limitation on the electronic device. Furthermore, the electronic device may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in Figure 1 can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits (ASICs).
[0065] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, memory, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of the electronic device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.
[0066] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0067] The execution of the model scheduling method provided in this application embodiment can be controlled by the processor 110 or by calling other components. For example, it can call the processing program of this application embodiment stored in the internal memory 121, or call the processing program of this application embodiment stored in a third-party device through the external memory interface 120 to control the wireless communication module 160 to perform data communication with other devices, thereby improving the intelligence and convenience of the electronic device 100 and enhancing the user experience. The processor 110 may include different devices. For example, when integrating a CPU and a GPU, the CPU and GPU can cooperate to execute the model scheduling method provided in this application embodiment. For example, some algorithms in the model scheduling method are executed by the CPU, and other algorithms are executed by the GPU to obtain faster processing efficiency.
[0068] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1. Display screen 194 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces (GUIs). For example, display screen 194 can display application windows, etc.
[0069] In this embodiment of the application, the display screen 194 can be a single flexible display screen, or it can be a splicing display screen composed of two rigid screens and a flexible screen located between the two rigid screens.
[0070] Camera 193 (a front-facing camera or a rear-facing camera, or a single camera that can function as both) is used to capture still images or videos. Typically, camera 193 may include a photosensitive element such as a lens assembly and an image sensor. The lens assembly includes multiple lenses (convex or concave lenses) for collecting light signals reflected from the object being photographed and transmitting these signals to the image sensor. The image sensor generates a raw image of the object being photographed based on the light signals.
[0071] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area can store the operating system, application program code (such as the functions corresponding to the scheme in this application), etc. The data storage area can store data created during the use of the electronic device 100.
[0072] The internal memory 121 may also store one or more computer programs corresponding to the algorithm of this application. The one or more computer programs are stored in the internal memory 121 and configured to be executed by one or more processors 110. The one or more computer programs include instructions that can be used to perform the various steps in the following embodiments.
[0073] In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.
[0074] Of course, the algorithm code of the embodiment of this application can also be stored in external memory. In this case, the processor 110 can run the algorithm code of the embodiment of this application stored in external memory through the external memory interface 120.
[0075] A touch sensor, also known as a "touch panel," can be located on the display screen 194. The touch sensor and display screen 194 together form a touch display screen, also called a "touch screen." The touch sensor detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 194.
[0076] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.
[0077] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.
[0078] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device. In this embodiment, the mobile communication module 150 can also be used for information interaction with other devices.
[0079] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.
[0080] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (WiFi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2. In this embodiment, the wireless communication module 160 can be used to establish connections with other electronic devices for data interaction. Alternatively, the wireless communication module 160 can be used to access access point devices, send control commands to other electronic devices, or receive data from other electronic devices.
[0081] In addition, the electronic device 100 can implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor, such as music playback and recording. The electronic device 100 can receive input from buttons 190, generating key signal inputs related to user settings and function control. The electronic device 100 can use a motor 191 to generate vibration alerts (such as vibration alerts for incoming calls). The indicator 192 in the electronic device 100 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. The SIM card interface 195 in the electronic device 100 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device 100.
[0082] It should be understood that in practical applications, the electronic device 100 may include more or fewer components than those shown in FIG. 1, and the embodiments of this application are not limited thereto. The illustrated electronic device 100 is merely an example, and the electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.
[0083] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. A layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example, as shown in Figure 2, this software architecture can be divided into four layers, from top to bottom: the application layer, the application framework layer (framework, FWK), the runtime and system libraries, and the (Linux) kernel layer.
[0084] The application layer is the top layer of the operating system and includes native operating system applications such as camera, gallery, calendar, Bluetooth, music, video, and messaging applications, as well as third-party applications. The applications discussed in this application embodiment are referred to as applications (APPs), which are software programs capable of performing one or more specific functions. Typically, multiple applications can be installed on an electronic device, such as camera applications and email applications. The applications mentioned below can be system applications pre-installed on the electronic device at the factory, or third-party applications downloaded by the user from the network or obtained from other electronic devices during the use of the electronic device.
[0085] Of course, for developers, they can write applications and install them into this layer. In one possible implementation, the application can be developed using the Java language, by calling the application programming interface (API) provided by the application framework layer. Developers can then interact with the underlying operating system (such as the kernel layer) through the application framework to develop their own applications.
[0086] The application framework layer provides the API and programming framework for the application layer. It can include predefined functions. The application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.
[0087] The window manager is used to manage window programs. The window manager can obtain the screen size, determine if a status bar is present, lock the screen (or display), and capture the screen, among other things.
[0088] Content providers are used to store and retrieve data, making that data accessible to applications. This data may include files (e.g., documents, videos, images, audio), text, and other information.
[0089] A view system includes visual controls, such as controls that display text, images, documents, and other content. View systems can be used to build applications. An interface in a display window can consist of one or more views. For example, a display interface including a text message notification icon could include a view that displays text and a view that displays images.
[0090] The phone manager provides communication functionality for electronic devices. The notification manager allows applications to display notification information in the status bar; it can be used to convey informative messages and can disappear automatically after a short pause without user interaction.
[0091] The runtime includes the core libraries and the virtual machine. The runtime is responsible for system scheduling and management.
[0092] The system's core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part is the system's core library. The application layer and application framework layer run in a virtual machine. Taking Java as an example, the virtual machine executes the Java files in the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.
[0093] The system library can include multiple functional modules. Examples include: a surface manager, a media library, a 3D graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), and an image processing library. The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.564, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D graphics.
[0094] The kernel layer provides core system services for the operating system, such as security, memory management, process management, network protocol stack, and driver models, all of which are implemented based on the kernel layer. The kernel layer also serves as an abstraction layer between the hardware and software stacks. This layer contains many drivers related to electronic devices, including: display drivers; keyboard drivers as input devices; Flash drivers for memory-based devices; camera drivers; audio drivers; Bluetooth drivers; and WiFi drivers.
[0095] It is important to understand that the functional services described above are just an example. In practical applications, electronic devices may be divided into more or fewer functional services based on other factors, or the functions of each service may be divided in other ways, or they may not be divided into functional services but work as a whole.
[0096] The solutions provided in this application will be described in detail below with reference to specific embodiments and accompanying drawings.
[0097] Example 1
[0098] Model scheduling refers to the technical strategy of coordinating and managing the operation of at least one model to achieve efficient resource utilization, task prioritization, and system performance optimization. Its core objective is to ensure the efficient and stable execution of tasks such as model training and inference through dynamic allocation of hardware resources and task orchestration. At least one model includes models currently awaiting execution and / or those already running. The models described in this application embodiment can include any model that can run on an electronic device, and no specific limitations are imposed in this application embodiment. For example, the models described in this application embodiment can include artificial intelligence (AI) models. AI models can include large models and small models. Exemplary examples include large language models, large code models, multimodal large models, etc., and small models can include image classification models, face recognition models, etc., which will not be listed exhaustively in this application embodiment.
[0099] In the model scheduling scheme provided in this application embodiment, electronic devices can uniformly manage models of different applications and services, and perform model scheduling based on the needs of different applications and services calling models. For ease of description, the following embodiments use the scenario of application calling models as an example to describe the scheme provided in this application embodiment. The scenario of service calling models is similar to the scenario of background application calling models, and will not be described again in this application embodiment.
[0100] Specifically, when an electronic device needs to schedule only a single model, the device can run that single model at full speed. In this method, the electronic device can provide the configuration and / or resources required for the single model to run at full speed (i.e., at maximum speed). When an electronic device needs to schedule multiple models, it can determine a scheduling strategy for multiple models based on the model type and / or the state of the application or application window corresponding to the model, and schedule multiple models based on the determined scheduling strategy. The model type can be categorized into large models and small models based on the number of model parameters. From the perspective of model ownership or invocation, the model type can include system models or non-system models. The state of the application or application window corresponding to the model can include foreground and background states; the foreground state can be further divided into focus and non-focus states. The application corresponding to the model is the application to which the inference task to be executed by the model belongs. The application window corresponding to the model is the application window to which the inference task to be executed by the model belongs, i.e., the application window currently invoking the model. The state of the application or application window corresponding to the model can also be considered as the state of the model; therefore, the states of the models described in the following embodiments all refer to the states of the application or application window corresponding to the model.
[0101] In one possible approach, the electronic device can determine the priority of each model based on its foreground / background status, focus status, model size, and whether it is a system model, and then perform model scheduling by combining these characteristics and the model priorities. For models with different priorities, the electronic device can run them sequentially according to their priority, thus ensuring a good user experience for high-priority models. Alternatively, the electronic device can run higher-priority models at a slightly reduced speed (i.e., high speed) from full speed while preventing lower-priority models from being "starved," thereby achieving concurrent model scheduling. This ensures a good user experience for high-priority services corresponding to high-priority models while preventing the shutdown of low-priority services corresponding to low-priority models. "Starved" refers to a runnable process being ignored indefinitely and unable to be scheduled for execution. In this embodiment, "not starved" means that a model is not killed (i.e., cleared from memory, stopped running) or suspended indefinitely (or for a long time). Optionally, in this embodiment, models can be kept from starving by running them at a certain speed. For models with the same priority, the electronic device can run them according to a fair competition principle. The fair competition method or rules can be pre-set, such as executing in a first-in, first-out (FIFO) order (i.e., the model used to execute the inference task received first runs first, and the model used to execute the inference task received later runs last), executing in a last-in, first-out (LIFO) order (i.e., the model used to execute the inference task received later runs first, and the model used to execute the inference task received first runs last), or executing according to inference latency (e.g., executing in order of inference latency from smallest to largest), etc. The specific fair competition rules are not limited in the embodiments of this application.
[0102] The above method will be described in detail below. For ease of description, the model type and / or the state of the application window corresponding to the model will be referred to as the model's reference item in the following embodiments.
[0103] The reference terms of the model will be explained in detail below.
[0104] In some embodiments of this application, the electronic device can simultaneously collect application-related information and user operation-related information, and determine the type of each model to be scheduled and / or the state of the application window corresponding to each model based on the collected information. The application may include system applications and non-system applications (e.g., third-party applications), and each application may have one or more models. Application-related information may include information related to application and model operation, such as the application window, the foreground / background state of the application window, the occlusion status of the application window, the application window currently being operated on by the user, the model called by the application, and the configuration information of the model called by the application. User operation-related information may include any information related to user operation, such as information indicating user operation, and the application window currently focused by the user can be determined based on the user operation-related information.
[0105] In this application, an application (or service) can have multiple application windows (or application interfaces), and each application window may generate one or more processes. A model's inference task at a certain moment or stage can be invoked by an application or a window of an application through a process, or it can be invoked by a system service or a third-party service. Therefore, in the model scheduling scheme provided in this application embodiment, a model corresponds to a unique application (or service) or application window during the execution of one inference task, while a model may correspond to different applications (or services) or application windows during the execution of different inference tasks.
[0106] 1) Large model or small model
[0107] In some embodiments of this application, the electronic device can classify the model into a large-latency model and a small-latency model based on the model's inference latency (which can also be understood as runtime latency). The large-latency model is the model with a relatively large inference latency, and the small-latency model is the model with a relatively small inference latency. For ease of description, in the embodiments of this application, the large-latency model is simply referred to as the large model, and the small-latency model is simply referred to as the small model.
[0108] In one possible approach, the large and small models in the electronic device can be pre-defined. For example, image generation models and semantic segmentation models can be pre-defined as large models, while super-resolution models and classification models can be pre-defined as small models.
[0109] In another possible approach, the large and small models in the electronic device can be determined according to a set rule. For example, the set rule may include: models with a size greater than or equal to a set value are defined as large models, and models with a size less than the set value are defined as small models. Here, model size refers to the physical space required to store all the model's parameters or the memory size occupied when the model runs or loads. For example, the set value could be 100 megabytes (MB), etc., and this application embodiment does not impose specific limitations. Of course, the set rule can also be other feasible rules for determining model size, which will not be listed one by one in this application embodiment.
[0110] In another possible implementation, the electronic device can identify models with inference latency greater than or equal to a set latency as large models and models with inference latency less than the set latency as small models. The inference latency of the model can be determined based on the actual runtime latency of the model within a historical time period, or it can be determined based on other information (such as the set inference latency of the model). This application embodiment does not impose specific limitations. As an optional implementation, the inference latency of the model can be determined based on the actual runtime latency of the model's most recent n runs, where n is a positive integer. For example, the model's inference latency can be any of the following: the average of the model's most recent n runtime latencies, the maximum value of the model's most recent n runtime latencies, the minimum value of the model's most recent n runtime latencies, or the product of the average of the model's most recent n runtime latencies and a set coefficient. As another optional implementation, the electronic device can persistently maintain a model latency list, which can record the inference latency of different models. The electronic device can determine whether the corresponding model belongs to a large or small model based on the inference latency recorded in this list. For a new model, the electronic device can first set it as a small model and add it to the model latency list. The inference latency recorded when the model is initially added to the model latency list can be the set hourly latency value. Subsequently, the electronic device can update the inference latency of the model recorded in the model latency list based on the actual runtime latency of the model. The method for determining the new inference latency based on the actual runtime latency of the model can be based on the actual runtime latency of the model's most recent n runs, as described in the above embodiments, and will not be repeated here.
[0111] 2) System model or non-system model
[0112] In one possible approach, electronic devices can be pre-configured to set their models as either system or non-system models. For example, the voice wake-up model of a system assistant can be set as a system model, while the image recognition model of a third-party application can be set as a non-system model.
[0113] In another possible approach, electronic devices can define the model of system application or system service calls as the system model, and the model of non-system application or non-system service calls as the non-system model.
[0114] 3) Front-end or back-end
[0115] In this embodiment, taking application window 1 as an example, when application window 1 is displayed in the foreground or when the application 1 providing application window 1 is running in the foreground (or can be understood as application window 1 running in the foreground), application window 1 can be considered to be in the foreground state. When application window 1 is not displayed in the foreground and the application 1 providing application window 1 is not running in the foreground (e.g., when application 1 is minimized), application window 1 can be considered to be in the background state. Optionally, for some applications, such as music playback applications and video playback applications, even if the application is minimized, the services provided by the application are still audible and / or perceptible to the user (hereinafter referred to as audible and perceptible). Therefore, audible and perceptible applications can have the same priority as foreground applications (non-focused). For simplicity, in this embodiment, the audible and perceptible state can be classified as the foreground state.
[0116] Example 1: In one example, using a laptop computer as an electronic device, as shown in Figure 3, a user can use two applications simultaneously. The user can use a document editing application (application window 1) to perform document-related knowledge-based question answering within application window 1, and a drawing application (application window 2) to generate an image using model B within application window 2. In this scenario, both application window 1 and application window 2 are in the foreground. When the user minimizes application window 2, application window 2 switches to the background.
[0117] 4) Focal or non-focal
[0118] In some electronic devices, there are multiple application windows that are in the foreground at the same time (for example, application window 1 and application window 2 in Figure 3 are both in the foreground). In this case, the focus window or non-focus window can be determined based on the occlusion of the application windows and relevant information of user operation.
[0119] As an optional implementation, the electronic device can designate the application window the user is currently interacting with as the focus window, regardless of whether application windows overlap. For example, as shown in Figure 3, when the user controls the cursor to click on application window 1, application window 1 becomes the focus window. In this scenario, application window 1 is the foreground focus window, and application window 2 is the foreground non-focus window. Subsequently, when the user controls the cursor to click on application window 2, application window 2 becomes the focus window.
[0120] In some embodiments of this application, when an electronic device needs to perform model scheduling, it can determine the reference terms for each model based on the above method, and then determine the priority of each model based on the reference terms. The method by which the electronic device determines the priority of a model based on its reference terms will be described in detail below.
[0121] In this embodiment of the application, the electronic device may follow at least one of the following rules 1 to 3 when determining the priority of the model:
[0122] Rule 1: Focus (i.e., front-end focus) > Non-focus on the front-end > Others
[0123] Among them, "focus" refers to the focus of the foreground, while "other" includes situations other than the focus of the foreground and non-focus of the foreground, such as the background, situations that are not perceived (i.e., the state cannot be determined), etc.
[0124] Since the foreground services are those that users are currently using or paying attention to, user experience must be prioritized. The foreground state can be further divided into focus and non-focus; therefore, focus windows have the highest priority, followed by non-focus windows, and so on. In other words, the model whose corresponding application window is the focus window has a higher priority than the model whose corresponding application window is the foreground non-focus window, and the model whose corresponding application window is the foreground non-focus window has a higher priority than the model whose corresponding application window is any other application window (i.e., application windows other than focus windows and foreground non-focus windows).
[0125] For example, in the scenario shown in Figure 3, since application window 1 is the focused window and application window 2 is a foreground non-focused window, the priority of model A called by application window 1 is higher than the priority of model B called by application window 2. In this scenario, when other applications running in the background of the electronic device also call model C, the priority of model C is lower than the priorities of model A and model B.
[0126] Rule 2: Smaller model > Larger model
[0127] Smaller models run faster and are typically those for which users expect an immediate response. Larger models run slower and have less stringent real-time requirements, allowing for interruptions from smaller models. Therefore, smaller models can be prioritized over larger models.
[0128] Rule 3: System model > Non-system model
[0129] In some embodiments of this application, the electronic device can determine the priority of multiple models based on any one of rules 1 to 3 above, or the electronic device can determine the priority of multiple models based on any multiple of rules 1 to 3 above. Specifically, when the electronic device determines the priority of multiple models based on any two of rules 1 to 3 above, the electronic device can first determine the priority of the multiple models based on the first rule of the two rules. If it is determined based on the first rule that there are different models with the same priority, the electronic device can further determine the priority of the different models based on the second rule of the two rules. When the electronic device determines the priority of multiple models based on three of rules 1 to 3 above, the electronic device can first determine the priority of the multiple models based on the first rule of the three rules. If it is determined based on the first rule that there are different models with the same priority, the electronic device can further determine the priority of the different models based on the second rule of the three rules. If it is determined based on the second rule that there are different models with the same priority, the electronic device can further determine the priority of the different models based on the third rule of the three rules. The order of the different rules in the above-mentioned multiple rules can be arbitrary, and this embodiment of the application does not impose any restrictions on this.
[0130] For example, if an electronic device determines the priority of multiple models based on three rules from rules 1 to 3 above, with the rules arranged in the order of rule 1, rule 2, and rule 3, the electronic device can prioritize determining the priority of multiple models based on rule 1. When rule 1 determines that there are different models with the same priority, the electronic device can further determine the priority of different models based on rule 2. When rule 2 determines that there are still different models with the same priority, the electronic device can further determine the priority of different models based on rule 3. Based on this method, the priority order determined by the electronic device for different models can be referred to in Table 1 below.
[0131] Table 1 Model Priority Order
[0132] In Table 1, numbers 1 through 12 represent the priority sequence numbers of the corresponding models, with smaller priority sequence numbers indicating higher priority. For example, number 1 indicates that the small system model in the focus state has the highest priority, and number 12 indicates that the large non-system model in other states has the lowest priority. The same applies to the other numbers, which will not be explained in detail here.
[0133] The above method will be illustrated by examples 2 and 3 below.
[0134] Example 2: In an example scenario, model A is a super-resolution small model called by the foreground non-focus window 1 of system application 1, model B is a classification small model called by the foreground non-focus window 2 of third-party application 2, model C is an image generation large model called by the focus window 3 of system application 3, and model D is a semantic segmentation large model called by the foreground non-focus window 4 of third-party application 2. Based on the above method, it can be seen that the priority order numbers of models A to D are 5, 6, 3, and 8, respectively, and the priority order of models A to D is C > A > B > D.
[0135] Example 3: In one example, based on the scenario described in Example 2, if the electronic device also contains a small model E called by a system application 4 running in the background, a small model F called by a third-party application 5 running in the background, a large model G called by a system application 6 running in the background, and a large model G called by a third-party application 7 running in the background, then based on the above method, the sequence numbers of models E to H are 9, 10, 11, and 12, respectively, and the priority order of models E to H is E > F > G > H. Therefore, the priority order of models A to H is C > A > B > D > E > F > G > H.
[0136] In some embodiments of this application, after the electronic device determines the priority of each model to be scheduled based on the above method, it can perform model scheduling by combining the reference items of each model and the priority of each model. The method of model scheduling by the electronic device is described in detail below.
[0137] In some embodiments of this application, for a single model, the electronic device can run the model at full speed directly. For multiple models with different priorities, as an optional implementation, the electronic device can run the multiple models sequentially based on their priority from high to low. As another optional implementation, the electronic device can run the model with the highest priority slightly at a reduced speed from full speed, while keeping other lower priority models running (i.e., not starving them). For multiple models with the same priority, the electronic device can run each model according to the principle of fair competition.
[0138] In the above method, the electronic device can adjust the model's running speed by adjusting hardware configuration (e.g., frequency modulation) and / or resource adjustment. The model's running speed refers to its inference speed. Adjusting hardware configuration means adjusting the configuration of the hardware used to run the model. For example, adjusting hardware configuration may include adjusting the clock frequency and cores of the processor used to run the model (e.g., CPU, GPU, neural processing unit (NPU)). A core can also be understood as a processing core; adjusting the cores in the processor used to run the model may include adjusting the core type and / or number. Adjusting resources means adjusting the resources used for model operation. For example, adjusting resources may include adjusting computing power resources, bandwidth resources, etc. For example, bandwidth resources may include double data rate (DDR) bandwidth, where DDR bandwidth is the amount of data that DDR memory can transfer in one second. In the above method, running the model at full speed means adjusting the model's inference speed to the maximum speed within the allowed range (e.g., a set maximum speed) by adjusting hardware configuration and / or resource adjustment, so that the model runs at full speed. The electronic device operates the model at a slightly reduced speed from its full speed. This means reducing the model's inference speed from the maximum speed by a set percentage and then running the model at that speed. For example, the set percentage could be 5%, meaning the model's inference speed could be reduced to 95% of its maximum speed, and then run at 95% of the maximum speed. This reserves some hardware capacity and resources for running other models. Optionally, the electronic device can prevent the model from "starving" by using a small amount of reserved hardware capacity and resources to run the model, thus keeping it running without shutting it down or stopping it.
[0139] For ease of description, the following embodiments will refer to operation at a slightly reduced speed from full speed as slightly reduced speed operation.
[0140] Example 4: Taking the other states mentioned above as background states as an example, and combining the scenarios in Table 2, we will provide an exemplary description of the specific strategy for model scheduling of electronic devices based on the above method.
[0141] Table 2 Model scheduling strategies in different scenarios
[0142] As shown in Table 2, in scenarios where a single model needs to be scheduled, regardless of whether the single model is in the focus, foreground non-focus, or background state, whether the model is a large model or a small model, or whether the model is a system model, the electronic device can run the single model at full speed.
[0143] As shown in Table 2, scenarios requiring the scheduling of multiple models can include:
[0144] Scenario 1: Scheduling multiple small models A to C
[0145] As shown in Table 2, the scheduling strategy for scenario 1 allows electronic devices to run multiple small models sequentially based on priority.
[0146] For example, when small models A to C are respectively the system small model A in the focus state, the system small model B in the background state, and the third-party small model C in the foreground non-focus state, the electronic device can determine the priority order of small models A to C as A > C > B based on the aforementioned method. Therefore, the electronic device can run small model A, small model C, and small model B in sequence.
[0147] Scenario 2: Scheduling multiple large system models D to F or multiple large non-system models D to F
[0148] As shown in Table 2, there are two scenarios in this case:
[0149] 2.1 When multiple large models have different foreground and background states, for example, when model D is in the foreground state and models E and F are in the background state, the electronic device can slightly reduce the speed of model D while keeping models E and F from running out of power. Alternatively, when models D and E are in the foreground state and model F is in the background state, the electronic device can determine the running order of models D and E according to the principle of fair competition, and slightly reduce the speed of models D and E according to the determined order, while keeping model F from running out of power.
[0150] 2.2 When multiple large models have the same foreground and background states, for example, when multiple large models D to F are all in the foreground state or all in the background state, the electronic device can determine the running order of large models D to F according to the principle of fair competition, and run large models D to F in the determined order.
[0151] Scenario 3: Scheduling a large system model G and a large non-system model H.
[0152] As shown in Table 2, there are two scenarios in this case:
[0153] 3.1 When a system model G and a non-system model H have different foreground and background states, the electronic device can slightly reduce the speed of the higher-priority model between G and H, while keeping the other, lower-priority model from starving. For example, when system model G is in the foreground non-focus state and non-system model H is in the focus state (a third-party model), the electronic device can determine that the priority of non-system model H is higher than that of system model G based on the aforementioned method. Therefore, the electronic device can slightly reduce the speed of non-system model H while keeping system model G from starving.
[0154] 3.2 When a system model G and a non-system model H have the same foreground and background states (i.e., both are in the foreground non-focus state or both are in the background state), the electronic device can slightly reduce the speed of the system model G while ensuring that the non-system model H does not become unresponsive. For example, when the system model G is in the foreground non-focus state and the non-system model H is a third-party model H in the foreground non-focus state, the electronic device can slightly reduce the speed of the system model G while ensuring that the non-system model H does not become unresponsive.
[0155] Scenario 4: Schedule at least one small model and at least one large model.
[0156] As shown in Table 2, there are four possible scenarios in this case:
[0157] 4.1 When an electronic device needs to schedule a small model I and a large model J, and the small model I is in the background and the large model J is in the foreground, the electronic device can slightly reduce the speed of the large model J and keep the small model I from starving.
[0158] 4.2 When an electronic device needs to schedule a small model K and multiple large models L~M, and the small model K is in the background and the multiple large models L~M are in the foreground, the electronic device can determine the execution order of the multiple large models L~M according to the principle of fair competition, and run the multiple large models L~M at a slightly slower speed according to the determined execution order, while keeping the small model L from starving.
[0159] 4.3 When an electronic device needs to schedule multiple small models N~O and one large model P, and all small models N~O are in the foreground and the large model P is in the background, as an optional implementation, the electronic device can determine the execution order of the multiple small models N~O according to the principle of fair competition, and run the multiple small models N~O at a slightly slower speed according to the determined execution order, while keeping the large model P from starving; as another optional implementation, the electronic device can run the multiple small models N~O sequentially according to their priority order, while keeping the large model P from starving.
[0160] 4.4 In cases other than those described in 4.1 to 4.3 above, where it is necessary to schedule at least one small model and at least one large model, the electronic device may execute at least one small model and at least one large model sequentially based on their priority order.
[0161] In some embodiments of this application, after the electronic device schedules multiple models based on the above method, when the foreground / background state or running state of any of the multiple models changes, and / or when a new model needs to be scheduled simultaneously, the model scheduling scenario of the electronic device changes, entering a new scenario. The electronic device can then re-determine the scheduling strategy for the multiple models in the new scenario based on the above method, and perform model scheduling in the new scenario based on the determined scheduling strategy. For example, when the electronic device needs to schedule a third-party large model A in the foreground state and a third-party large model B in the background state, based on the aforementioned method, it can be determined that the electronic device can slightly reduce the speed of third-party large model A while keeping third-party large model B from starving. After the electronic device starts running third-party large model A and third-party large model B, when the user switches third-party large model A to the background state and switches third-party large model B to the foreground state, the electronic device can re-determine the scheduling strategy in the new scenario (i.e., the scenario of running third-party large model A in the background state and large model B in the foreground state) based on the aforementioned method: slightly reduce the speed of third-party large model B while keeping third-party large model A from starving. Therefore, electronic devices can adjust the three-way large model A, which was originally running at a slightly reduced speed, to keep it from starving, and adjust the large model B, which was originally kept from starving, to run at a slightly reduced speed.
[0162] Based on the above method, electronic devices can dynamically and more efficiently schedule different models in different model scheduling scenarios, depending on the specific scenario. This can always ensure the experience of high-priority services and avoid excessive occupation or consumption of the hardware capabilities and resources of electronic devices during model operation.
[0163] Example 2
[0164] In some embodiments of this application, based on the solution described in Embodiment 1, or in scenarios where the electronic device uses other methods for model scheduling, the electronic device can also manage the process of running the model by considering the overall state of the electronic device. The overall state of the electronic device can include heat dissipation, temperature, power consumption, load (e.g., processor computing load), bandwidth (e.g., DDR bandwidth as described in the preceding embodiments), etc. When the overall resource consumption of the electronic device is too high, problems such as lag, freezing, overheating, and system crashes are likely to occur. Therefore, it is necessary to control the running speed of the model to avoid excessive consumption of the overall resources of the electronic device, thereby preventing problems such as lag, freezing, overheating, and system crashes.
[0165] Specifically, in the first possible approach, the electronic device can detect whether it is under high load based on the computing power load, bandwidth resource usage, and memory resource usage of its processors (e.g., CPU, GPU, NPU, etc.). When the electronic device detects a high load, it can manage the required scheduling models.
[0166] In this solution, when the electronic device is under high load, the running speed of the focused model and the smaller, non-focused models in the foreground must be guaranteed to ensure a good user experience for the focused window. Therefore, the electronic device can run the focused model and the smaller, non-focused models at full speed under high load. When it cannot support running the focused model and the smaller, non-focused models at full speed, the electronic device can prompt the user that resources are insufficient.
[0167] In this solution, when an electronic device is under high load, for the large model in the foreground non-focus state and other models, the electronic device can perform at least one of the following actions based on its load: slowing down, pausing, or stopping operation. It can also prompt the user that resources are insufficient. For example, under high load, the electronic device can first slow down the large model in the foreground non-focus state and other models, such as slowing down to a set speed or slowing down until the device is no longer under high load. If the electronic device remains under high load after slowing down the large model in the foreground non-focus state and other models, it can pause the operation of these models, such as pausing for a set duration or pausing until the device is no longer under high load before resuming execution. If the electronic device remains under high load after pausing the operation of these large models in the foreground non-focus state and other models, it can stop the operation of these models. Each adjustment to the model's running state (e.g., slowing down, pausing, stopping, etc.) can prompt the user that resources are insufficient.
[0168] The above scheme will be illustrated by example in Example 5 below.
[0169] Example 5: In one example, when an electronic device detects that it is in a high-load state, the electronic device can manage the scheduling model in the electronic device by referring to the method shown in Table 3 below.
[0170] Table 3 Model control methods under high load conditions
[0171] As shown in Table 3, the electronic device can operate at full speed without limiting the running speed of the focused model and the foreground non-focused small model under high load. In other words, the focused model and the foreground non-focused small model can still run at full speed even when the electronic device is under high load. Optionally, if the hardware capabilities and resources of the electronic device cannot support the full-speed operation of the focused model and the foreground non-focused small model, the electronic device can prompt the user that resources are insufficient.
[0172] As shown in Table 3, the electronic device can strive to run the large model in the focus state at full speed under high load. Specifically, the electronic device can provide resources to the large model in the focus state according to the goal of providing the resources required for its full-speed operation. When the resources provided by the electronic device are insufficient to guarantee the full-speed operation of the large model in the focus state, the electronic device can prompt the user that resources are insufficient. For example, in the scenario described in Example 1, when application window 1 is the focus window and model A called by application window 1 is the large system model, and model B called by application window 2 is the small system model, based on the model scheduling method described in Embodiment 1, the electronic device can first run model A at full speed, and then run model B at full speed. When the electronic device detects a high load state during the operation of model A, it can not limit the running speed of model A, that is, the electronic device can continue to run model A at full speed. When the electronic device determines that the resources provided to model A are insufficient to guarantee its full-speed operation, it can display the prompt message shown in Figure 4 to indicate that resources are insufficient.
[0173] As shown in Table 3, when an electronic device is under high load, it can first reduce the speed of the foreground non-focused system model. If the electronic device is still under high load after reducing the speed, it can pause the running of the foreground non-focused system model and also prompt the user that system resources are insufficient.
[0174] As shown in Table 3, when an electronic device is under high load, it can first reduce the speed of the foreground non-focus, non-system large model. If the electronic device is still under high load after reducing the speed, it can prompt the user that resources are insufficient.
[0175] As shown in Table 3, when an electronic device is under high load, it can first slow down the operation of models in other states (such as background states). If the electronic device is still under high load after slowing down, it can pause the operation of models in other states. If the electronic device is still under high load after pausing, it can stop the operation of models in other states.
[0176] In the second possible approach, the electronic device can manage the models that need to be scheduled when it detects that the temperature exceeds a set temperature (i.e., a high-temperature state). For example, the electronic device can directly detect the temperature using a temperature sensor, or it can determine the temperature using heat dissipation information. This application embodiment does not limit the specific temperature detection method.
[0177] In this scheme, since small models generally run faster—for example, some can complete in milliseconds—they do not require restrictions. Therefore, when the electronic device detects that the temperature exceeds a set temperature, it can continue running small models without limiting their speed. Large models, on the other hand, run much slower, sometimes taking several seconds to tens of seconds. Therefore, temperature-based management of the electronic device is necessary. Thus, when the electronic device detects that the temperature exceeds a set temperature, it can slow down the operation of large models to prevent the device's temperature from rising too quickly. Optionally, the electronic device can also prevent new large models from running.
[0178] For example, the specific way for an electronic device to reduce the speed of a large model can be to reduce the speed to a set speed, or to reduce the speed by a set percentage before running the large model, or other feasible methods. In this application embodiment, there is no limitation on the specific way for the electronic device to reduce the speed of a large model.
[0179] The above scheme will be illustrated by example in the following example 6.
[0180] Example 6: In one example, when the electronic device detects that the temperature exceeds the set temperature (i.e., high temperature state), the electronic device can refer to the method shown in Table 4 below to control the scheduling model in the electronic device.
[0181] Table 4 Model control methods under high temperature conditions
[0182] As shown in Table 4, in scenarios where an electronic device schedules a single model, when the single model is a small model, the electronic device can operate at full speed without limiting the running speed of the small model under high temperature conditions. When the single model is a large model, the electronic device can reduce the running speed of the large model under high temperature conditions.
[0183] As shown in Table 4, in scenarios where an electronic device schedules multiple models, when all models are small models (i.e., multiple models are multiple small models), the electronic device can not limit the running speed of the small models under high temperature conditions, and each small model can run at full speed. When all models are large system models, or when all models are non-large system models, or when multiple models include both large system models and non-large system models, or when multiple large models include both small models and large models, the electronic device can reduce the running speed of multiple models and can prevent new large models from running. Optionally, when there is a new large model that needs to be run, the electronic device can also prompt the user that the current electronic device temperature is too high to run the large model.
[0184] In the third possible solution, the electronic device can manage the models it needs to schedule when it detects that the power consumption exceeds a set power consumption level (i.e., a high power consumption state). In this solution, the electronic device can slow down the operation of large models when it detects that the power consumption exceeds the set power consumption level, thereby preventing the power consumption of the electronic device from increasing too rapidly. Optionally, the electronic device can also prevent new large models from running. Specifically, the model management method when the electronic device detects that the power consumption exceeds the set power consumption level can refer to the model management method when the temperature exceeds the set temperature described in the second possible solution above. For example, the model management method shown in Table 4 can be used for model management in a high power consumption state, which will not be detailed here.
[0185] In the fourth possible solution, the electronic device can manage the model to be scheduled when it detects that the battery level is lower than a set level. This method can refer to the model management method described in the second possible solution above, specifically the method for managing the model when the temperature exceeds a set level. For example, the model management method shown in Table 4 can be used for model management in scenarios where the battery level is lower than a set level; details will not be elaborated here.
[0186] Of course, the first to fourth possible solutions described above can be combined. For example, when an electronic device detects that the condition for triggering model control is met as described in at least one of the first to fourth possible solutions, it can perform model control based on at least one of the first to fourth possible solutions. No specific limitations are made in the embodiments of this application.
[0187] It should be understood that the specific control methods provided in the examples of the above possible solutions (such as the specific control methods shown in Tables 3 and 4) are merely illustrative examples of the control methods available in the solutions provided in this application, and do not limit the specific control methods available in the solutions of this application. The control methods or the same technical concepts provided in any of the above possible solutions can also be applied in other possible scenarios, or can be combined with the methods provided in other possible solutions. Specifically, they can be applied in conjunction with specific embodiments or specific scenarios, and will not be listed and described one by one in this application.
[0188] Based on the above methods, electronic devices can manage and control the models that need to be scheduled within the electronic device by combining the overall system status, thereby avoiding problems such as lag, freezing, overheating, and crashes caused by excessive consumption of the hardware capabilities and resources of the electronic device during model operation.
[0189] Based on the above embodiments and the same technical concept, this application also provides a system applicable to the model scheduling method. As shown in Figure 5, this system can be deployed in an electronic device. The system may include at least one application, a model management module, a model inference framework, inference hardware, a scheduling awareness module, and a system scheduling module.
[0190] At least one application may include system applications and / or third-party applications within the electronic device. Each application may invoke a model within the electronic device to perform inference tasks.
[0191] The model management module can be used to access and manage models in electronic devices, including models of system applications and models of third-party applications.
[0192] The scheduling awareness module can be used to determine the models invoked by applications in an electronic device, collect relevant information about applications, user operations, and models, and determine the type of each model that needs to be scheduled and / or the state of the application window corresponding to each model based on the collected information. The scheduling awareness module can also be used to collect information related to the overall state of the electronic device. The information collected by the scheduling awareness module can be referred to the relevant descriptions in the foregoing embodiments, and will not be detailed here.
[0193] As one optional implementation, when an application calls a model, it can directly register with the scheduling awareness module and send the inference tasks that the model needs to execute. Alternatively, when an application calls a model, it can register with the scheduling awareness module through the model management module and send the inference tasks that the model needs to execute. Registration is the process by which the application obtains permission to use the model. The scheduling awareness module can determine the model to be called by the application based on the registration made by the application.
[0194] In some embodiments of this application, as shown in FIG5, the scheduling perception module may include an application status acquisition module and a system status acquisition module. The application status acquisition module can be used to collect information related to applications, user operations, and models. The system status acquisition module can be used to collect information related to the overall status of the electronic device.
[0195] The system scheduling module can determine the priority of each model based on the reference items for each model that needs to be scheduled (as described in the foregoing embodiments) determined by the scheduling awareness module, and determine the model scheduling strategy based on the reference items and the priority of each model, and instruct the model scheduling strategy to the model inference framework. The system scheduling module can also determine the control strategy for the models scheduled in the electronic device based on the overall system status collected by the scheduling awareness module, and instruct the control strategy to the model inference framework.
[0196] The model inference framework can be used to run models invoked by applications through inference hardware according to the model scheduling strategy determined by the system scheduling module, so that the models can execute inference tasks from the applications, thereby realizing model scheduling. For details, please refer to the relevant methods in Embodiment 2 above, which will not be repeated here. The model inference framework can also be used to manage and control the models scheduled in electronic devices according to the control strategy determined by the system scheduling module. For details, please refer to the relevant methods in Embodiment 2 above, which will not be repeated here.
[0197] Inference hardware can be used to run models under the control of a model inference framework and to perform inference tasks through the models. For example, inference hardware may include processors for running models, such as CPUs, GPUs, NPUs, etc.
[0198] In some embodiments of this application, when the electronic device adopts the software architecture shown in FIG2, at least one of the above-mentioned applications can be deployed at the application layer, and the model management module, model inference framework, scheduling awareness module and system scheduling module can be deployed at the application framework layer.
[0199] It should be understood that the system architecture described above is merely an example. In practical applications, modules can be added or removed, or modules can be split or merged. This application does not impose specific limitations. The division of functional modules in the system architecture described above is illustrative and only represents a logical functional division. In actual implementation, there can be other division methods, and the specific functions performed by each functional module can also be divided in other ways. For example, electronic devices can also be divided into more or fewer functional services according to other factors, or the functions of each service can be divided in other ways, or they can not be divided into functional services but work as a whole. The names of the functional modules described above are merely examples, and the names of the functional modules in the embodiments of this application are not limited.
[0200] It should be noted that the application scenarios provided in the above embodiments are merely illustrative examples of the applicable scenarios of the embodiments of this application, and do not limit the applicable scenarios of the solutions of this application. Some methods or the same technical concepts provided in any of the above embodiments can also be applied in other embodiments or other scenarios, or can be combined with the methods provided in other embodiments. Specifically, they can be applied in combination with specific embodiments or specific scenarios, and will not be listed and described one by one in this application.
[0201] Based on the above embodiments and the same technical concept, this application also provides a model scheduling method, as shown in FIG6, which may include:
[0202] S601: The electronic device determines the operating order of the first model and the second model based on the first reference item and / or the second reference item; wherein the first reference item includes the type of the first model and the type of the second model, and the second reference item includes the operating state of the first model and the operating state of the second model.
[0203] The electronic device includes a first model and a second model. Applications or services within the electronic device can invoke the first model and the second model to perform inference tasks.
[0204] In some embodiments of this application, any model type may conform to at least one of the following: a system model or a non-system model; a large model or a small model; wherein, a large model is a model with a size greater than or equal to a set size, and a small model is a model with a size less than a set size; or, a large model is a model with an inference latency greater than or equal to a set latency, and a small model is a model with an inference latency less than a set latency. Inference latency, also known as runtime latency, refers to the total time required for the model to generate a complete response from the start of inference during the execution of the inference task.
[0205] Optionally, in the above method, the size of the model can be determined based on the number of parameters in the model. Therefore, a large model is a model whose size is greater than or equal to the set size, which can also be understood as a model whose number of parameters is greater than or equal to the set number of parameters. Similarly, a small model is a model whose size is less than the set size, which can also be understood as a model whose number of parameters is less than the set number of parameters.
[0206] Optionally, the inference latency of any model in the above method is the average, maximum, or minimum inference latency of any model performing at least one inference task within a historical time period. The definitions of system model, non-system model, large model, and small model in this method can be found in the descriptions in Embodiment 1 above, and will not be repeated here.
[0207] Optionally, the method for determining whether an electronic device's model belongs to a large model or a small model can refer to the method for distinguishing between large and small models described in the aforementioned Embodiment 1, and will not be repeated here.
[0208] In some embodiments of this application, the priority of the system model may be higher than that of the non-system model, and / or the priority of the small model may be higher than that of the large model.
[0209] In some embodiments of this application, the running state of any model can be a foreground state or a state other than a foreground state (such as the background state described in the foreground embodiments). Optionally, the foreground state may include a foreground focus state or a foreground non-focus state. The definition of each state can be referred to the description in the foreground embodiment one, and will not be repeated here.
[0210] In one example, when the first model and the second model are respectively model A and model B as described in Example 1, both the first model and the second model are in the foreground state. When the user controls the cursor to click on application window 1, the first model is in the foreground focused state, and the second model is in the foreground non-focused state.
[0211] In some embodiments of this application, the state of any model is the same as the state of the application, application window, or service that calls the model. Specifically, when an application has a foreground application window and that application window calls the model, the state of the model is the same as the state of the application window. When a background application calls the model (e.g., a scenario where the application does not have a foreground application window but calls the model), the state of the model is the same as the state of the background application. When a service (e.g., a system service or a third-party service) calls the model, the state of the model is the same as the state of the service.
[0212] In some embodiments of this application, the priority of the model in the foreground state may be higher than the priority of the models in other states. The priority of the model in the foreground focus state may be higher than the priority of the model in the foreground non-focus state.
[0213] In one possible approach, the process by which the electronic device determines the operating order of the first model and the second model based on a first reference item and / or a second reference item may include: the electronic device first determining the priorities of the first model and the second model based on the first reference item and / or the second reference item, and then determining the operating order based on the priorities of the first model and the second model. Wherein, when the priorities of the first model and the second model are different, the operating order may be the same as the order of their priorities; or, when the priorities of the first model and the second model are different, the operating order may be parallel operation; or, when the priorities of the first model and the second model are the same, the operating order may be an order determined according to a set rule. Optionally, the set rule may be the fair competition rule described in Embodiment 1.
[0214] In one possible approach, the electronic device can identify a specific model scheduling scenario based on a first reference term and / or a second reference term, and adopt the model execution order applicable to that scenario. For example, when the electronic device needs to schedule a first model and a second model, the specific model scheduling scenario and the applicable model execution order may include any of the following:
[0215] 1) When both the first model and the second model are small models, the running order of the first model and the second model can be the same as the order of priority of the first model and the second model.
[0216] For example, when the first model and the second model are the small model A and the small model C described in Example 4, the priority of the first model and the second model is in the order that the priority of the first model is higher than that of the second model.
[0217] 2) When both the first model and the second model are large models, and the running states of the first model and the second model are different, the running order of the first model and the second model can be parallel.
[0218] For example, when the first model and the second model are the large model D and the large model F described in Example 4, respectively, the first model and the second model can run in parallel.
[0219] 3) When both the first model and the second model are large models, and the running states of the first model and the second model are the same, the running order of the first model and the second model can be determined according to a set rule. Optionally, this set rule can be the fair competition rule described in Example 1.
[0220] For example, when the first model and the second model are respectively the large model D and the large model E described in Example 4, or when the first model and the second model are respectively the large model L and the large model M described in Example 4, the running order of the first model and the second model can be determined in a fair competition manner.
[0221] 4) When the first model belongs to the large model and the non-system model, and the second model belongs to the large model and the system model, the first model and the second model can be run in parallel.
[0222] For example, when the first model and the second model are the large model H and the large model G described in Example 4, respectively, the first model and the second model can run in parallel.
[0223] 5) When the first model is a small model and the second model is a large model, and the first model is in another running state while the second model is in the foreground running state, the first model and the second model can run in parallel.
[0224] For example, when the first model and the second model are the large model I and the large model G described in Example 4, respectively, the first model and the second model can run in parallel.
[0225] 6) When the first model is a small model and the second model is a large model, and the first model is not in any other running state while the second model is in the foreground running state, the running order of the first model and the second model can be the same as the order of priority of the first model and the second model.
[0226] In the above scenario, the priority of the first model and the second model is determined based on the first reference term and / or the second reference term. When the execution order of the first model and the second model is the same as the order of their priorities, the execution order of the first model and the second model can be determined according to the set rules.
[0227] The methods described in the above scenarios can also refer to the methods described in Example 4 of the aforementioned Embodiment 1, which will not be detailed here.
[0228] Optionally, the method for determining the priority of the first model and the second model in the above method can refer to the method for determining the model priority described in the aforementioned Embodiment 1, and will not be described in detail here.
[0229] In some embodiments of this application, the first model and the second model described above may be some of a plurality of models that need to be scheduled. In this scenario, the method for determining the running order of any two models among the plurality of models can refer to the method for determining the running order of the first model and the second model. The plurality of models that need to be scheduled may include models that are currently running and / or models that are yet to be run.
[0230] In some embodiments of this application, the electronic device can dynamically adjust the running order of different models based on the above method and in combination with specific scenarios. For example, when the running state of at least one of the first and second models changes, the electronic device redetermines the running order of the first and second models based on a first reference item and / or a third reference item, and runs the first and second models according to the redetermined running order. The third reference item may include the state of the first and second models after the running state of at least one model changes. As another example, when the electronic device needs to schedule a new model, it can also redetermine the running order of different models based on the above method and run the different models according to the redetermined order. Based on this method, during the operation of the first and second models, when the third model is called, the electronic device can determine the running order of the first, second, and third models based on a fourth reference item and / or a fifth reference item, and run the first, second, and third models according to this running order. The fourth reference item may include the type of the first model, the type of the second model, and the type of the third model, and the fifth reference item may include the running state of the first model, the running state of the second model, and the running state of the third model. In this scenario, the method for determining the running order of the first model, the second model, and the third model by the electronic device can refer to the method for determining the running order of the first model and the second model by the electronic device, and will not be repeated here.
[0231] S602: The electronic device runs the first and second models based on the running sequence.
[0232] In some embodiments of this application, when the running order of the first model and the second model determined based on the above method is the same as the priority order of the first model and the second model, the specific way in which the electronic device runs the first model and the second model based on the running order can be: the first model and the second model are executed sequentially according to the priority order of the first model and the second model.
[0233] In some embodiments of this application, when the running order of the first model and the second model determined by the above method is the same as the order determined by a set rule, the specific way in which the electronic device runs the first model and the second model according to the running order can be: executing the first model and the second model in sequence according to the order of the first model and the second model determined by the set rule.
[0234] In some embodiments of this application, when the running order of the first model and the second model determined by the above method is parallel and the priority of the first model is higher than that of the second model, the specific way in which the electronic device runs the first model and the second model according to the running order can be any of the following:
[0235] 1) When the electronic device can support running the first model and the second model at full speed at the same time, the electronic device can run the first model and the second model at full speed at the same time, thereby achieving the effect of running the first model and the second model in parallel.
[0236] "Running the first model at full speed" means running the first model at its maximum speed, and "running the second model at full speed" means running the second model at its maximum speed. The electronic device supports running both the first and second models at full speed simultaneously, meaning it has the hardware capability and resources to do so.
[0237] 2) When the electronic device cannot support running the first model and the second model at full speed at the same time, the electronic device can run the first model at the first running speed and the second model at the second running speed, which can also achieve the effect of running the first model and the second model in parallel; wherein, the first running speed is higher than the second running speed.
[0238] Optionally, the first running speed can be the speed obtained by reducing the maximum running speed of the first model by a set percentage. The second running speed can be the running speed that ensures the second model does not fail. Optionally, the second running speed can be less than or equal to the set speed.
[0239] In some embodiments of this application, when an electronic device detects that certain control conditions are met during the operation of a first model and / or a second model, it can perform control processing to manage the model's operation. These control conditions may include at least one of the following: the electronic device is in a set high-load state; the temperature of the electronic device is higher than a set temperature; the power consumption of the electronic device is higher than a set power consumption; or the battery level of the electronic device is lower than a set battery level. Optionally, the set high-load state may specifically include at least one of the following: the load (e.g., computing power load) of the processor used to run the model in the electronic device is higher than or equal to a set load; the bandwidth of the processor used to run the model in the electronic device is higher than or equal to a set bandwidth; or the free memory capacity of the electronic device is less than or equal to a set capacity. Optionally, the control conditions may also include any of the following: the currently running model is in another state; the currently running model is in a foreground state and is a large model; or a large model exists within the currently running model. In other words, the electronic device can perform control processing for specific operating states and / or types of models.
[0240] In some embodiments of this application, the above-described control processing may include at least one of the following:
[0241] 1) Reduce the running speed of the first model and / or the second model.
[0242] Optionally, when performing control processing, the electronic device may first reduce the running speed of the first model and / or the second model.
[0243] 2) Pause the operation of the first model and / or the second model.
[0244] Optionally, the electronic device may suspend the operation of the first model and / or the second model if the control conditions are still met after reducing the operating speed of the first model and / or the second model.
[0245] Optionally, the operation can be paused for a set duration, or paused until the control conditions are no longer met.
[0246] 3) Stop running the first model and / or the second model.
[0247] Optionally, the electronic device may stop running the first model and / or the second model only if the control conditions are still met after pausing the operation of the first model and / or the second model.
[0248] 4) Do not run new models.
[0249] In some embodiments of this application, when the above-mentioned control conditions include the electronic device being in a high-load state, the electronic device may also output a prompt message to indicate that the first model and / or the second model has insufficient operating resources when performing the above-mentioned control process (for example, the prompt message shown in FIG4 can be referred to).
[0250] In some embodiments of this application, when the electronic device is running the first model and the second model, and it is determined that the running state of any currently running model is the foreground focus state, and the electronic device cannot support the running of any model at the maximum speed, the electronic device can output a prompt message to indicate that the running resources of any model are insufficient.
[0251] Optionally, the method for controlling and managing the above-mentioned electronic devices can also be implemented with reference to the method provided in Embodiment 2 above, and will not be repeated here.
[0252] The specific steps performed by the electronic device in the above method can be found in the relevant descriptions in Embodiment 1 and Embodiment 2 above, and will not be repeated here.
[0253] Based on the above embodiments and the same technical concept, this application also provides an electronic device for implementing the model scheduling method provided in this application. As shown in FIG7, the electronic device 700 may include: a memory 701, one or more processors 702, and one or more computer programs (not shown in the figure). The above-mentioned devices can be coupled through one or more communication buses 703. Optionally, the electronic device 700 may also include a display screen 704.
[0254] The memory 701 stores one or more computer programs (code), and the one or more computer programs include computer instructions; one or more processors 702 call the computer instructions stored in the memory 701, causing the electronic device 700 to execute the model scheduling method provided in the embodiments of this application.
[0255] In a specific implementation, memory 701 may include high-speed random access memory and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 701 may store an operating system (hereinafter referred to as the system), such as embedded operating systems like Android, iOS, Windows, or Linux. Memory 701 can be used to store implementation programs of the embodiments of this application. Memory 701 may also store network communication programs, which can be used to communicate with one or more additional devices, one or more user devices, or one or more network devices.
[0256] One or more processors 702 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs according to the present application.
[0257] Display screen 704 is used to display application interfaces and other related user interfaces.
[0258] It should be noted that Figure 7 is only one implementation of the electronic device 700 provided in this application embodiment. In actual applications, the electronic device 700 may include more or fewer components. For details, please refer to the specific structure and description shown in Figure 1. No limitations are imposed here.
[0259] Based on the above embodiments and the same technical concept, this application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method provided in the above embodiments.
[0260] Based on the above embodiments and the same technical concept, this application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer performs the method provided in the above embodiments.
[0261] Based on the above embodiments and the same technical concept, this application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer performs the method provided in the above embodiments.
[0262] The methods provided in this application can be implemented, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs), or semiconductor media (e.g., SSDs), etc.
[0263] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A model scheduling method applied to electronic devices, characterized in that, The method includes: Based on a first reference term and / or a second reference term, the running order of the first model and the second model is determined; wherein, the first reference term includes the type of the first model and the type of the second model, and the second reference term includes the running state of the first model and the running state of the second model; The first model and the second model are run according to the aforementioned running order.
2. The method as described in claim 1, characterized in that, The type of any model meets at least one of the following: System model or non-system model; A large model or a small model; wherein the large model is a model with a size greater than or equal to a set size, and the small model is a model with a size less than the set size; or, the large model is a model with an inference latency greater than or equal to a set latency, and the small model is a model with an inference latency less than the set latency.
3. The method as described in claim 2, characterized in that, The system model has a higher priority than the non-system model.
4. The method as described in claim 2 or 3, characterized in that, The smaller model has a higher priority than the larger model.
5. The method according to any one of claims 2 to 4, characterized in that, The running state of any model is either the foreground state or any other state besides the foreground state.
6. The method as described in claim 5, characterized in that, The model in the foreground state has a higher priority than the models in other states.
7. The method as described in claim 5 or 6, characterized in that, The foreground state includes either the foreground focus state or the foreground non-focus state.
8. The method as described in claim 7, characterized in that, The model for the foreground focus state has a higher priority than the model for the foreground non-focus state.
9. The method according to any one of claims 5 to 8, characterized in that, When both the first model and the second model belong to the small model, the running order is the same as the priority order of the first model and the second model; or When both the first model and the second model belong to the large model, and the running states of the first model and the second model are different, the running order is parallel running; or When both the first model and the second model belong to the large model, and the running state of the first model and the running state of the second model are the same, the running order is the order determined according to the set rules. or When the first model belongs to both the large model and the non-system model, and the second model belongs to both the large model and the system model, the running order is parallel execution; or When the first model belongs to the small model, the second model belongs to the large model, and the first model is in the "other" running state while the second model is in the "foreground" running state, the running order is parallel running; or When the first model belongs to the small model and the second model belongs to the large model, and the first model's running state is not in the other state and the second model's running state is in the foreground state, the running order is the same as the priority order of the first model and the second model. The priority of the first model and the second model is determined based on the first reference term and / or the second reference term.
10. The method according to any one of claims 1 to 8, characterized in that, Determining the operating order of the first model and the second model based on the first reference term and / or the second reference term includes: Based on the first reference item and / or the second reference item, determine the priority of the first model and the second model; The execution order is determined based on the priorities of the first model and the second model.
11. The method as described in claim 10, characterized in that, When the priorities of the first model and the second model are different, the running order is the same as the order of priority between the first model and the second model. or When the priorities of the first model and the second model are different, the running order is parallel execution; or When the first model and the second model have the same priority, the running order is determined according to the set rules.
12. The method as described in claim 9 or 11, characterized in that, The step of running the first model and the second model based on the running order includes: When the running order is parallel running and the priority of the first model is higher than the priority of the second model, the first model runs at a first running speed and the second model runs at a second running speed; wherein the first running speed is higher than the second running speed.
13. The method as described in claim 12, characterized in that, The first operating speed is the speed obtained by reducing the maximum operating speed of the first model by a set percentage.
14. The method according to any one of claims 1 to 13, characterized in that, The method further includes: When a set control condition is detected during the operation of the first model and / or the second model, control processing is performed; wherein the control processing includes at least one of the following: Reduce the running speed of the first model and / or the second model; Pause the operation of the first model and / or the second model; Stop running the first model and / or the second model; Do not run new models.
15. The method as described in claim 14, characterized in that, The control conditions include at least one of the following: The electronic device is in a set high-load state; The temperature of the electronic device is higher than the set temperature; The power consumption of the electronic device is higher than the set power consumption. The battery level of the electronic device is lower than the set battery level.
16. The method as described in claim 15, characterized in that, The method further includes: If the control conditions include the electronic device being in the high-load state, when performing the control process, a prompt message is output to indicate that the first model and / or the second model has insufficient operating resources.
17. The method according to any one of claims 14 to 16, characterized in that, The suspension of the first model and / or the second model includes: If the control conditions are still met after reducing the running speed of the first model and / or the second model, the first model and / or the second model shall be suspended.
18. The method according to any one of claims 14 to 17, characterized in that, The stopping of the first model and / or the second model includes: If the control conditions are still met after suspending the operation of the first model and / or the second model, then the operation of the first model and / or the second model shall be stopped.
19. The method according to any one of claims 1 to 18, characterized in that, The method further includes: When the running state of at least one of the first model and the second model changes, the running order of the first model and the second model is re-determined based on the first reference item and / or the third reference item; wherein, the third reference item includes the state of the first model and the state of the second model after the running state of the at least one model changes; Run the first model and the second model in the redefined running order.
20. An electronic device, characterized in that, The electronic device includes a memory and one or more processors; The memory is used to store computer program code, which includes computer instructions; when the computer instructions are executed by the one or more processors, the electronic device performs the method as described in any one of claims 1 to 19.
21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 19.
22. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 19.
