Resource scheduling method, electronic device and storage medium

Abstract

The present application relates to a resource scheduling method, electronic device and storage medium. The embodiments of the present application include: obtaining an intelligent application processing request; obtaining current resource usage information; matching an intelligent application instance according to the intelligent application processing request; creating a task according to the resource usage information and the intelligent application instance to process the intelligent application processing request. The embodiments of the present application can adaptively schedule system resources, use the idle resources of the system to call the intelligent application model, improve resource utilization, thereby effectively improving the flexibility of the intelligent application model deployment and the overall operating efficiency of the device, and does not affect the operation of the existing functional modules of the device.

Description

Technical Field

[0001] The present application relates to the field of intelligent application technology, and in particular to a resource scheduling method, electronic device, and storage medium. Background Art

[0002] Intelligent applications refer to the application of intelligent technologies and management, primarily driven by artificial intelligence and driven by big data intelligence. By establishing an intelligent application model database and storing commonly used intelligent application models in the database, the corresponding intelligent application models in the database can be directly called in different intelligent application scenarios (for example, smart home, smart transportation, smart education, smart retail, etc.) without having to repeat the intelligent application model creation process. This accelerates the deployment of intelligent applications and is of great significance to the deployment and promotion of intelligent applications.

[0003] However, calling intelligent application models requires certain system resources (for example, the number of CPU cores, GPU, memory, chip resources, etc.). Known intelligent application models are deployed on devices with limited system resources. Due to the limited system resources of the device, it is easy to cause problems such as poor flexibility in the deployment of intelligent application models, low overall operating efficiency of the device, and impact on the operation of existing functional modules of the device. Summary of the Invention

[0004] The embodiments of the present application provide a resource scheduling method, electronic device and storage medium, which can adaptively schedule system resources, use the system's idle resources to call intelligent application models, improve resource utilization, thereby effectively improving the flexibility of intelligent application model deployment and the overall operating efficiency of the device, without affecting the operation of existing functional modules of the device.

[0005] In a first aspect, an embodiment of the present application provides a resource scheduling method, including:

[0006] Get intelligent application processing request;

[0007] Get current resource usage information;

[0008] Matching intelligent application instances based on intelligent application processing requests;

[0009] Create tasks based on resource usage information and smart application instances to handle smart application processing requests.

[0010] In a second aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a program stored in the memory and runnable on the processor. When the program is executed by the processor, the resource scheduling method of some embodiments of the first aspect of the present application is implemented.

[0011] In a third aspect, an embodiment of the present application provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the resource scheduling method of some embodiments of the first aspect of the present application.

[0012] The embodiment of the present application includes: obtaining an intelligent application processing request; obtaining current resource usage information; matching an intelligent application instance according to the intelligent application processing request; and creating a task based on the resource usage information and the intelligent application instance to process the intelligent application processing request. The embodiment of the present application can adaptively schedule system resources, utilize the system's idle resources to call the intelligent application model, improve resource utilization, and thus effectively enhance the flexibility of intelligent application model deployment and the overall operating efficiency of the device without affecting the operation of the device's existing functional modules.

[0013] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flowchart of resource scheduling provided by an embodiment of the present application;

[0015] Figure 2 yes Figure 1 Flowchart of an embodiment before step S0130;

[0016] Figure 3 yes Figure 2 Flowchart of an embodiment of step S0220;

[0017] Figure 4 yes Figure 1 Flowchart of another embodiment before step S0130;

[0018] Figure 5 yes Figure 3 Flowchart of an embodiment of step S0330;

[0019] Figure 6 yes Figure 1 Flowchart of an embodiment of step S0140;

[0020] Figure 7 yes Figure 6 Flowchart of an embodiment of step S0640;

[0021] Figure 8 yes Figure 1Flowchart of an embodiment before step S0140;

[0022] Figure 9 yes Figure 1 Flowchart of another embodiment before step S0140;

[0023] Figure 10 This is a system block diagram of a resource scheduling system provided by an embodiment of the present application;

[0024] Figure 11 yes Figure 10 Functional diagram of the resource monitoring unit. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of this application more clearly understood, this application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. In the absence of conflict, the embodiments and features in the embodiments of this application can be combined with each other in any manner.

[0026] It should be noted that although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown in the flowcharts. The terms "first," "second," and the like in the specification, claims, and drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0027] In the embodiment of the present application, the intelligent application model includes a face / fingerprint recognition model, an image / file classification model, a network flow / data flow / call volume control model, etc. For example, if the intelligent application processing request is an application requirement for face recognition, then according to the face recognition application requirement, the corresponding face recognition model is searched in the model database or the corresponding instance of the face recognition model is directly searched, and the face recognition instance is run using the idle computing resources of the system, creating a face recognition task, and then performing face recognition through the face recognition task.

[0028] As artificial intelligence and deep learning are applied more and more widely in various industries, the types of tasks that intelligent applications can complete are also increasing, usually involving multiple services of multiple models (solving problems in different scenarios and with different algorithms).

[0029] In related technologies, intelligent application models are generally deployed on devices with "unlimited" (dynamically scalable) system resources, such as servers. This approach has problems such as long latency, poor flexibility, difficulty in managing intelligent application services, and high data security risks. For example, in a CS architecture composed of servers and base stations, a common way to deploy intelligent applications is to use the system resources on existing servers in the core network (such as servers) and add one or more intelligent application services for specific models. Edge devices such as base stations send inference (prediction) and other requests through network interfaces, and the server processes and returns the results. This approach has the following disadvantages:

[0030] 1) High latency. There is a certain amount of communication overhead between the base station and the server. For some scenarios with high requirements for inference latency or large data volumes, the effect is not satisfactory.

[0031] 2) Poor flexibility. Deploying all models on the server is not flexible enough. For example, on some base stations, there may be a need to flexibly adjust the model deployment and operation according to actual conditions (such as adjusting the number of concurrent requests supported by the model). This solution cannot be flexibly adjusted, that is, it cannot achieve personalization.

[0032] 3) Intelligent application service management is difficult. As the true data source, base stations are responsible for generating training and inference requests. However, these requests are not limited to a single type and often involve multiple services across multiple models (solving problems with different scenarios and algorithms). This approach introduces design difficulties and complexity to the base station, as well as management inconveniences.

[0033] 4) High data security risk. More importantly, given the current emphasis on data privacy, if the base station and server belong to different service providers, there is a risk of exposing user data.

[0034] On resource-constrained devices, however, due to limited system resources, problems such as poor deployment flexibility of intelligent application models, low overall device operating efficiency, and impact on the operation of existing functional modules of the device can easily arise. For example, in a CS architecture consisting of servers and base stations, if intelligent application models are deployed in base stations (resource-constrained devices), the following problems arise:

[0035] 1) Poor deployment flexibility. Although there are "idle" resources on base stations and other equipment, this is also limited. If a single model is deployed individually, the subsequent tasks will inevitably be affected, resulting in serious "queuing" of tasks.

[0036] 2) The overall operating efficiency of the base station is low. The models on the server do not affect each other and can be scheduled independently. However, due to limited resources on the base station, independent scheduling between models will lead to resource preemption. As a result, models with frequent demand cannot execute tasks in a timely manner, while models with less demand continue to occupy resources and not release them, resulting in low overall system operating efficiency.

[0037] 3) Impact on the existing functional modules of the base station. The base station itself provides many functions, and many functional modules are already resident on it. Improper deployment of the intelligent system can, in extreme cases, affect the operation of existing functional modules and bring hidden dangers.

[0038] Based on this, the embodiments of the present application provide a resource scheduling method, electronic device and storage medium, which can adaptively schedule system resources, use the system's idle resources to call intelligent application models, improve resource utilization, thereby effectively improving the flexibility of intelligent application model deployment and the overall operating efficiency of the device, without affecting the operation of the device's existing functional modules.

[0039] In the first aspect, the embodiment of the present application provides a resource scheduling method, referring to Figure 1 , the method comprises the following specific steps:

[0040] S0110. Get smart application processing request;

[0041] S0120. Get current resource usage information;

[0042] S0130. Matching smart application instances based on smart application processing requests;

[0043] S0140. Create tasks based on resource usage information and smart application instances to process smart application processing requests.

[0044] In some embodiments, a task can only process one intelligent application processing request, and an intelligent application instance can only process one task at a time. An intelligent application processing request is obtained, a corresponding intelligent application instance is matched to the intelligent application processing request, and the resource requirements and current resource usage information of the intelligent application instance are obtained. If there are currently idle resources and the amount of idle resources is not less than the resource estimate of the intelligent application instance, a task is created using the intelligent application model, which is used to process the intelligent application processing request. The resource usage information includes the resource information currently in use by the system and the current idle resource information. The resource requirements of an intelligent application instance are the amount of resources required to run the intelligent application instance, that is, the resource estimate of the intelligent application instance. If the resource estimate is unknown, it is necessary to record the actual amount of resources occupied by the intelligent application instance when running the intelligent application instance, and use this resource estimate as the resource estimate. The resource estimate can be the actual resource amount recorded when running the intelligent application instance once, or it can be the average resource amount, minimum resource amount, or maximum resource amount calculated based on the actual resource amount recorded each time the intelligent application instance is run multiple times.

[0045] In some embodiments, reference Figure 1 and Figure 2 , before step S0130, the following steps are also included:

[0046] S0210 obtains at least one intelligent application model according to the intelligent application processing request;

[0047] S0220. Create at least one intelligent application instance based on the intelligent application model, resource requirements corresponding to the intelligent application model, and resource usage information.

[0048] Get an intelligent application processing request, match a corresponding intelligent application instance for the intelligent application processing request, if the corresponding intelligent application instance does not currently exist, then you need to create an intelligent application instance. To create an intelligent application instance, you need to get an intelligent application model, match a corresponding intelligent application model for the intelligent application processing request by traversing the model database, obtain the resource requirements and current resource usage information of the intelligent application model, if there are currently idle resources, and the amount of idle resources is not less than the resource estimate of the intelligent application model, then use the intelligent application model to create an intelligent application instance, and the intelligent application instance is used to create a task. Among them, the resource usage information includes the resource information currently being used by the system and the current idle resource information. The resource requirements of an intelligent application model are the amount of resources required to run the intelligent application model, that is, the resource estimate of the intelligent application model. If the resource estimate is unknown, it is necessary to record the actual resource amount occupied by the intelligent application model when running the intelligent application model, and use this resource amount as the resource estimate. The resource estimate can be the actual resource amount recorded when running the intelligent application model once, or it can be the average resource amount, minimum resource amount, or maximum resource amount calculated based on the actual resource amount recorded each time when running the intelligent application model multiple times. In some embodiments, the model database is traversed once according to the intelligent application processing request to find at least one corresponding intelligent application model, and a corresponding intelligent application instance is created using each intelligent application model. If there is a need for concurrent processing tasks, additional intelligent application instances need to be created. By traversing the model database a second time, the intelligent application model that matches the intelligent application processing request is re-executed, and the corresponding intelligent application instance is created again using each intelligent application model. In the process of traversing the model database, only one intelligent application instance can be created each time, or multiple intelligent application instances can be created in one traversal. When processing concurrent tasks, multiple intelligent application instances can be created based on one intelligent application model in one traversal, or multiple traversals can be performed, with only one corresponding intelligent application instance created for each intelligent application model in each traversal. In some embodiments, the intelligent application model can also be used to obtain the model's network topology, trained weights, data information, and more. Each model has at least one corresponding instance, and the total number of models meets the following conditions: the total number of models is no greater than the total number of instances, and each model can provide external services and functions. Each task has a corresponding instance, and the number of active tasks meets the following conditions: the total number of active tasks is no greater than the total number of active model instances.

[0049] In some embodiments, reference Figure 2 and Figure 3 , step S0220 includes the following steps:

[0050] S0310 obtains the first resource estimate according to the resource requirements corresponding to the intelligent application model;

[0051] S0320. Get the current amount of free resources based on resource usage information;

[0052] S0330 compares the first resource estimate and the idle resource amount. If the first resource estimate is not greater than the idle resource amount, execute step S0340; if the first resource estimate is greater than the idle resource amount, execute step S0350;

[0053] S0340. Create at least one intelligent application instance based on the intelligent application model;

[0054] S0350. Cache intelligent application processing requests.

[0055] To create an intelligent application instance, you need to obtain an intelligent application model, match a corresponding intelligent application model to the intelligent application processing request, obtain the resource requirements and current resource usage information of the intelligent application model, and if there are currently idle resources and the amount of idle resources is not less than the first resource estimate of the intelligent application model, then use the intelligent application model to create an intelligent application instance, and use this intelligent application instance to create a task. If there are currently no idle resources, the intelligent application processing request is cached in a queue and waits for the queue to be scheduled. The number of queues must not be less than the number of intelligent application models stored in the model unit. The queues can be scheduled using the Round Robin polling scheduling algorithm or according to the first-in-first-out rule.

[0056] In other embodiments, referring to Figure 1 and Figure 4 , before step S0130, the following steps are also included:

[0057] S0410. Prioritize the intelligent application models in the model database and obtain priority information;

[0058] S0420 obtains an intelligent application model based on the intelligent application processing request and priority information;

[0059] S0430. Create at least one intelligent application instance based on the intelligent application model, resource requirements corresponding to the intelligent application model, and resource usage information.

[0060] The intelligent application models in the model database are prioritized, and intelligent application models with high priorities are used first. Models with high priorities have an advantage in initial resource allocation, and the initialization priority can be default or manually configured. The priority can be set according to the frequency of use of each intelligent application model, and intelligent application models with high frequency of use are set to high priority. If the priorities of multiple intelligent application models are the same, one intelligent application model is randomly selected for calling. Obtain an intelligent application processing request, match a corresponding intelligent application model for the intelligent application processing request according to the priority information of the intelligent application model, obtain the resource requirements and current resource usage information of the intelligent application model, and if there are currently idle resources and the amount of idle resources is not less than the first resource estimate of the intelligent application model, use the intelligent application model to create an intelligent application instance, and the intelligent application instance is used to create a task.

[0061] In some embodiments, reference Figure 4 and Figure 5 Step S0430 includes the following steps:

[0062] S0510 obtains scheduling information of the intelligent application model according to priority information;

[0063] S0520. Determine the intelligent application model to be scheduled according to the scheduling information;

[0064] S0530 obtains a second resource estimate based on the resource requirements of the intelligent application model to be scheduled;

[0065] S0540. Get the current amount of free resources based on resource usage information;

[0066] S0550 compares the second resource estimate and the idle resource amount. If the second resource estimate is not greater than the idle resource amount, execute step S0560; if the second resource estimate is greater than the idle resource amount, execute step S0570;

[0067] S0560. Create at least one intelligent application instance based on the intelligent application model to be scheduled;

[0068] S0570. Cache intelligent application processing requests.

[0069] To create an intelligent application instance, a smart application model is obtained. The scheduling order of the smart application models is determined based on their priority ranking. A corresponding smart application model, i.e., the to-be-scheduled smart application model, is matched to the smart application processing request in descending order of priority. The resource requirements and current resource usage information of the to-be-scheduled smart application model are obtained. If there are currently idle resources and the amount of idle resources is not less than the second resource estimate of the smart application model, an intelligent application instance is created using the to-be-scheduled smart application model. This intelligent application instance is then used to create a task. If there are currently no idle resources, the smart application processing request is cached in a queue, awaiting queue scheduling. The number of queues must be no less than the number of smart application models stored in the model unit. Queues can be scheduled using a Round Robin scheduling algorithm or a first-in-first-out (FIFO) algorithm. If there are currently no idle resources, the smart application processing request is cached in a queue, awaiting queue scheduling. The number of queues must be no less than the number of smart application models stored in the model unit. Queues can be scheduled using a Round Robin scheduling algorithm or a FIFO (first-in-first-out) algorithm.

[0070] In some embodiments, reference Figure 1 , before step S0140 , it also includes: merging, caching or sorting the intelligent application processing request with the current pending request according to the current pending request information.

[0071] Obtain an intelligent application processing request. If there are currently no idle resources, cache the intelligent application processing request in a queue. The queue can cache multiple pending intelligent application processing requests. Pending request information includes the number of pending intelligent application processing requests cached in the queue, the queue's scheduling order, and the data information for each pending intelligent application processing request. If there are multiple pending intelligent application processing requests of the same or same type in the queue, they can be merged into a single pending intelligent application processing request.

[0072] In some embodiments, reference Figure 1 and Figure 6 Step S0140 includes the following specific steps:

[0073] S0610. Get status information of smart application instances;

[0074] S0620. Get the current task concurrent information;

[0075] S0630. Create a task based on the intelligent application instance, the status information of the intelligent application instance, and the task concurrency information.

[0076] Task concurrency information includes the task concurrency count, which refers to the number of tasks being processed simultaneously. The status information of an intelligent application instance indicates whether the intelligent application instance is idle or running. The current status of an intelligent application instance depends on the current resource usage information and the resource requirements of the intelligent application instance. The current resource usage information is affected by the current task concurrency count. The higher the current task concurrency count, the more resources are being used and the fewer idle resources are available. When obtaining an intelligent application processing request, if there are currently no idle resources, the intelligent application processing request is cached in a queue. The queue can cache multiple pending intelligent application processing requests. The processing order of the intelligent application processing request is determined by the queue's scheduling order. Each intelligent application processing request corresponds to an intelligent application instance. When the queue begins scheduling the intelligent application processing request, if the intelligent application instance is currently idle, a task is created using the intelligent application instance to process the intelligent application processing request. If the intelligent application instance is currently running, the intelligent application processing request remains cached in the queue.

[0077] In some embodiments, reference Figure 6 and Figure 7 Step S0640 includes the following specific steps:

[0078] S0710. Get the number of concurrent tasks based on the concurrent task information;

[0079] S0720. Compare the number of concurrent tasks with the maximum number of concurrent tasks. If the number of concurrent tasks is not greater than the maximum number of concurrent tasks, execute step S0730; if the number of concurrent tasks is greater than the maximum number of concurrent tasks, execute step S0740.

[0080] S0730. Create a task based on the intelligent application instance and the status information of the intelligent application instance;

[0081] S0740. Continue caching the intelligent application processing request.

[0082] The current status of an intelligent application instance depends on its current resource usage and resource requirements. This information is affected by the current concurrency of tasks. A higher concurrency indicates more resources in use and fewer idle resources. Before determining the status of an intelligent application instance, you must verify its current resource usage. You can obtain the maximum concurrency of tasks, which is the upper limit of task concurrency and is subject to system resource constraints. If the current concurrency of tasks is not greater than the maximum concurrency, idle resources are considered to exist. If the current concurrency of tasks is greater than the maximum concurrency of tasks, no idle resources are considered to be available for scheduling. When obtaining an intelligent application processing request, if there are no idle resources, the request is cached in a queue. The queue can cache multiple pending intelligent application processing requests. The order in which the intelligent application processing request is processed is determined by the queue's scheduling order. The intelligent application processing request corresponds to an intelligent application instance. If the current number of concurrent tasks is not greater than the maximum number of concurrent tasks, it means that there are currently idle resources. When the queue starts to schedule the intelligent application processing request, if the current intelligent application instance is in an idle state, a task can be created using the intelligent application instance, and the task is used to process the intelligent application processing request. If the intelligent application instance is currently in a running state, the intelligent application processing request continues to be cached in the queue. By comparing the current number of concurrent tasks with the maximum number of concurrent tasks, the current resource usage information can be obtained, and then the status information of the intelligent application instance can be obtained based on the current resource usage information and the resource requirements of the intelligent application instance.

[0083] In some embodiments, reference Figure 7 Before step S0720, the process also includes: obtaining the current number of waiting tasks; if the current number of waiting tasks is greater than a preset waiting task threshold, adjusting the maximum concurrent number of tasks according to the current maximum concurrent number of tasks, the system upper limit number of tasks, and a preset adjustment factor.

[0084] The number of waiting tasks refers to the number of cached tasks in the queue. If at a certain moment, the number of cached tasks in the queue exceeds the upper limit of the cache number, that is, the current number of waiting tasks is greater than the preset waiting task number threshold, then the current maximum concurrent task number needs to be adjusted. Use formula (1) to adjust the maximum concurrent task number:

[0085] T t+1 =min(T t (1+a),T top ) (1)

[0086] Among them, T t Indicates the maximum number of concurrent tasks at time t, T t+1 Indicates the maximum number of concurrent tasks at time t+1, Ttop Represents the system's upper limit on task counts, which is limited by system resources. a represents the adjustment factor, which is a natural number. The current maximum concurrent task count is adjusted using the adjustment factor. The adjusted maximum concurrent task count is compared with the system's upper limit on task count. If the adjusted maximum concurrent task count is greater than the system's upper limit on task count, the system's upper limit on task count is set as the maximum concurrent task count for the next moment. If the adjusted maximum concurrent task count is less than the system's upper limit on task count, the adjusted maximum concurrent task count is set as the maximum concurrent task count for the next moment. If the adjusted maximum concurrent task count and the system's upper limit on task count are equal, either the adjusted maximum concurrent task count or the system's upper limit on task count can be set as the maximum concurrent task count for the next moment.

[0087] In some embodiments, reference Figure 1 and Figure 8 , before step S0140, the following specific steps are also included:

[0088] S0810. Obtain the corresponding intelligent application model according to the intelligent application instance;

[0089] S0820. Get the number of pending requests corresponding to the intelligent application model in the cache queue;

[0090] S0830. Adjust the number of concurrent instances of the intelligent application model based on the number of pending requests.

[0091] An intelligent application model can create at least one intelligent application instance, and each intelligent application instance can create at least one task. A task is used to process one intelligent application processing request. Therefore, each intelligent application instance corresponds to a corresponding intelligent application model, and each intelligent application processing request corresponds to a corresponding intelligent application instance. In other words, each intelligent application model can be used to process at least one intelligent application processing request. All pending requests corresponding to an intelligent application model can be extracted from the pending request information to determine the number of pending requests corresponding to the intelligent application model. The instance concurrency is the number of concurrently running instances, and the setting of the instance concurrency is limited by system resources. Current resource usage is affected by the instance concurrency. The higher the current instance concurrency, the more resources the running instances occupy, and the fewer idle resources available for scheduling. If the number of pending requests corresponding to a certain intelligent application model reaches the upper limit of the cache at a certain moment, the instance concurrency corresponding to the current intelligent application model needs to be increased. If the intelligent application instance corresponding to a certain intelligent application model remains idle for a period of time, meaning that the pending requests corresponding to the intelligent application model have not been processed, the instance concurrency corresponding to the current intelligent application model needs to be reduced.

[0092] In some embodiments, reference Figure 1 and Figure 9 , before step S0140, the following specific steps are also included:

[0093] S0910. Obtain at least two corresponding intelligent application models based on the current concurrent intelligent application instance;

[0094] S0920. Get the number of pending requests corresponding to each intelligent application model in the cache queue;

[0095] S0930. Obtain priority information of the intelligent application model;

[0096] S0940. Obtain weight information corresponding to the intelligent application model according to the number of pending requests corresponding to each intelligent application model;

[0097] S0950 adjusts the priority of the intelligent application model according to the priority information and weight information;

[0098] S0960. Adjust the number of concurrent instances corresponding to the intelligent application model based on the priority of the intelligent application model and the number of pending requests.

[0099] If the number of pending requests in the cache queue reaches the upper limit at a certain point, the current concurrent instance count needs to be increased. If the current concurrent intelligent application instances involve multiple different intelligent application models, the priority of each intelligent application model needs to be determined and the concurrent instance count for each intelligent application model adjusted in order of priority. Each intelligent application model has an initial priority, which does not take into account the number of pending requests in the cache queue. The initial priority is determined by sorting all intelligent application models in the model database, with intelligent application models with higher priorities being used first. The initial priority can be set based on the frequency of use of each intelligent application model, with more frequently used intelligent application models being assigned higher priority. If multiple intelligent application models have the same priority, a randomly selected intelligent application model is invoked. The initial priority information for each intelligent application model can be obtained. The pending requests corresponding to each intelligent application model can be extracted from the pending request information to determine the number of pending requests corresponding to each intelligent application model and the total number of pending requests. Furthermore, the weight of the number of pending requests corresponding to a particular intelligent application model relative to the total number of pending requests can be determined. The weight information can be set according to the importance or urgency or data size of the pending request corresponding to the intelligent application model. According to the initial priority information of an intelligent application model and the weight information corresponding to the intelligent application model, the priority of the intelligent application model is adjusted, and then the priority of each intelligent application model can be obtained, and the number of concurrent instances corresponding to each intelligent application model can be adjusted according to the order of priority. Use formula (2) to adjust the priority of the intelligent application model:

[0100]

[0101] Among them, P i Indicates the original priority of model i, P i ' represents the priority of model i after update. b represents the weight factor when updating the priority. The weight factor b is the weight of the number of pending requests corresponding to model i to the total number of pending requests. b is a natural number. i Indicates the number of pending requests for model i within a period of time, Cache j Indicates the number of pending requests for all models during the period.

[0102] In the embodiment of the present application, through the layer-by-layer control of models, instances, and tasks, it is possible to adaptively schedule system resources, use the system's idle resources to call intelligent application models, improve resource utilization, and then quickly and effectively process intelligent application processing requests, improve the processing efficiency of intelligent applications, enhance the flexibility of intelligent application model deployment and the overall operating efficiency of the device, and do not affect the operation of the existing functional modules of the device. The relationship between computing resources and computing tasks can be dynamically adjusted to achieve a balance between the two. When executing model reasoning and training, since data processing is only performed within the device and does not interact with other devices, it is possible to protect user data and reduce task processing latency.

[0103] In some embodiments, reference Figure 10 and Figure 11 , shows a resource scheduling system 1000, which can execute the resource scheduling method of the above embodiment, including: a model unit 1010, used to store at least one intelligent application model, and record the first resource requirement and the first concurrency requirement of each intelligent application model; an instance unit 1020, connected to the model unit 1010, used to store at least one intelligent application instance, and record the second resource requirement and the second concurrency requirement of each intelligent application instance; a task unit 1030, connected to the instance unit 1020, used to obtain at least one intelligent application processing request and process the intelligent application processing request; a resource monitoring unit 1040, respectively connected to the model unit 1010, the instance unit 1020 and the task unit 1030, used to monitor the resource usage of the resource scheduling system 1000 in real time and obtain resource usage information.

[0104] In the model unit, the first resource requirement of an intelligent application model is the amount of resources required to run the intelligent application model, that is, the estimated resource amount of the intelligent application model. If the estimated resource amount is unknown, it is necessary to record the actual amount of resources occupied by the intelligent application model when running the intelligent application model, and use the amount of resources as the estimated resource amount. The estimated resource amount can be the actual amount of resources recorded when running the intelligent application model once, or it can be the average amount of resources, the minimum amount of resources, or the maximum amount of resources calculated based on the actual amount of resources recorded each time when running the intelligent application model multiple times. The first concurrency requirement of an intelligent application model is the number of models that expect to run the intelligent application model simultaneously. The number of models is preset to 1 and can be adjusted according to the actual task request.

[0105] In the instance unit, the second resource requirement of an intelligent application instance is the amount of resources required to run the intelligent application instance, that is, the estimated resource amount of the intelligent application instance. If the estimated resource amount is unknown, it is necessary to record the actual amount of resources occupied by the intelligent application instance when running the intelligent application instance, and use the amount of resources as the estimated resource amount. The estimated resource amount can be the actual amount of resources recorded when running the intelligent application instance once, or it can be the average amount of resources, the minimum amount of resources, or the maximum amount of resources calculated based on the actual amount of resources recorded each time when running the intelligent application instance multiple times. The second concurrency requirement of an intelligent application instance is the number of instances expected to run the intelligent application instance at the same time. The number of instances is preset to 1 and can be adjusted according to the actual task request.

[0106] In a task unit, a task can only handle one intelligent application processing request. If there are insufficient resources to handle all intelligent application processing requests, the unprocessed intelligent application processing requests are cached in a queue. The number of queues must be no less than the number of intelligent application models stored in the model unit. Queues can be scheduled using a Round Robin scheduling algorithm or a first-in-first-out (FIFO) approach. To create a task, an instance capable of handling the task must be found. If the instance's estimated resource capacity is no greater than the current idle resource capacity, a task is created based on the instance. If the instance's estimated resource capacity is greater than the current idle resource capacity, the corresponding intelligent application processing request is cached in the queue, awaiting scheduling.

[0107] The resource monitoring unit monitors the resource usage of the resource scheduling system in real time, and synchronizes the resource usage information to the model unit, instance unit and task unit in real time, so as to further adjust the relationship between resources and operation status. Resource usage information includes the resource information currently being used by the system and the current idle resource information. For any request, the resource monitoring unit checks whether there is an active and idle instance of the corresponding model of its request. If so, it uses the request and instance to create a task and allocate resources to run. Otherwise, it decides whether to cache the request and wait or directly return failure based on the policy. During the operation of the system, the resource monitoring unit will dynamically track the operation of tasks and the activity status of instances, and adjust the upper limit of the number of instances and the number of tasks when necessary, that is, to establish a closed loop between system resource occupancy and allocation, so as to achieve the purpose of rational resource utilization by dynamically adjusting the number of concurrent models and the number of concurrent tasks. In some embodiments, such as Figure 11 As shown, the system reports the number of active tasks, the number of waiting tasks, the number of active instances or the number of cache requests, and the resource monitoring unit performs operation monitoring and outputs the adjusted number of active tasks, the number of waiting tasks, the number of active instances or the number of cache requests.

[0108] In some embodiments, the model unit is also used to set the priority of the intelligent application model, and the intelligent application model with a high priority is used first. The model with a high priority has an advantage in the initial resource allocation, and the initialization priority can be default or manually configured. The priority can be set according to the frequency of use of each intelligent application model, and the intelligent application model with a high frequency of use is set to a high priority. If the priorities of multiple intelligent application models are the same, an intelligent application model is randomly selected from them for calling. In other embodiments, the model unit is also used to store the network topology, weight and data information of each intelligent application model.

[0109] In some embodiments, the instance unit is also used to create an intelligent application instance based on the intelligent application model, the resource requirements corresponding to the intelligent application model, and the current resource usage information. An intelligent application model can be used to create at least one intelligent application instance, and an intelligent application instance can be used to create at least one task. A task can only process one intelligent application processing request. At a certain moment, an intelligent application instance can only process one task, so the task concurrency is not greater than the instance concurrency. The task concurrency refers to the number of tasks processed simultaneously, and the instance concurrency refers to the number of instances running simultaneously.

[0110] When the system is powered on, the number of active model instances is determined based on the current resource availability, the number of models, the estimated resource usage of each model, and the expected number of concurrent users. A corresponding number of running instances are created for each model. During system operation, for a specific training or inference request for a model, the system determines whether to create a task for execution or cache data based on the idle and active status of the corresponding instance. For each created task, the system determines whether to process it immediately or wait based on the current system concurrency. When processing a task, the system collects statistics and evaluates the instance's active running status, the task's time-sharing running status, and the data cache status to determine whether to dynamically load the instance or release its resources. In some embodiments, the system creates model instances upon power-up and sorts the models in the model database from high to low priority. Initially, the model priorities can be default or manually configured. The system prepares to begin the first iteration of all models. Let i = 0. For the i-th model, the system creates the first running instance. It determines whether the resources requested by model i are less than the system's idle resources. If so, an instance is created for model i; otherwise, the power-up process ends. Create an instance for model i. If model i knows its own resource requirements, determine whether all models in the warehouse have been traversed. Otherwise, record the minimum amount of resources required to start the model instance to facilitate evaluation calculations when the system is subsequently started. Determine whether all models in the warehouse have been traversed. If so, prepare to traverse the model database a second time. Otherwise, execute the next model to continue traversing. Traverse the model database a second time and re-execute the first model. For model i, if model i has the need for concurrent processing tasks, it is considered that it needs to create additional active instances (that is, for model i, the total number of instances is >= 2). If additional active instances need to be created, determine whether the resources requested by model i are less than the system's idle resources. If so, create a new instance for model i. Otherwise, determine whether all models have been traversed. If no additional active instances are needed, determine whether all models have been traversed. Determine whether all models have been traversed. If so, end the entire process. Otherwise, point to the next model and continue traversing. The system ensures an initial balance of system resources and models upon power-up. If all models are powered on successfully, it means that all models can provide basic services to the outside world, there is at least one active instance, and there will be no impact on the existing system. Otherwise, the model with the highest priority will successfully create an instance first, and only some of the model's service functions can be provided to the outside world.

[0111] In the embodiment of the present application, each model has at least one corresponding instance, and the total number of models meets the following conditions: the total number of models is not greater than the total number of instances, and each model can provide services and functions to the outside world. Each task has a corresponding instance, and the number of active tasks meets the following conditions: the total number of active tasks is not greater than the total number of active model instances. The resource scheduling system ensures that while providing the required reasoning and training functions, resource utilization efficiency is maximized by controlling the models, instances, and tasks layer by layer. Without affecting the original functions, the required intelligent reasoning and training system can be built using existing idle computing resources. Models can be flexibly deployed and scheduled to fully utilize resources without waste, and the relationship between computing resources and computing tasks can be dynamically adjusted to achieve a balance between the two. When executing model reasoning and training, since data processing is only performed within the device and no data interaction is performed with other devices, user data can be protected and task processing latency can be reduced.

[0112] In a second aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a program stored in the memory and runnable on the processor. When the program is executed by the processor, the resource scheduling method of some embodiments of the first aspect of the present application is implemented.

[0113] In some embodiments, the electronic device may be a mobile terminal device or a non-mobile terminal device. The mobile terminal device may be a mobile phone, tablet computer, laptop computer, PDA, vehicle-mounted terminal device, wearable device, ultra-mobile personal computer, netbook, personal digital assistant, etc.; the non-mobile terminal device may be a personal computer, television, ATM, or self-service machine, etc.; the present application embodiment does not specifically limit this.

[0114] In a third aspect, an embodiment of the present application provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the resource scheduling method of some embodiments of the first aspect of the present application.

[0115] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0116] In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0117] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the above-mentioned implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A resource scheduling method, comprising: Get intelligent application processing request; Get current resource usage information; Matching intelligent application instances based on intelligent application processing requests; Creating a task according to the resource usage information and the intelligent application instance to process the intelligent application processing request; Before creating a task based on the resource usage information and the smart application instance, the method further includes: Acquire at least two corresponding intelligent application models according to the currently concurrent intelligent application instances; Obtain the number of pending requests corresponding to each of the intelligent application models in the cache queue; Obtaining priority information of the intelligent application model; Obtaining weight information corresponding to each intelligent application model according to the number of pending requests corresponding to each intelligent application model; Adjusting the priority of the intelligent application model according to the priority information and the weight information; The concurrent number of instances corresponding to the intelligent application model is adjusted according to the priority of the intelligent application model and the number of pending requests.

2. The resource scheduling method according to claim 1, characterized in that: Before matching the intelligent application instance according to the intelligent application processing request, the method further includes: Acquire at least one intelligent application model according to the intelligent application processing request; At least one of the smart application instances is created according to the smart application model, the resource requirements corresponding to the smart application model, and the resource usage information.

3. The resource scheduling method according to claim 2, characterized in that: The creating at least one smart application instance according to the smart application model, the resource requirements corresponding to the smart application model, and the resource usage information includes: Obtaining a first resource estimate according to the resource demand corresponding to the intelligent application model; Obtaining the current amount of idle resources according to the resource usage information; comparing the first estimated amount of resources with the amount of idle resources; If the first estimated amount of resources is not greater than the amount of idle resources, at least one smart application instance is created according to the smart application model.

4. The resource scheduling method according to claim 1, characterized in that: Before matching the intelligent application instance according to the intelligent application processing request, the method further includes: Prioritize the intelligent application models in the model database and obtain priority information; Acquire the intelligent application model according to the intelligent application processing request and the priority information; At least one of the smart application instances is created according to the smart application model, the resource requirements corresponding to the smart application model, and the resource usage information.

5. The resource scheduling method according to claim 4, characterized in that: The creating at least one smart application instance according to the smart application model, the resource requirements corresponding to the smart application model, and the resource usage information includes: Acquiring scheduling information of the intelligent application model according to the priority information; Determining a smart application model to be scheduled according to the scheduling information; Obtaining a second resource estimate according to the resource demand of the intelligent application model to be scheduled; Obtaining the current amount of idle resources according to the resource usage information; comparing the second estimated amount of resources with the amount of idle resources; If the second estimated amount of resources is not greater than the amount of idle resources, at least one of the intelligent application instances is created according to the intelligent application model to be scheduled.

6. The resource scheduling method according to any one of claims 1 to 5, characterized in that: The creating a task according to the resource usage information and the intelligent application instance includes: Obtaining status information of the intelligent application instance; Get the current task concurrent information; A task is created according to the intelligent application instance, the status information of the intelligent application instance, and the task concurrency information.

7. The resource scheduling method according to claim 6, characterized in that: The creating a task according to the intelligent application instance, the state information of the intelligent application instance, and the task concurrency information includes: Obtain the number of concurrent tasks according to the concurrent task information; Comparing the concurrent number of tasks with the maximum concurrent number of tasks; If the concurrent number of tasks is not greater than the maximum concurrent number of tasks, a task is created according to the intelligent application instance and the status information of the intelligent application instance.

8. The resource scheduling method according to claim 7, characterized in that: Before comparing the number of concurrent tasks with the maximum number of concurrent tasks, the method further includes: Get the current number of waiting tasks; If the current number of waiting tasks is greater than the preset waiting task number threshold, the maximum concurrent task number is adjusted according to the current maximum concurrent task number, the system upper limit task number and the preset adjustment factor.

9. The resource scheduling method according to any one of claims 1 to 5, characterized in that: Before creating a task according to the resource usage information and the smart application instance, the method further includes: Acquire a corresponding intelligent application model according to the intelligent application instance; Obtain the number of pending requests corresponding to the intelligent application model in the cache queue; The concurrent number of instances of the intelligent application model is adjusted according to the number of pending requests.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the resource scheduling method according to any one of claims 1 to 9 is implemented.

11. A storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the resource scheduling method according to any one of claims 1 to 9.

Images