Resource scheduling method, system, device and storage medium

Abstract

The embodiments of the present application provide a resource scheduling method, system, device and storage medium. Among them, the method includes: predicting the first resource demand based on the user's access records to the algorithm model in the historical period; wherein the algorithm model is used to provide corresponding computing services for the user; obtaining the user's access frequency to the algorithm model in the current period, and determining the second resource demand; combining the first resource demand and the second resource demand, and expanding or shrinking the resources corresponding to the algorithm model accordingly. Through the above scheme, it is possible to obtain accurate resource demands even in the face of diversified scheduling requirements, and it is possible to cope with resource scheduling scenarios with diversified and variable scheduling requirements, and achieve accurate and timely scheduling effects. The technical solution of the present application has higher sensitivity when performing resource scheduling.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a resource scheduling method, system, device and storage medium. Background Art

[0002] With the development of cloud computing technology, more and more users choose to use cloud computing platforms to meet their task requirements.

[0003] In the prior art, cloud computing platforms can provide users with a variety of directly callable models, which are directly deployed on cloud computing platforms, such as image recognition algorithm models. When a user task requires a model to be called, it can be called directly. In actual applications, there may be many users who need to call a certain model at the same time, resulting in insufficient resources and unable to meet the calling needs of so many users. It is also possible that not many users call the model at the current time, resulting in a waste of model resources. In addition, since there are many users who call the model, these users have different resource scheduling habits and resource scheduling rules, which also leads to complex and changeable resource scheduling requirements. Therefore, a resource scheduling solution with better scheduling effect is needed. Summary of the invention

[0004] In order to solve or improve the problems existing in the prior art, the embodiments of the present application provide a resource scheduling method, system, device and storage medium.

[0005] In a first aspect, in one embodiment of the present application, a resource scheduling method is provided. The method comprises:

[0006] Predicting the first resource demand based on the user's access records to the algorithm model during the historical period; wherein the algorithm model is used to provide the user with corresponding computing services;

[0007] Obtaining the access frequency of users to the algorithm model in the current period, and determining the second resource demand;

[0008] Based on the first resource demand and the second resource demand, the resources corresponding to the algorithm model are expanded or reduced accordingly.

[0009] In a second aspect, in one embodiment of the present application, a resource scheduling method is provided. The method comprises:

[0010] In response to a user's access request, an access record to an algorithm model deployed on a cloud platform is generated, so that the algorithm model calls the resources of the cloud platform based on the access record request; the cloud platform predicts a first resource demand based on the user's access record to the algorithm model in a historical period; and determines a second resource demand based on the access frequency of the access record in a current period; and the resources corresponding to the algorithm model are expanded or reduced accordingly based on the first resource demand and the second resource demand, so that the algorithm model meets the computing needs of the user;

[0011] Receive the calculation result fed back by the algorithm model.

[0012] In a third aspect, in one embodiment of the present application, a resource scheduling system is provided, the system comprising:

[0013] A cloud platform, for predicting a first resource demand based on a user's access record to an algorithm model during a historical period; wherein the algorithm model is used to provide corresponding computing services to the user; obtaining a user's access frequency to the algorithm model during a current period to determine a second resource demand; and, based on the first resource demand and the second resource demand, expanding or reducing the resource corresponding to the algorithm model accordingly;

[0014] The client is used to generate an access record to the algorithm model deployed on the cloud platform in response to the user's access request, so that the algorithm model can call the resources of the cloud platform based on the access record request; the cloud platform predicts the first resource demand based on the user's access record to the algorithm model in the historical period; and determines the second resource demand based on the access frequency of the access record in the current period; the resources corresponding to the algorithm model are expanded or reduced accordingly based on the first resource demand and the second resource demand, so that the algorithm model meets the user's computing needs; and the computing results fed back by the algorithm model are received.

[0015] In a fourth aspect, in one embodiment of the present application, an electronic device is provided, including a memory and a processor; wherein:

[0016] The memory is used to store programs;

[0017] The processor is coupled to the memory, and is used to execute the program stored in the memory, so as to implement the resource scheduling method described in the first aspect or the resource scheduling method described in the second aspect.

[0018] In the fifth aspect, in one embodiment of the present application, a non-temporary machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes a resource scheduling method as described in the first aspect or a resource scheduling method as described in the second aspect.

[0019] In the technical solution provided by the embodiment of the present application, various models that can be called by users, such as algorithm models, are deployed on the cloud platform to provide corresponding services for users. After the algorithm model is deployed on the cloud platform, the user's access to the algorithm model or the algorithm model to the cloud platform resources can be monitored in real time, and the current resource demand can be determined in time, and resources can be scheduled according to the resource demand to meet complex and changeable scheduling needs. Specifically, the first resource demand with active scheduling effect is generated by using historical access records; and the second resource demand for passive resource adjustment is generated by real-time monitoring of the access frequency of the access record. Then, the first resource demand and the second resource demand are comprehensively utilized, and resource scheduling is performed according to the comprehensive processing results, so that a more timely and accurate resource scheduling effect can be obtained. The first resource demand can be predicted by an online prediction model, and the online prediction model can be trained and optimized in real time online. Therefore, accurate resource demand can be obtained in the face of diversified scheduling needs. In order to avoid the situation where the prediction model fails to predict the resource scheduling due to inaccurate prediction, a second resource demand is also configured. Through the above scheme, it is possible to cope with diverse resource scheduling scenarios with changing scheduling requirements and achieve accurate and timely scheduling effects. This scheduling scheme is more sensitive and flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A schematic diagram of a resource scheduling method according to an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a resource scheduling method is provided as an example to illustrate an embodiment of the present application;

[0023] Figure 3 A flowchart of a model training method provided in an embodiment of the present application;

[0024] Figure 4A schematic diagram of the framework of an online prediction model illustrated in an embodiment of the present application;

[0025] Figure 5 A flowchart of another resource scheduling method provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a resource scheduling process for illustrating an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of a resource scheduling device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the structure of another resource scheduling device provided in an embodiment of the present application;

[0030] Fig.10 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0032] In some processes described in the specification, claims and the above-mentioned figures of the present application, multiple operations appearing in a specific order are included, and these operations may not be executed or executed in parallel in the order in which they appear in this article. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit "first" and "second" to different types. In addition, the embodiments described below are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0033] With the development of cloud computing technology, more and more scenarios, such as image recognition and text recognition scenarios using cloud computing, need to call algorithm models deployed on the cloud platform to provide computing services to complete the corresponding computing tasks. Algorithm models deployed on the cloud platform usually have good hardware configuration and can provide computing services for many users. Generally speaking, before the algorithm model is deployed on the cloud platform, the hardware resource specifications required for model deployment need to be determined before the algorithm model is deployed online (the conventional method is to perform stress testing after amplifying the access frequency of the access information by N times), such as: 1cpu, 4G memory. And after evaluating the access frequency of the algorithm model (queries per second, Queries-per second, QPS is a measure of the amount of traffic processed by a specific query server within a specified time (per second)), amplify N times for stress testing and other processes. However, when the access volume in the real environment explodes and the access frequency exceeds the carrying capacity of the currently configured resources of the algorithm model, the algorithm model needs to urgently expand the resources. This expansion method is passive expansion, and it is difficult to meet the resource requirements of the burst access volume, resulting in poor user experience. In addition, in some real environments where the access volume is small, it will lead to resource waste. Also, in some cases, if multiple algorithm models are using the same set of resources, some algorithm model resources are idle and cannot be released in time, while other algorithm model resources are tight and cannot be expanded in time. Therefore, a solution that can perform resource scheduling in time is needed.

[0034] In the technical solution of the present application, the specific working process will be described in the following embodiments.

[0035] like Figure 1 The resource scheduling method provided in the embodiment of the present application is a flow chart. The execution subject of the resource scheduling method can be a server device (such as a cloud server, etc.) Figure 1 As shown, the resource scheduling method includes the following steps:

[0036] Step 101: predicting a first resource demand based on a user's access record to an algorithm model in a historical period; wherein the algorithm model is used to provide corresponding computing services for the user.

[0037] Step 102: Obtain the access frequency of users to the algorithm model in the current period and determine the second resource demand.

[0038] Step 103: Based on the first resource demand and the second resource demand, the resources corresponding to the algorithm model are expanded or reduced accordingly.

[0039] It should be noted that the method for predicting the first resource demand can be to use a trained fixed machine learning model and an online prediction model for prediction. Among them, the trained machine learning model is fixed, usually trained based on the same access initiator, and only has a good prediction effect on the access initiator, and cannot play a predictive role for other access initiators. The online prediction model (for example, Alink) is a type of machine learning, which can optimize the training model in real time according to the changes in online data, so that the model can adapt to real-time changes online, thereby improving the accuracy of online predictions.

[0040] The online prediction model is particularly suitable for situations where multiple users or clients need to use the algorithm model, and each user or client uses the algorithm model according to their own habits and needs. The usage needs of each user or client change in real time and are not very regular. The online prediction model can be preliminarily trained using the historical period (that is, before being connected or deployed to the cloud platform) to obtain an initial model. After the online prediction model is connected to the cloud platform, the online prediction model can be optimized using the real-time data, so that the online prediction model can have better sensitivity after training based on the latest data, and can achieve more accurate prediction results based on the latest user or client resource call situation. The first resource demand obtained by using the online prediction model is the result of advance prediction, which can be used to expand or shrink the algorithm model in advance. It is an active expansion or shrinking method that can avoid the problem of untimely expansion caused by passive expansion, or the problem of idle waste of resources and lack of resources for other algorithm models caused by untimely passive shrinking.

[0041] The cloud platform mentioned here can be, for example, kubernetes: K8s for short, which can be used to manage containerized applications on multiple hosts in the cloud platform, making the deployment of containerized applications simple and efficient (powerful). Kubernetes provides a mechanism for application deployment, planning, updating, and maintenance. Of course, when scheduling resources, scheduling can be performed at the granularity of pod containers or other containers.

[0042] The first resource requirement and the second resource requirement mentioned here can be understood as the requirements of the algorithm model for hardware (such as CPU, memory, GPU, etc.) related resources. The resource requirement can be the amount of expansion (increasing the current resource requirement) or the amount of reduction (reducing the current resource requirement and releasing the currently occupied resources). For example, expand 4G memory, or reduce 4G memory.

[0043] The access frequency mentioned here can be the query rate per second (QPS), which is the amount of traffic processed by a specific query server within a specified time (per second). Therefore, the access frequency can be used as a reference for adjusting resource capacity. Generally speaking, the access frequency that the algorithm model can withstand needs to be tested by extreme stress, that is, in theory, the algorithm model can withstand the resource scheduling requirements under high access frequencies. However, there is a certain upper limit of access frequency in the initial resource configuration of the algorithm model. When it reaches or approaches the upper limit, it needs to be expanded. Simply put, when the access frequency is greater than a certain threshold, it means that the current resources cannot meet the resource call requirements of the access requests that may continue to grow in the future, so it is necessary to expand the resources; or, when the access frequency is less than a certain threshold, it means that the current algorithm model resources are too sufficient, resulting in idle resources. Waste, therefore, the currently configured resources can be appropriately reduced to release some resources for use by other algorithm models.

[0044] The comprehensive first resource demand and the second resource demand mentioned here can be understood as comprehensive processing of the first resource demand and the second resource demand. The comprehensive processing method can be to use the configuration strategy to set the priority of the results of the resource demand obtained by evaluating in different ways. For example, when the online prediction model has a high prediction accuracy, the result "first resource demand" predicted by the online prediction model is used as the basis for the corresponding expansion or reduction of the resources corresponding to the algorithm model; the result predicted by the online prediction model has a leading effect and can make corresponding expansion or reduction adjustments to the resources in advance.

[0045] The historical period access records mentioned here can be understood as records of algorithm model access requests initiated by at least one user or client before the current period.

[0046] In the embodiment of the present application, a resource demand is obtained through the online prediction model and the real-time calculation model. Then, the resources of the current algorithm model are adjusted based on the priority order of the first resource demand and the second resource demand, including resource expansion or resource reduction according to demand, so as to avoid resource idleness while meeting the resource usage needs of users or clients.

[0047] In one or more embodiments of the present application, predicting the first resource demand based on the user's access records for the algorithm model during the historical period includes: obtaining the access subject identifier contained in the user's access records for the algorithm model during the historical period. Based on the access subject identifier, predicting the first resource demand using an online prediction model.

[0048] In actual applications, algorithm models deployed on cloud platforms can provide computing services to multiple users at the same time. It is easy to understand that different users may not have exactly the same calling habits for algorithm models. Therefore, an online prediction model can be used to predict the first resource demand, and the model can be optimized and trained online as needed. For example, when the online prediction model is connected to the platform to perform prediction work, access records within a period of time will be collected online, and the access records will be used as training samples to optimize the online prediction model, so that the online prediction model can be updated in real time and dynamically, and make more accurate predictions based on the latest situation.

[0049] Assume that there are currently two users, User A and User B, who use the same algorithm model and the same set of resources to complete the image recognition task. Further assume that in the first week, User A releases a new product, which attracts a large number of users to experience the product provided by User A, which will lead to an explosion in the number of visits. However, User B's product has a small problem and needs further optimization and adjustment, resulting in a gradual decrease in the number of users. Then, when the online prediction model is in normal use, the access situation of User A and User B to the algorithm model is collected online, and then, based on the access records, training samples for optimizing the online prediction model are obtained. The training samples need to clearly include the access subject identifier so that the online prediction model obtained after training optimization can realize the resource demand for each user and the overall user. The latest online access records are used as training samples to meet the requirements for accurate prediction of multi-user resource demand.

[0050] In one or more embodiments of the present application, Figure 2 The following is a flow chart of a resource scheduling method for illustrating an embodiment of the present application. Figure 2 It can be seen that the first resource demand and the second resource demand are comprehensively considered to expand or shrink the resources corresponding to the algorithm model, including: 201: If the prediction result of the online prediction model meets the expected result, then based on the configuration strategy, it is determined that the resources corresponding to the algorithm model are expanded or shrunk according to the first resource demand. 202: If the prediction result of the online prediction model does not meet the expected result, then based on the configuration strategy, it is determined that the resources corresponding to the algorithm model are expanded or shrunk according to the second resource demand. 203: The online prediction model uses the access records before the current time period for training and optimization. If the prediction result after training and optimization does not meet the expected result, then based on the configuration strategy, it is determined that the resources corresponding to the algorithm model are expanded or shrunk according to the second resource demand.

[0051] In practical applications, online prediction models can only be deployed online to provide prediction services for algorithm models after training based on historical access records. Specifically, when the prediction results of the trained online prediction model meet the expected results (the resource usage of the algorithm model after a period of time) (of course, it is also possible to determine whether the online prediction model can be used for online prediction based on the loss function), the prediction results of the online prediction model are considered reliable. Therefore, the priorities of the resource demands obtained in different ways can be adjusted according to the configuration strategy, and the first resource demand can be adjusted to the highest priority. In other words, the resources of the algorithm model are expanded or reduced accordingly according to the first resource demand.

[0052] When providing online predictions, the online prediction model can be optimized and trained based on online access records as training samples. It should be noted that although the online prediction model can realize prediction and actively perform resource scheduling, inaccurate predictions are inevitable. If, after comparing the prediction results with the actual resource scheduling situation, it is found that the prediction results do not match the expected results in the actual resource scheduling process, the online prediction model obtained by the current optimization training is considered inaccurate. In order to avoid adverse effects, the resource demands obtained by different methods will be prioritized according to the configuration strategy, the second resource demand will be adjusted to the highest priority, and the resources corresponding to the algorithm model will be expanded or reduced accordingly according to the second resource demand.

[0053] It can be seen that, through the configuration strategy, the resource requirements obtained in various different ways can be comprehensively processed according to the actual situation, and the priority can be adjusted, so as to select the resource requirements that are more in line with the actual situation so as to meet the actual resource scheduling requirements of the algorithm model. Based on the above embodiment, while using multiple methods to evaluate the resource requirements, the configuration strategy can be further used to screen out the resource requirements with higher accuracy according to the actual situation, with higher sensitivity and more accurate prediction effect, effectively solving the problem of insufficient resources or idle resources.

[0054] In one or more embodiments of the present application, Figure 3 A flow chart of a model training method provided in an embodiment of the present application. Figure 3 It can be seen that the online prediction model uses the access records before the current period for training optimization, including: 301: data processing of the sequentially acquired access records to generate multiple training samples containing the access subject identifier. 302: training the online prediction model according to the acquisition order of the training samples. 303: if the loss function of the trained online prediction model meets the threshold, it is determined that the prediction result meets the expected result.

[0055] For ease of understanding, the overall framework of the online prediction model will be illustrated below with reference to the accompanying drawings. Figure 4 The following is a schematic diagram of the framework of the online prediction model used as an example in the embodiment of the present application. Figure 4 As can be seen in the figure, it includes the initial model, streaming vector training data, streaming vector prediction data, online prediction model training, and online prediction model prediction. During online training, the training samples are input into the online prediction model training unit in the form of data streams for training. After the training is completed, the optimized model parameters are passed to the online prediction model prediction unit so that relevant prediction work can be performed based on the optimized model parameters.

[0056] In actual applications, after the online prediction model access platform starts to perform prediction work, the access records obtained online in real time are used to perform corresponding prediction work. At the same time, access records are also collected, for example, collected at a certain period, and the collected accesses are processed into data streams carrying access subject identifiers as training samples to train the online prediction model. It should be noted that in order to ensure the training effect, when using data streams for training, the data streams must be strictly input into the training unit of the online prediction model to be trained and optimized in the order of the data streams. The loss function of the online prediction model can be obtained through training. When the loss function meets the threshold, the training optimization is considered to be completed, and the latest online prediction model can be used to perform the corresponding prediction task.

[0057] In one or more embodiments of the present application, the obtaining of the access frequency of the user to the algorithm model in the current period and determining the second resource requirement includes: if the access frequency is greater than the first frequency threshold, determining the second resource requirement including expanding resource scheduling and expanding quantity. If the access frequency is less than the second frequency threshold, determining the second resource requirement including reducing resource scheduling and reducing quantity.

[0058] In actual applications, the user's access to the algorithm model can also be monitored, so as to directly determine whether the current resources can meet the resource demand and provide a basis for resource adjustment. This adjustment method is to make corresponding resource adjustments when there are signs of insufficient resources or idle resources. This adjustment method is a passive adjustment compared to the adjustment method based on the online prediction model. However, in this solution, the second resource demand is determined based on information such as the access frequency obtained in real time. Although it is a passive adjustment, it is responsive and can improve the timely effect of passive adjustment.

[0059] For example, assuming that the first frequency threshold is set to 30 times / second, the access frequency Qps of user A at the first moment is 10 times / second, and the frequency threshold at the second moment is 50 times / second, which is greater than the first frequency threshold of 30 times / second, and it exceeds the first frequency threshold for multiple consecutive moments (or multiple times within a period of time), it is considered that it is necessary to expand resource scheduling and increase the number. Set the second frequency threshold to 5 times / second, the access frequency Qps of user A at the third moment is 10 times / second, and the frequency threshold at the fourth moment is 4 times / second, which is less than the second frequency threshold of 5 times / second, and it is less than the first frequency threshold for multiple consecutive moments (or multiple times within a period of time), it is considered that it is necessary to reduce resource scheduling and reduce the number.

[0060] Since the access frequency is collected at a high frequency, the latest resource demand can be discovered in time, and the normal operation of the algorithm model can be avoided as much as possible due to insufficient resources. In scenarios where resource demand is complex and changeable, the accuracy of resource demand determined by only one method is not high. Therefore, this solution uses the method of calculating resource demand through online prediction model and real-time access frequency model at the same time to comprehensively evaluate how to schedule resources, so as to obtain more accurate and timely resource scheduling results to meet the resource demand of the algorithm model and avoid resource waste.

[0061] In addition, basic resource scheduling rules can be preset, such as setting adjustment thresholds for various resources. For example, the CPU expansion threshold is set to 90% utilization, and the reduction threshold is set to 30% utilization. In other words, when monitoring finds that the current CPU utilization reaches 90% and lasts for a period of time, it means that there is a risk of insufficient resources and it needs to be expanded; when monitoring finds that the current CPU utilization is greater than 30% and lasts for a period of time, it means that there are idle resources, so it needs to be reduced to release idle resources.

[0062] In one or more embodiments of the present application, the integrating the first resource demand and the second resource demand to expand or shrink the resources corresponding to the algorithm model includes: determining the comprehensive resource demand by the first resource demand and the second resource demand according to the weight relationship in the preset configuration strategy. Expand or shrink the resources accordingly according to the comprehensive resource demand.

[0063] In practical applications, in addition to adjusting the priorities of resource demands obtained in different ways, resource allocation strategies can also set corresponding weights for resource demands obtained in different ways, and determine the comprehensive resource demand through comprehensive calculation.

[0064] For example, the weight coefficient of the first resource demand obtained by the online prediction model is set to 0.8, and the weight coefficient of the second resource demand determined based on the real-time access frequency is set to 0.2. When the first resource demand is: expand the memory to 4G; the second resource demand is: expand the memory to 2G; after calculation, the comprehensive resource demand is 4*0.8+2*0.2=3.6. Since there is no 3.6G memory, the memory is expanded to 4G by rounding.

[0065] In practical applications, the weight coefficient can also be dynamically adjusted according to the resource allocation strategy. For example, when the prediction effect of the online prediction model after optimization is not good, the weight coefficient corresponding to the first resource demand can be adjusted to 0.1, and the weight coefficient corresponding to the second resource demand can be adjusted to 0.9, so as to obtain a more accurate prediction result.

[0066] When the resource demand evaluation of multiple algorithm models that are deployed on the same cloud platform and call the same set of resources is completed through the above embodiment, resources will be allocated to the algorithm model with a greater possibility of expansion demand. For example,

[0067] When a certain algorithm model has a capacity expansion demand, but the resources that can be scheduled at this time are not enough to meet the resource expansion demand of the algorithm model. At this time, an emergency resource scheduling request can be issued, and then the current resource usage of other algorithm models and the possibility of future capacity expansion can be obtained. Thus, some resources are released from the configuration resources of at least one algorithm model with more idle resources and less possibility of future capacity expansion to meet a certain algorithm model with urgent capacity expansion demand. Therefore, the capacity expansion or contraction scheme provided by the above embodiment can achieve efficient, accurate and timely resource evaluation and scheduling. The technical solution of the present application adopts a combination of active resource scheduling (capacity expansion or contraction) and passive resource scheduling (capacity expansion or contraction), has high sensitivity, is suitable for various situations of sudden resource scheduling demand, and can better improve resource utilization.

[0068] It should be noted that the algorithm model is used as an example to illustrate the implementation process of the solution in this application scheme. In practical applications, in addition to the algorithm model, other models can also be used, such as resource management models and other models with resource scheduling and configuration requirements.

[0069] Based on the same idea, the embodiment of the present application also provides another resource scheduling method. Figure 5 A flow chart of another resource scheduling method provided in an embodiment of the present application, which can be applied to a client, specifically includes the following steps:

[0070] 501: In response to the user's access request, generate an access record to the algorithm model deployed on the cloud platform, so that the algorithm model can call the resources of the cloud platform based on the access record request. The cloud platform predicts the first resource demand based on the user's access record to the algorithm model in the historical period; and determines the second resource demand based on the access frequency of the access record in the current period; and expands or shrinks the resources corresponding to the algorithm model accordingly based on the first resource demand and the second resource demand, so that the algorithm model meets the user's computing needs.

[0071] 502: Receive the calculation result fed back by the algorithm model.

[0072] It should be noted that when a user sends an access request to a corresponding algorithm model through a client, the access record can be recorded by the client and then sent to the algorithm model. Of course, it can also be recorded by the cloud platform, or the client and the cloud platform can record at the same time. In the following embodiments, the access record will be recorded on the client as an example, but it does not constitute a limitation on the technical solution of the present application.

[0073] As an optional solution, the user can send an access request to the corresponding algorithm module through the client. Since the algorithm model is deployed on the cloud platform and can be called by many users, although only one client sending an access request is used as an example here, in actual situations, there may be many users who initiate access requests to the algorithm model through the client. Since there are many users participating in the access and the resource requirements are complex and changeable, different methods are used to obtain the first resource requirement and the second resource requirement respectively. Then, the resource requirement that can meet the user's needs is determined according to the resource allocation strategy, and the resource capacity is expanded or reduced for the algorithm model according to the resource demand. The relevant technical solutions for determining the first resource demand and the second resource demand, as well as the relevant solutions for resource scheduling based on the first resource demand and the second resource demand have been explained in the above embodiments, and will not be repeated here. For details, please refer to Figures 1 to 4 The corresponding embodiments.

[0074] For ease of understanding, the following will take the cloud platform as a K8s cloud platform for resource scheduling as an example to illustrate this application solution. Figure 6 The resource scheduling process diagram is illustrated in the embodiment of the present application. Figure 6As can be seen in , after the algorithm model is connected to the cloud platform, the cloud platform will monitor the request information during the interaction of the algorithm model. The request information will be sent to the real-time Qps perception module through the message queue to perform Qps calculation in real time, and the Qps calculation result will be sent to the collaborative module that plays the role of configuration strategy adjustment. At the same time, the online prediction model will use the online access records to train the online prediction model one by one, so as to complete the iterative optimization of the online prediction model, and the online prediction model can achieve a good prediction effect. The online prediction model will output the first resource demand that plays a decision-making role based on the access record, and send the first resource demand to the collaborative module. In addition, basic rules for resource scheduling can be configured for the collaborative module (for example, the basic resource scheduling rule is set as: when the CPU utilization rate reaches 90%, the CPU is expanded). The collaborative module ultimately decides how to schedule resources based on the configuration strategy. The configuration strategy can set the priority, upper and lower limits, weight coefficient, etc. of the resource demand obtained in various ways. At the same time, the configuration strategy can also adjust the priority, upper and lower limits, weight coefficient, etc. as needed to obtain more accurate resource adjustment requirements. The resource adjustment module adjusts the resource requirements according to the different cloud platform resources connected (for example Figure 6 In the example, it is the K8s cloud platform), and the corresponding API is used to send and execute resource adjustment commands.

[0075] Based on the same idea, the embodiment of the present application also provides a resource scheduling device. Figure 7 A schematic diagram of the structure of a resource scheduling device provided in an embodiment of the present application. The resource scheduling device includes:

[0076] The prediction module 71 is used to predict the first resource demand based on the user's access records to the algorithm model in the historical period; wherein the algorithm model is used to provide corresponding computing services for the user.

[0077] The determination module 72 is used to obtain the access frequency of users to the algorithm model in the current period and determine the second resource demand.

[0078] The resource adjustment module 73 is used to comprehensively consider the first resource demand and the second resource demand, and to expand or reduce the resources corresponding to the algorithm model accordingly.

[0079] Optionally, the prediction module 71 is used to obtain the access subject identifier contained in the access record of the user to the algorithm model in the historical period; based on the access subject identifier, use the online prediction model to predict the first resource demand.

[0080] Optionally, a resource adjustment module 73 is configured to determine, based on a configuration strategy, whether to expand or reduce the resources corresponding to the algorithm model according to the first resource demand if the prediction result of the online prediction model meets the expected result;

[0081] If the prediction result of the online prediction model does not meet the expected result, it is determined based on the configuration strategy to expand or reduce the capacity of the resources corresponding to the algorithm model according to the second resource demand.

[0082] Optionally, the resource adjustment module 73 is also used to train and optimize the online prediction model using access records before the current time period. If the prediction result after training and optimization does not meet the expected result, it is determined based on the configuration strategy to expand or reduce the resources corresponding to the algorithm model according to the second resource demand.

[0083] Optionally, a training module 74 is further included, which is used to perform data processing on the access records acquired sequentially to generate a plurality of training samples containing the access subject identifiers;

[0084] Training the online prediction model according to the order in which the training samples are obtained;

[0085] If the loss function of the trained online prediction model meets the threshold, it is determined that the prediction result meets the expected result.

[0086] Optionally, the determination module 72 is configured to determine a second resource demand including expanding resource scheduling and expanding quantity if the access frequency is greater than a first frequency threshold;

[0087] If the access frequency is less than a second frequency threshold, a second resource requirement including reducing resource scheduling and reducing quantity is determined.

[0088] Optionally, the resource adjustment module 73 is further used to determine a comprehensive resource demand through the first resource demand and the second resource demand according to the weight relationship in the preset configuration strategy; and perform corresponding expansion or reduction according to the comprehensive resource demand.

[0089] An embodiment of the present application also provides an electronic device. The electronic device is a master node electronic device in a computing unit. Figure 8 The electronic device includes a memory 801, a processor 802 and a communication component 803;

[0090] The memory 801 is used to store programs;

[0091] The processor 802 is coupled to the memory and is configured to execute the program stored in the memory to:

[0092] Predicting the first resource demand based on the user's access records to the algorithm model during the historical period; wherein the algorithm model is used to provide the user with corresponding computing services;

[0093] Obtaining the access frequency of users to the algorithm model in the current period, and determining the second resource demand;

[0094] Based on the first resource demand and the second resource demand, the resources corresponding to the algorithm model are expanded or reduced accordingly.

[0095] The memory 801 can be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any application or method for operating on the electronic device. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0096] Furthermore, the processor 802 in this embodiment may specifically be: a programmable switching processing chip, in which a data replication engine is configured to replicate received data.

[0097] When the processor 802 executes the program in the memory, in addition to the above functions, it can also realize other functions, which can be specifically referred to the description of the above embodiments. Figure 8 As shown, the electronic device also includes: a power supply component 804 and other components.

[0098] The present application also provides a non-transitory machine-readable storage medium, wherein the non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes Figure 1 The method described in the corresponding embodiment.

[0099] The present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement Figure 1 The method described in the corresponding embodiment.

[0100] Based on the same idea, the embodiment of the present application also provides another resource scheduling device. Fig. 9A schematic diagram of the structure of another resource scheduling device provided in an embodiment of the present application. The resource scheduling device includes:

[0101] The sending module 91 is used to generate an access record to the algorithm model deployed on the cloud platform in response to the user's access request, so that the algorithm model can call the resources of the cloud platform based on the access record request. The cloud platform predicts the first resource demand based on the user's access record to the algorithm model in the historical period; and determines the second resource demand based on the access frequency of the access record in the current period; and the resources corresponding to the algorithm model are expanded or reduced accordingly based on the first resource demand and the second resource demand, so that the algorithm model meets the user's computing needs.

[0102] The receiving module 92 is used to receive the calculation result fed back by the algorithm model.

[0103] The present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement Figure 6 The method described in the corresponding embodiment.

[0104] An embodiment of the present application also provides an electronic device. The electronic device is a standby node electronic device in a computing unit. Fig.10 This is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application. The electronic device includes a memory 1001, a processor 1002 and a communication component 1003; wherein:

[0105] The memory 1001 is used to store programs;

[0106] The processor 1002 is coupled to the memory and is configured to execute the program stored in the memory to:

[0107] In response to a user's access request, an access record to an algorithm model deployed on a cloud platform is generated, so that the algorithm model calls the resources of the cloud platform based on the access record request; the cloud platform predicts a first resource demand based on the user's access record to the algorithm model in a historical period; and determines a second resource demand based on the access frequency of the access record in a current period; and the resources corresponding to the algorithm model are expanded or reduced accordingly based on the first resource demand and the second resource demand, so that the algorithm model meets the computing needs of the user;

[0108] Receive the calculation result fed back by the algorithm model.

[0109] The memory 1001 can be configured to store various other data to support operations on the electronic device. Examples of these data include instructions for any application or method for operating on the electronic device. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0110] Furthermore, the processor 1002 in this embodiment may specifically be: a programmable switching processing chip, in which a data replication engine is configured to replicate received data.

[0111] When the processor 1002 executes the program in the memory, in addition to the above functions, it can also realize other functions, which can be specifically referred to the description of the above embodiments. Fig.10 As shown, the electronic device also includes: a power supply component 1004 and other components.

[0112] The present application also provides a non-transitory machine-readable storage medium, wherein the non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes Figure 5 The method described in the corresponding embodiment.

[0113] The present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement Figure 5 The method described in the corresponding embodiment.

[0114] Based on the above embodiments, various models, such as algorithm models, are deployed on the cloud platform for users to call, so as to provide corresponding services for users. After the algorithm model is deployed on the cloud platform, the user's access to the algorithm model or the algorithm model to the cloud platform resources can be monitored in real time, and the current resource demand can be determined in time, and resources can be scheduled according to the resource demand to meet complex and changeable scheduling requirements. Specifically, the first resource demand with active scheduling effect is generated by using historical access records; and the second resource demand for passive resource adjustment is generated by real-time monitoring of the access frequency of the access record. Then, the first resource demand and the second resource demand are comprehensively utilized, and resource scheduling is performed according to the comprehensive processing result, so that a more timely and accurate resource scheduling effect can be obtained. The first resource demand can be predicted by an online prediction model, and the online prediction model can be trained and optimized in real time online. Therefore, accurate resource demand can be obtained in the face of diversified scheduling requirements. In order to avoid the situation where the prediction model is inaccurate and causes resource scheduling failure, a second resource demand is also configured. Through the above scheme, it is possible to cope with diversified and variable scheduling demand resource scheduling scenarios, achieve accurate and timely scheduling effects, and the scheduling scheme is more sensitive and flexible.

[0115] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A resource adjustment method, characterized in that: The method comprises: Based on the user's access records to the algorithm model during the historical period, the online prediction model is used to predict the first resource demand; wherein the algorithm model is used to provide the user with corresponding computing services; Obtaining the access frequency of users to the algorithm model in the current period, and determining the second resource demand; Based on the first resource demand and the second resource demand, resources corresponding to the algorithm model are expanded or reduced accordingly, including: If the prediction result of the online prediction model meets the expected result, determining based on the configuration strategy to expand or reduce the capacity of the resources corresponding to the algorithm model according to the first resource demand; If the prediction result of the online prediction model does not meet the expected result, it is determined based on the configuration strategy to expand or reduce the capacity of the resources corresponding to the algorithm model according to the second resource demand.

2. The method according to claim 1, characterized in that The predicting the first resource demand based on the user's access records to the algorithm model during the historical period includes: Obtaining the access subject identifier contained in the access record of the user to the algorithm model during the historical period; Based on the access subject identifier, the first resource demand is predicted using an online prediction model.

3. The method according to claim 2, characterized in that Also includes: The online prediction model is trained and optimized using access records before the current period. If the prediction result after training and optimization does not meet the expected result, it is determined based on the configuration strategy to expand or reduce the resources corresponding to the algorithm model according to the second resource demand.

4. The method according to claim 3, characterized in that: The online prediction model uses the access records before the current period for training optimization, including: Performing data processing on the sequentially acquired access records to generate a plurality of training samples including the access subject identifiers; Training the online prediction model according to the order in which the training samples are obtained; If the loss function of the trained online prediction model meets the threshold, it is determined that the prediction result meets the expected result.

5. The method according to claim 1, characterized in that: The obtaining the access frequency of the user to the algorithm model in the current period and determining the second resource demand includes: If the access frequency is greater than the first frequency threshold, determining a second resource demand including expanded resource scheduling and expanded quantity; If the access frequency is less than a second frequency threshold, a second resource requirement including reducing resource scheduling and reducing quantity is determined.

6. A resource scheduling method, characterized in that: Applied to a client, the method comprises: In response to a user's access request, an access record for an algorithm model deployed on a cloud platform is generated so that the algorithm model calls the resources of the cloud platform based on the access record request; the cloud platform predicts a first resource demand using an online prediction model based on the user's access record for the algorithm model in a historical period; and, based on the access frequency of the access record in the current period, determines a second resource demand; and based on the first resource demand and the second resource demand, the resources corresponding to the algorithm model are expanded or reduced accordingly so that the algorithm model meets the user's computing needs, including: if the prediction result of the online prediction model meets the expected result, then based on the configuration strategy, determine that the resources corresponding to the algorithm model are expanded or reduced according to the first resource demand; if the prediction result of the online prediction model does not meet the expected result, then based on the configuration strategy, determine that the resources corresponding to the algorithm model are expanded or reduced according to the second resource demand; Receive the calculation result fed back by the algorithm model.

7. A resource scheduling system, characterized in that: The system comprises: A cloud platform, for predicting a first resource demand using an online prediction model based on a user's access record to an algorithm model during a historical period; wherein the algorithm model is used to provide corresponding computing services to users; obtaining a user's access frequency to the algorithm model during a current period to determine a second resource demand; and, based on the first resource demand and the second resource demand, expanding or reducing the resource corresponding to the algorithm model accordingly; A client is used to generate an access record to an algorithm model deployed on a cloud platform in response to a user's access request, so that the algorithm model calls the resources of the cloud platform based on the access record request; the cloud platform predicts a first resource demand based on the user's access record to the algorithm model in a historical period; and determines a second resource demand based on the access frequency of the access record in the current period; and accordingly expands or shrinks the resources corresponding to the algorithm model based on the first resource demand and the second resource demand so that the algorithm model meets the user's computing needs, including: if the prediction result of the online prediction model meets the expected result, then determine based on the configuration strategy to expand or shrink the resources corresponding to the algorithm model according to the first resource demand; if the prediction result of the online prediction model does not meet the expected result, then determine based on the configuration strategy to expand or shrink the resources corresponding to the algorithm model according to the second resource demand; and receive the calculation result fed back by the algorithm model.

8. An electronic device comprising a memory and a processor; wherein: The memory is used to store programs; The processor is coupled to the memory, and is used to execute the program stored in the memory to implement the method described in any one of claims 1 to 5; or to implement the method described in claim 6.

9. A non-transitory machine-readable storage medium having executable code stored thereon, wherein when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described in any one of claims 1 to 5; or implement the method as described in claim 6 above.

Images