Computing resource scheduling system and method

Abstract

The present application discloses a computing resource scheduling system and method, which includes the following steps: a cluster management module detects computing resources from each server in a server cluster to obtain a set of available computing resources for the server; synchronizes the set of available computing resources for the server to the resource management module, and the resource management module constructs an available resource view based on the set of available computing resources for the server; in response to a control module receiving a deployment model instruction, the control module determines the computing resources required to deploy the model to be deployed based on the relevant information of the model to be deployed; the control module requests scheduling from a scheduling module, and the scheduling module obtains a list of allocated computing resources from the available resource view based on a scheduling algorithm; and, based on the allocated computing resource list, deploys the model to be deployed. In this way, the idle computing power on the unified server can provide a multi-machine and multi-card scheduling method, reduce idle computing resources, and improve resource utilization.

Description

Technical Field

[0001] The present application relates to the field of intelligent scheduling, and more specifically, to a computing resource scheduling system and method. Background Art

[0002] With the development of artificial intelligence technology, the application of large-scale AI models has gradually increased. AI large models have begun to be applied in various industries, bringing new momentum to the development of the industry. However, the deployment, operation and reasoning of these AI large models require powerful computing resources, such as GPU and NPU. For industry users who deploy AI large models in a privatized environment, they face the problem of diversified and limited computing resources. On the one hand, users in different industries have different requirements for AI large models, which leads to diversified demand for computing resources, such as generative language models of different scales (7B, 13B, 34B, 72B, etc.), BERT models, StableDiffusion, etc. On the other hand, the computing resources of industry users are usually limited. How to make full use of each unit of computing resources in the case of mixed deployment of multiple models has become an urgent problem to be solved.

[0003] As the scale of current AI models continues to grow, one computing resource such as a GPU is usually unable to provide model inference services, and multiple GPUs are required to run model inference in a distributed manner. Existing GPU scheduling methods usually only support a single-machine multi-card scheduling mode. Although this mode is simple and easy to implement, it has obvious shortcomings. For example, in a multi-machine multi-card scenario, the computing resource pool can usually only deploy one model, which may result in the idle computing power of multiple servers being unable to be effectively utilized and scheduled, resulting in a waste of resources. In addition, when part of the GPU resources of the server are occupied, the remaining GPUs may not be able to meet the deployment requirements of larger models, resulting in fragmentation of computing power, thereby reducing the overall utilization of resources.

[0004] Therefore, an optimized computing resource scheduling solution is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a computing resource scheduling system and method, which obtains the contextual semantic association features of each attribute of the model by encoding the specific relevant information of the model to be deployed (such as model type, parameter quantity, context length, data type, etc.) and context attribute association analysis, and then calculates the computing resources required for the model to be deployed based on the dynamic aggregation representation of the feature distribution and the expression of key information between the context semantics of these model attributes. In this way, the required computing resources can be automatically recommended and allocated according to the specific information of the model to be deployed. This automated computing resource allocation method can ensure the rationality and reliability of computing resource allocation, which helps to ensure the stability and cost-effectiveness of the overall system.

[0006] According to one aspect of the present application, a computing resource scheduling method is provided, which includes: when the system is initialized, the cluster management module detects computing resources from each server in the server cluster to obtain a set of available computing resources of the server; the set of available computing resources of the server is synchronized to the resource management module, and the resource management module constructs an available resource view based on the set of available computing resources of the server; in response to the control module receiving a deployment model instruction, the control module determines the computing resources required to deploy the model to be deployed according to relevant information of the model to be deployed; the control module requests scheduling from the scheduling module, and the scheduling module obtains a list of allocated computing resources from the available resource view based on a scheduling algorithm; after scheduling is completed, the available resource view is updated; and the model to be deployed is deployed based on the allocated computing resource list.

[0007] According to another aspect of the present application, a computing resource scheduling system is provided, which includes: a resource detection module, which is used when the system is initialized, and the cluster management module detects computing resources from each server in the server cluster to obtain a set of available computing resources of the server; a resource synchronization module, which is used to synchronize the set of available computing resources of the server to the resource management module, and the resource management module builds an available resource view based on the set of available computing resources of the server; a control response module, which is used to respond to the control module receiving a deployment model instruction, and the control module determines the computing resources required to deploy the model to be deployed according to the relevant information of the model to be deployed; a resource allocation module, which is used for the control module to request scheduling from the scheduling module, and the scheduling module obtains a list of allocated computing resources from the available resource view based on the scheduling algorithm; an available resource view update module, which is used to update the available resource view after scheduling is completed; and a to-be-deployed model deployment module, which is used to deploy the model to be deployed based on the allocated computing resource list.

[0008] Compared with the prior art, the present application provides a computing resource scheduling system and method, which obtains the context semantic association features of each attribute of the model by encoding the specific relevant information of the model to be deployed (such as model type, parameter quantity, context length, data type, etc.) and context attribute association analysis, and then calculates the computing resources required for the model to be deployed based on the dynamic aggregation representation of the feature distribution between the context semantics of these model attributes and the expression of key information. In this way, the required computing resources can be automatically recommended and allocated according to the specific information of the model to be deployed. This automated computing resource allocation method can ensure the rationality and reliability of computing resource allocation, which helps to ensure the stability and cost-effectiveness of the overall system. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 2 is a diagram of the computing resource scheduling system architecture according to an embodiment of the present application.

[0011] Figure 2 Flow chart of a computing resource scheduling method according to an embodiment of the present application.

[0012] Figure 3 Schematic diagram of data flow of a computing resource scheduling method according to an embodiment of the present application.

[0013] Figure 4 This is a flowchart of sub-step S3 of the computing resource scheduling method according to an embodiment of the present application.

[0014] Figure 5 A flowchart of a computing resource application scheduling process according to an embodiment of the present application.

[0015] Figure 6 The figure is a flowchart of computing resource recovery according to an embodiment of the present application.

[0016] Figure 7 4 is a block diagram of a computing resource scheduling system according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0018] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0019] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0020] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0022] Existing GPU scheduling methods usually only support the scheduling mode of single machine and multiple cards. Although this mode is simple and easy to implement, it has obvious shortcomings. For example, in the scenario of multiple machines and multiple cards, the computing resource pool can usually only deploy one model, which may result in the idle computing power of multiple servers being unable to be effectively utilized and scheduled, resulting in a waste of resources. In addition, when part of the GPU resources of the server are occupied, the remaining GPUs may not be able to meet the deployment requirements of larger models, resulting in fragmented computing power, thereby reducing the overall utilization of resources.

[0023] In view of the above technical problems, in the technical solution of this application, a computing resource scheduling method is proposed, which can give priority to single-machine multi-card scheduling and can realize multi-machine multi-card scheduling, so as to minimize the idle computing resources and improve the utilization rate of server resources. Specifically, Figure 1As shown in the figure, the system mainly includes four modules: cluster management module, resource management module, scheduling module, and control module. The computing resource scheduling system detects computing resources through the cluster management module, builds an available resource view through the resource management module, and dynamically schedules and allocates resources according to actual needs through the scheduling module, thereby minimizing the generation of idle computing resources and improving computing resource utilization. This not only optimizes the allocation of computing resources, but also reduces user costs, allowing users to more easily deploy AI models of various sizes.

[0024] In the technical solution of the present application, a computing resource scheduling method is proposed. Figure 2 Flow chart of a computing resource scheduling method according to an embodiment of the present application. Figure 3 Schematic diagram of data flow of the computing resource scheduling method according to an embodiment of the present application. Figure 2 and Figure 3 As shown, according to the computing resource scheduling method of the embodiment of the present application, the steps include: S1, when the system is initialized, the cluster management module detects computing resources from each server in the server cluster to obtain a set of available computing resources of the server; S2, the set of available computing resources of the server is synchronized to the resource management module, and the resource management module constructs an available resource view based on the set of available computing resources of the server; S3, in response to the control module receiving the deployment model instruction, the control module determines the computing resources required to deploy the model to be deployed according to the relevant information of the model to be deployed; S4, the control module requests scheduling from the scheduling module, and the scheduling module obtains the allocated computing resource list from the available resource view based on the scheduling algorithm; S5, after the scheduling is completed, the available resource view is updated; S6, based on the allocated computing resource list, the model to be deployed is deployed.

[0025] In particular, in said S1, when the system is initialized, the cluster management module detects computing resources from each server in the server cluster to obtain a set of available computing resources of the server. It should be understood that the resource management module can grasp the resource status in the cluster in real time. Through computing resource detection, the system can fully understand the computing resource status of all servers in the cluster, and provide accurate basic data for subsequent computing resource management and scheduling.

[0026] In particular, S2 synchronizes the set of available computing resources of the server to the resource management module, and the resource management module constructs an available resource view based on the set of available computing resources of the server. That is, the data structure of the available resource view is defined, and the data structure is: "Number of available computing resources": [node list], wherein the number of available computing resources is used as the granularity for scheduling computing resources in powers of two, that is, if a server has a total of 8 computing resources, then the "number of available computing resources" are "1", "2", "4", and "8" respectively. In a specific example, there are 4 servers numbered 0, 1, 2, and 3, each with 8 computing resources. No model is running at initialization, and the "number of available computing resources" of all nodes is 8. The initial available resource view is as follows:

[0027] {

[0028] "1":[],

[0029] "2":[],

[0030] "4":[],

[0031] “8”: [0,1,2,3]

[0032] }

[0033] In particular, in S3, in response to the control module receiving the deployment model instruction, the control module determines the computing resources required to deploy the model to be deployed according to the relevant information of the model to be deployed. In the above-mentioned computing resource scheduling method, the step of determining the computing resources required to deploy the model to be deployed according to the relevant information of the model to be deployed is crucial. This is because each AI model has its own unique computing requirements, and the relevant information of the model to be deployed contains information about the model type, parameter quantity, context length, data type, etc., which reflects the required computing resource requirements. Only by accurately knowing the resource requirements of the model can the corresponding computing resources be reasonably allocated to ensure that the model can run normally without insufficient resources, and at the same time, it can also avoid resource waste caused by excessive resource allocation. Therefore, in the technical solution of the present application, the contextual semantic association features of each attribute of the model are obtained by encoding the specific relevant information of the model to be deployed (such as model type, parameter quantity, context length, data type, etc.) and context attribute association analysis, and then the computing resources required for the model to be deployed are calculated based on the dynamic aggregation representation of the feature distribution between the context semantics of these model attributes and the expression of key information. Specifically, Figure 4As shown, S3 includes: S31, embedding and encoding each data item in the relevant information of the model to be deployed to obtain a set of model attribute embedding coding vectors; S32, inputting the set of model attribute embedding coding vectors into a context encoder based on a converter structure to obtain a set of model attribute context semantic coding vectors; S33, dynamically aggregating the feature distribution of the set of model attribute context semantic coding vectors to obtain a significantly aggregated representation of model attributes; S34, determining the computing resources required for the model to be deployed based on the significantly aggregated representation of model attributes.

[0034] Specifically, the S31 embeds and encodes each data item in the relevant information of the model to be deployed to obtain a set of model attribute embedding coding vectors. It should be understood that through embedded coding, the potential associations between data can be captured to better obtain the intrinsic structure and semantic information between each data item in the relevant information of the model to be deployed. It is worth mentioning that embedded coding is a technology that converts discrete, usually high-dimensional data into a low-dimensional continuous vector (i.e., embedded vector), which not only retains the key information of the original data, but also can capture the complex relationships between data, such as similarity, correlation, etc.

[0035] Specifically, in S32, the set of model attribute embedding coding vectors is input into a context encoder based on a converter structure to obtain a set of model attribute context semantic coding vectors. Considering that in computing resource scheduling, the attributes of the model (such as type, parameter quantity, context length and data type, etc.) do not exist in isolation, but are intrinsically related to each other. Therefore, after embedding and encoding these model-related information to map them into a common space, context encoding is performed on the embedded coding semantics of these model-related information to extract the context features and associations between the embedded semantics of each model attribute, thereby obtaining a set of model attribute feature representations that is more context-aware.

[0036] Specifically, the S33 performs dynamic aggregation of feature distribution on the set of model attribute context semantic coding vectors to obtain a significant aggregation representation of model attributes. It should be understood that each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors respectively contains context semantic association features of different aspects of model attributes, and there may be a large amount of redundant information in these feature information. Therefore, in order to be able to extract the most representative features from a large amount of model attribute context information and perform information aggregation representation, so as to focus on the information expression that best reflects the model characteristics and reduce redundancy, so as to facilitate the subsequent processing and execution of computing resource recommendation tasks, in the technical solution of the present application, the set of model attribute context semantic coding vectors is further subjected to dynamic aggregation of feature distribution to obtain a significant aggregation representation of model attributes. In particular, in the process of dynamic aggregation of feature distribution, each input model attribute context semantic coding vector is regarded as a "particle" in the field, and the interaction between these "particles" is modeled by a concept similar to a force field. This force field not only takes into account the direct spatial relationship between each model attribute context semantics (such as Euclidean distance), but also combines the properties of each model attribute context semantic coding vector (such as static energy factor) and the overall structure of the entire feature set. In this way, the model is able to dynamically adjust the importance of the contextual semantic encoding vectors of each model attribute, and integrate the information into a more compact and representative representation according to a specific aggregation strategy, namely the model attribute significant aggregation representation.

[0037] Specifically, the specific process of dynamically aggregating the feature distribution of the set of model attribute context semantic coding vectors includes: calculating the static energy factor of each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors to obtain a set of model attribute semantic static energy factors; based on the set of model attribute semantic static energy factors, performing feature significant aggregation on the set of model attribute context semantic coding vectors to obtain the model attribute significant aggregation representation. It should be understood that the dynamic aggregation process is not just a simple averaging or summing process. It emphasizes those vectors that occupy a more important position in the feature space by calculating the static energy factor and dynamic aggregation energy factor of each model attribute context semantic coding vector. This ensures that the model pays more attention to important model attribute context semantic features in subsequent processing. In addition, in the process of calculating the dynamic aggregation weight factor to perform weighted aggregation between different model attribute context semantics, those vectors that are considered less important will be given lower weights or even ignored. This can reduce the impact of noise on the final result and improve the robustness of the model. The model attribute significant aggregation representation obtained by dynamic aggregation can more clearly show the core characteristics of the model attribute. This is helpful for understanding model behavior, debugging models, and the required computing resources.

[0038] Among them, the static energy factor of each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors is calculated to obtain a set of model attribute semantic static energy factors, including: calculating the mean and variance of the model attribute context semantic coding vector to obtain the model attribute semantic mean and the model attribute semantic variance; subtracting the model attribute context semantic coding vector from the model attribute semantic mean by position, and calculating the fourth power of each position of the subtracted feature vector to obtain the model attribute semantic difference modulation vector; calculating the expected value of the model attribute semantic difference modulation vector to obtain the model attribute semantic expected value; dividing the model attribute semantic expected value by the square of the model attribute semantic variance, and inputting the divided value into the sigmoid function to obtain the model attribute semantic static energy factor.

[0039] More specifically, based on the set of model attribute semantic static energy factors, the set of model attribute context semantic coding vectors is feature-significantly aggregated to obtain the model attribute significantly aggregated representation, including: selecting the model attribute context semantic coding vector corresponding to the largest model attribute semantic static energy factor in the set of model attribute semantic static energy factors as the initial center vector of the model attribute semantic clustering; based on the spatial span between each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors and the initial center vector of the model attribute semantic clustering and the static energy factors of each model attribute context semantic coding vector and the static energy factors of the initial center vector of the model attribute semantic clustering, calculating the dynamic aggregation energy factors of each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors to obtain a set of model attribute semantic dynamic aggregation energy factors; inputting the set of model attribute semantic dynamic aggregation energy factors into a gated mask unit to obtain a set of model attribute semantic dynamic aggregation weight factors; using the set of model attribute semantic dynamic aggregation weight factors as weighted weights, calculating the weighted sum of the set of model attribute context semantic coding vectors to obtain a model attribute significantly aggregated representation vector as the model attribute significantly aggregated representation.In a specific example, based on the spatial span between each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors and the initial center vector of the model attribute semantic clustering, as well as the static energy factor of each model attribute context semantic coding vector and the static energy factor of the initial center vector of the model attribute semantic clustering, the process of calculating the dynamic aggregation energy factor of each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors to obtain a set of model attribute semantic dynamic aggregation energy factors includes: multiplying the static energy factor of the model attribute context semantic coding vector, the static energy factor of the initial center vector of the model attribute semantic clustering with a first weighting parameter to obtain a first model attribute semantic dynamic aggregation energy factor; The square of the spatial span between the semantic coding vector and the initial center vector of the model attribute semantic clustering is multiplied by a second weighting parameter to obtain a second model attribute semantic dynamic aggregation energy factor; the first model attribute semantic dynamic aggregation energy factor is divided by the second model attribute semantic dynamic aggregation energy factor to obtain the model attribute semantic dynamic aggregation energy factor; and the process of inputting the set of model attribute semantic dynamic aggregation energy factors into a gated mask unit to obtain a set of model attribute semantic dynamic aggregation weight factors includes: normalizing the set of model attribute semantic dynamic aggregation energy factors to obtain a set of normalized model attribute semantic dynamic aggregation energy factors; and masking the set of normalized model attribute semantic dynamic aggregation energy factors to obtain a set of model attribute semantic dynamic aggregation weight factors. More specifically, the process of normalizing the set of model attribute semantic dynamic aggregation energy factors includes: calculating the exponential function value with the natural constant e as the base and the negative of each model attribute semantic dynamic aggregation energy factor in the set of model attribute semantic dynamic aggregation energy factors as the exponent to obtain the set of model attribute semantic dynamic aggregation energy index factors; adding the set of model attribute semantic dynamic aggregation energy index factors and the constant one by position, and then calculating the inverse of each model attribute semantic dynamic aggregation energy index modulation factor in the set of model attribute semantic dynamic aggregation energy index modulation factors to obtain the set of normalized model attribute semantic dynamic aggregation energy factors. The process of masking the set of normalized model attribute semantic dynamic aggregation energy factors includes: in response to each normalized model attribute semantic dynamic aggregation energy factor in the set of normalized model attribute semantic dynamic aggregation energy factors being greater than a predetermined threshold, setting the normalized model attribute semantic dynamic aggregation energy factor to the original value and the rest to zero to obtain the set of model attribute semantic dynamic aggregation weight factors.

[0040] In summary, the set of model attribute context semantic coding vectors is dynamically aggregated by feature distribution to obtain a significant aggregation representation of the model attribute, including: the set of model attribute context semantic coding vectors is dynamically aggregated by feature distribution using the following feature dynamic aggregation formula to obtain a significant aggregation representation of the model attribute; wherein the feature dynamic aggregation formula is: ;in, is the set of model attribute context semantic encoding vectors, are the first, second, ..., and third in the set of the model attribute context semantic encoding vectors. ,..., model attribute context semantic encoding vector, It is The feature value of each position in the context semantic encoding vector of the model attribute, To calculate the expected value, and They are The mean and variance of yes function, yes The corresponding model attribute semantic static energy factor, To select the maximum value corresponding to value, is the maximum matching value, is the initial center vector of semantic clustering of model attributes, yes The corresponding model attribute semantic static energy factor, express and The space span between and is the weighting parameter, for The corresponding model attribute semantic dynamic aggregation energy factor, yes The corresponding normalized model attribute semantic dynamic aggregation energy factor, For masking, is a predetermined threshold, yes The corresponding model attribute semantic dynamic aggregation weight factor, is the number of vectors in the set of model attribute context semantic encoding vectors, is a significant aggregate representation vector of the model attributes.

[0041] Specifically, the S34 determines the computing resources required for the model to be deployed based on the significant aggregation representation of the model attributes. In a specific example of the present application, the model attribute significant aggregation representation vector is input into a decoder-based computing resource recommendation module to obtain the computing resources required for the model to be deployed. That is, the significant aggregation representation features between the contextual semantics of each attribute of the model are used to perform decoding regression, so as to calculate the computing resources required for the model to be deployed. In this way, the required computing resources can be automatically recommended and allocated according to the specific information of the model to be deployed. This automated computing resource allocation method can ensure the rationality and reliability of computing resource allocation, thereby ensuring that the model can run normally without insufficient resources, and at the same time, it can also avoid waste of resources caused by excessive resource allocation, which helps to ensure the stability and cost-effectiveness of the overall system.

[0042] In particular, in S4, the control module requests scheduling from the scheduling module, and the scheduling module obtains a list of allocated computing resources from the available resource view based on a scheduling algorithm. In a specific example of the present application, the computing resource application scheduling process is as follows: Figure 5 As shown, specifically, in response to receiving the request scheduling; traversing the available resource view to obtain the traversal result; and extracting the allocation method with the smallest server node number from the traversal result as the allocated computing resource list. In a specific example, when the scheduling model receives the request scheduling, the scheduling model traverses the available resource view, and if the current available resources meet the requirements, the result is added to the list in the form of [(resource quantity, node number)] to calculate the allocation plan, and then the allocation method with the least number of servers and the smallest node number is selected and returned to the control module.

[0043] For example, the current resource view is as follows:

[0044] {

[0045] "1":[2],

[0046] "2":[0],

[0047] "4": [0,1],

[0048] "8":[3]

[0049] }

[0050] When scheduling one computing resource, the available resources in the list are traversed and the resulting allocation scheme is [[(1,2)],[(1,0)],[(1,0)],[(1,1)],[1,3)]], and the server with the smallest node number is selected to allocate computing resources, that is, [(1,0)] is returned.

[0051] When scheduling two computing resources, the available resources in the list are traversed and the allocation scheme obtained is [[(2,0)],[(2,0),[(2,1)],[(2,3)]], and the server with the smallest node number is selected to allocate computing resources, that is, [(2,0)] is returned.

[0052] When scheduling 4 computing resources, the available resources in the list are traversed and the allocation scheme obtained is [[(4,0)],[(4,1)],[(4,3)]]; the server with the smallest node number is selected to allocate computing resources, that is, [(4,0)] is returned.

[0053] When scheduling 8 computing resources, the available resources in the list are traversed and the allocation scheme obtained is [[(4,0),(4,1)],[(8,3)]]; the scheme with the least number of servers is selected to allocate computing resources, that is, [(8,3)] is returned.

[0054] When scheduling 16 computing resources, if the model requirements cannot be met at present, it will return that there are no available resources. If there are 2 or more available computing resources equal to 8, the scheduling will be successful.

[0055] In particular, the S5 updates the available resource view after the scheduling is completed. That is, in the scheme of the present application, first, the server available computing resources of each server are split to obtain the resource splitting result; specifically, in response to the server available computing resources of each server being an integer power of two, the server available computing resources of each server are not split; and, in response to the server available computing resources of each server being not an integer power of two, the server available computing resources of each server are split from large to small to obtain the resource splitting result. In a specific example, when the scheduling module completes the computing resource scheduling, the available resources on the server are reduced. In order to ensure that the model can preferentially allocate computing resource splitting on one node, the algorithm splits the available resources from large to small according to the "number of available computing resources". For example: when server No. 0 is allocated 2 computing resources and 6 computing resources are left, the available computing resources can be split into a combination of 2 times 2 and 2 times 1. Therefore, the available computing resources on server 0 include two types of "available computing resources", 2 and 4; server 1 has 4 computing resources allocated and 4 computing resources remaining, so server 1 includes 4 "available computing resources"; server 2 has 7 computing resources allocated and 1 computing resources remaining, so it includes 1 "available computing resources"; server 3 is unallocated and includes 8 "available computing resources". The available resource view now becomes:

[0056] {

[0057] "1":[2],

[0058] "2": [0],

[0059] "4": [0,1],

[0060] "8":[3]

[0061] }

[0062] Then, the resource splitting result is synchronized to the available resource view to update the available resource view. Figure 6 As shown in the figure, when the control module deletes the model, the control module requests the scheduling module to release the resources, and the scheduling module updates the available resource view. In a specific example, after the system releases the computing resources, it first updates the node number to the corresponding "Number of Available Computing Resources" list. The algorithm traverses the updated view, and when it finds that the node number on a certain scheduling granularity is repeated, the available computing resources are merged to the next level. Iterate continuously until there are no repeated node numbers in the list on all "Number of Available Computing Resources", and the iteration ends.

[0063] For example, the available resources view before the model is released is:

[0064] {

[0065] "1":[2],

[0066] "2": [0],

[0067] "4": [0,1],

[0068] "8":[3]

[0069] }

[0070] When two computing resources are released on server 0, GPU 0 has two more resources. 0 is updated to the "2" list first, and the resource view first becomes:

[0071] {

[0072] "1":[2],

[0073] "2": [0,0],

[0074] "4": [0,1],

[0075] "8":[3]

[0076] }

[0077] The first iteration traverses the view and finds that "2" has the same server number, so it is merged into 4 available resources. The server number is deleted from the "2" list and added to the "4" list. The resource view becomes:

[0078] {

[0079] "1":[2],

[0080] "2":[],

[0081] "4": [0,0,1],

[0082] "8":[3]

[0083] }

[0084] The second iteration traverses the view, merging the available resources into 8. The server number is removed from the "4" list and added to the "8" list. The resource view becomes:

[0085] {

[0086] "1":[2],

[0087] "2":[],

[0088] "4":[1],

[0089] “8”: [0,3]

[0090] }

[0091] The third time the view is traversed, there are no duplicate node numbers in the resource lists of all types, and the iteration ends.

[0092] In particular, the S6 deploys the model to be deployed based on the list of allocated computing resources. That is, in the technical solution of the present application, the control module can accurately deploy the model to the specified computing resources according to the list of computing resources allocated by the scheduling module. In this way, it is ensured that the model can obtain all the required computing resources, avoid waste or shortage of resources, achieve efficient operation of the model, and improve the resource utilization of the system.

[0093] In summary, the computing resource scheduling method according to the embodiment of the present application is explained, which obtains the context semantic association features of each attribute of the model by encoding the specific relevant information of the model to be deployed (such as model type, parameter quantity, context length, data type, etc.) and context attribute association analysis, and then calculates the computing resources required for the model to be deployed based on the dynamic aggregation representation of the feature distribution between the context semantics of these model attributes and the expression of key information. In this way, the required computing resources can be automatically recommended and allocated according to the specific information of the model to be deployed. This automated computing resource allocation method can ensure the rationality and reliability of computing resource allocation, which helps to ensure the stability and cost-effectiveness of the overall system.

[0094] Furthermore, a computing resource scheduling system is also provided.

[0095] Figure 7FIG. 1 is a block diagram of a computing resource scheduling system according to an embodiment of the present application. Figure 7 As shown, according to the computing resource scheduling system 300 of the embodiment of the present application, it includes: a resource detection module 310, which is used when the system is initialized, and the cluster management module detects computing resources from each server in the server cluster to obtain a set of available computing resources of the server; a resource synchronization module 320, which is used to synchronize the set of available computing resources of the server to the resource management module, and the resource management module builds an available resource view based on the set of available computing resources of the server; a control response module 330, which is used to respond to the control module receiving a deployment model instruction, and the control module determines the computing resources required to deploy the model to be deployed according to the relevant information of the model to be deployed; a resource allocation module 340, which is used for the control module to request scheduling from the scheduling module, and the scheduling module obtains the allocated computing resource list from the available resource view based on the scheduling algorithm; an available resource view update module 350, which is used to update the available resource view after the scheduling is completed; a to-be-deployed model deployment module 360, which is used to deploy the model to be deployed based on the allocated computing resource list.

[0096] As described above, the computing resource scheduling system 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a computing resource scheduling algorithm. In a possible implementation, the computing resource scheduling system 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the computing resource scheduling system 300 can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the computing resource scheduling system 300 can also be one of the many hardware modules of the wireless terminal.

[0097] Alternatively, in another example, the computing resource scheduling system 300 and the wireless terminal may also be separate devices, and the computing resource scheduling system 300 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0098] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A computing resource scheduling method, characterized in that: include: When the system is initialized, the cluster management module detects computing resources from each server in the server cluster to obtain a set of available computing resources of the servers; Synchronizing the set of available computing resources of the server to a resource management module, wherein the resource management module constructs an available resource view based on the set of available computing resources of the server; In response to the control module receiving the deployment model instruction, the control module determines the computing resources required to deploy the model to be deployed according to the relevant information of the model to be deployed; the control module requests scheduling from the scheduling module, and the scheduling module obtains the list of allocated computing resources from the available resource view based on the scheduling algorithm; after the scheduling is completed, the available resource view is updated; based on the allocated computing resource list, the model to be deployed is deployed; Wherein, in response to the control module receiving a deployment model instruction, the control module determines the computing resources required to deploy the model to be deployed based on the relevant information of the model to be deployed, including: embedding and encoding each data item in the relevant information of the model to be deployed to obtain a set of model attribute embedding coding vectors; inputting the set of model attribute embedding coding vectors into a context encoder based on a converter structure to obtain a set of model attribute context semantic coding vectors; performing feature distribution dynamic aggregation on the set of model attribute context semantic coding vectors to obtain a model attribute significant aggregation representation, including: calculating the static energy factor of each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors to obtain a set of model attribute semantic static energy factors; based on the set of model attribute semantic static energy factors, performing feature significant aggregation on the set of model attribute context semantic coding vectors to obtain the model attribute significant aggregation representation; based on the model attribute significant aggregation representation, determining the computing resources required for the model to be deployed.

2. The computing resource scheduling method according to claim 1, characterized in that: The set of available computing resources of the server is synchronized to the resource management module, and the resource management module constructs an available resource view based on the set of available computing resources of the server, including: defining a data structure of the available resource view, the data structure is: "Number of available computing resources": [node list], wherein the number of available computing resources is a power of two as a granularity for scheduling computing resources.

3. The computing resource scheduling method according to claim 2, characterized in that: The control module requests scheduling from the scheduling module, and the scheduling module obtains a list of allocated computing resources from the available resource view based on a scheduling algorithm, including: responding to receiving the request for scheduling; traversing the available resource view to obtain a traversal result; and extracting the allocation method with the smallest server node number from the traversal result as the allocated computing resource list.

4. The computing resource scheduling method according to claim 3, characterized in that: After the scheduling is completed, the available resource view is updated, including: performing resource splitting on the server available computing resources of each server to obtain a resource splitting result; and synchronizing the resource splitting result to the available resource view to update the available resource view.

5. The computing resource scheduling method according to claim 4, characterized in that: The server available computing resources of each server are split to obtain a resource split result, including: in response to the server available computing resources of each server being an integer power of two, not splitting the server available computing resources of each server; in response to the server available computing resources of each server being not an integer power of two, splitting the server available computing resources of each server in a descending order to obtain the resource split result.

6. The computing resource scheduling method according to claim 5, characterized in that: Calculate the static energy factor of each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors to obtain a set of model attribute semantic static energy factors, including: calculating the mean and variance of the model attribute context semantic coding vector to obtain the model attribute semantic mean and the model attribute semantic variance; subtracting the model attribute context semantic coding vector from the model attribute semantic mean by position, and calculating the fourth power of each position of the subtracted feature vector to obtain the model attribute semantic difference modulation vector; calculating the expected value of the model attribute semantic difference modulation vector to obtain the model attribute semantic expected value; dividing the model attribute semantic expected value by the square of the model attribute semantic variance, and inputting the divided value into a sigmoid function to obtain the model attribute semantic static energy factor.

7. The computing resource scheduling method according to claim 6, characterized in that: Based on the set of model attribute semantic static energy factors, the set of model attribute context semantic coding vectors is feature-significantly aggregated to obtain the model attribute significantly aggregated representation, including: selecting the model attribute context semantic coding vector corresponding to the largest model attribute semantic static energy factor in the set of model attribute semantic static energy factors as the model attribute semantic clustering initial center vector; based on the spatial span between each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors and the model attribute semantic clustering initial center vector and the static energy factors of each model attribute context semantic coding vector and the static energy factors of the initial center vector of the model attribute semantic clustering, calculating the dynamic aggregation energy factors of each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors to obtain a set of model attribute semantic dynamic aggregation energy factors; inputting the set of model attribute semantic dynamic aggregation energy factors into a gated mask unit to obtain a set of model attribute semantic dynamic aggregation weight factors; using the set of model attribute semantic dynamic aggregation weight factors as weighted weights, calculating the weighted sum of the set of model attribute context semantic coding vectors to obtain a model attribute significantly aggregated representation vector as the model attribute significantly aggregated representation.

8. The computing resource scheduling method according to claim 7, characterized in that: Based on the model attribute significant aggregation representation, determining the computing resources required for the model to be deployed includes: inputting the model attribute significant aggregation representation vector into a decoder-based computing resource recommendation module to obtain the computing resources required for the model to be deployed.

9. A computing resource scheduling system, characterized in that: include: The resource detection module is used when the system is initialized. The cluster management module detects computing resources from each server in the server cluster to obtain a set of available computing resources of the server; A resource synchronization module, used to synchronize the set of available computing resources of the server to a resource management module, wherein the resource management module constructs an available resource view based on the set of available computing resources of the server; A control response module, configured to, in response to the control module receiving a deployment model instruction, determine the computing resources required to deploy the model to be deployed according to relevant information of the model to be deployed; A resource allocation module, used for the control module to request scheduling from the scheduling module, and the scheduling module obtains a list of allocated computing resources from the available resource view based on a scheduling algorithm; an available resource view updating module, used for updating the available resource view after scheduling is completed; A to-be-deployed model deployment module, used to deploy the to-be-deployed model based on the allocated computing resource list; Wherein, in response to the control module receiving a deployment model instruction, the control module determines the computing resources required to deploy the model to be deployed based on the relevant information of the model to be deployed, including: embedding and encoding each data item in the relevant information of the model to be deployed to obtain a set of model attribute embedding coding vectors; inputting the set of model attribute embedding coding vectors into a context encoder based on a converter structure to obtain a set of model attribute context semantic coding vectors; performing feature distribution dynamic aggregation on the set of model attribute context semantic coding vectors to obtain a model attribute significant aggregation representation, including: calculating the static energy factor of each model attribute context semantic coding vector in the set of model attribute context semantic coding vectors to obtain a set of model attribute semantic static energy factors; based on the set of model attribute semantic static energy factors, performing feature significant aggregation on the set of model attribute context semantic coding vectors to obtain the model attribute significant aggregation representation; based on the model attribute significant aggregation representation, determining the computing resources required for the model to be deployed.

Images