A method for managing and scheduling heterogeneous computing resources

Abstract

The application discloses a kind of methods for managing and scheduling heterogeneous computing resources, and its technical scheme main points are: including the following specific steps: step one, build management and scheduling platform, build the network management platform of the management and scheduling of heterogeneous computing resources, and with scheduler network connection, arrange management computing resources;Step two, the task collection of management and scheduling platform, receiving end receives the resource computing task sent by client, and determines the resource characteristics of this resource computing task allocation, and the kind of resource is distributed;Step three, the classification of resource priority, according to the dynamic allocation adjustment sequence of management and scheduling platform, dynamically allocate resources of different priority, unused resource computing task margin is scheduled to other resource computing tasks, improve the management and scheduling performance of heterogeneous computing resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a method for managing and scheduling heterogeneous computing resources. BACKGROUND

[0002] With the development of digital economy and the promotion of Internet of Things, the amount of data has shown explosive growth, and the competition for computing power has become an important part of the competition for artificial intelligence. The rapid growth of data and the increasing diversity of data sources have led to the emergence of unstructured data. How to efficiently and reliably process and use these diversified data has become one of the core challenges in the era of big data. Heterogeneous computing platforms can fully utilize hardware advantages, adapt algorithm models to hardware characteristics, adapt to complex and diverse data forms, and meet the diversified needs of various upper-layer applications for computing resources and computing power.

[0003] For example, a Chinese patent with the authorization announcement number CN103365713A discloses a resource scheduling and management method, which includes: a scheduling and management platform receiving a resource computing task sent by a client, the resource computing task including one or more subtasks; the scheduling and management platform determining resource characteristics allocated to the resource computing task according to the resource computing task, the resource characteristics including: resource quantity and resource specification attributes; the scheduling and management platform applying to a resource providing platform for resources determined to meet the resource characteristics; and the scheduling and management platform scheduling the resource computing task to the applied resources for execution.

[0004] The above resource scheduling and management method has the advantages of realizing resource application, scheduling and release; however, the above resource scheduling and management still has some disadvantages, such as: unreasonable allocation of resource computing, some unused resource computing task margins are not used by users, and some resource computing task margins are only occasionally used, while the use of remaining resource computing task margins is fiercely competitive, which cannot meet the management and scheduling of heterogeneous computing resources. SUMMARY

[0005] The present application aims to provide a method for managing and scheduling heterogeneous computing resources to solve the problems in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0007] A method for managing and scheduling heterogeneous computing resources includes the following specific steps:

[0008] Step 1: Construct a management and scheduling platform, build a network management platform commonly used for the management and scheduling of heterogeneous computing resources, and connect with a scheduler to arrange management computing resources;

[0009] Step two, the task collection of the management and scheduling platform receives the resource calculation task sent by the client, determines the resource characteristics of the resource calculation task allocation, and allocates the types of resources;

[0010] Step three, resource priority classification, according to the dynamic allocation adjustment sequence of the management and scheduling platform, dynamically allocate resources of different priorities, and use genetic algorithm to classify the adaptive level of the dynamic resource allocation of the management and scheduling platform;

[0011] Step four, classification of computing scenarios, according to the massive heterogeneous computing power, different scenarios need different computing power for collaborative processing, and through the computing power network to coordinate and manage the scheduling of heterogeneous computing power, and allocate different resources to different computing scenarios after classification;

[0012] Step five, resource scheduling throughput calculation, using scheduling algorithm to calculate the throughput of the computing power resources and resource computing power of the management and scheduling platform, real-time identification and monitoring of used resource calculation tasks and unused resource calculation tasks, and scheduling the unused resource calculation task margin to other resource calculation tasks.

[0013] Preferably, the management and scheduling platform is provided with a data storage library, and the data storage library is provided with a plurality of independent data storage areas.

[0014] Preferably, the resource characteristics include large data volume, various types, low value density, high speed and high timeliness.

[0015] Preferably, the management and scheduling platform is provided with a user behavior analysis module, which submits tasks and checks personnel results, statistically analyzes historical data, and uses a corresponding distribution model to describe the running state and expected psychological task time characteristics of the user time slice. User behavior data is grouped by service type.

[0016] Preferably, the different computing scenarios include high-performance computing, Internet of Things, edge computing, and artificial intelligence scenarios.

[0017] Preferably, in step five, the scheduling algorithm includes polling algorithm, maximum carrier-to-interference ratio algorithm and proportional fair algorithm.

[0018] Preferably, the calculation formula of the proportional fair algorithm is as follows:

[0019]

[0020] Where R m,n (t)n=1,2,...N is the instantaneous rate of the mth user in the nth resource block at the tth transmission time interval, Tm,n (t) is the moving average throughput of user m at each resource block n in the window length of t time slots, M is the throughput.

[0021] Preferably, the instantaneous rate of the mth user at the nth subcarrier can be obtained by the following formula

[0022] Rm,n(t) = B / N log2(1 + SNR)

[0023] Where Rm,n(t) is the transmission rate of the mth user at the tth time slot, B is the total bandwidth, N is the number of subcarriers, and SNR is the signal-to-noise ratio.

[0024] Preferably, the algorithm formula of the maximum carrier-to-interference ratio algorithm is as follows:

[0025] k = argmax Rl(t);

[0026] Where k is the scheduled user, and Rl(t) is the instantaneous transmission rate of the ith user.

[0027] Compared with the prior art, the present application has the following beneficial effects:

[0028] In the method for managing and scheduling heterogeneous computing power resources, the management and scheduling platform is constructed, network connection with the scheduler is facilitated, management computing resources are arranged, resource characteristics of resource computing task allocation are determined, and different priority resources are dynamically allocated. The adaptive level of dynamic resource allocation of the management and scheduling platform is classified by using a genetic algorithm, different resources after classification are allocated to different computing scenarios through the computing power network, and the scheduling algorithm is used to calculate the computing power resources of the management and scheduling platform and the throughput of the resource computing power. The used resource computing tasks and the unused resource computing tasks are identified and monitored in real time, the unused resource computing task margin is scheduled to other resource computing tasks, and the management and scheduling performance of the heterogeneous computing power resources is improved. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The structure flow chart of the present application is shown in the figure. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0031] Embodiment 1

[0032] Referring to Figure 1 The application provides a method for managing and scheduling heterogeneous computing resources, wherein the technical solution is as follows:

[0033] The method comprises the following specific steps:

[0034] Step 1: Building a management and scheduling platform, building a network management platform shared by the management and scheduling of heterogeneous computing resources, and connecting with the scheduler to arrange the management of computing resources.

[0035] Step 2: Task collection of the management and scheduling platform, receiving the resource computing task sent by the client at the receiving end, determining the resource characteristics of the resource computing task allocation, and allocating the types of resources.

[0036] Step 3: Classification of resource priority, dynamically allocating resources of different priorities according to the dynamic allocation adjustment order of the management and scheduling platform, and classifying the adaptive level of the dynamic resource allocation of the management and scheduling platform using a genetic algorithm.

[0037] Step 4: Classification of computing scenarios, according to the massive heterogeneous computing power, different scenarios need different computing power for collaborative processing, and the classified different resources are allocated to different computing scenarios through the computing power network to collaboratively and manage and schedule the heterogeneous computing power.

[0038] Step 5: Resource scheduling throughput calculation, using a scheduling algorithm to calculate the throughput of the computing power resources and resource computing power of the management and scheduling platform, and identifying and monitoring the used and unused resource computing tasks in real time, and scheduling the unused resource computing task margin to other resource computing tasks.

[0039] In this embodiment, preferably, the management and scheduling platform is provided with a data storage library, and the data storage library is provided with a plurality of independent data storage areas.

[0040] In this embodiment, preferably, the resource characteristics include large data volume, various types, low value density, high speed, and high timeliness.

[0041] In this embodiment, preferably, the management and scheduling platform is provided with a user behavior analysis module, the user behavior analysis module submits tasks and checks personnel results, statistically analyzes historical data, uses a corresponding distribution model to describe the running state and expected psychological task time characteristics of the user time slice, and groups the user behavior data according to service types.

[0042] In this embodiment, preferably, the different computing scenarios include high-performance computing, Internet of Things, edge computing, and artificial intelligence scenarios.

[0043] In this embodiment, preferably, in step five, the scheduling algorithm includes a polling algorithm, a maximum carrier-to-interference ratio algorithm, and a proportional fair algorithm.

[0044] In this embodiment, preferably, the calculation formula of the proportional fair algorithm is as follows:

[0045]

[0046] where R m,n (t)n=1,2,...N is the instantaneous rate of the mth user in the nth resource block at the tth transmission time interval, T m,n (t) is the moving average throughput of user m in each resource block n in the tth time slot with a window length of M is the throughput.

[0047] Embodiment 2

[0048] Please refer to Figure 1 The present application provides a method for managing and scheduling heterogeneous computing resources, wherein the technical solution is as follows:

[0049] The method comprises the following specific steps:

[0050] Step one, build a management and scheduling platform, build a network management platform for the management and scheduling of heterogeneous computing resources, and connect with the scheduler to arrange the management of computing resources;

[0051] Step two, task collection of the management and scheduling platform, the receiving end receives the resource computing task sent by the client, determines the resource characteristics of the resource computing task allocation, and allocates the types of resources;

[0052] Step three, resource priority classification, according to the dynamic allocation adjustment order of the management and scheduling platform, dynamically allocate resources of different priorities, and use genetic algorithm to classify the adaptive level of dynamic resource allocation of the management and scheduling platform;

[0053] Step four, classification of computing scenarios, according to the massive heterogeneous computing power, different scenarios need different computing power for collaborative processing, through the computing power network to coordinate and manage the scheduling of heterogeneous computing power, and allocate different resources after classification to different computing scenarios;

[0054] Step five, resource scheduling throughput calculation, use scheduling algorithm to calculate the throughput of the computing power resources and resource computing power of the management and scheduling platform, real-time identify and monitor the used resource computing task and unused resource computing task, and schedule the unused resource computing task margin to other resource computing tasks.

[0055] In this embodiment, preferably, the management and scheduling platform is provided with a data storage library, and the data storage library is provided with a plurality of independent data storage areas.

[0056] In this embodiment, preferably, the resource characteristics include large data volume, various types, low value density, high speed and high timeliness.

[0057] In this embodiment, preferably, the management and scheduling platform is provided with a user behavior analysis module, which submits tasks and checks personnel results, and statistically analyzes historical data, uses a corresponding distribution model to describe the running state in the time slice of the user and the time characteristics of the expected psychological task, and groups the user behavior data according to the service type.

[0058] In this embodiment, preferably, the different computing scenarios include high-performance computing, Internet of Things, edge computing and artificial intelligence scenarios.

[0059] In this embodiment, preferably, in step five, the scheduling algorithm includes a round-robin algorithm, a maximum carrier-to-interference ratio algorithm and a proportional fairness algorithm.

[0060] In this embodiment, preferably, the calculation formula of the proportional fairness algorithm is as follows:

[0061]

[0062] Where R m,n (t)n=1,2,...N is the instantaneous rate of the mth user in the nth resource block at the tth transmission time interval, T m,n (t) is the moving average throughput of user m in each resource block n in the tth time slot with a window length of M is the throughput.

[0063] In this embodiment, preferably, the instantaneous rate of the mth user in the nth subcarrier can be obtained by the following formula:

[0064] Rm,n(t)=B / Nlog2(1+SNR)

[0065] Where Rm,n(t) is the transmission rate of the mth user in the tth time slot, B is the total bandwidth, N is the number of subcarriers, and SNR is the signal-to-noise ratio.

[0066] Embodiment 3

[0067] Please refer to Figure 1 The present application provides a kind of method for managing and scheduling heterogeneous computing resource, wherein technical scheme is as follows:

[0068] Comprise the following specific steps:

[0069] Step one, build a management and scheduling platform, build a network management platform for the management and scheduling of heterogeneous computing resources, and connect with the scheduler, and arrange the management of computing resources;

[0070] Step two, task collection of the management and scheduling platform, the receiving end receives the resource computing task sent by the client, determines the resource characteristics of the resource computing task allocation, and allocs the types of resources;

[0071] Step three, resource priority classification, according to the dynamic allocation adjustment sequence of the management and scheduling platform, dynamically allocate resources of different priorities, and use genetic algorithm to classify the adaptive level of the dynamic resource allocation of the management and scheduling platform;

[0072] Step four, classification of computing scenarios, according to the massive heterogeneous computing power, different scenarios need different computing power for collaborative processing, through the computing power network to coordinate and manage the scheduling of heterogeneous computing power, and allocate different resources to different computing scenarios after classification;

[0073] Step five, resource scheduling throughput calculation, using scheduling algorithm to calculate the throughput of the computing power resources and resource computing power of the management and scheduling platform, and real-time identification and monitoring of used and unused resource computing tasks, and scheduling the unused resource computing task margin to other resource computing tasks.

[0074] In this embodiment, preferably, the management and scheduling platform is provided with a data storage library, and the data storage library is provided with a plurality of independent data storage areas.

[0075] In this embodiment, preferably, the resource characteristics include large data volume, various types, low value density, high speed, and high timeliness.

[0076] In this embodiment, preferably, the management and scheduling platform is provided with a user behavior analysis module, the user behavior analysis module submits tasks and checks personnel results, statistically analyzes historical data, and uses a corresponding distribution model to describe the running state and expected psychological task time characteristics of the user time slice. User behavior data is grouped by service type.

[0077] In this embodiment, preferably, the different computing scenarios include high-performance computing, Internet of Things, edge computing, and artificial intelligence scenarios.

[0078] In this embodiment, preferably, in step five, the scheduling algorithm includes a polling algorithm, a maximum carrier-to-interference ratio algorithm, and a proportional fair algorithm.

[0079] In this embodiment, preferably, the calculation formula of the proportional fair algorithm is as follows:

[0080]

[0081] wherein R m,n (t)n = 1, 2,... N is the instantaneous rate of the mth user in the nth resource block in the tth transmission time interval, T m,n (t) is the moving average throughput of the user m in each resource block n in the tth time slot with a window length of M is the throughput.

[0082] In this embodiment, the instantaneous rate of the mth user in the nth subcarrier can be obtained by the following formula

[0083] Rm,n(t) = B / N log2(1 + SNR)

[0084] wherein Rm,n(t) is the transmission rate of the mth user in the tth time slot, B is the total bandwidth, N is the number of subcarriers, and SNR is the signal-to-noise ratio.

[0085] In this embodiment, the algorithm formula of the maximum carrier-to-interference ratio algorithm is as follows:

[0086] k = argmax Ri(t) ;

[0087] wherein k is the scheduled user, and Ri(t) is the instantaneous transmission rate of the ith user.

[0088] Working principle and use flow of the application:

[0089] The method for managing and scheduling heterogeneous computing power resources, when in use, constructs a management and scheduling platform, facilitates network connection with a scheduler, arranges management computing resources, allocates resources according to resource computing task allocation characteristics, and allocates different priority resources, classifies the adaptive level of the management and scheduling platform dynamic resource allocation using a genetic algorithm, collaborates and manages and schedules heterogeneous computing power through a computing power network, allocates different resources after classification to different computing scenarios, calculates the throughput of the management and scheduling platform computing power resources and resource computing power using a scheduling algorithm, identifies and monitors used resource computing tasks and unused resource computing tasks in real time, schedules unused resource computing task margins to other resource computing tasks, and improves the management and scheduling performance of heterogeneous computing power resources.

[0090] Although embodiments of the application have been shown and described, it is to be understood that the application is not limited to the details of the foregoing embodiment, and that the application can be practiced with modification and change within the scope and spirit of the appended claims.

Claims

1. A method for managing and scheduling heterogeneous computing resources, the method comprising: Comprise the following specific steps: ​ Step one, build management and scheduling platform, build the management and scheduling of heterogeneous computing resources of network management platform, and network connection with the dispatcher, arrange management computing resources; Step two, task collection of management and scheduling platform, receiving end receives the resource computing task sent by the client, and determines the resource characteristics of the resource computing task allocation, and allocates the type of resources; Step three, resource priority classification, according to the dynamic allocation adjustment sequence of management and scheduling platform, dynamically allocate resources with different priority, and use genetic algorithm to classify the adaptive level of dynamic resource allocation of management and scheduling platform; Step four, classification of computing scenarios, according to the massive heterogeneous computing power, different scenes need different computing power for collaborative processing, through the computing power network to coordinate and manage the scheduling of heterogeneous computing power, and allocate different resources to different computing scenarios after classification; Step five, resource scheduling throughput calculation, using scheduling algorithm to calculate the throughput of management and scheduling platform computing resources and resource computing power, real-time identification and monitoring of used resource computing tasks and unused resource computing tasks, and scheduling the unused resource computing task margin to other resource computing tasks; In step five, the scheduling algorithm includes polling algorithm, maximum carrier to interference ratio algorithm and proportional fair algorithm; The calculation formula of the proportional fair algorithm is as follows: ； wherein n = 1, 2,... N is the instantaneous rate of the mth user in the nth resource block in the tth transmission time interval, is the moving average throughput of user m in each resource block n in the tth time slot with a window length of M is the throughput; The instantaneous rate of the mth user at the nth subcarrier can be obtained by the following formula: Rm,n(t)=B / Nlog2(1+SNR) Where Rm,n(t) is the transmission rate of the mth user at the tth time slot, B is the total bandwidth, N is the number of subcarriers, and SNR is the signal to noise ratio; The algorithm formula of the maximum carrier to interference ratio algorithm is as follows: k=argmaxRi(t) Where k is the user to be scheduled, and Ri(t) is the instantaneous transmission rate of the ith user.

2. The method of claim 1, wherein: The management and scheduling platform is provided with a data storage library, and the data storage library is provided with a plurality of independent data storage areas.

3. The method of claim 1, wherein: The resource characteristics include large data volume, various types, low value density, high speed and high timeliness.

4. The method of claim 1, wherein: The management and scheduling platform is provided with a user behavior analysis module, which submits tasks and checks personnel results, statistically analyzes historical data, uses corresponding distribution model to describe the running state and expected psychological task time characteristics of user time segment, and groups user behavior data according to service type.

5. The method of claim 1, wherein: The different computing scenarios include high performance computing, Internet of things, edge computing and artificial intelligence scenarios.

Images