A method and system for managing and scheduling computing resources

Abstract

The present invention discloses a management and scheduling method and system for computing power resources, comprising: a cloud data server generates a computing power container pool corresponding to a computing power node according to computing power node data collected by a computing power node data collection device; a historical data statistics module obtains the operating characteristics of the computing power node according to the historical data of the computing power node, and the computing power container pool corresponding to the computing power node generates a computing power demand task according to the operating characteristics of the computing power node, and sends it to the cloud data server; the cloud data server generates a computing power container pool sequence corresponding to the first computing power node according to the first matching feature in the computing power demand task; according to the computing power container confirmation information returned by the client, the computing power container corresponding to the computing power node connects the computing power container with the client module according to the computing power container information, and sends the computing power demand information to the computing power container to complete the scheduling of computing power resources.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method and system for managing and scheduling computing resources. Background Art

[0002] In today's information age, computing resources have become an important driving force for social progress and development. However, with the rapid development of technologies such as cloud computing, big data, and artificial intelligence, the scheduling and management of computing resources face many challenges. How to optimize the configuration of computing resources and improve utilization has become the focus of the industry. In the existing computing resource management and scheduling methods, there are the following problems:

[0003] Low resource utilization: Traditional resource allocation methods cannot be adjusted dynamically according to actual needs, resulting in a large number of resources being idle or wasted. Low scheduling efficiency: Complex tasks and large-scale data processing require efficient scheduling algorithms and system support, but existing technologies are difficult to meet the requirements of real-time and reliability. Excessive energy consumption: The energy consumption problem of computing infrastructure such as data centers is becoming increasingly serious. How to reduce energy consumption while increasing computing power has become an urgent problem to be solved. Security risks: Computing resources involve a large amount of sensitive information and privacy data during use. How to ensure data security and privacy is not leaked is also an important issue that needs attention. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for managing and scheduling computing resources, comprising the following steps:

[0005] Step 1: The cloud data server obtains the computing power information of the computing power node according to the computing power node data collected by the computing power node data collection device, and the cloud data server generates a computing power container pool corresponding to the computing power node according to the computing power node and the computing power node information;

[0006] Step 2: The cloud data server connects the computing power container pool of the corresponding computing power node to the computing power node. The historical data statistics module obtains the operation characteristics of the computing power node according to the historical data of the computing power node. The computing power container pool of the corresponding computing power node generates a list of available computing power containers according to the operation characteristics of the computing power node, and sends the generated list of available computing power containers to the cloud data server.

[0007] Step 3: The client module generates a computing power demand task and sends it to the cloud data server. The cloud data server generates a computing power container pool sequence of the first corresponding computing power node according to the first matching feature in the computing power demand task.

[0008] Step 4: Obtain the computing power container sequence of the second corresponding computing power node according to the computing power requirement information in the computing power requirement task and the computing power container sequence of the first corresponding computing power node, obtain the computing power container sequence of the candidate corresponding computing power node according to the computing power container sequence of the second corresponding computing power node, and send it to the client module;

[0009] Step 5: Based on the computing power container confirmation information returned by the client, the cloud data server sends the computing power demand task and computing power container information to the computing power node corresponding to the computing power container information; the computing power container of the corresponding computing power node connects the computing power container with the client module based on the computing power container information, and sends the computing power demand information to the computing power container to complete the scheduling of computing power resources.

[0010] Furthermore, the cloud data server obtains computing power information of the computing power node based on the computing power node data collected by the computing power node data collection device, including: location information of the computing power node, total computing power of the computing power node, and computing power unit information of the computing power node.

[0011] Furthermore, the cloud data server generates a computing power container pool corresponding to the computing power node according to the computing power node and the computing power node information, including:

[0012] The cloud data server generates corresponding computing power containers according to the computing power unit information of the computing power node, and all computing power containers constitute the computing power container pool of the corresponding computing power node.

[0013] Furthermore, the historical data statistics module obtains the operation characteristics of the computing power node according to the historical data of the computing power node, and the computing power container corresponding to the computing power node generates a list of available computing power containers according to the operation characteristics of the computing power node, including:

[0014] The historical data statistics module obtains the demand frequency of different computing power sizes based on the computing power demand information data of the computing power node during the data collection period. The computing power container corresponding to the computing power node generates available computing power containers corresponding to different computing power sizes based on the demand frequency of different computing power sizes and the computing power container, and obtains a list of available computing power containers.

[0015] Furthermore, the cloud data server generates a computing power container pool sequence of a first corresponding computing power node according to the first matching feature in the computing power demand task, including:

[0016] According to the communication delay between the client module that issues the computing power demand task and each computing power node, the computing power container pool sequence of the first corresponding computing power node is obtained according to the communication delay size.

[0017] Further, the step of obtaining the computing power container sequence of the second corresponding computing power node according to the computing power requirement information in the computing power requirement task and the computing power container pool sequence of the first corresponding computing power node includes:

[0018] According to the computing power required in the computing power requirement task, it is matched in the computing power container pool sequence of the first corresponding computing power node. If the difference between the computing power required in the required task and the available computing power containers in the computing power container pool of the corresponding computing power node is within the set difference threshold, it is the computing power container pool of the matching corresponding computing power node, and the idle available computing power containers in the computing power container pool of the matching corresponding computing power node are the computing power container sequence of the second corresponding computing power node.

[0019] Further, obtaining the computing power container sequence of the candidate corresponding computing power node according to the computing power container sequence of the second corresponding computing power node includes:

[0020] The difference between the computing power of the idle available computing power containers in the computing power container pool of the matching corresponding computing power node and the computing power required in the required task, within the set difference threshold, is the computing power container of the candidate corresponding computing power node. The computing power containers of all candidate corresponding computing power nodes are used to obtain the computing power container sequence of the candidate corresponding computing power nodes according to the computing power size.

[0021] A computing resource management and scheduling system, applying the computing resource management and scheduling method, comprises a task allocation module, a historical data statistics module, a task management module, a cloud data server, a network detection module and a communication module;

[0022] The cloud data server, network detection module and communication module are connected in sequence; the task allocation module, historical data statistics module and task management module are respectively connected to the communication module for communication.

[0023] The beneficial effect of the present invention is that: this paper proposes a method and system for managing and scheduling computing resources. The method realizes dynamic scheduling and optimal configuration of resources by real-time monitoring, prediction and analysis of computing resources, thereby improving utilization, reducing energy consumption and ensuring data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a method for managing and scheduling computing resources is provided;

[0025] Figure 2 The figure is a schematic diagram of the principle of a computing resource management and scheduling system. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0028] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative work are within the scope of protection of the present invention. It should be noted that relational terms such as the terms "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0029] Moreover, the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus. In the absence of more restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0030] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0031] like Figure 1 As shown, a method for managing and scheduling computing resources includes the following steps:

[0032] Step 1: The cloud data server obtains the computing power information of the computing power node according to the computing power node data collected by the computing power node data collection device, and the cloud data server generates a computing power container pool corresponding to the computing power node according to the computing power node and the computing power node information;

[0033] Step 2: The cloud data server connects the computing power container pool of the corresponding computing power node to the computing power node. The historical data statistics module obtains the operation characteristics of the computing power node according to the historical data of the computing power node. The computing power container pool of the corresponding computing power node generates a list of available computing power containers according to the operation characteristics of the computing power node, and sends the generated list of available computing power containers to the cloud data server.

[0034] Step 3: The client module generates a computing power demand task and sends it to the cloud data server. The cloud data server generates a computing power container pool sequence of the first corresponding computing power node according to the first matching feature in the computing power demand task.

[0035] Step 4: Obtain the computing power container sequence of the second corresponding computing power node according to the computing power requirement information in the computing power requirement task and the computing power container sequence of the first corresponding computing power node, obtain the computing power container sequence of the candidate corresponding computing power node according to the computing power container sequence of the second corresponding computing power node, and send it to the client module;

[0036] Step 5: Based on the computing power container confirmation information returned by the client, the cloud data server sends the computing power demand task and computing power container information to the computing power node corresponding to the computing power container information; the computing power container of the corresponding computing power node connects the computing power container with the client module based on the computing power container information, and sends the computing power demand information to the computing power container to complete the scheduling of computing power resources.

[0037] The cloud data server obtains the computing power information of the computing power node based on the computing power node data collected by the computing power node data collection device, including: the location information of the computing power node, the total computing power of the computing power node and the computing power unit information of the computing power node.

[0038] The cloud data server generates a computing power container pool corresponding to the computing power node according to the computing power node and computing power node information, including:

[0039] The cloud data server generates corresponding computing power containers according to the computing power unit information of the computing power node, and all computing power containers constitute the computing power container pool of the corresponding computing power node.

[0040] The historical data statistics module obtains the operation characteristics of the computing power node according to the historical data of the computing power node, and the computing power container corresponding to the computing power node generates a list of available computing power containers according to the operation characteristics of the computing power node, including:

[0041] The historical data statistics module obtains the demand frequency of different computing power sizes based on the computing power demand information data of the computing power node during the data collection period. The computing power container corresponding to the computing power node generates available computing power containers corresponding to different computing power sizes based on the demand frequency of different computing power sizes and the computing power container, and obtains a list of available computing power containers.

[0042] The cloud data server generates a computing power container pool sequence of a first corresponding computing power node according to the first matching feature in the computing power demand task, including:

[0043] According to the communication delay between the client module that issues the computing power demand task and each computing power node, the computing power container pool sequence of the first corresponding computing power node is obtained according to the communication delay size.

[0044] The step of obtaining the computing power container sequence of the second corresponding computing power node according to the computing power requirement information in the computing power requirement task and the computing power container pool sequence of the first corresponding computing power node includes:

[0045] According to the computing power required in the computing power requirement task, it is matched in the computing power container pool sequence of the first corresponding computing power node. If the difference between the computing power required in the required task and the available computing power containers in the computing power container pool of the corresponding computing power node is within the set difference threshold, it is the computing power container pool of the matching corresponding computing power node, and the idle available computing power containers in the computing power container pool of the matching corresponding computing power node are the computing power container sequence of the second corresponding computing power node.

[0046] The step of obtaining the computing power container sequence of the candidate corresponding computing power node according to the computing power container sequence of the second corresponding computing power node includes:

[0047] The difference between the computing power of the idle available computing power containers in the computing power container pool of the matching corresponding computing power node and the computing power required in the required task, within the set difference threshold, is the computing power container of the candidate corresponding computing power node. The computing power containers of all candidate corresponding computing power nodes are used to obtain the computing power container sequence of the candidate corresponding computing power nodes according to the computing power size.

[0048] like Figure 2 As shown, a computing resource management and scheduling system, applying the computing resource management and scheduling method, includes a task allocation module, a historical data statistics module, a task management module, a cloud data server, a network detection module and a communication module;

[0049] The cloud data server, network detection module and communication module are connected in sequence; the task allocation module, historical data statistics module and task management module are respectively connected to the communication module for communication.

[0050] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art do not deviate from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for managing and scheduling computing resources, characterized in that: The steps include: Step 1: The cloud data server obtains the computing power information of the computing power node according to the computing power node data collected by the computing power node data collection device, and the cloud data server generates a computing power container pool corresponding to the computing power node according to the computing power node and the computing power node information; Step 2: The cloud data server connects the computing power container pool of the corresponding computing power node to the computing power node. The historical data statistics module obtains the operation characteristics of the computing power node according to the historical data of the computing power node. The computing power container pool of the corresponding computing power node generates a list of available computing power containers according to the operation characteristics of the computing power node, and sends the generated list of available computing power containers to the cloud data server. Step 3: The client module generates a computing power demand task and sends it to the cloud data server. The cloud data server generates a computing power container pool sequence of the first corresponding computing power node according to the first matching feature in the computing power demand task. Step 4: Obtain the computing power container sequence of the second corresponding computing power node according to the computing power demand information in the computing power demand task and the computing power container pool sequence of the first corresponding computing power node, obtain the computing power container sequence of the candidate corresponding computing power node according to the computing power container sequence of the second corresponding computing power node, and send it to the client module; Step 5: According to the computing power container confirmation information returned by the client, the cloud data server sends the computing power demand task and computing power container information to the computing power node corresponding to the computing power container information; the computing power container of the corresponding computing power node connects the computing power container with the client module according to the computing power container information, and sends the computing power demand information to the computing power container to complete the scheduling of computing power resources; The cloud data server generates a computing power container pool sequence of a first corresponding computing power node according to the first matching feature in the computing power demand task, including: According to the communication delay between the client module that issues the computing power demand task and each computing power node, the computing power container pool sequence of the first corresponding computing power node is obtained according to the communication delay size; The step of obtaining the computing power container sequence of the second corresponding computing power node according to the computing power requirement information in the computing power requirement task and the computing power container pool sequence of the first corresponding computing power node includes: According to the computing power required in the computing power demand task, the computing power container pool sequence of the first corresponding computing power node is matched. If the difference between the computing power required in the demand task and the available computing power containers in the computing power container pool of the corresponding computing power node is within the set difference threshold, the computing power container pool of the matching corresponding computing power node is the computing power container sequence of the second corresponding computing power node. The step of obtaining the computing power container sequence of the candidate corresponding computing power node according to the computing power container sequence of the second corresponding computing power node includes: The difference between the computing power of the idle available computing power containers in the computing power container pool of the matching corresponding computing power node and the computing power required in the required task, within the set difference threshold, is the computing power container of the candidate corresponding computing power node. The computing power containers of all candidate corresponding computing power nodes are used to obtain the computing power container sequence of the candidate corresponding computing power nodes according to the computing power size.

2. A method for managing and scheduling computing resources according to claim 1, characterized in that: The cloud data server obtains the computing power information of the computing power node based on the computing power node data collected by the computing power node data collection device, including: the location information of the computing power node, the total computing power of the computing power node and the computing power unit information of the computing power node.

3. A method for managing and scheduling computing resources according to claim 2, characterized in that: The cloud data server generates a computing power container pool corresponding to the computing power node according to the computing power node and computing power node information, including: The cloud data server generates corresponding computing power containers according to the computing power unit information of the computing power node, and all computing power containers constitute the computing power container pool of the corresponding computing power node.

4. A method for managing and scheduling computing resources according to claim 3, characterized in that: The historical data statistics module obtains the operation characteristics of the computing power node according to the historical data of the computing power node, and the computing power container corresponding to the computing power node generates a list of available computing power containers according to the operation characteristics of the computing power node, including: The historical data statistics module obtains the demand frequency of different computing power sizes based on the computing power demand information data of the computing power node during the data collection period. The computing power container corresponding to the computing power node generates available computing power containers corresponding to different computing power sizes based on the demand frequency of different computing power sizes and the computing power container, and obtains a list of available computing power containers.

5. A computing resource management and scheduling system, characterized in that: A method for managing and scheduling computing resources according to any one of claims 1 to 4 is applied, comprising a task allocation module, a historical data statistics module, a task management module, a cloud data server, a network detection module and a communication module; The cloud data server, network detection module and communication module are connected in sequence; the task allocation module, historical data statistics module and task management module are respectively connected to the communication module for communication.

Images