Computing power scheduling optimization system and method based on graph neural network

Abstract

The invention discloses a computing power scheduling optimization system and method based on a graph neural network, and the system comprises the following modules: a computing power resource intelligent recommendation module which carries out the appropriate computing power resource recommendation based on the graph neural network according to the adaptability of historical consumption records and computing power product records with user demands; the computing power resource intelligent scheduling module is used for performing computing power scheduling of proper computing nodes according to the conditions of the current computing nodes and the network nodes based on the graph neural network on the basis of determining the proper computing power resources; and the node fault module is used for selecting a new node or temporarily freezing the operation according to the operation computing power demand when the computing node has a fault on the basis of the dynamic network perception of the graph neural network. According to the invention, the reliability and stability of the scheduling process are improved, and the user experience of using computing power is improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a computing power scheduling optimization system and method based on graph neural network. Background Art

[0002] Computing power scheduling refers to the process of rationally allocating and utilizing computer resources in a certain area or system. Its main purpose is to improve the utilization rate of computer resources, reduce resource waste, and ensure efficient execution of tasks. That is, based on the brain of the computing network, intelligent orchestration and flexible scheduling of computing power resources across the entire network are carried out. In today's information explosion and increasing demand for computing power, more and more scenarios, from university research, company R&D, government governance to personal needs, require huge computing power to support. Computing power is productivity, and as the country vigorously develops digital infrastructure, the improvement and inclusiveness of computing power are becoming more and more important. However, as computing power continues to innovate with technology, computing power distribution is more ubiquitous, and the distribution of computing power infrastructure locations extends from the center to the edge and the end side, and a ubiquitous evolution trend of cloud, edge, and end three-level computing power architecture emerges. As a result, the distribution of computing power will no longer be concentrated in the data center, but will be widely distributed at any location on the edge or end side. If all these computing power nodes are not interconnected through the network, these computing power resources cannot be shared, scheduled, used, or coordinated. And with the in-depth advancement of the "East Data West Computing" project, how to improve the level of cross-regional computing power scheduling and provide high-quality computing power services has become an important research direction in the computing power field. Therefore, the improvement of computing power scheduling is even more important.

[0003] In this context, the coordination of computing resources is the key. In this case, the computing network came into being. The core idea of ​​the computing network is to connect geographically distributed computing center nodes through new network technologies, dynamically and in real time perceive the status of computing resources, storage resources, and network resources (including but not limited to node computing, network, and storage, etc.), and then coordinate and schedule computing tasks, transmit data, and form a network that perceives, allocates, and schedules computing power on a global scale. On this basis, it gathers and shares computing power, data, and application resources. According to different business needs, it provides the industry with the best resource allocation plan, thereby realizing the optimal allocation and use of the entire network resources on demand.

[0004] The data explosion in recent years has led to a significant increase in data volume, data density and data complexity, and the difficulty of resource processing for various business programs has increased. Processing large-scale data sets requires programs to apply more computing power, storage space, network bandwidth and other resources. Currently, computing power scheduling services are mainly divided into two parts: the purchase of computing power resources and the scheduling of computing power resources.

[0005] In terms of purchasing computing power resources, the major computing power platforms currently provide a wide variety of computing power resource products, and the resource combinations required for different businesses are also different. It is time-consuming and labor-intensive to rely solely on customers to select the appropriate combination from a large number of products, and there is a lack of convenient and reasonable tools and means to provide selection strategies.

[0006] In terms of computing resource scheduling, it is currently difficult to coordinate computing power across regions and networks, and the degree of intelligence is insufficient, which makes the unified orchestration and scheduling of computing power face challenges. Computing power scheduling requires comprehensive consideration of the status of multiple clusters, nodes, and networks, but current computing power scheduling methods lack this consideration and simply divide each part into computing power scheduling methods as an independent individual. At the same time, under the current situation of data explosion, force majeure factors cause source service interruptions and even direct downtime, restarts, or partial hardware failures, which may cause node failures and poor user experience. This also requires the optimization of computing power scheduling methods to minimize user losses and experience. Summary of the invention

[0007] The purpose of the present invention is to provide a computing power scheduling optimization system and method based on graph neural network.

[0008] The technical solution adopted by the present invention is: A computing power scheduling optimization system based on graph neural network includes the following modules: Computing power resource intelligent recommendation module: Based on graph neural network, it recommends appropriate computing power resources according to the compatibility of historical consumption records and computing power product records with user needs; Computing power resource intelligent scheduling module: Based on the determination of appropriate computing power resources, the computing power of appropriate computing nodes is scheduled according to the current status of computing nodes and network nodes based on the graph neural network; Node failure module: Based on the dynamic network perception of the graph neural network, when a computing node fails, a new node is selected or the job is temporarily frozen according to the job computing power requirements.

[0009] A computing power scheduling optimization method based on graph neural network includes the following steps: Step 1: Intelligent recommendation of computing power resources: Based on the graph neural network, according to the historical consumption records and computing power product records, the adaptability with user needs is calculated respectively, so as to recommend appropriate computing power resources; Step 2: Intelligent scheduling of computing resources: Based on the graph neural network, the scheduling strategy is dynamically adjusted according to the computing network status of the current computing nodes and network nodes to select appropriate computing nodes; Step 3, node failure processing: Based on the dynamic network perception of the graph neural network, when a computing node fails, a new node is selected or the job is temporarily frozen according to the job computing power requirements.

[0010] Furthermore, the specific steps for intelligent recommendation of computing resources are as follows: Step 1-1, extracting user demand keywords to form a user demand graph structure; Specifically, when users make a computing resource request, they need to enter keywords or tags of the required resources, such as application type, application industry, data set size, network bandwidth requirement, video memory size requirement, memory size requirement, etc. Before making subsequent adaptive computing power recommendations, these keywords are first extracted into the input data format of the graph neural network - the graph structure.

[0011] Step 1-2: Check the database to determine whether the user has a historical consumption record on the platform; if so, extract all historical consumption record keywords to form a historical consumption graph structure; Step 1-3, calculate the similarity between the user demand graph structure and the historical consumption graph structure, and obtain the top three graph structure results with the highest similarity as candidate recommendation results; Specifically, in steps 1-3, the user demand graph structure and the historical consumption graph structure are input together into a similarity model constructed based on a graph neural network to perform similarity calculation.

[0012] Step 1-4, determine whether the similarity of the candidate recommendation results is greater than the similarity threshold set in advance; if so, take the corresponding candidate recommendation result as the final recommendation result and execute step 1-7; otherwise, execute step 1-5; Furthermore, the similarity threshold set in advance in steps 1-4 is 70%.

[0013] Step 1-5, check whether there are records of computing power resource products in the database; if so, extract all computing power product data notes to form a computing power product graph structure; otherwise, directly end the intelligent computing power resource recommendation; Step 1-6: Calculate the recommendation scores of the user demand graph structure and the computing power product graph structure, and obtain the top three graph structure results with the highest recommendation scores as the final recommendation results; Specifically, in steps 1-6, the user demand graph structure and the computing power product graph structure are input together into the recommendation model built based on the graph neural network to calculate the recommendation score.

[0014] Steps 1-7: Deliver the final recommendation result to the user and end the intelligent recommendation of computing resources.

[0015] Furthermore, the specific steps of intelligent scheduling of computing resources are as follows: Step 2-1, the client initiates a computing service request; Specifically, the request includes the status of computing resource requirements (i.e., computing power) and network resource requirements (e.g., latency, bandwidth, and packet loss).

[0016] Step 2-2: The network monitoring module obtains the network status in real time. Specifically, the computing network state refers to the state of links and nodes. Link state contains information about the paths through the links, the connected node pairs, link capacity, latency, and other network metrics. The state of nodes involves node characteristics, their neighborhood, and the state of the computations in each computing node (e.g., computation type, load, and computational capacity).

[0017] Step 2-3, determine whether the network computing status is normal; if so, extract the graph structure of the network computing status topology information; otherwise, exit directly and report the situation in the form of a work order; Step 2-4, dynamically adjust the scheduling strategy according to the real-time changes of the network and computing nodes to select the best computing nodes and allocate network resources.

[0018] Furthermore, the scheduling strategy includes path selection, service scheduling (selection of qualified computing nodes), and allocation of bandwidth resources for traffic.

[0019] Furthermore, in steps 2-4, the computing network status obtained by the computing network monitoring module is abstracted into a graph structure and input into the trained graph neural network. According to the real-time changes of the network and computing nodes, the path selection, service scheduling (selecting qualified computing nodes) and the strategy of allocating bandwidth resources to traffic are dynamically adjusted; by selecting the best computing nodes and allocating network resources (i.e. bandwidth).

[0020] Furthermore, the computing power scheduling node failure processing specifically includes the following steps: Step 3-1, construct a node fault handling model of the graph neural network, and train the model based on the computing status and network resource information of each computing node; Furthermore, the computing status includes computing type, load and computing capacity; the network resource information includes delay, bandwidth and packet loss.

[0021] Specifically, the inherent advantages of graph neural networks for network topology are utilized to train the model based on the computing status (e.g., computing type, load, and computing power) and network resource information (e.g., latency, bandwidth, and packet loss) in each computing node.

[0022] Step 3-2, obtaining the computing network status: obtaining the computing network status according to the computing network perception function and determining whether there is a node error in the computing network status; if yes, inputting the current computing network information into the node fault handling model of the graph neural network; otherwise, ending the fault handling process; Step 3-3, selection of new nodes: determine whether the node fault handling model of the graph neural network can find a similar new computing node; if so, input the backup data of the damaged node into the new computing node; otherwise, arrange the computing job into the waiting list to freeze the job training, dynamically waiting for the appearance of idle nodes or the repair of faulty nodes.

[0023] The present invention adopts the above technical scheme, based on the advantages of graph neural networks in processing the graph structure of computing resource nodes and network nodes such as computing power scheduling, comprehensively considers the status of all indicators, clusters and nodes involved during computing power scheduling, completes intelligent recommendations when purchasing computing power resources, intelligent scheduling after computing power resources are selected, and construction of three method flows for handling subsequent node failures, thereby improving the reliability and stability of the scheduling process and enhancing the user experience of using computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments; Figure 1 This is a schematic diagram of the principle architecture of a computing power scheduling optimization system based on a graph neural network of the present invention; Figure 2 A schematic diagram of the process of intelligently recommending computing resources according to the present invention; Figure 3 A schematic diagram of the process of intelligent scheduling of computing resources of the present invention; Figure 4 The figure is a flowchart of node failure processing according to the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0026] like Figures 1 to 4 As shown in one, the present invention discloses a computing power scheduling optimization system based on graph neural network, which includes the following modules: Computing power resource intelligent recommendation module: Based on graph neural network, it recommends appropriate computing power resources according to the compatibility of historical consumption records and computing power product records with user needs; Computing power resource intelligent scheduling module: Based on the determination of appropriate computing power resources, the computing power of appropriate computing nodes is scheduled according to the current status of computing nodes and network nodes based on the graph neural network; Node failure module: Based on the dynamic network perception of the graph neural network, when a computing node fails, a new node is selected or the job is temporarily frozen according to the job computing power requirements.

[0027] A computing power scheduling optimization method based on graph neural network includes the following steps: Step 1: Intelligent recommendation of computing power resources: Based on the graph neural network, according to the historical consumption records and computing power product records, the adaptability with user needs is calculated respectively, so as to recommend appropriate computing power resources; Step 2: Intelligent scheduling of computing resources: Based on the graph neural network, the scheduling strategy is dynamically adjusted according to the computing network status of the current computing nodes and network nodes to select appropriate computing nodes; Step 3, node failure processing: Based on the dynamic network perception of the graph neural network, when a computing node fails, a new node is selected or the job is temporarily frozen according to the job computing power requirements.

[0028] Furthermore, the specific steps for intelligent recommendation of computing resources are as follows: Step 1-1, extracting user demand keywords to form a user demand graph structure; Specifically, when users make a computing resource request, they need to enter keywords or tags of the required resources, such as application type, application industry, data set size, network bandwidth requirement, video memory size requirement, memory size requirement, etc. Before making subsequent adaptive computing power recommendations, these keywords are first extracted into the input data format of the graph neural network - the graph structure.

[0029] Step 1-2: Check the database to determine whether the user has a historical consumption record on the platform; if so, extract all historical consumption record keywords to form a historical consumption graph structure; Step 1-3, calculate the similarity between the user demand graph structure and the historical consumption graph structure, and obtain the top three graph structure results with the highest similarity as candidate recommendation results; Specifically, in steps 1-3, the user demand graph structure and the historical consumption graph structure are input together into a similarity model constructed based on a graph neural network to perform similarity calculation.

[0030] Step 1-4, determine whether the similarity of the candidate recommendation results is greater than the similarity threshold set in advance; if so, take the corresponding candidate recommendation result as the final recommendation result and execute step 1-7; otherwise, execute step 1-5; Furthermore, the similarity threshold set in advance in steps 1-4 is 70%.

[0031] Step 1-5, check whether there are records of computing power resource products in the database; if so, extract all computing power product data notes to form a computing power product graph structure; otherwise, directly end the intelligent computing power resource recommendation; Step 1-6: Calculate the recommendation scores of the user demand graph structure and the computing power product graph structure, and obtain the top three graph structure results with the highest recommendation scores as the final recommendation results; Specifically, in steps 1-6, the user demand graph structure and the computing power product graph structure are input together into the recommendation model built based on the graph neural network to calculate the recommendation score.

[0032] Steps 1-7: Deliver the final recommendation result to the user and end the intelligent recommendation of computing resources.

[0033] Furthermore, the specific steps of intelligent scheduling of computing resources are as follows: Step 2-1, the client initiates a computing service request; Specifically, the request includes the status of computing resource requirements (i.e., computing power) and network resource requirements (e.g., latency, bandwidth, and packet loss).

[0034] Step 2-2: The network monitoring module obtains the network status in real time. Specifically, the computing network state refers to the state of links and nodes. Link state contains information about the paths through the links, the connected node pairs, link capacity, latency, and other network metrics. The state of nodes involves node characteristics, their neighborhood, and the state of the computations in each computing node (e.g., computation type, load, and computational capacity).

[0035] Step 2-3, determine whether the network computing status is normal; if so, extract the graph structure of the network computing status topology information; otherwise, exit directly and report the situation in the form of a work order; Step 2-4, dynamically adjust the scheduling strategy according to the real-time changes of the network and computing nodes to select the best computing nodes and allocate network resources.

[0036] Furthermore, the scheduling strategy includes path selection, service scheduling (selection of qualified computing nodes), and allocation of bandwidth resources for traffic.

[0037] Furthermore, in steps 2-4, the computing network status obtained by the computing network monitoring module is abstracted into a graph structure and input into the trained graph neural network. According to the real-time changes of the network and computing nodes, the path selection, service scheduling (selecting qualified computing nodes) and the strategy of allocating bandwidth resources to traffic are dynamically adjusted; by selecting the best computing nodes and allocating network resources (i.e. bandwidth).

[0038] Furthermore, the computing power scheduling node failure processing specifically includes the following steps: Step 3-1, construct a node fault handling model of the graph neural network, and train the model based on the computing status and network resource information of each computing node; Furthermore, the computing status includes computing type, load and computing capacity; the network resource information includes delay, bandwidth and packet loss.

[0039] Specifically, the inherent advantages of graph neural networks for network topology are utilized to train the model based on the computing status (e.g., computing type, load, and computing power) and network resource information (e.g., latency, bandwidth, and packet loss) in each computing node.

[0040] Step 3-2, obtaining the computing network status: obtaining the computing network status according to the computing network perception function and determining whether there is a node error in the computing network status; if yes, inputting the current computing network information into the node fault handling model of the graph neural network; otherwise, ending the fault handling process; Step 3-3, selection of new nodes: determine whether the node fault handling model of the graph neural network can find a similar new computing node; if so, input the backup data of the damaged node into the new computing node; otherwise, arrange the computing job into the waiting list to freeze the job training, dynamically waiting for the appearance of idle nodes or the repair of faulty nodes.

[0041] The computing power scheduling of the present invention can have important applications in the following fields: 1. Smart city: The city management system, i.e., the brain of the smart city (such as traffic management and public safety), needs to rely on computing power scheduling to process and analyze data from different sensors, such as electronic sensors, infrared sensors, thermal sensors, short-range sensors, and lidar, etc., to provide real-time decision support and improve the efficiency of urban operations. 2. Cloud computing: Cloud service providers optimize resource utilization through computing power scheduling, integrate computing power resources in different regions, dynamically adjust resource allocation to cope with different workloads, ensure service quality and meet the needs of different tenants. For example, e-commerce websites need additional computing resources to handle the surge in traffic during promotional activities, and the scheduling system will automatically increase resources to meet the demand. 3. Financial services: High-frequency trading, risk analysis, and real-time market monitoring require efficient computing power scheduling to ensure fast data processing and analysis. 4. Media and entertainment on the cloud: Movie production and special effects rendering require a large amount of computing resources, and computing power scheduling optimizes resource utilization and improves rendering speed. For example, to make a complex special effects scene, the scheduling system will assign rendering tasks to multiple GPUs for parallel processing. Cloud gaming requires low latency and high performance. Computing power scheduling can provide the best computing resource services based on the player's geographic location and network conditions.

[0042] The present invention uses graph neural networks as the core technology to optimize the three main processes of computing power scheduling, namely, adaptive computing power resource recommendation, computing power scheduling, and node fault handling in computing power scheduling. 1. Provide intelligent adaptive computing power resource recommendations: According to the user's business scenarios and computing power resource requirements, appropriate computing power resources are recommended based on historical consumption records and the products currently provided by the computing power platform. 2. Intelligent scheduling of computing power: According to the user's job purpose, data volume, job scale, network conditions, and current computing power resource distribution, the most appropriate computing power is selected and allocated. 3. Node fault handling: When a node is damaged, quickly find a similar node to restart the computing power job. If there is no suitable node, the computing power job is temporarily frozen, and the job is restored when there is a suitable node or the node fault is restored.

[0043] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. A computing power scheduling optimization system based on graph neural network, characterized by: It includes the following modules: Computing power resource intelligent recommendation module: Based on graph neural network, it recommends appropriate computing power resources according to the compatibility of historical consumption records and computing power product records with user needs; Intelligent computing resource scheduling module: Based on the graph neural network, it schedules the computing power of appropriate computing nodes according to the status of current computing nodes and network nodes; Node failure module: Based on the dynamic network perception of graph neural network, when a computing node fails, a new node is selected or the job is temporarily frozen according to the computing power requirements of the job.

2. A method for optimizing computing power scheduling based on a graph neural network, which adopts a computing power scheduling optimization system based on a graph neural network as described in claim 1, characterized in that: The method comprises the following steps: Step 1: Intelligent recommendation of computing power resources: Based on the graph neural network, according to the historical consumption records and computing power product records, the adaptability with user needs is calculated respectively, so as to recommend appropriate computing power resources; Step 2: Intelligent scheduling of computing resources: Based on the graph neural network, the scheduling strategy is dynamically adjusted according to the computing network status of the current computing nodes and network nodes to select appropriate computing nodes; Step 3, node failure processing: Based on the dynamic network perception of the graph neural network, when a computing node fails, a new node is selected or the job is temporarily frozen according to the job computing power requirements.

3. The method for optimizing computing power scheduling based on graph neural network according to claim 2, characterized in that: The specific steps for intelligent recommendation of computing resources are as follows: Step 1-1, extracting user demand keywords to form a user demand graph structure; Step 1-2: Check the database to determine whether the user has a historical consumption record on the platform; if so, extract all historical consumption record keywords to form a historical consumption graph structure; Step 1-3, calculate the similarity between the user demand graph structure and the historical consumption graph structure, and obtain the top three graph structure results with the highest similarity as candidate recommendation results; Step 1-4, determining whether the similarity of the candidate recommendation results is greater than a similarity threshold set in advance; If yes, the corresponding candidate recommendation result is currently used as the final recommendation result and steps 1-7 are executed; Otherwise, execute steps 1-5; Step 1-5, check whether there are records of computing power resource products in the database; if so, extract all computing power product data notes to form a computing power product graph structure; otherwise, directly end the intelligent computing power resource recommendation; Step 1-6: Calculate the recommendation scores of the user demand graph structure and the computing power product graph structure, and obtain the top three graph structure results with the highest recommendation scores as the final recommendation results; Steps 1-7: Deliver the final recommendation result to the user and end the intelligent recommendation of computing resources.

4. The method for optimizing computing power scheduling based on graph neural network according to claim 3 is characterized in that: In steps 1-3, the user demand graph structure and the historical consumption graph structure are input together into the similarity model built based on the graph neural network to calculate the similarity.

5. The method for optimizing computing power scheduling based on graph neural network according to claim 3 is characterized in that: In steps 1-6, the user demand graph structure and the computing power product graph structure are input into the recommendation model built based on the graph neural network to calculate the recommendation score.

6. The method for optimizing computing power scheduling based on graph neural network according to claim 2, characterized in that: The specific steps for intelligent scheduling of computing resources are as follows: Step 2-1, the client initiates a computing service request; the computing service request includes computing resource requirements and network resource requirements status; Step 2-2, the computing network monitoring module obtains the computing network status in real time, and the computing network status includes the status of links and nodes; Step 2-3, determine whether the network computing status is normal; if so, extract the graph structure of the network computing status topology information; otherwise, exit directly and report the situation in the form of a work order; Step 2-4, dynamically adjust the scheduling strategy according to the real-time changes of the network and computing nodes to select the best computing nodes and allocate network resources.

7. The method for optimizing computing power scheduling based on graph neural network according to claim 6, characterized in that: The link state in step 2-2 includes information about the path through the link, the connected node pairs, the link capacity, and the delay; the node state includes the node characteristics, the node neighborhood, and the computing state in each computing node.

8. The method for optimizing computing power scheduling based on graph neural network according to claim 6, characterized in that: In steps 2-4, the computing network status obtained by the computing network monitoring module is abstracted into a graph structure and input into the trained graph neural network. The scheduling strategy is dynamically adjusted according to the real-time changes of the network and computing nodes to select the best computing nodes and allocate network resources. The scheduling strategy includes path selection, service scheduling, and allocation of bandwidth resources for traffic.

9. The method for optimizing computing power scheduling based on graph neural network according to claim 2, characterized in that: The specific steps for handling computing power scheduling node failures include: Step 3-1, construct a node fault handling model of the graph neural network, and train the model based on the computing status and network resource information of each computing node; Step 3-2, obtaining the computing network status: obtaining the computing network status according to the computing network perception function and determining whether there is a node error in the computing network status; if yes, inputting the current computing network information into the node fault handling model of the graph neural network; otherwise, ending the fault handling process; Step 3-3, selection of new nodes: determine whether the node fault handling model of the graph neural network can find a similar new computing node; if so, input the backup data of the damaged node into the new computing node; otherwise, arrange the computing job into the waiting list to freeze the job training, dynamically waiting for the appearance of idle nodes or the repair of faulty nodes.

10. The method for optimizing computing power scheduling based on graph neural network according to claim 9, characterized in that: The computing status in step 3-1 includes computing type, load, and computing capacity; the network resource information includes delay, bandwidth, and packet loss.

Images