An intelligent computing resource scheduling method based on cloud-edge collaboration

Abstract

The present invention discloses a method for scheduling intelligent computing resources based on cloud-edge collaboration, comprising the following steps: S1, dividing computing resources into cloud layer, edge layer and terminal layer; S2, constructing a hierarchical data set; S3, constructing and training a hierarchical model using a graph neural network and a Transformer model; S4, making hierarchical decisions and determining a first task execution layer; S5, constructing a sharding data set; S6, constructing and training a sharding model using a generative adversarial network and a hybrid attention mechanism; S7, optimizing task sharding and predicting an optimal task sharding scheme; S8, obtaining a task sharding set, optimizing the execution of task shards in the task sharding set in the first task execution layer, and determining a second task execution layer for the task shards; S9, monitoring task execution in real time, collecting execution data and feeding it back to the hierarchical model and the sharding model, and dynamically adjusting task allocation and sharding strategies. The present invention adopts a hierarchical and sharding scheduling method of cloud-edge collaboration, and combines deep learning to achieve dynamic task optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent computing power resource scheduling, and in particular to an intelligent computing power resource scheduling method based on cloud-edge collaboration. Background Art

[0002] With the rapid development of cloud computing and edge computing technologies, the distribution and utilization of computing resources have changed significantly. Cloud computing can handle large-scale data and complex computing tasks through its powerful computing power and storage capacity, and is widely used in scientific computing, data analysis, artificial intelligence and other fields. However, a major drawback of cloud computing is its high latency, which is not enough for application scenarios that require real-time response. To make up for this shortcoming, edge computing came into being. By deploying computing resources at the edge of the network, edge computing can process data closer to the data source, significantly reducing the latency of data transmission, and is suitable for tasks with high real-time requirements. However, the computing power and storage capacity of edge computing are relatively limited, and it cannot handle overly complex computing tasks.

[0003] In the prior art, resource scheduling for cloud computing and edge computing is often performed separately, lacking an effective coordination mechanism. This separation leads to reduced resource utilization and unsatisfactory task execution efficiency. Specifically, the high latency problem of cloud computing and the resource limitation problem of edge computing cannot be effectively solved. In practical applications, many tasks are diverse and complex, and need to meet the requirements of high computing power and low latency at the same time. The prior art cannot provide an efficient scheduling method to dynamically adjust computing resources to meet the needs of different tasks.

[0004] In addition, most traditional resource scheduling methods are based on static rules or simple algorithms, which are difficult to adapt to complex and changing computing environments. When dealing with task allocation and resource scheduling, these methods lack the ability to deeply analyze task characteristics and resource status and dynamically adjust them, and cannot achieve optimal utilization of computing resources. With the increase in task complexity and data volume, the defects of traditional methods in resource scheduling efficiency and task execution performance have become more and more obvious.

[0005] In response to the above problems, some resource scheduling methods based on machine learning and deep learning have emerged in recent years. These methods can optimize resource allocation and task scheduling to a certain extent by learning from historical data. However, there are still many shortcomings in the application of existing machine learning and deep learning methods in resource scheduling. First, a single machine learning model is difficult to fully capture the complex relationship between task characteristics and resource status, resulting in poor scheduling results. Second, the existing methods lack a real-time feedback mechanism, cannot dynamically adjust the scheduling strategy according to the task execution status, and are difficult to cope with changes in the computing environment.

[0006] Therefore, how to provide an intelligent computing resource scheduling method based on cloud-edge collaboration is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0007] One purpose of the present invention is to propose an intelligent computing resource scheduling method based on cloud-edge collaboration. The present invention adopts a layered and sharded scheduling method of cloud-edge collaboration, and realizes efficient allocation of computing resources and dynamic optimization of tasks through graph neural networks, Transformer models, generative adversarial networks and hybrid attention mechanisms. It has the advantages of efficient resource utilization, reduced computing delays and improved task success rates, can meet complex and changeable computing needs, and significantly improve the overall performance and response speed of the system. Through real-time monitoring and feedback mechanisms, the scheduling strategy is dynamically adjusted, which improves the stability and reliability of the system.

[0008] According to an embodiment of the present invention, a method for intelligent computing resource scheduling based on cloud-edge collaboration includes the following steps:

[0009] S1. Divide computing resources into a cloud layer, an edge layer, and a terminal layer, wherein the cloud layer, the edge layer, and the terminal layer have different computing capabilities and delay characteristics;

[0010] S2, collect resource usage and task characteristics of the cloud layer, edge layer, and terminal layer, and build a hierarchical dataset;

[0011] S3. Build and train hierarchical models using graph neural networks and Transformer models;

[0012] S4, making a hierarchical decision, using the hierarchical model, and determining the first task execution layer according to the real-time task characteristics and the current resource status;

[0013] S5. Collect the task fragmentation, execution time, resource consumption, and success rate of each fragment, and construct a fragmentation data set.

[0014] S6. Build and train a sharding model using a generative adversarial network and a hybrid attention mechanism.

[0015] S7, task slicing optimization, using the slicing model to predict the optimal task slicing solution based on task characteristics and resource usage;

[0016] S8, slicing the task according to the task slicing scheme to obtain a task slicing set, optimizing the execution of the task slicing in the first task execution layer, and determining a second task execution layer for the task slicing;

[0017] S9. Monitor the task execution in real time, collect execution data and feed it back to the hierarchical model and sharding model. Based on the real-time feedback, the hierarchical model and sharding model dynamically adjust the task allocation and sharding strategy.

[0018] Furthermore, the computing power of the cloud layer is more than 10 trillion floating-point operations per second, the storage capacity is more than 100 TB, and the network latency is more than 50 milliseconds, which is suitable for processing computing-intensive and non-real-time tasks or task slices;

[0019] The computing power of the edge layer is between 1 teraflops and 10 teraflops, the storage capacity is between 10 TB and 100 TB, and the network latency is between 10 and 50 milliseconds, which is suitable for processing tasks or task slices of medium computing complexity with fast response;

[0020] The computing power of the terminal layer is less than 1 trillion floating-point operations per second, the storage capacity is less than 10 TB, and the network delay is less than 10 milliseconds, which is suitable for processing real-time tasks or task slices with extremely low delay.

[0021] Furthermore, the S3 specifically includes:

[0022] S31, preprocessing the hierarchical data set in S2 to obtain a feature matrix X;

[0023] S32. Build a graph neural network model and define the graph structure , where V represents the node set, each node represents a computing node, E represents the edge set, each edge represents the communication link between computing nodes, and the node feature vector is initialized For the corresponding feature vector in the feature matrix X, update the node features through graph convolution operation:

[0024] ;

[0025] in, Indicates The feature vector of layer node v, represents the feature vector of node u in the kth layer, N(v) represents the set of neighbor nodes of node v, represents the normalization constant, represents the weight matrix of the kth layer, represents the bias vector of the kth layer, represents the activation function, Attention(Q,K,V) represents the attention mechanism function, where , , ; , and represents the weight matrix;

[0026] S33, build the Transformer model, As the input sequence, the input sequence contains time series data of task characteristics and resource usage, performs position encoding, and calculates the position encoding matrix P:

[0027] ;

[0028] ;

[0029] Among them, pos represents the sequence position, i represents the dimension index, and d represents the feature dimension;

[0030] The attention weights are calculated through the multi-head self-attention mechanism, and the outputs of each head are connected together through linear transformation to form the output of the multi-head attention;

[0031] S34. Perform nonlinear transformation on the attention output through the feedforward neural network to obtain the output of the Transformer model ;

[0032] S35. Combine the outputs of the graph neural network model and the Transformer model to obtain a joint feature representation :

[0033] ;

[0034] in, and represents the weight parameter, satisfying ;

[0035] S36. Define a comprehensive loss function, combining task priority, resource consumption and execution time, and express it in the form of linear combination:

[0036] ;

[0037] in, , and Represents the loss weight coefficient, satisfying ;

[0038] Represents the task priority loss:

[0039] ;

[0040] Where N represents the number of samples, represents the true value of the priority of the i-th task, represents the predicted value;

[0041] Indicates resource consumption loss:

[0042] ;

[0043] in, represents the actual value of resource consumption of the i-th task, represents the predicted value;

[0044] Represents the execution time loss:

[0045] ;

[0046] in, represents the actual value of the execution time of the i-th task, represents the predicted value;

[0047] S37, will As the input of hierarchical model training, the data is divided into training set and validation set, and the gradient descent algorithm is used on the training set to iteratively optimize the parameters of the hierarchical model; in each iteration, the comprehensive loss function is calculated The gradient of , and update the hierarchical model parameters;

[0048] S38. During the training process, the validation set is used to evaluate the performance of the hierarchical model and monitor the overfitting of the hierarchical model;

[0049] S39. Deploy the trained hierarchical model on nodes at the cloud layer, edge layer, and terminal layer to make hierarchical decisions.

[0050] Furthermore, the S4 specifically includes:

[0051] S41, set the task characteristic vector T = [{t}_{1,}{t}_{2},...,{t}_{n}] and the current resource state vector R=[{r}_{1,}{r}_{2},...,{r}_{m}] Merge and get the input vector {X}_{input}=[T,R] ,in represents the i-th task feature, Indicates the jth resource status;

[0052] S42, input vector Input into the hierarchical model, the hierarchical model outputs a probability distribution of the task execution level ,in represents the probability of executing the task in the cloud, represents the probability of task execution at the edge layer, Indicates the probability of task execution at the terminal layer;

[0053] S43. According to probability distribution , get the first task execution layer :

[0054] ;

[0055] S44, assigning tasks to the first task execution layer ,like , then send the task to the cloud computing center; if , then send the task to the edge node; if , then execute the task on the terminal device.

[0056] Furthermore, the S6 specifically includes:

[0057] S61, preprocessing the sharded data set in S5 to obtain a preprocessed sharded data set ;

[0058] S62. Construct a generative adversarial network model, including a generator G and a discriminator D. The generator G inputs random noise z to generate a fake data sample G(z). The discriminator D inputs the real data sample x and the fake data sample G(z) generated by the generator, and outputs the probability of the sample being real data. The loss functions of the generator and the discriminator are:

[0059] ;

[0060] ;

[0061] in, represents the loss function of the generator, represents the loss function of the discriminator;

[0062] S63. Construct a hybrid attention mechanism. Based on the generative adversarial network, a hybrid attention mechanism is introduced to better capture the temporal and spatial characteristics of task slicing:

[0063] ;

[0064] Among them, Q represents the query matrix, K represents the key matrix, and V represents the value matrix. , , , , and represents the weight matrix of the attention mechanism, X represents the input feature matrix, Represents the dimension of the key matrix, calculates the output of multiple attention heads through the multi-head attention mechanism, and performs concatenation and linear transformation:

[0065] ;

[0066] in, , , and represents the weight matrix of the i-th attention head, Represents the linear transformation matrix of the output;

[0067] S64, training the sharding model, the preprocessed sharding data set Input into the generative adversarial network and hybrid attention mechanism model for training, and optimize the loss functions of the generator and discriminator;

[0068] S65. Combined loss function optimization of generative adversarial network and hybrid attention mechanism, defining the total loss function :

[0069] ;

[0070] in, , and represents the weight parameter;

[0071] ;

[0072] S66. The trained sharding model is used to predict the task sharding scheme.

[0073] Through the above steps, the sharding model is constructed and trained using the generative adversarial network and hybrid attention mechanism, so as to better optimize the task sharding scheme and improve the efficiency and success rate of task execution.

[0074] Furthermore, task slicing optimization calculates the optimal task slicing solution based on the prediction results, combined with the current resource usage and task characteristics:

[0075] ;

[0076] Among them, O represents the optimal task slicing scheme, S represents the set of all possible task slicing schemes, The execution time cost is shown. represents the resource consumption cost, represents the success rate cost, , and represents the weight parameter, satisfying .

[0077] Furthermore, the S8 specifically includes:

[0078] S81. Generate a task slicing set based on the optimal task slicing scheme O ,in represents the i-th shard of task S;

[0079] S82. Split each task into pieces Assigned to the first task execution layer Based on the current load and resource conditions of the computing nodes, the best computing node is obtained. :

[0080] ;

[0081] in, Represents task slicing The best computing node to be assigned, represents the jth computing node, Represents a compute node The current load, Represents a compute node computing power;

[0082] S83, at the best computing node Execute task sharding , monitor execution time during task execution , Resource consumption and success rate :

[0083] ;

[0084] ;

[0085] ;

[0086] in, Represents a compute node The computing power of Represents task slicing The data size, Represents task slicing The computational complexity of Represents task slicing The number of successful attempts, Represents task slicing Total number of attempts;

[0087] S84, monitor data execution time , Resource consumption and success rate Feedback to the sharding model to update the task feature vector and resource status vector , forming a new input vector ;

[0088] S85, according to the new input vector Recalculate the execution level probability distribution of task slices , get the second task execution layer :

[0089] ;

[0090] S86, if , the intermediate data of the task slice is transferred from Compute nodes Migrate to Compute nodes , the calculation formula of data migration amount M is:

[0091] ;

[0092] in, Represents task slicing Percentage of completion;

[0093] S87, in Compute nodes Restart the task slice , continue to execute the remaining computing tasks and restart the task slice Time and restarting the task slice Resource consumption for:

[0094] ;

[0095] .

[0096] The beneficial effects of the present invention are:

[0097] First, by dividing computing resources into cloud layer, edge layer and terminal layer, the present invention can make full use of computing resources at different levels to handle computing-intensive, fast-response and real-time-intensive tasks respectively. The powerful computing power and storage capacity of the cloud layer enable it to efficiently handle complex and large-scale computing tasks; the edge layer adapts to fast-response medium-complexity tasks through medium computing power and low network latency; the extremely low network latency of the terminal layer enables it to meet the needs of tasks with high real-time requirements. This multi-level resource configuration greatly improves the flexibility and adaptability of the system.

[0098] On this basis, the present invention further introduces sharding technology, and performs refined sharding optimization on tasks through generative adversarial networks and hybrid attention mechanisms. The organic combination of stratification and sharding strategies is one of the core innovations of the present invention. The stratified model can dynamically determine the initial execution level of the task according to the real-time task characteristics and current resource status through the combination of graph neural networks and Transformer models; while the sharding model refines the execution plan of the task through generative adversarial networks and hybrid attention mechanisms, so that the task can be reasonably distributed and efficiently executed among different computing nodes.

[0099] The combination of the layering strategy and the sharding strategy has brought about significant beneficial effects. The layering strategy ensures that tasks are executed at the most suitable computing level by dividing tasks into macro-level layers, reducing computing delays and improving overall execution efficiency. The sharding strategy decomposes tasks into smaller fragments through micro-optimization of tasks, and performs refined allocation and execution based on resource status, further improving the success rate of tasks and resource utilization. The combination of these two strategies enables the system to not only perform effective resource scheduling at the macro level, but also achieve efficient execution of tasks at the micro level.

[0100] In addition, the present invention realizes dynamic adjustment of scheduling strategies through real-time monitoring and feedback mechanisms. Prometheus, Grafana and other tools are used to monitor task execution in real time, collect execution data and feed it back to the hierarchical model and sharding model, so that the system can respond to changes in the computing environment in a timely manner. According to real-time feedback, the hierarchical model and sharding model dynamically adjust task allocation and sharding strategies to ensure optimal utilization of resources and efficient completion of tasks. This dynamic adjustment mechanism not only improves the stability and reliability of the system, but also significantly improves resource utilization and task completion efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0102] Figure 1 A flowchart of an intelligent computing resource scheduling method based on cloud-edge collaboration proposed by the present invention;

[0103] Figure 2 A hierarchical decision-making diagram of an intelligent computing resource scheduling method based on cloud-edge collaboration proposed by the present invention;

[0104] Figure 3 This is a task slicing optimization flowchart of an intelligent computing resource scheduling method based on cloud-edge collaboration proposed by the present invention. DETAILED DESCRIPTION

[0105] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0106] refer to Figure 1-3 , an intelligent computing resource scheduling method based on cloud-edge collaboration, comprising the following steps:

[0107] S1. Divide computing resources into a cloud layer, an edge layer, and a terminal layer, wherein the cloud layer, the edge layer, and the terminal layer have different computing capabilities and delay characteristics;

[0108] S2, collect resource usage and task characteristics of the cloud layer, edge layer, and terminal layer, and build a hierarchical dataset;

[0109] S3. Build and train hierarchical models using graph neural networks and Transformer models;

[0110] S4, making a hierarchical decision, using the hierarchical model, and determining the first task execution layer according to the real-time task characteristics and the current resource status;

[0111] S5. Collect the task fragmentation, execution time, resource consumption, and success rate of each fragment, and construct a fragmentation data set.

[0112] S6. Build and train a sharding model using a generative adversarial network and a hybrid attention mechanism.

[0113] S7, task slicing optimization, using the slicing model to predict the optimal task slicing solution based on task characteristics and resource usage;

[0114] S8, slicing the task according to the task slicing scheme to obtain a task slicing set, optimizing the execution of the task slicing in the first task execution layer, and determining a second task execution layer for the task slicing;

[0115] S9. Monitor the task execution in real time, collect execution data and feed it back to the hierarchical model and sharding model. Based on the real-time feedback, the hierarchical model and sharding model dynamically adjust the task allocation and sharding strategy.

[0116] In this embodiment, the computing power of the cloud layer is more than 10 trillion floating-point operations per second, the storage capacity is more than 100 TB, and the network delay is more than 50 milliseconds, which is suitable for processing computing-intensive and non-real-time tasks or task slices;

[0117] The computing power of the edge layer is between 1 teraflops and 10 teraflops, the storage capacity is between 10 TB and 100 TB, and the network latency is between 10 and 50 milliseconds, which is suitable for processing tasks or task slices of medium computing complexity with fast response;

[0118] The computing power of the terminal layer is less than 1 trillion floating-point operations per second, the storage capacity is less than 10 TB, and the network delay is less than 10 milliseconds, which is suitable for processing real-time tasks or task slices with extremely low delay.

[0119] In this implementation, S3 specifically includes:

[0120] S31, preprocessing the hierarchical data set in S2 to obtain a feature matrix X;

[0121] S32. Build a graph neural network model and define the graph structure , where V represents the node set, each node represents a computing node, E represents the edge set, each edge represents the communication link between computing nodes, and the node feature vector is initialized For the corresponding feature vector in the feature matrix X, update the node features through graph convolution operation:

[0122] ;

[0123] in, Indicates The feature vector of layer node v, represents the feature vector of node u in the kth layer, N(v) represents the set of neighbor nodes of node v, represents the normalization constant, represents the weight matrix of the kth layer, represents the bias vector of the kth layer, represents the activation function, Attention(Q,K,V) represents the attention mechanism function, where , , ; , and represents the weight matrix;

[0124] S33, build the Transformer model, As the input sequence, the input sequence contains time series data of task characteristics and resource usage, performs position encoding, and calculates the position encoding matrix P:

[0125] ;

[0126] ;

[0127] Among them, pos represents the sequence position, i represents the dimension index, and d represents the feature dimension;

[0128] The attention weights are calculated through the multi-head self-attention mechanism, and the outputs of each head are connected together through linear transformation to form the output of the multi-head attention;

[0129] S34. Perform nonlinear transformation on the attention output through the feedforward neural network to obtain the output of the Transformer model ;

[0130] S35. Combine the outputs of the graph neural network model and the Transformer model to obtain a joint feature representation :

[0131] ;

[0132] in, and represents the weight parameter, satisfying ;

[0133] S36. Define a comprehensive loss function, combining task priority, resource consumption and execution time, and express it in the form of linear combination:

[0134] ;

[0135] in, , and Represents the loss weight coefficient, satisfying ;

[0136] Represents the task priority loss:

[0137] ;

[0138] Where N represents the number of samples, represents the true value of the priority of the i-th task, represents the predicted value;

[0139] Indicates resource consumption loss:

[0140] ;

[0141] in, represents the actual value of resource consumption of the i-th task, represents the predicted value;

[0142] Represents the execution time loss:

[0143] ;

[0144] in, represents the actual value of the execution time of the i-th task, represents the predicted value;

[0145] S37, will As the input of hierarchical model training, the data is divided into training set and validation set, and the gradient descent algorithm is used on the training set to iteratively optimize the parameters of the hierarchical model; in each iteration, the comprehensive loss function is calculated The gradient of , and update the hierarchical model parameters;

[0146] S38. During the training process, the validation set is used to evaluate the performance of the hierarchical model and monitor the overfitting of the hierarchical model;

[0147] S39. Deploy the trained hierarchical model on nodes at the cloud layer, edge layer, and terminal layer to make hierarchical decisions.

[0148] In this implementation, S4 specifically includes:

[0149] S41, set the task characteristic vector T = [{t}_{1,}{t}_{2},...,{t}_{n}] and the current resource state vector R=[{r}_{1,}{r}_{2},...,{r}_{m}] Merge and get the input vector {X}_{input}=[T,R] ,in represents the i-th task feature, Indicates the jth resource status;

[0150] S42, input vector Input into the hierarchical model, the hierarchical model outputs a probability distribution of the task execution level ,in represents the probability of executing the task in the cloud, represents the probability of task execution at the edge layer, Indicates the probability of task execution at the terminal layer;

[0151] S43. According to probability distribution , get the first task execution layer :

[0152] ;

[0153] S44, assigning tasks to the first task execution layer ,like , then send the task to the cloud computing center; if , then send the task to the edge node; if , then execute the task on the terminal device.

[0154] In this implementation manner, S6 specifically includes:

[0155] S61, preprocessing the sharded data set in S5 to obtain a preprocessed sharded data set ;

[0156] S62. Construct a generative adversarial network model, including a generator G and a discriminator D. The generator G inputs random noise z to generate a fake data sample G(z). The discriminator D inputs the real data sample x and the fake data sample G(z) generated by the generator, and outputs the probability of the sample being real data. The loss functions of the generator and the discriminator are:

[0157] ;

[0158] ;

[0159] in, represents the loss function of the generator, represents the loss function of the discriminator;

[0160] S63. Construct a hybrid attention mechanism. Based on the generative adversarial network, a hybrid attention mechanism is introduced to better capture the temporal and spatial characteristics of task slicing:

[0161] ;

[0162] Among them, Q represents the query matrix, K represents the key matrix, and V represents the value matrix. , , , , and represents the weight matrix of the attention mechanism, X represents the input feature matrix, Represents the dimension of the key matrix, calculates the output of multiple attention heads through the multi-head attention mechanism, and performs concatenation and linear transformation:

[0163] ;

[0164] in, , , and represents the weight matrix of the i-th attention head, Represents the linear transformation matrix of the output;

[0165] S64, training the sharding model, the preprocessed sharding data set Input into the generative adversarial network and hybrid attention mechanism model for training, and optimize the loss functions of the generator and discriminator;

[0166] S65. Combined loss function optimization of generative adversarial network and hybrid attention mechanism, defining the total loss function :

[0167] ;

[0168] in, , and represents the weight parameter;

[0169] ;

[0170] S66. The trained sharding model is used to predict the task sharding scheme.

[0171] Through the above steps, the sharding model is constructed and trained using the generative adversarial network and hybrid attention mechanism, so as to better optimize the task sharding scheme and improve the efficiency and success rate of task execution.

[0172] In this implementation, task slicing optimization calculates the optimal task slicing solution based on the prediction results, combined with the current resource usage and task characteristics:

[0173] ;

[0174] Among them, O represents the optimal task slicing scheme, S represents the set of all possible task slicing schemes, The execution time cost is shown. represents the resource consumption cost, represents the success rate cost, , and represents the weight parameter, satisfying .

[0175] In this implementation manner, S8 specifically includes:

[0176] S81. Generate a task slicing set based on the optimal task slicing scheme O ,in represents the i-th shard of task S;

[0177] S82. Split each task into pieces Assigned to the first task execution layer Based on the current load and resource conditions of the computing nodes, the best computing node is obtained. :

[0178] ;

[0179] in, Represents task slicing The best computing node to be assigned, represents the jth computing node, Represents a compute node The current load, Represents a compute node computing power;

[0180] S83, at the best computing node Execute task sharding , monitor execution time during task execution , Resource consumption and success rate :

[0181] ;

[0182] ;

[0183] ;

[0184] in, Represents a compute node The computing power of Represents task slicing The data size, Represents task slicing The computational complexity of Represents task slicing The number of successful attempts, Represents task slicing Total number of attempts;

[0185] S84, monitor data execution time , Resource consumption and success rate Feedback to the sharding model to update the task feature vector and resource status vector , forming a new input vector ;

[0186] S85, according to the new input vector Recalculate the execution level probability distribution of task slices , get the second task execution layer :

[0187] ;

[0188] S86, if , the intermediate data of the task slice is transferred from Compute nodes Migrate to Compute nodes , the calculation formula of data migration amount M is:

[0189] ;

[0190] in, Represents task slicing Percentage of completion;

[0191] S87, in Compute nodes Restart the task slice , continue to execute the remaining computing tasks and restart the task slice Time and restarting the task slice Resource consumption for:

[0192] ;

[0193] .

[0194] Embodiment 1:

[0195] In order to verify the feasibility of the present invention in implementation, the present invention is applied to the management platform of a smart city. The platform needs to process a large amount of data from traffic monitoring, environmental monitoring, intelligent security and other aspects. These data are characterized by high real-time performance, large amount of calculation and a wide variety of types. The traditional single cloud computing method can no longer meet its needs for low latency and high computing power. Therefore, the smart city management platform decided to adopt the intelligent computing resource scheduling method based on cloud-edge collaboration proposed in the present invention to achieve efficient allocation of computing resources and dynamic optimization of tasks.

[0196] The smart city management platform divides computing resources into cloud layer, edge layer and terminal layer. The cloud layer is deployed in the central data center, with powerful computing power and storage capacity, and is mainly responsible for processing complex historical data analysis and large-scale data storage. The edge layer is deployed in edge nodes in various areas of the city, with relatively medium computing power and storage capacity, and mainly processes tasks of medium computing complexity with high real-time requirements, such as traffic flow prediction and environmental data processing. The terminal layer is deployed on various smart devices, with low computing power and storage capacity, but with extremely low network latency, and mainly processes tasks with extremely high real-time requirements, such as real-time monitoring and emergency response of smart security.

[0197] Through the method of the present invention, the usage of computing resources at each layer and task characteristic data are first collected to construct a hierarchical data set. The hierarchical data set is trained using a graph neural network and a Transformer model to construct a hierarchical model. The hierarchical model can dynamically determine the initial execution level of the task based on the real-time task characteristics and the current resource status. For example, when a smart security camera detects suspicious activity, the task is initially assigned to the terminal layer for real-time processing; while a complex traffic flow prediction task is assigned to the edge layer for processing in order to quickly respond to traffic management needs.

[0198] In terms of task slicing optimization, the system collects the execution time, resource consumption, and success rate of each task slicing and constructs a slicing dataset. The slicing model is trained using a generative adversarial network and a hybrid attention mechanism. The slicing model can perform fine-grained slicing of tasks and predict the optimal task slicing scheme based on real-time resource usage and task characteristics. For example, a traffic flow prediction task can be decomposed into traffic analysis subtasks in multiple regions, which are assigned to different edge nodes for parallel processing, thereby improving the efficiency and success rate of task execution.

[0199] The test site was a central data center and several edge nodes in a smart city, and the data sources included traffic monitoring, environmental monitoring, and intelligent security systems. During the test, the system processed more than 100 million task requests, including about 50 million traffic flow prediction tasks, about 30 million environmental data processing tasks, and about 20 million intelligent security real-time monitoring tasks.

[0200] Table 1 Comparison of test data between traditional cloud computing method and the method of the present invention

[0201] ;

[0202] From the data analysis in Table 1, it can be seen that in the test of this smart city management platform, by comparing the different performance indicators of the traditional cloud computing method and the method of the present invention, the superiority of the method of the present invention in resource scheduling and task execution is significantly demonstrated. First, the system processed more than 100 million task requests in six months, including 50 million traffic flow prediction tasks, 30 million environmental data processing tasks, and 20 million intelligent security real-time monitoring tasks. Compared with the traditional cloud computing method, the method of the present invention has significantly improved the response time of various tasks.

[0203] Specifically, the average response time of the traffic flow prediction task was reduced from 500.34 milliseconds in the traditional cloud computing method to 149.87 milliseconds, an improvement of 70.06%. The average response time of the environmental data processing task was reduced from 300.21 milliseconds to 99.65 milliseconds, an improvement of 66.80%. The average response time of the intelligent security monitoring task was reduced from 200.15 milliseconds to 49.78 milliseconds, an improvement of 75.13%. These data show that the method of the present invention can significantly reduce the response time and improve the real-time performance and response speed of the system when processing different types of tasks.

[0204] In terms of task success rate, the method of the present invention also performed well. The success rate of the traffic flow prediction task increased from 85.45% of the traditional cloud computing method to 94.67%, an increase of 9.22%. The success rate of the environmental data processing task increased from 89.98% to 97.15%, an increase of 7.17%. The success rate of the intelligent security monitoring task increased from 92.34% to 98.12%, an increase of 5.78%. These improvements show that the method of the present invention not only improves the processing speed, but also significantly improves the success rate of task execution, reduces the probability of task failure, and improves the reliability and stability of the system.

[0205] In terms of resource utilization, the cloud layer CPU utilization of the traditional cloud computing method is 60.50%, which is increased to 79.82% with the application of the method of the present invention, an increase of 19.32%. The CPU utilization of the edge layer increased from 49.73% to 74.68%, an increase of 24.95%. The CPU utilization of the terminal layer increased from 40.27% to 69.85%, an increase of 29.58%. These data show that the method of the present invention greatly improves the utilization efficiency of computing resources at each layer by optimizing resource scheduling and task allocation, avoids resource waste, and improves the performance of the overall system.

[0206] In summary, the data comparison of this test fully demonstrates the application effect of the method of the present invention in the smart city management platform. It not only greatly reduces the response time of the task and improves the success rate of the task, but also significantly improves the utilization rate of computing resources at all levels, and solves the problems of high computing delay and low resource utilization in traditional cloud computing methods. This method demonstrates strong advantages and broad application prospects when processing large amounts of real-time data and complex computing tasks, and provides solid technical support for the efficient management of smart cities.

[0207] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for scheduling intelligent computing resources based on cloud-edge collaboration, characterized in that: The steps include: S1. Divide computing resources into a cloud layer, an edge layer, and a terminal layer, wherein the cloud layer, the edge layer, and the terminal layer have different computing capabilities and delay characteristics; S2, collect resource usage and task characteristics of the cloud layer, edge layer, and terminal layer, and build a hierarchical dataset; S3. Build and train hierarchical models using graph neural networks and Transformer models; S4, making a hierarchical decision, using the hierarchical model, and determining the first task execution layer according to the real-time task characteristics and the current resource status; S5. Collect the task fragmentation, execution time, resource consumption, and success rate of each fragment, and construct a fragmentation data set. S6. Build and train a sharding model using a generative adversarial network and a hybrid attention mechanism. S7, task slicing optimization, using the slicing model to predict the optimal task slicing solution based on task characteristics and resource usage; S8, slicing the task according to the task slicing scheme to obtain a task slicing set, optimizing the execution of the task slicing in the first task execution layer, and determining a second task execution layer for the task slicing; S9. Monitor the task execution in real time, collect execution data and feed it back to the hierarchical model and sharding model. Based on the real-time feedback, the hierarchical model and sharding model dynamically adjust the task allocation and sharding strategy.

2. According to the method of intelligent computing resource scheduling based on cloud-edge collaboration according to claim 1, it is characterized in that: The cloud layer has a computing capability of more than 10 trillion floating-point operations per second, a storage capacity of more than 100 TB, and a network latency of more than 50 milliseconds, and is suitable for processing computing-intensive and non-real-time tasks or task slices; The computing power of the edge layer is between 1 teraflops and 10 teraflops, the storage capacity is between 10 TB and 100 TB, and the network latency is between 10 and 50 milliseconds, which is suitable for processing tasks or task slices of medium computing complexity with fast response; The computing power of the terminal layer is less than 1 trillion floating-point operations per second, the storage capacity is less than 10 TB, and the network delay is less than 10 milliseconds, which is suitable for processing real-time tasks or task slices with extremely low delay.

3. According to the method of intelligent computing resource scheduling based on cloud-edge collaboration according to claim 1, it is characterized in that: The S3 specifically includes: S31, preprocessing the hierarchical data set in S2 to obtain a feature matrix X; S32. Build a graph neural network model and define the graph structure , where V represents the node set, each node represents a computing node, E represents the edge set, each edge represents the communication link between computing nodes, and the node feature vector is initialized For the corresponding feature vector in the feature matrix X, update the node features through graph convolution operation: ; in, Indicates The feature vector of layer node v, represents the feature vector of node u in the kth layer, N(v) represents the set of neighbor nodes of node v, represents the normalization constant, represents the weight matrix of the kth layer, represents the bias vector of the kth layer, represents the activation function, Attention(Q,K,V) represents the attention mechanism function, where , , ; , and represents the weight matrix; S33, build the Transformer model, As the input sequence, the input sequence contains time series data of task characteristics and resource usage, performs position encoding, and calculates the position encoding matrix P: ; ; Among them, pos represents the sequence position, i represents the dimension index, and d represents the feature dimension; The attention weights are calculated through the multi-head self-attention mechanism, and the outputs of each head are connected together through linear transformation to form the output of the multi-head attention; S34. Perform nonlinear transformation on the attention output through the feedforward neural network to obtain the output of the Transformer model ; S35. Combine the outputs of the graph neural network model and the Transformer model to obtain a joint feature representation : ; in, and represents the weight parameter, satisfying ; S36. Define a comprehensive loss function, combining task priority, resource consumption and execution time, and express it in the form of linear combination: ; in, , and Represents the loss weight coefficient, satisfying ; Represents the task priority loss: ; Where N represents the number of samples, represents the true value of the priority of the i-th task, represents the predicted value; Indicates resource consumption loss: ; in, represents the actual value of resource consumption of the i-th task, represents the predicted value; Represents the execution time loss: ; in, represents the actual value of the execution time of the i-th task, represents the predicted value; S37, will As the input of hierarchical model training, the data is divided into training set and validation set, and the gradient descent algorithm is used on the training set to iteratively optimize the parameters of the hierarchical model; in each iteration, the comprehensive loss function is calculated The gradient of , and update the hierarchical model parameters; S38. During the training process, the validation set is used to evaluate the performance of the hierarchical model and monitor the overfitting of the hierarchical model; S39. Deploy the trained hierarchical model on nodes at the cloud layer, edge layer, and terminal layer to make hierarchical decisions.

4. According to the method of intelligent computing resource scheduling based on cloud-edge collaboration in claim 1, it is characterized in that: The S4 specifically includes: S41, the task characteristic vector and the current resource state vector Merge to get the input vector ,in represents the i-th task feature, Indicates the jth resource status; S42, input vector Input into the hierarchical model, the hierarchical model outputs a probability distribution of the task execution level ,in represents the probability of executing the task in the cloud, represents the probability of task execution at the edge layer, Indicates the probability of task execution at the terminal layer; S43. According to probability distribution , get the first task execution layer : ； S44, assigning tasks to the first task execution layer ,like , then send the task to the cloud computing center; if , then send the task to the edge node; if , then execute the task on the terminal device.

5. According to the method of intelligent computing resource scheduling based on cloud-edge collaboration in claim 1, it is characterized in that: The S6 specifically includes: S61, preprocessing the sharded data set in S5 to obtain a preprocessed sharded data set ; S62. Construct a generative adversarial network model, including a generator G and a discriminator D. The generator G inputs random noise z to generate a fake data sample G(z). The discriminator D inputs the real data sample x and the fake data sample G(z) generated by the generator, and outputs the probability of the sample being real data. The loss functions of the generator and the discriminator are: ; ; in, represents the loss function of the generator, represents the loss function of the discriminator; S63. Construct a hybrid attention mechanism. Based on the generative adversarial network, introduce a hybrid attention mechanism: ; Among them, Q represents the query matrix, K represents the key matrix, and V represents the value matrix. , , , , and represents the weight matrix of the attention mechanism, X represents the input feature matrix, Represents the dimension of the key matrix, calculates the output of multiple attention heads through the multi-head attention mechanism, and performs concatenation and linear transformation: ; in, , , and represents the weight matrix of the i-th attention head, Represents the linear transformation matrix of the output; S64, training the sharding model, the preprocessed sharding data set Input into the generative adversarial network and hybrid attention mechanism model for training, and optimize the loss functions of the generator and discriminator; S65. Combined loss function optimization of generative adversarial network and hybrid attention mechanism, defining the total loss function : ； in, , and represents the weight parameter; ; S66. The trained sharding model is used to predict the task sharding scheme.

6. According to the method of intelligent computing resource scheduling based on cloud-edge collaboration in claim 1, it is characterized in that: Task slicing optimization: Based on the prediction results, combined with the current resource usage and task characteristics, the optimal task slicing solution is calculated: ; Among them, O represents the optimal task slicing scheme, S represents the set of all possible task slicing schemes, The execution time cost is shown. represents the resource consumption cost, represents the success rate cost, , and represents the weight parameter, satisfying .

7. According to claim 1, a method for scheduling intelligent computing resources based on cloud-edge collaboration is characterized in that: The S8 specifically includes: S81. Generate a task slicing set based on the optimal task slicing scheme O ,in represents the i-th shard of task S; S82. Split each task into pieces Assigned to the first task execution layer Based on the current load and resource conditions of the computing nodes, the best computing node is obtained. : ; in, Represents task slicing The best computing node to be assigned, represents the jth computing node, Represents a compute node The current load, Represents a compute node computing power; S83, at the best computing node Execute task sharding , monitor execution time during task execution , Resource consumption and success rate : ; ; ; in, Represents a compute node The computing power of Represents task slicing The data size, Represents task slicing The computational complexity of Represents task slicing The number of successful attempts, Represents task slicing Total number of attempts; S84, monitor data execution time , Resource consumption and success rate Feedback to the sharding model to update the task feature vector and resource status vector , forming a new input vector ; S85, according to the new input vector Recalculate the execution level probability distribution of task slices , get the second task execution layer : ; S86, if , the intermediate data of the task slice is transferred from Compute nodes Migrate to Compute nodes , the calculation formula of data migration amount M is: ; in, Represents task slicing Percentage of completion; S87, in Compute nodes Restart the task slice , continue to execute the remaining computing tasks and restart the task slice Time and restarting the task slice Resource consumption for: ; 。

Images