A collaborative allocation method for computing, network, and storage resources based on optimization and machine learning

Abstract

The present invention discloses a collaborative allocation method for computing, network and storage resources based on dual drive of optimization and machine learning, which includes obtaining all user requests for resource allocation and storing them in a request set; inputting the current network resource state and algorithm decision state and the user state of the service function m of the service request φ that has not been traversed in the request set into a trained DRL model, obtaining the service function m of the service request φ and the benefit weight w of the static objects required by the service function m and placed on each node respectively. f 、w s ;W f 、w s The weight matrix W is spliced ​​to the algorithm decision state f and W s , determine whether m is equal to the total number of service functions of the service request φ. If so, proceed to the next step, otherwise update m = m + 1 and return to the model identification step; determine whether the service request φ is equal to H. If so, obtain the resource allocation strategy by solving the ILP optimization problem and the LP optimization problem, otherwise update φ = φ + 1 and return to the model identification step.

Description

Technical Field

[0001] The present invention relates to network resource allocation technology, and in particular to a method for collaborative allocation of computing, network and storage resources based on dual drive of optimization and machine learning. Background Art

[0002] Computing networks coordinate computing and storage resources across heterogeneous nodes, such as cloud, fog, and edge nodes, along with wide area network resources, to form a next-generation information infrastructure. This significantly promotes the widespread application of Data-Intensive Augmented Information (D-AgI) technology in a variety of emerging business scenarios. By integrating, processing, and enhancing various real-time data streams, D-AgI technology provides users with more personalized and customized services and interactions. This extends beyond immersive experiences (such as extended reality and holographic communication) to areas such as Artificial Intelligence Generated Content (AIGC) and Large Language Models (LLMs). This broadens the possibilities for users to interact with digital environments and effectively blurs the boundaries between the real and virtual worlds, ushering users into an unprecedented dimension of digital experience. Future application scenarios enabled by D-AgI, such as the metaverse, panoramic learning, remote surgery, and smart factories, will profoundly transform how people learn, work, and entertain.

[0003] To meet D-AgI's high demand for multi-dimensional resources, some work has proposed using distributed node resources in multi-layer computing networks to form service chains. This approach improves system efficiency by distributing the service functions and storage resources required across multiple nodes and combining this with the scheduling of user request traffic. The most advanced solution employs dynamic programming to make traffic scheduling decisions and updates cache placement based on multiple rounds of scoring results.

[0004] As the network size and number of users increase, the overhead of dynamic programming and scoring iterative optimization increases significantly. For example, in some work, the complexity of dynamic programming is , which makes it difficult to ensure the real-time resource allocation in a dynamic environment. Summary of the Invention

[0005] In response to the above-mentioned deficiencies in the existing technology, the present invention provides a collaborative allocation method for computing, network and storage resources based on dual-drive optimization and machine learning, which solves the problem that the existing technology is difficult to ensure the real-time allocation of resources in a dynamic environment.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0007] A method for collaboratively allocating computing, network, and storage resources based on optimization and machine learning is provided, which includes the following steps:

[0008] S1. Obtain all user requests to be allocated resources and store them in a request set; each user request corresponds to a service graph, including multiple service functions and static objects executed in sequence;

[0009] S2. The current network resource status and algorithm decision status and the service requests that have not been traversed in the request set are combined. The user state of the service function m is input into the trained DRL model to obtain the service request The service function m and its required static objects are placed on each node with a benefit weight w f 、w s ;

[0010] S3, the benefit weight w f 、w s The weight matrix W is spliced ​​to the algorithm decision state f and W s , and determine whether m is equal to the service request The total number of service functions If yes, go to step S4, otherwise update m=m+1 and return to step S2, 1≤m≤ ;

[0011] S4. Determine service request Is it equal to H? If so, go to step S5, otherwise update = +1, and return to step S2; 1≤ ≤H, where H is the total number of service requests in the request set;

[0012] S5. Solve the ILP optimization problem to obtain the allocation strategies x and y of all service functions and required static objects on nodes for all service requests. The ILP optimization problem is to maximize the weight matrix W. f and W s and the weighted sum of allocation strategies x and y;

[0013] S6. Solve the LP optimization problem to obtain the service request under the allocation strategy x and y. Real-time user traffic on the feasible path σ in time slot t , using allocation strategies x, y and user real-time traffic As the final resource allocation strategy, the LP optimization problem is to minimize the short-term queue backlog.

[0014] Furthermore, service requests The user status of the service function m includes the service request Request volume ,starting point ,end and the service request quad ,

[0015] For service requests The static object type required by the service function m; For service requests The aggregation rate of service function m, For service requests The workload of the service function m, For service requests The reduction factor of the service function m; For the time slot Next, service request All feasible paths The sum of the real-time traffic of users is expressed as:

[0016]

[0017] in, For the time slot Time Service Request Upper feasible path; For service requests Constructed layered graph In [1], the K shortest path algorithm is used to find the shortest path between the source and the destination, and the set of K feasible paths that meet the service request constraints is obtained.

[0018] Furthermore, maximizing the weight matrix and The expression of the weighted sum of the allocation strategies x and y is:

[0019]

[0020] st

[0021]

[0022]

[0023] Among them, max is the maximum value; is the size of the static object k; K is the set of all static objects; Whether to cache static object k at node i; Is the service request deployed at node i at time slot t? Service function m; is the storage resource available to node i at time slot t; x is all The allocation strategy composed of the allocation strategy of the components; is the dot product.

[0024] Furthermore, the expression for minimizing the short-term queue backlog is:

[0025]

[0026] st

[0027]

[0028]

[0029]

[0030]

[0031]

[0032]

[0033] Among them, min is the minimum value; and Node and links In the time slot Virtual queue; and are the accumulated traffic of node i and link ij in the physical topology at time slot t; For service requests workload of the service function; For user requests In the layered graph, the link traffic from node i in the mth layer to node i in the m+1th layer is For user requests In the layered graph, the link traffic from node i in the mth layer to node j in the mth layer; is the computation frequency of node i in time slot t; is the traffic on link ij associated with the static object k used in the service request; is the transmission bandwidth of link ij in time slot t; For service requests Constructed layered graph In [1], the K shortest path algorithm is used to find the shortest path between the source and the destination to obtain a set of K feasible paths that meet the service request constraints; For the time slot Time Service Request Upper feasible path; To make service request Traffic flows through , the computational load weight borne by node i; To make service request Traffic flows through The cumulative traffic weight generated on link ij when .

[0034] Furthermore, and The expressions are:

[0035]

[0036]

[0037]

[0038] in, and Service Request Cumulative scaling factors for the mth and m-1th service functions in ; Indicates that when the link A feasible path When it is 1, it does not belong to Time is 0; For service requests The reduction factor of the service function m-1.

[0039] Furthermore, the virtual queue and The expressions are:

[0040]

[0041]

[0042] in, and Node and links In the time slot Virtual queue; is the computation frequency of node i in time slot t-1; is the transmission bandwidth of link ij in time slot t-1; and are the accumulated traffic of node i and link ij in the physical topology at time slot t-1; Indicates that the result of the equation in this symbol must be greater than or equal to 0, otherwise it is forcibly set to 0.

[0043] Furthermore, the flow The expression is:

[0044]

[0045]

[0046]

[0047] in, From the super source The shortest path to node l; is the traffic on node l associated with the use of static object k in service requests at time slot t; is the traffic on node i associated with the use of static object k in service requests at time slot t; For service requests The reduction factor of the service function m; For user requests In the layered graph, the link traffic from node i in the mth layer to node i in the m+1th layer; For service requests The static object type required by the service function m; For user requests The end point; is the node i in the mth layer of the hierarchical graph.

[0048] Furthermore, the DRL model is a model designed based on the Actor-Critic architecture; the Critic network evaluates the benefit weight w generated by the Actor network by estimating the value function f 、w s , the value function is determined by the parameters of the Critic network. The parameters of the Critic network are updated according to the reward of reinforcement learning. The expression of reward is:

[0049]

[0050] in, Reward for the model; is the target value obtained by solving the LP optimization problem, that is ; C is a constant.

[0051] Furthermore, the output of the DRL model is A value, The first value in Value and The values ​​are respectively used as benefit weights w f and w s , is the total number of nodes in the network combined with the number of nodes in V.

[0052] Furthermore, user requests generate content for artificial intelligence, large language models need to transmit user-provided multimodal data, or visual data in virtual reality, augmented reality, holographic communication, and the metaverse.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] This solution designs DRL as a multi-stage output. Specifically, each stage of each user request generates an inference output. Each output is defined as a weight, representing the contribution of each integer decision variable to the optimization objective. Subsequently, a simple ILP method with a constrained weighted sum is solved to output the final deployment and caching actions. This design significantly reduces the DRL state and action space, improves algorithm efficiency, and ensures that the final output satisfies the constraints and globally optimizes the action.

[0055] 2. This solution leverages the dual-granularity solution framework of Lyapunov optimization and deep reinforcement learning (DRL) to decompose complex multi-stage stochastic optimization problems into easily solvable single-stage subproblems: the ILP optimization problem and the LP optimization problem. This reduces the difficulty of solving the problem at scale and provides real-time solution feasibility, thereby ensuring real-time and efficient resource allocation while also ensuring the long-term stability of the data queue (achieved through the LP optimization problem). The combination of DRL and ILP also significantly reduces algorithmic complexity, making the solution scalable to larger networks and higher user request volumes, and improving the algorithm's versatility and adaptability.

[0056] 3. This solution comprehensively considers service function deployment, static object caching, and traffic planning, achieving joint optimization of transmission, computing, and storage resources, significantly improving user request satisfaction rates and system resource utilization efficiency. Based on deep reinforcement learning, this solution can quickly adapt to changes and maintain efficient operation under highly dynamic user requests and network resource conditions, ensuring higher user traffic carrying capacity and better queue backlog control. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 Request a service graph for the user.

[0058] Figure 2 An auxiliary graph constructed based on the service graph in the user request; (a) is the original graph structure, (b) is the layered graph corresponding to the user request, and (c) is the super source point graph.

[0059] Figure 3 This is a flowchart of the collaborative allocation method of computing, network and storage resources based on dual-drive optimization and machine learning.

[0060] Figure 4 This is a block diagram of the principle of the collaborative allocation method of computing, network and storage resources based on dual drive of optimization and machine learning.

[0061] Figure 5 This is the principle block diagram of the Actor-Critic architecture.

[0062] Figure 6 Schematic diagram of queue backlog in dynamic network scenarios; (a) and (b) are schematic diagrams of queue backlog for node processing and link transmission in cloud-edge heterogeneous network resource scenarios, (c) and (d) are schematic diagrams of queue backlog for node processing and link transmission in edge network resource scenarios, and (e) and (f) are schematic diagrams of queue backlog for cloud-edge network resource scenarios.

[0063] Figure 7 Schematic diagram of resource utilization in a dynamic network scenario; (a) schematic diagram of node resource utilization; (b) schematic diagram of link resource utilization. DETAILED DESCRIPTION

[0064] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0065] To facilitate understanding of this solution, before introducing the collaborative allocation method of computing, network and storage resources, we introduce the multi-layer heterogeneous network model, user requests, layered graphs and super source point graphs involved in this solution:

[0066] Multi-layer heterogeneous network model

[0067] This solution models the multi-layer network considering transmission, calculation and storage as an undirected graph ,in Represents the set of all network nodes, Represents the set of all physical links connecting network nodes. and There are traffic inflow and outflow nodes respectively The multi-layer heterogeneous network model of this solution assumes that each node has computing and storage capabilities, and can perform various service functions and cache various static objects under the condition that the amount of resources allows. ,use Indicates that it is in the time slot The computing frequency of , its physical meaning is the maximum number of executable instructions in a time slot; Representation node The storage capacity is the maximum total volume of static objects that can be cached at this point.

[0068] For each link ,use Representative link The transmission bandwidth is the maximum total transmission volume within the time slot. is the set of all static objects; Is a static object The size of It should be noted that in this model, 、 and These are all time-varying. This is based on the following considerations: First, centralized or distributed resource management systems can dynamically and flexibly adjust resource availability to address fluctuations in user demand or maintain costs. Second, available resources may also fluctuate due to equipment aging, failures, and upgrades. Therefore, the present invention considers network systems with a certain degree of time-varying resource availability.

[0069] This model defines the cache vector As one of the decision variables, it indicates whether Caching static objects :

[0070]

[0071] A set of feasible cache vectors needs to satisfy the following storage resource constraints:

[0072]

[0073] A static object can be stored in multiple network nodes, and the storage resource constraint indicates that the total volume of static objects that can be stored in a network node cannot exceed the storage capacity of that point.

[0074] User Request

[0075] This solution defines that all user requests constitute a request set , each user request All required service functions are represented as service chains: a service chain has In each stage, the user's real-time data must pass through the corresponding service function of each stage in sequence and generate the available results of the next stage. At the same time, in order to process the real-time data of each unit of user, the service function of each stage also needs to accept a static object as input. This object can be reused by different user requests and needs to be pre-cached in the network. The above process can be represented by a directed acyclic graph (DAG). Figure 1 For each service function , this scheme uses a four-tuple To record service resource requirements:

[0076] For service requests The static object type required by the service function m; For service requests The aggregation rate of service function m, For service requests The workload of the service function m, For service requests The reduction factor of the service function m. Since the service function processes the data, the amount of data after processing may change. The reduction factor is the ratio of the data before and after processing.

[0077] The values ​​of the above four tuples are consistent with the user request It has nothing to do with how it is deployed on the network or the time slot. Each user request has a starting point and an end point, indicating the source and destination of the real-time data flow. , user traffic reaches compliance The Poisson distribution of .

[0078] Considering the characteristics of chain services and static object selection of D-AgI, it is often necessary to Figure 2 Traffic planning is performed on the service graph shown in (a), which is highly complex. Figure 2 (b) and Figure 2 The two graph structures and corresponding graph theory methods shown in (c) are used to assist in modeling, simplify traffic planning ideas, and obtain feasible paths for traffic.

[0079] Layered graph: Since user requests need to be processed and transmitted in different computing nodes in the form of service chains, this structure is complex and prone to loops, which is not conducive to the execution of graph algorithms. Therefore, a layered graph is introduced to simplify it. Assuming that the deployment of service functions is known, the service chain The length is , a service chain can be built for each user request (corresponding to a service chain) Layer diagram . Each layer in the network replicates the original physical network topology for data transmission targeting a specific service function. Layer Node Deployed the service function, then the node is connected to the Nodes in the layer Flowing through such a cross-layer link means that the data has passed the service function Processing. In the constructed On the top, user traffic is input from the corresponding source point on the first layer and The present invention uses the K shortest path algorithm to find the shortest path between the source and the sink, and can obtain K feasible paths that meet the service chain constraints, and let their set be At the same time, considering that the data volume will be scaled after being processed by each layer of service function, the present invention further defines a cumulative scaling factor:

[0080]

[0081] In order to facilitate the understanding of the layered diagram, combined Figure 2 For example, in (b): Assume the shortest path from A1 to C3 is: A1-C1-C2-D2-D3-C3. In the original graph, user traffic is planned as: A - C(F1) - D(F2) - C(F3). As can be seen, traffic passes through node C twice, which explains why a hierarchical graph is used to simplify path calculation and avoid loops.

[0082] Super Source Graph: Assuming Static Objects The deployment location is known, and a super source point is added to the original physical network topology , connect all deployed Nodes and set the weight to 0. After the construction is completed, using Dijkstra algorithm, you can find To any node The shortest path , indicating the slave node To the node Transferring static objects is the shortest transmission path.

[0083] For example: In the original graph (a), nodes A and C both deploy static object 1 (green). An auxiliary graph (c) is constructed, and the shortest paths from O1 to B and D are calculated as O1-AB and O1-CD, respectively. It can be seen that if B needs static object O1, it should be called from node A, and D needs to be called from node C.

[0084] refer to Figure 3 , Figure 3 A collaborative allocation method for computing, network, and storage resources based on optimization and machine learning is presented. Figure 4 The principle block diagram of the collaborative allocation method of computing, network and storage resources based on optimization and machine learning is shown; Figure 3 and Figure 4 As shown, the method S includes steps S1 to S6.

[0085] In step S1, all user requests to be allocated resources are obtained and stored in a request set; each user request corresponds to a service graph, including multiple service functions and static objects executed sequentially; user requests are content generated by artificial intelligence, multimodal data provided by users that need to be transmitted in large language models, or visual data of virtual reality, augmented reality, holographic communication and metaverse.

[0086] In step S2, the current network resource state and algorithm decision state and the service requests that have not been traversed in the request set are combined. The user state of the service function m is input into the trained DRL model to obtain the service request The service function m and its required static objects are placed on each node with a benefit weight w f 、w s ; Specifically, the output of the DRL model is A value, The first value in Value and The values ​​are respectively used as benefit weights w f and w s , is the total number of nodes in the network combined with the number of nodes in V.

[0087] When implemented, this solution prioritizes service requests The user status of the service function m includes the service request Request volume ,starting point ,end and the service request quad , For the time slot Next, service request All feasible paths The sum of the real-time traffic of users is expressed as:

[0088]

[0089] in, For the time slot Time Service Request Upper feasible path; For service requests Constructed layered graph In [1], the K shortest path algorithm is used to find the shortest path between the source and the destination, and the set of K feasible paths that meet the service request constraints is obtained.

[0090] In step S3, the benefit weight w f 、w s The weight matrix W is spliced ​​to the algorithm decision state fand W s , and determine whether m is equal to the service request The total number of service functions If yes, go to step S4, otherwise update m=m+1 and return to step S2, 1≤m≤ ;

[0091] In step S4, the service request is judged Is it equal to H? If so, go to step S5, otherwise update = +1, and return to step S2; 1≤ ≤H, where H is the total number of service requests in the request set;

[0092] In step S5, the ILP optimization problem is solved to obtain the allocation strategies x and y of all service functions and required static objects on the nodes for all service requests. The ILP optimization problem is to maximize the weight matrix W f and W s The weighted sum of the allocation strategies x and y.

[0093] When implemented, this solution prefers to maximize the weight matrix and The expression of the weighted sum of the allocation strategies x and y is:

[0094]

[0095] st

[0096]

[0097]

[0098] Among them, max is the maximum value; is the size of the static object k; K is the set of all static objects; Whether to cache static object k at node i; Is the service request deployed at node i at time slot t? Service function m; is the storage resource available to node i at time slot t; x is all The allocation strategy composed of the allocation strategy of the components; is the dot product.

[0099] In step S6, solve the LP optimization problem to obtain the service request under the allocation strategy x, y Real-time user traffic on the feasible path σ in time slot t , using allocation strategies x, y and user real-time traffic As the final resource allocation strategy, the LP optimization problem is to minimize the short-term queue backlog.

[0100] In one embodiment of the present invention, the expression for minimizing the short-term queue backlog is:

[0101]

[0102] st

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] Among them, min is the minimum value; and Node and links In the time slot Virtual queue; and are the accumulated traffic of node i and link ij in the physical topology at time slot t; For service requests workload of the service function; For user requests In the layered graph, the link traffic from node i in the mth layer to node i in the m+1th layer is For user requests In the layered graph, the link traffic from node i in the mth layer to node j in the mth layer; is the computation frequency of node i in time slot t; is the traffic on link ij associated with the static object k used in the service request; is the transmission bandwidth of link ij in time slot t; For service requests Constructed layered graph In [1], the K shortest path algorithm is used to find the shortest path between the source and the destination to obtain a set of K feasible paths that meet the service request constraints; For the time slot Time Service Request Upper feasible path; To make service request Traffic flows through , the computational load weight borne by node i; To make service request Traffic flows through The cumulative traffic weight generated on link ij when .

[0110] In the process of minimizing the short-term queue backlog, and The expressions are:

[0111]

[0112]

[0113]

[0114] in, and Service Request Cumulative scaling factors for the mth and m-1th service functions in ; Indicates that when the link A feasible path When it is 1, it does not belong to Time is 0; For service requests The reduction factor of the service function m-1.

[0115] Virtual Queue and The expressions are:

[0116]

[0117]

[0118] in, and Node and links In the time slot Virtual queue; is the computation frequency of node i in time slot t-1; is the transmission bandwidth of link ij in time slot t-1; and are the accumulated traffic of node i and link ij in the physical topology at time slot t-1; Indicates that the result of the equation in this symbol must be greater than or equal to 0, otherwise it is forcibly set to 0.

[0119] flow The expression is:

[0120]

[0121]

[0122]

[0123] in, From the super source The shortest path to node l; is the traffic on node l associated with the use of static object k in service requests at time slot t; is the traffic on node i associated with the use of static object k in service requests at time slot t; For service requests The reduction factor of the service function m; For user requests In the layered graph, the link traffic from node i in the mth layer to node i in the m+1th layer; For service requests The static object type required by the service function m; For user requests The end point; is the node i in the mth layer of the hierarchical graph.

[0124] The DRL model of this solution is a model designed based on the Actor-Critic architecture; Figure 5 As shown in the figure, the Actor-Critic architecture increases stability by introducing a residual network module. Among them, the Actor network generates action probabilities based on the current state input. It uses a series of residual blocks, each of which contains two linear transformations and a ReLU activation function, as well as batch normalization to stabilize the learning process, and optionally uses dropout for regularization. This architecture helps alleviate the gradient vanishing problem during training by introducing short-circuit connections, enhancing the network's ability to learn deep representations. The final output layer uses a Sigmoid activation function to ensure that the output value is in the range of [0, 1] to ensure that it can be interpreted as a probability. The Critic network evaluates the proposed action by estimating the value function, which represents the expected reward for a given state-action pair. Specifically:

[0125] The Critic network evaluates the benefit weight w generated by the Actor network by estimating the value function f 、w s , the value function is determined by the parameters of the Critic network. The parameters of the Critic network are updated according to the reward of reinforcement learning. The expression of reward is:

[0126]

[0127] in, Reward for the model; is the target value obtained by solving the LP optimization problem, that is ; C is a constant.

[0128] Similar to the Actor Network, the Critic Network uses a series of residual blocks to process the concatenated input of state and action, providing a powerful feature extraction mechanism. The output is a single value from the linear layer, representing the value of the action under the current policy instructed by the Actor Network.

[0129] To verify the effectiveness of this solution, the following is a detailed explanation with specific examples:

[0130] Experimental scenario: In terms of network topology selection, the present invention adopts 14-node NSFNET and 24-node USBNET topologies; in terms of network resource quantity, the present invention sets the node calculation frequency , node storage capacity and link transmission bandwidth The scope includes resource allocation for three network scenarios: edge, cloud-edge heterogeneous and cloud-end; on the user request service graph, the present invention sets the service graph quadruple Range and volume of static objects range, and randomly generate the service graph of each user from it. It is worth noting that the present invention uses Zipf distribution to allow overlapping static object demands between different users, making it possible to utilize the sharing of static objects; Regarding the user request volume, the user request volume in the experiment obeys the parameter Poisson distribution, the starting point and end point of the traffic are randomly generated; the number of user requests is the size of the request set ; The number of service functions per request is ; For the setting of simulation parameters, please refer to Table 1.

[0131] Table 1 Simulation parameter settings

[0132]

[0133] The algorithm proposed in this invention (hereinafter referred to as Bridge algorithm) takes each stage of each user request as input and determines the final action through multiple outputs.

[0134] Comparison Algorithms: For performance comparison, this paper considers two benchmark methods:

[0135] Greedy Random Algorithm: (1) Sort static objects in descending order by size and evenly distribute them to nodes, ensuring that each static object is cached at least once. Then, randomly select a node and a static object and check whether they can be cached. Repeat this process until all nodes can no longer cache them. (2) Find a shortest path between the starting point and the end point of a specific user request and map the service functions to the nodes on the path at equal distances. Then, under the premise of ensuring that each service function is deployed at least once, randomly select a mapping list and add, delete, or replace the mapping position of a service function in it. Iterate this process 1000 times. It should be noted that this algorithm performs service function deployment and static object caching separately, and does not consider joint optimization.

[0136] DRL algorithm: In the DRL comparison algorithm designed based on LyDROO, all stages of all user requests are directly used as input, and all deployment and caching actions are output at once. This method has a larger state space and action space, and the convergence difficulty is increased. Therefore, the DRL algorithm restricts the model output to a continuous value of [0, 1] and uses a quantizer to quantize the output 10 times, obtaining 10 sets of feasible 0-1 integer action plans. LP is solved using each of the 10 sets of plans, and the plan with the best solution is selected for training. This algorithm jointly optimizes the deployment of service functions and the caching of static objects. At the same time, the use of the quantizer accelerates model training and convergence, improving the quality of the algorithm solution, but also introduces a high time overhead. In addition, it is necessary to introduce a penalty term for constraint violation in the reward function so that the DRL algorithm produces an output that satisfies the constraints of the optimization problem.

[0137] Performance indicators:

[0138] Average Queue Backlog: This paper uses the average virtual queue backlog to reflect system stability, including both node processing queue backlog and link transmission queue backlog. Queue backlog indicates the effectiveness of the current resource allocation algorithm. Lower backlogs mean faster packet processing and routing, thereby improving overall network performance.

[0139] Average resource utilization: By statistically analyzing the utilization of computing power and transmission resources across the entire network, we can help determine whether the resource allocation algorithm is reasonable. For example, if the average queue backlog is large but the average resource utilization is low, it means that the algorithm is causing unnecessary hotspot congestion and limiting the upper limit of the traffic that can be accommodated.

[0140] The greedy-random algorithm, DRL algorithm and the Bridge algorithm of the present invention are all simulated under the simulation conditions shown in Table 1. The simulation results of these three methods can be referred to Figure 6 and Figure 7 :

[0141] Figure 6 The performance of the Bridge algorithm and its two benchmark methods in different network resource scenarios is shown. , all user requests are randomly updated every 10 time slots; in each time slot, the amount of network resources will appear randomly It is worth mentioning that, considering that the network queue will crash when it increases to a certain extent, the present invention sets 1300 as the upper limit of the queue backlog. Exceeding this value indicates that the resource allocation scheme is inappropriate or the current network resources cannot bear the user request volume. Figure 6 The average data queue backlog changes within 3000 runtime slots are plotted in the three resource scenarios of cloud-edge heterogeneous, pure edge and pure cloud networks. At the same time, the average resource utilization of each algorithm in the three scenarios is plotted in Figure 7 In the example, L and H are used to distinguish the same algorithm in different The performance below.

[0142] from Figure 6 As can be seen from (a) and (b) of the Greedy Random algorithm, the node processing queue backlog is very low, but at the expense of a high backlog in the link transmission queue. The transmission queue frequently reaches the backlog limit when When the speed was reduced to 45Mbps, the transmission queue backlog reached its upper limit, but it still remained extremely high (750Mb on average). This was because the Greedy Random algorithm's greedy strategy only considered allocating resources for each user's request along the shortest path, ignoring global load balancing and the joint optimization of transmission, computing, and storage resources. This easily led to unbalanced resource allocation and localized congestion.

[0143] pass Figure 7 In (a) and (b), we can observe that the Greedy Random algorithm has the highest link resource utilization and the lowest node resource utilization, which supports the rationality of the above analysis. In contrast, the DRL and Bridge algorithms with global optimization considerations also have the same The processing queue backlog of Bridge is slightly higher than that of DRL, but the transmission queue backlog is very small (less than 100Mb). The transmission queue backlog of DRL is 300Mb on average, with some fluctuations, up to 750Mb. Figure 7As can be seen from the figure, Bridge (L) has higher node computing power utilization and lower link resource utilization than DRL. To a certain extent, it transfers the link load to the node, resulting in a slight increase in processing queue backlog and a significant decrease in transmission queue backlog. This shows that Bridge has learned how to jointly optimize transmission, computing, and storage resources.

[0144] Finally, the present invention attempts to improve the DRL and Bridge , observe whether it can carry higher user request traffic. After a long period of training, the DRL algorithm is still difficult to converge. Either the output cannot meet the constraints, or the queue backlog reaches the upper limit. This is because under limited resources and high load, the optimization strategy is more difficult to explore. The Bridge algorithm can be used even in It can be predicted that as the load increases, the The processing queue backlog of the Bridge algorithm continues to rise to an average of 450Mb, and the transmission queue backlog is the same as On the one hand, thanks to the multi-stage action conversion and the constraint guarantee of the ILP solver, the Bridge algorithm is significantly better than the DRL algorithm in terms of convergence performance, making it possible to carry larger user traffic; on the other hand, The performance of Bridge on the transmission queue backlog once again demonstrates the superiority of Bridge's resource allocation strategy.

[0145] exist Figure 6 In (c) and (d), we can see that as the amount of network resources decreases, the original When the transmission queue can be stabilized by the Greedy Random algorithm, the backlog in the edge network increases and reaches the upper limit. Even at 40Mbps, the backlog remains high, near the upper limit. In contrast, the DRL and Bridge algorithms can still maintain high loads. Compared to the cloud-edge heterogeneous scenario, the queue backlogs for processing and transmission increase slightly and are more volatile, but they remain stable within an acceptable backlog range.

[0146] And in Figure 6 In (e) and (f), as the amount of resources increases, the queue backlog of each algorithm is significantly improved. The queue backlog of DRL and Bridge algorithms does not exceed 100Mb. Greedy Random algorithm can already support The load is not high and the queue backlog is not high. But it is still unbearable At the same time, the DRL algorithm It can also converge under the load of , which is because the constraints are easier to satisfy. It is worth noting that in the three scenarios, the node and link resource utilization characteristics of each algorithm are similar, that is, Figure 6 The analysis of (a) and (b) is also applicable to Figure 6 (c and (d) and Figure 6 (e) and (f) were analyzed.

Claims

1. A collaborative computing, network, and storage resource allocation method based on optimization and machine learning, characterized by: Including steps: S1. Obtain all user requests to be allocated resources and store them in a request set; each user request corresponds to a service graph, including multiple service functions and static objects executed in sequence; S2. The current network resource status and algorithm decision status and the service requests that have not been traversed in the request set are combined. The user state of the service function m is input into the trained DRL model to obtain the service request The service function m and its required static objects are placed on each node with a benefit weight w f 、w s ; S3, the benefit weight w f 、w s The weight matrix W is spliced ​​to the algorithm decision state f and W s , and determine whether m is equal to the service request The total number of service functions If yes, go to step S4, otherwise update m=m+1 and return to step S2, 1≤m≤ ; S4. Determine service request Is it equal to H? If so, go to step S5, otherwise update = +1, and return to step S2; 1≤ ≤H, where H is the total number of service requests in the request set; S5. Solve the ILP optimization problem to obtain the allocation strategies x and y of all service functions and required static objects on nodes for all service requests. The ILP optimization problem is to maximize the weight matrix W. f and W s and the weighted sum of allocation strategies x and y; S6. Solve the LP optimization problem to obtain the service request under the allocation strategy x and y. Real-time user traffic on the feasible path σ in time slot t , using allocation strategies x, y and user real-time traffic As the final resource allocation strategy, the LP optimization problem is to minimize the short-term queue backlog.

2. The method for collaborative allocation of computing, network and storage resources according to claim 1, characterized in that: Service Request The user status of the service function m includes the service request Request volume ,starting point ,end and the service request quad , For service requests The static object type required by the service function m; For service requests The aggregation rate of service function m, For service requests The workload of the service function m, For service requests The reduction factor of the service function m; For the time slot Next, service request All feasible paths The sum of the real-time traffic of users is expressed as: in, For the time slot Time Service Request feasible paths on ; For service requests Constructed layered graph In [1], the K shortest path algorithm is used to find the shortest path between the source and the destination, and the set of K feasible paths that meet the service request constraints is obtained.

3. The method for collaborative allocation of computing, network and storage resources according to claim 1 or 2, characterized in that: Maximize the weight matrix and The expression of the weighted sum of the allocation strategies x and y is: st Among them, max is the maximum value; is the size of the static object k; K is the set of all static objects; Whether to cache static object k at node i; Is the service request deployed at node i at time slot t? Service function m; is the storage resource available to node i at time slot t; x is all The allocation strategy composed of the allocation strategy of the components; is the dot product.

4. The method for collaborative allocation of computing network storage resources according to claim 3, characterized in that: The expression for minimizing the short-term queue backlog is: st Among them, min is the minimum value; and Node and links In the time slot Virtual queue; and are the accumulated traffic of node i and link ij in the physical topology at time slot t; For service requests workload of the service function; For user requests In the layered graph, the link traffic from node i in the mth layer to node i in the m+1th layer is For user requests In the layered graph, the link traffic from node i in the mth layer to node j in the mth layer; is the computation frequency of node i in time slot t; is the traffic on link ij associated with the static object k used in the service request; is the transmission bandwidth of link ij in time slot t; For service requests Constructed layered graph In [1], the K shortest path algorithm is used to find the shortest path between the source and the destination to obtain a set of K feasible paths that meet the service request constraints; For the time slot Time Service Request Upper feasible path; To make service request Traffic flows through , the computational load weight borne by node i; To make service request Traffic flows through The cumulative traffic weight generated on link ij when .

5. The method for collaborative allocation of computing, network and storage resources according to claim 4, characterized in that: and The expressions are: in, and Service Request Cumulative scaling factors for the mth and m-1th service functions in ; Indicates that when the link A feasible path When it is 1, it does not belong to Time is 0; For service requests The reduction factor of the service function m-1.

6. The method for collaborative allocation of computing, network and storage resources according to claim 4, characterized in that: Virtual Queue and The expressions are: in, and Node and links In the time slot Virtual queue; is the computation frequency of node i in time slot t-1; is the transmission bandwidth of link ij in time slot t-1; and are the accumulated traffic of node i and link ij in the physical topology at time slot t-1; Indicates that the result of the equation in this symbol must be greater than or equal to 0, otherwise it is forcibly set to 0.

7. The method for collaborative allocation of computing, network and storage resources according to claim 4, characterized in that: flow The expression is: in, From the super source The shortest path to node l; is the traffic on node l associated with the use of static object k in service requests at time slot t; is the traffic on node i associated with the use of static object k in service requests at time slot t; For service requests The reduction factor of the service function m; For user requests In the layered graph, the link traffic from node i in the mth layer to node i in the m+1th layer; For service requests The static object type required by the service function m; For user requests The end point; is the node i in the mth layer of the hierarchical graph.

8. The method for collaborative allocation of computing, network and storage resources according to claim 4, characterized in that: The DRL model is a model designed based on the Actor-Critic architecture; the Critic network evaluates the benefit weight w generated by the Actor network by estimating the value function f 、w s , the value function is determined by the parameters of the Critic network, and the parameters of the Critic network are updated according to the reward of reinforcement learning. The expression of reward is: in, Reward for the model; is the target value obtained by solving the LP optimization problem, that is ; C is a constant.

9. The method for collaborative allocation of computing, network and storage resources according to any one of claims 1 to 8, characterized in that: The output of the DRL model is A value, The first value in Value and The values ​​are respectively used as benefit weights w f and w s , is the total number of nodes in the network combined with the number of nodes in V.

10. The method for collaborative allocation of computing, network and storage resources according to any one of claims 1 to 8, characterized in that: The user request is for artificial intelligence-generated content, multimodal data provided by the user that needs to be transmitted in a large language model, or visual data in virtual reality, augmented reality, holographic communication, and the metaverse.

Images