Information center network-based computing power network task scheduling method and device

Abstract

The application provides a kind of computing power network task scheduling method and device based on information center network, the steps of the method include: forwarding node receives the processing request of terminal user, and the processing request is sent to control node, the control node is provided with pre-trained reinforcement learning model, the target network in the reinforcement learning model is based on the action of the processing task corresponding to processing request for processing task Allocation computing node, the computing node is provided with the computing resource for executing processing task;The control node calculates the total time delay and the total energy consumption of action based on the action of allocating computing node made by the reinforcement learning model, the control node calculates reward function based on total time delay and total energy consumption of action, calculates loss function based on reward function;The parameters of value network in reinforcement learning model are updated based on loss function, and the parameters in value network are synchronized to target network every preset step.

Description

Technical Field

[0001] The present invention relates to the field of computing power network technology, and in particular to a computing power network task scheduling method and device based on an information center network. Background Art

[0002] In recent years, many researchers have proposed the concept of CFN (Compute First Networking) and built a CFN architecture that is deeply integrated with the "cloud-edge-end" computing network.

[0003] Compared to traditional edge computing, CFN can perceive the resource status of the infrastructure layer and orchestrate computing, network, and storage resources more finely. Furthermore, CFN can locate services on the data plane and perform appropriate task scheduling and routing planning upon receiving computing requests from consumers. However, computing resources in CFN are heterogeneous and distributed across different computing nodes, making effective unified perception and coordination impossible. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a computing network task scheduling method based on an information-centric network to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a method for scheduling computing network tasks based on an information-centric network. The method is applied to a computing network comprising forwarding nodes, control nodes, and computing nodes. The method comprises the following steps:

[0006] The forwarding node receives a processing request from a terminal user and sends the processing request to a control node. The control node is provided with a pre-trained reinforcement learning model. The target network in the reinforcement learning model allocates a computing node to the processing task based on the processing task corresponding to the processing request. The computing node is provided with computing resources for executing the processing task.

[0007] The control node calculates the total action delay and the total action energy consumption based on the action of the allocated computing node made by the reinforcement learning model, the control node calculates the reward function based on the total action delay and the total action energy consumption, and calculates the loss function based on the reward function;

[0008] Update the parameters of the value network in the reinforcement learning model based on the loss function, and synchronize the parameters in the value network to the target network every preset number of steps.

[0009] By adopting the above scheme, in the scheme of the present invention, when a task arrives, the task execution is comprehensively planned to ensure global optimization. This scheme also takes into account both total latency and total energy consumption, consuming the lowest energy while ensuring user service quality. Therefore, in this scenario, this scheme is a better choice that is closer to the global optimal goal of the actual scenario, and can enable resources distributed in different computing nodes to be uniformly processed, effectively unifying perception and coordination.

[0010] In some embodiments of the present invention, the step of allocating a computing node to a processing task based on the processing task corresponding to the processing request by the target network in the reinforcement learning model includes:

[0011] The control node calculates state parameters of the power network, the state parameters including forwarding node state parameters and computing node state parameters, and the forwarding node state parameters and computing node state parameters are combined into a state vector;

[0012] The state vector is input into a reinforcement learning model, which outputs an action vector. The dimension corresponding to the largest parameter among the multiple dimensions of the action vector is extracted, and the action corresponding to the dimension is used as the output action.

[0013] In some embodiments of the present invention, the forwarding node status parameter is used to mark whether each forwarding node is sending a processing request to the control node. If so, the forwarding node is marked as the first parameter, otherwise, the forwarding node is marked as the second parameter; the node status parameter is composed of a service function parameter, a node resource parameter and a remaining time parameter. The service function parameter is used to mark whether each computing node is executing a processing request. If so, the computing node is marked as the first parameter, otherwise, the computing node is marked as the second parameter; the node resource parameter is the computing resource size of the computing node; the remaining time parameter is the remaining time required for the computing node to execute the current processing request.

[0014] In some embodiments of the present invention, the action includes a transmission path for processing a task and a pull path for a container corresponding to the processing task, the total action delay includes transmission delay, queuing delay, container pull delay, and computation delay, the total action energy consumption includes transmission energy consumption, container pull energy consumption, and computation energy consumption, and the step of calculating the total action delay and the total action energy consumption by the control node for allocating computing nodes based on the reinforcement learning model includes:

[0015] Calculate the transmission delay based on the bandwidth parameters in the transmission path, calculate the container pulling delay based on the bandwidth parameters in the pulling path, calculate the computing delay required for the processing task based on the CPU resources and GPU resources in the destination computing node, and calculate the queuing delay of the processing task based on the computing delay of the processing task that arrived earlier than the processing task; calculate the total action delay based on the transmission delay, queuing delay, container pulling delay, and computing delay;

[0016] The transmission energy consumption is calculated based on the transmission power of the sending router in the transmission path, the container pulling energy consumption is calculated based on the transmission power of the sending router in the pulling path, and the computing energy consumption is calculated based on the CPU resources and GPU resources in the destination computing node; the total energy consumption of the action is calculated based on the transmission energy consumption, container pulling energy consumption and computing energy consumption.

[0017] In some embodiments of the present invention, in the step of calculating the transmission delay based on the bandwidth parameter in the transmission path, the transmission delay is calculated based on the following formula:

[0018]

[0019] Among them, T trans represents the transmission delay, R represents the set of routers in the transmission path, i and j represent any two adjacent routers in the transmission path, B (i,j) represents the bandwidth between router i and router j, Indicates the size of the processing task k;

[0020] In the step of calculating the container pulling delay based on the bandwidth parameter in the pulling path, the pulling delay is calculated based on the following formula:

[0021]

[0022] Among them, T pull represents the pull delay, U represents the set of routers in the pull path, x and y represent any two adjacent routers in the pull path, B (x,y) represents the bandwidth between router x and router y, Z docker Indicates the size of the container corresponding to the processing task;

[0023] In the step of calculating the computing delay required for the processing task based on the CPU resources and GPU resources in the destination computing node, the computing delay is calculated according to the following formula:

[0024]

[0025] Among them, T comp Indicates calculation delay, Max{} indicates taking the maximum value, represents the size of the processing task k, Indicates the GPU resource size of the destination computing node. Indicates the CPU resource size of the target computing node, λ is the preset task parameter, ε is the preset GPU positive correlation coefficient parameter, and μ is the preset CPU positive correlation coefficient parameter;

[0026] In the step of calculating the queuing delay of the processing task based on the computational delays of the processing tasks that arrived earlier than the processing task, the sum of the computational delays of the processing tasks that have not yet been completed when the processing task arrives and that arrived earlier than the processing task is calculated to obtain the queuing delay;

[0027] In the step of calculating the total action delay based on the transmission delay, queuing delay, container pull delay, and calculation delay, the total action delay is calculated based on the following formula:

[0028] T total =T trans +T queue +T pull +T comp ;

[0029] Among them, T total Indicates the total action delay, T queue Indicates the queuing delay.

[0030] In some embodiments of the present invention, in the step of calculating the transmission energy consumption based on the transmission power of the transmitting router in the transmission path, the transmission energy consumption is calculated based on the following formula:

[0031]

[0032] Among them, E trans represents the transmission energy consumption, i and j represent any two adjacent routers in the transmission path, P (i,j) represents the transmission power of the sending router in the adjacent router i and router j, T trans Indicates transmission delay;

[0033] In the step of calculating the container pulling energy consumption based on the transmission power of the sending end router in the pulling path, the container pulling energy consumption is calculated based on the following formula:

[0034]

[0035] Among them, E pull represents the container pulling energy consumption, U represents the set of routers in the pulling path, x and y represent any two adjacent routers in the pulling path, P (x,y) represents the transmission power of the sending router in the adjacent router x and router y, T pull Indicates transmission delay;

[0036] In the step of calculating the computing energy consumption based on the CPU resources and GPU resources in the destination computing node, the computing energy consumption is calculated based on the following formula:

[0037] E comp =P com T comp ;

[0038] Among them, E comp represents computing energy consumption, P com Indicates the computing capacity parameter of the destination computing node, T comp Indicates the computation delay;

[0039] In the step of calculating the total energy consumption of an action based on the transmission energy consumption, container pulling energy consumption, and calculation energy consumption, the total energy consumption of the action is calculated according to the following formula:

[0040] E total =E trans +E pull +E comp ;

[0041] Among them, E total Indicates the total energy consumption of the action.

[0042] In some embodiments of the present invention, the computing capacity parameter of the destination computing node is calculated according to the following formula:

[0043]

[0044] Among them, P com Represents the computing power parameter of the destination computing node, η G Represents the GPU power factor preset by the destination computing node, η C Indicates the CPU power factor preset for the destination computing node. Indicates the GPU resource size of the destination computing node. Indicates the CPU resource size of the destination compute node.

[0045] In some embodiments of the present invention, in the step of calculating the reward function based on the total action delay and the total action energy consumption, the reward function value is calculated according to the following formula:

[0046] R=ξ-ρ(αT total +βE total );

[0047] Among them, R represents the reward function value, α, ξ and ρ are preset calculation parameters, E total Represents the total energy consumption of the action, T total Indicates the total action delay.

[0048] In some embodiments of the present invention, obtaining the state parameters of the computing power network after executing the action, the step of calculating the loss function based on the reward function includes:

[0049] Encode the state parameters of the computing network before the action, the action, the reward function, and the state parameters of the computing network after the action, and combine them into a four-tuple, and add the four-tuple to the memory space;

[0050] Randomly collect any quadruple from the memory space and obtain the state parameters of the computing power network after the action in the quadruple.

[0051] Based on the state parameters of the post-action computing network in the extracted quadruple, the corresponding state vector is obtained, and the state vector is input into the target network in the reinforcement learning model to obtain the action vector corresponding to the state vector. The largest parameter among the multiple dimensions of the action vector is output as the training dimension parameter.

[0052] The transition function is calculated based on the reward function value and training dimension parameters in the extracted quadruple according to the following formula:

[0053]

[0054] Among them, y δ represents the transition function, Represents the reward function value in the extracted quadruple δ, represents the training dimension parameter, and γ is the preset training parameter;

[0055] The loss function is calculated based on the transition function according to the following formula:

[0056] L=[y δ -q] 2 ;

[0057] Among them, L represents the loss function, y δ Represents the transition function, and q represents the dimension parameter value corresponding to the action in the extracted quadruple.

[0058] Another aspect of the present invention also provides a computing network task scheduling device based on an information center network, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps implemented by the method described above.

[0059] Additional advantages, objects, and features of the present invention will be described in part in the following description and will become apparent to those skilled in the art after studying the following or may be learned by practice of the present invention. The objects and other advantages of the present invention may be particularly pointed out and attained in the description and drawings.

[0060] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0062] Figure 1 A schematic diagram of an implementation of a computing network task scheduling method based on an information-centric network according to the present invention;

[0063] Figure 2 This is a schematic diagram of the implementation architecture of the present invention;

[0064] Figure 3 This is a schematic diagram of the implementation process structure of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0066] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0067] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0068] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0069] Introduction to existing technology:

[0070] Prior art 1 adopts the concept of named function network (NFN), where the network acts as a parser, and designs a universal parser for λ-expressions to solve the problem of orchestration of information access.

[0071] Prior art 2 adopts a named Function as a Service (NFaaS) architecture, in which functions can be deployed in the network and can be moved between nodes according to user needs.

[0072] Existing technology 3 adopts efficient resource discovery, computation reuse, mobility management, and security in AR scenarios using NFN and NFaaS.

[0073] Prior art 4 adopts a unified method of remote function calling.

[0074] Existing technology 5 adopts the service placement problem in the edge computing scenario and proposes a service caching strategy through the ICN paradigm, which considers caching service instances in network nodes so that users can respond quickly when repeatedly requesting services.

[0075] The above existing technologies help deploy, discover, and call function services in edge and core networks.

[0076] The prior art 6 combines ICN and CFN to propose a named CFN, namely, NCFN solution.

[0077] In the NCFN scheme, first, the named addressing mode enables flexible computing resource discovery at the network layer; second, the in-network computing mechanism effectively shortens the service call latency and improves the user's quality of experience (QoE), where the lightweight container used to carry the computing service replica can be easily migrated and deployed in the network; finally, the in-network caching mechanism can avoid repeated service calls by reusing the computing results. Based on the NFCN scheme, researchers proposed a naming service access control scheme based on proxy signature (NSACS-PS). In this scheme, the source server provides the authorized user with the authorization certificate for the subscription service, and the authorized user can sign the interest packet for calling the computing service, where the signature has the same validity as the signature of the source server. On the router side, the deployed service replica only needs to hold the public key of the source server to verify who is the authorized user.

[0078] There are three participants in the system model: the source server, authorized users, and service replicas deployed on routers. To support in-network computing, the source server clones service replicas and distributes them across the network. NCFN routers load service replicas with a high request frequency based on the least frequently used (LFU) strategy. Since service replicas deployed on routers are often closer to users, they can provide computing services to nearby authorized users.

[0079] Disadvantages of existing technology:

[0080] 1. Existing technologies focus more on analyzing the security of service calls in the network, while ignoring the overhead caused by task scheduling and the imbalance of resource utilization;

[0081] 2. The existing task scheduling solution caches the calculation results in the network. In actual scenarios, different users rarely request the same calculation results, so this solution has limitations in actual scenarios.

[0082] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0083] To solve the above problems, the present invention proposes a computing network task scheduling method based on an information-centric network. The method is applied to a computing network comprising forwarding nodes, control nodes, and computing nodes. The method comprises the following steps:

[0084] In the specific implementation process, the architecture of this solution is as follows Figure 2 As shown in Figure 1, the architecture is divided into a control plane and a data plane. The data plane is a set of network entities that perform basic operations, including forwarding, caching, and computing. The control plane centrally controls and manages these underlying physical resources based on the status information of computing resources and network resources collected from the data plane.

[0085] In the above network architecture, the control plane perceives the data plane resources, makes intelligent decisions on user requests, and schedules tasks to the corresponding containers of the computing nodes for computing.

[0086] Compute nodes are network devices with computing capabilities, whose primary function is to provide various computing services. A service provider launches a container with various allocated resources and deploys different service functions within it. The computing resources allocated to each container vary, primarily including a CPU for general computing tasks, a GPU for neural network training and graphics rendering, and memory for fast read and write access and caching of intermediate computational results. Furthermore, compute nodes can pull containers from the network for hot start, ensuring that all user service requests are met.

[0087] A forwarding node is a router equipped with a CS (memory storage), a PIT (pending interest table), and a FIB (forwarding database). Its primary functions are to forward user task requests and cache containers providing services within the network. The CS caches containers that can be requested by compute nodes; the PIT stores the names and request interfaces of all containers requested by compute nodes; and the FIB records the container forwarding interfaces.

[0088] End users initiate computing task requests. Tasks are categorized as latency-sensitive, compute-intensive, and I / O-intensive, depending on their needs. To fulfill these user-initiated task requests, the reinforcement learning model selects appropriate compute nodes to provide computing services.

[0089] The control node is used to collect computing and network resource status information of the data plane, then make decisions on task requests initiated by users and publish routing tables to forwarding nodes.

[0090] In its implementation, the present invention considers combining CFN with ICN to effectively address the aforementioned issues. Compared to traditional IP networks, ICN prioritizes the successful acquisition of service content, rather than its source. Therefore, CFN enables intelligent decision-making and scheduling to ensure user service quality. Secondly, the hierarchical naming mechanism within ICN effectively represents user needs and network resources, facilitating awareness by sensing nodes within the CFN. Furthermore, routing nodes within the ICN possess cache tables for caching services, as well as PITs (Pending Interest Tables) and FIBs (Forward Information Databases) for automated routing to obtain service resources within the network.

[0091] ICN stands for Information-Centric Networking.

[0092] In some embodiments of the present invention, the forwarding node may be a router, the computing node may be a computer, and the control node may be a computer or a server. In the computing power network, each forwarding node and each computing node are connected to the control node, and each computing node has a forwarding node connected to it.

[0093] like Figure 1 As shown, in step S100, the forwarding node receives a processing request from a terminal user and sends the processing request to a control node. The control node is provided with a pre-trained reinforcement learning model. The target network in the reinforcement learning model allocates a computing node to the processing task based on the processing task corresponding to the processing request. The computing node is provided with computing resources for executing the processing task.

[0094] In some embodiments of the present invention, the action includes an instruction to transfer the processing task to the computing node and an instruction to transfer the container corresponding to the processing task to the computing node, as well as a transmission path for transferring the processing task to the computing node and a pull path for transferring the container corresponding to the processing task to the computing node, and the container corresponding to the processing task is set in any forwarding node.

[0095] The transmission path and the pull path may both include a plurality of routers connected to each other, and the plurality of routers are connected to each other for forwarding.

[0096] In some embodiments of the present invention, the reinforcement learning model is a DQN model, namely, Deep Q-Network; the computing resources of the computing node include CPU resources and GPU resources.

[0097] Step S200, the control node calculates the total action delay and the total action energy consumption based on the action of the allocated computing node made by the reinforcement learning model, the control node calculates the reward function based on the total action delay and the total action energy consumption, and calculates the loss function based on the reward function;

[0098] In some embodiments of the present invention, the total action delay includes transmission delay, queuing delay, container pulling delay and computing delay, and the total action energy consumption includes transmission energy consumption, container pulling energy consumption and computing energy consumption.

[0099] Step S300: Update the parameters of the value network in the reinforcement learning model based on the loss function, and synchronize the parameters in the value network to the target network every preset number of steps.

[0100] In some embodiments of the present invention, in the step of updating the parameters of the value network in the reinforcement learning model based on the loss function, the parameters of the value network in the reinforcement learning model are updated using a gradient descent method based on the loss function.

[0101] In some embodiments of the present invention, in the step of synchronizing the parameters in the value network to the target network every preset number of steps, each update of the parameters of the value network is one step, and the parameters in the value network are synchronized to the target network after the preset number of updates, and the value network and the target network have the same structure.

[0102] By adopting the above scheme, in the scheme of the present invention, when a task arrives, the task execution is comprehensively planned to ensure global optimization. This scheme also takes into account both total latency and total energy consumption, consuming the lowest energy while ensuring user service quality. Therefore, in this scenario, this scheme is a better choice that is closer to the global optimal goal of the actual scenario, and can enable resources distributed in different computing nodes to be uniformly processed, effectively unifying perception and coordination.

[0103] like Figure 3As shown, in the resource awareness phase, forwarding nodes and compute nodes periodically register the status of computing resources, network resources, and cache resources with the control plane. The control plane then establishes a resource topology map and constructs a network state space, enabling it to perceive and control the computing and network resources of the data plane. In the request, decision, and forwarding phase, users initiate task requests to forwarding nodes. The control plane makes intelligent decisions based on the task type and parameters, scheduling the task to the appropriate compute node container and publishing a routing table to the forwarding node. In the container pull request phase, if a compute node does not have a container that can provide computing services for the user, it pulls the corresponding container from the network. The compute node accesses the forwarding node's cache. If the cache does not contain the same request record, it searches the PIT to check for the existence of a matching request record. If so, the requesting interface is added to the interface column. Otherwise, the node creates a new tuple and forwards the request to another forwarding node based on the FIB. Finally, the forwarding node returns the cached container to the compute node. In the task computation phase, the compute node performs the computation service, completing the user task within the specified timeframe and returning the computation result.

[0104] In the above network model, the control node needs to make decisions on user tasks, match tasks with computing nodes, and then schedule computing tasks to computing nodes for calculation, and finally return the calculation results.

[0105] In some embodiments of the present invention, the step of allocating a computing node to a processing task based on the processing task corresponding to the processing request by the target network in the reinforcement learning model includes:

[0106] The control node calculates state parameters of the power network, the state parameters including forwarding node state parameters and computing node state parameters, and the forwarding node state parameters and computing node state parameters are combined into a state vector;

[0107] The state vector is input into a reinforcement learning model, which outputs an action vector. The dimension corresponding to the largest parameter among the multiple dimensions of the action vector is extracted, and the action corresponding to the dimension is used as the output action.

[0108] In some embodiments of the present invention, the forwarding node status parameter is used to mark whether each forwarding node is sending a processing request to the control node, that is, whether the forwarding node is triggered by the processing request. If so, the forwarding node is marked as the first parameter, otherwise, the forwarding node is marked as the second parameter; the node status parameter is composed of a service function parameter, a node resource parameter and a remaining time parameter. The service function parameter is used to mark whether each computing node is executing a processing request. If so, the computing node is marked as the first parameter, otherwise, the computing node is marked as the second parameter; the node resource parameter is the computing resource size of the computing node; the remaining time parameter is the remaining time required for the computing node to execute the current processing request.

[0109] During the specific implementation process, the first parameter can be 1 and the second parameter can be 0. In the step where the remaining time parameter is the remaining time required for the computing node to execute the current processing request, the total delay required for the computing node to execute the current processing request is obtained by calculating the computing delay, and then subtracting the time consumed by the computing node to execute the current processing request to obtain the remaining time required for the computing node to execute the current processing request.

[0110] In some embodiments of the present invention, the action includes a transmission path for processing a task and a pull path for a container corresponding to the processing task, the total action delay includes transmission delay, queuing delay, container pull delay, and computation delay, the total action energy consumption includes transmission energy consumption, container pull energy consumption, and computation energy consumption, and the step of calculating the total action delay and the total action energy consumption by the control node for allocating computing nodes based on the reinforcement learning model includes:

[0111] Calculate the transmission delay based on the bandwidth parameters in the transmission path, calculate the container pulling delay based on the bandwidth parameters in the pulling path, calculate the computing delay required for the processing task based on the CPU resources and GPU resources in the destination computing node, and calculate the queuing delay of the processing task based on the computing delay of the processing task that arrived earlier than the processing task; calculate the total action delay based on the transmission delay, queuing delay, container pulling delay, and computing delay;

[0112] The transmission energy consumption is calculated based on the transmission power of the sending router in the transmission path, the container pulling energy consumption is calculated based on the transmission power of the sending router in the pulling path, and the computing energy consumption is calculated based on the CPU resources and GPU resources in the destination computing node; the total energy consumption of the action is calculated based on the transmission energy consumption, container pulling energy consumption and computing energy consumption.

[0113] In some embodiments of the present invention, in the step of calculating the transmission delay based on the bandwidth parameter in the transmission path, the transmission delay is calculated based on the following formula:

[0114]

[0115] Among them, T trans represents the transmission delay, R represents the set of routers in the transmission path, i and j represent any two adjacent routers in the transmission path, B (i,j) represents the bandwidth between router i and router j, Indicates the size of the processing task k;

[0116] In some embodiments of the present invention, the router i and the router j are forwarding nodes, and the transmission path is composed of at least one router and a final destination computing node.

[0117] By adopting the above scheme, the time delay consumed by every two adjacent routers in the transmission path is calculated separately and superimposed to obtain the transmission delay, thereby improving the accuracy of transmission delay calculation.

[0118] In the step of calculating the container pulling delay based on the bandwidth parameter in the pulling path, the pulling delay is calculated based on the following formula:

[0119]

[0120] Among them, T pull represents the pull delay, U represents the set of routers in the pull path, x and y represent any two adjacent routers in the pull path, B (x,y) represents the bandwidth between router x and router y, Z docker Indicates the size of the container corresponding to the processing task;

[0121] In some embodiments of the present invention, the delay consumed by every two adjacent routers in the pull path is calculated separately and superimposed to obtain the transmission delay, thereby improving the accuracy of transmission delay calculation.

[0122] During the specific implementation process, if the computing node caches the container corresponding to the processing task, the pull delay is 0.

[0123] In the step of calculating the computing delay required for the processing task based on the CPU resources and GPU resources in the destination computing node, the computing delay is calculated according to the following formula:

[0124]

[0125] Among them, T comp Indicates calculation delay, Max{} indicates taking the maximum value, represents the size of the processing task k, Indicates the GPU resource size of the destination computing node. Indicates the CPU resource size of the target computing node, λ is the preset task parameter, ε is the preset GPU positive correlation coefficient parameter, and μ is the preset CPU positive correlation coefficient parameter;

[0126] In some embodiments of the present invention, the CPU resources and GPU resources in the computing node can run simultaneously. In this case, the time it takes to finally complete the computing task in the CPU resources and the GPU resources is the computing latency.

[0127] In the step of calculating the queuing delay of the processing task based on the computational delays of the processing tasks that arrived earlier than the processing task, the sum of the computational delays of the processing tasks that have not yet been completed when the processing task arrives and that arrived earlier than the processing task is calculated to obtain the queuing delay;

[0128] In the step of calculating the total action delay based on the transmission delay, queuing delay, container pull delay, and calculation delay, the total action delay is calculated based on the following formula:

[0129] T total =T trans +T queue +T pull +T comp ;

[0130] Among them, T total Indicates the total action delay, T queue Indicates the queuing delay.

[0131] In some embodiments of the present invention, in the step of calculating the transmission energy consumption based on the transmission power of the transmitting router in the transmission path, the transmission energy consumption is calculated based on the following formula:

[0132]

[0133] Among them, E trans represents the transmission energy consumption, i and j represent any two adjacent routers in the transmission path, P (i,j) represents the transmission power of the sending router in the adjacent router i and router j, T trans Indicates transmission delay;

[0134] In some embodiments of the present invention, if, among adjacent routers i and j, router i is a transmitting end and router j is a receiving end, the transmission power of router i is included in the calculation.

[0135] In the step of calculating the container pulling energy consumption based on the transmission power of the sending end router in the pulling path, the container pulling energy consumption is calculated based on the following formula:

[0136]

[0137] Among them, E pull represents the container pulling energy consumption, U represents the set of routers in the pulling path, x and y represent any two adjacent routers in the pulling path, P (x,y) represents the transmission power of the sending router in the adjacent router x and router y, T pull Indicates transmission delay;

[0138] Using the above method, the energy consumption of each two adjacent routers during the transmission process is calculated one by one, and the energy consumption of container pulling and the transmission energy consumption required for processing tasks are accurately calculated.

[0139] In the step of calculating the computing energy consumption based on the CPU resources and GPU resources in the destination computing node, the computing energy consumption is calculated based on the following formula:

[0140] E comp =P com T comp ;

[0141] Among them, E comp represents computing energy consumption, P com Indicates the computing capacity parameter of the destination computing node, T comp Indicates the computation delay;

[0142] In the step of calculating the total energy consumption of an action based on the transmission energy consumption, container pulling energy consumption, and calculation energy consumption, the total energy consumption of the action is calculated according to the following formula:

[0143] E total =E trans +E pull +E comp ;

[0144] Among them, R total Indicates the total energy consumption of the action.

[0145] In some embodiments of the present invention, the computing capacity parameter of the destination computing node is calculated according to the following formula:

[0146]

[0147] Among them, P com Represents the computing power parameter of the destination computing node, η G Represents the GPU power factor preset by the destination computing node, η C Indicates the CPU power factor preset for the destination computing node. Indicates the GPU resource size of the destination computing node. Indicates the CPU resource size of the destination compute node.

[0148] Using the above solution, this solution calculates the computing power parameters of the computing node through the GPU resources and CPU resources of the computing node, and calculates the computing energy consumption based on the computing power parameters to ensure the computing accuracy of the computing node.

[0149] In some embodiments of the present invention, in the step of calculating the reward function based on the total action delay and the total action energy consumption, the reward function value is calculated according to the following formula:

[0150] R=ξ-ρ(αT total +βE total );

[0151] Among them, R represents the reward function value, α, ξ and ρ are preset calculation parameters, E total Represents the total energy consumption of the action, T total Indicates the total action delay.

[0152] With this approach, during each training round, the system selects actions from the action space based on the current state at each epoch to receive rewards. As rewards accumulate over multiple rounds, the system will tend to develop routing strategies that help optimize the goal.

[0153] In some embodiments of the present invention, obtaining the state parameters of the computing power network after executing the action, the step of calculating the loss function based on the reward function includes:

[0154] Encode the state parameters of the computing network before the action, the action, the reward function, and the state parameters of the computing network after the action, and combine them into a four-tuple, and add the four-tuple to the memory space;

[0155] Randomly collect any quadruple from the memory space and obtain the state parameters of the computing power network after the action in the quadruple.

[0156] Based on the state parameters of the post-action computing network in the extracted quadruple, the corresponding state vector is obtained, and the state vector is input into the target network in the reinforcement learning model to obtain the action vector corresponding to the state vector. The largest parameter among the multiple dimensions of the action vector is output as the training dimension parameter.

[0157] The transition function is calculated based on the reward function value and training dimension parameters in the extracted quadruple according to the following formula:

[0158]

[0159] Among them, y δ represents the transition function, Represents the reward function value in the extracted quadruple δ, represents the training dimension parameter, and γ is the preset training parameter;

[0160] The loss function is calculated based on the transition function according to the following formula:

[0161] L=[y δ -q] 2 ;

[0162] Among them, L represents the loss function, y δ Represents the transition function, and q represents the dimension parameter value corresponding to the action in the extracted quadruple.

[0163] In the specific implementation process, q is the dimensional parameter value corresponding to the action when the reinforcement learning model outputs the action in the extracted quadruple.

[0164] In the specific implementation process, in order to reduce energy consumption and delay, the problem can be modeled as a multi-index optimization problem, the goal is to minimize the weighted sum of delay and energy consumption. The problem can be expressed as follows:

[0165] min(αT total +βE total );

[0166]

[0167] in, Indicates the preset upper limit time for processing task k.

[0168] This problem is a multi-objective non-convex optimization problem. The system state in the next time slot is affected by the system action in the previous time slot. It is a Markov decision problem, so reinforcement learning algorithm is used to solve this problem.

[0169] In the network model, the control node needs to make decisions on user tasks, match tasks with computing nodes, then schedule computing tasks to computing nodes for calculation, and finally return the calculation results.

[0170] The solution process of the entire model includes: task sorting, the control node needs to prioritize the tasks within a cycle to ensure that latency-sensitive tasks can be executed first; node matching, the network selects the best computing node for each task based on the task parameters and requirements of each task and the computing power status of the computing node; forwarding path selection, after completing node matching, it is also necessary to find the best path to reach the computing node from multiple paths to ensure the shortest total latency; pulling containers, after the task arrives at the computing node, if the node does not provide the container for the service, it is necessary to pull the container image from the network.

[0171] For node matching and task sequencing, we are considering using deep reinforcement learning to unify network resources, computing resources, and cache resources into a unified representation of the network state. Task sequencing is represented as system actions, and the optimization objective is expressed as a system reward function. For container forwarding path selection and fetching, we are considering using shortest path and greedy algorithms for simulation, rather than observing them as states, to prevent an excessively large state space from affecting training results. Through multiple rounds of iterative deep reinforcement learning training, the memory space is updated, and the neural network is continuously improved, ultimately achieving better training results.

[0172] The training process can be represented as shown in the following table:

[0173]

[0174]

[0175] The hierarchical CFN architecture based on ICN provided by the present invention aims to apply the characteristics of ICN to CFN, including hierarchical naming mechanism, caching strategy and workflow, to solve the problems of resource perception and task scheduling. CFN combines heterogeneous computing power information with network information, and improves resource utilization and task execution efficiency through resource perception, service positioning and task scheduling. However, since heterogeneous computing power is difficult to express and perceive, and is distributed on each node of the edge network, it is difficult to schedule tasks in a unified manner. The architecture of the present invention uses the naming mechanism of ICN to characterize computing power and tasks, and uses a caching mechanism and a routing forwarding mechanism to solve the problem of heterogeneous computing power being difficult to characterize and perceive in CFN, and the problem of dispersed distribution of computing power resources; on the other hand, CFN can perceive resources at the data level, formulate task forwarding strategies and coordinate computing resources, thereby solving the flooding problem of ICN, ensuring the service quality of user tasks, improving computing efficiency in the network, and improving the utilization of system network resources.

[0176] The beneficial effects of the present invention include:

[0177] 1. The ICN-based CFN architecture proposed in this paper uses the ICN naming mechanism to characterize computing power and tasks, and employs a caching mechanism and routing forwarding mechanism to solve the problems of difficult expression and perception of heterogeneous computing power in CFN and the dispersed distribution of computing power resources.

[0178] 2. In the solution proposed by this invention, as the number of routing hops increases, more forwarding nodes will cache containers in the task forwarding path, which helps reduce container pull time and improve service hit rate. This shows that the network architecture proposed by this invention has advantages in large-scale networks, which can improve task processing efficiency and the utilization of computing and network resources across the entire network.

[0179] 3. The reinforcement learning model designed in this paper prioritizes tasks and coordinates overall task execution when multiple tasks arrive within a cycle to achieve global optimization, ensuring user service quality while minimizing energy consumption. Therefore, in this scenario, the DQN solution designed in this paper is a better choice that is closer to the global optimal goal in real-world scenarios.

[0180] An embodiment of the present invention also provides a computing network task scheduling device based on an information center network, which includes a computer device, the computer device including a processor and a memory, the memory storing computer instructions, and the processor being used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps implemented by the method described above.

[0181] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps implemented by the aforementioned information-centric network-based computing network task scheduling method. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.

[0182] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0183] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0184] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0185] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A computing network task scheduling method based on an information-centric network, characterized in that: The method is applied to a computing power network, which includes forwarding nodes, control nodes, and computing nodes. The method includes the following steps: The forwarding node receives a processing request from a terminal user and sends the processing request to a control node. The control node is provided with a pre-trained reinforcement learning model. The target network in the reinforcement learning model allocates a computing node to the processing task based on the processing task corresponding to the processing request. The computing node is provided with computing resources for executing the processing task. The control node calculates the total action delay and the total action energy consumption based on the action of allocating the computing node made by the reinforcement learning model, the control node calculates the reward function based on the total action delay and the total action energy consumption, and calculates the loss function based on the reward function, the total action delay includes the container pulling delay, and the total action energy consumption includes the container pulling energy consumption; Obtaining the state parameters of the computing power network after executing the action, the step of calculating the loss function based on the reward function includes: encoding the state parameters of the computing power network before the action, the action, the reward function and the state parameters of the computing power network after the action, and combining them into a four-tuple, and adding the four-tuple to the memory space; randomly collecting any four-tuple from the memory space, obtaining the state parameters of the computing power network after the action in the four-tuple, obtaining the corresponding state vector based on the state parameters of the computing power network after the action in the extracted four-tuple, inputting the state vector into the target network in the reinforcement learning model to obtain the action vector corresponding to the state vector, and outputting the largest parameter among the parameters of multiple dimensions of the action vector as the training dimension parameter, calculating the transition function based on the reward function value and the training dimension parameter in the extracted four-tuple, and calculating the loss function based on the transition function; Update the parameters of the value network in the reinforcement learning model based on the loss function, and synchronize the parameters in the value network to the target network every preset number of steps.

2. The computing network task scheduling method based on information center network according to claim 1 is characterized in that: The step of allocating computing nodes to processing tasks based on the processing tasks corresponding to the processing requests by the target network in the reinforcement learning model includes: The control node calculates state parameters of the power network, the state parameters including forwarding node state parameters and computing node state parameters, and the forwarding node state parameters and computing node state parameters are combined into a state vector; The state vector is input into a reinforcement learning model, which outputs an action vector. The dimension corresponding to the largest parameter among the multiple dimensions of the action vector is extracted, and the action corresponding to the dimension is used as the output action.

3. The computing network task scheduling method based on information center network according to claim 2 is characterized in that: The forwarding node status parameter is used to mark whether each forwarding node is sending a processing request to the control node. If so, the forwarding node is marked as the first parameter, otherwise, the forwarding node is marked as the second parameter; the node status parameter is composed of a service function parameter, a node resource parameter and a remaining time parameter. The service function parameter is used to mark whether each computing node is executing a processing request. If so, the computing node is marked as the first parameter, otherwise, the computing node is marked as the second parameter; the node resource parameter is the computing resource size of the computing node; the remaining time parameter is the remaining time required for the computing node to execute the current processing request.

4. The method for scheduling computing network tasks based on information-centric network according to claim 1, characterized in that: The action includes a transmission path for processing the task and a pull path for the container corresponding to the processing task; the total action delay includes transmission delay, queuing delay, and computation delay; the total action energy consumption includes transmission energy consumption and computation energy consumption; and the step of calculating the total action delay and the total action energy consumption by the control node for allocating computation nodes based on the reinforcement learning model includes: Calculate the transmission delay based on the bandwidth parameters in the transmission path, calculate the container pulling delay based on the bandwidth parameters in the pulling path, calculate the computing delay required for the processing task based on the CPU resources and GPU resources in the destination computing node, and calculate the queuing delay of the processing task based on the computing delay of the processing task that arrived earlier than the processing task; calculate the total action delay based on the transmission delay, queuing delay, container pulling delay, and computing delay; The transmission energy consumption is calculated based on the transmission power of the sending router in the transmission path, the container pulling energy consumption is calculated based on the transmission power of the sending router in the pulling path, and the computing energy consumption is calculated based on the CPU resources and GPU resources in the destination computing node; the total energy consumption of the action is calculated based on the transmission energy consumption, container pulling energy consumption and computing energy consumption.

5. The method for scheduling computing network tasks based on information-centric network according to claim 4 is characterized in that: In the step of calculating the transmission delay based on the bandwidth parameter in the transmission path, the transmission delay is calculated based on the following formula: in, represents the transmission delay, R represents the set of routers in the transmission path, i and j represent any two adjacent routers in the transmission path, represents the bandwidth between router i and router j, Indicates the size of the processing task k; In the step of calculating the container pulling delay based on the bandwidth parameter in the pulling path, the pulling delay is calculated based on the following formula: in, represents the pull delay, U represents the set of routers in the pull path, x and y represent any two adjacent routers in the pull path, represents the bandwidth between router x and router y, Indicates the size of the container corresponding to the processing task; In the step of calculating the computing delay required for the processing task based on the CPU resources and GPU resources in the destination computing node, the computing delay is calculated according to the following formula: in, represents the computation delay, Indicates taking the maximum value, represents the size of the processing task k, Indicates the GPU resource size of the destination computing node. Indicates the CPU resource size of the destination computing node. are the preset task parameters. is the preset GPU positive correlation coefficient parameter, It is the preset CPU positive correlation coefficient parameter; In the step of calculating the queuing delay of the processing task based on the computational delays of the processing tasks that arrived earlier than the processing task, the sum of the computational delays of the processing tasks that have not yet been completed when the processing task arrives and that arrived earlier than the processing task is calculated to obtain the queuing delay; In the step of calculating the total action delay based on the transmission delay, queuing delay, container pull delay, and calculation delay, the total action delay is calculated based on the following formula: in, Indicates the total delay of the action, Indicates the queuing delay.

6. The method for scheduling computing network tasks based on information-centric network according to claim 4 is characterized in that: In the step of calculating the transmission energy consumption based on the transmission power of the transmitting end router in the transmission path, the transmission energy consumption is calculated based on the following formula: in, represents the transmission energy consumption, i and j represent any two adjacent routers in the transmission path, represents the transmission power of the sending router in the adjacent routers i and j, Indicates transmission delay; In the step of calculating the container pulling energy consumption based on the transmission power of the sending end router in the pulling path, the container pulling energy consumption is calculated based on the following formula: in, represents the container pulling energy consumption, U represents the set of routers in the pulling path, x and y represent any two adjacent routers in the pulling path, represents the transmission power of the sending router in the adjacent routers x and y, Indicates transmission delay; In the step of calculating the computing energy consumption based on the CPU resources and GPU resources in the destination computing node, the computing energy consumption is calculated based on the following formula: in, represents the computing energy consumption, Indicates the computing capacity parameter of the destination computing node. Indicates the computation delay; In the step of calculating the total energy consumption of an action based on the transmission energy consumption, container pulling energy consumption, and calculation energy consumption, the total energy consumption of the action is calculated according to the following formula: in, Indicates the total energy consumption of the action.

7. The method for scheduling computing network tasks based on information-centric network according to claim 6, characterized in that: Calculate the computing capacity parameters of the destination computing node according to the following formula: in, Indicates the computing capacity parameter of the destination computing node. Indicates the preset GPU power factor of the destination computing node. Indicates the CPU power factor preset for the destination computing node. Indicates the GPU resource size of the destination computing node. Indicates the CPU resource size of the destination compute node.

8. The method for scheduling computing network tasks based on information-centric network according to claim 1, characterized in that: In the step of calculating the reward function based on the total action delay and the total action energy consumption, the reward function value is calculated according to the following formula: in, represents the reward function value, and These are all preset calculation parameters. represents the total energy consumption of the action, Indicates the total action delay.

9. The computing network task scheduling method based on information center network according to claim 1 is characterized in that: The transition function is calculated based on the reward function value and training dimension parameters in the extracted quadruple according to the following formula: in, represents the transition function, Represents the extracted quadruple The reward function value in , represents the training dimension parameter, are the preset training parameters; The loss function is calculated based on the transition function according to the following formula: in, represents the loss function, Represents the transition function, and q represents the dimension parameter value corresponding to the action in the extracted quadruple.

10. A computing network task scheduling device based on an information center network, characterized in that: The apparatus includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus implements the steps implemented by the method according to any one of claims 1 to 9.

Images