Intelligent calculation fusion network function unloading method based on energy perception
Through deep reinforcement learning DQN algorithm and passive INT detection technology, the service function chain deployment strategy of the intelligent computing converged network is dynamically adjusted, and energy consumption is optimized, which solves the problems of low energy consumption and performance efficiency in the intelligent computing network, and realizes efficient energy utilization and high QoS services.
Patent Information
- Application Number
- CN202411022566.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-22
AI Technical Summary
Existing intelligent computing converged networks rarely pay attention to energy consumption when improving network performance, resulting in low server utilization and performance efficiency, making it difficult to meet high QoS needs.
The service function chain SFC deployment strategy is generated using deep reinforcement learning DQN algorithm, and the control plane interacts with the decision plane into a forwarding path, and the network status information is collected through passive INT detection, the energy consumption of each SFC deployment is calculated, the server status is updated, and the server status is updated. Adjust node status to optimize energy consumption.
It realizes intelligent management and optimization of network resources, reduces energy consumption, improves energy utilization efficiency, improves server utilization and performance efficiency, and responds to carbon peak and carbon neutrality goals.
Smart Images

Figure CN119011392A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent computing fusion network technology, and in particular to an intelligent computing fusion network function unloading method based on energy perception. Background Art
[0002] The existing network architecture has problems such as limited deep integration of computing and network, difficulty in cross-network scheduling and low comprehensive resource utilization. These drawbacks make it difficult to guarantee diversified service needs and fail to meet users' intelligent needs for network functions and performance. The intelligent computing fusion network studies the "three-layer three-domain" network model, unified representation and dynamic identification technology, and through the intelligent integration of multi-space and multi-dimensional resources across the entire network, vertically associates the service layer and network layer of the existing Internet, and uses identification as the basis for network access and control; the "three layers" include the intelligent service layer, resource adaptation layer and network component layer, and the "three domains" include the entity domain, behavior domain and knowledge domain, and the "three domains" run through the "three layers"; the knowledge space is innovatively introduced, and through intelligent scheduling strategies, differentiated and fine-grained network services are provided for different business scenarios, realizing the coordinated mobilization and intelligent integration of the service space and the network space on their own resources, and providing effective support for on-demand supply of personalized services and flexible networking.
[0003] In traditional networks, hardware is tightly coupled with specific network functions, resulting in high deployment and maintenance costs and low flexibility of network functions. The introduction of new architectures such as Software Defined Network (SDN), Network Function Virtualization (NFV) and Intelligent Identity Network has improved the flexibility, scalability and integration of the network. SDN technology achieves centralized management and dynamic configuration of network resources by decoupling the network control plane from the data forwarding plane, greatly improving the flexibility and programmability of the network. NFV technology virtualizes traditional network functions into software entities that can run on general-purpose hardware, thereby decoupling network functions from the underlying hardware.
[0004] In the intelligent computing network, network functions can be deployed in the form of software on general hardware servers, so that network functions no longer rely on dedicated hardware devices, realizing the decoupling of software and hardware. By flexibly scheduling the traffic in the virtual network and flowing through multiple virtualized network functions (VNF), a customizable network function set with low latency, high bandwidth and strong security can be dynamically and flexibly built for different users and business groups. In addition, the intelligent computing converged network uses the P4 (Programming Protocol-independent Packet Processors) language with the help of the programmable data plane (PDP) to realize user-defined programmability of network functions and more flexible differentiated service transmission control on the intelligent computing network components. The programmable switch directly executes VNF on the data plane through hardware processing capabilities, reducing latency and increasing data packet processing speed. It can not only optimize resource utilization efficiency and improve parallel processing capabilities, but also has good scalability and adapts to changes in network topology and traffic patterns, thereby providing higher network service quality.
[0005] In recent years, Smart NICs have been able to implement complex data processing and protocol processing at the network layer due to their powerful computing power and flexible programming interfaces, combined with multi-core processors and dedicated hardware accelerators. Through the P4 programming language, Smart NICs have achieved high programmability, allowing network functions to be flexibly defined and dynamically adjusted according to specific needs. In the intelligent computing center, Smart NICs release server CPU resources and improve system performance and energy efficiency by offloading computing-intensive tasks such as virtual network functions.
[0006] When actually providing network services, the functional requirements and quality requirements of network services are constructed into a customizable service function chain (SFC) according to user needs. SFC technology allows dynamic definition, deployment and management of service functions during data packet transmission, realizing flexible combination of network functions and customized service delivery. The intelligent computing converged network system introduces knowledge space and intelligent scheduling strategies to provide differentiated and fine-grained network services for different business scenarios, helping to achieve intelligent adaptation of SFC resources. Based on the associated nature of the entity domain, behavior domain and knowledge domain, the intelligent computing converged network constructs upper-layer applications into differentiated network function groups, and uses its hidden terminal heterogeneous network resource fusion, flexible network function chaining, SFC dynamic configuration and other characteristics to achieve coordinated mobilization and intelligent integration of service space and network space to their resources through mapping execution with underlying components. At the same time, with the substantial increase in computing power demand, when the intelligent computing center processes server computing power demand, QoS (Quality of Service) as a typical indicator of service demand behavior description needs to efficiently and dynamically deploy service functions and optimize service paths according to real-time network status and user needs. However, solutions that consider network performance rarely focus on energy consumption. In response to this, the present invention focuses on improving server utilization and performance efficiency, and reducing energy consumption and carbon emissions.
[0007] At present, although the existing NFV technology and SDN technology have achieved the decoupling of network functions from the underlying hardware, the decoupling of the network control plane from the data forwarding plane, and supported business diversification and elasticity, as well as automated deployment and operation and maintenance, the virtualization methods used by NFV technology and SDN technology are relatively limited, and the application scenarios of the service function chain are less considered. Although virtual machines and smart network cards have relatively flexible deployment capabilities, the service function chain constructed by them has a large bottleneck in performance and is difficult to support services with high QoS (Quality of Service) requirements.
[0008] Although intelligent computing networks can provide efficient, flexible, differentiated and customized communication network services to different industries and users through the "three-layer" and "three-domain" architecture, they rarely pay attention to energy consumption when considering network performance. Summary of the invention
[0009] An embodiment of the present invention provides an energy-aware intelligent computing fusion network function unloading method to achieve effective dynamic adjustment and optimization of resource allocation of the intelligent computing fusion network.
[0010] In order to achieve the above object, the present invention adopts the following technical scheme.
[0011] A method for offloading network functions based on energy perception and intelligent computing fusion, comprising:
[0012] The servers in the intelligent computing fusion network use the deep reinforcement learning DQN algorithm to generate the service function chain SFC deployment strategy;
[0013] The SFC deployment strategy is converted into a forwarding path through interaction between the control plane and the decision plane, and the forwarding path is associated with the decision of the decision plane and saved;
[0014] When executing this decision, obtain the forwarding path corresponding to this decision, implement the SFC data packet forwarding of this decision according to the forwarding path, collect the network status information of each node through passive INT detection, and output the actual network status information matrix;
[0015] After each SFC deployment strategy is completed, the energy consumption of each SFC deployment is calculated according to the parameters in the network state information matrix, and the state of the server is updated through the server node state conversion mechanism.
[0016] Preferably, the server in the intelligent computing fusion network generates a service function chain SFC deployment strategy using a deep reinforcement learning DQN algorithm, including:
[0017] The intelligent deployment of energy-aware SFC is modeled as a Markov process, and the five-tuple in the Markov process is defined.<S,A,P,R,γ> , where S represents the state set, A represents the action set, P represents the state transition probability distribution, R represents the reward function, and γ is the discount factor;
[0018] 1) State: Each state s of the system t ∈S, represented as a state space vector s t =[B t , D t , D t_sum ,
[0019] V t ], where B t represents the required bandwidth matrix of the current SFC request, D t Denotes the required delay matrix of the current SFC request, D t_sum Displays the accumulated delay information of the current SFC request, V t Mark the order of the currently deployed VNF in a group of VNFs. This information is preprocessed and used as the input of the neural network.
[0020] 2) Action: Each action a t∈A contains the node group vector consisting of nodes selected in order for each virtual network function provided. If the required bandwidth exceeds the bandwidth threshold or the current accumulated delay exceeds the delay threshold, the current action selection operation will be terminated, that is, the deployment fails;
[0021] 3) Reward function: Design a VNF resource allocation optimization scheme with the optimization objectives of minimizing server energy consumption and maximizing SFC deployment performance QoS. Define the reward as the difference between the actual QoS of an SFC and the normalized energy consumption of each physical node under the running efficiency of the SFC.
[0022]
[0023] The QoS of an SFC is equal to the QoS of bandwidth minus the QoS of latency.
[0024]
[0025] 4) State transition: The state transition of MDP is defined as: t+1 =f(s t 、a t 、r t 、s t+1 ), where s t Indicates the current network status, a t Indicates the action selected in the current network state, r t Indicates execution of action a t The reward obtained after t+1 Indicates execution of action a t The new network state that is transferred to later;
[0026] The DQN algorithm is used to optimize the Markov process. Through the continuous interaction between the agent and the environment, the estimation error in the agent will be gradually reduced, and a trained agent will be obtained. The trained agent is used to generate the service function chain SFC deployment strategy.
[0027] Preferably, converting the SFC deployment strategy into a forwarding path through interaction between the control plane and the decision plane, and associating and saving the forwarding path with the decision of the decision plane includes:
[0028] The agent generates a deployment strategy for the virtual network function VNF, establishes a corresponding action array based on the service path identifier SPI of the SFC corresponding to the current service function chain, writes the action array into the decision list, starts monitoring the feedback signal sent back by the software defined network SDN controller after completing the relevant actions, and creates a receipt check function to continuously monitor the changes in the receipt file;
[0029] After the SDN controller captures the updated action in the receipt file, it converts the SFC deployment strategy into a forwarding path according to the VNF deployment node and server topology, and sends down the flow table, and sequentially calls the function function corresponding to the service function on the service function forwarding switch node in the forwarding path to implement the service function;
[0030] When the SFC packet forwarding and service function implementation actions are completed, the SDN controller writes a receipt signal "Action completed" to the receipt file, indicating that it is ready to accept the next decision. It monitors whether the decision list is updated by reading the decision function. Once the agent reads a new receipt signal, it immediately sends a new decision, realizing the automated agent policy delivery and controller action execution process.
[0031] Preferably, when executing this decision, the forwarding path corresponding to this decision is obtained, the SFC data packet forwarding of this decision is implemented according to the forwarding path, the network status information of each node is collected through passive INT detection, and the actual network status information matrix is output, including:
[0032] Based on the P4 programmable data platform, a data packet header protocol is custom designed, and the SFC data packet is encapsulated using the NSH protocol. The header definition of the SFC data packet includes an Ethernet header, a service function chain header, an IPv4 header, a TCP header, and a UDP header. The etherType field in the Ethernet header identifies the data packet type: 0x800 indicates that the data packet type is an IPv4 packet, and 0x894F identifies the data packet as an SFC data packet. The service path identifier defined in the SFC header is used to uniquely identify the service function chain path, which is used to select a specific service path to achieve directional routing and service link selection for data packets; the service identifier SI is used to indicate the service function that the data packet currently needs to pass through, and to determine the service function of the next hop. The SI field of the node increases or decreases in sequence, indicating the complete path of the service function chain through which the data packet passes, guiding the network device to perform the service function in the specified order, and forwarding the SFC data stream by matching the SPI and SI;
[0033] Traverse each switch node in the path to configure the flow table entry. For the network function forwarding switch node in the path, obtain the MAC address of the next hop node of the current node and the port number from the current node to the next hop node, add the sfc_forward rule in the sfc_egress table of the current node, and forward the data packet to the address by looking up the address or port information of the next node to achieve traffic forwarding;
[0034] While the SFC flow is forwarding, the INT header structure is defined on the P4 plane, the packet parsing logic is written, the metadata type, path information and timestamp fields are filled, and INT table entries are added on the control plane. The network status information of each node in the service function path is collected through passive INT detection, and the network status information matrix is output.
[0035] Preferably, when executing this decision, the forwarding path corresponding to this decision is obtained, the SFC data packet forwarding of this decision is implemented according to the forwarding path, the network status information of each node is collected through passive INT detection, and the actual network status information matrix is output, which also includes:
[0036] For the first network function forwarding switch node in the path, add a flow table entry on the controller of the node, add the sfc_encapsulation rule in the sfc_classifier table to implement the identification and encapsulation of the SFC according to the service path identifier SPI of the data packet, and for the last forwarding node in the path, add the sfc_decapsulation rule in the sfc_termination table to process the incoming traffic;
[0037] Five network service function modules based on P4 programming are designed, namely normal forwarding, firewall, NAT, Counter, and load balancing. After receiving the upper-level decision, the service function modules are scheduled and dynamically combined according to business needs to realize on-demand combination and deployment of network functions. The P4 controller calls the table_add method to add entries to the flow table of each service function, specify the name of the action, matching fields, and transfer parameters, so as to realize the decoupling of service functions and management scheduling.
[0038] Preferably, after each SFC deployment strategy is completed, the server status is updated through the server node status conversion mechanism, including:
[0039] Set up a node state adaptive conversion mechanism. This mechanism distributes SFC traffic by dynamically adjusting the node state when processing SFC requests. Set up three different states of the server: OFF, IDLE, and ACTIVE, as well as the conversion method between the three states. The server state must be updated according to the latest VNF deployment status of each server after each SFC deployment. The specific rules are as follows:
[0040] 1) When the server node is powered on and receives an SFC deployment request, the node needs to process the request and implement the service function. At this time, the node is in the ACTIVE state;
[0041] 2) If the node in the ACTIVE state no longer processes SFC requests, it will change to the IDLE state. In this state, a parameter idletime is defined as the IDLE state countdown counter. When it just changes to the IDLE state, the counter is adjusted to the maximum value, that is, three time steps, and the countdown is reduced by 1 as each step is completed;
[0042] 3) If the node still has no new SFC deployment request after the countdown ends, the node will automatically switch to the OFF state to reduce energy consumption; but if there is a new VNF deployment request in the time slot before the countdown ends, it will return to the ACTIVE state.
[0043] Preferably, the energy consumption of each SFC deployment is calculated according to the parameters in the network state information matrix, and VNFs are dynamically allocated to the server with the best energy efficiency, including:
[0044] The SFC energy consumption is divided into two parts: operating power consumption E p and no-load power consumption E m The running power consumption is calculated by considering whether the node is unloaded, and the power parameters P corresponding to the switch and the server are set respectively. sw and P se , τ sfc The lifetime introduced for processing each SFC request; the maintenance power consumption Em calculates the power consumption generated by the power switch and the power consumption generated by the unloaded equipment, where P idle is the no-load power, E on / off is the power consumption of on / off, Num(on / off_nodes) is the number of nodes that perform on / off actions in an SFC;
[0045] Energy=E m +E p (3)
[0046] E p =B sfc ×τ sfc ×(P sw ×Num(sw_nodes)+P se ×Num(se_nodes)) (4)
[0047] E m =Num(on / off_nodes)×E on / off +P idle ×τ sfc (5)
[0048]
[0049] The operating power consumption Ep and no-load power consumption E m After addition, the values are normalized and written into the reward value of the algorithm. By monitoring the demand for network services and the current status of the nodes, an energy-aware VNF dynamic allocation strategy is implemented to dynamically allocate VNFs to servers with optimal energy efficiency.
[0050] It can be seen from the technical solution provided by the above-mentioned embodiment of the present invention that the embodiment of the present invention uses an intelligent network card to realize heterogeneous unloading of service functions and optimizes the quality of network services. The present invention realizes the unloading of virtual network functions on the basis of a heterogeneous NFV framework based on servers and intelligent network cards, thereby realizing the balancing of network loads and the effective use of resources. The present invention aims to provide a service function chain deployment method based on reinforcement learning to realize energy consumption perception, which can dynamically adjust the deployment mode and traffic path of the service function chain according to the real-time network status and user QoS requirements, and realize intelligent management and optimization of network resources. At the same time, by perceiving the energy consumption of the network device server and combining the reinforcement learning algorithm for optimization scheduling, energy consumption is reduced and energy utilization efficiency is improved. The present invention also focuses on improving server utilization and performance efficiency, reducing energy consumption and carbon emissions, in response to my country's goal of achieving carbon peak and carbon neutrality.
[0051] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0053] Figure 1 A processing flow chart of a method for offloading network functions based on energy-aware intelligent computing fusion provided by an embodiment of the present invention;
[0054] Figure 2 A schematic diagram of offloading a network function from a server to a smart network card provided by an embodiment of the present invention, wherein the network function implemented on a general server is a VNF, and the network function offloaded to the smart network card for execution is defined as a PNF;
[0055] Figure 3 A schematic diagram of an entity domain forwarding SFC data flow and function implementation provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0056] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.
[0057] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.
[0058] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.
[0059] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.
[0060] The present invention is studied on the basis of the theoretical model of "three layers and three domains". The generalized service layer has the service management function of managing service identification and service behavior description in the three domains. It parses user service requests and sends service requirements to the mapping adaptation layer. Combined with the deep reinforcement learning algorithm, it generates deployment strategies in the knowledge domain and sends them to the converged network layer. The converged network layer is a collection of underlying network devices and network functions, integrating computing resources, and collecting component device status to the mapping adaptation layer for upper-level decision-making. The deep integration of the network and cross-network collaborative scheduling of resources can be achieved through the intelligent computing converged network architecture. The knowledge domain can achieve adaptive deployment through an intelligent decision-making mechanism, dynamically adjust strategies according to the environment and feedback, and optimize system performance and resource utilization efficiency. In addition, reinforcement learning can also reduce manual intervention, realize automated management and optimize the deployment process, thereby reducing management costs and improving the intelligence level of the system.
[0061] In the knowledge domain, the SFC deployment strategy is generated through the deep reinforcement learning algorithm to guide the collaboration between the entity domain and the sensor control domain to complete the specific implementation process of the service, and further optimize itself in the generalized service layer based on the feedback of the implementation results; in the sensor control domain, it is responsible for the distribution control of the service, the corresponding component group strategy management and the management of the component transmission control strategy, and then calls the perception method to perceive the policy execution and provide feedback, forming the perception feedback experience of "service-strategy-execution result" to support the knowledge extraction of the knowledge domain. Finally, the energy-aware intelligent computing fusion network function offloading method is realized by designing the energy consumption state conversion and calling the entity domain network component layer.
[0062] The processing flow of a method for offloading network functions based on energy perception and intelligent computing fusion provided by an embodiment of the present invention is as follows: Figure 1 As shown, the processing steps include the following:
[0063] Step S10: Generate a service function chain SFC deployment strategy based on the deep reinforcement learning DQN algorithm.
[0064] Input state information (described in detail below), output the node vectors to which the virtual network functions VNFs are deployed in order, and the vectors indicating whether each VNF is uninstalled (that is, implemented in the smart network card); the DQN algorithm is an existing deep reinforcement learning algorithm.
[0065] Step S20: SFC deployment strategy issuance and control plane information processing.
[0066] The interaction between the decision plane and the control plane is realized by "passing notes". The SFC deployment strategy in step S10 is written into the established decision list document. The control plane reads and outputs the SFC deployment strategy as a forwarding path through the set information processing function, and calls the function function in sequence. After the forwarding and function are completed, a receipt signal is written to the established receipt file, indicating that the deployment has been completed. The decision plane receives the receipt signal by reading the receipt file and then starts the next decision. This part is innovative.
[0067] Step S30: SFC flow forwarding and service function implementation.
[0068] First, the forwarding of the NSH protocol and SFC flow is realized on the P4 programmable data plane, and then the flow table is issued according to the forwarding path output after the control plane information processing after each decision, so as to realize the SFC data packet forwarding of this decision. At the same time, the network status information of each node is collected through the passive INT (Inband Network Telemetry) detection technology, and the output is the actual network status information matrix; secondly, five network service functions are deployed on the P4 platform and function call functions are established. When the control plane calls the function function, the function flow table of the corresponding node is issued and the function is realized. In other words, the action execution of the network component layer deployed by SFC is realized. Among them, INT technology and P4 programmable platform are existing technologies, and the rest are realized by the present invention based on these technologies.
[0069] Step S40: Server node state transition and energy consumption awareness calculation.
[0070] After each SFC deployment strategy is completed, the server status is updated through the server node status conversion mechanism designed by the present invention, and then an energy consumption evaluation module is designed to calculate the energy consumption of each SFC deployment through the parameters in the actual network status information matrix output in step S30. This energy consumption is written into the reward of the algorithm after normalization, thereby realizing the continuous learning of the intelligent body and the optimization of the energy consumption of SFC deployment. This part is all innovatively proposed.
[0071] The specific processing procedures of each step are introduced below.
[0072] Step S10: Knowledge domain based on energy consumption aware SFC deployment module.
[0073] The present invention models the intelligent deployment of energy-aware SFC as a Markov process (Markov Decision Process, MDP), and uses a deep reinforcement learning (DQN) algorithm to implement and optimize it. DQN is a reinforcement learning algorithm that combines deep learning and Q-learning techniques, and has the following characteristics: ① Using neural networks to approximate Q functions, it can work in continuous state spaces and large-scale action spaces; ② Experience replay mechanism to break the correlation between data and improve learning efficiency; ③ Introducing the target network (Target Network), the Q network parameters are used to update the parameters of the target network in each step of learning. This mechanism prevents instability that may occur during frequent parameter updates. Usually, MDP is defined as a 5-tuple<S,A,P,R,γ> , where S represents the state set, A represents the action set, P represents the state transition probability distribution, R represents the reward function, and γ is the discount factor, which reflects the agent's consideration of the future rewards brought by the current decision, helping the agent to strike a balance between long-term and short-term rewards. The definition of the quintuple in this invention is as follows:
[0074] 1) State: Each state s of the system t ∈S, represented as a state space vector s t =[B t , D t , D t_sum ,
[0075] V t ], where B t represents the required bandwidth matrix of the current SFC request, D t Denotes the required delay matrix of the current SFC request, D t_sum Displays the accumulated delay information of the current SFC request, V t The order of the currently deployed VNF in a group of VNFs is marked, and this information is used as the input of the neural network after preprocessing.
[0076] 2) Action: Each action a t ∈A contains a node group vector consisting of nodes selected in order for each virtual network function provided. If the required bandwidth exceeds the bandwidth threshold or the current accumulated delay exceeds the delay threshold, the current action selection operation will be terminated, that is, the deployment fails.
[0077] 3) Reward function: The present invention takes minimizing server energy consumption and maximizing SFC deployment performance QoS as the optimization goals, and designs a VNF resource allocation optimization scheme based on this. Therefore, the reward is defined as the difference between the actual QoS of an SFC and the normalized value of the energy consumption of each physical node under the running efficiency of the SFC, aiming to encourage the selection of high-bandwidth, low-latency, and low-energy consumption actions.
[0078]
[0079] Among them, the QoS of an SFC is equal to the QoS of bandwidth minus the QoS of delay. If the obtained value is large, it means that the difference between the bandwidth QoS and the delay QoS is large, that is, high bandwidth and low delay are obtained, which means that better network performance is obtained.
[0080]
[0081] 4) State transition: The state transition of MDP is defined as: t+1 =f(s t 、a t 、r t 、s t+1 ), where s t Indicates the current network status, a t Indicates the action selected in the current network state, r t Indicates execution of action a t The reward obtained after t+1 Indicates execution of action a t Then transfer to the new network state.
[0082] Through the continuous interaction between the agent and the environment, the policy model will gradually reduce the estimation error, improve the training stability, and ultimately achieve better performance and higher accuracy. The trained agent can make online decisions and is responsible for executing VNF deployment decisions based on the assessed energy consumption and service requirements.
[0083] Step S20. Automatically issue the decision based on the entity domain network component:
[0084] Figure 2A schematic diagram of offloading a network function from a server to a smart network card provided in an embodiment of the present invention. The present invention implements the offloading operation of SFC data packets on a smart network card that supports a programmable data plane by writing a p4 program. Virtual network function offloading based on servers and smart network cards is an important strategy for optimizing network resource utilization. By dynamically adjusting the deployment location of VNFs between servers and smart network cards, network load balancing and effective resource utilization can be achieved. Specifically, when the server load is too high or the network traffic changes greatly, the smart network card can offload part of the VNF from the server to the smart network card for execution according to a preset strategy, thereby reducing the server load and improving the overall network performance and response speed. It can realize dynamic allocation and utilization of resources, improve network performance and service quality, and also reduce energy consumption and operation and maintenance costs, providing important support for the sustainable development and intelligent operation of the network. The following introduces the process of automatic issuance and function realization of deep reinforcement learning strategies:
[0085] Create a decision list and receipt acceptance file. First, the intelligent agent generates the VNF deployment strategy and the service path identifier (SPI) of the service function path corresponding to the current service function chain, establishes the corresponding action array and writes it into the decision list. It immediately starts monitoring the feedback signal sent back after the SDN controller completes the relevant actions, and creates a receipt check function to continuously monitor the changes in the receipt file so as to process new signals in time.
[0086] At the same time, the SDN controller will continue to issue instructions and read the decision-making file. Once it captures an update to the action in the file, it will immediately read and process the information, generate a complete SFC path based on the VNF deployment node and server topology, and issue the flow table. It will also call the function functions corresponding to the service function in sequence on the service function forwarding switch node in the SFC path to implement the service function.
[0087] When the SFC packet forwarding and service function implementation actions are completed, the SDN controller writes a receipt signal "Action completed" to the receipt file, indicating that the next decision can be accepted, and establishes a decision reading function to continuously monitor whether the decision list is updated. Once the agent reads a new receipt signal, it will immediately send a new decision, thereby realizing the automated agent strategy delivery and controller action execution process.
[0088] Step S30. Entity domain forwards SFC data flow and function implementation module:
[0089] Figure 3A schematic diagram of an entity domain forwarding SFC data flow and function implementation provided by an embodiment of the present invention. A data packet header protocol is custom designed based on a P4 programmable data platform. The NSH (Network Service Header) protocol needs to be used to encapsulate the SFC data packet. The header definition of the SFC data packet includes an Ethernet header, a service function chain header, an IPv4 header, a TCP header, and a UDP header. Among them, the etherType field of the Ethernet header in the Ethernet header identifies the data packet type: 0x800 indicates that the data packet type is an IPv4 packet, and 0x894F identifies the data packet as an SFC data packet. The service path identifier defined in the SFC header is used to uniquely identify the service function chain path, which is used to select a specific service path, thereby realizing directional routing and service link selection for the data packet; a service identifier (Service Index, SI) is used to indicate the service function that the data packet currently needs to pass through, and the service function of the next hop is determined. The SI field of the node increases or decreases in sequence, which can indicate the complete path of the service function chain through which the data packet passes, and guides the network device to perform the service function in the specified order. The SFC data stream can be forwarded by matching the SPI and SI.
[0090] Traverse each switch node in the path to configure the flow table entry to achieve path-based traffic forwarding and processing. For the network function forwarding switch node in the path, it is necessary to obtain the MAC address of the next hop node of the current node and the port number from the current node to the next hop node. Add the sfc_forward rule in the sfc_egress table of the current node, and forward the data packet to the address by looking up the address or port information of the next node to achieve traffic forwarding.
[0091] For the first network function forwarding switch node in the path, a flow table entry needs to be added to the controller of the node, that is, the sfc_encapsulation rule needs to be added to the sfc_classifier table to realize the identification and encapsulation of the SFC according to the service path identifier (SPI) of the data packet, including adding the header information required for the service function chain, such as identifier, path information, etc.; for the last forwarding node in the path, the sfc_decapsulation rule needs to be added to the sfc_termination table to process the incoming traffic and decapsulate the NSH header information in it so as to correctly route and process the data packet.
[0092] While the SFC flow is forwarding, the INT header structure is defined on the P4 plane, the packet parsing logic is written, the metadata type, path information, timestamp and other fields are filled, and the INT table entry is added to the control plane. The collected network status information includes queue length, latency, throughput and other information. Passive detection of INT is implemented to collect the network status information of each node in the service function path and output it as a network status information matrix.
[0093] In addition, the present invention designs five network service functions based on P4 programming, namely, ordinary forwarding, firewall, NAT, Counter, and load balancing. Through modularization and functionalization of functions, it is suitable for building flexible and efficient network service links. After receiving the upper-level decision, these service function modules can be quickly scheduled and dynamically combined according to business needs, realizing on-demand combination and flexible deployment of network functions. The P4 controller calls the table_add method to add table entries to the flow table of each service function, accurately controls the data packet processing according to SPI, SI, destination IP address and other related parameters, realizes the decoupling of service functions and management scheduling, and improves the service response speed.
[0094] Step S40: Energy consumption perception module of the sensing control domain. The present invention mainly considers the server energy consumption of the green intelligent computing center, including the power-on / off energy consumption and the no-load energy consumption. In this regard, (1) a node state adaptive conversion mechanism is designed. This mechanism dynamically adjusts the node state when processing SFC requests, thereby allocating SFC traffic to achieve the purpose of optimizing energy consumption. Three different states of the server are set: OFF, IDLE, and ACTIVE, as well as the conversion method between the three states. The server state must be updated after each SFC deployment. The specific rules are as follows:
[0095] 1) When the server node is powered on and receives an SFC deployment request, the node needs to process the request and implement the service function. At this time, the node is in the ACTIVE state;
[0096] 2) If the node in the ACTIVE state no longer processes SFC requests, it will change to the IDLE state. In this state, a parameter idletime is defined as the IDLE state countdown counter. When it just changes to the IDLE state, the counter is adjusted to the maximum value, that is, three time steps, and the countdown is reduced by 1 as each step is completed;
[0097] 3) If the node still has no new SFC deployment request after the countdown ends, the node will automatically switch to the OFF state to reduce energy consumption; but if there is a new VNF deployment request in the time slot before the countdown ends, it will return to the ACTIVE state, thereby reducing the energy waste caused by frequent power on and off.
[0098] By monitoring the demand for network services and the current status of nodes, the present invention implements a VNF dynamic allocation strategy based on energy consumption awareness. The strategy reduces overall energy consumption by dynamically allocating VNFs to servers with optimal energy efficiency.
[0099] (2) The present invention proposes a method for evaluating the overall energy consumption by considering the energy consumption of the server in different states. As shown in formula (3-6), the SFC energy consumption is calculated by dividing it into two parts: operating power consumption E and p and no-load power consumption E m , in which the running power consumption takes into account whether the node is unloaded when calculating, and sets the power parameters P corresponding to the switch and server respectively sw and P se , τ sfc The lifetime introduced for processing each SFC request; the maintenance power consumption Em calculates the power consumption generated by the power switch and the power consumption generated by the unloaded equipment, where P idle is the no-load power, E on / off is the power consumption of on / off, and Num(on / off_nodes) is the number of nodes that perform on / off actions in an SFC. The two parts of power consumption are added together and normalized and written into the reward value of the algorithm to optimize energy consumption.
[0100] Energy=E m +E p (3)
[0102] E p =B sfc ×τ sfc ×(P sw ×Num(sw_nodes)+P se ×Num(se_nodes)) (4)
[0104] E m =Num(on / off_nodes)×E on / off +P idle ×τ sfc (5)
[0106]
[0107]
[0108] In summary, the energy-aware intelligent computing fusion network function offloading method of the embodiment of the present invention can bring the following beneficial effects:
[0109] Energy consumption optimization and green computing: The energy consumption perception module of the sensing and control domain proposed in the present invention dynamically adjusts the node status by setting three different energy consumption states of the node (off, idle, and active) and the conversion method between the three states, thereby optimizing energy consumption, reducing the waste of computing resources, and promoting the practice of green computing.
[0110] Intelligent deployment and optimized decision-making: Based on the deep reinforcement learning (DQN) algorithm, VNF resource allocation is optimized through reinforcement learning. The intelligent agent improves the accuracy and stability of the policy model through learning and interaction with the environment, and finally realizes online decision-making. According to the evaluated energy consumption and service requirements, it dynamically executes state transition and resource allocation, improving deployment efficiency and network performance.
[0111] Heterogeneous offloading and flexible deployment of automated control: The present invention designs a decision-making automatic issuance and function realization module based on P4 programming, which can realize the offloading of SFC data packets on the programmable data plane smart network card. The flexibility and modular service function design provided by P4 programming enable network service links to be combined and flexibly deployed on demand, improving network response speed and service quality.
[0112] The resource allocation strategy deployed by the traditional static service function chain is usually unable to dynamically adjust and optimize resource allocation, and rarely considers the real-time changes in energy consumption, making it difficult to minimize energy consumption and maximize service quality. The present invention proposes a systematic solution, including a state transition method, an energy consumption evaluation method, and a dynamic resource allocation strategy, which significantly optimizes energy utilization; traditional machine learning methods usually require a large number of predefined rules and manual adjustments, and are difficult to adapt to complex and dynamic network environments. The introduction of DQN can automatically adjust strategies through continuous learning and self-optimization, improve resource utilization efficiency and service quality, and achieve a balance between long-term and short-term returns; by dynamically allocating virtual network functions (VNFs) to different types of computing resources (servers and smart network cards), flexible allocation and scheduling of resources can be achieved. In summary, traditional methods and existing alternatives are difficult to achieve the effects of the present invention in terms of flexibility, dynamic adjustment capabilities, overall optimization efficiency, and innovative applications.
[0113] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.
[0114] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.
[0115] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0116] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A method for offloading network functions based on energy awareness and intelligent computing fusion, characterized in that: include: The servers in the intelligent computing fusion network use the deep reinforcement learning DQN algorithm to generate the service function chain SFC deployment strategy; The SFC deployment strategy is converted into a forwarding path through interaction between the control plane and the decision plane, and the forwarding path is associated with the decision of the decision plane and saved; When executing this decision, obtain the forwarding path corresponding to this decision, implement the SFC data packet forwarding of this decision according to the forwarding path, collect the network status information of each node through passive INT detection, and output the actual network status information matrix; After each SFC deployment strategy is completed, the energy consumption of each SFC deployment is calculated according to the parameters in the network state information matrix, and the state of the server is updated through the server node state conversion mechanism.
2. The method according to claim 1, characterized in that The server in the intelligent computing fusion network generates a service function chain SFC deployment strategy using a deep reinforcement learning DQN algorithm, including: The intelligent deployment of energy-aware SFC is modeled as a Markov process, and the five-tuple in the Markov process is defined.<S,A,P,R,γ> , where S represents the state set, A represents the action set, P represents the state transition probability distribution, R represents the reward function, and γ is the discount factor; 1) State: Each state s of the system t ∈S, represented as a state space vector s t =[B t , D t , D t_sum , V t ], where B t represents the required bandwidth matrix of the current SFC request, D t Denotes the required delay matrix of the current SFC request, D t_sum Displays the accumulated delay information of the current SFC request, V t Mark the order of the currently deployed VNF in a group of VNFs. This information is preprocessed and used as the input of the neural network. 2) Action: Each action a t ∈A contains the node group vector consisting of nodes selected in order for each virtual network function provided. If the required bandwidth exceeds the bandwidth threshold or the current accumulated delay exceeds the delay threshold, the current action selection operation will be terminated, that is, the deployment fails; 3) Reward function: Design a VNF resource allocation optimization scheme with the optimization objectives of minimizing server energy consumption and maximizing SFC deployment performance QoS. Define the reward as the difference between the actual QoS of an SFC and the normalized energy consumption of each physical node under the running efficiency of the SFC. The QoS of an SFC is equal to the QoS of bandwidth minus the QoS of latency. 4) State transition: The state transition of MDP is defined as: t+1 =f(s t 、a t 、r t 、s t+1 ), where s t Indicates the current network status, a t Indicates the action selected in the current network state, r t Indicates execution of action a t The reward obtained after t+1 Indicates execution of action a t The new network state that is transferred to later; The DQN algorithm is used to optimize the Markov process. Through the continuous interaction between the agent and the environment, the estimation error in the agent will be gradually reduced, and a trained agent will be obtained. The trained agent is used to generate the service function chain SFC deployment strategy.
3. The method according to claim 2, characterized in that The step of converting the SFC deployment strategy into a forwarding path through interaction between the control plane and the decision plane, and associating and saving the forwarding path with the decision of the decision plane includes: The agent generates a deployment strategy for the virtual network function VNF, establishes a corresponding action array based on the service path identifier SPI of the SFC corresponding to the current service function chain, writes the action array into the decision list, starts monitoring the feedback signal sent back by the software defined network SDN controller after completing the relevant actions, and creates a receipt check function to continuously monitor the changes in the receipt file; After the SDN controller captures the updated action in the receipt file, it converts the SFC deployment strategy into a forwarding path according to the VNF deployment node and server topology, and sends down the flow table, and sequentially calls the function function corresponding to the service function on the service function forwarding switch node in the forwarding path to implement the service function; When the SFC packet forwarding and service function implementation actions are completed, the SDN controller writes a receipt signal "Action completed" to the receipt file, indicating that it is ready to accept the next decision. It monitors whether the decision list is updated by reading the decision function. Once the agent reads a new receipt signal, it immediately sends a new decision, realizing the automated agent strategy delivery and controller action execution process.
4. The method according to claim 3, characterized in that When executing this decision, the forwarding path corresponding to this decision is obtained, the SFC data packet forwarding of this decision is implemented according to the forwarding path, the network status information of each node is collected through passive INT detection, and the actual network status information matrix is output, including: Based on the P4 programmable data platform, a data packet header protocol is custom designed, and the SFC data packet is encapsulated using the NSH protocol. The header definition of the SFC data packet includes an Ethernet header, a service function chain header, an IPv4 header, a TCP header, and a UDP header. The etherType field in the Ethernet header identifies the data packet type: 0x800 indicates that the data packet type is an IPv4 packet, and 0x894F identifies the data packet as an SFC data packet. The service path identifier defined in the SFC header is used to uniquely identify the service function chain path, which is used to select a specific service path to achieve directional routing and service link selection for data packets; the service identifier SI is used to indicate the service function that the data packet currently needs to pass through, and to determine the service function of the next hop. The SI field of the node increases or decreases in sequence, indicating the complete path of the service function chain through which the data packet passes, guiding the network device to perform the service function in the specified order, and forwarding the SFC data stream by matching the SPI and SI; Traverse each switch node in the path to configure the flow table entry. For the network function forwarding switch node in the path, obtain the MAC address of the next hop node of the current node and the port number from the current node to the next hop node, add the sfc_forward rule in the sfc_egress table of the current node, and forward the data packet to the address by looking up the address or port information of the next node to achieve traffic forwarding; While the SFC flow is forwarding, the INT header structure is defined on the P4 plane, the packet parsing logic is written, the metadata type, path information and timestamp fields are filled, and INT table entries are added on the control plane. The network status information of each node in the service function path is collected through passive INT detection, and the network status information matrix is output.
5. The method according to claim 4, characterized in that When executing this decision, the forwarding path corresponding to this decision is obtained, the SFC data packet forwarding of this decision is implemented according to the forwarding path, the network status information of each node is collected through passive INT detection, and the actual network status information matrix is output, which also includes: For the first network function forwarding switch node in the path, add a flow table entry on the controller of the node, add the sfc_encapsulation rule in the sfc_classifier table to implement the identification and encapsulation of the SFC according to the service path identifier SPI of the data packet, and for the last forwarding node in the path, add the sfc_decapsulation rule in the sfc_termination table to process the incoming traffic; Five network service function modules based on P4 programming are designed, namely normal forwarding, firewall, NAT, Counter, and load balancing. After receiving the upper-level decision, the service function modules are scheduled and dynamically combined according to business needs to realize on-demand combination and deployment of network functions. The P4 controller calls the table_add method to add entries to the flow table of each service function, specify the name of the action, matching fields, and transfer parameters, so as to realize the decoupling of service functions and management scheduling.
6. The method according to claim 4 or 5, characterized in that: After each SFC deployment strategy is completed, the server status is updated through the server node status conversion mechanism, including: Set up a node state adaptive conversion mechanism. This mechanism distributes SFC traffic by dynamically adjusting the node state when processing SFC requests. Set up three different states of the server: OFF, IDLE, and ACTIVE, as well as the conversion method between the three states. The server state must be updated according to the latest VNF deployment status of each server after each SFC deployment. The specific rules are as follows: 1) When the server node is powered on and receives an SFC deployment request, the node needs to process the request and implement the service function. At this time, the node is in the ACTIVE state; 2) If the node in the ACTIVE state no longer processes SFC requests, it will change to the IDLE state. In this state, a parameter idletime is defined as the IDLE state countdown counter. When it just changes to the IDLE state, the counter is adjusted to the maximum value, that is, three time steps, and the countdown is reduced by 1 as each step is completed; 3) If the node still has no new SFC deployment request after the countdown ends, the node will automatically switch to the OFF state to reduce energy consumption; but if there is a new VNF deployment request in the time slot before the countdown ends, it will return to the ACTIVE state.
7. The method according to claim 6, characterized in that The energy consumption of each SFC deployment is calculated according to the parameters in the network state information matrix, and VNFs are dynamically allocated to the server with the best energy efficiency, including: The SFC energy consumption is divided into two parts: operating power consumption E p and no-load power consumption E m The running power consumption is calculated by considering whether the node is unloaded, and the power parameters P corresponding to the switch and the server are set respectively. sw and P se , τ sfc The lifetime introduced for processing each SFC request; the maintenance power consumption Em calculates the power consumption generated by the power switch and the power consumption generated by the unloaded equipment, where P idle is the no-load power, E on / off is the power consumption of on / off, Num(on / off_nodes) is the number of nodes that perform on / off actions in an SFC; Energy=E m +E p (3) IN p =B sfc ×τ sfc ×(P sw ×Num(sw_nodes)+P se ×Num(se_nodes)) (4) E m =Num(on / off_nodes)×E on / off +P idle ×τ sfc (5) The operating power consumption E p and no-load power consumption E m After addition, the values are normalized and written into the reward value of the algorithm. By monitoring the demand for network services and the current status of the nodes, an energy-aware VNF dynamic allocation strategy is implemented to dynamically allocate VNFs to servers with optimal energy efficiency.
Citation Information
Cited By
Network slice arrangement method for programmable data plane network
CN119906644A
